Considering Execution Environment Resilience: A White-Box Approach - - PowerPoint PPT Presentation

▶
considering execution environment resilience a white box
SMART_READER_LITE
LIVE PREVIEW

Considering Execution Environment Resilience: A White-Box Approach - - PowerPoint PPT Presentation

Jul 29, 2023 100 likes •458 views

Considering Execution Environment Resilience: A White-Box Approach Stefan Klikovits 1 , 2 , David PY Lawrence 1 , 3 , Manuel Gonzalez-Berges 2 , Didier Buchs 1 1 Universit de Genve, Carouge, Switzerland 2 CERN, Geneva, Switzerland 3 Honeywell

slide-1
SLIDE 1

Considering Execution Environment Resilience: A White-Box Approach

Stefan Klikovits1,2, David PY Lawrence1,3, Manuel Gonzalez-Berges2, Didier Buchs1

1Université de Genève, Carouge, Switzerland 2CERN, Geneva, Switzerland 3Honeywell International Sarl., Rolle, Switzerland

Tuesday 11th August, 2015

slide-2
SLIDE 2

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

What is this all about?

How to

◮ generate test cases w. little user interaction ◮ on a large scale ◮ unit/component level

2 / 21

slide-3
SLIDE 3

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

Welcome to CERN

Credit: CERN (www.cern.ch)

3 / 21

slide-4
SLIDE 4

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

Welcome to CERN

◮ LHC, experiments, infrastructure (e.g. power grid) ◮ large-scale, widespread, complex systems ◮ many types of hard- and software ◮ > 100 subsystems,

10,000s of devices, 100,000s of measurement points

◮ thousands of physicists/engineers/workers

high reliability and resilience expectations

3 / 21

slide-5
SLIDE 5

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

How do we supervise it?

◮ two frameworks (UNICOS, JCOP) built on top ◮ Control (CTRL): proprietary scripting language

4 / 21

slide-6
SLIDE 6

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

So where is the problem?

◮ until recently no automated unit test support ◮ frequent changes in execution environment ◮ (mostly) manual verification ◮ big expenses (time) on QA side

5 / 21

slide-7
SLIDE 7

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

Testing

1 f ( x ){ 2 i f GLOBAL_VAR: 3 r e t u r n dbGet ( x ) 4 e l s e : 5 r e t u r n −1 6 }

f(x)

1 t e s t _ f ( ) { 2 dbSet ( " t e s t " ,5) // p r e p a r e 3 GLOBAL_VAR = True 4 x = f ( " t e s t " ) // a c t 5 a s s e r t ( x == 5) // a s s e r t 6 }

Test case for f(x)

6 / 21

slide-8
SLIDE 8

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

What do we want?

code test cases Iterative TEst Case System

◮ regression testing ◮ consider dependencies ◮ automatic test case generation (ATCG) ◮ build on existing research & tools ◮ generate unit & component tests

7 / 21

slide-9
SLIDE 9

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

Automated Test Case Generation

◮ source code based ◮ black-box (function signature) vs. white-box (function

body)

8 / 21

slide-10
SLIDE 10

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

Semi-purification

◮ replace dependencies with parameters

1 f ( x ){ 2 i f GLOBAL_VAR : 3 r e t u r n dpGet(x) 4 e l s e : 5 r e t u r n −1 6 }

A non-pure function

1 f_sp ( x , a ,b){ 2 i f a : 3 r e t u r n b 4 e l s e : 5 r e t u r n −1 6 }

Semi-purified f(x)

a a a a 9 / 21

slide-11
SLIDE 11

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

Semi-purification

◮ replace dependencies with parameters

1 f ( x ){ 2 i f GLOBAL_VAR : 3 r e t u r n dpGet(x) 4 e l s e : 5 r e t u r n −1 6 }

A non-pure function

1 f_sp ( x , a ,b){ 2 i f a : 3 r e t u r n b 4 e l s e : 5 r e t u r n −1 6 }

Semi-purified f(x)

1 test_f_sp ( ) { 2 x = f ( " t e s t " , True , 5 ) // a c t 3 a s s e r t ( x == 5) // a s s e r t 4 }

Test case

9 / 21

slide-12
SLIDE 12

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

Semi-purification (cont.)

◮ replace dependencies with parameters

1 f u n c t i o n A ( x ){ 2 a = f u n c t i o n B ( x ) 3 r e t u r n a 4 } 5 6 f u n c t i o n B ( x ){ 7 b = GLOBAL_VAR 8 b++ 9 r e t u r n b 10 }

Function with SRC

1 functionA_sp ( x , y){ 2 a = f u n c t i o n B ( x , y) 3 r e t u r n a 4 } 5 6 functionB_sp ( x , y){ 7 b = y 8 b++ 9 r e t u r n b 10 }

Semi-purified w. SRC

10 / 21

slide-13
SLIDE 13

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

Semi-purification (cont.)

◮ replace dependencies with parameters

1 f u n c t i o n A ( x ){ 2 a = f u n c t i o n B ( x ) 3 r e t u r n a 4 } 5 6 f u n c t i o n B ( x ){ 7 b = GLOBAL_VAR 8 b++ 9 r e t u r n b 10 }

Function with SRC

1 functionA_sp ( x , y){ 2 a = f u n c t i o n B ( x , y) 3 r e t u r n a 4 } 5 6 functionB_sp ( x , y){ 7 b = y 8 b++ 9 r e t u r n b 10 }

Semi-purified w. SRC

10 / 21

slide-14
SLIDE 14

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

Semi-purification: Concept

◮ code contains dependencies

◮ global variables, data base values, subroutine calls,

  • ther resources

◮ manual way: test doubles (mocks, stubs, fakes, . . . )

[ME06]

◮ remove dependencies

◮ based on localization [SW03, SK13] ◮ input parameters instead of dependencies ◮ use any ATCG (black- and white-box) 11 / 21

slide-15
SLIDE 15

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

“Shortcut”

f (P){D} f (P ∪ PD){} TI, TS TI ∪ TID SP SP−1 ATCG

Figure: Test case generation schema

◮

P . . . Parameters

◮

D . . . Dependencies

◮

TI . . . Test Input

◮

TS . . . Test Setup Routine 12 / 21

slide-16
SLIDE 16

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

Identified Bottlenecks

◮ Loops ◮ Shared Subroutines ◮ Concurrency

13 / 21

slide-17
SLIDE 17

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

SP: Bottlenecks (cont.)

◮ Loops

1 s l e e p U n t i l R e a d y ( ){ // a = bool 2 3 w h i l e dpGet(notReadyDP) : 4 s l e e p ( 5 ) // s l e e p f o r 5 seconds 5 6 }

A semi-purified loop

14 / 21

slide-18
SLIDE 18

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

SP: Bottlenecks (cont.)

◮ Loops

1 s l e e p U n t i l R e a d y (a){ // a = bool 2 3 w h i l e a : // replaces dpGet(notReadyDP) 4 s l e e p ( 5 ) // s l e e p f o r 5 seconds 5 6 }

A semi-purified loop

Test Cases:

◮ a: False ⇒ loop not executed ◮ a: True ⇒ endless loop

14 / 21

slide-19
SLIDE 19

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

SP: Bottlenecks (cont.)

◮ Loops

1 s l e e p U n t i l R e a d y (a){ // a = [bool] 2 i = 0 3 w h i l e a[i] : // replaces dpGet(notReadyDP) 4 s l e e p ( 5 ) // s l e e p f o r 5 seconds 5 i++ 6 }

A semi-purified loop

Test Cases:

◮ a: [False]⇒ loop not executed ◮ a: [True, True, . . . , False] ⇒ loop execution

Questions:

◮ how long should the list be? ◮ how to modify correctly? ◮ Test modified code or w. threads?

14 / 21

slide-20
SLIDE 20

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

SP: Bottlenecks (cont.)

◮ Shared subroutine dependencies

1 var SPEED_VAR = 1 2 a d j u s t S p e e d ( ){ 3 x = getTheSpeed ( ) 4 i f x < 10 : 5 doubleTheSpeed ( ) 6 }

CUT

1 getTheSpeed ( ){ 2 r e t u r n SPEED_VAR 3 }

Subroutine 1

1 doubleTheSpeed ( ){ 2 speed = SPEED_VAR 3 SPEED_VAR

Subroutine 2

15 / 21

slide-21
SLIDE 21

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

SP: Bottlenecks (cont.)

◮ Shared subroutine dependencies

1 var SPEED_VAR = 1 2 a d j u s t S p e e d ( ){ 3 x = getTheSpeed ( ) 4 i f x < 10 : 5 doubleTheSpeed ( ) 6 }

CUT

1 getTheSpeed ( ){ 2 r e t u r n SPEED_VAR 3 }

Subroutine 1

1 doubleTheSpeed ( ){ 2 speed = SPEED_VAR 3 SPEED_VAR

Subroutine 2

15 / 21

slide-22
SLIDE 22

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

SP: Bottlenecks (cont.)

◮ Shared subroutine dependencies

1 2 a d j u s t S p e e d (a){ 3 x = getTheSpeed (a) 4 i f x < 10 : 5 doubleTheSpeed ( ) 6 }

CUT

1 getTheSpeed (a){ 2 r e t u r n a // SPEED_VAR 3 }

Subroutine 1

1 doubleTheSpeed ( ){ 2 speed = SPEED_VAR 3 SPEED_VAR

Subroutine 2

15 / 21

slide-23
SLIDE 23

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

SP: Bottlenecks (cont.)

◮ Shared subroutine dependencies

1 2 a d j u s t S p e e d (a, b){ 3 x = getTheSpeed (a) 4 i f x < 10 : 5 doubleTheSpeed (b) 6 }

CUT

1 getTheSpeed (a){ 2 r e t u r n a // SPEED_VAR 3 }

Subroutine 1

1 doubleTheSpeed (b){ 2 speed = b // SPEED_VAR 3 b = speed*2

Subroutine 2

15 / 21

slide-24
SLIDE 24

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

SP: Bottlenecks (cont.)

◮ Shared subroutine dependencies

1 2 a d j u s t S p e e d (a, b){ 3 x = getTheSpeed (a) 4 i f x < 10 : 5 doubleTheSpeed (b) 6 }

CUT

1 getTheSpeed (a){ 2 r e t u r n a // SPEED_VAR 3 }

Subroutine 1

1 doubleTheSpeed (b){ 2 speed = b // SPEED_VAR 3 b = speed*2

Subroutine 2

15 / 21

slide-25
SLIDE 25

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

SP: Bottlenecks (cont.)

◮ Shared subroutine dependencies

1 2 a d j u s t S p e e d (a, b){ 3 x = getTheSpeed (a) 4 i f x < 10 : 5 doubleTheSpeed (b) 6 }

CUT

1 getTheSpeed (a){ 2 r e t u r n a // SPEED_VAR 3 }

Subroutine 1

1 doubleTheSpeed (b){ 2 speed = b // SPEED_VAR 3 b = speed*2

Subroutine 2

15 / 21

slide-26
SLIDE 26

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

SP: Bottlenecks (cont.)

◮ Shared subroutine dependencies

1 2 a d j u s t S p e e d (a, b){ 3 x = getTheSpeed (a) 4 i f x < 10 : 5 doubleTheSpeed (b) 6 }

CUT

1 getTheSpeed (a){ 2 r e t u r n a // SPEED_VAR 3 }

Subroutine 1

1 doubleTheSpeed (b){ 2 speed = b // SPEED_VAR 3 b = speed*2

Subroutine 2

◮ BUT: a and b replace the same dependency

15 / 21

slide-27
SLIDE 27

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

SP: Bottlenecks (cont.)

◮ Concurrency manager Ctrl UI manager Device Drivers Dist manager DB manager EV manager

Figure: WinCC OA’s manager concept

16 / 21

slide-28
SLIDE 28

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

SP: Bottlenecks (cont.)

◮ access to ALL data points ◮ 100+ sub-systems ◮ discover dirty read/write scenarios (adjustSpeed())

16 / 21

slide-29
SLIDE 29

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

ITEC implementation

IDE SP engine CTRL test gen. test driver

code SP data test cases results

ITEC ATCG

SP code t e s t i n p u t s

Figure: ITEC workflow

17 / 21

slide-30
SLIDE 30

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

ITEC implementation

IDE SP engine CTRL test gen. test driver

code SP data test cases results

ITEC ATCG source code translator test input translator

SP CTRL CTRL test inputs SP tool code test inputs

TI generator

Figure: ITEC workflow

17 / 21

slide-31
SLIDE 31

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

Conclusion

Semi-purification

◮ remove dependencies ◮ facilitate unit tests generation ◮ use black-box techniques on all code

18 / 21

slide-32
SLIDE 32

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

Future works

What’s next?

◮ connect to test framework & roll-out ◮ intrinsic domain information ◮ compare different ATCG ◮ research into bottlenecks

19 / 21

slide-33
SLIDE 33

Considering Execution Environment Resilience: A White-Box Approach S.Klikovits,

  • D. Lawrence,
  • M. Gonzalez-

Berges,

  • D. Buchs

References

[ME06] Meszaros, G.: XUnit Test Patterns: Refactoring Test Code, Chapter 23. Test Double Patterns, pages 521–590. Prentice Hall PTR, Upper Saddle River, NJ, USA, (2006) [SC03] Sward, R. E., Chamillard, A. T.: Re-engineering Global Variables in Ada. In: Proc. 2004 ACM SIGAda international conference on Ada, pp. 29–34, ACM, New York, (2003). [SK13] Sankaranarayanan, H., Kulkarni, P.: Source-to-Source Refactoring and Elimination of Global Variables in C Programs. In: Journal of Software Engineering and Applications,

  • Vol. 6 No. 5, pp. 264–273, (2013).

20 / 21

slide-34
SLIDE 34

Considering Execution Environment Resilience: A White-Box Approach

Stefan Klikovits1,2, David PY Lawrence1,3, Manuel Gonzalez-Berges2, Didier Buchs1

1Université de Genève, Carouge, Switzerland 2CERN, Geneva, Switzerland 3Honeywell International Sarl., Rolle, Switzerland

Tuesday 11th August, 2015