Software Reliability Estimation Based on Static Error Detection M. - - PowerPoint PPT Presentation

▶
software reliability estimation based on static error
SMART_READER_LITE
LIVE PREVIEW

Software Reliability Estimation Based on Static Error Detection M. - - PowerPoint PPT Presentation

Sep 16, 2022 559 likes •886 views

7 th Central and Eastern European Software Engineering Conference in Russia - CEE-SECR 2011 October 31 November 3, Moscow Software Reliability Estimation Based on Static Error Detection M. Moiseev, M. Glukhikh , A. Karpenko, H. Richter

slide-1
SLIDE 1

7th Central and Eastern European Software Engineering Conference in Russia - CEE-SECR 2011

October 31 – November 3, Moscow

  • M. Moiseev, M. Glukhikh, A. Karpenko,
  • H. Richter

Software Reliability Estimation Based on Static Error Detection

slide-2
SLIDE 2

Importance of Software Reliability Analysis

Software Reliability Estimation Based on Static Error Detection 2

Modern software contains errors Errors can lead to disasters Software Reliability Analysis Error detection should be

  • rganized
slide-3
SLIDE 3

Known Approaches

Software Reliability Estimation Based on Static Error Detection 3

Heuristics approaches Dynamic approach Architecture- based approach Program metrics Development process

slide-4
SLIDE 4

Known Approaches – Program Metrics

Based on simple code properties, such as

number of statements number of conditions number of loops number of functions ...

Software Reliability Estimation Based on Static Error Detection 4

slide-5
SLIDE 5

Known Approaches – Development Process Metrics

Based on development process properties, such as

duration of development number & qualification of developers number & qualification of testers methodology used automation tools used

Software Reliability Estimation Based on Static Error Detection 5

slide-6
SLIDE 6

Known Approaches – Others

Runtime

Based on failures observed at run-time

Architecture-based

Based on known reliability of program components

Software Reliability Estimation Based on Static Error Detection 6

slide-7
SLIDE 7

Our Approach

Based on source code static analysis Delivers

Ranking of errors (based on failure probability) Reliability characteristics

Limitations

Single-threaded C programs

Error types

uninitialized variable use incorrect pointer dereference pointer out of bounds

Software Reliability Estimation Based on Static Error Detection 7

slide-8
SLIDE 8

Features of Our Approach

Analysis of a program model Analysis of all possible execution paths Advantages

Reliability estimations is based on real errors Results are applicable for any exploitation conditions Makes debugging more effective

Drawbacks

Does not consider quantitative time Does not consider normal program exploitation Execution path probability estimation False positives problem

Software Reliability Estimation Based on Static Error Detection 8

slide-9
SLIDE 9

Program Classes

Programs Computational Server

Software Reliability Estimation Based on Static Error Detection 9

slide-10
SLIDE 10

Reliability characteristics used

Computational programs

Probability of whole program

successful execution P(∞)

Server programs

Probability of n statements

successful execution P(n)

Mean executed statement

number before failure

Software Reliability Estimation Based on Static Error Detection 10

n

slide-11
SLIDE 11

Algorithms

Software Reliability Estimation Based on Static Error Detection 11

Model building State determination Error detection Error ranking Reliability estimation

slide-12
SLIDE 12

Program Model Features

Control flow graph Three-operand assignment form A = B op C If and Phi statements

Software Reliability Estimation Based on Static Error Detection 12

If Phi Phi If

slide-13
SLIDE 13

State Determination Algorithms

State representation Control flow analysis

Statement analysis

Sequential If statement analysis Phi statement analysis

Loop analysis Interprocedural analysis

Software Reliability Estimation Based on Static Error Detection 13

slide-14
SLIDE 14

Program State Representation

Based on objects, values, and probabilities

set of triples state probability

Object values

intervals pointers resource descriptors

Software Reliability Estimation Based on Static Error Detection 14

( ) { }

jk k j

p v

  • Q

, , =

( )

Q P

slide-15
SLIDE 15

Probability normalization

Control flow normalization State normalization

Software Reliability Estimation Based on Static Error Detection 15

( )

( )

( )

( )

∑ ∑

∈ ∈ ∀

=

s Output Q

  • ut

j s Input Q in j

  • ut

j in j

Q P Q P

( )

( )

( )

Q P p Q p v

  • Q

p v

  • jk

jk k j j

jk k j

= ⇒ ∈ ∃ ∀

∑

∈ ∀ , ,

, , :

slide-16
SLIDE 16

Sequential Statement Analysis

Software Reliability Estimation Based on Static Error Detection 16

a = b + c

( ) ( ) ( ) ( )

      = ,... 1 , 6 .. 3 , , 1 , 2 .. 1 , c b Qin

( ) ( ) { }

,... 1 , 8 .. 4 , a Qout =                                                       = ,... 4 1 , 6 , , 4 1 , 5 , , 4 1 , 4 , , 4 1 , 3 , 2 1 , 2 , , 2 1 , 1 , c c c c b b                                                 = ,... 8 1 , 8 , , 4 1 , 7 , , 4 1 , 6 , , 4 1 , 5 , , 8 1 , 4 , a a a a a

slide-17
SLIDE 17

If Statement Analysis

True and false combination consideration Normalization of state probabilities Normalization of non-affected triples probabilities

Software Reliability Estimation Based on Static Error Detection 17

( )

( )

( )

( )

. ,

, , , ,

∑ ∏ ∑ ∏

∈ ∈ ∈ ∈

= =

false jk k j true jk k j

C c c p v

  • jk

false C c c p v

  • jk

true

p Q P p Q P

slide-18
SLIDE 18

If Statement Analysis Example

172 combinations where a < b 28 combinations where a >= b Normalization: 0.86 for true, 0.14 for false

Software Reliability Estimation Based on Static Error Detection 18

( ) ( ) ( ) ( )

      = ,... 1 , 23 .. 4 , , 1 , 10 .. 1 , b a Qin

true

Q

false

Q

( ) ( ) ( ) ( )

     86 . , 23 .. 4 , 86 . , 10 .. 1 , b a

( ) ( ) ( ) ( )

     14 . , 10 .. 4 , 14 . , 10 .. 4 , b a

slide-19
SLIDE 19

Phi Statement Analysis

Identical triples are added together Control flow normalization

Software Reliability Estimation Based on Static Error Detection 19

( ) ( ) ( )

  • ut

jk jk k j in jk k j in jk k j k j

Q r p v

  • Q

r v

  • Q

p v

  • v
  • ∈

+ ⇒ ∈ ∈ ∀ , , , , , , , : ,

2 1

In1 In2 Phi

( ) ( ) ( )

in in

  • ut

Q P Q P Q P

2 1

+ =

slide-20
SLIDE 20

Error Detection

Based on incorrect values in state

uninitialized variable use pointer dereference

  • ut of bounds

correct if

  • therwise error is detected

Software Reliability Estimation Based on Static Error Detection 20

( )

k noninit j

p v

  • ,

,

( )

k noninit j

p v

  • ,

,

( )

k invalid j

p v

  • ,

,

( )

k null j

p v

  • ,

,

( ) ( )

k j j i

p

  • ffset
  • ,

, ,

( )

j j

  • sizeof
  • ffset <

≤

slide-21
SLIDE 21

Error Inhibition

Software Reliability Estimation Based on Static Error Detection 21

  • bj use

(obj, valid, p1) (obj, invalid, p2) P(Q)=p1+p2 (obj, valid, p1) P(Q)=p1

slide-22
SLIDE 22

Error Ranking

Errors are sorted according to probability of

  • ccurrence

Most dangerous errors can be corrected first

Probabilities are summarized

for same errors in the same statement

Software Reliability Estimation Based on Static Error Detection 22

slide-23
SLIDE 23

Overall reliability estimation

probability of successful execution probability of n statements successful execution mean executed statements number before failure

Software Reliability Estimation Based on Static Error Detection 23

( )

∑

=

⋅ + − =

max

) 1 ( ) (

n n

n n P n P n

( )

∑

=

executed statements n

Q P n P ) (

( )

∑

=

statements end

Q P n P ) (

slide-24
SLIDE 24

Implementation

AEGIS static analyzer

analysis of C/C++ source code interval, points to, resource analysis loop & interprocedural analysis spread range of program errors detected

Results

error ranking table P(n) table P(∞) mean executed statements number before failure

Software Reliability Estimation Based on Static Error Detection 24

slide-25
SLIDE 25

Experiments made

Purpose

Testing of our approach Debugging example

Software Reliability Estimation Based on Static Error Detection 25

Test programs Students' projects Real-world projects (embedded software)

slide-26
SLIDE 26

Sample of reliability analysis

while (!(feof(f))) // 0.5 { i = t = 0; // Failure in one of three cases prov(&t, strlen(st), st); }

Probability of successful execution is

0.75 = 0.5 + 0.5 * 0.33 + 0.5 * 0.332 + ...

Software Reliability Estimation Based on Static Error Detection 26

slide-27
SLIDE 27

Amount of errors in real-world projects

Software Reliability Estimation Based on Static Error Detection 27

20 40 60 80 100 A B C D E F G H I J K L Projec t name E rror number

  • More than 500 errors, 2/3 of considered types
  • Density about 0.8/1KLOC
slide-28
SLIDE 28

Distribution of error number

Software Reliability Estimation Based on Static Error Detection 28

25 50 75 100 125 150 175 1.E

  • 06 1.E
  • 05 1.E
  • 04 1.E
  • 03 1.E
  • 02 1.E
  • 01

0,25 0,5 1,0 E rror probability E rror number

slide-29
SLIDE 29

Debugging results

Software Reliability Estimation Based on Static Error Detection 29

0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 32 33 34 35 36 37 38 39 40 41

Original C

  • rrected

n, Ms tatements

slide-30
SLIDE 30

Directions for Future Work

Reliability estimation

Annotations for path probability estimations Run-time analysis for path probability estimation Execution time estimation

Static analysis itself

Soundness & precision Parallel program analysis Annotations for functional error detection

Software Reliability Estimation Based on Static Error Detection 30

slide-31
SLIDE 31

Conclusion

Approach for software reliability estimation

based on error detection using static analysis

Implementation in AEGIS tool (prototype)

ranking of errors by the probability of occurrence probability of successful execution probability of N statement successful execution mean number of executed statements before failure

Software Reliability Estimation Based on Static Error Detection 31

slide-32
SLIDE 32

Contacts

Saint Petersburg State Polytechnical University Digitek Labs http://digiteklabs.ru Mikhail Glukhikh, Mikhail Moiseev, Anatoly Karpenko E-mail: glukhikh@kspt.ftk.spbstu.ru E-mail: mikhail.moiseev@gmail.com E-mail: karpenko@kspt.ftk.spbstu.ru Clausthal University of Technology Harald Richter E-mail: hri@tu-clausthal.de

32 Software Reliability Estimation Based on Static Error Detection