[PPT] - 1 PowerPoint Presentation, free download

SLIDE 1

1

SLIDE 2

Validation

Purpose is to uncover probl Combination of reasoning a

Debugging

Finding out why a program Finding out why a program

Defensive programming

Programming with validatio

Testing ≠ debugging

test: reveals existence debug: pinpoint location+

problems and increase confidence

ning and test gram is not functioning as intended

2 gram is not functioning as intended lidation and debugging in mind tence of problem ation+cause of problem

SLIDE 3

3

SLIDE 4

Defect – mistake committed

Error – incorrect computation Failure – visible error: progr Debugging starts when a fail

Unit testing Unit testing Integration testing In the field

itted by a human utation program violates its specification a failure is observed

4

SLIDE 5

1.

Make errors impossible

Java makes memory overwrite bugs

2.

Don’t introduce defects

Correctness: get things right the first

3.

Make errors immediately visible

Local visibility of errors: best to fail Local visibility of errors: best to fail Example: checkRep() routine to che

4.

Last resort is debugging

Needed when effect of bug is distant Design experiments to gain informat

! Fairly easy in a program with good

unit tests etc.

! Much harder and more painstaking

exposure

bugs impossible

he first time to fail immediately

5

to fail immediately to check representation invariants distant from cause formation about bug

good modularity, representation hiding, specs, aking with a poor design, e.g., with rampant rep

SLIDE 6

" #$

In the language

Java makes memory overwrite b

In the protocols/libraries/mod

TCP/IP will guarantee that data i BigInteger will guarantee that th BigInteger will guarantee that th

In self7imposed conventions

Hierarchical locking makes dead Banning the use of recursion wil stack bugs go away Immutable data structures will g Caution: You must maintain the

$%

rite bugs impossible

es/modules

t data is not reordered that there will be no overflow

6

that there will be no overflow

ntions

s deadlock bugs impossible

n will make infinite recursion/insufficient

will guarantee behavioral equality ain the discipline

SLIDE 7

Get things right the first time

Don’t code before you think! Th If you're making lots of easy7to to7find bugs – don't use compile

Especially true, when debugging

Concurrency Difficult test and instrument env Difficult test and instrument env Program must meet timing deadl

Simplicity is key

Modularity

! Divide program into chunks tha ! Use abstract data types with we ! Use defensive programming; av

Specification

! Write specs for all modules, so

exists between each module and

#

nk! Think before you code. to7find bugs, you're also making hard7 mpiler as crutch

ugging is going to be hard

nt environments

7

nt environments deadlines

nks that are easy to understand ith well7defined interfaces ing; avoid rep exposure les, so that an explicit, well7defined contract ule and its clients

SLIDE 8

& #

If we can't prevent bugs, we small part of the program

Assertions: catch bugs early to contaminate (and be obsc Unit testing: when you test be confident that any bug yo be confident that any bug yo it's in the test driver) Regression testing: run tests changing code. If there is a mistake in the code you just

When localized to a single m can be found simply by study

#'%

s, we can try to localize them to a

s early, before failure has a chance e obscured by) further computation u test a module in isolation, you can bug you find is in that unit (unless

8 bug you find is in that unit (unless n tests as often as possible when re is a failure, chances are there's a

u just changed

gle method or small module, bugs studying the program text

SLIDE 9

Key difficulty of debugging i responsible for an observed p

A method may return an err error free, if there is prior co

The earlier a problem is obse The earlier a problem is obse

For example, frequently che the above problem

General approach: fail7fast

Check invariants, don't just Don't try to recover from bu

'%

gging is to find the code fragment rved problem

an erroneous result, but be itself rior corruption of representation

s observed, the easier it is to fix

9 s observed, the easier it is to fix

tly checking the rep invariant helps 't just assume them

m bugs – this just obscures them

SLIDE 10

()

Multiple passes

Each operate on a complex IR Lot of information passing Very complex Rep Invariant Code generation at the end Code generation at the end

Bugs

Compiler crashes ☺ Generated program is buggy

%

ex IR

Program

Front End

Intermediate Representation

10

Optimization

Intermediate Representation

Optimization

Intermediate Representation

Code Generation

Executable

SLIDE 11

*

This code fragment searches

Value is guaranteed to be in If that guarantee is broken ( exception and dies.

Temptation: make code more

*

11

rches an array for a value .

be in the array.
ken (by a bug), the code throws an

e more “robust” by not failing

SLIDE 12

*

Now at least the loop will alw

But no longer guaranteed th If rest of code relies on this,

*
12

ill always terminate

teed that n this, then problems arise later

SLIDE 13

*

Assertions let us document a

Abort program as soon as p

*

13

!"

ent and check invariants

n as problem is detected

SLIDE 14

$+

Insert checks galore with an

Precondition checks Consistency checks Bug7specific checks

Goal: stop the program as clo Goal: stop the program as clo

Use debugger to see where

+,

ith an intelligent checking strategy as close to bug as possible

14 as close to bug as possible

here you are, explore program a bit

SLIDE 15

+,"-

Precondition violated? Get

"-

15

!"

? Get an assertion!

SLIDE 16

)

#$%&

(

%)*+

Assertion not checked until we use the

Fault occurs when bad index inserted May be a long distance between fault $'( +,- 16 use the data erted into list n fault activation and error detection

SLIDE 17

,.#

++.$ ( %)$+

Perform check after all updates

between bug occurrence and b Can also write a single procedu structures, then scatter calls to code

++,

%&$'( /0+- 17

pdates to minimize distance and bug detection rocedure to check ALL data alls to this procedure throughout

SLIDE 18

!

++$% ( 12345%)$+

Bug shows up as 1234 in list

Check for that specific conditio

+,

%&$'( /0+- 18

ndition

SLIDE 19

+,$-

Should you include assertions and

Yes: stop program if check chance program will do som No: may need program to k not have such bad conseque Correct answer depends on Correct answer depends on

Ariane 5 – program halted becaus exception thrown but not handled

+
ns and checks in production code?

check fails 7 don’t want to take do something wrong m to keep going, maybe bug does sequences ds on context!

19 ds on context!

because of overflow in unused value, handled until top level, rocket crashes…

SLIDE 20

.

Whenever you find and fix a

Add a test for it Re7run all your tests

Why this is a good idea

Often reintroduce old bugs Often reintroduce old bugs Helps to populate test suite If a bug happened once, it c

Run regression tests as frequ

Automate process Make concise test sets, with

d fix a bug

bugs while fixing new ones

20 bugs while fixing new ones t suite with good tests ce, it could well happen again

frequently as you can afford to

s, with few superfluous tests

SLIDE 21

/

Often you would like to have som check fails Design a logging infrastructure

Dump events to a file (strings) Events have consistent format to Sometimes (usually for timing re Sometimes (usually for timing re not on disk Circular logs to avoid resource e

Important in debugging in custom

May not have access to the custo Only the log is available Information on the log to help re

/'

ve some indication of past when a

mat to enable efficient searches ing reasons) must keep lot in memory,

21

ing reasons) must keep lot in memory, urce exhaustion

n customer environments

e customer use help reproduce the bug

SLIDE 22

#

Bugs happen

Industry average: 10 bugs p

Bugs that are not immediatel

Found during integration tes Or reported by user Or reported by user

step 1 – Clarify symptom step 2 – Find and understand step 3 – Fix step 4 – Rerun all tests

#

bugs per 1000 lines of code (“kloc”)

diately localizable happen

tion testing

22 rstand cause, create test

SLIDE 23

Quick, easy bugs (few minut Medium bugs (hours) Hard bugs (small number of Really Bad bugs (many days Look for bugs in this order! Different debugging strategie minutes) ber of days) y days to never)

23 rder! rategies for each

SLIDE 24

"/

Hope for a quick bug, take a

Look at backtrace in the deb Look at code where you thin maybe use a debugger or a f Try to get lucky

Make the first shot quick! Do Look for medium bug with n

Use print statements Design an organized print s Legible, easy to read error m

Make the medium shot mediu

/

take a first quick shot

the debugger

u think there might be a problem,

r or a few print statements in

24 ck! Don’t get sucked in! with next shot

print strategy error messages

t medium! Don’t get sucked in!

SLIDE 25

&, (

Rebuild system from scratch Explain bug to a friend Make sure it is a bug – progr correctly and you don’t realiz Minimize input required to e Minimize input required to e Add checks to program

Minimize distance between Use binary search to narrow

Use logs to record events in h

(

cratch and reboot program may be working t realize it! d to exercise bug

25 d to exercise bug

tween error and detection narrow down possible locations

nts in history

SLIDE 26

.$

0%0 returns false for s = “The wworld is ggreat! Liffe i happy all of the ttime!” b = “very happy” even though “very happy” is a subst Wrong approach: try to trace the exe Right approach: try to reduce the siz

1/2%

Liffe is wwonderful! I am so vvery

26 substring of s the execution of substr for this case the size of the test case

SLIDE 27

.$

)$#,,!!" ),!!"#1-%) )$#,,!!-%),! )$#,,!!-%)!- )$#,,!!-%),!- )$#,,!!-%),!- )$#,,!!-%),- ),,!!-%),-" ),,!!-%),- ),,!-%),-" ),,-%),-" ),-%),-

$1

#1-%),!!-" 1-%),!!- !-" !-

"

27

"

"

SLIDE 28

3%

In general: find simplest inpu

Usually not the input that re

Start with data that revealed

Keep paring it down (binary Often leads directly to an un Often leads directly to an un

When not dealing with simpl

Think of “test input” as the trigger the bug Same basic idea

#%

st input that will provoke bug

that revealed existence of the bug

ealed bug

(binary search can help)

an understanding of the cause

28

an understanding of the cause

simple method calls

as the set of steps needed to reliably

SLIDE 29

%1

Take advantage of modularit

Start with everything, take a Start with nothing, add piec

Take advantage of modular r

Trace through program, vie Trace through program, vie

Can use binary search to spee

Bug happens somewhere be So can do binary search on

%1

dularity

, take away pieces until bug goes d pieces back in until bug appears

dular reasoning

m, viewing intermediate results

29 m, viewing intermediate results

to speed things up

ere between first and last statement ch on that ordered set of statements

SLIDE 30

6/+

" 6(, 6( * !6(+ " , +

6(# 6(

6(# 6( /""+,%+%# !7#%#%2 $##%# 8 8## 69: !7#%#% * ;# ,+!+

no problem yet

Check 30

%2
%
problem exists

Check intermediate result at half7way point

SLIDE 31

6/+

" 6(, 6( * !6(+ " , +

6(# 6(

6(# 6( /""+,%+%# !7#%#%2 $##%# 8 8## 69: !7#%#% * ;# ,+!+

no problem yet

Check intermediate result 31

%2
%
problem exists

result at half7way point Quickly home in

n bug in O(log n) time

by repeated subdivision

SLIDE 32

A

B C

Front end Optimization 1 Optimization 2 Optimization 3

Optimization 3

Optimization 4 Optimization 5 Optimization 6 Code generation Link and Run Test

+%

Class D E F G H I

32

SLIDE 33

A

B C

Front end Optimization 1 Optimization 2 Optimization 3 Optimization 3 Optimization 4 Optimization 5 Optimization 6 Code generation Link and Run Test

+%

Class D E F G H I 33

SLIDE 34

(

Sequential, deterministic program But the real world is not that nice…

Continuous input/environment cha Timing dependencies Concurrency and Parallelism

Bug occurs randomly Hard to reproduce

Use of debugger or assertions Only happens when under heavy lo Only happens once in a while

gram – bug is repeatable

nice…

nt changes

34

bug goes away eavy load

SLIDE 35

$(

Harsh environments

Bug is nondeterministic, difficul Can’t print or use debugger Can’t change timing of program

Build an event log (circular buffe Build an event log (circular buffe Log events during execution of pr When detect error, stop program a

(/'

ifficult to reproduce

gram (or bug has to do with timing)

buffer) 35 buffer)

n of program as it runs at speed
gram and examine logs

SLIDE 36

The bug is not where you thi

Ask yourself where it canno

Look for stupid mistakes firs

Reversed order of argument Spelling of identifiers: int h Spelling of identifiers: int h

! <,can help catch

Same object vs. equal: a == Failure to reinitialize a varia Deep vs. shallow copy

Make sure that you have corr

Recompile everything

4

u think it is

cannot be; explain why

es first, e.g.,

uments: Collections.copy(src,dest) : int hashcode()

36 : int hashcode()

p catch method name typos

l: a == b versus a.equals(b) a variable

e correct source code

SLIDE 37

Reconsider assumptions

E.g., has the OS changed? I Debug the code, not the com

Start documenting your syste

Gives a fresh angle, and hig Gives a fresh angle, and hig

Get help

We all develop blind spots Explaining the problem ofte

Walk away

Trade latency for efficiency One good reason to start ear

ged? Is there room on the hard drive?

he comments

r system

nd highlights area of confusion

37 nd highlights area of confusion spots m often helps ciency – ! tart early

SLIDE 38

Real Systems are…

Large and complex (duh!) Collection of modules, writ Complex input Many external interactions Non7deterministic

Replication can be an issue

Infrequent bug Instrumentation eliminates t

Bugs cross abstraction barrie Large time lag from corruptio

.%%

uh!) s, written by multiple people tions

38 nates the bug

barriers rruption to detection