When Abstraction Fails Andreas Zeller Saarland University - PowerPoint PPT Presentation

When Abstraction Fails Andreas Zeller Saarland University http://www.st.cs.uni-sb.de/

Mining 2

The First Bug (September 9, 1947) 3

4

How to Debug (Sommerville 2004) Design Repair Re-test Locate error error repair error program 5

How to Debug (not Sommerville 2004) Design error repair Locate error Repair error Re-test program 6

The Tra ffj c Principle T rack the problem R eproduce A utomate F ind Origins F ocus I solate C orrect 7

“The definitive book on debugging” – WALTER F. TICHY TU Karlsruhe WHY PROGRAMS FAIL On Display! A Guide to Systematic Debugging ANDREAS ZELLER 8

Socrates Plato Aristotle Ptolemy Parmenides Diogenes Euclid Pythagoras Heraclitus 9

Idealism Empiricism Abstract Concrete Deduction Induction Models Facts 10

Code Runs Static Dynamic Future Past Prevention Cure 11

12

bug.c double bug(double z[], int n) { int i, j; i = 0; for (j = 0; j < n; j++) { i = i + j + 1; z[i] = z[i] * (z[0] + 1.0); } return z[n]; } 13

What is the cause of this failure? 14

Causality Actual world Effect does not occur Alternate Effect does occur world Causes 15

bug.c } ✘ double bug(double z[], int n) { int i, j; i = 0; for (j = 0; j < n; j++) { i = i + j + 1; z[i] = z[i] * (z[0] + 1.0); } return z[n]; 16

empty.c } ✔ double bug(double z[], int n) { int i, j; i = 0; for (j = 0; j < n; j++) { i = i + j + 1; z[i] = z[i] * (z[0] + 1.0); } return z[n]; 17

Causes as Di fg erences Actual world empty.c: GCC works fine Alternate bug.c: GCC crashes world Cause: bug.c 18

Actual Causes “The” cause ( actual cause) is a minimal difference Actual cause 19

Isolating Causes } ✔ double bug(double z[], int n) { int i, j; i = 0; for (j = 0; j < n; j++) { i = i + j + 1; z[i] = z[i] * (z[0] + 1.0); } return z[n]; 20

Isolating Causes } ✔ double bug(double z[], int n) { int i, j; i = 0; for (j = 0; j < n; j++) { i = i + j + 1; z[i] = z[i] * (z[0] + 1.0); } return z[n]; 21

Isolating Causes } ✘ double bug(double z[], int n) { int i, j; i = 0; for (j = 0; j < n; j++) { i = i + j + 1; z[i] = z[i] * (z[0] + 1.0); } return z[n]; 22

Isolating Causes double bug(double z[], int n) { int i, j; i = 0; for (j = 0; j < n; j++) { i = i + j + 1; z[i] = z[i] * (z[0] + 1.0); } return z[n]; } Actual cause narrowed down 23

Isolating Causes } ✔ double bug(double z[], int n) { int i, j; i = 0; for (j = 0; j < n; j++) { i = i + j + 1; z[i] = z[i] * (z[0] + 1.0); } return z[n]; 24

Isolating Causes } ✘ double bug(double z[], int n) { int i, j; i = 0; for (j = 0; j < n; j++) { i = i + j + 1; z[i] = z[i] * (z[0] + 1.0); } return z[n]; 25

Isolating Causes double bug(double z[], int n) { int i, j; i = 0; for (j = 0; j < n; j++) { i = i + j + 1; z[i] = z[i] * (z[0] + 1.0); } return z[n]; } Actual cause of the GCC crash 26

Isolating Causes Alternate world Actual world ✔ ✘ ? Test Mixed world 27

Isolating Causes Alternate world Actual world “+ 1.0” ✔ ✘ ? Test Mixed world 28

The Scientific Method Problem Report Hypothesis is supported: Code refine hypothesis Observation Prediction Hypothesis Experiment + Conclusion Run Hypothesis is rejected: create new hypothesis More Runs Diagnosis 29

The Scientific Method 30

Circumstances • Any aspect that may influence a problem is a circumstance: • Aspects of the problem environment • Individual steps of the problem history 31

Experimentation • By experimentation , one finds out whether a circumstance is relevant or not: • Omit the circumstance and try to reproduce the problem. • The circumstance is relevant iff the problem no longer occurs. 32

Mozilla Bug #24735 Ok the following operations cause mozilla to crash consistently on my machine -> Start mozilla -> Go to bugzilla.mozilla.org -> Select search for bug -> Print to file setting the bottom and right margins to .50 (I use the file /var/tmp/netscape.ps) -> Once it's done printing do the exact same thing again on the same file (/var/tmp/netscape.ps) -> This causes the browser to crash with a segfault 33

<SELECT NAME="op_sys" MULTIPLE SIZE=7> <OPTION VALUE="All">All<OPTION VALUE="Windows 3.1">Windows 3.1<OPTION VALUE="Windows 95">Windows 95<OPTION VALUE="Windows 98">Windows bugzilla.mozilla.org 98<OPTION VALUE="Windows ME">Windows ME<OPTION VALUE="Windows 2000">Windows 2000<OPTION VALUE="Windows NT">Windows NT<OPTION VALUE="Mac System 7">Mac System 7<OPTION VALUE="Mac System 7.5">Mac System 7.5<OPTION VALUE="Mac System 7.6.1">Mac System 7.6.1<OPTION VALUE="Mac System 8.0">Mac System 8.0<OPTION VALUE="Mac System 8.5">Mac System 8.5<OPTION VALUE="Mac System 8.6">Mac System 8.6<OPTION VALUE="Mac System 9.x">Mac System 9.x<OPTION VALUE="MacOS X">MacOS X<OPTION VALUE="Linux">Linux<OPTION VALUE="BSDI">BSDI<OPTION VALUE="FreeBSD">FreeBSD<OPTION VALUE="NetBSD">NetBSD<OPTION VALUE="OpenBSD">OpenBSD<OPTION VALUE="AIX">AIX<OPTION VALUE="BeOS">BeOS<OPTION VALUE="HP-UX">HP-UX<OPTION What’s relevant in here? VALUE="IRIX">IRIX<OPTION VALUE="Neutrino">Neutrino<OPTION VALUE="OpenVMS">OpenVMS<OPTION VALUE="OS/2">OS/2<OPTION VALUE="OSF/ 1">OSF/1<OPTION VALUE="Solaris">Solaris<OPTION VALUE="SunOS">SunOS<OPTION VALUE="other">other</SELECT> </td> <td align=left valign=top> <SELECT NAME="priority" MULTIPLE SIZE=7> <OPTION VALUE="--">--<OPTION VALUE="P1">P1<OPTION VALUE="P2">P2<OPTION VALUE="P3">P3<OPTION VALUE="P4">P4<OPTION VALUE="P5">P5</SELECT> 34

Why simplify? • Ease of communication. A simplified test case is easier to communicate. • Easier debugging. Smaller test cases result in smaller states and shorter executions. • Identify duplicates. Simplified test cases subsume several duplicates. 35

The Gecko BugAThon • Download the Web page to your machine. • Using a text editor, start removing HTML from the page. Every few minutes, make sure it still reproduces the bug. • Code not required to reproduce the bug can be safely removed. • When you’ve cut away as much as you can, you’re done. 36

Rewards 5 bugs - invitation to the Gecko launch party 10 bugs - the invitation, plus an attractive Gecko stuffed animal 12 bugs - the invitation, plus an attractive Gecko stuffed animal autographed by Rick Gessner, the Father of Gecko 15 bugs - the invitation, plus a Gecko T-shirt 20 bugs - the invitation, plus a Gecko T-shirt signed by the whole raptor team 37

Binary Search • Proceed by binary search. Throw away half the input and see if the output is still wrong. • If not, go back to the previous state and discard the other half of the input. HTML input ✔ ✔ ✘ ✘ 38

Simplified Input <SELECT NAME="priority" MULTIPLE SIZE=7> • Simplified from 896 lines to one single line • Required 12 tests only 39

Benefits • Ease of communication. All one needs is “Printing <SELECT> crashes”. • Easier debugging. We can directly focus on the piece of code that prints <SELECT>. • Identify duplicates. Check other test cases whether they’re <SELECT>-related, too. 40

Why automate? • Manual simplification is tedious . • Manual simplification is boring . • We have machines for tedious and boring tasks. 41

Basic Idea • We set up an automated test that checks whether the failure occurs or not (= Mozilla crashes when printing or not) • We implement a strategy that realizes the binary search. 42

Automated Test 1. Launch Mozilla 2. Replay (previously recorded) steps from problem report 3. Wait to see whether • Mozilla crashes (= the test fails ) • Mozilla still runs (= the test passes ) 4. If neither happens, the test is unresolved 43

Binary Search ✘ <SELECT NAME="priority" MULTIPLE SIZE=7> ✔ <SELECT NAME="priority" MULTIPLE SIZE=7> What do we do if both halves pass? ✔ <SELECT NAME="priority" MULTIPLE SIZE=7> <SELECT NAME="priority" MULTIPLE SIZE=7> ✔ ✘ <SELECT NAME="priority" MULTIPLE SIZE=7> ✘ <SELECT NAME="priority" MULTIPLE SIZE=7> ✔ <SELECT NAME="priority" MULTIPLE SIZE=7> 44

Configuration Circumstance δ All circumstances C = { δ 1 , δ 2 , . . . } Configuration c ⊆ C c = { δ 1 , δ 2 , . . . δ n } 45