Replaying and Isolating Failing Multi-Object Interactions Martin - - PowerPoint PPT Presentation

▶
replaying and isolating failing multi object interactions
SMART_READER_LITE
LIVE PREVIEW

Replaying and Isolating Failing Multi-Object Interactions Martin - - PowerPoint PPT Presentation

Apr 17, 2023 410 likes •1.27k views

Replaying and Isolating Failing Multi-Object Interactions Martin Burger Andreas Zeller Saarland University e-mail client written in Java 100,200 LOC ~ 1,600 Java classes 17 developers Actively developed from February

slide-1
SLIDE 1

Replaying and Isolating Failing Multi-Object Interactions

Martin Burger • Andreas Zeller Saarland University

slide-2
SLIDE 2
slide-3
SLIDE 3
  • e-mail client written in Java
  • 100,200 LOC
  • ~ 1,600 Java classes
  • 17 developers
  • Actively developed from February 2001 to August 2007
slide-4
SLIDE 4
slide-5
SLIDE 5
slide-6
SLIDE 6

Debugging 101

  • 1. reproduce the original failure
  • manually by using GUI
  • test driver that reproduce faulty behavior
slide-7
SLIDE 7

Debugging 101

  • 1. reproduce the original failure
  • manually by using GUI
  • test driver that reproduce faulty behavior
  • 2. fix the actual defect
  • focus on relevant behavior
  • simplify faulty behavior
slide-8
SLIDE 8

component

Faulty Component Behavior

set of classes

application

slide-9
SLIDE 9

component

Faulty Component Behavior

set of classes

interaction

  • method calls
  • field accesses
  • exception flow
  • ...

application

crosses border

slide-10
SLIDE 10

component

Faulty Component Behavior

set of classes

interaction

  • method calls
  • field accesses
  • exception flow
  • ...

failure

application

crosses border

slide-11
SLIDE 11

component

Reproducing

slide-12
SLIDE 12

component

Reproducing

  • 1. interaction
slide-13
SLIDE 13

component

Reproducing

  • 1. interaction
  • 2. … component

computes in the same way

slide-14
SLIDE 14

component

Reproducing

  • 1. interaction
  • 2. … component

computes in the same way

  • 3. … failure

reproduced

slide-15
SLIDE 15

Simplifying

slide-16
SLIDE 16

Simplifying

many interactions

slide-17
SLIDE 17

Simplifying

replaying causes

  • riginal failure

many interactions

slide-18
SLIDE 18

Simplifying

minimal set of interactions replaying causes

  • riginal failure
slide-19
SLIDE 19

Simplifying

minimal set of interactions replaying causes

  • riginal failure

delta debugging binary search

slide-20
SLIDE 20

Implementation: JINSI

  • debugging tool to reproduce and simplify

failing components in Java programs

  • reproduce

capture and replay of interactions at component level

  • simplify

delta debugging minimizes interactions

Isolating Relevant Component Interactions with JINSI. Orso + Joshi + Burger + Zeller

WODA 2006

slide-21
SLIDE 21

Original Import Run

address book importer (GUI) Columba

slide-22
SLIDE 22

Capture

address book importer (GUI) Columba JINSI instrumentation

slide-23
SLIDE 23

Capture

address book importer (GUI) Columba JINSI instrumentation event log “incoming call”

  • method name
  • arguments

written to

slide-24
SLIDE 24

Replay

address book importer (GUI) Columba

slide-25
SLIDE 25

Replay

address book importer (GUI) JINSI replaces Columba

slide-26
SLIDE 26

Replay

address book importer (GUI) JINSI replaces Columba event log read from

slide-27
SLIDE 27

Replay

address book importer (GUI) JINSI replaces Columba event log

  • riginal failure

reproduced read from

slide-28
SLIDE 28

Delta Debugging

address book importer (GUI) JINSI uses delta debugging event log

slide-29
SLIDE 29

Delta Debugging

address book importer (GUI) JINSI uses delta debugging event log

AddressbookImporter()

slide-30
SLIDE 30

Delta Debugging

address book importer (GUI) JINSI uses delta debugging event log

AddressbookImporter() init(file)

slide-31
SLIDE 31

Delta Debugging

address book importer (GUI) JINSI uses delta debugging event log

AddressbookImporter() init(file) wizardFinished()

slide-32
SLIDE 32

Delta Debugging

address book importer (GUI) JINSI uses delta debugging event log

AddressbookImporter() init(file) wizardFinished()

Problem

  • already minimal
  • too high level

Solution

  • much finer grained
slide-33
SLIDE 33

black box (JINSI 2006)

Inside the Component

coarse grained view

slide-34
SLIDE 34

Object Interactions

  • bject

finer grained view

slide-35
SLIDE 35

Object Interactions

  • bject

finer grained view

slide-36
SLIDE 36

Object Interactions

  • bject

finer grained view

slide-37
SLIDE 37

Object Interactions

  • bject

finer grained view

slide-38
SLIDE 38

Object Interactions

event log

captured by JINSI captured by JINSI border crossing interactions are captured by JINSI 2006

slide-39
SLIDE 39

Object Interactions

all object interactions are captured by JINSI 2008

slide-40
SLIDE 40

Object Trace

event log

  • bject

trace

for all objects

all object interactions are captured by JINSI 2008

slide-41
SLIDE 41

Importing the Addresses

Object Trace

addressbook. model

(19 classes)

ContactModel() setSortString() setFormattedName() setNickName() setFamilyName()

49,849 more

JINSI

slide-42
SLIDE 42

Importing the Addresses

Object Trace

addressbook. model

(19 classes)

ContactModel() setSortString() setFormattedName() setNickName() setFamilyName()

49,849 more

JINSI

  • calling

delta debugging: > 2 hours

slide-43
SLIDE 43

Object Slice

  • dynamic backward slice: subset that may

have influenced a specific object

  • focuses on objects, not on statements
  • objects are a natural abstraction
slide-44
SLIDE 44

Object Interactions

slide-45
SLIDE 45

Object Interactions

suspicious

  • bject
slide-46
SLIDE 46

Object Slice

suspicious

  • bject

at first, include suspicious object

slide-47
SLIDE 47

Object Slice

include interacting

  • bjects
slide-48
SLIDE 48

Object Slice

include objects that interact with those already in slice

slide-49
SLIDE 49

Object Slice

stop when reached largest connected graph

slide-50
SLIDE 50

Object Slice

simplified set of failure-inducing interactions within the component

slide-51
SLIDE 51

Object Slice

simplified set of failure-inducing interactions within the component

slide-52
SLIDE 52

Importing Addresses

Object Slice

addressbook. model

(19 classes)

ContactModel() setGivenName() setFamilyName() setSortString() getGivenName() JINSI getSortString() getPreferredEmail()

slide-53
SLIDE 53

Delta Debugging

apply delta debugging to

  • bject slice
slide-54
SLIDE 54

Delta Debugging

  • riginal

failure

minimized set

  • f failure-inducing

interactions within the component

slide-55
SLIDE 55

Importing Addresses

Delta Debugging

addressbook. model

(19 classes)

JINSI ContactModel() getPreferredEmail()

slide-56
SLIDE 56

Slicing + Delta Debugging

Incoming Interactions

49,855

  • riginal run

delta debugging

2

> 2 hours

minimized

slide-57
SLIDE 57

Slicing + Delta Debugging

Incoming Interactions

7

  • bject slice

49,855

  • riginal run

simplified

delta debugging

2

> 2 hours

minimized

slide-58
SLIDE 58

Slicing + Delta Debugging

Incoming Interactions

7

2

  • bject slice

delta debugging

49,855

  • riginal run

simplified minimized

delta debugging

2

> 2 hours

minimized

slide-59
SLIDE 59

Slicing + Delta Debugging

Incoming Interactions

7

2

  • bject slice

delta debugging

49,855

  • riginal run

simplified minimized

delta debugging

2

> 2 hours < 1 min

minimized

slide-60
SLIDE 60

Slicing + Delta Debugging

Incoming Interactions

7

2

  • bject slice

delta debugging

49,855

  • riginal run

simplified minimized

delta debugging

2

> 2 hours < 1 min < 1 min

minimized

slide-61
SLIDE 61

Method from Minimal Trace

public String getPreferredEmail() { Iterator it = getEmailIterator(); // get first item IEmailModel model = (IEmailModel) it.next(); // backwards compatiblity -> its not possible // anymore to create a model without email if (model == null) return null; return model.getAddress(); }

slide-62
SLIDE 62

Method from Minimal Trace

public String getPreferredEmail() { Iterator it = getEmailIterator(); // get first item IEmailModel model = (IEmailModel) it.next(); // backwards compatiblity -> its not possible // anymore to create a model without email if (model == null) return null; return model.getAddress(); }

slide-63
SLIDE 63

Method from Minimal Trace

public String getPreferredEmail() { Iterator it = getEmailIterator(); // get first item IEmailModel model = (IEmailModel) it.next(); // backwards compatiblity -> its not possible // anymore to create a model without email if (model == null) return null; return model.getAddress(); } IEmailModel model = null; if (it.hasNext()) model = (IEmailModel) it.next();

slide-64
SLIDE 64

Related Work

  • Test Factoring David Saff et al.

Selective Capture-Replay Shrinivas Joshi et al. JINSI is based on these methods

  • Efficient Test Case Minimization Andreas Leitner et al.

method calls in random tests; static slicing

  • ReCrash Shay Artzi et al.
  • ReCrash focuses on stack trace
  • JINSI focuses on object interactions
slide-65
SLIDE 65

Future Work

application failing but not defective component

slide-66
SLIDE 66

Future Work

application failing but not defective component

slide-67
SLIDE 67

Future Work

application when observing application: too many interactions failing but not defective component

slide-68
SLIDE 68

Future Work

application failing but not defective component

slide-69
SLIDE 69

Future Work

application failing but not defective component

slide-70
SLIDE 70

Future Work

Cause-Effect-Chain

final state e.g. contact model

slide-71
SLIDE 71

Future Work

Cause-Effect-Chain

final state e.g. contact model intermediate states

slide-72
SLIDE 72

Future Work

Cause-Effect-Chain

final state e.g. contact model input e.g. keystrokes in GUI intermediate states

slide-73
SLIDE 73

Future Work

Capture Component Level

application event log

JINSI captures component interaction

component

slide-74
SLIDE 74

event log

JINSI replays component interaction...

component component

Future Work

Replay Component Level

slide-75
SLIDE 75

event log

JINSI replays component interaction...

component

Future Work

Replay Component Level

slide-76
SLIDE 76

event log

  • bject

trace

for all objects JINSI replays component interaction... ... and captures

  • bject interactions

component

Future Work

Replay Component + Capture Objects

slide-77
SLIDE 77

Challenges

  • capture constructors
  • super call
  • constructor call as argument
  • who is calling
  • caller registry to get caller of method
slide-78
SLIDE 78

Debugging 101

  • 1. reproduce the original failure
  • manually by using GUI
  • test driver that reproduce faulty behavior
  • 2. fix the actual defect
  • focus on relevant behavior
  • simplify faulty behavior
slide-79
SLIDE 79

Debugging 101

  • 1. reproduce the original failure
  • manually by using GUI
  • test driver that reproduce faulty behavior
  • 2. fix the actual defect
  • focus on relevant behavior
  • simplify faulty behavior
slide-80
SLIDE 80

Debugging 101

  • 1. reproduce the original failure
  • manually by using GUI
  • test driver that reproduce faulty behavior
  • 2. fix the actual defect
  • focus on relevant behavior
  • simplify faulty behavior
slide-81
SLIDE 81

Contributions

slide-82
SLIDE 82

Contributions

slide-83
SLIDE 83

Contributions

slide-84
SLIDE 84

Contributions

slide-85
SLIDE 85

Contributions