Using Feature Locality: Can We Motivation Leverage History to Avoid - - PowerPoint PPT Presentation

▶

Feb 10, 2024 141 likes •677 views

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Using Feature Locality: Can We Motivation Leverage History to Avoid Failures Failure Avoidance During Reconfiguration? Study Results

SLIDE 1

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration?

Brady J. Garvin∗ Myra B. Cohen Matthew B. Dwyer

University of Nebraska–Lincoln

September 4, 2011

Slide 1/52

SLIDE 2

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Outline

1

Motivation

2

Failure Avoidance

3

Study

4

Results

5

Conclusions and Future Work

Slide 2/52

SLIDE 3

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Spirit

Adirondack, Spirit’s first rock, whose investigation would be interrupted by a bout of insomnia, fever, and delirium.

Slide 3/52

SLIDE 4

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

What we Know from Testing Highly-Configurable Systems

Spirit, a highly-configurable system, was exhibiting a configuration-dependent failure [Adler 2004]. Highly-configurable systems tend to do this [Kuhn et al. 2004,Yilmaz et al. 2006,Qu et al. 2008]. Configuration-dependence makes failures harder to find in testing, but easier to work around.

Slide 4/52

SLIDE 5

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

What we Know from Designing Autonomous Systems

Adaptations, including online reconfiguration, affect reliability. Reliability can be predicted fairly well from a few key parameters [Ahmed et al. 2010]. But until the system has been running for a while under various configurations, we will have observed only a handful of discrete failures, which makes it difficult to build a reliability model and identify those parameters.

Slide 5/52

SLIDE 6

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

What we Know from Fault Prediction

Faults exhibit many forms of locality, both temporal and spacial [Hassan et al. 2005,Kim et al. 2007]. In consequence, failures also show locality.

Slide 6/52

SLIDE 7

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Hypothesis

Reconfiguration workarounds effective against one failure tend to be effective against others. We call this phenomenon feature locality of failures.

Slide 7/52

SLIDE 8

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Outline

1

Motivation

2

Failure Avoidance

3

Study

4

Results

5

Conclusions and Future Work

Slide 8/52

SLIDE 9

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Configuration Space

Navigation IDD . . . Blind Stowed . . . Hazard Avoidance APXS . . . Autonomous MB . . . Wheel Wiggle RAT . . .

Slide 9/52

SLIDE 10

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Configuration Space

Navigation IDD . . . Blind Stowed . . . Hazard Avoidance APXS . . . Autonomous MB . . . Wheel Wiggle RAT . . .

Slide 10/52

SLIDE 11

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Configuration Space

Navigation

Slide 11/52

SLIDE 12

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Configuration Space

Navigation IDD . . . Blind Stowed . . . Hazard Avoidance APXS . . . Autonomous MB . . . Wheel Wiggle RAT . . .

Slide 12/52

SLIDE 13

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Configuration Space

Navigation IDD

Slide 13/52

SLIDE 14

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Configuration Space

Navigation IDD . . . Blind Stowed . . . Hazard Avoidance APXS . . . Autonomous MB . . . Wheel Wiggle RAT . . .

Slide 14/52

SLIDE 15

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Configuration Space

Navigation IDD

Slide 15/52

SLIDE 16

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Configuration Space

Navigation IDD

Slide 16/52

SLIDE 17

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Configuration Space

Navigation IDD

Slide 17/52

SLIDE 18

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Algorithms

1 Analyze failures 2 Guard against dangerous configurations 3 Workaround new failures Slide 18/52

SLIDE 19

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Algorithm 1: Analyze

Navigation IDD

Slide 19/52

SLIDE 20

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Algorithm 1: Analyze

Navigation IDD

Slide 20/52

SLIDE 21

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Algorithm 1: Analyze

Navigation IDD

Slide 21/52

SLIDE 22

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Algorithm 1: Analyze

Navigation IDD

Slide 22/52

SLIDE 23

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Algorithm 1: Analyze

Navigation IDD

Slide 23/52

SLIDE 24

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Algorithm 2: Guard

Navigation IDD

Slide 24/52

SLIDE 25

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Algorithm 2: Guard

Navigation IDD

Slide 25/52

SLIDE 26

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Algorithm 3: Reconfigure

Navigation IDD

Slide 26/52

SLIDE 27

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Algorithm 3: Reconfigure

Navigation IDD

Slide 27/52

SLIDE 28

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Algorithm 3: Reconfigure

Navigation IDD

Slide 28/52

SLIDE 29

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Algorithm 3: Reconfigure

Navigation IDD

Slide 29/52

SLIDE 30

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Algorithm 3: Reconfigure

Navigation IDD

Slide 30/52

SLIDE 31

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Outline

1

Motivation

2

Failure Avoidance

3

Study

4

Results

5

Conclusions and Future Work

Slide 31/52

SLIDE 32

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

A First Approximation of an Autonomous System

I n i t i a l C

fi g . I n i t i a l C

fi g . A d a p t a t i

N e w C

fi g . A d a p t a t i

N e w C

fi g . A d a p t a t i

N e w C

fi g . Autonomous System GCC GCC also has a large configuration space, an open bug database, and a broad and active user community whose members tend to write good bug reports.

Slide 32/52

SLIDE 33

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

GCC

Versions 4.4.0–4.4.2 Features:

Command-line options that, when toggled, neither require changes to the input nor alter the semantics of the output 339 in 168 groups, with 132 clauses to encode feature constraints

Slide 33/52

SLIDE 34

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

GCC Failures

Reported 360 Incomplete 7 Platform-Dependent 92 Require Alternate Bootstrap Options 3 Nonfunctional 13 Nondeterministic 8 Remaining 237 Fixed before Release 109 Reproducible on Releases 128

Slide 34/52

SLIDE 35

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Research Questions

RQ1: Can failures be avoided by reconfiguration? RQ2: To what extent do failures depend on similar combinations of features? RQ3: Can feature locality be exploited to avoid failures?

Slide 35/52

SLIDE 36

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Outline

1

Motivation

2

Failure Avoidance

3

Study

4

Results

5

Conclusions and Future Work

Slide 36/52

SLIDE 37

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

RQ1: High-Priority Failures Tend to have Reconfiguration Workarounds

4.4.0 4.4.1 4.4.2 P1 3 of 5 (60%) 3 of 4 (75%) 2 of 3 (67%) P2 7 of 23 (30%) 7 of 19 (36%) 5 of 17 (29%) P3 21 of 84 (25%) 20 of 80 (25%) 18 of 75 (24%) P4 0 of 11 (0%) 0 of 11 (0%) 0 of 8 (0%) P5 0 of 3 (0%) 0 of 3 (0%) 0 of 2 (0%) Total 31 of 126 (25%) 30 of 117 (26%) 25 of 105 (24%)

Slide 37/52

SLIDE 38

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

RQ1: High-Priority Failures Tend to have Reconfiguration Workarounds

4.4.0 4.4.1 4.4.2 P1 3 of 5 (60%) 3 of 4 (75%) 2 of 3 (67%) P2 7 of 23 (30%) 7 of 19 (36%) 5 of 17 (29%) P3 21 of 84 (25%) 20 of 80 (25%) 18 of 75 (24%) P4 0 of 11 (0%) 0 of 11 (0%) 0 of 8 (0%) P5 0 of 3 (0%) 0 of 3 (0%) 0 of 2 (0%) Total 31 of 126 (25%) 30 of 117 (26%) 25 of 105 (24%)

Slide 38/52

SLIDE 39

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

RQ1: High-Priority Failures Tend to have Reconfiguration Workarounds

4.4.0 4.4.1 4.4.2 P1 3 of 5 (60%) 3 of 4 (75%) 2 of 3 (67%) P2 7 of 23 (30%) 7 of 19 (36%) 5 of 17 (29%) P3 21 of 84 (25%) 20 of 80 (25%) 18 of 75 (24%) P4 0 of 11 (0%) 0 of 11 (0%) 0 of 8 (0%) P5 0 of 3 (0%) 0 of 3 (0%) 0 of 2 (0%) Total 31 of 126 (25%) 30 of 117 (26%) 25 of 105 (24%)

Slide 39/52

SLIDE 40

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

RQ1: High-Priority Failures Tend to have Reconfiguration Workarounds

4.4.0 4.4.1 4.4.2 P1 3 of 5 (60%) 3 of 4 (75%) 2 of 3 (67%) P2 7 of 23 (30%) 7 of 19 (36%) 5 of 17 (29%) P3 21 of 84 (25%) 20 of 80 (25%) 18 of 75 (24%) P4 0 of 11 (0%) 0 of 11 (0%) 0 of 8 (0%) P5 0 of 3 (0%) 0 of 3 (0%) 0 of 2 (0%) Total 31 of 126 (25%) 30 of 117 (26%) 25 of 105 (24%)

Slide 40/52

SLIDE 41

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

RQ2: Reconfiguration Workarounds Tend to Avoid Multiple Failures

5 10 15

35 339 Single-Feature Reconfigurations in GCC Number of Failures Avoided

Slide 41/52

SLIDE 42

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

RQ2: Reconfiguration Workarounds Tend to Avoid Multiple Failures

5 10 15

35 339 Single-Feature Reconfigurations in GCC Number of Failures Avoided

Slide 42/52

SLIDE 43

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

RQ3: Biased Random Reconfiguration as an Alternative to Algorithm 3

Choose a single-feature reconfiguration randomly, reconfigurations that will fail with probability p and reconfigurations that will succeed with probability q

Slide 43/52

SLIDE 44

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

RQ3: Failure History Tends to Suggest Effective Reconfiguration Workarounds

5 10 15 20 25 10 20 30

Reconfiguration Attempts after Detection Failures Avoided in GCC 4.4.0 Our Technique Average Case, Biased Random Reconfiguration with q/p = 1, 2, 4, 8, 16, 32, 64

Slide 44/52

SLIDE 45

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

RQ3: Failure History Tends to Suggest Effective Reconfiguration Workarounds

5 10 15 20 25 10 20 30

Reconfiguration Attempts after Detection Failures Avoided in GCC 4.4.0 Our Technique Average Case, Biased Random Reconfiguration with q/p = 1, 2, 4, 8, 16, 32, 64

Slide 45/52

SLIDE 46

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

RQ3: Failure History Tends to Suggest Effective Reconfiguration Workarounds

5 10 15 20 25 10 20 30

Reconfiguration Attempts after Detection Failures Avoided in GCC 4.4.0 Our Technique Average Case, Biased Random Reconfiguration with q/p = 1, 2, 4, 8, 16, 32, 64

Slide 46/52

SLIDE 47

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

RQ3: Failure History Tends to Suggest Effective Reconfiguration Workarounds

5 10 15 20 25 10 20 30

Reconfiguration Attempts after Detection Failures Avoided in GCC 4.4.0 Our Technique Average Case, Biased Random Reconfiguration with q/p = 1, 2, 4, 8, 16, 32, 64

Slide 47/52

SLIDE 48

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Outline

1

Motivation

2

Failure Avoidance

3

Study

4

Results

5

Conclusions and Future Work

Slide 48/52

SLIDE 49

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Conclusions

In systems like GCC, where we can

recreate failures, reasonably run test cases under several configurations, and change configuration in the field,

feature locality allows us to avoid failures.

Slide 49/52

SLIDE 50

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Future Work

The next step is to try extending these results to autonomous systems, by

identifying autonomous systems with which we can experiment, generalizing the approach to accommodate those systems, and incorporating our algorithms into the utility function that drives adaptation.

Slide 50/52

SLIDE 51

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Questions

This work is supported in part by the National Science Foundation through awards CNS-0720654, CCF-0747009, CCF-0915526, and CFDA-47.076, by the Air Force Office of Scientific Research through awards FA9550-09-1-0129 and FA9550-09-1-0687, the National Aeronautics and Space Administration under grant number NNX08AV20A. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the position or policy of NSF, AFOSR, or NASA.

Slide 51/52

SLIDE 52

Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration? ASAS 2011 Motivation Failure Avoidance Study Results Conclusions and Future Work

Jump to Slide (1–52)

Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Slide 6 Slide 7 Slide 8 Slide 9 Slide 10 Slide 11 Slide 12 Slide 13 Slide 14 Slide 15 Slide 16 Slide 17 Slide 18 Slide 19 Slide 20 Slide 21 Slide 22 Slide 23 Slide 24 Slide 25 Slide 26 Slide 27 Slide 28 Slide 29 Slide 30 Slide 31 Slide 32 Slide 33 Slide 34 Slide 35 Slide 36 Slide 37 Slide 38 Slide 39 Slide 40 Slide 41 Slide 42 Slide 43 Slide 44 Slide 45 Slide 46 Slide 47 Slide 48 Slide 49 Slide 50 Slide 51 Slide 52 Slide 52/52