without Regret Barbara Jobstmann EPFL and Jasper DA CNRS, Verimag - - PowerPoint PPT Presentation

▶
without regret
SMART_READER_LITE
LIVE PREVIEW

without Regret Barbara Jobstmann EPFL and Jasper DA CNRS, Verimag - - PowerPoint PPT Presentation

Aug 26, 2023 244 likes •550 views

Program Repair without Regret Barbara Jobstmann EPFL and Jasper DA CNRS, Verimag Joint Work with Christian von Essen UJF, Verimag Google Zurich 4th April 2014 B. Jobstmann 1 Agenda Introduction Motivation Program Repair,

slide-1
SLIDE 1

Program Repair without Regret

Barbara Jobstmann EPFL and Jasper DA CNRS, Verimag Joint Work with Christian von Essen UJF, Verimag Google Zurich

4th April 2014

  • B. Jobstmann

1

slide-2
SLIDE 2

Agenda

  • Introduction

– Motivation – Program Repair, related choices, our choices – Example

  • Our Repair approach

– Exact and relaxed repair problem – Reduction to classical synthesis – General and efficient algorithm – Implementation and results

  • Conclusion and future work

4th April 2014

  • B. Jobstmann

2

slide-3
SLIDE 3

Motivation

4th April 2014

  • B. Jobstmann

3

  • Debugging can be tedious

– Find the bug – Locate it – Fix it

slide-4
SLIDE 4

Motivation

4th April 2014

  • B. Jobstmann

4

  • Debugging can be tedious

– Find the bug: model checking – Locate it: automatically analyze/modify/explain CEX/witness – Fix it: automatically repair

slide-5
SLIDE 5

Automatic Repair

  • Given faulty program +

(explicit/implicit) specification

  • Search for modification s. t. modified program is

– “correct” and – “similar” to the original program

  • Key choices in an repair approach:

– Type of programs and specifications – Which modification do you allow? – How to you find and check corrections? – What do you mean by “similar”?

4th April 2014

  • B. Jobstmann

5

slide-6
SLIDE 6

Key choices

  • 1. Type of programs and specifications

– Data or control-oriented – Specific properties (e.g., deadlock), general properties (e.g., given in a logic), explicit/implicit

  • 2. Which modification do you allow?

– Syntactic modifications, e.g., based on expression language, genetic algorithms, …

  • 3. How to you find and check corrections?

– “Smart” enumeration and verification – “Synthesize” (combine search and verification)

  • 4. What do you mean by “similar”?

– Focus on syntactic similarity (e.g., edit distance, …)

4th April 2014

  • B. Jobstmann

6

slide-7
SLIDE 7

Our Choices [following CAV’05]

  • 1. Type of programs and specifications

– Reactive finite-state programs (Mealy machines) – General properties specified using LTL)

  • 2. Which modification do you allow?

– Theory: functions over state/input variables – Implementation: expression language

  • 3. How to you find and check corrections?

– Combine search and verification using game theory

  • 4. What do you mean by “similar”?

– Syntactic similarity (given by expression language) – Semantic similarity (NEW in this work)

4th April 2014

  • B. Jobstmann

7

slide-8
SLIDE 8

Choice 1: Programs and specifications

mainLight = Red; sideLight = Red; always @(posedge clock) begin case (mainLight) Red: if (mainSensor) mainLight = Yellow; Yellow: mainLight = Green; Green: mainLight = Red; endcase // case (mainLight) case (sideLight) Red: if (sideSensor) sideLight = Yellow; Yellow: sideLight = Green; Green: sideLight = Red; endcase // case (sideLight) end

4th April 2014 8

State variables: mainLight in {Red, Yellow, Green} sideLight in {Red, Yellow, Green} Input variables: mainSensor in {True, False} sideSensor in {True, False} Behavior represented as (infinite) sequence of evaluations of state and input variables:w ∈ 𝐹(𝑊)𝜕

Step 1 2 3 4 … mL Red Yellow Green Red … sL Red Red Red Yellow … mS True True False … … sS False False True … …

Program represented as set of behaviors: 𝑀 𝑄

slide-9
SLIDE 9

Choice 1: Programs and specifications

mainLight = Red; sideLight = Red; always @(posedge clock) begin case (mainLight) Red: if (mainSensor) mainLight = Yellow; Yellow: mainLight = Green; Green: mainLight = Red; endcase // case (mainLight) case (sideLight) Red: if (sideSensor) sideLight = Yellow; Yellow: sideLight = Green; Green: sideLight = Red; endcase // case (sideLight) end

4th April 2014 9

State variables: mainLight in {Red, Yellow, Green} sideLight in {Red, Yellow, Green} Input variables: mainSensor in {True, False} sideSensor in {True, False} Specification represented as set of behaviors: 𝑀(φ) Program represented as set of behaviors: 𝑀 𝑄 Specification: never(mainLight == Green and sideLight == Green) Behavior represented as (infinite) sequence of evaluations of state and input variables:w ∈ 𝐹(𝑊)𝜕

slide-10
SLIDE 10

Choice 2: Modifications

mainLight = Red; sideLight = Red; always @(posedge clock) begin case (mainLight) Red: if (mainSensor) mainLight = Yellow; Yellow: mainLight = Green; Green: mainLight = Red; endcase // case (mainLight) case (sideLight) Red: if (sideSensor) sideLight = Yellow; Yellow: sideLight = Green; Green: sideLight = Red; endcase // case (sideLight) end

4th April 2014 10

State variables: mainLight in {Red, Yellow, Green} sideLight in {Red, Yellow, Green} Input variables: mainSensor in {True, False} sideSensor in {True, False}

Specification: never(mainLight == Green and sideLight == Green)

??? Allowed modifications: function over state and input variables Specification represented as set of behaviors: 𝑀(φ) Program represented as set of behaviors: 𝑀 𝑄 Behavior represented as (infinite) sequence of evaluations of state and input variables:w ∈ 𝐹(𝑊)𝜕

slide-11
SLIDE 11

Choice 3: Repair Search using Games

mainLight = Red; sideLight = Red; always @(posedge clock) begin case (mainLight) Red: if (mainSensor) mainLight = Yellow; Yellow: mainLight = Green; Green: mainLight = Red; endcase // case (mainLight) case (sideLight) Red: if (sideSensor) sideLight = Yellow; Yellow: sideLight = Green; Green: sideLight = Red; endcase // case (sideLight) end

4th April 2014 11

State variables: mainLight in {Red, Yellow, Green} sideLight in {Red, Yellow, Green} Input variables: mainSensor in {True, False} sideSensor in {True, False}

Specification: never(mainLight == Green and sideLight == Green)

??? Allowed modifications: function over state and input variables Input variables: mainSensor in {True, False} sideSensor in {True, False} Winning objective: repaired program is correct, i.e., 𝑀 𝑄′ ⊆ 𝑀(φ) Specification represented as set of behaviors: 𝑀(φ) Program represented as set of behaviors: 𝑀 𝑄 Behavior represented as (infinite) sequence of evaluations of state and input variables:w ∈ 𝐹(𝑊)𝜕

slide-12
SLIDE 12

Choice 4: Similarity

mainLight = Red; sideLight = Red; always @(posedge clock) begin case (mainLight) Red: if (mainSensor) mainLight = Yellow; Yellow: mainLight = Green; Green: mainLight = Red; endcase // case (mainLight) case (sideLight) Red: if (sideSensor) sideLight = Yellow; Yellow: sideLight = Green; Green: sideLight = Red; endcase // case (sideLight) end

4th April 2014 12

State variables: mainLight in {Red, Yellow, Green} sideLight in {Red, Yellow, Green} Input variables: mainSensor in {True, False} sideSensor in {True, False} ??? Allowed modifications: “simple” function over state and input variables Input variables: mainSensor in {True, False} sideSensor in {True, False}

Specification: never(mainLight == Green and sideLight == Green)

Specification represented as set of behaviors: 𝑀(φ) Program represented as set of behaviors: 𝑀 𝑄 Winning objective: repaired program is correct, i.e., 𝑀 𝑄′ ⊆ 𝑀(φ) Behavior represented as (infinite) sequence of evaluations of state and input variables:w ∈ 𝐹(𝑊)𝜕

slide-13
SLIDE 13

Simple Repair

mainLight = Red; sideLight = Red; always @(posedge clock) begin case (mainLight) Red: if (mainSensor) mainLight = Yellow; Yellow: mainLight = Green; Green: mainLight = Red; endcase // case (mainLight) case (sideLight) Red: if (sideSensor) sideLight = Yellow; Yellow: sideLight = Green; Green: sideLight = Red; endcase // case (sideLight) end

4th April 2014

false

Specification: never(mainLight == Green and sideLight == Green)

 No car crash: correct repair  Main street blocked What went wrong? Lost intended behavior; changed behaviors unnecessarily Idea: semantic similarity

  • Keep correct behaviors
  • Modifications must not

effect correct behaviors

Extend objective: repair keeps correct behaviors 𝑀 𝑄 ∩ 𝑀 φ ⊆ 𝑀 𝑄′ (mainSensor & !(sideLight == Red & sideSensor)) Winning objective: repaired program is correct, i.e., 𝑀 𝑄′ ⊆ 𝑀(φ)

[Angelic debugging, Chandra et al.]

slide-14
SLIDE 14

Agenda

  • Introduction

– Motivation – Program Repair, related choices, our choices – Example

  • Our Repair approach

– Exact and relaxed repair problem – Reduction to classical synthesis – General and efficient algorithm – Implementation and results

  • Conclusion and future work

4th April 2014

  • B. Jobstmann

14

slide-15
SLIDE 15

Exact Repair Problem

Program 𝑄′ is an “exact repair” of program 𝑄 for specification φ if (i) all correct behaviors of 𝑄 w.r.t. φ are part of 𝑄′, (ii) all behaviors of 𝑄’ are correct w.r.t. φ, i.e., 𝑀 𝑄 ∩ 𝑀 φ ⊆ 𝑀 𝑄′ ⊆ 𝑀(φ)

4th April 2014

  • B. Jobstmann

15

P’ Ideal but sometimes too restrictive:

  • exact repair might not exists
  • exact repair might not be required

φ = φ𝑏 → φ𝑕

Behaviors that do not satisfy φ𝑏 are correct but might not need to be preserved φ P

slide-16
SLIDE 16

Relaxed Repair Problem

4th April 2014

  • B. Jobstmann

16

P’ φ P ψ

Program 𝑄′ is an “relaxed repair” of program 𝑄 for specifications φ and 𝜔 if (i) all correct behaviors of 𝑄 w.r.t. 𝜔 are part of 𝑄′, (ii) all behaviors of 𝑄’ are correct w.r.t. φ, i.e., 𝑀 𝑄 ∩ 𝑀 𝜔 ⊆ 𝑀 𝑄′ ⊆ 𝑀(φ)

slide-17
SLIDE 17

Some choices for 𝜔

  • Exact repair 𝜔 = φ
  • Assume-Guarantee:

if φ = φ𝑏 → φ𝑕, then 𝜔 = φ𝑏 ∧ φ𝑕

  • Classical repair 𝜔 = ∅

4th April 2014

  • B. Jobstmann

17

slide-18
SLIDE 18

Reduction to Classical Synthesis

Given two specifications φ and 𝜔 over variables 𝑊 = 𝐽 ∪ 𝑃 and two programs 𝑄 and 𝑄’, then 𝑄′is a relaxed repair of 𝑄 if and only if 𝑄′satisfies the following formula 𝑀 𝑄′ ⊆ 𝛽 with 𝛽 = 𝐹(𝑊)𝜕 ∖ 𝑀 𝑄 ∩ 𝑀 𝜔 ↓𝐽↑𝑊∪ 𝑀 𝑄 ∩ 𝑀(φ)

4th April 2014

  • B. Jobstmann

18

(All behaviors with a sequence of inputs for which P violates the spec)

There exists a relaxed repair for program 𝑄 w.r.t. φ and 𝜔 if and only if language 𝛽 is realizable.

slide-19
SLIDE 19

General Algorithm

  • 1. Construct an (Buchi) automaton 𝐵𝛽 for

𝛽 = 𝐹(𝑊)𝜕 ∖ 𝑀 𝑄 ∩ 𝑀 𝜔 ↓𝐽↑𝑊∪ 𝑀 𝑄 ∩ 𝑀(φ) 𝐵𝛽 can be constructed because

  • 𝑄, 𝜔, φ can be represented as Buchi automata and
  • Buchi automata are closed under intersection,

union, projection, and complementation

  • 2. Synthesize 𝑄’ using 𝐵𝛽 as specification

4th April 2014

  • B. Jobstmann

19

slide-20
SLIDE 20

Efficient Algorithm (1)

4th April 2014

  • B. Jobstmann

20

Lemma: Given program 𝑄 and LTL formula 𝜔 over variables 𝑊 = 𝐽 ∪ 𝑃, for all words 𝑥 ∈ 𝐹(𝑊)𝜕 𝑥 ∈ 𝑀 𝑄 ∩ 𝑀 𝜔 ↓𝐽↑𝑊⟺ 𝑄(𝑥 ↓𝐽) ∈ 𝑀(𝜔)

  • This enables a simple procedure to check if a word

produced by 𝑄′ lies in 𝑀 𝑄 ∩ 𝑀 𝜔 ↓𝐽↑𝑊

  • A synthesizer searching for 𝑄′ that satisfies α, can

simulate 𝑄 and check it against 𝜔 to decide if 𝑄′ is allowed to deviate.

slide-21
SLIDE 21

Efficient Algorithm (2)

4th April 2014

  • B. Jobstmann

21

P faulty program P* P extended with allowed modifications 𝐵𝜒 𝐵¬𝜔 𝐵𝑓𝑟 Inputs New Repair Approach: 𝐵¬𝜔 … checks when P’ can deviate from P 𝐵𝑓𝑟 … checks when the output

  • f P and P’ coincide

Classical Repair Approach: 𝐵𝜒 … defines objective of synthesis/repair game New Objective: 𝐵𝜒 and (𝐵¬𝜔 or 𝐵𝑓𝑟)

slide-22
SLIDE 22

Implementation

  • Ideas/choice:

– Modification restricted to expressions – Use MC instead of game engine:

  • encode “interesting” strategies using initial states (e.g., all

memoryless strategies)

  • an initial state that does not lead to a CEX gives a correct

repair.

  • Drawbacks: explodes with considered strategies
  • Benefits: can make use of any MC (and SEC optimizations)
  • Prototype based on NuSMV (with Cudd)

– Tiny modification to return initial state(s) without CEX

4th April 2014

  • B. Jobstmann

22

slide-23
SLIDE 23

PCI arbiter (partial specification)

  • PCI Bus
  • n Devices
  • n specifications

always(ri implies eventually gi) checked in isolation

  • Off by 1 error
  • Approach without lower bound gives access to

device i forever

  • Our approach fixes off by 1 error

4th April 2014

  • B. Jobstmann

23

slide-24
SLIDE 24

Processor (unclear error location)

  • Error in one of the ALUs
  • Partial specification
  • Multiple suspected error locations
  • Approach without lower bound approach may

modify all ALUs

  • Our approach only modifies faulty ALU

4th April 2014

  • B. Jobstmann

24

slide-25
SLIDE 25

Read-Write-Lock (minimal locking)

  • Faulty implementation leads to dead-lock
  • Allow introduction of a lot of locking
  • Approach without lower bound may lock

everything

  • Our approach introduces only necessary locks

4th April 2014

  • B. Jobstmann

25

slide-26
SLIDE 26

Preliminary Results

4th April 2014

  • B. Jobstmann

26

slide-27
SLIDE 27

Agenda

  • Introduction

– Motivation – Program Repair, related choices, our choices – Example

  • Our Repair approach

– Exact and relaxed repair problem – Reduction to classical synthesis – General and efficient algorithm – Implementation and results

  • Conclusion and future work

4th April 2014

  • B. Jobstmann

27

slide-28
SLIDE 28

Conclusions

  • New notion of program repair that

– ensures that correct behaviors are kept – supports fixing bugs incrementally – facilitates repair with incomplete specifications – avoids degenerated repairs

  • Algorithm based on reactive program

synthesis

  • Preliminary implementation supporting our

claims of obtaining “better” repairs

4th April 2014

  • B. Jobstmann

28

slide-29
SLIDE 29

Future Work

  • Quantitative notions of semantic similarity

– Count ratio of modified paths vs all paths – Count ratio of modified symbols – Define distance measure between symbols

  • Repairs with look-ahead to increase the

number of repairable systems

  • Repairing infinite-state system

4th April 2014

  • B. Jobstmann

29