to to Termination Analysis in in Lin inear Tim ime Roman - - PowerPoint PPT Presentation

Shape Analysis Termination Analysis Lin inear Tim ime

Roman Manevich Ben-Gurion University of the Negev Noam Rinetzky Tel Aviv University Boris Dogadov Tel Aviv University

From to to in in

2

UPGRADE YOUR SHAPE ANALYSIS

2

UPGRADE YOUR SHAPE ANALYSIS

START PROVING TOTAL CORRECTNESS TODAY

2

UPGRADE YOUR SHAPE ANALYSIS

START PROVING TOTAL CORRECTNESS TODAY

FOR JUST 5% OF THE RUNNING TIME

Research problem

• Automatically verify termination of heap-manipulating programs
• Dynamic memory allocation
• Destructive updates
• Applications
• Systems codes, e.g., Windows device drivers containing linked data structures:

lists, trees, etc.

• Object-oriented programs utilizing containers: sets, maps, and graphs

3

Scalability “dimensions”

4

Code size Shape complexity

(complexity of heap invariants)

PL features

recursion concurrency numeric data recursive containers hierarchical

• verlaid

Integer Program

[Berdine et al. CAV 2006] [Berdine et al. POPL 2007] [Magill et al. POPL 2010]

Classic approach

5

Instrumented Shape Analyzer Termination Analyzer

Heap-manipulating Program Safety property 

Always terminates May not terminate

Integer Program

[Berdine et al. CAV 2006] [Berdine et al. POPL 2007] [Magill et al. POPL 2010]

Classic approach

5

Instrumented Shape Analyzer Termination Analyzer

Heap-manipulating Program Safety property 

Always terminates May not terminate Shape analysis-specific

Integer Program

[Berdine et al. CAV 2006] [Berdine et al. POPL 2007] [Magill et al. POPL 2010]

Classic approach

5

Instrumented Shape Analyzer Termination Analyzer

Heap-manipulating Program Safety property 

Always terminates May not terminate Shape analysis-specific Termination undecidable

Integer Program

[Berdine et al. CAV 2006] [Berdine et al. POPL 2007] [Magill et al. POPL 2010]

Classic approach

5

Instrumented Shape Analyzer Termination Analyzer

Heap-manipulating Program Safety property 

Always terminates May not terminate Shape analysis-specific Termination undecidable Heavyweight

Integer Program

[Berdine et al. CAV 2006] [Berdine et al. POPL 2007] [Magill et al. POPL 2010]

Classic approach

5

Instrumented Shape Analyzer Termination Analyzer

Heap-manipulating Program Safety property 

Always terminates May not terminate Shape analysis-specific Termination undecidable Heavyweight

TRS

[AProVE Giesl et al. CAV 2012, RTA 2011, IJCAR 2014]

Integer Program

[Berdine et al. CAV 2006] [Berdine et al. POPL 2007] [Magill et al. POPL 2010]

Classic approach

5

Instrumented Shape Analyzer Termination Analyzer

Heap-manipulating Program Safety property 

Always terminates May not terminate Shape analysis-specific Termination undecidable Heavyweight

TRS

[AProVE Giesl et al. CAV 2012, RTA 2011, IJCAR 2014]

Logic Program

[Albert et al. FMOOBDS 2008] [Spoto et al. TOPLAS 2010]

Integer Program

[Berdine et al. CAV 2006] [Berdine et al. POPL 2007] [Magill et al. POPL 2010]

Classic approach

5

Instrumented Shape Analyzer Termination Analyzer

Heap-manipulating Program Safety property 

Always terminates May not terminate Shape analysis-specific Termination undecidable Heavyweight

Naughty idea: Once shape analysis does the heavy lifting termination is easy

TRS

[AProVE Giesl et al. CAV 2012, RTA 2011, IJCAR 2014]

Logic Program

[Albert et al. FMOOBDS 2008] [Spoto et al. TOPLAS 2010]

Our solution

6

Partition-based Shape Analyzer Termination Analyzer

Heap-manipulating Program Safety property 

Evolution relation Abstract transition relation

Always terminates May not terminate

Our solution

6

Partition-based Shape Analyzer Termination Analyzer

Heap-manipulating Program Safety property 

Evolution relation Abstract transition relation

Always terminates May not terminate Most shape analyses (Sep. Logic, TVLA, Boolean heaps, TRS)

Our solution

6

Partition-based Shape Analyzer Termination Analyzer

Heap-manipulating Program Safety property 

Evolution relation Abstract transition relation

Always terminates May not terminate Most shape analyses (Sep. Logic, TVLA, Boolean heaps, TRS) Easy to implement Induced by shape analysis

Our solution

6

Partition-based Shape Analyzer Termination Analyzer

Heap-manipulating Program Safety property 

Evolution relation Abstract transition relation

Always terminates May not terminate Most shape analyses (Sep. Logic, TVLA, Boolean heaps, TRS) Easy to implement Induced by shape analysis Termination checked in linear time

Main results

7

Main results

• 1. Termination analysis parametrized

by partition-based shape analysis

• Enables handling wide range of shape invariants

(both inductive data structures and unstructured graphs)

7

Main results

• 1. Termination analysis parametrized

by partition-based shape analysis

• Enables handling wide range of shape invariants

(both inductive data structures and unstructured graphs)

7

 Shape complexity

Main results

• 1. Termination analysis parametrized

by partition-based shape analysis

• Enables handling wide range of shape invariants

(both inductive data structures and unstructured graphs)

• Novel ranking function based on evolution relation

7

 Shape complexity

Main results

• 1. Termination analysis parametrized

by partition-based shape analysis

• Enables handling wide range of shape invariants

(both inductive data structures and unstructured graphs)

• Novel ranking function based on evolution relation
• 3. Featherweight analysis
• Linear time modulo shape analysis
• Modular

7

 Shape complexity

Main results

• 1. Termination analysis parametrized

by partition-based shape analysis

• Enables handling wide range of shape invariants

(both inductive data structures and unstructured graphs)

• Novel ranking function based on evolution relation
• 3. Featherweight analysis
• Linear time modulo shape analysis
• Modular

7

 Shape complexity  Code size

Main results

• 1. Termination analysis parametrized

by partition-based shape analysis

• Enables handling wide range of shape invariants

(both inductive data structures and unstructured graphs)

• Novel ranking function based on evolution relation
• 3. Featherweight analysis
• Linear time modulo shape analysis
• Modular
• 4. Handles recursion very precisely
• Limited support for concurrency

7

 Shape complexity  Code size

Main results

• 1. Termination analysis parametrized

by partition-based shape analysis

• Enables handling wide range of shape invariants

(both inductive data structures and unstructured graphs)

• Novel ranking function based on evolution relation
• 3. Featherweight analysis
• Linear time modulo shape analysis
• Modular
• 4. Handles recursion very precisely
• Limited support for concurrency

7

 Shape complexity  Code size  PL features

Main results

• 1. Termination analysis parametrized

by partition-based shape analysis

• Enables handling wide range of shape invariants

(both inductive data structures and unstructured graphs)

• Novel ranking function based on evolution relation
• 3. Featherweight analysis
• Linear time modulo shape analysis
• Modular
• 4. Handles recursion very precisely
• Limited support for concurrency
• 5. Precise enough on a variety of benchmarks

7

 Shape complexity  Code size  PL features

Agenda

• Our recipe for proving termination
• Apply recipe for shape analysis
• Handling nested loops and recursion
• Experiments and conclusion

8

Reminder: general recipe for proving termination

To show that a transition system (, ) does not contain infinite paths:

9

Reminder: general recipe for proving termination

To show that a transition system (, ) does not contain infinite paths:

• 1. Find well-founded ordering  :   

(all descending chains finite)

9

Reminder: general recipe for proving termination

To show that a transition system (, ) does not contain infinite paths:

• 1. Find well-founded ordering  :   

(all descending chains finite)

• 2. Show that every infinite path must contain an infinite -descending chain

9

1 i j k * * * *   

Our recipe

To show that a transition system (, ) does not contain infinite paths:

10

Our recipe

To show that a transition system (, ) does not contain infinite paths:

• 1. Find well-founded ordering  :   

that is monotone:   ’  ’  

10

Our recipe

To show that a transition system (, ) does not contain infinite paths:

• 1. Find well-founded ordering  :   

that is monotone:   ’  ’  

• 2. Compute a (finite) abstract transition system (, )

10

Our recipe

To show that a transition system (, ) does not contain infinite paths:

• 1. Find well-founded ordering  :   

that is monotone:   ’  ’  

• 2. Compute a (finite) abstract transition system (, )

10

• 3. Find all decreasing transitions

 := {} for each   ’   do if   ’ then  :=   (, ’) fi

• d // linear time

Our recipe

To show that a transition system (, ) does not contain infinite paths:

• 1. Find well-founded ordering  :   

that is monotone:   ’  ’  

• 2. Compute a (finite) abstract transition system (, )

10

• 3. Find all decreasing transitions

 := {} for each   ’   do if   ’ then  :=   (, ’) fi

• d // linear time
• 4. Check for cutting set

if  \  contains no cycles then answer true else answer “may not terminate” fi // linear time

Our recipe

To show that a transition system (, ) does not contain infinite paths:

• 1. Find well-founded ordering  :   

that is monotone:   ’  ’  

• 2. Compute a (finite) abstract transition system (, )

10

• 3. Find all decreasing transitions

 := {} for each   ’   do if   ’ then  :=   (, ’) fi

• d // linear time
• 4. Check for cutting set

if  \  contains no cycles then answer true else answer “may not terminate” fi // linear time

Agenda

• Our recipe for proving termination
• Apply recipe for shape analysis
• Handling nested loops and recursion
• Experiments and conclusion

11

Part rtition-based shape analysis: : symbolic heaps example

Concrete heaps abstracted into heap descriptors that partition the set of concrete objects into a finite number of heap regions

12

x y

nil

Part rtition-based shape analysis: : symbolic heaps example

Concrete heaps abstracted into heap descriptors that partition the set of concrete objects into a finite number of heap regions

12

x y

nil

ls(x, y) ls(y, nil) * 1 =

Abstract transition relation

• A shape analysis results in a finite abstract transition relation

13

x y

nil

ls(x, y) ls(y, nil) * 1 =

Abstract transition relation

• A shape analysis results in a finite abstract transition relation

13

x y

nil

ls(x, y) ls(y, nil) * y := y.n

x y

nil

ls(x, y) ls(y, nil) * 2 = 1 =

Concrete evolution relation

• A consistent renaming of object identifiers

14

x y

nil

x y

nil

y := y.n

Concrete evolution relation

• A consistent renaming of object identifiers

14

x y

nil

x y

nil

y := y.n

Abstract evolution relation

• May/must edges express how objects change membership in regions

15

x y

nil

ls(x, y) ls(y, nil) *

x y

nil

ls(x, y) ls(y, nil) * y := y.n 2 = 1 =

Abstract evolution relation

• May/must edges express how objects change membership in regions

15

x y

nil

ls(x, y) ls(y, nil) *

x y

nil

ls(x, y) ls(y, nil) * y := y.n 2 = 1 =

Must evolution edges help deplete regions

• May/must edges express how objects change membership in regions

16

x y

nil

ls(x, y) ls(y, nil) *

x y

nil

ls(x, y) ls(y, nil) * y := y.n 2 = 1 =

Must evolution edges help deplete regions

• May/must edges express how objects change membership in regions

16

x y

nil

ls(x, y) ls(y, nil) *

x y

nil

ls(x, y) ls(y, nil) * y := y.n 2 = 1 =

Region size decreases

Modeling dynamic allocation/deallocation

17

x

nil

ls(x, nil) free * yv * y := new 2 = 1 =

x

nil

ls(x, nil)

y

free *

… …

Modeling dynamic allocation/deallocation

17

x

nil

ls(x, nil) free * yv * y := new 2 = 1 =

x

nil

ls(x, nil)

y

free *

Infinite region

… …

Defi fining a monotone WF for shape analysis

• 1. Use evolution relation to define monotone WF

relation for objects

• 2. Use it to define monotone WF relation for heaps

18

Abstract evolution relation is a fi finite graph

• Nodes are region:descriptor pairs

19

ls(x, y) ls(y, nil) * ls(x, y) ls(y, nil) * y := y.n 2 = 1 = free * free *

Abstract evolution relation is a fi finite graph

• Nodes are region:descriptor pairs

19

ls(x, y) ls(y, nil) ls(x, y) ls(y, nil) 2 = 1 = free free

Abstract evolution relation to to WF over regions

• Nodes are region:descriptor pairs
• Lemma: collapsing strongly-connected component yields WF relation

(linear time)

20

1:ls(x, y) 1:ls(y, nil) 2:ls(x, y) 2:ls(y, nil) free free

Abstract evolution relation to to WF over regions

• Nodes are region:descriptor pairs
• Lemma: collapsing strongly-connected component yields WF relation

(linear time)

20

1:ls(x, y) 1:ls(y, nil) 2:ls(x, y) 2:ls(y, nil) free free

Ranking objects

• Merged nodes = region ranks
• Assign each object its respective region ranks

21

1:ls(x, y) 2:ls(x, y) free

Suffix rank Prefix rank

1:ls(y, nil) 2:ls(y, nil)

Condensation graph

x y

nil S S S S P P P

Before y:=y.n

• Merged nodes = region ranks
• Assign each object its respective region ranks
• Lemma: monotone WF

22

free

Suffix rank Prefix rank

x y

nil S S S S P P P

1:ls(y, nil) 2:ls(y, nil) 1:ls(x, y) 2:ls(x, y)

Aft fter y:=y.n

• Merged nodes = region ranks
• Assign each object its respective region ranks
• Lemma: monotone WF

23

free

Suffix rank Prefix rank

x y

nil S S S P P P P

1:ls(y, nil) 2:ls(y, nil) 1:ls(x, y) 2:ls(x, y)

Monotone WF relation for heaps

• Order heaps by comparing the ranks of their objects points-wise
• Lemma: monotone WF (have to be careful with memory allocation)

24

x y

nil S S S P P P P

x y

nil S S S S P P P

y := y.n = = = = = = 

Agenda

• Our recipe for proving termination
• Apply recipe for shape analysis
• Handling nested loops and recursion
• Experiments and conclusion

25

Assume control-flow graphs are structured

26

if xnil assume innil

• ut := nil

x := in return out y := out if ynil  y.nnil  y.d<x.d if y=out then // replace list head

• ut := new Node(out, x.d)

else // insert element after y y.n := new Node(y.n, x.d) x := x.n y := y.n

N1 N2 N3 N4 N5 N6 N7

FindPos InsertionSort

Handling nested loops

• Problem: proving termination for nested loops often requires

lexicographic ranking functions

• Solution: summarize loops
• Simply compose relations across statements: precise
• Linear time
• Prove termination from inner loop up to outer loop
• Limited to structured programs
• Improved in ongoing work [Boris Dogadov, M.Sc. thesis]

27

Termination for recursive procedures

• Model call stack as linked list [Rinetzky & Sagiv, CC 2001]

[Rival & Chang POPL 2011]

• Call modelled as allocation of stack object and insertion to call stack list
• Return modelled as removing from call stack
• For precision use predicates to capture shape patterns across calls
• Our algorithms can be applied without change
• Future work: extend to local interprocedural analysis

based on heap cutpoints [Rinetzky et al. POPL 2005, SAS 2005]

28

Agenda

• Our recipe for proving termination
• Apply recipe for shape analysis
• Handling nested loops and recursion
• Experiments and conclusion

29

Im Implementation

• Use TVLA as underlying

shape analysis

• No need to refine abstraction

(two exceptions)

• Overhead usually < 5%

30

Im Implementation

• Use TVLA as underlying

shape analysis

• No need to refine abstraction

(two exceptions)

• Overhead usually < 5%

30

Required (manual) abstraction refinement Required (manual) abstraction refinement

Conclusion

• Upgrade almost any shape analysis to automatically prove termination
• Simple to implement
• Featherweight reasoning
• Linear time in output of shape analysis
• Usually 5% of overall running time
• Naturally supports recursion
• Some support for concurrency
• Improving with ongoing work
• Practically quite precise

31

Conclusion

• Upgrade almost any shape analysis to automatically prove termination
• Simple to implement
• Featherweight reasoning
• Linear time in output of shape analysis
• Usually 5% of overall running time
• Naturally supports recursion
• Some support for concurrency
• Improving with ongoing work
• Practically quite precise

31

Thanks for lis listening Questions?

Sugg ggested questions

• How easy it is to extend a shape analysis with an abstract evolution

relation?

• Which programs defeat your analysis?
• How would you handle integer/string-valued fields?

How would you handle integer variables?

32

v, Q  H * ls(E, F) v, Q  H * (EF) Unroll1 v, Q  H * ls(E, F) v, Q  H * (Ez) * ls(z, F) where z is fresh Unroll>1 v, Q  H * (Ez) * ls(z, F) v, Q  H * ls(E, F) Fold v, Q  H * (yw) y := y.n v’, Q  H[y/w, z/y] * (zy) where z is fresh Next v, Q  H assume EF v’, Q  EF  H assume EF

In Inference rules ext xtended with evolution relation

33