Boosting Verification Scalability via Structural Grouping and - - PowerPoint PPT Presentation

SLIDE 1

Boosting Verification Scalability via Structural Grouping and Semantic Partitioning of Properties

Rohit Dureja*, Jason Baumgartner†, Alexander Ivrii†, Robert Kanzelman†, Kristin Y. Rozier*

* Iowa State University

† IBM Corporation

October 23, 2019

SLIDE 2

Model Checking

2 Motivation Cone-of-Influence Structural Semantic Summary

Usually multiple properties to be verified

SLIDE 3

Model Checking

3

Make multi-property verification scalable

Motivation Cone-of-Influence Structural Semantic Summary

Report

SLIDE 4

Multi-Property Verification

4

• Properties checked concurrently, or one-at-a-time
• Doesn’t optimally exploit sub-problem sharing

Motivation Cone-of-Influence Structural Semantic Summary

Report

Opportunity to save verification resources!

SLIDE 5

Improved Multi-Property Verification

5

• Group ‘high-affinity’ properties; similarity metric
• Properties in a group are concurrently solved; parallel groups
• Engine effort reused across properties in a group

Motivation Cone-of-Influence Structural Semantic Summary

What similarity metric to use?

Report

SLIDE 6

Similarity Measure

• Every property has distinct minimal cone-of-influence (COI)
• Multiple properties à exponential complexity w.r.t to collective COI
• Concurrent verification slower that one-at-a-time
• Nearly identical COI à save verification resource*
• Experimental demonstrated, offline-grouping

6

P3 P2 P1

Inputs

Design Structurally Similar

Motivation Cone-of-Influence Structural Semantic Summary 6

* G. Cabodi, P. E. Camurati, C. Loiacono, M. Palena, P. Pasini, D. Patti, and S. Quer, “To split or to group: from divide-and-conquer to sub- task sharing for verifying multiple properties in model checking,” International Journal on Software Tools for Technology Transfer (STTT), vol. 20, no. 3, pp. 313–325, Jun 2018

SLIDE 7

Our Contributions

• Online procedure to partition properties into high-affinity groups
• Near-linear runtime and automated; provable affinity bounds

7

Initial Grouping

P0 P1 P2 P3 P4 P5 P6 P7 P8 P9

Motivation Cone-of-Influence Structural Semantic Summary

SLIDE 8

Our Contributions

• Online procedure to partition properties into high-affinity groups
• Near-linear runtime and automated; provable affinity bounds
• Property grouping based on cone-of-influence
• Structural information (static)
• Structurally-similar properties may have different semantics
• Subset of design logic in cone-of-influence

8

Initial Grouping

P0 P1 P2 P3 P4 P5 P6 P7 P8 P9

Structural Affinity Grouping

P2 P4 P5 P1 P3 P7 P9 P0 P6 P8

Motivation Cone-of-Influence Structural Semantic Summary

SLIDE 9

Our Contributions

• Online procedure to partition properties into high-affinity groups
• Near-linear runtime and automated; provable affinity bounds
• Property grouping based on cone-of-influence
• Structural information (static)
• Structurally-similar properties may have different semantics
• Subset of design logic in cone-of-influence
• Property-group refinement using localization abstraction
• Semantic information (dynamic)

9

Initial Grouping

P0 P1 P2 P3 P4 P5 P6 P7 P8 P9

Structural Affinity Grouping

P2 P4 P5 P1 P3 P7 P9 P0 P6 P8

Semantic Affinity Partitioning

P2 P4 P5 P1 P3 P7 P9 P0 P6 P8

Motivation Cone-of-Influence Structural Semantic Summary

SLIDE 10

Cone-of-Influence Computation

10

Iterative

P3 P2 P1

Inputs

Design

Motivation Cone-of-Influence Structural Semantic Summary

SLIDE 11

Cone-of-Influence Computation

11

P3 P2 P1 P1

Inputs

Design

Iterative

Motivation Cone-of-Influence Structural Semantic Summary

SLIDE 12

Cone-of-Influence Computation

12

P3 P2 P1 P2

Inputs

Design

Iterative

Motivation Cone-of-Influence Structural Semantic Summary

SLIDE 13

Cone-of-Influence Computation

13

P3 P2 P1 P3

Inputs

Design

Iterative

Motivation Cone-of-Influence Structural Semantic Summary

SLIDE 14

Cone-of-Influence Computation

14

Iterative

• Repeated traversals
• Does not scale!

Our Method

P3 P2 P1

Inputs

Design P3 P2 P1 P3 P2 P1

Inputs

Design

Motivation Cone-of-Influence Structural Semantic Summary

SLIDE 15

Cone-of-Influence Computation

15

Iterative

• Repeated traversals
• Does not scale!

Our Method

P3 P2 P1 P1

Inputs

Design P3 P2 P1 P3 P2 P1

Inputs

Design

Motivation Cone-of-Influence Structural Semantic Summary

SLIDE 16

Cone-of-Influence Computation

16

Iterative

• Repeated traversals
• Does not scale!

Our Method

P3 P2 P1 P1

Inputs

Design P3 P2 P1 P3 P2 P1

Inputs

Design

Motivation Cone-of-Influence Structural Semantic Summary

SLIDE 17

Cone-of-Influence Computation

17

Iterative

• Repeated traversals
• Does not scale!

Our Method

P3 P2 P1 P2 P1

Inputs

Design P3 P2 P1 P3 P2 P1

Inputs

Design

Motivation Cone-of-Influence Structural Semantic Summary

SLIDE 18

P3 P2 P1 P2 P1

Inputs

Design

Cone-of-Influence Computation

18

Iterative

• Repeated traversals
• Does not scale!

Our Method

P3 P2 P1 P3 P2 P1

Inputs

Design

Motivation Cone-of-Influence Structural Semantic Summary

• One traversal
• Very scalable

SLIDE 19

COI Computation via Support Vectors

19

• Support variable – registers and inputs in COI
• Represent every support variable as a bit
• Bitvector operations to compute support (linear)

v0 v1 v2 v3 v4 v5 v6 v7 v8 v9

P0 P1

Support Vectors

1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 Motivation Cone-of-Influence Structural Semantic Summary 19

• G. Cabodi, P. Camurati, and S. Quer, “A graph-labeling approach for efficient cone-of-influence computation in model-checking problems with multiple

properties,” Software: Practice and Experience, vol. 46, no. 4, pp. 493–511, 2016.

SLIDE 20

COI Computation via Support Vectors

• Support variable – registers and inputs in COI
• Represent every support variable as a bit
• Bitvector operations to compute support (linear)

v0 v1 v2 v3 v4 v5 v6 v7 v8 v9

P0 P1

Support Vectors

1

1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 Motivation Cone-of-Influence Structural Semantic Summary 20 20

• G. Cabodi, P. Camurati, and S. Quer, “A graph-labeling approach for efficient cone-of-influence computation in model-checking problems with multiple

properties,” Software: Practice and Experience, vol. 46, no. 4, pp. 493–511, 2016.

SLIDE 21

COI Computation via Support Vectors

• Support variable – registers and inputs in COI
• Represent every support variable as a bit
• Bitvector operations to compute support (linear)
• Constant-time inspection

v0 v1 v2 v3 v4 v5 v6 v7 v8 v9

P0 P1

Support Vectors

1 1

1 2

1

3

1

4 5

1

6

1

7

1

8 9

1 1

1

1

2

1

3 4

1

5

1

6 7 8

1

9 Motivation Cone-of-Influence Structural Semantic Summary 21 21

• G. Cabodi, P. Camurati, and S. Quer, “A graph-labeling approach for efficient cone-of-influence computation in model-checking problems with multiple

properties,” Software: Practice and Experience, vol. 46, no. 4, pp. 493–511, 2016.

SLIDE 22

Support Vector Computation

• Several optimizations to improve time/memory
• Directed acyclic graph – SCCs à shorter bitvectors
• Garbage collection à peak memory requirement

Motivation Cone-of-Influence Structural Semantic Summary

0.01 0.1 1 10 100 1000 Iterative time (sec) 0.01 0.1 1 10 100 1000 Support vector time (sec)

Several orders of magnitude faster!

22 22

• G. Cabodi, P. Camurati, and S. Quer, “A graph-labeling approach for efficient cone-of-influence computation in model-checking problems with multiple

properties,” Software: Practice and Experience, vol. 46, no. 4, pp. 493–511, 2016.

SLIDE 23

Structural Grouping

23

• Properties with ‘similar’ support bitvectors above threshold t
• Classical clustering – very slow, at least O(n2)
• Three-level approximate clustering (near-linear runtime)

Initial Grouping

P0 P1 P2 P3 P4 P5 P6 P7 P8 P9

Motivation Cone-of-Influence Structural Semantic Summary

SLIDE 24

Structural Grouping

24

• Properties with ‘similar’ support bitvectors above threshold t
• Classical clustering – very slow, at least O(n2)
• Three-level approximate clustering (near-linear runtime)

Initial Grouping

P0 P1 P2 P3 P4 P5 P6 P7 P8 P9

Level-1 Grouping

(identical support) P0 P2 P1 P3 P4 P7 P9 P5 P6 P8

Motivation Cone-of-Influence Structural Semantic Summary

SLIDE 25

Structural Grouping

25

• Properties with ‘similar’ support bitvectors above threshold t
• Classical clustering – very slow, at least O(n2)
• Three-level approximate clustering (near-linear runtime)

Initial Grouping

P0 P1 P2 P3 P4 P5 P6 P7 P8 P9

Level-1 Grouping

(identical support) P0 P2 P1 P3 P4 P7 P9 P5 P6 P8

Level-2 Grouping

(SCC sharing) P0 P2 P1 P3 P4 P7 P9 P5 P8 P6

Motivation Cone-of-Influence Structural Semantic Summary

SLIDE 26

Level 2 – SCC Sharing

26 Motivation Cone-of-Influence Structural Semantic Summary

• Several designs contain large SCCs in cone-of-influence
• Every SCC has a weight – number of registers in SCC
• Group properties that share large SCCs – at least weight t

P1, P2

1

1

1

2

1

3

1

4 5 6 7

1

8

1

9 10

1

11 12

1

13

1

14 15 16 17

1

18

1

19

P3

1

1

1

2

1

3 4

1

5 6

1

7 8

1

9

1

10 11

1

12 13 14

1

15 16 17

1

18 19

P4

1

1

2 3 4

1

5

1

6 7

1

8 9 10

1

11 12

1

13

1

14

1

15 16

1

17 18 19

P5

1

1

2 3 4

1

5

1

6 7

1

8 9 10

1

11 12

1

13

1

14

1

15 16

1

17

1

18 19

“N” SCC bits

SLIDE 27

Structural Grouping

27

• Properties with ‘similar’ support bitvectors above threshold t
• Classical clustering – very slow, at least O(n2)
• Three-level approximate clustering (near-linear runtime)

Initial Grouping

P0 P1 P2 P3 P4 P5 P6 P7 P8 P9

Level-1 Grouping

(identical support) P0 P2 P1 P3 P4 P7 P9 P5 P6 P8

Level-2 Grouping

(SCC sharing) P0 P2 P1 P3 P4 P7 P9 P5 P8 P6

Motivation Cone-of-Influence Structural Semantic Summary

SLIDE 28

Structural Grouping

28

• Properties with ‘similar’ support bitvectors above threshold t
• Classical clustering – very slow, at least O(n2)
• Three-level approximate clustering (near-linear runtime)

Initial Grouping

P0 P1 P2 P3 P4 P5 P6 P7 P8 P9

Level-1 Grouping

(identical support) P0 P2 P1 P3 P4 P7 P9 P5 P6 P8

Level-2 Grouping

(SCC sharing) P0 P2 P1 P3 P4 P7 P9 P5 P8 P6

Level-3 Grouping

(hamming distance) P0 P2 P3 P1 P4 P7 P9 P6 P5 P8

Motivation Cone-of-Influence Structural Semantic Summary

SLIDE 29

Level 3 – Hamming Distance

29 Motivation Cone-of-Influence Structural Semantic Summary

• Exact Hamming distance calculation is slow, O(n2)
• Generate normalized support bitvectors
• Map generated offline or on-the-fly, < 1sec
• Group properties with identical mapped bitvectors

0101000101101101

V

0011110010111000 1100011000011111 1101110000111001

. . .

16-bits v0 = 20854 v1 = 15544 v2 = 50719 v3 = 56377 M(v0) M(v1) M(v2) M(v3) 391

M(V)

145 231 92

. . .

SLIDE 30

Structural Grouping

30

• Properties with ‘similar’ support bitvectors above threshold t
• Classical clustering – very slow, at least O(n2)
• Three-level approximate clustering (near-linear runtime)
• Proof: affinity >= 3*t - 2
• Properties in a group are checked concurrently; groups in parallel

Initial Grouping

P0 P1 P2 P3 P4 P5 P6 P7 P8 P9

Level-1 Grouping

(identical support) P0 P2 P1 P3 P4 P7 P9 P5 P6 P8

Level-2 Grouping

(SCC sharing) P0 P2 P1 P3 P4 P7 P9 P5 P8 P6

Level-3 Grouping

(hamming distance) P0 P2 P3 P1 P4 P7 P9 P6 P5 P8

Motivation Cone-of-Influence Structural Semantic Summary

SLIDE 31

Grouping Time

• Largest benchmarks (HWMCC)
• Simplified by logic synthesis; hard properties only
• 100 – 2,500 properties in a benchmark

31 Motivation Cone-of-Influence Structural Semantic Summary

SLIDE 32

Grouping Time

32 Motivation Cone-of-Influence Structural Semantic Summary

1 10 20 30 40 48 0.1 1 10

Level-1 Level-2 Level-3 Overall

Time (ms) Benchmarks

Grouping takes <10 ms

SLIDE 33

End-to-End Speedup

33 Motivation Cone-of-Influence Structural Semantic Summary

• Engine portfolio – BMC, IC3, and Localization (LOC)
• BMC and IC3 can process multiple properties
• Localization concurrently

SLIDE 34

End-to-End Speedup

Median 4.3X speedup

34

0.01 0.1 1 10 50 Multiple (hrs) 0.01 0.1 1 10 50 With Grouping (hrs)

Motivation Cone-of-Influence Structural Semantic Summary

SLIDE 35

Impact on Localization Abstraction

• Technique to remove irrelevant logic
• Iterative method, repeated cutpointing and refinement
• Concurrent localization of low-affinity properties
• Large localized designs, disjoint logic subsets, slow proofs
• Our procedure ensures high-affinity property localization
• Small localized designs, faster proofs

Motivation 35 Cone-of-Influence Structural Semantic Summary

SLIDE 36

Impact on Localization Abstraction

Motivation 36 Cone-of-Influence Structural Semantic Summary

0.01 0.1 1 10 50 Low Affinity (hrs) 0.01 0.1 1 10 50 High Affinity (hrs)

Median 2.5X speedup

• Compare with low-affinity groups – sort then partition
• First efficient multi-property localization solution!

SLIDE 37

Structural Grouping

Motivation 37 Cone-of-Influence Structural Semantic Summary

• Structurally-similar properties may have different semantics
• Subset of design logic in cone-of-influence

SLIDE 38

Structural Grouping

Motivation 38 Cone-of-Influence Structural Semantic Summary

• Structurally-similar properties may have different semantics
• Subset of design logic in cone-of-influence, mix of hittable/unhittable
• Learn semantic information via localization abstraction

SLIDE 39

Semantic Partitioning

Motivation 39 Cone-of-Influence Structural Semantic Summary

• Concurrently localize high-affinity property group

SLIDE 40

Semantic Partitioning

Motivation 40 Cone-of-Influence Structural Semantic Summary

• Concurrently localize high-affinity property group
• Repeated BMC steps to generate localized design

SLIDE 41

Semantic Partitioning

Motivation 41 Cone-of-Influence Structural Semantic Summary

• Concurrently localize high-affinity property group
• Repeated BMC steps to generate localized design

SLIDE 42

Semantic Partitioning

Motivation 42 Cone-of-Influence Structural Semantic Summary

• Concurrently localize high-affinity property group
• Repeated BMC steps to generate localized design

SLIDE 43

Semantic Partitioning

Motivation 43 Cone-of-Influence Structural Semantic Summary

• Concurrently localize high-affinity property group
• Repeated BMC steps to generate localized design
• Attempt partitioning after N consecutive steps with no refinement

SLIDE 44

Semantic Partitioning

Motivation 44 Cone-of-Influence Structural Semantic Summary

• Concurrently localize high-affinity property group
• Repeated BMC steps to generate localized design
• Attempt partitioning after N consecutive steps with no refinement
• Structural grouping procedure w.r.t localized design

SLIDE 45

Impact on Localization Abstraction

Motivation 45 Cone-of-Influence Structural Semantic Summary

0.1 1 10 130 Disabled (mins) 0.1 1 10 130 Enabled (mins)

• Selected benchmarks; some property groups solved by localization
• Single proof run; no spurious counterexamples

SLIDE 46

Summary

• Fast and online algorithm to group ”high-affinity” properties
• Three leveled grouping; identical, SCC sharing, and Hamming distance
• 4.3x speedup, minimal resource overhead
• Yields groups with provable affinity bounds; might err (tradeoff)
• First approach to optimize multi-property localization
• Ongoing and future work
• Sequential equivalence checking (SEC) – each equivalence point is a property
• Structural vs. semantic – hard to know without consuming verification resource

46 Motivation Cone-of-Influence Structural Semantic Summary

Initial Grouping

P0 P1 P2 P3 P4 P5 P6 P7 P8 P9

Structural Affinity Grouping

P2 P4 P5 P1 P3 P7 P9 P0 P6 P8

Semantic Affinity Partitioning

P2 P4 P5 P1 P3 P7 P9 P0 P6 P8

Thank you!

http://temporallogic.org/research/FMCAD19/