SpaceSearch: A Library for Building and Verifying Solver-Aided - - PowerPoint PPT Presentation

▶
spacesearch a library for building and verifying solver
SMART_READER_LITE
LIVE PREVIEW

SpaceSearch: A Library for Building and Verifying Solver-Aided - - PowerPoint PPT Presentation

Dec 27, 2023 170 likes •749 views

SpaceSearch: A Library for Building and Verifying Solver-Aided Tools Konstantin Steven S. Stefan Emina Michael Zachary Weitz Lyubomirsky Heule Torlak D. Ernst Tatlock Reduction SMT Reduction SMT Reduction SpaceSearch Reduction

slide-1
SLIDE 1

SpaceSearch: A Library for Building and Verifying Solver-Aided Tools

Konstantin Weitz Steven S. Lyubomirsky Michael

  • D. Ernst

Zachary Tatlock Emina Torlak Stefan Heule

slide-2
SLIDE 2
slide-3
SLIDE 3
slide-4
SLIDE 4

Reduction

slide-5
SLIDE 5

Reduction

SMT

slide-6
SLIDE 6

Reduction

SMT

slide-7
SLIDE 7

SpaceSearch

Reduction

SMT

slide-8
SLIDE 8

The Border Gateway Protocol

slide-9
SLIDE 9

ISP ISP

The Border Gateway Protocol

slide-10
SLIDE 10

ISP ISP ISP ISP

The Border Gateway Protocol

ISP

slide-11
SLIDE 11

ISP ISP ISP ISP

The Border Gateway Protocol

ISP

slide-12
SLIDE 12

ISP ISP ISP ISP

The Border Gateway Protocol

ISP

slide-13
SLIDE 13

ISP ISP ISP ISP

The Border Gateway Protocol

configure terminal router bgp 1 neighbor 8.212.226.20 remote-as 2 neighbor 172.16.254.1 remote-as 3 neighbor 95.112.23.51 remote-as 4 ! prefix lists ip prefix-list
  • utCustomer permit 0.0.0.0/0 le 32
ip prefix-list outPeer permit 8.212.226.0/24 le p prefix-list
  • utProvider permit
slide-14
SLIDE 14

ISP ISP ISP ISP

The Border Gateway Protocol

configure terminal router bgp 1 neighbor 8.212.226.20 remote-as 2 neighbor 172.16.254.1 remote-as 3 neighbor 95.112.23.51 remote-as 4 ! prefix lists ip prefix-list
  • utCustomer permit 0.0.0.0/0 le 32
ip prefix-list outPeer permit 8.212.226.0/24 le p prefix-list
  • utProvider permit
slide-15
SLIDE 15

ISP ISP ISP ISP

The Border Gateway Protocol

configure terminal router bgp 1 neighbor 8.212.226.20 remote-as 2 neighbor 172.16.254.1 remote-as 3 neighbor 95.112.23.51 remote-as 4 ! prefix lists ip prefix-list
  • utCustomer permit 0.0.0.0/0 le 32
ip prefix-list outPeer permit 8.212.226.0/24 le p prefix-list
  • utProvider permit
slide-16
SLIDE 16

ISP ISP ISP ISP

The Border Gateway Protocol

configure terminal router bgp 1 neighbor 8.212.226.20 remote-as 2 neighbor 172.16.254.1 remote-as 3 neighbor 95.112.23.51 remote-as 4 ! prefix lists ip prefix-list
  • utCustomer permit 0.0.0.0/0 le 32
ip prefix-list outPeer permit 8.212.226.0/24 le p prefix-list
  • utProvider permit
slide-17
SLIDE 17

ISP ISP ISP ISP

The Border Gateway Protocol

configure terminal router bgp 1 neighbor 8.212.226.20 remote-as 2 neighbor 172.16.254.1 remote-as 3 neighbor 95.112.23.51 remote-as 4 ! prefix lists ip prefix-list
  • utCustomer permit 0.0.0.0/0 le 32
ip prefix-list outPeer permit 8.212.226.0/24 le p prefix-list
  • utProvider permit
slide-18
SLIDE 18

ISP ISP ISP ISP

The Border Gateway Protocol

configure terminal router bgp 1 neighbor 8.212.226.20 remote-as 2 neighbor 172.16.254.1 remote-as 3 neighbor 95.112.23.51 remote-as 4 ! prefix lists ip prefix-list
  • utCustomer permit 0.0.0.0/0 le 32
ip prefix-list outPeer permit 8.212.226.0/24 le p prefix-list
  • utProvider permit
slide-19
SLIDE 19

ISP ISP ISP ISP

The Border Gateway Protocol

configure terminal router bgp 1 neighbor 8.212.226.20 remote-as 2 neighbor 172.16.254.1 remote-as 3 neighbor 95.112.23.51 remote-as 4 ! prefix lists ip prefix-list
  • utCustomer permit 0.0.0.0/0 le 32
ip prefix-list outPeer permit 8.212.226.0/24 le p prefix-list
  • utProvider permit
slide-20
SLIDE 20

Bagpipe

configure terminal router bgp 1 neighbor 8.212.226.20 remote-as 2 neighbor 172.16.254.1 remote-as 3 neighbor 95.112.23.51 remote-as 4 ! prefix lists ip prefix-list outC permit 0.0.0.0/0 le 32 ip prefix-list outPeer permit 8.212.226.0/24 le p prefix-list ou permit

Configuration

Bagpipe

Specification Spec Violation Spec Holds

slide-21
SLIDE 21

Bagpipe

configure terminal router bgp 1 neighbor 8.212.226.20 remote-as 2 neighbor 172.16.254.1 remote-as 3 neighbor 95.112.23.51 remote-as 4 ! prefix lists ip prefix-list outC permit 0.0.0.0/0 le 32 ip prefix-list outPeer permit 8.212.226.0/24 le p prefix-list ou permit

Configuration

Bagpipe

Specification Spec Violation Spec Holds

slide-22
SLIDE 22

∀t:trace( ), check( , t)

Bagpipe

configure terminal router bgp 1

Spec Violation Spec Holds Specification

configure terminal router bgp 1 neighbor 8.212.226.20 remote-as 2 neighbor 172.16.254.1 remote-as 3 neighbor 95.112.23.51 remote-as 4 ! prefix lists ip prefix-list outC permit 0.0.0.0/0 le 32 ip prefix-list outPeer permit 8.212.226.0/24 le p prefix-list ou permit

Configuration

slide-23
SLIDE 23

∀t:trace( ), check( , t)

Bagpipe

∞

configure terminal router bgp 1

Spec Violation Spec Holds Specification

configure terminal router bgp 1 neighbor 8.212.226.20 remote-as 2 neighbor 172.16.254.1 remote-as 3 neighbor 95.112.23.51 remote-as 4 ! prefix lists ip prefix-list outC permit 0.0.0.0/0 le 32 ip prefix-list outPeer permit 8.212.226.0/24 le p prefix-list ou permit

Configuration

slide-24
SLIDE 24

∀t:trace( ), check( , t)

Bagpipe

∞

fin

configure terminal router bgp 1

Spec Violation Spec Holds

∀t:initTrace( ), check( , t)

configure terminal router bgp 1

Reduce

Specification

configure terminal router bgp 1 neighbor 8.212.226.20 remote-as 2 neighbor 172.16.254.1 remote-as 3 neighbor 95.112.23.51 remote-as 4 ! prefix lists ip prefix-list outC permit 0.0.0.0/0 le 32 ip prefix-list outPeer permit 8.212.226.0/24 le p prefix-list ou permit

Configuration

[OOPSLA’16]

slide-25
SLIDE 25

∀t:trace( ), check( , t)

Bagpipe

∞

fin

configure terminal router bgp 1

Spec Violation Spec Holds

∀t:initTrace( ), check( , t)

configure terminal router bgp 1

Reduce

Specification

configure terminal router bgp 1 neighbor 8.212.226.20 remote-as 2 neighbor 172.16.254.1 remote-as 3 neighbor 95.112.23.51 remote-as 4 ! prefix lists ip prefix-list outC permit 0.0.0.0/0 le 32 ip prefix-list outPeer permit 8.212.226.0/24 le p prefix-list ou permit

Configuration

[OOPSLA’16]

slide-26
SLIDE 26

∀t:trace( ), check( , t)

Bagpipe

∞

fin

configure terminal router bgp 1

Spec Violation Spec Holds

Reduce ?

∀t:initTrace( ), check( , t)

configure terminal router bgp 1

Reduce

SMT

Specification

configure terminal router bgp 1 neighbor 8.212.226.20 remote-as 2 neighbor 172.16.254.1 remote-as 3 neighbor 95.112.23.51 remote-as 4 ! prefix lists ip prefix-list outC permit 0.0.0.0/0 le 32 ip prefix-list outPeer permit 8.212.226.0/24 le p prefix-list ou permit

Configuration

[OOPSLA’16]

slide-27
SLIDE 27

∀t:initTrace( ), check( , t)

configure terminal router bgp 1 neighbor 8.212.226.20
slide-28
SLIDE 28

initTrace( )

configure terminal router bgp 1

∀t:initTrace( ), check( , t)

configure terminal router bgp 1 neighbor 8.212.226.20
slide-29
SLIDE 29

initTrace( )

configure terminal router bgp 1

∀t:initTrace( ), check( , t)

configure terminal router bgp 1 neighbor 8.212.226.20

{ t:initTrace( ) | ¬check( , t) }

configure terminal router bgp 1
slide-30
SLIDE 30

initTrace( )

configure terminal router bgp 1

∀t:initTrace( ), check( , t)

configure terminal router bgp 1 neighbor 8.212.226.20

search( ) = None

{ t:initTrace( ) | ¬check( , t) }

configure terminal router bgp 1
slide-31
SLIDE 31

SpaceSearch Interface

a

empty = singleton(a) = union( , ) =

slide-32
SLIDE 32

SpaceSearch Interface

a

empty = singleton(a) = union( , ) =

a b

f(a) f(b)

a b

bind(S,f) = Ux:S f(x) = bind( , ) =

slide-33
SLIDE 33

SpaceSearch Interface

a

empty = singleton(a) = union( , ) =

a

search( ) = None search( ) = Some a

a b

f(a) f(b)

a b

bind(S,f) = Ux:S f(x) = bind( , ) =

slide-34
SLIDE 34

∀t:trace( ), check( , t) ∀t:initTrace( ), check( , t)

Bagpipe

∞

fin

Configuration Specification

configure terminal router bgp 1 neighbor 8.212.226.20 remote-as 2 neighbor 172.16.254.1 remote-as 3 neighbor 95.112.23.51 remote-as 4 ! prefix lists ip prefix-list outC permit 0.0.0.0/0 le 32 ip prefix-list outPeer permit 8.212.226.0/24 le p prefix-list ou permit configure terminal router bgp 1 configure terminal router bgp 1

Spec Violation Spec Holds

Reduce

slide-35
SLIDE 35

∀t:trace( ), check( , t) ∀t:initTrace( ), check( , t)

Bagpipe

∞

fin

Configuration Specification

configure terminal router bgp 1 neighbor 8.212.226.20 remote-as 2 neighbor 172.16.254.1 remote-as 3 neighbor 95.112.23.51 remote-as 4 ! prefix lists ip prefix-list outC permit 0.0.0.0/0 le 32 ip prefix-list outPeer permit 8.212.226.0/24 le p prefix-list ou permit configure terminal router bgp 1 configure terminal router bgp 1

Spec Violation Spec Holds

Reduce

search(
 bind(initTraceSpace( ), (λ t. if check( , t)
 then empty
 else singleton(t))) = None

configure terminal router bgp 1

Reduce

slide-36
SLIDE 36

∀t:trace( ), check( , t) ∀t:initTrace( ), check( , t)

Bagpipe

∞

fin

Configuration Specification

configure terminal router bgp 1 neighbor 8.212.226.20 remote-as 2 neighbor 172.16.254.1 remote-as 3 neighbor 95.112.23.51 remote-as 4 ! prefix lists ip prefix-list outC permit 0.0.0.0/0 le 32 ip prefix-list outPeer permit 8.212.226.0/24 le p prefix-list ou permit configure terminal router bgp 1 configure terminal router bgp 1

Spec Violation Spec Holds

Reduce

search(
 bind(initTraceSpace( ), (λ t. if check( , t)
 then empty
 else singleton(t))) = None

configure terminal router bgp 1

Reduce Extract ?

SMT

slide-37
SLIDE 37

Meet

slide-38
SLIDE 38

Meet

∀ x y. (x /\ y) ⟺ ¬(¬x \/ ¬y) De Morgan's Law

slide-39
SLIDE 39

(let ((x (symbolic-bool)) (y (symbolic-bool)))

Meet

∀ x y. (x /\ y) ⟺ ¬(¬x \/ ¬y) De Morgan's Law

slide-40
SLIDE 40

(let ((x (symbolic-bool)) (y (symbolic-bool)))

Meet

∀ x y. (x /\ y) ⟺ ¬(¬x \/ ¬y)

(eq? (and x y) (not (or (not x) (not y))))

De Morgan's Law

slide-41
SLIDE 41

(let ((x (symbolic-bool)) (y (symbolic-bool)))

Meet

∀ x y. (x /\ y) ⟺ ¬(¬x \/ ¬y)

(eq? (and x y) (not (or (not x) (not y)))) (solve (if (assert false) ‘counter-example))

De Morgan's Law

slide-42
SLIDE 42

(let ((x (symbolic-bool)) (y (symbolic-bool))) (declare-const x Bool) (declare-const y Bool) (define-const a Bool (and x y)) (define-const b Bool (not (or (not x) (not y)))) (assert (not (and (=> a b) (=> b a)))) (check-sat)

type-driven state merging

Meet

∀ x y. (x /\ y) ⟺ ¬(¬x \/ ¬y)

(eq? (and x y) (not (or (not x) (not y)))) (solve (if (assert false) ‘counter-example))

De Morgan's Law

slide-43
SLIDE 43

=> union(s,t) => (lambda (v) (if (symbolic-bool) (s v) (t v))) empty => (lambda (v) (assert false)) search(s) => (solve s) single(a) => (lambda (v) a) bind(s,f) => (lambda (v) (f (s v) v))

SpaceSearch Extraction

slide-44
SLIDE 44

∀t:trace( ), check( , t) ∀t:initTrace( ), check( , t)

Bagpipe

∞

fin

Configuration Specification

configure terminal router bgp 1 neighbor 8.212.226.20 remote-as 2 neighbor 172.16.254.1 remote-as 3 neighbor 95.112.23.51 remote-as 4 ! prefix lists ip prefix-list outC permit 0.0.0.0/0 le 32 ip prefix-list outPeer permit 8.212.226.0/24 le p prefix-list ou permit configure terminal router bgp 1 configure terminal router bgp 1

Spec Violation Spec Holds

Reduce

search(
 bind(initTraceSpace( ), (λ t. if check( , t)
 then empty
 else singleton(t))) = None

configure terminal router bgp 1

Reduce

slide-45
SLIDE 45

∀t:trace( ), check( , t) ∀t:initTrace( ), check( , t)

Bagpipe

∞

fin

Configuration Specification

configure terminal router bgp 1 neighbor 8.212.226.20 remote-as 2 neighbor 172.16.254.1 remote-as 3 neighbor 95.112.23.51 remote-as 4 ! prefix lists ip prefix-list outC permit 0.0.0.0/0 le 32 ip prefix-list outPeer permit 8.212.226.0/24 le p prefix-list ou permit configure terminal router bgp 1 configure terminal router bgp 1

Spec Violation Spec Holds

Reduce

search(
 bind(initTraceSpace( ), (λ t. if check( , t)
 then empty
 else singleton(t))) = None

configure terminal router bgp 1

Reduce Extract

/SMT

slide-46
SLIDE 46

Summary

SpaceSearch

  • Interface & Semantics
  • Extraction
slide-47
SLIDE 47

Summary

SpaceSearch

  • Interface & Semantics
  • Extraction

More in the paper:

  • Infinite Search Spaces
  • Other Backends
  • Parallelization
  • Incrementalization
slide-48
SLIDE 48

Evaluation

  • 1. BGP Verification

Bagpipe

SpaceSearch

  • Interface & Semantics
  • Extraction
slide-49
SLIDE 49

Evaluation

  • 2. SQL Rewrite
  • 1. BGP Verification

Bagpipe

SpaceSearch

  • Interface & Semantics
  • Extraction
slide-50
SLIDE 50

Evaluation

  • 2. SQL Rewrite
  • 3. x86 Semantics
  • 1. BGP Verification

Bagpipe

SpaceSearch

  • Interface & Semantics
  • Extraction
slide-51
SLIDE 51

Bagpipe

configure terminal router bgp 1 neighbor 8.212.226.20 remote-as 2 neighbor 172.16.254.1 remote-as 3 neighbor 95.112.23.51 remote-as 4 ! prefix lists ip prefix-list outC permit 0.0.0.0/0 le 32 ip prefix-list outPeer permit 8.212.226.0/24 le p prefix-list ou permit
  • 1. BGP Verification

Specification Configuration Counter Example Correct

slide-52
SLIDE 52

Bagpipe

configure terminal router bgp 1 neighbor 8.212.226.20 remote-as 2 neighbor 172.16.254.1 remote-as 3 neighbor 95.112.23.51 remote-as 4 ! prefix lists ip prefix-list outC permit 0.0.0.0/0 le 32 ip prefix-list outPeer permit 8.212.226.0/24 le p prefix-list ou permit
  • 10 Juniper Scenarios
  • No Martian
  • Block To External
  • Gao & Rexford
  • Internet2 >100K
  • BelWü >200K
  • Selfnet >50

19 Spec Violations, No False Positives 14 Specs Hold

  • 1. BGP Verification
slide-53
SLIDE 53

Counter Example Equal

search(
 counterexamples( , ))

  • 2. SQL Rewrite Verification

Query A Query B

=

E x t r a c t Reduce

SMT

slide-54
SLIDE 54

Counter Example Equal

search(
 bind(int32, (λ x. bind(int32, (λ y. if stoke(ADD x, y) =
 rocksalt(ADD x, y)
 then empty
 else singleton(x,y)))))

Instruction

  • 3. x86 Semantics Validation

ADD x, y

E x t r a c t

SMT

slide-55
SLIDE 55

Counter Example Equal

search(
 bind(int32, (λ x. bind(int32, (λ y. if stoke(ADD x, y) =
 rocksalt(ADD x, y)
 then empty
 else singleton(x,y)))))

Instruction

  • 3. x86 Semantics Validation

ADD x, y

E x t r a c t

Bugs found:

  • 7 Rocksalt Bugs
  • 1 Stoke Bug

SMT

slide-56
SLIDE 56

Related Work

  • Solver Aided Languages:

Rosette Torlak et al. PLDI’14 Smten Uhler et al. CAV’13

  • Solver Aided Tool Verification:

XCert Tatlock et al. PLDI’10

  • Verified SAT Solvers & SAT Tactics:

Marić TCS’10 Oe et al. VMCAI’12

slide-57
SLIDE 57

Thank You

Konstantin Weitz Steven S. Lyubomirsky Michael

  • D. Ernst

Zachary Tatlock Emina Torlak Stefan Heule

SpaceSearch

  • 2. SQL Rewrite
  • 3. x86 Semantics
  • 1. BGP Verification

Bagpipe

  • Interface & Semantics
  • Extraction

github.com/konne88/SpaceSearch