CAQE: A Certifying QBF Solver FMCAD Austin, Texas, September 29 - - PowerPoint PPT Presentation

▶
caqe a certifying qbf solver
SMART_READER_LITE
LIVE PREVIEW

CAQE: A Certifying QBF Solver FMCAD Austin, Texas, September 29 - - PowerPoint PPT Presentation

Jan 06, 2023 592 likes •981 views

CAQE: A Certifying QBF Solver FMCAD Austin, Texas, September 29 2015 1 / 15 Markus N. Rabe 1 Leander Tentrup 2 1 University of California at Berkeley, 2 Saarland University Quantified boolean formulas 2 / 15 TrueQBF is the prototypical

slide-1
SLIDE 1

CAQE: A Certifying QBF Solver

Markus N. Rabe1 Leander Tentrup2

1University of California at Berkeley, 2Saarland University

FMCAD Austin, Texas, September 29 2015

1 / 15

slide-2
SLIDE 2

Quantified boolean formulas

▶ TrueQBF is the prototypical PSPACE problem ▶ Compact version of SAT ▶ Verification/synthesis/artificial intelligence

2 / 15

slide-3
SLIDE 3

Contribution - A QBF Algorithm

▶ Simple and CEGAR-based (∼3K loc w/o SAT solver) ▶ Competitive performance ▶ Produces certificates ▶ Handles deep quantifier alternations

3 / 15

slide-4
SLIDE 4

QBF - Example ∃x∀y∃z : (x ∨ y ∨ z) ∧ (x ∨ y ∨ z) ∧ (x ∨ y ∨ z)

Choose x true y z y z y z Case y true z z Case y false z z

This formula is true!

4 / 15

slide-5
SLIDE 5

QBF - Example ∃x∀y∃z : (x ∨ y ∨ z) ∧ (x ∨ y ∨ z) ∧ (x ∨ y ∨ z)

Choose x = true : ∀y ∃z : (y ∨ z) ∧ (y ∨ z) Case y true z z Case y false z z

This formula is true!

4 / 15

slide-6
SLIDE 6

QBF - Example ∃x∀y∃z : (x ∨ y ∨ z) ∧ (x ∨ y ∨ z) ∧ (x ∨ y ∨ z)

Choose x = true : ∀y ∃z : (y ∨ z) ∧ (y ∨ z) Case y = true : ∃z : z Case y = false : ∃z : z

This formula is true!

4 / 15

slide-7
SLIDE 7

QBF - Example ∃x∀y∃z : (x ∨ y ∨ z) ∧ (x ∨ y ∨ z) ∧ (x ∨ y ∨ z)

Choose x = true : ∀y ∃z : (y ∨ z) ∧ (y ∨ z) Case y = true : ∃z : z Case y = false : ∃z : z

This formula is true!

4 / 15

slide-8
SLIDE 8

Clausal abstractions

Construct one SAT solver per quantifier level.

∃x∀y∃z : (x ∨ b t y b t ∨ z) (x ∨ b t y b t ∨ z) (x ∨ b t y b t ∨ z)

x y z

5 / 15

slide-9
SLIDE 9

Clausal abstractions

Construct one SAT solver per quantifier level.

∃x∀y∃z : (x ∨ b t y b t ∨ z) (x ∨ b t y b t ∨ z) (x ∨ b t y b t ∨ z)

x y z

5 / 15

slide-10
SLIDE 10

Clausal abstractions

Construct one SAT solver per quantifier level.

∃x∀y∃z : (x ∨ b1) (t1 → (y → b1)) (t1 ∨ z) (x ∨ b t y b t ∨ z) (x ∨ b t y b t ∨ z)

x y z

5 / 15

slide-11
SLIDE 11

Clausal abstractions

Construct one SAT solver per quantifier level.

∃x∀y∃z : (x ∨ b1) (t1 → (y → b1)) (t1 ∨ z) (x ∨ b2) (t2 → (y → b2)) (t2 ∨ z) (x ∨ b3) (t3 → (y → b3)) (t3 ∨ z)

x y z

5 / 15

slide-12
SLIDE 12

Clausal abstractions

Construct one SAT solver per quantifier level.

∃x∀y∃z : (x ∨ b1) (t1 ∨ y) (t1∨ z) (x ∨ b2) (t2 ∨ y) (t2∨ z) (x ∨ b3) (t3 ∨ y) (t3∨ z)

x y z

5 / 15

slide-13
SLIDE 13

Clausal abstractions

Construct one SAT solver per quantifier level.

∃x∀y∃z : (x ∨ b1) (t1 ∨ y) (t1∨ z) (x ∨ b2) (t2 ∨ y) (t2∨ z) (x ∨ b3) (t3 ∨ y) (t3∨ z) ϕ∃x ϕ∀y ϕ∃z

5 / 15

slide-14
SLIDE 14

Clausal abstractions - general case

Given Q1X1 . . . QnXn : ∧ Ci ϕ∃Xm = ∧

Ci

(( ∨

l∈Ci,level(l)=m l

) ∨ ti ∨ bi ) ϕ∀Xm = ∧

Ci

( ∧

l∈Ci,level(l)=m(l ∨ ti)

) Let t be a assignment to the variables ti. Represents the clauses that have been satisfied already. Two algorithms: solve Xm QnXn t solve Xm QnXn t Return value: (result, minimized assumptions, unsat core over assumptions)

6 / 15

slide-15
SLIDE 15

Clausal abstractions - general case

Given Q1X1 . . . QnXn : ∧ Ci ϕ∃Xm = ∧

Ci

(( ∨

l∈Ci,level(l)=m l

) ∨ ti ∨ bi ) ϕ∀Xm = ∧

Ci

( ∧

l∈Ci,level(l)=m(l ∨ ti)

) Let t be a assignment to the variables ti. Represents the clauses that have been satisfied already. Two algorithms:

▶ solve∃(∃Xm . . . QnXn : ψ, t) ▶ solve∀(∀Xm . . . QnXn : ψ, t)

Return value: (result, minimized assumptions, unsat core over assumptions)

6 / 15

slide-16
SLIDE 16

Algorithm

1: procedure solve∃(∃X. Ψ, t) 2:

while true do

3:

result, b, failed ← sat(ϕX, t)

4:

if result = UNSAT then

5:

return UNSAT, _, failed

6:

else if Ψ is propositional then

7:

return SAT, t, _

8:

tb ← {ti | bi / ∈ b, 1 ≤ i ≤ k}

9:

result, t′, failed′ ← solve∀(Ψ, t ∪ tb)

10:

if result = UNSAT then

11:

ϕX ← ϕX ∧ (∨

t∈failed′ ¬bt)

12:

else

13:

return SAT, t′, _

7 / 15

slide-17
SLIDE 17

Algorithm (2)

1: procedure solve∀(∀X. Ψ, t) 2:

while true do

3:

result, t′, failed ← sat(ϕX, t+)

4:

if result = UNSAT then

5:

return SAT, failed, _

6:

result, t′′, failed′ ← solve∃(Ψ, t′)

7:

if result = SAT then

8:

ϕX ← ϕX ∧ (∨

t∈t′′ ¬t)

9:

else

10:

return UNSAT, _, failed′

8 / 15

slide-18
SLIDE 18

Example (2)

(x ∨ b1) (t1 ∨ y) (t1∨ z) (x ∨ b2) (t2 ∨ y) (t2∨ z) (x ∨ b3) (t3 ∨ y) (t3∨ z) ϕ∃x ϕ∀y t t ϕ∃z Variable assignments Interface variable assignments Interface variable assumptions

9 / 15

slide-19
SLIDE 19

Example (2)

(x ∨ b1) (t1 ∨ y) (t1∨ z) (x ∨ b2) (t2 ∨ y) (t2∨ z) (x ∨ b3) (t3 ∨ y) (t3∨ z) ϕ∃x ϕ∀y t t ϕ∃z Variable assignments Interface variable assignments Interface variable assumptions

9 / 15

slide-20
SLIDE 20

Example (2)

(x ∨ b1) (t1 ∨ y) (t1∨ z) (x ∨ b2) (t2 ∨ y) (t2∨ z) (x ∨ b3) (t3 ∨ y) (t3∨ z) ϕ∃x ϕ∀y t t ϕ∃z Variable assignments Interface variable assignments Interface variable assumptions

9 / 15

slide-21
SLIDE 21

Example (2)

(x ∨ b1) (t1 ∨ y) (t1∨ z) (x ∨ b2) (t2 ∨ y) (t2∨ z) (x ∨ b3) (t3 ∨ y) (t3∨ z) ϕ∃x ϕ∀y t t ϕ∃z Variable assignments Interface variable assignments Interface variable assumptions

9 / 15

slide-22
SLIDE 22

Example (2)

(x ∨ b1) (t1 ∨ y) (t1∨ z) (x ∨ b2) (t2 ∨ y) (t2∨ z) (x ∨ b3) (t3 ∨ y) (t3∨ z) ϕ∃x ϕ∀y t t ϕ∃z Variable assignments Interface variable assignments Interface variable assumptions

9 / 15

slide-23
SLIDE 23

Example (2)

(x ∨ b1) (t1 ∨ y) (t1∨ z) (x ∨ b2) (t2 ∨ y) (t2∨ z) (x ∨ b3) (t3 ∨ y) (t3∨ z) ϕ∃x ϕ∀y t t ϕ∃z Variable assignments Interface variable assignments Interface variable assumptions

9 / 15

slide-24
SLIDE 24

Example (2)

(x ∨ b1) (t1 ∨ y) (t1∨ z) (x ∨ b2) (t2 ∨ y) (t2∨ z) (x ∨ b3) (t3 ∨ y) (t3∨ z) ϕ∃x ϕ∀y t t ϕ∃z Variable assignments Interface variable assignments Interface variable assumptions

9 / 15

slide-25
SLIDE 25

Example (2)

(x ∨ b1) (t1 ∨ y) (t1∨ z) (x ∨ b2) (t2 ∨ y) (t2∨ z) (x ∨ b3) (t3 ∨ y) (t3∨ z) ϕ∃x ϕ∀y ∧ t2 t ϕ∃z refine! Variable assignments Interface variable assignments Interface variable assumptions

9 / 15

slide-26
SLIDE 26

Example (2)

(x ∨ b1) (t1 ∨ y) (t1∨ z) (x ∨ b2) (t2 ∨ y) (t2∨ z) (x ∨ b3) (t3 ∨ y) (t3∨ z) ϕ∃x ϕ∀y ∧ t2 t ϕ∃z Variable assignments Interface variable assignments Interface variable assumptions

9 / 15

slide-27
SLIDE 27

Example (2)

(x ∨ b1) (t1 ∨ y) (t1∨ z) (x ∨ b2) (t2 ∨ y) (t2∨ z) (x ∨ b3) (t3 ∨ y) (t3∨ z) ϕ∃x ϕ∀y ∧ t2 t ϕ∃z Variable assignments Interface variable assignments Interface variable assumptions

9 / 15

slide-28
SLIDE 28

Example (2)

(x ∨ b1) (t1 ∨ y) (t1∨ z) (x ∨ b2) (t2 ∨ y) (t2∨ z) (x ∨ b3) (t3 ∨ y) (t3∨ z) ϕ∃x ϕ∀y ∧ t2 t ϕ∃z Variable assignments Interface variable assignments Interface variable assumptions

9 / 15

slide-29
SLIDE 29

Example (2)

(x ∨ b1) (t1 ∨ y) (t1∨ z) (x ∨ b2) (t2 ∨ y) (t2∨ z) (x ∨ b3) (t3 ∨ y) (t3∨ z) ϕ∃x ϕ∀y ∧ t2 ∧ t3 ϕ∃z refine! Variable assignments Interface variable assignments Interface variable assumptions

9 / 15

slide-30
SLIDE 30

Certification

1

CAQE

2

cap2aig

3

check_strategy

4

p cnf 3 3 e 1 a 2 e 3 1 2 -3 0

  • 1 2 -3 0
  • 1 -2 3 0

p cap 3 3 d d 6 -3 u SAT d 4 5 3 u SAT u SAT 1 u SAT r SAT

skolem.aig

x SAT SAT t x SAT t t x SAT

u d u d u d u

10 / 15

slide-31
SLIDE 31

Certification

1

CAQE

2

cap2aig

3

check_strategy

4

p cnf 3 3 e 1 a 2 e 3 1 2 -3 0

  • 1 2 -3 0
  • 1 -2 3 0

p cap 3 3 d d 6 -3 u SAT d 4 5 3 u SAT u SAT 1 u SAT r SAT

skolem.aig

x SAT SAT t x SAT t t x SAT

u d u d u d u

10 / 15

slide-32
SLIDE 32

Certification

1

CAQE

2

cap2aig

3

check_strategy

4

p cnf 3 3 e 1 a 2 e 3 1 2 -3 0

  • 1 2 -3 0
  • 1 -2 3 0

p cap 3 3 d d 6 -3 u SAT d 4 5 3 u SAT u SAT 1 u SAT r SAT

skolem.aig

⟨∅, {x1}, SAT⟩ ⟨∅, ∅, SAT⟩ ⟨{t3}, {x3}, SAT⟩ ⟨{t1, t2}, {x3}, SAT⟩

↑ u ↓ d ↑ u ↙ d ↗ u ↘ d ↖ u

10 / 15

slide-33
SLIDE 33

Certification

✓

1

CAQE

2

cap2aig

3

check_strategy

4

p cnf 3 3 e 1 a 2 e 3 1 2 -3 0

  • 1 2 -3 0
  • 1 -2 3 0

p cap 3 3 d d 6 -3 u SAT d 4 5 3 u SAT u SAT 1 u SAT r SAT

skolem.aig

⟨∅, {x1}, SAT⟩ ⟨∅, ∅, SAT⟩ ⟨{t3}, {x3}, SAT⟩ ⟨{t1, t2}, {x3}, SAT⟩

↑ u ↓ d ↑ u ↙ d ↗ u ↘ d ↖ u

10 / 15

slide-34
SLIDE 34

Experimental Evaluation

Implementation

▶ CAQE (Clausal Abstraction for Quantifier Elimination) ▶ ∼3K loc w/o SAT solver ▶ https://www.react.uni-saarland.de/tools/caqe/

Evaluation

▶ Compared against state-of-the-art QBF solvers DepQBF,

RAReQS, GhostQ

▶ Benchmark: QBFGallery2014 ▶ With/without preprocessing ▶ PicoSAT/MiniSAT

11 / 15

slide-35
SLIDE 35

Performance - with preprocessing

Number of instances solved within 10 minutes.

CAQE RAReQS GhostQ DepQBF Family total picosat+bloqqer minisat+bloqqer rareqs+bloqqer ghostq∗ depqbf+bloqqer eval2012r2 276 112 98 129 124 128 bomb 132 74 59 82 75 80 complexity 104 67 67 91 26 57 dungeon 107 31 69 62 45 66 hardness 114 103 94 68 57 81 planning 147 79 55 135 31 47 testing 131 77 84 92 102 76 all 1011 543 526 659 460 535

▶ Second-best performance

12 / 15

slide-36
SLIDE 36

Performance - without preprocessing

Number of instances solved within 10 minutes.

CAQE RAReQS GhostQ DepQBF Family total picosat minisat rareqs ghostq depqbf eval2012r2 276 75 55 81 124 88 bomb 132 91 75 84 75 67 complexity 104 50 60 75 26 49 dungeon 107 46 22 57 45 44 hardness 114 78 58 15 57 8 planning 147 84 50 146 31 57 testing 131 54 25 36 102 57 all 1011 478 345 494 460 370

▶ Competitive performance

13 / 15

slide-37
SLIDE 37

Performance - certification

Number of instances solved within 10 minutes and certified within another 10 minutes.

Solver # solved # verified # unique CAQE 428 340 146 DepQBF 312 239 44 virtual best 468 384

  • ▶ Significant improvement in certification performance.

14 / 15

slide-38
SLIDE 38

Conclusions

Contributions

▶ New CEGAR algorithm1 ▶ Competitive performance ▶ Best certification performance

Questions

▶ Quantification as a theory in SMT solvers?

1Similar: Janota, Marques-Silva, “Solving QBF by Clause Selection”, IJCAI’15 15 / 15