On the Sparsity of XORs in Approximate Model Counting Durgesh - - PowerPoint PPT Presentation

On the Sparsity of XORs in Approximate Model Counting

Durgesh Agrawal1 Bhavishya1 Kuldeep S. Meel2

1Indian Institute of Technology, Kanpur 2School of Computing, National University of Singapore

SAT 2020

1/18

Model Counting

• Given

– Boolean variables X1, X2, · · · Xn – Formula F over X1, X2, · · · Xn

• Sol(F) = { solutions of F }

2/18

Model Counting

• Given

– Boolean variables X1, X2, · · · Xn – Formula F over X1, X2, · · · Xn

• Sol(F) = { solutions of F }
• Model Counting: Determine |Sol(F)|

2/18

Model Counting

• Given

– Boolean variables X1, X2, · · · Xn – Formula F over X1, X2, · · · Xn

• Sol(F) = { solutions of F }
• Model Counting: Determine |Sol(F)|
• Given F := (X1 ∨ X2)

2/18

Model Counting

• Given

– Boolean variables X1, X2, · · · Xn – Formula F over X1, X2, · · · Xn

• Sol(F) = { solutions of F }
• Model Counting: Determine |Sol(F)|
• Given F := (X1 ∨ X2)
• Sol(F) = {(0, 1), (1, 0), (1, 1)}

2/18

Model Counting

• Given

– Boolean variables X1, X2, · · · Xn – Formula F over X1, X2, · · · Xn

• Sol(F) = { solutions of F }
• Model Counting: Determine |Sol(F)|
• Given F := (X1 ∨ X2)
• Sol(F) = {(0, 1), (1, 0), (1, 1)}
• |Sol(F)| = 3

2/18

Model Counting

• Given

– Boolean variables X1, X2, · · · Xn – Formula F over X1, X2, · · · Xn

• Sol(F) = { solutions of F }
• Model Counting: Determine |Sol(F)|
• Given F := (X1 ∨ X2)
• Sol(F) = {(0, 1), (1, 0), (1, 1)}
• |Sol(F)| = 3
• Model counting is #P-complete

(Valiant 1979)

2/18

Different Shades of Approximation

• Probabilistic (1 + ε)-Approximation

Pr |Sol(F)| 1 + ε ≤ ApproxCount(F, ε, δ) ≤ |Sol(F)|(1 + ε)

• ≥ 1 − δ

3/18

Different Shades of Approximation

• Probabilistic (1 + ε)-Approximation

Pr |Sol(F)| 1 + ε ≤ ApproxCount(F, ε, δ) ≤ |Sol(F)|(1 + ε)

• ≥ 1 − δ
• Constant Factor Approximation: (4,δ)

Pr |Sol(F)| 4 ≤ ConstantCount(F, δ) ≤ 4 · |Sol(F)|

• ≥ 1 − δ

3/18

Different Shades of Approximation

• Probabilistic (1 + ε)-Approximation

Pr |Sol(F)| 1 + ε ≤ ApproxCount(F, ε, δ) ≤ |Sol(F)|(1 + ε)

• ≥ 1 − δ
• Constant Factor Approximation: (4,δ)

Pr |Sol(F)| 4 ≤ ConstantCount(F, δ) ≤ 4 · |Sol(F)|

• ≥ 1 − δ
• From 4 to 2-factor

Let G = F1 ∧ F2 (i.e., two identical copies of F) |Sol(G)| 4 ≤C ≤ 4 · |Sol(G)| = ⇒ |Sol(F)| 2 ≤ √ C ≤ 2 · |Sol(F)|

3/18

Different Shades of Approximation

• Probabilistic (1 + ε)-Approximation

Pr |Sol(F)| 1 + ε ≤ ApproxCount(F, ε, δ) ≤ |Sol(F)|(1 + ε)

• ≥ 1 − δ
• Constant Factor Approximation: (4,δ)

Pr |Sol(F)| 4 ≤ ConstantCount(F, δ) ≤ 4 · |Sol(F)|

• ≥ 1 − δ
• From 4 to 2-factor

Let G = F1 ∧ F2 (i.e., two identical copies of F) |Sol(G)| 4 ≤C ≤ 4 · |Sol(G)| = ⇒ |Sol(F)| 2 ≤ √ C ≤ 2 · |Sol(F)|

• From 4 to (1 + ε)-factor

Construct G = F1 ∧ F2 . . . F 1

ε And then we can take 1

ε-root

3/18

Applications across Computer Science

Model Counting

Network Reliability Hardware Validation

Computational Biology

Neural Network Robustness Quantified Information Flow

4/18

As Simple as Counting Dots

The Rise of Hashing-based Approach: Promise of Scalability and Guarantees

(S83,GSS06,GHSS07,CMV13b,EGSS13b,CMV14,CDR15,CMV16,ZCSE16,AD16 KM18,ATD18,SM19,ABM20,SGM20)

5/18

As Simple as Counting Dots

5/18

As Simple as Counting Dots

5/18

As Simple as Counting Dots

Pick a random cell Estimate = Number of solutions in a cell × Number of cells

5/18

Challenges

Challenge 1 What is exactly a small cell ?

6/18

Challenges

Challenge 1 What is exactly a small cell ? Challenge 2 How to partition into roughly equal small cells of solutions without knowing the distribution of solutions?

6/18

Challenges

Challenge 1 What is exactly a small cell ?

• A cell is small cell if it has ≈ thresh solutions.
• Two choices for thresh.

– thresh = constant → 4-factor approximation – thresh = O( 1

ε2 ) gives (1 + ε)-approximation directly

6/18

Challenges

Challenge 1 What is exactly a small cell ?

• A cell is small cell if it has ≈ thresh solutions.
• Two choices for thresh.

– thresh = constant → 4-factor approximation – thresh = O( 1

ε2 ) gives (1 + ε)-approximation directly

• Zm be the number of solutions in a randomly chosen

cell ; we are interested in cases E[Zm] ≥ 1

6/18

Challenges

Challenge 1 What is exactly a small cell ?

• A cell is small cell if it has ≈ thresh solutions.
• Two choices for thresh.

– thresh = constant → 4-factor approximation – thresh = O( 1

ε2 ) gives (1 + ε)-approximation directly

• Zm be the number of solutions in a randomly chosen

cell ; we are interested in cases E[Zm] ≥ 1

• For thresh = O( 1

ε2 ), we need

dispersion index: σ2[Zm]

(E[Zm]) ≤ some constant

• For thresh = constant, sufficient to have

coefficient of variation:

σ2[Zm] (E[Zm])2 ≤ some constant

6/18

Challenges

Challenge 1 What is exactly a small cell ?

• A cell is small cell if it has ≈ thresh solutions.
• Two choices for thresh.

– thresh = constant → 4-factor approximation – thresh = O( 1

ε2 ) gives (1 + ε)-approximation directly

• Zm be the number of solutions in a randomly chosen

cell ; we are interested in cases E[Zm] ≥ 1

• For thresh = O( 1

ε2 ), we need

dispersion index: σ2[Zm]

(E[Zm]) ≤ some constant

• For thresh = constant, sufficient to have

coefficient of variation:

σ2[Zm] (E[Zm])2 ≤ some constant

Techniques based on thresh = O( 1

ε2 ) such as ApproxMC scale

significantly better than those based on thresh = constant.

6/18

Challenges

Challenge 1 What is exactly a small cell ? Challenge 2 How to partition into roughly equal small cells of solutions without knowing the distribution of solutions?

7/18

Challenges

Challenge 1 What is exactly a small cell ? Challenge 2 How to partition into roughly equal small cells of solutions without knowing the distribution of solutions?

• Designing function h : assignments → cells (hashing)
• Solutions in a cell α: Sol(F) ∩ {y | h(y) = α}

7/18

Challenges

Challenge 1 What is exactly a small cell ? Challenge 2 How to partition into roughly equal small cells of solutions without knowing the distribution of solutions?

• Designing function h : assignments → cells (hashing)
• Solutions in a cell α: Sol(F) ∩ {y | h(y) = α}
• Choose h randomly from a specially constructed large

family H of hash functions Carter and Wegman 1977

7/18

2-wise Independent Hashing

• Let H be family of 2-wise independent hash functions mapping

{0, 1}n to {0, 1}m ∀y1, y2 ∈ {0, 1}n, α1, α2 ∈ {0, 1}m, h

R

← − H Pr[h(y1) = α1] = Pr[h(y2) = α2] = 1 2m

• Pr[h(y1) = α1 ∧ h(y2) = α2] =

1 2m 2

8/18

2-wise Independent Hashing

• Let H be family of 2-wise independent hash functions mapping

{0, 1}n to {0, 1}m ∀y1, y2 ∈ {0, 1}n, α1, α2 ∈ {0, 1}m, h

R

← − H Pr[h(y1) = α1] = Pr[h(y2) = α2] = 1 2m

• Pr[h(y1) = α1 ∧ h(y2) = α2] =

1 2m 2

• The power of 2-wise independence

– Zm be the number of solutions in a randomly chosen cell – E[Zm] = |Sol(F)|

2m

– σ2[Zm] ≤ E[Zm]

8/18

2-wise Independent Hashing

• Let H be family of 2-wise independent hash functions mapping

{0, 1}n to {0, 1}m ∀y1, y2 ∈ {0, 1}n, α1, α2 ∈ {0, 1}m, h

R

← − H Pr[h(y1) = α1] = Pr[h(y2) = α2] = 1 2m

• Pr[h(y1) = α1 ∧ h(y2) = α2] =

1 2m 2

• The power of 2-wise independence

– Zm be the number of solutions in a randomly chosen cell – E[Zm] = |Sol(F)|

2m

– σ2[Zm] ≤ E[Zm] (Interested only in the cases when E[Zm] ≥ 1 )

8/18

2-wise Independent Hashing

• Let H be family of 2-wise independent hash functions mapping

{0, 1}n to {0, 1}m ∀y1, y2 ∈ {0, 1}n, α1, α2 ∈ {0, 1}m, h

R

← − H Pr[h(y1) = α1] = Pr[h(y2) = α2] = 1 2m

• Pr[h(y1) = α1 ∧ h(y2) = α2] =

1 2m 2

• The power of 2-wise independence

– Zm be the number of solutions in a randomly chosen cell – E[Zm] = |Sol(F)|

2m

– σ2[Zm] ≤ E[Zm] (Interested only in the cases when E[Zm] ≥ 1 ) – Zm be the number of solutions in a randomly chosen cell – dispersion index:

σ2[Zm] E[Zm] ≤ 1

– coefficient of variation:

σ2[Zm] (E[Zm])2 ≤ 1

8/18

2-wise Independent Hashing

• Variables: X1, X2, · · · Xn
• To construct h : {0, 1}n → {0, 1}m, choose m random XORs
• Pick every Xi with prob. 1

2 and XOR them

– X1 ⊕ X3 ⊕ X6 · · · ⊕ Xn−2 – Expected size of each XOR: n

2

9/18

2-wise Independent Hashing

• Variables: X1, X2, · · · Xn
• To construct h : {0, 1}n → {0, 1}m, choose m random XORs
• Pick every Xi with prob. 1

2 and XOR them

– X1 ⊕ X3 ⊕ X6 · · · ⊕ Xn−2 – Expected size of each XOR: n

2

• To choose α ∈ {0, 1}m, set every XOR equation to 0 or 1 randomly

X1 ⊕ X3 ⊕ X6 · · · ⊕ Xn−2 = 0 (Q1) X2 ⊕ X5 ⊕ X6 · · · ⊕ Xn−1 = 1 (Q2) · · · (· · · ) X1 ⊕ X2 ⊕ X5 · · · ⊕ Xn−2 = 1 (Qm)

• Solutions in a cell: F ∧ Q1 · · · ∧ Qm

9/18

2-wise Independent Hashing

• Variables: X1, X2, · · · Xn
• To construct h : {0, 1}n → {0, 1}m, choose m random XORs
• Pick every Xi with prob. 1

2 and XOR them

– X1 ⊕ X3 ⊕ X6 · · · ⊕ Xn−2 – Expected size of each XOR: n

2

• To choose α ∈ {0, 1}m, set every XOR equation to 0 or 1 randomly

X1 ⊕ X3 ⊕ X6 · · · ⊕ Xn−2 = 0 (Q1) X2 ⊕ X5 ⊕ X6 · · · ⊕ Xn−1 = 1 (Q2) · · · (· · · ) X1 ⊕ X2 ⊕ X5 · · · ⊕ Xn−2 = 1 (Qm)

• Solutions in a cell: F ∧ Q1 · · · ∧ Qm
• Performance of state of the art SAT solvers degrade with increase

in the size of XORs (SAT Solvers != SAT oracles)

9/18

The Hope of Short XORs

• If we pick every variable Xi with probability p

(Sparse: p ≪ 1

2; Dense: p = 1 2)

– Expected Size of each XOR: np – E[Zm] = |Sol(F)|

2m

(Interested only in the cases when E[Zm] ≥ 1 )

10/18

The Hope of Short XORs

• If we pick every variable Xi with probability p

(Sparse: p ≪ 1

2; Dense: p = 1 2)

– Expected Size of each XOR: np – E[Zm] = |Sol(F)|

2m

(Interested only in the cases when E[Zm] ≥ 1 ) – σ2[Zm] ≤ E[Zm] +

• σ1∈Sol(F)
• σ2∈Sol(F)

w=d(σ1,σ2)

r(w, p, m)

◮ where, r(w, p, m) =

• 1

2 + (1−2p)w 2

m −

1 2m

• – For p = 1

2, we have σ2[Zm] E[Zm] ≤ 1

10/18

The Core Technical Challenge

• Bounding the variance

(EGSS14,ZCSE16)

– σ2[Zm] ≤ E[Zm] + η

11/18

The Core Technical Challenge

• Bounding the variance

(EGSS14,ZCSE16)

– σ2[Zm] ≤ E[Zm] + η –

σ2[Zm] (E[Zm])2 ≤ 1 for p = O( log m m )

11/18

The Core Technical Challenge

• Bounding the variance

(EGSS14,ZCSE16)

– σ2[Zm] ≤ E[Zm] + η –

σ2[Zm] (E[Zm])2 ≤ 1 for p = O( log m m )

– Sparse XORs can be used in techniques that first compute constant factor approximation – From linear to logarithmic size XORs!

11/18

The Core Technical Challenge

• Bounding the variance

(EGSS14,ZCSE16)

– σ2[Zm] ≤ E[Zm] + η –

σ2[Zm] (E[Zm])2 ≤ 1 for p = O( log m m )

– Sparse XORs can be used in techniques that first compute constant factor approximation – From linear to logarithmic size XORs!

BUT Constant-factor approximation techniques don’t scale

11/18

The Core Technical Challenge

• Bounding the variance

(EGSS14,ZCSE16)

– σ2[Zm] ≤ E[Zm] + η –

σ2[Zm] (E[Zm])2 ≤ 1 for p = O( log m m )

– Sparse XORs can be used in techniques that first compute constant factor approximation – From linear to logarithmic size XORs!

BUT Constant-factor approximation techniques don’t scale

• Can we use these bounds for techniques such as ApproxMC that

compute (1 + ε)-approximation directly?

11/18

Our Contributions (1/2)

Explicit identification of the need for stronger bounds

• σ2[Zm] ≤ E[Zm] + η

(EGSS14,ZCSE16)

12/18

Our Contributions (1/2)

Explicit identification of the need for stronger bounds

• σ2[Zm] ≤ E[Zm] + η

(EGSS14,ZCSE16)

• We show η ∈ Ω((E[Zm])2)

Theorem (Informal) The currently best known bounds on σ2[Zm] are insufficient for techniques such as ApproxMC that have thresh = O(1/ε2)

• Open Problem: Derive better bounds

12/18

Our Contributions (2/2)

Do sparse XORs + constant factor techniques fare better than dense XOR-based techniques?

13/18

Our Contributions (2/2)

Do sparse XORs + constant factor techniques fare better than dense XOR-based techniques?

SparseCount2 SparseCount + Search Technique of CMV16

13/18

Our Contributions (2/2)

Do sparse XORs + constant factor techniques fare better than dense XOR-based techniques?

SparseCount2 SparseCount + Search Technique of CMV16 Fair comparison Same code base in C++ and identical underlying SAT solver: CryptoMiniSat Parameters ε = 0.8, δ = 0.2

13/18

Our Contributions (2/2)

Do sparse XORs + constant factor techniques fare better than dense XOR-based techniques?

14/18

Our Contributions(2/2)

Do sparse XORs + constant factor techniques fare better than dense XOR-based techniques?

15/18

Decoding the Surprise

• The sparse XORs do make the SAT calls of SparseCount2 easier

16/18

Decoding the Surprise

• The sparse XORs do make the SAT calls of SparseCount2 easier

BUT

16/18

Decoding the Surprise

• The sparse XORs do make the SAT calls of SparseCount2 easier

BUT

• Not All SAT calls are Equal

16/18

Decoding the Surprise

• The sparse XORs do make the SAT calls of SparseCount2 easier

BUT

• Not All SAT calls are Equal

– 4-factor to (1 + ε) requires multiple copies; so SparseCount2 has to work with large formula – Weak bounds on

σ2[Zm] (E[Zm])2 leads to larger number of SAT calls

16/18

Epilogue on Sparse XORs

• Low Density Parity Code-based Sparse XORs for approximate

model counting

(AT, SAT-16; ADT, SAT-18)

• Loss of theoretical guarantees

... the number of repetitions t explodes to over 1 million, making the derivation of rigorous (ε, δ)-approximations via Theorem 4 unrealistic.

17/18

Conclusion

• The rise of hashing-based techniques for approximate counting
• The runtime of SAT solvers depend on the size of XORs

18/18

Conclusion

• The rise of hashing-based techniques for approximate counting
• The runtime of SAT solvers depend on the size of XORs
• Short XORs suffice

18/18

Conclusion

• The rise of hashing-based techniques for approximate counting
• The runtime of SAT solvers depend on the size of XORs
• Short XORs suffice in theory

– In particular, there exist hashing-based techniques that can use short XORs

18/18

Conclusion

• The rise of hashing-based techniques for approximate counting
• The runtime of SAT solvers depend on the size of XORs
• Short XORs suffice in theory

– In particular, there exist hashing-based techniques that can use short XORs – Such techniques are significantly inefficient (379 vs 1169 benchmarks)

18/18

Conclusion

• The rise of hashing-based techniques for approximate counting
• The runtime of SAT solvers depend on the size of XORs
• Short XORs suffice in theory

– In particular, there exist hashing-based techniques that can use short XORs – Such techniques are significantly inefficient (379 vs 1169 benchmarks)

One last thing . . . some news to cheer you up

18/18

Conclusion

• The rise of hashing-based techniques for approximate counting
• The runtime of SAT solvers depend on the size of XORs
• Short XORs suffice in theory

– In particular, there exist hashing-based techniques that can use short XORs – Such techniques are significantly inefficient (379 vs 1169 benchmarks)

One last thing . . . some news to cheer you up

• New bounds on σ2[Zm] via Isoperimetric Inequalities

(Meel r Akshay, LICS 2020)

• Achieves σ2[Zm]

E[Zm] ≤ 1.1 (=constant)

• Speedup over ApproxMC3 – The first instance of sparse XORs

leading to runtime speedup over state of the art.

18/18