FPRASs for DNF-Counting Kuldeep S. Meel 1 , Aditya A. Shrotri 2 , - - PowerPoint PPT Presentation

β–Ά
fprass for dnf counting
SMART_READER_LITE
LIVE PREVIEW

FPRASs for DNF-Counting Kuldeep S. Meel 1 , Aditya A. Shrotri 2 , - - PowerPoint PPT Presentation

Oct 09, 2022 752 likes β€’1.17k views

Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel 1 , Aditya A. Shrotri 2 , Moshe Y. Vardi 2 1 School of Computing, 2 Department of Computer Science, National University of Singapore Rice

slide-1
SLIDE 1

Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting

Kuldeep S. Meel1, Aditya A. Shrotri2, Moshe Y. Vardi2

1 School of Computing, 2 Department of Computer Science,

National University of Singapore Rice University

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 1

slide-2
SLIDE 2
  • SAT: Does there exist a satisfying assignment?
  • #SAT: How many satisfying assignments?
  • Complexity: #P-Complete (contains entire polynomial hierarchy)
  • In Practice: Harder than SAT

Beyond SAT: #SAT

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 2

Probabilistic Inference Measuring Information Leakage Network Reliability Probabilistic Databases #SAT

slide-3
SLIDE 3
  • F = Β¬π‘Œ1 ∧ π‘Œ2 ∨ π‘Œ2 ∧ π‘Œ3 ∧ π‘Œ4 ∨ (π‘Œ1 ∧ π‘Œ3 ∧ Β¬π‘Œ5)
  • Disjunction of Cubes
  • DNF-SAT is in P
  • #DNF is #P-Complete [Valiant, ’79]
  • Need to Approximate!

The Disjunctive Normal Form

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 3

slide-4
SLIDE 4

Input: DNF Formula F Tolerance Ξ΅ 0 < Ξ΅ < 1 Confidence Ξ΄ 0 < Ξ΄ < 1 Output: Approximate Count C s.t. Pr [#𝐆 β‹…(1-Ξ΅) < C < #𝐆 β‹…(1+Ξ΅) ] > 1-Ξ΄

Approximate DNF-Counting

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 4

slide-5
SLIDE 5

Input: DNF Formula F Tolerance Ξ΅ 0 < Ξ΅ < 1 Confidence Ξ΄ 0 < Ξ΄ < 1 Output: Approximate Count C s.t. Pr [#𝐆 β‹…(1-Ξ΅) < C < #𝐆 β‹…(1+Ξ΅) ] > 1-Ξ΄ Challenge: Design a poly(m, n, 1

πœ— , log( 1 πœ€ )) time algorithm

Fully Polynomial Randomized Approximation Scheme

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 5

where m = #cubes n = #vars

slide-6
SLIDE 6
  • Monte Carlo Sampling
  • Best complexity: 𝑷(𝒏 β‹… 𝒐 β‹… π’Žπ’‘π’‰(

𝟐 𝜺) β‹… 𝟐 πœ»πŸ‘)

  • Hashing – based
  • Best complexity: ΰ·©

𝑷(𝒏 β‹… 𝒐 β‹… π’Žπ’‘π’‰(

𝟐 𝜺) β‹… 𝟐 πœ»πŸ‘)

Paradigms for #DNF FPRAS

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 6

slide-7
SLIDE 7

1.

Can hashing – based approach have same complexity as Monte Carlo?

2.

How do the various algorithms compare empirically?

Motivation

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 7

slide-8
SLIDE 8

1.

Can hashing – based approach have same complexity as Monte Carlo?

2.

How do the various algorithms compare empirically? In this work:

  • Improved complexity of hashing – based algorithm
  • Improves practical performance
  • First comprehensive empirical evaluation of FPRASs for #DNF
  • Much more nuanced than complexity analysis alone

Our Contribution

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 8

slide-9
SLIDE 9

Method 1: Monte Carlo Sampling

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 9

U F

  • Draw independent samples from U
slide-10
SLIDE 10

Method 1: Monte Carlo Sampling

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 10

U F

  • Draw independent samples from U

U F

slide-11
SLIDE 11

Method 1: Monte Carlo Sampling

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 11

U F

  • Draw independent samples from U
  • Count samples that satisfy F

U F U F

slide-12
SLIDE 12

Method 1: Monte Carlo Sampling

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 12

U F

  • Draw independent samples from U
  • Count samples that satisfy F
  • #F β‰ˆ

#β€’ #β€’+#β€’ * |U|

U F U F

slide-13
SLIDE 13

Method 1: Monte Carlo Sampling

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 13

U F

  • Draw independent samples from U
  • Count samples that satisfy F
  • #F β‰ˆ

#β€’ #β€’+#β€’ * |U|

  • Requirement:

#𝐺 𝑉 is large

  • Solution density plays crucial role

U F U F

slide-14
SLIDE 14
  • KL Counter [Karp, Luby, β€˜83]
  • Insight: Transform solution space
  • KLM Counter [Karp et al., β€˜89] 𝑃(𝑛 β‹… π‘œ β‹… π‘šπ‘π‘•(

1 πœ€) β‹… 1 𝜁2)

  • Insight: Sample using geometric distribution
  • Vazirani Counter [Vazirani, ’13]
  • Insight: Low variance ⟹ fewer samples

Monte Carlo FPRAS

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 14

slide-15
SLIDE 15

Method 2: Hashing-Based

9/5/2018 On Hashing-Based Approaches to Approximate DNF Counting 15

U F

  • Partition U into small cells
  • Count solutions in a random cell β€˜Ccell’
  • #F β‰ˆ Ccell * no. of cells

U F

slide-16
SLIDE 16

Method 2: Hashing-Based

9/5/2018 On Hashing-Based Approaches to Approximate DNF Counting 16

U F

  • Partition U into small cells
  • Count solutions in a random cell β€˜Ccellβ€˜
  • #F β‰ˆ Ccell * no. of cells
  • Requirements:
  • Roughly equal solutions in each cell
  • Right size of cell

U F

slide-17
SLIDE 17
  • Requirement: Roughly Equal Solutions in each cell
  • Solution: Hash Functions
  • Conjunction of random XOR formulas
  • h = 𝐼1 ∧ 𝐼2 ∧ … ∧ 𝐼𝑗
  • h: 2π‘œ β†’ 2𝑗

Method 2: Hashing-Based

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 17

slide-18
SLIDE 18
  • Requirement: Right size cell
  • Solution: Count up to threshold π‘ˆ
  • π‘ˆ depends on 𝜁 and πœ€
  • π·π‘‘π‘“π‘šπ‘š > π‘ˆ ⟹ cell too large
  • Increase no. of constraints 𝑗
  • Find 𝑗 such that π·π‘‘π‘“π‘šπ‘š ≀ π‘ˆ

Method 2: Hashing-Based

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 18

slide-19
SLIDE 19
  • Requirement: Right size partition
  • Solution: Count up to threshold π‘ˆ
  • Find 𝑗 such that π·π‘‘π‘“π‘šπ‘š ≀ π‘ˆ

Forward Search

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 19

𝒋 π‘«π’…π’‡π’Žπ’Ž π·π‘‘π‘“π‘šπ‘š > π‘ˆ Cell size Too large

slide-20
SLIDE 20
  • Requirement: Right size partition
  • Solution: Count up to threshold π‘ˆ
  • Find 𝑗 such that π·π‘‘π‘“π‘šπ‘š ≀ π‘ˆ

Forward Search

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 20

𝒋 1 π‘«π’…π’‡π’Žπ’Ž π·π‘‘π‘“π‘šπ‘š > π‘ˆ π·π‘‘π‘“π‘šπ‘š > π‘ˆ Cell size Too large Too large

slide-21
SLIDE 21
  • Requirement: Right size partition
  • Solution: Count up to threshold π‘ˆ
  • Find 𝑗 such that π·π‘‘π‘“π‘šπ‘š ≀ π‘ˆ

Forward Search

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 21

𝒋 1 2 π‘«π’…π’‡π’Žπ’Ž π·π‘‘π‘“π‘šπ‘š > π‘ˆ π·π‘‘π‘“π‘šπ‘š > π‘ˆ π·π‘‘π‘“π‘šπ‘š > π‘ˆ Cell size Too large Too large Too large

slide-22
SLIDE 22
  • Requirement: Right size partition
  • Solution: Count up to threshold π‘ˆ
  • Find 𝑗 such that π·π‘‘π‘“π‘šπ‘š ≀ π‘ˆ
  • #F β‰ˆ Ccell * 2π‘š

Forward Search

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 22

𝒋 1 2 … π‘š βˆ’ 1 π‘š π‘«π’…π’‡π’Žπ’Ž π·π‘‘π‘“π‘šπ‘š > π‘ˆ π·π‘‘π‘“π‘šπ‘š > π‘ˆ π·π‘‘π‘“π‘šπ‘š > π‘ˆ … π·π‘‘π‘“π‘šπ‘š > π‘ˆ π·π‘‘π‘“π‘šπ‘š ≀ π‘ˆ Cell size Too large Too large Too large … Too large Perfect

slide-23
SLIDE 23

1.

DNFApproxMC [Chakraborty et al., β€˜16]

  • Insight: Satisfiability of partitioned DNF formulas is polytime

2.

SymbolicDNFApproxMC [Meel et al., β€˜17] ΰ·¨ 𝑃(𝑛 β‹… π‘œ β‹… π‘šπ‘π‘•(

1 πœ€) β‹… 1 𝜁2)

  • Insight: Symbolic hash constraints applied to transformed space

Hashing-Based FPRAS

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 23

slide-24
SLIDE 24
  • Requirement: Right size partition
  • Solution: Count up to threshold π‘ˆ
  • Find 𝑗 such that π·π‘‘π‘“π‘šπ‘š ≀ π‘ˆ
  • #F β‰ˆ Ccell * 2π‘š

Forward Search

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 24

𝒋 1 2 … π‘š βˆ’ 1 π‘š π‘«π’…π’‡π’Žπ’Ž π·π‘‘π‘“π‘šπ‘š > π‘ˆ π·π‘‘π‘“π‘šπ‘š > π‘ˆ π·π‘‘π‘“π‘šπ‘š > π‘ˆ … π·π‘‘π‘“π‘šπ‘š > π‘ˆ π·π‘‘π‘“π‘šπ‘š ≀ π‘ˆ Cell size Too large Too large Too large … Too large Perfect Drawback: Re-enumerates solutions multiple times

slide-25
SLIDE 25
  • Insight: Search in reverse direction
  • Advantages:
  • Each solution is enumerated exactly once
  • π‘š is close to π‘œ for DNF formulas

Reverse Search

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 25

𝒋 1 2 … π‘š βˆ’ 1 π‘š … n-1 n π‘«π’…π’‡π’Žπ’Ž π·π‘‘π‘“π‘šπ‘š > π‘ˆ π·π‘‘π‘“π‘šπ‘š > π‘ˆ π·π‘‘π‘“π‘šπ‘š > π‘ˆ … π·π‘‘π‘“π‘šπ‘š > π‘ˆ π·π‘‘π‘“π‘šπ‘š ≀ π‘ˆ … π·π‘‘π‘“π‘šπ‘š ≀ π‘ˆ π·π‘‘π‘“π‘šπ‘š ≀ π‘ˆ Cell size Too large Too large Too large … Too large Perfect … Too small Too small

slide-26
SLIDE 26
  • Insight: Search in reverse direction
  • Advantages:
  • Each solution is enumerated exactly once
  • π‘š is close to π‘œ for DNF formulas
  • Theorem: The complexity of SymbolicDNFApproxMC with

Reverse Search is 𝑃(𝑛 β‹… π‘œ β‹… π‘šπ‘π‘•(

1 πœ€) β‹… 1 𝜁2)

  • Reverse Search

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 26

𝒋 1 2 … π‘š βˆ’ 1 π‘š … n-1 n π‘«π’…π’‡π’Žπ’Ž π·π‘‘π‘“π‘šπ‘š > π‘ˆ π·π‘‘π‘“π‘šπ‘š > π‘ˆ π·π‘‘π‘“π‘šπ‘š > π‘ˆ … π·π‘‘π‘“π‘šπ‘š > π‘ˆ π·π‘‘π‘“π‘šπ‘š ≀ π‘ˆ … π·π‘‘π‘“π‘šπ‘š ≀ π‘ˆ π·π‘‘π‘“π‘šπ‘š ≀ π‘ˆ Cell size Too large Too large Too large … Too large Perfect … Too small Too small

slide-27
SLIDE 27

Reverse Search vs. Forward Search

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 27

900+ benchmarks

  • X-axis: Time taken by Binary Search
  • Y-axis: Time taken by Reverse

Search Reverse Search is 4-5 times faster!

slide-28
SLIDE 28
  • Monte Carlo – Based
  • KL Counter 𝑃(𝑛2 β‹… π‘œ β‹… π‘šπ‘π‘•(

1 πœ€) β‹… 1 𝜁2)

  • Vazirani Counter 𝑃(𝑛2 β‹… π‘œ β‹… π‘šπ‘π‘•(

1 πœ€) β‹… 1 𝜁2)

  • KLM Counter 𝑃(𝑛 β‹… π‘œ β‹… π‘šπ‘π‘•(

1 πœ€) β‹… 1 𝜁2)

  • Hashing – Based
  • DNFApproxMC

𝑃(𝑛 β‹… π‘œ2 β‹… π‘šπ‘π‘•(

1 πœ€) β‹… 1 𝜁2)

  • SymbolicDNFApproxMC

𝑃(𝑛 β‹… π‘œ β‹… π‘šπ‘π‘•(

1 πœ€) β‹… 1 𝜁2)

Experiments: Algorithms

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 28

slide-29
SLIDE 29

Runtime Variation

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 29

1.

Runtime Variation: How does the running time of the algorithms vary across different benchmarks?

  • Parameters
  • # vars = 100,000
  • # cubes ∈ [ 104, 8 Γ— 106 ]
  • Cube width ∈ {3, 13, 23, 33, 43}
  • 20 random benchmarks for each setting of parameters
slide-30
SLIDE 30

Runtime Variation

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 30

  • X-axis: Number of cubes
  • Y-axis: Median run time over 20

instances

  • Timeout: 500 sec

Observations:

  • DNFApproxMC performs very well
  • All other algorithms quickly timeout

Reason: Low density of solutions

  • Monte Carlo algorithms and

SymbolicDNFApproxMC are sensitive to solution density

slide-31
SLIDE 31

Runtime Variation

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 31

  • X-axis: Number of cubes
  • Y-axis: Median run time over 20

instances

  • Timeout: 500 sec

Observations:

  • DNFApproxMC performance

degrades gracefully

  • Performance of other algorithms

improves dramatically Reason: Density of solutions increases with cube-width

slide-32
SLIDE 32

Total Benchmarks Solved

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 32

2.

Total Benchmarks Solved: How many benchmarks can the algorithms solve overall?

  • Measures raw problem solving ability
  • Same 900 benchmarks as Runtime Variation
slide-33
SLIDE 33

Total Benchmarks Solved

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 33

  • Point (X,Y) represents X number of

benchmarks out of 900 were solved in Y seconds or less Observations:

  • DNFApproxMC does not timeout
  • n any formulas

Reason: Efficient data structures mitigate dependence on solution density

slide-34
SLIDE 34

1.

Runtime Variation: How does the running time of the algorithms vary across different benchmarks?

2.

Total Benchmarks Solved: How many benchmarks can the algorithms solve overall?

3.

Accuracy: How accurate are the counts returned by the algorithms?

4. 4.

𝝑 βˆ’ 𝜺 Scalability: How do the algorithms scale with the input tolerance and confidence?

Other Experiments

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 34

slide-35
SLIDE 35
  • Theory:
  • Introduced Reverse Search for hashing – based counting
  • Improved the complexity of hashing FPRAS by polylog factors
  • 4 – 5 times speedup in practice

Summary

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 35

slide-36
SLIDE 36
  • Empirical Evaluation
  • Presented nuanced picture
  • Different algorithms are well suited for different formula types
  • Algorithm with poor time complexity (DNFApproxMC) is most robust

and solves largest number of benchmarks

  • Takeaways for practitioners:
  • High solution density ⟹ use KLM Counter
  • Low or unknown solution density ⟹ use DNFApproxMC

Summary

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 36

slide-37
SLIDE 37
  • New counter that is robust as DNFApproxMC and fast as KLM

Counter?

  • Portfolio of algorithms approach
  • Real-world adoption of techniques
  • Break vicious cycle
  • Study effect of Reverse Search on CNF and SMT Counting

Future Directions

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 37

slide-38
SLIDE 38

Backup Slides

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 38

slide-39
SLIDE 39

Algorithm Mean Error Max Error DNFApproxMC 0.09 0.36 SymbolicDNFApproxMC 0.21 0.42 KLM Counter 0.11 0.55 KL Counter 0.007 0.20 Vazirani Counter 0.001 0.04

Experiments: Accuracy

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 39

slide-40
SLIDE 40

Experiments: 𝝑 βˆ’ 𝜺 Scalability

9/5/2018 Not All FPRASs are Equal: Demystifying FPRASs for DNF-Counting Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi 40