Approximate Counting in SMT and Value Estimation for Probabilistic - PowerPoint PPT Presentation

Approximate Counting in SMT and Value Estimation for Probabilistic Programs Dmitry Chistikov Rayna Dimitrova Rupak Majumdar Max Planck Institute for Software Systems (MPI-SWS) Kaiserslautern and Saarbr¨ ucken, Germany TACAS 2015 April 15, 2015

#SMT : Quickstart SAT 2/37

#SMT : Quickstart SAT Add counting #SAT 2/37

#SMT : Quickstart SAT Add counting Add theories #SAT SMT 2/37

#SMT : Quickstart SAT Add counting Add theories #SAT SMT #SMT 2/37

#SMT : Quickstart SAT Add counting NP Add theories #SAT SMT # P #SMT PH ⊆ P # P [Toda, FOCS’89] 2/37

Our contributions 1. Simple logical framework for #SMT problems 2. Approximate #SMT via reduction to black-box SMT (extend from #SAT ) ◮ Bounded integer arithmetic ◮ Linear real arithmetic 3. An application: value estimation for small probabilistic programs with nondeterminism 3/37

Approximate #SMT via reduction to SMT Idea: Use known SMT techniques as a black box and reduce approximate counting to decision problem Running time: Polynomial (randomized), but with queries to SMT oracle Output: Approximate, but with user-given precision 4/37

Our contributions 1. Simple logical framework for #SMT problems 2. Approximate #SMT via reduction to black-box SMT (extend from #SAT ) ◮ Bounded integer arithmetic ◮ Linear real arithmetic 3. An application: value estimation for small probabilistic programs with nondeterminism 5/37

Outline 1. Example: Probabilistic programs 2. #SMT : Logical framework 3. Approximate #SMT via reduction to SMT 4. Discussion and further directions 6/37

What are probabilistic programs? Probabilistic programs are a way to express probability distributions. This talk: Imperative, loop-free programs with: ◮ Coin-flipping (uniform distributions) ◮ Bayesian reasoning ◮ Nondeterminism 7/37

Value estimation problem Input: probabilistic program Output: value of the program Value of program = Pr( Accept | Accept or Reject ) Pr( Accept ) = Pr( Accept or Reject ) 8/37

Example: The Monty Hall problem [Selvin, American Statistician (1975)] 1 2 3 9/37

Example: The Monty Hall problem [Selvin, American Statistician (1975)] 1 2 3 9/37

Example: The Monty Hall problem [Selvin, American Statistician (1975)] 1 2 3 9/37

Example: The Monty Hall problem [Selvin, American Statistician (1975)] 1 2 3 I. Player chooses door i ∈ { 1 , 2 , 3 } . II. Host opens door j � = i with goat. III. Should the player change her choice? 9/37

Example: The Monty Hall problem [Selvin, American Statistician (1975)] 1 2 3 I. Player chooses door i ∈ { 1 , 2 , 3 } . II. Host opens door j � = i with goat. III. Should the player change her choice? 9/37

Example: The Monty Hall problem [Selvin, American Statistician (1975)] 1 2 3 I. Player chooses door i ∈ { 1 , 2 , 3 } . II. Host opens door j � = i with goat. III. Should the player change her choice? 9/37

Example: The Monty Hall problem [Selvin, American Statistician (1975)] 1 2 3 I. Player chooses door i ∈ { 1 , 2 , 3 } . II. Host opens door j � = i with goat. III. Should the player change her choice? 9/37

Example: A probabilistic program for Monty Hall a-proc:mh c ∼ Uniform ( { 1 , 2 , 3 } ) /* position of car */ i := 1 /* initial choice of player */ choice: /* host opens door j with goat */ case: j := 2; assume ( j � = c ) ; case: j := 3; assume ( j � = c ) ; if i � = c then accept; else reject; /* player switches from door i */ 10/37

Example: A probabilistic program for Monty Hall a-proc:mh c ∼ Uniform ( { 1 , 2 , 3 } ) /* position of car */ i := 1 /* initial choice of player */ choice: /* host opens door j with goat */ case: j := 2; assume ( j � = c ) ; case: j := 3; assume ( j � = c ) ; if i � = c then accept; else reject; /* player switches from door i */ c = 1 c = 2 c = 3 (with Pr = 1 / 3 each) × j := 2 reject accept × j := 3 reject accept Reject Accept Accept 10/37

Example: A probabilistic program for Monty Hall a-proc:mh c ∼ Uniform ( { 1 , 2 , 3 } ) /* position of car */ i := 1 /* initial choice of player */ choice: /* host opens door j with goat */ case: j := 2; assume ( j � = c ) ; case: j := 3; assume ( j � = c ) ; if i � = c then accept; else reject; /* player switches from door i */ c = 1 c = 2 c = 3 (with Pr = 1 / 3 each) × j := 2 reject accept × j := 3 reject accept Reject Accept Accept 10/37

Example: A probabilistic program for Monty Hall a-proc:mh c ∼ Uniform ( { 1 , 2 , 3 } ) /* position of car */ i := 1 /* initial choice of player */ choice: /* host opens door j with goat */ case: j := 2; assume ( j � = c ) ; case: j := 3; assume ( j � = c ) ; if i � = c then accept; else reject; /* player switches from door i */ c = 1 c = 2 c = 3 (with Pr = 1 / 3 each) × j := 2 reject accept × j := 3 reject accept Reject Accept Accept Pr( Accept ) Value of program = Pr( Accept or Reject ) = 2/3 10/37

Value estimation problem Input: probabilistic program Output: value of the program Value of program = Pr( Accept | Accept or Reject ) Pr( Accept ) = Pr( Accept or Reject ) 11/37

Value estimation reduces to #SMT Formulae Program ϕ acc , ϕ acc ∨ rej Accept = � ϕ acc � Accept or Reject = � ϕ acc ∨ rej � mc ( ϕ acc ) val ( Program ) = mc ( ϕ acc ∨ rej ) 12/37

Value estimation reduces to #SMT [Sankaranarayanan et al., PLDI’13] Formulae Program ϕ acc , ϕ acc ∨ rej Accept = � ϕ acc � Accept or Reject = � ϕ acc ∨ rej � mc ( ϕ acc ) val ( Program ) = mc ( ϕ acc ∨ rej ) Our work: 2 calls to #SMT 12/37

Outline 1. Example: Probabilistic programs 2. #SMT : Logical framework 3. Approximate #SMT via reduction to SMT 4. Discussion and further directions 13/37

Logical theories Logical theory T with fixed interpretation: ◮ formula : ϕ ( x 1 . . . , x k ) , variables with domain D ◮ model : ( a 1 , . . . , a k ) ∈ D k such that ϕ ( a 1 , . . . , a k ) is true ◮ satisfiability problem : � ϕ � � = ∅ ? ◮ model counting problem : ? 14/37

Measures The domain D is a measure space : it comes with ◮ σ -algebra F ⊆ 2 D ◮ ∅ ∈ F , F is closed under complement and countable ∪ ◮ measure µ : F → R ◮ µ is non-negative, µ ( ∅ ) = 0 and µ is σ -additive Lift µ to D k : µ ( A 1 × . . . × A k ) = µ ( A 1 ) . . . µ ( A k ) 15/37

Measured theories T is measured iff every � ϕ � is measurable. The model count of ϕ is mc ( ϕ ) = µ ( � ϕ � ) . 16/37

Measured theories: Examples Theory Domain Connectives Quantifiers mc ( ϕ ) Boolean { 0 , 1 } ∧ , ∨ , ¬ None Number of satisfiability satisfying assignments Integer Z ∩ [ a, b ] ∧ , ∨ , ¬ ∃ Number of arithmetic models Linear R ∩ [ a, b ] ∧ None Volume of programming polytope Linear real R ∩ [ a, b ] ∧ , ∨ , ¬ ∃ Volume arithmetic 17/37

Monty Hall in formulas ◮ Random variable c ∈ { 1 , 2 , 3 } ◮ Nondeterministic variables ◮ i, j ∈ { 1 , 2 , 3 } ◮ b 1 , . . . , b 5 , b init , b acc , b rej ∈ { 0 , 1 } (for program locations) ◮ Formula ϕ acc ( c ) : there exists an execution of the program that reaches accept ◮ Formula ϕ acc ∨ rej ( c ) : there exists an execution of the program that reaches accept or reject 18/37

Monty Hall in formulas ◮ Random variable c ∈ { 1 , 2 , 3 } ◮ Nondeterministic variables ◮ i, j ∈ { 1 , 2 , 3 } ◮ b 1 , . . . , b 5 , b init , b acc , b rej ∈ { 0 , 1 } (for program locations) ◮ Formula ϕ acc ( c ) : ∃ i ∃ j ∃ b 1 . . . ∃ b 5 ∃ b init ∃ b acc ∃ b rej ( b acc → ( i � = c ∧ b 5 )) ∧ ( b rej → ( i = c ∧ b 5 )) ∧ ( b 5 → (( j � = c ∧ b 3 ) ∨ ( j � = c ∧ b 4 ))) ∧ ( b 4 → ( j = 3 ∧ b 2 )) ∧ ( b 3 → ( j = 2 ∧ b 2 )) ∧ ( b 2 → ( i = 1 ∧ b 1 )) ∧ ( b 1 → b init ) ∧ b init ∧ b acc ◮ Formula ϕ acc ∨ rej ( c ) : replace last b acc with ( b acc ∨ b rej ) 18/37

Monty Hall in formulas ◮ Random variable c ∈ { 1 , 2 , 3 } ◮ Nondeterministic variables ◮ i, j ∈ { 1 , 2 , 3 } ◮ b 1 , . . . , b 5 , b init , b acc , b rej ∈ { 0 , 1 } (for program locations) ◮ Formula ϕ acc ( c ) : ∃ i ∃ j ∃ b 1 . . . ∃ b 5 ∃ b init ∃ b acc ∃ b rej ( b acc → ( i � = c ∧ b 5 )) ∧ ( b rej → ( i = c ∧ b 5 )) ∧ ( b 5 → (( j � = c ∧ b 3 ) ∨ ( j � = c ∧ b 4 ))) ∧ ( b 4 → ( j = 3 ∧ b 2 )) ∧ ( b 3 → ( j = 2 ∧ b 2 )) ∧ ( b 2 → ( i = 1 ∧ b 1 )) ∧ ( b 1 → b init ) ∧ b init ∧ b acc ◮ Formula ϕ acc ∨ rej ( c ) : replace last b acc with ( b acc ∨ b rej ) 18/37

Monty Hall in formulas ◮ Random variable c ∈ { 1 , 2 , 3 } ◮ Nondeterministic variables ◮ i, j ∈ { 1 , 2 , 3 } ◮ b 1 , . . . , b 5 , b init , b acc , b rej ∈ { 0 , 1 } (for program locations) ◮ Formula ϕ acc ( c ) : ∃ i ∃ j ∃ b 1 . . . ∃ b 5 ∃ b init ∃ b acc ∃ b rej ( b acc → ( i � = c ∧ b 5 )) ∧ ( b rej → ( i = c ∧ b 5 )) ∧ ( b 5 → (( j � = c ∧ b 3 ) ∨ ( j � = c ∧ b 4 ))) ∧ ( b 4 → ( j = 3 ∧ b 2 )) ∧ ( b 3 → ( j = 2 ∧ b 2 )) ∧ ( b 2 → ( i = 1 ∧ b 1 )) ∧ ( b 1 → b init ) ∧ b init ∧ b acc ◮ Formula ϕ acc ∨ rej ( c ) : replace last b acc with ( b acc ∨ b rej ) 18/37