Revisiting Enumerative Instantiation Andrew Reynolds 1 , Haniel Barbosa 1 , 2 and Pascal Fontaine 2 1 University of Iowa, Iowa City, U.S.A. 2 University of Lorraine, CNRS, Inria, LORIA, Nancy, France TACAS 2018/Matryoshka 2018/SMT 2018 1/15
Formal Program Veri fi cation Analysis Automated Reasoning Program Automatic Synthesis T esting 1/15
Formal Program Veri fi cation Analysis SMT Solvers Program Automatic Synthesis T esting 1/15
Formal Program Veri fi cation Analysis SMT Solvers + Quant Program Automatic Synthesis T esting 1/15
Outline ◮ Quantifier handling in SMT solving ◮ Strengthening the Herbrand Theorem ◮ Effective enumerative instantiation ◮ Combination with other instantiation strategies ◮ Implementation ◮ Evaluation a t i f c t r A * C o m t * p l e n t e * t e * A s i W S s E n e A o l l C C D C * o c * A e s u u m T e R e E o t n * t y s * d e v d a E a e u l t a 2/15
Quantifier handling in SMT 2/15
Problem statement SMT solver Instantiation module Model SMT formula Assignment Instance UNSAT Ground SMT solver Ground solver enumerates assignments E ∪ Q ◮ E is a set of ground literals { a ≤ b , b ≤ a + x , x ≃ 0 , f ( a ) �≃ f ( b ) } ◮ Q is a set of quantified clauses {∀ xyz . f ( x ) �≃ f ( z ) ∨ g ( y ) ≃ h ( z ) } Instantiation generates instances of Q f ( a ) �≃ f ( b ) ∨ g ( a ) ≃ h ( b ) 2/15
Instantiation strategies: trigger-based [Detlefs et al. J. ACM’05] Trigger-based instantiation (E-matching) : search for relevant instantiations according to a set of triggers and E -matching 3/15
Instantiation strategies: trigger-based [Detlefs et al. J. ACM’05] Trigger-based instantiation (E-matching) : search for relevant instantiations according to a set of triggers and E -matching ◮ E = {¬ P ( a ) , ¬ P ( b ) , P ( c ) , ¬ R ( b ) } and Q = {∀ x . P ( x ) ∨ R ( x ) } ◮ Assume trigger P ( x ) ◮ Find substitution σ for x such P ( x ) is a know term (in E ) ◮ Suitable substitutions are x �→ a , x �→ b , or x �→ c . E.g. E | = P ( x )[ x / a ] = P ( a ) and P ( a ) ∈ E ◮ Formally Select a set of triggers { ¯ t 1 , . . . ¯ e (E, ∀ ¯ x . ϕ ) 1. t n } for ∀ ¯ x . ϕ 2. For each i = 1 , . . . , n , select a set of substitutions S i s.t = ¯ for each σ ∈ S i , E | t i σ ≃ ¯ g i ∈ T (E) g i for some tuple ¯ Return � n 3. i =1 S i 3/15
Instantiation strategies: conflict-based [Reynolds et al. FMCAD’14] Conflict-based instantiation : search for instantiations of a quantified formula in Q that makes E unsatisfiable 4/15
Instantiation strategies: conflict-based [Reynolds et al. FMCAD’14] Conflict-based instantiation : search for instantiations of a quantified formula in Q that makes E unsatisfiable ◮ E = {¬ P ( a ) , ¬ P ( b ) , P ( c ) , ¬ R ( b ) } and Q = {∀ x . P ( x ) ∨ R ( x ) } ◮ Since E , P ( b ) ∨ R ( b ) | = ⊥ , this strategy returns x �→ b ◮ Formally c (E, ∀ ¯ x . ϕ ) Either returns σ where E , ϕσ | = ⊥ , or return ∅ 4/15
Instantiation strategies: model-based [Ge and de Moura CAV’09] Model-based instantiation (MBQI) : build a candidate model for E ∪ Q and instantiate with counter-examples from model checking 5/15
Instantiation strategies: model-based [Ge and de Moura CAV’09] Model-based instantiation (MBQI) : build a candidate model for E ∪ Q and instantiate with counter-examples from model checking ◮ E = {¬ P ( a ) , ¬ P ( b ) , P ( c ) , ¬ R ( b ) } and Q = {∀ x . P ( x ) ∨ R ( x ) } ◮ Assume that P M = λ x . ite( x ≃ c , ⊤ , ⊥ ) and R M = λ x . ⊥ ◮ Since M �| = P ( a ) ∨ R ( a ), MBQI may return x �→ a ◮ Formally m (E, ∀ ¯ x . ϕ ) 1. Construct a model M for E x �→ ¯ t where ¯ x / ¯ 2. Return ¯ t ∈ T (E) and M | = ¬ ϕ [¯ t ], or ∅ if none exists 5/15
Shortcomings ◮ Conflict-based instantiation ( c ) ◮ Inherently incomplete ◮ E -matching ( e ) ◮ Too many instances ◮ Butterfly effect ◮ MBQI ( m ) ◮ Complete for many fragments, but slow convergence for UNSAT ◮ Better suited for model finding Generally SMT solvers implement complete techniques by applying m as a last resort after trying c and e 6/15
Strengthening the Herbrand Theorem 7/15
Why can we use instantiation? Theorem (Herbrand) A set of pure first-order logic formulas is unsatisfiable if and only if there exists a finite unsatisfiable set of its instances 7/15
Why can we use instantiation? Theorem (Herbrand) A set of pure first-order logic formulas is unsatisfiable if and only if there exists a finite unsatisfiable set of its instances ◮ The earliest theorem provers relied on Herbrand instantiation ◮ Instantiate with all possible terms in the language ◮ Enumerating all instances is unfeasible in practice! ◮ Enumerative instantiation was then discarded 7/15
Why can we use instantiation? Theorem (Herbrand) A set of pure first-order logic formulas is unsatisfiable if and only if there exists a finite unsatisfiable set of its instances ◮ The earliest theorem provers relied on Herbrand instantiation ◮ Instantiate with all possible terms in the language ◮ Enumerating all instances is unfeasible in practice! ◮ Enumerative instantiation was then discarded We make enumerative instantiation beneficial for state-of-the-art SMT ◮ strengthening of Herbrand theorem ◮ efficient implementation techniques 7/15
Theorem (Strengthened Herbrand) If R is a (possibly infinite) set of instances of Q closed under Q-instantiation w.r.t. itself and if E ∪ R is satisfiable, then E ∪ Q is satisfiable. 8/15
Theorem (Strengthened Herbrand) If there exists an infinite sequence of finite satisfiable sets of ground literals E i and of finite sets of ground instances Q i of Q such that ◮ Q i = � � ϕσ | ∀ ¯ x . ϕ ∈ Q , dom( σ ) = { ¯ x } ∧ ran( σ ) ⊆ T (E i ) ; ◮ E 0 = E , E i +1 | = E i ∪ Q i ; then E ∪ Q is satisfiable in the empty theory with equality 8/15
Theorem (Strengthened Herbrand) If there exists an infinite sequence of finite satisfiable sets of ground literals E i and of finite sets of ground instances Q i of Q such that ◮ Q i = � � ϕσ | ∀ ¯ x . ϕ ∈ Q , dom( σ ) = { ¯ x } ∧ ran( σ ) ⊆ T (E i ) ; ◮ E 0 = E , E i +1 | = E i ∪ Q i ; then E ∪ Q is satisfiable in the empty theory with equality Direct application at SMT solver Instantiation module Model SMT formula Assignment Instance UNSAT Ground SMT solver ◮ Ground solver enumerates assignments E ∪ Q ◮ Instantiation module generates instances of Q 8/15
Effective enumerative instantiation 9/15
Enumerative instantiation u (E, ∀ ¯ x . ϕ ) Choose an ordering � on tuples of ground terms 1. x �→ ¯ t where ¯ 2. Return ¯ t is a minimal tuple of terms w.r.t � , such that ¯ x / ¯ t ∈ T (E) and E �| t ], or ∅ if none exist = ϕ [¯ ◮ E = {¬ P ( a ) , ¬ P ( b ) , P ( c ) , ¬ R ( b ) } and Q = {∀ x . P ( x ) ∨ R ( x ) } ◮ u chooses an ordering on tuples of terms, e.g. a ≺ b ≺ c ◮ Since E �| = P ( a ) ∨ R ( a ), enumerative instantiation returns x �→ a 9/15
u as an alternative for m ◮ Enumerative instantiation plays a similar role to MBQI ◮ It can also serve as a “completeness fallback” to c and e ◮ However, u has advantages over m for UNSAT problems ◮ Moreover it is significantly simpler to implement ◮ No model building ◮ No model checking 10/15
Example E = {¬ P ( a ) , R ( b ) , S ( c ) } Q = {∀ x . R ( x ) ∨ S ( x ) , ∀ x . ¬ R ( x ) ∨ P ( x ) , ∀ x . ¬ S ( x ) ∨ P ( x ) } P M = λ x . ⊥ , R M M = = λ x . ite( x ≃ b , ⊤ , ⊥ ) , , a ≺ b ≺ c S M = λ x . ite( x ≃ c , ⊤ , ⊥ ) ϕ x s.t. M | = ¬ ϕ x s.t. E �| = ϕ m (E , ∀ x . ϕ ) u (E , ∀ x . ϕ ) R ( x ) ∨ S ( x ) a a x �→ a x �→ a ¬ R ( x ) ∨ P ( x ) b a , b , c x �→ b x �→ a ¬ S ( x ) ∨ P ( x ) c a , b , c x �→ c x �→ a ◮ u instantiates uniformly so that new terms are introduced less often ◮ m instantiates depending on how model was built ◮ Moreover, u leads to E ∧ Q[ x / a ] | = ⊥ ◮ m requires considering E ′ which satisfies E along the new instances 11/15
Implementation Implementing enumerative instantiation efficiently depends on: ◮ Restricting enumeration space ◮ Avoiding entailed instantiations ◮ Term ordering to introduce new terms less often 12/15
Evaluation 13/15
CVC4 configurations on unsatisfiable benchmarks e+u 10 2 e;u e+m e;m CPU time (s) 10 1 e u m 10 0 10 − 1 6000 8000 10000 12000 14000 16000 18000 20000 ◮ 42 065 benchmarks: 14 731 TPTP + 27 334 SMT-LIB ◮ e + u stands for “interleave e and u ”, while e ; u for “apply e first, then u if it fails” ◮ All CVC4 configurations have “ c ;” as prefix 13/15
Impact of u on satisfiable benchmarks Library # u e ; u e + u m e ; m e + m e TPTP 14731 471 492 464 17 930 808 829 UF 7293 39 42 42 0 70 69 65 Theories 20041 3 3 3 3 350 267 267 Total 42065 513 537 509 20 1350 1144 1161 ◮ As expected, m greatly outperforms u ◮ u answers SAT half as often as m in empty theory ◮ u solves 13 problems m does not 14/15
Recommend
More recommend
