Sentential Decision Diagrams and their Applications Guy Van den Broeck, Arthur Choi, and Adnan Darwiche Nov 4, 2015, INFORMS
Basing Decisions on Sentences US Senate: 54 Rep., 44 Dem., and 2 Indep. p1 s1 p2 s2 p3 s3
Basing Decisions on Sentences US Senate: 54 Rep., 44 Dem., and 2 Indep. p1 s1 p2 s2 p3 s3 > 50 Rep.
Basing Decisions on Sentences US Senate: 54 Rep., 44 Dem., and 2 Indep. p1 s1 p2 s2 p3 s3 > 50 Veto Rep.
Basing Decisions on Sentences US Senate: 54 Rep., 44 Dem., and 2 Indep. p1 s1 p2 s2 p3 s3 > 50 Veto 48-50 Rep. Rep.
Basing Decisions on Sentences US Senate: 54 Rep., 44 Dem., and 2 Indep. p1 s1 p2 s2 p3 s3 > 50 Veto 48-50 Convince Rep. Rep. Indeps.
Basing Decisions on Sentences US Senate: 54 Rep., 44 Dem., and 2 Indep. p1 s1 p2 s2 p3 s3 < 48 > 50 Veto 48-50 Convince Rep. Rep. Rep. Indeps.
Basing Decisions on Sentences US Senate: 54 Rep., 44 Dem., and 2 Indep. p1 s1 p2 s2 p3 s3 < 48 Vote > 50 Veto 48-50 Convince Rep. Nay Rep. Rep. Indeps.
Basing Decisions on Sentences p1 s1 p2 s2 p3 s3
Basing Decisions on Sentences p1 s1 p2 s2 p3 s3 Branch on sentences p1, p2, and p3:
Basing Decisions on Sentences p1 s1 p2 s2 p3 s3 Branch on sentences p1, p2, and p3: p1, p2, p3 are mutually exclusive , exhaustive and not false
Basing Decisions on Sentences p1 s1 p2 s2 p3 s3 Branch on sentences p1, p2, and p3: p1, p2, p3 are mutually exclusive , exhaustive and not false p1, p2, p3 are called primes and represented by SDDs
Basing Decisions on Sentences p1 s1 p2 s2 p3 s3 Branch on sentences p1, p2, and p3: p1, p2, p3 are mutually exclusive , exhaustive and not false p1, p2, p3 are called primes and represented by SDDs s1, s2, s3 are called subs and represented by SDDs
Basing Decisions on Sentences f ( A , B , C , D ) = ( A B ) ( C D ) A B ¬ A A ¬ B A ¬ B ¬ A C D C D ¬ C
Basing Decisions on Sentences f ( A , B , C , D ) = ( A B ) ( C D ) A B A B A B ¬ A A ¬ B A ¬ B ¬ A C D C D ¬ C
Basing Decisions on Sentences f ( A , B , C , D ) = ( A B ) ( C D ) A B A B primes,subs primes,subs A B ¬ A A ¬ B A ¬ B ¬ A C D C D ¬ C
SDDs as Boolean Circuits f ( A , B , C , D ) ( A ( B D )) C = C C A ¬ A ¬ A A A ¬ A ¬ A A ¬ B B ¬ D ¬ B B D ¬ B B ¬ D ¬ B B D
( X , Y ) - Partitions US Senate: 54 Rep., 44 Dem., and 2 Indep. p1 s1 p2 s2 p3 s3 > 50 Veto 48-50 Convince < 48 Vote Rep. Rep. Indeps. Rep. Nay
( X , Y ) - Partitions US Senate: 54 Rep., 44 Dem., and 2 Indep. p1 s1 p2 s2 p3 s3 > 50 Veto 48-50 Convince < 48 Vote Rep. Rep. Indeps. Rep. Nay
( X , Y ) - Partitions US Senate: 54 Rep., 44 Dem., and 2 Indep. p1( X ) s1( Y ) p2( X ) s2( Y ) p3( X ) s3( Y ) > 50 Veto 48-50 Convince < 48 Vote Rep. Rep. Indeps. Rep. Nay
( X , Y ) - Partitions US Senate: 54 Rep., 44 Dem., and 2 Indep. p1( X ) s1( Y ) p2( X ) s2( Y ) p3( X ) s3( Y ) f ( X , Y ) = p 1 ( X ) s 1 ( Y ) … p n ( X ) s n ( Y )
Variable order becomes variable tree (vtree)
Variable order becomes variable tree (vtree)
Variable order becomes variable tree (vtree) vtree 6 Vtree 2 5 4 1 3 0 B A D C
Variable order becomes variable tree (vtree) vtree 6 Vtree 2 5 4 1 3 0 B A D C
Variable order becomes variable tree (vtree) vtree 6 Vtree 2 5 4 1 3 0 B A D C
OBDDs are SDDs 6 right-linear vtree 0 A 5 1 B 4 3 2 C D
OBDDs are SDDs 6 right-linear vtree 0 A 5 1 B 4 3 2 C D
OBDDs are SDDs 6 right-linear vtree A 0 B A 5 C 1 B 4 D 3 2 0 1 C D
Ingredients for Delicious Decision Diagrams • Minimization M • Apply Function A • Succinctness S • Queries Q
Ingredients for Delicious Decision Diagrams • Minimization M • Apply Function A • Succinctness S • Queries Q
M Compression • An ( X , Y ) - partition: f ( X , Y ) = p 1 ( X ) s 1 ( Y ) … p n ( X ) s n ( Y ) is compressed when subs are distinct: s i ( Y ) ≠ s i ( Y ) if i≠j • f ( X , Y ) has a unique compressed ( X,Y) -partition
M Compression • An ( X , Y ) - partition: f ( X , Y ) = p 1 ( X ) s 1 ( Y ) … p n ( X ) s n ( Y ) is compressed when subs are distinct: s i ( Y ) ≠ s i ( Y ) if i≠j • f ( X , Y ) has a unique compressed ( X,Y) -partition
M Compression • An ( X , Y ) - partition: f ( X , Y ) = p 1 ( X ) s 1 ( Y ) … p n ( X ) s n ( Y ) is compressed when subs are distinct: s i ( Y ) ≠ s i ( Y ) if i≠j • f ( X , Y ) has a unique compressed ( X,Y) -partition
M SDDs are Canonical For a fixed vtree (fixing X , Y throughout the SDD), compressed SDDs are canonical ! Equivalent sentences ≡ A (C ∨ D) (A C) ∨ (A D) have identical circuits. =
M OBDD Minimization 24 ordering of 4 variables ABCD ABDC ADBC DABC DACB ADCB ACDB ACBD CABD CADB CDAB DCAB DCBA CDBA CBDA CBAD BCAD BCDA BDCA DBCA DBAC BDAC BADC BACD 24 OBDDs for every function over 4 variables Searching for an optimal OBDD is searching for an optimal variable order
M SDD Minimization rrotate swap lrotate swap rrotate swap swap lrotate swap rrotate swap lrotate
Ingredients for Delicious Decision Diagrams • Minimization M • Apply Function A • Succinctness S • Queries Q
A Efficient Apply Function • Build Boolean combinations of existing circuits • Compile arbitrary sentence incrementally ( A ( B D )) ( C ∨ D ) ( A ( B D )) ( C ∨ D ) = • Polytime Apply: one Apply cannot blow up size ) = O( x
A Is Apply for SDDs Polytime? • | α |x| β | recursive calls • Polytime !
Ingredients for Delicious Decision Diagrams • Minimization M • Apply Function A • Succinctness S • Queries Q
S Succinctness • Theory – OBDD SDD thus SDD never larger than OBDD – Quasi-polynomial separation with OBDD OBDD can be much larger than SDD – Treewidth upper bounds (important in AI!) • Practice – SDD Compiler available and effective M A – SDD Package: http://reasoning.cs.ucla.edu/sdd/ – Can obtain orders of magnitude improvements
Ingredients for Delicious Decision Diagrams • Minimization M • Apply Function A • Succinctness S • Queries Q
Q Queries • OBDDs are Swiss army knife of supported queries • SDDs are equally powerful • Some enabled by canonicity + apply A • E.g., (Weighted) Model Counting for Probabilistic reasoning (E.g., Pr(bill passes|Vote1=Yea))
Application: Bayesian Networks • Incrementally compile network M A
Application: Bayesian Networks • Incrementally compile network M A
Application: Bayesian Networks • Incrementally compile network M A
Application: Bayesian Networks • Incrementally compile network M A =
Application: Bayesian Networks • Incrementally compile network M A =
Application: Bayesian Networks • Incrementally compile network M A • Compute probability of any query Q =
Application: Bayesian Networks • Incrementally compile network M A • Compute probability of any query Q • Better than state of the art (treewidth) S =
Application: Probabilistic Programming Model = program with random numbers 0.6 M P 0.9 reach(X,Y) :- flight(X,Y). 0.8 0.9 reach(X,Y) :- flight(X,Z), reach(Z,Y). 0.7 L A 0.8 M S Q A State of the art inference: SDDs
Application: Tractable Learning • Given: data • Objective: – learn a probability distribution – ensure distribution is tractable for querying • Unstructured space: Voting data • Structured space: Movie recommendation
Learning in Unstructured Spaces • Voting data from US House 1764 votes of 453 congressmen • Learn distribution (Markov network) M A • Represent as SDD to ensure tractability S • Query efficiency Q
Learning in Structured Spaces Student enrollment constraints: • Must take at least one of Probability or Logic. • Probability is a prerequisite for AI. • The prerequisites for KR is either AI or Logic. w = A K L P impossible
Example: Rankings and Permutations rank user 1 rank user 2 rank user 3 1 The Godfather 1 Star Wars V: The Empire Strikes Back 1 The Usual Suspects 2 Raiders of the Lost Ark 2 Star Wars IV: A New Hope 2 One Flew over the Cuckoo’s Nest 3 Casablanca 3 The Godfather 3 The Godfather: Part II 4 The Shawshank Redemption 4 The Shawshank Redemption 4 Monty Python and the Holy Grail 5 Schindler’s List 5 The Usual Suspects 5 Star Wars IV: A New Hope ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ Learn rankings of movies (permutations): Predict new movies given preferences
Recommend
More recommend
