Global Structure: Treewidth w ( exp( )) O n w 1 Local - PDF document

Readings: K&F: 4.1, 4.2, 4.3, 4.4, 8.4, 8.5, 8.6 “Recursive Conditioning”, Adnan Darwiche. In Artificial Intelligence Journal, 125:1, pp. 5-41 Context-specific independence Graphical Models – 10708 Carlos Guestrin Carnegie Mellon University October 16 th , 2006 Global Structure: Treewidth w ( exp( )) O n w 1

Local Structure 1: Context specific indepencence Battery Age Alternator Fan Belt Charge Delivered Battery Fuel Pump Fuel Line Starter Distributor Gas Battery Power Spark Plugs Gas Gauge Engine Start Radio Lights Engine Turn Over Local Structure 1: Context specific indepencence Context Specific I ndependence (CSI ) After observing a variable, some vars become independent Battery Age Alternator Fan Belt Charge Delivered Battery Fuel Pump Fuel Line Starter Distributor Gas Battery Power Spark Plugs Gas Gauge Engine Start Radio Lights Engine Turn Over 2

CSI example: Tree CPD Apply SAT Letter � Represent P(X i | Pa Xi ) using a decision tree � Path to leaf is an assignment to (a subset Job of) Pa Xi � Leaves are distributions over X i given assignment of Pa Xi on path to leaf � Interpretation of leaf : � For specific assignment of Pa Xi on path to this leaf – X i is independent of other parents � Representation can be exponentially smaller than equivalent table Tabular VE with Tree CPDs � If we turn a tree CPD into table � “Sparsity” lost ! � Need inference approach that deals with tree CPD directly ! 3

Local Structure 2: Determinism Determinism I f Battery Power = Dead , Battery Age Alternator Fan Belt then Lights = OFF Lights Charge Delivered ON Battery OFF Fuel Pump Fuel Line Battery OK .99 .01 Power Starter Distributor .80 Gas WEAK .20 Battery Power 0 1 DEAD Gas Gauge Spark Plugs Engine Start Radio Lights Engine Turn Over Determinism and inference � Determinism gives a little Lights ON sparsity in table, but much OFF Battery bigger impact on inference OK .99 .01 Power .80 � Multiplying deterministic factor WEAK .20 0 1 with other factor introduces DEAD many new zeros � Operations related to theorem proving, e.g., unit resolution 4

Today’s Models … � Often characterized by: � Richness in local structure (determinism, CSI) � Massiveness in size (10,000’s variables) � High connectivity (treewidth) � Enabled by: � High level modeling tools: relational, first order � Advances in machine learning � New application areas (synthesis): � Bioinformatics (e.g. linkage analysis) � Sensor networks � Exploiting local structure a must! Exact inference in large models is possible… � BN from a relational model 5

Recursive Conditioning � Treewidth complexity (worst case) � Better than treewidth complexity with local structure � Provides a framework for time-space tradeoffs � Only quick intuition today, details in readings The Computational Power of Assumptions Alternator Fan Belt Battery Age Leak Charge Delivered Battery Fuel Line Starter Gas Distributor Battery Power Spark Plugs Gas Gauge Engine Start Lights Engine Turn Over Radio A. Darwiche 6

The Computational Power of Assumptions Alternator Fan Belt Battery Age Leak Charge Delivered Battery Fuel Line Starter Gas Distributor Battery Power Spark Plugs Gas Gauge Engine Start Lights Engine Turn Over Radio A. Darwiche Decomposition Alternator Fan Belt Battery Age Leak Charge Delivered Battery Fuel Line Starter Gas Distributor Battery Power Spark Plugs Gas Gauge Engine Start Lights Engine Turn Over Radio A. Darwiche 7

Case Analysis Battery Age Alternator Fan Belt Battery Age Alternator Fan Belt Leak Leak Charge Delivered Charge Delivered Battery Fuel Line Battery Fuel Line Starter Gas Starter Distributor Gas Distributor Battery Power Battery Power Spark Plugs Spark Plugs Gas Gauge Gas Gauge Lights Engine Turn Over Engine Start Radio Lights Engine Turn Over Engine Start Radio + p p A. Darwiche Case Analysis Battery Age Alternator Fan Belt Battery Age Alternator Fan Belt Leak Leak Charge Delivered Charge Delivered Battery Fuel Line Battery Fuel Line Starter Gas Starter Distributor Gas Distributor Battery Power Battery Power Spark Plugs Spark Plugs Gas Gauge Gas Gauge Lights Engine Turn Over Engine Start Radio Lights Engine Turn Over Engine Start Radio * + p l p r p A. Darwiche 8

Case Analysis Battery Age Alternator Fan Belt Battery Age Alternator Fan Belt Leak Leak Charge Delivered Charge Delivered Battery Fuel Line Battery Fuel Line Starter Gas Starter Distributor Gas Distributor Battery Power Battery Power Spark Plugs Spark Plugs Gas Gauge Gas Gauge Lights Engine Turn Over Engine Start Radio Lights Engine Turn Over Engine Start Radio * + * p l p r p l p r A. Darwiche Case Analysis Alternator Fan Belt Battery Age Battery Age Alternator Fan Belt Leak Charge Delivered Leak Charge Delivered Battery Fuel Line Battery Fuel Line Starter Gas Starter Distributor Gas Distributor Battery Power Battery Power Spark Plugs Spark Plugs Gas Gauge Gas Gauge Lights Radio Engine Turn Over Engine Start Lights Radio Engine Turn Over Engine Start * + * p l p r p l p r A. Darwiche 9

Case Analysis Alternator Fan Belt Battery Age Battery Age Alternator Fan Belt Leak Charge Delivered Leak Charge Delivered Battery Fuel Line Battery Fuel Line Starter Gas Starter Distributor Gas Distributor Battery Power Battery Power Spark Plugs Gas Gauge Spark Plugs Gas Gauge Lights Radio Engine Turn Over Engine Start Lights Radio Engine Turn Over Engine Start * + * p l p r p l p r A. Darwiche Decomposition Tree A B C D E Cutset B A B B C A f(A) f(B,C) f(A,B) B C D E D f(C,D) f(B,D,E) A. Darwiche 10

Decomposition Tree A B C D E Cutset B A B B C A f(A) f(B,C) f(A,B) B C D E D f(C,D) f(B,D,E) A. Darwiche Decomposition Tree A B C D E Cutset B A B C A f(A) f(B,C) f(A,B) Time: O(n exp(w log n)) C D E Space: Linear D f(C,D) (using appropriate dtree) f(B,D,E) A. Darwiche 11

RC1 RC1(T,e) // compute probability of evidence e on dtree T If T is a leaf node Return Lookup(T,e) Else p := 0 for each instantiation c of cutset(T)-E do p := p + RC1(Tl,ec) RC1(Tr,ec) return p A. Darwiche Lookup(T, e ) Θ X| U : CPT associated with leaf T If X is instantiated in e , then x: value of X in e u : value of U in e Return θ x| u Else return 1 = Σ x θ x| u A. Darwiche 12

Caching A B C D E F A B Context A C ABC C .27 ABC A B C B ABC A C .39 ABC B C ABC ABC C D ABC ABC E F D E A. Darwiche Caching A B C D E F Recursive Conditioning A An any-space algorithm with treewidth complexity B Darwiche AIJ-01 Context A C ABC Time: O(n exp(w)) C .27 ABC Space: O(n exp(w)) A B C B ABC A C (using appropriate dtree) .39 ABC B C ABC ABC C D ABC ABC E F D E A. Darwiche 13

RC2 RC2(T, e ) If T is a leaf node, return Lookup(T,e) y := instantiation of context(T) If cache T [ y ] < > nil, return cache T [ y ] p := 0 For each instantiation c of cutset(T)- E do p := p + RC2(T l , ec ) RC2(T r , ec ) cache T [ y ] := p Return p A. Darwiche Decomposition with Local Structure X I ndependent of B, C given A A, B, C A X B C A. Darwiche 14

Decomposition with Local Structure X I ndependent of B, C given A A, B, C A X B C A. Darwiche Decomposition with Local Structure X I ndependent of B, C given A A, B, C No need to consider an exponential number of cases (in the cutset size) given local structure A X B C A. Darwiche 15

Structural cache Caching with Local A B C Structure A B C A B C B,C A B C A B C A A B C A B C A,B,C A B C A X B C A. Darwiche Structural cache Caching with Local A B C Structure A B C A B C B,C A B C A B C A A B C A B C A,B,C A B C A X B C A. Darwiche 16

Structural cache Caching with Local A B C Structure No need to cache an exponential number of A B C results (in the context size) given local structure A B C B,C Non- A B C Structural A B C cache A A B C A A B C A B C A,B,C A B C A B C A B C A A B C X B C A. Darwiche Determinism… ¬ ∧ ¬ ∧ ¬ ⇒ ¬ A B C X ⇒ A X A, B, C ⇒ B X ⇒ C X A natural setup to incorporate SAT technology: A • Unit resolution to: X • Derive values of variables B • Detect/ skip inconsistent cases • Dependency directed backtracking • Clause learning C A. Darwiche 17

CSI Summary � Exploit local structure � Context-specific independence � Determinism � Significantly speed-up inference � Tackle problems with tree-width in the thousands � Acknowledgements � Recursive conditioning slides courtesy of Adnan Darwiche � Implementation available: � http://reasoning.cs.ucla.edu/ace 18