Exact Lifted Inference with Distinct Soft Evidence on Every Object Hung Hai Bui, Tuyen N. Huynh, Rodrigo de Salvo Braz Artificial Intelligence Center SRI International Menlo Park, CA, USA July 26, 2012 AAAI 2012 1/18
Outline 1 Outline 2 Distinct Soft Evidence is Problematic 3 LIDE (Lifted Inference with Distinct Evidence) 4 Experiments AAAI 2012 2/18
Lifted Inference and the Problematic Soft Evidence • The main idea of lifted inference is to exploit symmetry of the probabilistic models. This leads to algorithms that can be very efficient on high-tree width, but symmetric models • Soft evidence at the level of every object destroys the model’s symmetry • Everyone has different weight, cholesterol level, etc Symmetric ¡ Symmetry ¡destroyed ¡ • Aim: lifted inference with distinct soft evidence on every object AAAI 2012 3/18
Distinct Soft Evidence on a Unary Predicate • The simplest form of distinct soft evidence: on every grounding of a single unary predicate • Consider an MLN M consists of • An MLN M 0 with a unary predicate q . • A set of soft evidence of the form w i : q ( i ) for every object i . Evidence M 0 1 . 4 : ¬ Smokes ( x ) w 1 : Cancer ( P 1 ) 2 . 3 : ¬ Cancer ( x ) w 2 : Cancer ( P 2 ) 4 . 6 : ¬ Friends ( x , y ) ... 1 . 5 : Smokes ( x ) ⇒ Cancer ( x ) w 1000 : Cancer ( P 1000 ) 1 . 1 : Smokes ( x ) ∧ Friends ( x , y ) ⇒ Smokes ( y ) (tree-width = 1000) AAAI 2012 4/18
LIDE (Lifted Inference with Distinct Evidence) • Most lifted inference methods applied to M would completely shatter the model, thus reverting to ground inference. • LIDE’s approach 1 Perform lifted inference on M 0 only 2 Use special operations to absorb the soft evidences • Instead of exploiting symmetry of the model, we exploit symmetry of the partition function AAAI 2012 5/18
Symmetric Function Definition A n -variable function F ( t 1 , . . . , t n ) is symmetric if for all permutation π , permuting the variables of F by π does not change the output value, that is, F ( t 1 , . . . , t n ) = F ( t π ( 1 ) . . . , t π ( n ) ) . • F depends only on the histogram of its arguments. • If t i ∈ { 0 , 1 } , the set { c k } , k = 0 , . . . , n , where c k = F ( t ) for any t such that � t � 1 = k is termed the counting representation of the symmetric function F . • An exchangable distribution is a symmetric function, so it admits a counting representation. AAAI 2012 6/18
Exchangeability of Groundings of a Unary Predicate Theorem Let D ∗ = { d 1 , . . . , d n } be the set of individuals that do not appear as constants in the MLN M 0 and q be a unary predicate in M 0 . Let P 0 ( . ) = Pr ( q ( d 1 ) . . . q ( d n ) | M 0 ) . Then, the random vector ( q ( d 1 ) . . . q ( d n )) is exchangeable under P 0 . • Proof is in the paper. • This seems trivial: d 1 , . . . d n do not appear in M 0 so they are “indistinguishable”. But beware, “indistinguishable” does not necessarily imply exchangeable: groundings of an n -ary predicate in general are NOT exchangeable when n > 1. AAAI 2012 7/18
LIDE as a Wrapper 1 Step 1: apply any applicable lifted inference technique on M 0 to compute the counting representation { c k } of P 0 () . • One natural method is counting elimination. 2 Step 2: Absorb the soft evidence • Equivalent to compute the posterior of a set of exchangable binary random variables n P ( q 1 , . . . , q n ) = 1 � Z P 0 ( q 1 . . . q n ) φ i ( q i ) i = 1 where q i = q ( d i ) AAAI 2012 8/18
Posterior of Exchangeable Binary RVs n Pr ( q 1 , . . . , q n ) = 1 � Z P 0 ( q 1 . . . q n ) φ i ( q i ) i = 1 We discuss three related problems, to compute • The MAP configuration q under the marginal Pr ( q ) (a.k.a the marginal-map problem) • The partition function Z • The marginal Pr ( q i ) for each individual d i AAAI 2012 9/18
MAP Inference φ i ( 0 ) , Φ = � φ i ( 0 ) . Then Let α i = φ i ( 1 ) n P ( q ) = Φ � α q i Z P 0 ( q 1 . . . q n ) i i = 1 n P ( q ) = Φ � α q i max Z max max c k i q k q : � q � 1 = k i = 1 • Observation: the 2nd maximization simply picks k largest elements of α . • By sorting the vector α , the MAP problem can be solved in O ( n log ( n )) given { c k } as input. AAAI 2012 10/18
Partition Function Z n � � α q i Z ( α 1 , . . . , α n ) = Φ P 0 ( q 1 , . . . , q n ) i q 1 ... q n i = 1 • Observation: Z is a polynomial in α . More importantly Z is a symmetric polynomial. • According to the fundamental theorem of symmetric polynomials, it can be expressed in terms of a small number of building units called elementary symmetric polynomials. n � Z ( α ) = Φ c k e k ( α ) k = 0 AAAI 2012 11/18
Elementary Symmetric Polynomials • e k ( α ) is the k -th order elementary symmetric polynomial in α , the sum of all products of distinct k elements of α � e k ( α ) = α i 1 . . . α i k 1 ≤ i 1 <...< i k ≤ n � n � • Sum of terms, so naive evaluation is a bad idea. k • Newton’s Identity: Let p k ( α 1 . . . α n ) = � n i = 1 α k i be the k -th power sum. Then � k e k ( α ) = 1 i = 1 ( − 1 ) i − 1 e k − i ( α ) p i ( α ) k • This yields a recursive method to compute all e k ( α ) in O ( n 2 ) . • Thus, Z can be computed in O ( n 2 ) given { c k } . AAAI 2012 12/18
Marginal on Each Individual • As usual, the marginals Pr ( q i ) can be computed in a way similar to the computation of the normalization term Z , as the following theorem shows. • Let α ( i ) be the vector such that α ( i ) = 0 and α ( i ) = α j for i j every j � = i . Then Pr ( q i = 0 ) = Z ( α ( i ) ) � n k = 0 c k e k ( α ( i ) ) = � n Z ( α ) k = 0 c k e k ( α ) AAAI 2012 13/18
Experimental Setup • Friends and Smokes domain. • Task: compute the marginal probability of having cancer of each person given the cancer test readings of whole population. • Individual soft evidence uniformly sample from [0,2]. Thus, lifted BP reduces to ground BP • Two versions of the “Friends & and Smokes” MLN: • Original Friends-and-Smokes: encourage Smokes ( x ) and Smokes ( y ) to be the same if Friends ( x , y ) is unknown. • Attractive potential between Smokes ( x ) and Smokes ( y ) • Friends-and-Smokes-Neg: − 1 . 1 : Smokes ( x ) ∧ Friends ( x , y ) ⇒ Smokes ( y ) . • Repulsive potential between Smokes ( x ) and Smokes ( y ) • Difficult test case for BP AAAI 2012 14/18
Running Time on “Friends & Smokes” 900 800 Running time (seconds) 700 600 500 400 LIDE 300 BP 200 100 0 10 20 30 40 50 60 70 80 90 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 Number of persons *Note • Use a slightly modified C-FOVE for lifted inference without evidence • C-FOVE time dominates evidence absorbing time • Junction-tree ran out of memory for N = 30. AAAI 2012 15/18
Running Time on “Friends & Smokes-Neg” 9 8 Running time (seconds) 7 6 5 LIDE 4 BP with damping 3 (damping = 0.1) 2 1 0 10 20 30 40 50 60 70 80 90 100 Number of persons *Note: BP did not converge when N ≥ 200 AAAI 2012 16/18
Evidence Strength vs Probability 0.45 0.4 0.35 Prob(Cancer) 0.3 0.25 0.2 0.15 0.1 0.05 0 0 0.5 1 1.5 2 Evidence strength w *Note: • This is a scatter plot, not a function. • Distribution of Pr(Cancer) spreads, so quantization will loose accuracy. AAAI 2012 17/18
Conclusion and Future Direction • We propose a new strategy for handling distinct soft evidence • Perform lifted inference (e.g. C-FOVE) without the distinct soft evidence • Absorb the soft evidence by exploiting the symmetry of the partition polynomial Z • Future direction • Soft evidence on multiple ( L ) unary predicates. • Polynomial in domain size N , but super-exponential in L • Need to depart from exact inference and derive efficient approximation. • Soft evidence on one binary predicate • Intractable in general • Are there efficient approximations that can be derived from this approach? AAAI 2012 18/18
Recommend
More recommend
