Probabilistic Inference in BN2T Models by Weighted Model Counting - - PowerPoint PPT Presentation

Probabilistic Inference in BN2T Models by Weighted Model Counting

Jirka Vomlel

Institute of Information Theory and Automation Academy of Sciences of the Czech Republic http://www.utia.cz/vomlel

Aalborg, Denmark, November, 21, 2013

A medical example: Carcinoid Heart Disease (CHD), van Gerven (2003)

• Eleven CHD risk factors X1, . . . , X11 measured at patient

admission to the clinic: diarrhea, hepatic metastases, etc.

A medical example: Carcinoid Heart Disease (CHD), van Gerven (2003)

• Eleven CHD risk factors X1, . . . , X11 measured at patient

admission to the clinic: diarrhea, hepatic metastases, etc.

• Dependent variable Y has two values: either 0 if CHD does

not develop of 1 if it does.

A medical example: Carcinoid Heart Disease (CHD), van Gerven (2003)

• Eleven CHD risk factors X1, . . . , X11 measured at patient

admission to the clinic: diarrhea, hepatic metastases, etc.

• Dependent variable Y has two values: either 0 if CHD does

not develop of 1 if it does.

• The conditional probability P(Y |X1, . . . , X11) is modeled by a

noisy threshold model with ℓ = 6.

A medical example: Carcinoid Heart Disease (CHD), van Gerven (2003)

• Eleven CHD risk factors X1, . . . , X11 measured at patient

admission to the clinic: diarrhea, hepatic metastases, etc.

• Dependent variable Y has two values: either 0 if CHD does

not develop of 1 if it does.

• The conditional probability P(Y |X1, . . . , X11) is modeled by a

noisy threshold model with ℓ = 6.

• The threshold model (without noise) implies that the CHD

develop if at least 6 risk factors are positive.

A medical example: Carcinoid Heart Disease (CHD), van Gerven (2003)

• Eleven CHD risk factors X1, . . . , X11 measured at patient

admission to the clinic: diarrhea, hepatic metastases, etc.

• Dependent variable Y has two values: either 0 if CHD does

not develop of 1 if it does.

• The conditional probability P(Y |X1, . . . , X11) is modeled by a

noisy threshold model with ℓ = 6.

• The threshold model (without noise) implies that the CHD

develop if at least 6 risk factors are positive.

• The noise on inputs allows a non-zero probability of no CHD

even if at least 6 risk factors are positive.

BN2T - Bayesian Network with 2 Layers Consisting of Noisy Threshold Models

Y2 Y1 X1 X2 X3 X4

Yj takes value 1 iff at least ℓ out of k parents Xi take value 1.

BN2T - Bayesian Network with 2 Layers Consisting of Noisy Threshold Models

Y2 Y1 X1 X2 X3 X4

Yj takes value 1 iff at least ℓ out of k parents Xi take value 1. Assume ℓ = 2.

BN2T - Bayesian Network with 2 Layers Consisting of Noisy Threshold Models

Y2 Y1 X1 X2 X3 X4

Yj takes value 1 iff at least ℓ out of k parents Xi take value 1. Assume ℓ = 2. For deterministic threshold the value of p = P(Y1 = 1|X1, X2, X3) is

X1 X2 X3 p 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

BN2T - Bayesian Network with 2 Layers Consisting of Noisy Threshold Models

Y2 Y1 X1 X2 X3 X4

Yj takes value 1 iff at least ℓ out of k parents Xi take value 1. Assume ℓ = 2. For noisy threshold the value of p′ = P(Y1 = 1|X1, X2, X3) is

X1 X2 X3 p p′ 1 1 1 1 1 (1 − p2)(1 − p3) 1 1 1 1 (1 − p1)(1 − p3) 1 1 1 (1 − p1)(1 − p2) 1 1 1 1 (1 − p1)(1 − p2)(1 − p3) +p1(1 − p2)(1 − p3) +(1 − p1)p2(1 − p3) +(1 − p1)(1 − p2)p3

where p1, p2, p3 are inhibitory probabilities.

BN2T - Bayesian Network with 2 Layers Consisting of Noisy Threshold Models

Y2 Y1 X1 X2 X3 X4

Yj takes value 1 iff at least ℓ out of k parents Xi take value 1. Assume ℓ = 2. The joint probability of the Bayesian network P(X1, . . . , Xn, Y1, . . . , Ym) =

n

• i=1

P(Xi)

m

• j=1

P(Yj|pa(Yj)) .

Noisy-threshold with explicit deterministic and noisy parts

Xk . . . X′

2

X′

1

Y X1 X2 . . . X′

k

Noisy-threshold with explicit deterministic and noisy parts

Xk . . . X′

2

X′

1

Y X1 X2 . . . X′

k

PX′

i |Xi

=

1

pi 1 − pi

Noisy-threshold with explicit deterministic and noisy parts

Xk . . . X′

2

X′

1

Y X1 X2 . . . X′

k

PX′

i |Xi

=

1

pi 1 − pi

• PY |X′

1,X′ 2,X′ 3

=

     

• 1

1 1

• 1
• 1
• 1

1 1

• 

    

. where we visualize the CPT as a tensor using nested matrices.

Tensor of the threshold (for ℓ = 1) as 4D cube and its decomposition

1 1 1 1 1 1 1 1 Y X′ 1 X′ 2 X′ 3

Tensor of the threshold (for ℓ = 1) as 4D cube and its decomposition

−1 1 1 1 1 1 1 1

= +

1 1 1 1 1 1 1 1 1 1 Y X′ 3 X′ 2 X′ 1

Tensor of the threshold (for ℓ = 1) as 4D cube and its decomposition

−1 1 1 1 1 1 1 1

= +

1 1 1 1 1 1 1 1 1 1 Y X′ 3 X′ 2 X′ 1 −1

= +

1 1 1 1 1 1 1 1 1 1 1

CP tensor decomposition of the threshold CPT (Vomlel, Tichavsk´ y, 2012)

X′

1

X′

2

. . . X′

k

Y ′

CP tensor decomposition of the threshold CPT (Vomlel, Tichavsk´ y, 2012)

X′

1

X′

2

. . . X′

k

Y ′

PY =1|X′

1,X′ 2,X′ 3

=

• Y ′

3

• i=1

ψX′

i ,Y ′,

CP tensor decomposition of the threshold CPT (Vomlel, Tichavsk´ y, 2012)

X′

1

X′

2

. . . X′

k

Y ′

PY =1|X′

1,X′ 2,X′ 3

=

• Y ′

3

• i=1

ψX′

i ,Y ′,

ψX′

i ,Y ′

=

  

3

• 1

6

3

• 1

3

− 3

• 1

2

2 3

• 1

6

− 3

• 1

3

  

for i = 1, 2, 3.

CP tensor decomposition of the threshold CPT (Vomlel, Tichavsk´ y, 2012)

X′

1

X′

2

. . . X′

k

Y ′

PY =1|X′

1,X′ 2,X′ 3

=

• Y ′

3

• i=1

ψX′

i ,Y ′,

ψX′

i ,Y ′

=

  

3

• 1

6

3

• 1

3

− 3

• 1

2

2 3

• 1

6

− 3

• 1

3

  

for i = 1, 2, 3. Instead of an array with 2k entries we get k arrays with 2k entries!

Probabilistic inference by weighted model counting (WMC)

• The basic idea of WMC is to encode a Bayesian network using

a conjunctive normal form (CNF),

Probabilistic inference by weighted model counting (WMC)

• The basic idea of WMC is to encode a Bayesian network using

a conjunctive normal form (CNF),

• associate weights to literals according to the CPTs of the

Bayesian network, and

Probabilistic inference by weighted model counting (WMC)

• The basic idea of WMC is to encode a Bayesian network using

a conjunctive normal form (CNF),

• associate weights to literals according to the CPTs of the

Bayesian network, and

• compute the probability of evidence as the sum of weights of

all logical models consistent with that evidence.

Probabilistic inference by weighted model counting (WMC)

• The basic idea of WMC is to encode a Bayesian network using

a conjunctive normal form (CNF),

• associate weights to literals according to the CPTs of the

Bayesian network, and

• compute the probability of evidence as the sum of weights of

all logical models consistent with that evidence.

• The weight of a logical model is the product of weights of all

literals.

Probabilistic inference by weighted model counting (WMC)

• The basic idea of WMC is to encode a Bayesian network using

a conjunctive normal form (CNF),

• associate weights to literals according to the CPTs of the

Bayesian network, and

• compute the probability of evidence as the sum of weights of

all logical models consistent with that evidence.

• The weight of a logical model is the product of weights of all

literals.

• Efficient WMC solvers exploiting several advanced techniques

such as clause learning, component caching can be used – e.g. Cachet.

Probabilistic inference by weighted model counting (WMC)

• The basic idea of WMC is to encode a Bayesian network using

a conjunctive normal form (CNF),

• associate weights to literals according to the CPTs of the

Bayesian network, and

• compute the probability of evidence as the sum of weights of

all logical models consistent with that evidence.

• The weight of a logical model is the product of weights of all

literals.

• Efficient WMC solvers exploiting several advanced techniques

such as clause learning, component caching can be used – e.g. Cachet.

• If the Bayesian network exhibit a lot of determinism this is

much more efficient than standard techniques.

Encoding transformed BN2T as a CNF using Chavira and Darwiche (2008) encoding

Clauses for indicator (λ) and parameter (θ) logical variables: ”states of Xi are mutualy exclusive” ⊕x∈Xiλx

Xi

Encoding transformed BN2T as a CNF using Chavira and Darwiche (2008) encoding

Clauses for indicator (λ) and parameter (θ) logical variables: ”states of Xi are mutualy exclusive” ⊕x∈Xiλx

Xi

”states of Y ′

j are mutualy exclusive”

⊕y∈Y′

jλy

Y ′

j

Encoding transformed BN2T as a CNF using Chavira and Darwiche (2008) encoding

Clauses for indicator (λ) and parameter (θ) logical variables: ”states of Xi are mutualy exclusive” ⊕x∈Xiλx

Xi

”states of Y ′

j are mutualy exclusive”

⊕y∈Y′

jλy

Y ′

j

”when parameters of P(Xi) applies” θx

Xi ⇔ λx Xi

Encoding transformed BN2T as a CNF using Chavira and Darwiche (2008) encoding

Clauses for indicator (λ) and parameter (θ) logical variables: ”states of Xi are mutualy exclusive” ⊕x∈Xiλx

Xi

”states of Y ′

j are mutualy exclusive”

⊕y∈Y′

jλy

Y ′

j

”when parameters of P(Xi) applies” θx

Xi ⇔ λx Xi

”when parameters of ψ(Xi, Y ′

j ) applies”

θx,y

Xi,Y ′

j

⇔ λx

Xi ∧ λy Y ′

j

Encoding transformed BN2T as a CNF using Chavira and Darwiche (2008) encoding

Clauses for indicator (λ) and parameter (θ) logical variables: ”states of Xi are mutualy exclusive” ⊕x∈Xiλx

Xi

”states of Y ′

j are mutualy exclusive”

⊕y∈Y′

jλy

Y ′

j

”when parameters of P(Xi) applies” θx

Xi ⇔ λx Xi

”when parameters of ψ(Xi, Y ′

j ) applies”

θx,y

Xi,Y ′

j

⇔ λx

Xi ∧ λy Y ′

j

The weights of all positive literals: w(λx

Xi) = 1 and w(λy Y ′

j ) = 1

w(θx

Xi) = PXi(x) and w(θx,y Xi,Y ′

j ) = ψXi,Y ′ j (x, y)

Encoding transformed BN2T as a CNF using Chavira and Darwiche (2008) encoding

Clauses for indicator (λ) and parameter (θ) logical variables: ”states of Xi are mutualy exclusive” ⊕x∈Xiλx

Xi

”states of Y ′

j are mutualy exclusive”

⊕y∈Y′

jλy

Y ′

j

”when parameters of P(Xi) applies” θx

Xi ⇔ λx Xi

”when parameters of ψ(Xi, Y ′

j ) applies”

θx,y

Xi,Y ′

j

⇔ λx

Xi ∧ λy Y ′

j

The weights of all positive literals: w(λx

Xi) = 1 and w(λy Y ′

j ) = 1

w(θx

Xi) = PXi(x) and w(θx,y Xi,Y ′

j ) = ψXi,Y ′ j (x, y)

The weights of all negative literals ¬A are all one.

A comparison with the standard approach based on WMC

• Clauses consist of at most three literals.

A comparison with the standard approach based on WMC

• Clauses consist of at most three literals.
• In the standard approach the number of literals in some

clauses is the number of parents of corresponding variable.

A comparison with the standard approach based on WMC

• Clauses consist of at most three literals.
• In the standard approach the number of literals in some

clauses is the number of parents of corresponding variable.

• The number of clauses is polynomial with respect to the

maximal number of variable’s parents.

A comparison with the standard approach based on WMC

• Clauses consist of at most three literals.
• In the standard approach the number of literals in some

clauses is the number of parents of corresponding variable.

• The number of clauses is polynomial with respect to the

maximal number of variable’s parents.

• In the standard approach the number of clauses is exponential

with respect to the number of parents of corresponding variable.

Conclusions

• Noisy threshold models can be used in applications where

more than one factor need to be positive for a symptom to be present.

Conclusions

• Noisy threshold models can be used in applications where

more than one factor need to be positive for a symptom to be present.

• Noisy threshold is a generalization of noisy-or and noisy-max.

Conclusions

• Noisy threshold models can be used in applications where

more than one factor need to be positive for a symptom to be present.

• Noisy threshold is a generalization of noisy-or and noisy-max.
• The CP tensor decomposition of noisy-threshold should be

applied before the construction of CNF used for weighted model counting.

Conclusions

• Noisy threshold models can be used in applications where

more than one factor need to be positive for a symptom to be present.

• Noisy threshold is a generalization of noisy-or and noisy-max.
• The CP tensor decomposition of noisy-threshold should be

applied before the construction of CNF used for weighted model counting.

• The CP tensor decomposition as a preprocessing step for

WMC can be utilized also for other types of local structure of conditional probability tables.