Global Structure: Treewidth w O ( n exp( w )) 1 Local - - PDF document

1

Context-specific independence

Graphical Models – 10708 Carlos Guestrin Carnegie Mellon University October 16th, 2006

Readings: K&F: 4.1, 4.2, 4.3, 4.4, 8.4, 8.5, 8.6 “Recursive Conditioning”, Adnan Darwiche. In Artificial Intelligence Journal, 125:1, pp. 5-41

Global Structure: Treewidth w

)) exp( ( w n O

2

• !

"

• Local Structure 1:

Context specific indepencence

• !

"

• # $%$& $'&(

%!!) !$

Local Structure 1: Context specific indepencence

3

CSI example: Tree CPD

Represent P(Xi|PaXi) using a decision tree

Path to leaf is an assignment to (a subset

• f) PaXi

Leaves are distributions over Xi given

assignment of PaXi on path to leaf

Interpretation of leaf:

For specific assignment of PaXi on path to

this leaf – Xi is independent of other parents

Representation can be exponentially

smaller than equivalent table

Apply SAT Letter Job

Tabular VE with Tree CPDs

If we turn a tree CPD into table

“Sparsity” lost!

Need inference approach that deals with

tree CPD directly!

4

Local Structure 2: Determinism

• !

"

• *
• +

,+

• ..
• /0
• 1/
• 2/

/

&%3 3

• Determinism and inference

Determinism gives a little

sparsity in table, but much bigger impact on inference

Multiplying deterministic factor

with other factor introduces many new zeros

Operations related to theorem

proving, e.g., unit resolution

*

• +

,+

• ..
• /0
• 1/
• 2/

/

5

Today’s Models …

Often characterized by:

Richness in local structure (determinism, CSI) Massiveness in size (10,000’s variables) High connectivity (treewidth)

Enabled by:

High level modeling tools: relational, first order Advances in machine learning New application areas (synthesis):

Bioinformatics (e.g. linkage analysis) Sensor networks

Exploiting local structure a must!

Exact inference in large models is possible…

BN from a relational model

6

Recursive Conditioning

Treewidth complexity (worst case) Better than treewidth complexity with local

structure

Provides a framework for time-space tradeoffs Only quick intuition today, details in readings

• $

%

• "
• !

"

7

• $
• "
• !

"

• %
• $

$

• "
• !

"

8

• $
• !

"

• !

"

• "

"

4

• $
• !

"

• !

"

• "

"

4

"

5

9

• $
• !

"

• !

"

• "

4

"

5

" "

5

• $
• !

"

• !

"

• "

4

"

5

" "

5

10

• $
• !

"

• !

"

• "

4

"

5

" "

5

• $

$

A B C D E

A A B B C C D D B E

B

#$% #$% #$% #$% #$%

11

• $

$

A B C D E

A A B B C C D D B E

B

#$% #$% #$% #$% #$%

• $

$

A B C D E

A A B C C D D E

B

&'$ ("$ %% ")' $"""%

#$% #$% #$% #$% #$%

12

• $

*$% ++)&""!!#)

,## "$%

• "'-.

#))#)$%/ "'-"0*$)%*$)% "

• $

"$%

Θ126 ')# ,#1

('#1 '#6 θx|u

*-Σ( θx|u

13

• $

$

A B C D E F

A A B B C C D D E E F

A B C

ABC ABC ABC ABC ABC ABC ABC ABC

A B C

C C

.27 .39

(

• $

$

A B C D E F

A A B B C C D D E E F

A B C

ABC ABC ABC ABC ABC ABC ABC ABC

&'$ ("$%% ")'$ ("$%% $"""%

A B C

C C

.27 .39

(

$

/")& )&"( ) ,3/.*

14

• $

4$%

,## "$% '-#)($% ,#))5678))56 "'-. )$ #)$%/

"'-"04$$%4$$%

))56'-" "

1

• $

$ $ $

• 7
• ))

7& %)

15

• $

$ $ $

• 7
• ))

7& %)

• $

$ $ $

• 7
• ))

7& %)

*$# !% $'$ 8($$

16

• $

$$ $

• 7
• ))

)

• $

$$

• $

$$ $

• 7
• ))

)

• $

$$

17

• $

$$ $

• 7
• ))

)

• *9

$ $$

• $

$$

*$$# !% '$#8($$

• $

:

• 7
• ))

X C X B X A X C B A

• ¬
• ¬

∧ ¬ ∧ ¬

$ $; 96; 9%! 9$<" $$ 9 $$!$"$" 9

18

CSI Summary

Exploit local structure

Context-specific independence Determinism

Significantly speed-up inference

Tackle problems with tree-width in the thousands

Acknowledgements

Recursive conditioning slides courtesy of Adnan

Darwiche

Implementation available:

http://reasoning.cs.ucla.edu/ace