Global Structure: Treewidth w ( exp( )) O n w 1 Local - - PDF document

▶
global structure treewidth w
SMART_READER_LITE
LIVE PREVIEW

Global Structure: Treewidth w ( exp( )) O n w 1 Local - - PDF document

Apr 08, 2024 650 likes •838 views

Readings: K&F: 4.1, 4.2, 4.3, 4.4, 8.4, 8.5, 8.6 Recursive Conditioning, Adnan Darwiche. In Artificial Intelligence Journal, 125:1, pp. 5-41 Context-specific independence Graphical Models 10708 Carlos Guestrin Carnegie Mellon

slide-1
SLIDE 1

1

Context-specific independence

Graphical Models – 10708 Carlos Guestrin Carnegie Mellon University October 16th, 2006

Readings: K&F: 4.1, 4.2, 4.3, 4.4, 8.4, 8.5, 8.6 “Recursive Conditioning”, Adnan Darwiche. In Artificial Intelligence Journal, 125:1, pp. 5-41

Global Structure: Treewidth w

)) exp( ( w n O

slide-2
SLIDE 2

2

Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Fuel Pump Fuel Line Distributor Spark Plugs Engine Start

Local Structure 1: Context specific indepencence

Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Fuel Pump Fuel Line Distributor Spark Plugs Engine Start

Context Specific I ndependence (CSI ) After observing a variable, some vars become independent

Local Structure 1: Context specific indepencence

slide-3
SLIDE 3

3

CSI example: Tree CPD

Represent P(Xi|PaXi) using a decision tree Path to leaf is an assignment to (a subset

  • f) PaXi

Leaves are distributions over Xi given

assignment of PaXi on path to leaf

Interpretation of leaf: For specific assignment of PaXi on path to

this leaf – Xi is independent of other parents

Representation can be exponentially

smaller than equivalent table

Apply SAT Letter Job

Tabular VE with Tree CPDs

If we turn a tree CPD into table “Sparsity” lost! Need inference approach that deals with

tree CPD directly!

slide-4
SLIDE 4

4

Local Structure 2: Determinism

Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Fuel Pump Fuel Line Distributor Spark Plugs Engine Start

ON OFF OK WEAK DEAD

Lights Battery Power

.99 .01 .20 .80 1

I f Battery Power = Dead, then Lights = OFF

Determinism

Determinism and inference

Determinism gives a little

sparsity in table, but much bigger impact on inference

Multiplying deterministic factor

with other factor introduces many new zeros

Operations related to theorem

proving, e.g., unit resolution

ON OFF OK WEAK DEAD

Lights Battery Power

.99 .01 .20 .80 1

slide-5
SLIDE 5

5

Today’s Models …

Often characterized by:

Richness in local structure (determinism, CSI) Massiveness in size (10,000’s variables) High connectivity (treewidth)

Enabled by:

High level modeling tools: relational, first order Advances in machine learning New application areas (synthesis):

Bioinformatics (e.g. linkage analysis) Sensor networks

Exploiting local structure a must!

Exact inference in large models is possible…

BN from a relational model

slide-6
SLIDE 6

6

Recursive Conditioning

Treewidth complexity (worst case) Better than treewidth complexity with local

structure

Provides a framework for time-space tradeoffs Only quick intuition today, details in readings

  • A. Darwiche

The Computational Power

  • f Assumptions

Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start

slide-7
SLIDE 7

7

  • A. Darwiche

Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start

The Computational Power

  • f Assumptions
  • A. Darwiche

Decomposition

Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge

Gas

Leak Fuel Line Distributor Spark Plugs Engine Start

slide-8
SLIDE 8

8

  • A. Darwiche

Case Analysis

Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start

p p

+

  • A. Darwiche

Case Analysis

Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start

pl p

+

pr

*

slide-9
SLIDE 9

9

  • A. Darwiche

Case Analysis

Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start

pl

+

pr

*

pl pr

*

  • A. Darwiche

Case Analysis

Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start

pl

+

pr

*

pl pr

*

slide-10
SLIDE 10

10

  • A. Darwiche

Case Analysis

Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start

pl

+

pr

*

pl pr

*

  • A. Darwiche

Decomposition Tree

A B C D E

A A B B C C D D B E

B

f(A) f(A,B) f(B,C) f(C,D) f(B,D,E) Cutset

slide-11
SLIDE 11

11

  • A. Darwiche

Decomposition Tree

A B C D E

A A B B C C D D B E

B

f(A) f(A,B) f(B,C) f(C,D) f(B,D,E) Cutset

  • A. Darwiche

Decomposition Tree

A B C D E

A A B C C D D E

B

Time: O(n exp(w log n)) Space: Linear (using appropriate dtree)

f(A) f(A,B) f(B,C) f(C,D) f(B,D,E) Cutset

slide-12
SLIDE 12

12

  • A. Darwiche

RC1(T,e) // compute probability of evidence e on dtree T

If T is a leaf node Return Lookup(T,e) Else

p := 0 for each instantiation c of cutset(T)-E do p := p + RC1(Tl,ec) RC1(Tr,ec) return p

RC1

  • A. Darwiche

Lookup(T,e)

ΘX|U : CPT associated with leaf T

If X is instantiated in e, then

x: value of X in e

u: value of U in e

Return θx|u

Else return 1 = Σx θx|u

slide-13
SLIDE 13

13

  • A. Darwiche

Caching

A B C D E F

A A B B C C D D E E F

A B C

ABC ABC ABC ABC ABC ABC ABC ABC

A B C

C C

.27 .39

Context

  • A. Darwiche

Caching

A B C D E F

A A B B C C D D E E F

A B C

ABC ABC ABC ABC ABC ABC ABC ABC

Time: O(n exp(w)) Space: O(n exp(w)) (using appropriate dtree)

A B C

C C

.27 .39

Context

Recursive Conditioning

An any-space algorithm with treewidth complexity Darwiche AIJ-01

slide-14
SLIDE 14

14

  • A. Darwiche

RC2(T,e)

If T is a leaf node, return Lookup(T,e)

y := instantiation of context(T)

If cacheT[y] < > nil, return cacheT[y] p := 0 For each instantiation c of cutset(T)-E do

p := p + RC2(Tl,ec) RC2(Tr,ec)

cacheT[y] := p Return p

RC2

  • A. Darwiche

Decomposition with Local Structure

B C X A

A, B, C

X I ndependent of B, C given A

slide-15
SLIDE 15

15

  • A. Darwiche

Decomposition with Local Structure

B C X A

A, B, C

X I ndependent of B, C given A

  • A. Darwiche

Decomposition with Local Structure

B C X A

A, B, C

X I ndependent of B, C given A

No need to consider an exponential number of cases (in the cutset size) given local structure

slide-16
SLIDE 16

16

  • A. Darwiche

Caching with Local Structure

B C X A

A,B,C B,C A

C B A C B A C B A C B A C B A C B A C B A C B A

Structural cache

  • A. Darwiche

Caching with Local Structure

B C X A

A,B,C B,C A

C B A C B A C B A C B A C B A C B A C B A C B A

Structural cache

slide-17
SLIDE 17

17

  • A. Darwiche

Caching with Local Structure

B C X A

A,B,C B,C A

C B A C B A C B A C B A A

Non- Structural cache

C B A C B A C B A C B A C B A C B A C B A C B A

Structural cache

No need to cache an exponential number of results (in the context size) given local structure

  • A. Darwiche

Determinism…

B C X A

A, B, C X C X B X A X C B A ⇒ ⇒ ⇒ ¬ ⇒ ¬ ∧ ¬ ∧ ¬

A natural setup to incorporate SAT technology:

  • Unit resolution to:
  • Derive values of variables
  • Detect/ skip inconsistent cases
  • Dependency directed backtracking
  • Clause learning
slide-18
SLIDE 18

18

CSI Summary

Exploit local structure

Context-specific independence Determinism

Significantly speed-up inference

Tackle problems with tree-width in the thousands

Acknowledgements

Recursive conditioning slides courtesy of Adnan

Darwiche

Implementation available:

http://reasoning.cs.ucla.edu/ace