[PPT] - @ Leuven H. Blockeel, J. Davis, L. De Raedt, D. Fierens, W. Meert, PowerPoint Presentation

SLIDE 1

Recent advances in lifted inference @ Leuven

H. Blockeel, J. Davis, L. De Raedt, D. Fierens,
W. Meert, N. Taghipour, G. Van den Broeck

SML, April 19, 2012

SLIDE 2

Outline

Introduction to lifted inference
Four contributions
Arbitrary constraints
Completeness results
Conditioning
An approximate method

1

SLIDE 3

Lifted inference Exact Approximate Variable Elimination (2003) Belief propagation (2008) Knowledge compilation (2011) … …

2

and many more !

SLIDE 4

MLN 1.5 Attends(person) → Series 1.2 Topic → Attends(person)

3

SLIDE 5

MLN

Series Attends(p1) … Attends(p2) Attends(pN) … Topic

1.5 Attends(person) → Series 1.2 Topic → Attends(person)

3

SLIDE 6

MLN

Series Attends(p1) … Attends(p2) Attends(pN) … Topic ϕ1 ϕ1 ϕ1 ϕ1 ϕ2 ϕ2 ϕ2 ϕ2

1.5 Attends(person) → Series 1.2 Topic → Attends(person)

3

SLIDE 7

MLN

Series Attends(p1) … Attends(p2) Attends(pN) … Topic ϕ1 ϕ1 ϕ1 ϕ1 ϕ2 ϕ2 ϕ2 ϕ2

1.5 Attends(person) → Series 1.2 Topic → Attends(person)

A1 T ϕ2(A1,T) false false true true true false true false 3.3 3.3 1.0 3.3

3

SLIDE 8

MLN

Series Attends(p1) … Attends(p2) Attends(pN) … Topic ϕ1 ϕ1 ϕ1 ϕ1 ϕ2 ϕ2 ϕ2 ϕ2

1.5 Attends(person) → Series 1.2 Topic → Attends(person)

A1 T ϕ2(A1,T) false false true true true false true false 3.3 3.3 1.0 3.3 AN T ϕ2(AN,T) false false true true true false true false 3.3 3.3 1.0 3.3

SLIDE 9

Series Attends(p1) … Attends(p2) Attends(pN) … Topic ϕ1 ϕ1 ϕ1 ϕ1 ϕ2 ϕ2 ϕ2 ϕ2

4

SLIDE 10

Series Attends(p1) … Attends(p2) Attends(pN) … Topic ϕ1 ϕ1 ϕ1 ϕ1 ϕ2 ϕ2 ϕ2 ϕ2

 

 



N i i N i i N

A T S A Z T A A S P

1 2 1 1 1

) , ( ) , ( 1 ) , ,..., , (  

4

SLIDE 11

Series Attends(p1) … Attends(p2) Attends(pN) … Topic ϕ1 ϕ1 ϕ1 ϕ1 ϕ2 ϕ2 ϕ2 ϕ2

   

 



T A A N i i N i i

N

A T S A Z S P

1

1 2 1 1

) , ( ) , ( ... 1 ) (  

4

will it become

a series ?

SLIDE 12

Series Attends(p1) … Attends(p2) Attends(pN) … Topic ϕ1 ϕ1 ϕ1 ϕ1 ϕ2 ϕ2 ϕ2 ϕ2

   

 



T A A N i i N i i

N

A T S A Z S P

1

1 2 1 1

) , ( ) , ( ... 1 ) (  

2(N+1) terms

4

will it become

a series ?

SLIDE 13

Series Attends(p1) … Attends(p2) Attends(pN) … Topic ϕ1 ϕ1 ϕ1 ϕ1 ϕ2 ϕ2 ϕ2 ϕ2

   

  T A A N i i N i i

N

A T S A

1

1 2 1 1

) , ( ) , ( ...  

4

2(N+1) terms

SLIDE 14

Series Attends(p1) … Attends(p2) Attends(pN) … Topic ϕ1 ϕ1 ϕ1 ϕ1 ϕ2 ϕ2 ϕ2 ϕ2

               

  

N

A N N T A

A T S A A T S A ) , ( ) , ( ... ) , ( ) , (

2 1 1 2 1 1

1

   

1 for every person

4

SLIDE 15

Series Attends(p1) … Attends(p2) Attends(pN) … Topic ϕ1 ϕ1 ϕ1 ϕ1 ϕ2 ϕ2 ϕ2 ϕ2

               

  

N

A N N T A

A T S A A T S A ) , ( ) , ( ... ) , ( ) , (

2 1 1 2 1 1

1

   

N times the same product !

4

SLIDE 16

Series Attends(p1) … Attends(p2) Attends(pN) … Topic ϕ1 ϕ1 ϕ1 ϕ1 ϕ2 ϕ2 ϕ2 ϕ2

               

  

N

A N N T A

A T S A A T S A ) , ( ) , ( ... ) , ( ) , (

2 1 1 2 1 1

1

   

N times the same sum !

4

SLIDE 17

Series Attends(p1) … Attends(p2) Attends(pN) … Topic ϕ1 ϕ1 ϕ1 ϕ1 ϕ2 ϕ2 ϕ2 ϕ2

 

     

T N A

A T S A ) , ( ) , (

2 1

 

lifted:

4

compute only once !

SLIDE 18

Series Attends(p1) … Attends(p2) Attends(pN) … Topic ϕ1 ϕ1 ϕ1 ϕ1 ϕ2 ϕ2 ϕ2 ϕ2 “lifted multiplication”

 

     

T N A

A T S A ) , ( ) , (

2 1

 

lifted:

“lifted sum-out”

4

SLIDE 19

Lifted Variable Elimination

[Poole ’03,…]

Repeatedly apply certain operators on

the model

Lifted multiplication
Lifted sum-out
…
Until the desired result is found

5

SLIDE 20

Lifted Knowledge Compilation

[Van den Broeck et al ‘11,…]

8

Compile the model into a “lifted” circuit

(“FO d-DNNF”)

How? Compilation rules
Inference = traversing the circuit
Time = poly(domain size)

SLIDE 21

Outline

Introduction to lifted inference
Four contributions
Arbitrary constraints
Completeness results
Conditioning
An approximate method

9

SLIDE 22

S A(p1) T … A(pN/2) A(pN/2+1) A(pN) …

10

SLIDE 23

S A(p1) T … A(pN/2) A(pN/2+1) A(pN) …

10

SLIDE 24

S A(p1) T … A(pN/2) A(pN/2+1) A(pN) …

10

SLIDE 25

S A(p1) T … A(pN/2) A(pN/2+1) A(pN) …

10

SLIDE 26

S A(p1) T … A(pN/2) A(pN/2+1) A(pN) …

10

SLIDE 27

S A(p1) T … A(pN/2) A(pN/2+1) A(pN) …

10

SLIDE 28

S A(p1) T … A(pN/2) A(pN/2+1) A(pN) …

Bigger groups = more lifting !

10

SLIDE 29

S A(p1) T … A(pN/2) A(pN/2+1) A(pN) …

Bigger groups = more lifting ! The groups are specified by constraints

10

SLIDE 30

11

Importance of constraints

[Taghipour et al, AISTATS'12]

Exact lifted algorithms use a particular

constraint language group → constraint →

Often leads to unnecessarily small groups

→ less lifting can it be expressed in the language ?

SLIDE 31

11

Importance of constraints

[Taghipour et al, AISTATS'12]

Exact lifted algorithms use a particular

constraint language group → constraint →

Often leads to unnecessarily small groups

→ less lifting

We avoid using a particular constraint language

Instead: arbitrary constraints + relational algebra can it be expressed in the language ?

SLIDE 32

12

more evidence arbitrary constraints pairwise constraints (C-FOVE) runtime (log)

SLIDE 33

Outline

Introduction to lifted inference
Four contributions
Arbitrary constraints
Completeness results
Conditioning
An approximate method

13

SLIDE 34

Outline

Introduction to lifted inference
Four contributions
Arbitrary constraints
Completeness result
Conditioning
Approximate inference

SLIDE 35

Outline

Introduction to lifted inference
Four contributions
Arbitrary constraints
Completeness result
Conditioning
Approximate inference

SLIDE 36

What is Lifted Inference?

Propositional inference is intractable

Solution: lifted inference

“Exploit symmetries” “Reason at first-order level” “Reason about groups of objects as a whole” “Avoid repeated computations” “Mimic resolution in theorem proving”

There is a common understanding but

no formal definition of lifted inference!

SLIDE 37

What is Lifted Inference?

What is commonly understood as

exact lifted inference? Definition: Domain-Lifted Inference Complexity of computing P(q|e) in model m is polynomial time in the domain sizes of the logical variables in q,e,m

[Van den Broeck NIPS11]

1.5 Attends(person) → Series 1.2 Topic → Attends(person)

SLIDE 38

What is Lifted Inference?

What is commonly understood as

exact lifted inference? Definition: Domain-Lifted Inference Complexity of computing P(q|e) in model m is polynomial time in the domain sizes of the logical variables in q,e,m

Possibly exponential in the size of q,e,m

# predicates, # parfactors, # atoms, # arguments, # formulas, # constants in model

[Van den Broeck NIPS11]

SLIDE 39

What is Lifted Inference?

Motivation: Large domains lead to intractable

propositional inference.

A formal framework for lifted inference
Definition + complexity considerations
~ PAC-learnability (Valiant)
Other notions, e.g., for approximate inference.

[Van den Broeck NIPS11]

SLIDE 40

Completeness

A procedure that is domain-lifted for all models

in a class M is called complete for M All models in M are “liftable”

There was no completeness result

for existing algorithms If you give me a model, I cannot say if grounding will be needed, untill I run the inference algorithm itself.

[Van den Broeck NIPS11]

SLIDE 41

Completeness Result

Probabilistic inference in models with

universal quantifiers ∀ and
2 logical variables per formula

is domain-liftable.

A non-trivial class of models
First completeness results in exact lifted inference
Lifted knowledge compilation procedure
Lifted variable elimination procedure

[Van den Broeck NIPS11], [Taghipour et al.]

SLIDE 42

Completeness Game

Expressivity

FOL , ∀ =, 2 variables [Van den Broeck 11] FOL , ,= ∀ ∃

[Jaeger 99]

... [Jaeger 12]

?

Complete domain-lifted inference procedure No domain-lifted inference procedure exists

SLIDE 43

Outline

Introduction to lifted inference
Four contributions
Arbitrary constraints
Completeness result
Conditioning
Approximate inference

SLIDE 44

Conditioning

Task: Probability of query q given evidence e: P(q|e)

Domain-lifted inference is exponential in the size of e.

Can we compute conditional probabilities efficiently?

Depends on the arity of literals conditioned on:

Positive and negative result for lifted inference

[Van den Broeck, Davis AAAI12]

SLIDE 45

Outline

Introduction to lifted inference
Four contributions
Arbitrary constraints
Completeness result
Conditioning
Approximate inference

SLIDE 46

Lifted RCR

Practical usefulness of lifted inference shown

for approximate inference with lifted BP

Lifted Relax, Compensate and Recover

(1) Clone all atoms in a model (2) Relax equivalences between clones (3) Compensate for removed equivalences (4) Recover equivalences until model too complex

Exact lifted inference black box in (3)

[Van den Broeck, Choi, Darwiche]

SLIDE 47

Lifted RCR

Special case: Lifted BP Tractable Exact lifted inference Intractable

[Van den Broeck, Choi, Darwiche]

Approximation Error

SLIDE 48

Outline

Introduction to lifted inference
Four contributions
Arbitrary constraints
Completeness result
Conditioning
Approximate inference

SLIDE 49

Posters!

Website & Implementation: http://dtai.cs.kuleuven.be/ml/systems/wfomc