Lifted Probabilistic Inference by First-Order Knowledge Compilation - - PowerPoint PPT Presentation

▶
lifted probabilistic inference by first order knowledge
SMART_READER_LITE
LIVE PREVIEW

Lifted Probabilistic Inference by First-Order Knowledge Compilation - - PowerPoint PPT Presentation

May 09, 2023 138 likes •950 views

Lifted Probabilistic Inference by First-Order Knowledge Compilation Guy Van den Broeck Nima Taghipour Wannes Meert Jesse Davis Luc De Raedt Lifted Inference in Probabilistic Logical Models - Tutorial - IJCAI11 18/07/11 Outline Overview

slide-1
SLIDE 1

Lifted Probabilistic Inference by First-Order Knowledge Compilation

Guy Van den Broeck Nima Taghipour Wannes Meert Jesse Davis Luc De Raedt

Lifted Inference in Probabilistic Logical Models - Tutorial - IJCAI11 18/07/11

slide-2
SLIDE 2

18/07/11 First-Order Knowledge Compilation 2

Outline

  • Overview Approach
  • First-Order d-DNNF Circuits
  • First-Order Knowledge Compilation
  • Experiments
  • Conclusions
slide-3
SLIDE 3

18/07/11 First-Order Knowledge Compilation 3

Outline

  • Overview Approach
  • First-Order d-DNNF Circuits
  • First-Order Knowledge Compilation
  • Experiments
  • Conclusions
slide-4
SLIDE 4

18/07/11 First-Order Knowledge Compilation 4

Context

Variable Elimination Belief Propagation Knowledge Compilation Ground [Zhang94] [Pearl82] [Darwiche03] Lifted [Poole03] [Singla08] Our approach

slide-5
SLIDE 5

18/07/11 First-Order Knowledge Compilation 5

Advantages of Knowledge Compilation

  • Compile once, then run polytime inference for

multiple queries and evidence

  • Efficient data structures
  • Principled logical approach
  • Exploits context-specific independences
  • State of the art for exact inference in
  • Bayesian networks
  • Statistical relational learning
  • Used in many domains, not just probabilistic

reasoning

slide-6
SLIDE 6

18/07/11 First-Order Knowledge Compilation 6

Question?

  • Can we lift knowledge compilation to a first-order

setting?

  • First step taken: first-order d-DNNFs for
  • weighted first-order model counting
  • lifted probabilistic inference
  • Many open questions remaining!
slide-7
SLIDE 7

18/07/11 First-Order Knowledge Compilation 7

What is Lifted Inference?

slide-8
SLIDE 8

18/07/11 First-Order Knowledge Compilation 8

What is Lifted Inference?

  • Variables X,Y range over

domain People

  • Represents propositional

model for given domain (50 people)

  • Propositional inference in

factor graph is expensive

  • However: symmetries
slide-9
SLIDE 9

18/07/11 First-Order Knowledge Compilation 9

What is Lifted Inference?

  • We compile to a circuit

independent of |People|

  • Inference linear

in |People|

→ Lifted Inference

slide-10
SLIDE 10

18/07/11 First-Order Knowledge Compilation 10

Knowledge Compilation

Factor Graph MLN ... Bayesian Network

slide-11
SLIDE 11

18/07/11 First-Order Knowledge Compilation 11

Knowledge Compilation

Weighted CNF Factor Graph MLN ...

  • Step : Convert model to weighted CNF

Bayesian Network 1 1

slide-12
SLIDE 12

18/07/11 First-Order Knowledge Compilation 12

Knowledge Compilation

Weighted CNF d-DNNF Circuit Factor Graph MLN ...

  • Step : Convert model to weighted CNF
  • Step : Convert CNF to d-DNNF circuit

Bayesian Network 1 1 2 2

slide-13
SLIDE 13

18/07/11 First-Order Knowledge Compilation 13

Knowledge Compilation

Weighted CNF d-DNNF Circuit Weighted Model Counting Factor Graph MLN ...

  • Step : Convert model to weighted CNF
  • Step : Convert CNF to d-DNNF circuit
  • Step : Perform weighted model counting

Bayesian Network 1 1 2 3 2 3

slide-14
SLIDE 14

18/07/11 First-Order Knowledge Compilation 14

Our Approach: First-Order Knowledge Compilation:

Weighted FO CNF FO d-DNNF Circuit Weighted FO Model Counting Parfactor Graph MLN ... 2 3

  • Step : Convert model to weighted FO CNF
  • Step : Convert CNF to FO d-DNNF circuit
  • Step : Perform weighted FO model counting

1 2 3 1

slide-15
SLIDE 15

18/07/11 First-Order Knowledge Compilation 15

Step 1: Converting to Weighted FO CNF

MLN

slide-16
SLIDE 16

18/07/11 First-Order Knowledge Compilation 16

Step 1: Converting to Weighted FO CNF

Weighted FO Theory MLN

slide-17
SLIDE 17

18/07/11 First-Order Knowledge Compilation 17

Step 1: Converting to Weighted FO CNF

Weighted FO Theory MLN Weighted FO CNF

slide-18
SLIDE 18

18/07/11 First-Order Knowledge Compilation 18

Step 3: Weighted FO Model Counting

  • Weight function on ground atoms
slide-19
SLIDE 19

18/07/11 First-Order Knowledge Compilation 19

Step 3: Weighted FO Model Counting

  • Weight function on ground atoms
  • Weight of a model (possible world)
slide-20
SLIDE 20

18/07/11 First-Order Knowledge Compilation 20

Step 3: Weighted FO Model Counting

  • Weight function on ground atoms
  • Weight of a model (possible world)
  • Weight of all models is
slide-21
SLIDE 21

18/07/11 First-Order Knowledge Compilation 21

Step 3: Weighted FO Model Counting

  • Weight function on ground atoms
  • Weight of a model (possible world)
  • Weight of all models is

Weight of models where Alice smokes is

slide-22
SLIDE 22

18/07/11 First-Order Knowledge Compilation 22

Step 3: Weighted FO Model Counting

  • Weight function on ground atoms
  • Weight of a model (possible world)
  • Weight of all models is

Weight of models where Alice smokes is

slide-23
SLIDE 23

18/07/11 First-Order Knowledge Compilation 23

Outline

  • Overview Approach
  • First-Order d-DNNF Circuits
  • First-Order Knowledge Compilation
  • Experiments
  • Conclusions
slide-24
SLIDE 24

18/07/11 First-Order Knowledge Compilation 24

Propositional d-DNNF Circuits

[Darwiche and Marquis, 2002]

Logical theory:

slide-25
SLIDE 25

18/07/11 First-Order Knowledge Compilation 25

Propositional d-DNNF Circuits

[Darwiche and Marquis, 2002]

Logical theory:

Literal (leaf)

slide-26
SLIDE 26

18/07/11 First-Order Knowledge Compilation 26

Propositional d-DNNF Circuits

[Darwiche and Marquis, 2002]

Logical theory:

Literal (leaf)

Logical operators (inner node)

slide-27
SLIDE 27

18/07/11 First-Order Knowledge Compilation 27

Propositional d-DNNF Circuits

[Darwiche and Marquis, 2002]

Logical theory:

Literal (leaf)

Deterministic disjunction Logical operators (inner node)

slide-28
SLIDE 28

18/07/11 First-Order Knowledge Compilation 28

Propositional d-DNNF Circuits

[Darwiche and Marquis, 2002]

Logical theory:

Literal (leaf)

Deterministic disjunction Decomposable conjunction Logical operators (inner node)

slide-29
SLIDE 29

18/07/11 First-Order Knowledge Compilation 29

First-Order d-DNNF Circuits

Logical Theory:

slide-30
SLIDE 30

18/07/11 First-Order Knowledge Compilation 30

First-Order d-DNNF Circuits

First-Order Literal (leaf) Logical Theory:

slide-31
SLIDE 31

18/07/11 First-Order Knowledge Compilation 31

First-Order d-DNNF Circuits

  • Deterministic disjunction
  • Decomposable conjunction
  • 3 additional first-order operators (inner nodes)

First-Order Literal (leaf) Logical Theory:

slide-32
SLIDE 32

18/07/11 First-Order Knowledge Compilation 32

Outline

  • Overview Approach
  • First-Order d-DNNF Circuits
  • First-Order Knowledge Compilation
  • Experiments
  • Conclusions
slide-33
SLIDE 33

18/07/11 First-Order Knowledge Compilation 33

Step 2: Our Compilation Algorithm

  • Recursively apply
  • Unit Propagation
  • Independence
  • Inclusion-Exclusion (Shannon Decomposition)
  • Shattering
  • Independent Partial Grounding
  • Atom Counting
  • (Grounding)

Weighted FO CNF FO d-DNNF Circuit

slide-34
SLIDE 34

18/07/11 First-Order Knowledge Compilation 34

Step 2: Our Compilation Algorithm

  • Recursively apply
  • Unit Propagation
  • Independence
  • Inclusion-Exclusion (Shannon Decomposition)
  • Shattering
  • Independent Partial Grounding
  • Atom Counting
  • (Grounding)

Weighted FO CNF FO d-DNNF Circuit

Generate first-order

  • perators in inner nodes
slide-35
SLIDE 35

18/07/11 First-Order Knowledge Compilation 35

Step 2: Our Compilation Algorithm

  • Recursively apply
  • Unit Propagation
  • Independence
  • Inclusion-Exclusion (Shannon Decomposition)
  • Shattering
  • Independent Partial Grounding
  • Atom Counting
  • (Grounding)

Weighted FO CNF FO d-DNNF Circuit

slide-36
SLIDE 36

18/07/11 First-Order Knowledge Compilation 36

Unit Propagation

slide-37
SLIDE 37

18/07/11 First-Order Knowledge Compilation 37

Unit Propagation

Unit clause

slide-38
SLIDE 38

18/07/11 First-Order Knowledge Compilation 38

Unit Propagation

Unit clause

slide-39
SLIDE 39

18/07/11 First-Order Knowledge Compilation 39

Unit Propagation

Clauses split w.r.t. unit clause atom 'residuals' → independent

slide-40
SLIDE 40

18/07/11 First-Order Knowledge Compilation 40

Unit Propagation

Resolvent of unit and 2nd clause

slide-41
SLIDE 41

18/07/11 First-Order Knowledge Compilation 41

Step 2: Our Compilation Algorithm

  • Recursively apply
  • Unit Propagation
  • Independence
  • Inclusion-Exclusion (Shannon Decomposition)
  • Shattering
  • Independent Partial Grounding
  • Atom Counting
  • (Grounding)

Weighted FO CNF FO d-DNNF Circuit

slide-42
SLIDE 42

18/07/11 First-Order Knowledge Compilation 42

Atom Counting

slide-43
SLIDE 43

18/07/11 First-Order Knowledge Compilation 43

Atom Counting

Atom with one logical variable X {luc,jesse} ∈

slide-44
SLIDE 44

18/07/11 First-Order Knowledge Compilation 44

Atom Counting

Atom with 1 logical variable

slide-45
SLIDE 45

18/07/11 First-Order Knowledge Compilation 45

Atom Counting

All partial interpretations for fun(X)

  • deterministic
  • 2|People|
slide-46
SLIDE 46

18/07/11 First-Order Knowledge Compilation 46

Atom Counting

Same weighted model count

slide-47
SLIDE 47

18/07/11 First-Order Knowledge Compilation 47

Atom Counting

2|People| → |People|+1

slide-48
SLIDE 48

18/07/11 First-Order Knowledge Compilation 48

Atom Counting

Isomorphic circuits

slide-49
SLIDE 49

18/07/11 First-Order Knowledge Compilation 49

Atom Counting

|People|+1 → 1

slide-50
SLIDE 50

18/07/11 First-Order Knowledge Compilation 50

Atom Counting

slide-51
SLIDE 51

18/07/11 First-Order Knowledge Compilation 51

Outline

  • Overview Approach
  • First-Order d-DNNF Circuits
  • First-Order Knowledge Compilation
  • Experiments
  • Conclusions
slide-52
SLIDE 52

18/07/11 First-Order Knowledge Compilation 52

Evaluated Models

  • Sick Death [de Salvo Braz 2005]
  • WebKB [Lowd 2007]
  • Competing Workshops [Milch 2008]
  • Workshop Attributes [Milch 2008]
  • Friends Smoker [Singla 2008]
  • Friends Smoker Drinker
slide-53
SLIDE 53

18/07/11 First-Order Knowledge Compilation 53

Competing Workshops [Milch 2008]

Probabilistic Model:

slide-54
SLIDE 54

18/07/11 First-Order Knowledge Compilation 54

Competing Workshops [Milch 2008]

slide-55
SLIDE 55

18/07/11 First-Order Knowledge Compilation 55

slide-56
SLIDE 56

18/07/11 First-Order Knowledge Compilation 56

Friends Smoker [Singla 2008]

Probabilistic Model:

slide-57
SLIDE 57

18/07/11 First-Order Knowledge Compilation 57

Friends Smoker [Singla 2008]

slide-58
SLIDE 58

18/07/11 First-Order Knowledge Compilation 58

Friends Smoker Drinker

New Probabilistic Model:

slide-59
SLIDE 59

18/07/11 First-Order Knowledge Compilation 59

Friends Smoker Drinker

slide-60
SLIDE 60

18/07/11 First-Order Knowledge Compilation 60

Outline

  • Overview Approach
  • First-Order d-DNNF Circuits
  • First-Order Knowledge Compilation
  • Experiments
  • Conclusions
slide-61
SLIDE 61

18/07/11 First-Order Knowledge Compilation 61

Benefits of First-Order Knowledge Compilation?

  • Compile once for a given set of evidence then run

polytime inference

  • Efficient data structure
  • Principled logical approach
  • First model theoretic approach to lifted probabilistic

inference

  • Uses concepts from logical inference: model counting, unit

propagation, Shannon decomposition, etc.

  • Exploits context-specific independences
  • State of the art for exact lifted inference
  • Lifts more models than C-FOVE
slide-62
SLIDE 62

18/07/11 First-Order Knowledge Compilation 62

Contributions

  • We introduced first-order ...
  • knowledge compilation
  • d-DNNF circuits
  • weighted model counting
  • smoothing
  • Algorithm to compile a first-order probabilistic model into

FO d-DNNF circuits

  • Closer to understanding the connection between lifted

inference in first-order logic (resolution) and lifted inference in graphical models

slide-63
SLIDE 63

18/07/11 First-Order Knowledge Compilation 63

Advertisement

  • Poster

Wednesday 10:30 UAI session

  • Talk

Thursday 10:30 UAI session

slide-64
SLIDE 64

18/07/11 First-Order Knowledge Compilation 64

Thanks

slide-65
SLIDE 65

18/07/11 First-Order Knowledge Compilation 65

Extra Slides

slide-66
SLIDE 66

18/07/11 First-Order Knowledge Compilation 66

Logic-based Probabilistic Inference

Bayesian Network Weighted Propositional CNF Weighted Model Counting Factor Graph MLN ProbLog

  • DPLL Search Weighted Model Counting

[Sang 2005]

  • Knowledge Compilation [Darwiche]
slide-67
SLIDE 67

18/07/11 First-Order Knowledge Compilation 67

Additional Operator Nodes

FO d-DNNF Circuit

  • Deterministic set-disjunction
  • Decomposable set-conjunction
  • Inclusion-Exclusion

(non-deterministic disjunction)

slide-68
SLIDE 68

18/07/11 First-Order Knowledge Compilation 68

Auxiliary Operations

  • Splitting w.r.t. an atom [Poole 2003]:
  • Similar to splitting in (C-)FOVE,

but with domain constraints

  • Shattering [de Salvo Braz 2005]

Splitting w.r.t. any atom in theory, until convergence

Before Splitting Atom After Splitting

slide-69
SLIDE 69

18/07/11 First-Order Knowledge Compilation 69

Unit Propagation

slide-70
SLIDE 70

18/07/11 First-Order Knowledge Compilation 70

Independence

  • Set of clauses independent from rest
  • Independence when no unifying atoms
slide-71
SLIDE 71

18/07/11 First-Order Knowledge Compilation 71

Inclusion-Exclusion

  • Clause has set of literals that share no logical

variables with rest

  • Non-deterministic disjunction & intersection
slide-72
SLIDE 72

18/07/11 First-Order Knowledge Compilation 72

Inclusion-Exclusion

  • Clause has set of literals that share no logical

variables with rest

  • Non-deterministic disjunction & intersection
slide-73
SLIDE 73

18/07/11 First-Order Knowledge Compilation 73

Independent Partial Grounding

  • Single logical variable in every atom (position!)
  • Different partial groundings are independent
slide-74
SLIDE 74

18/07/11 First-Order Knowledge Compilation 74

Independent Partial Grounding

  • Single logical variable in every atom (position!)
  • Different partial groundings are independent
slide-75
SLIDE 75

18/07/11 First-Order Knowledge Compilation 75

Special Inclusion Exclusion Case: Shannon Decomposition

  • CNF has ground atom
  • Ground atoms do not share

logical variables

  • We can always add

clause

  • Intersection is

unsatisfiable

  • Inclusion-Exclusion

becomes deterministic disjunction

slide-76
SLIDE 76

18/07/11 First-Order Knowledge Compilation 76

First-Order Smoothing

FO d-DNNF

slide-77
SLIDE 77

18/07/11 First-Order Knowledge Compilation 77

First-Order Smoothing

FO d-DNNF Smooth FO d-DNNF

slide-78
SLIDE 78

18/07/11 First-Order Knowledge Compilation 78

First-Order Smoothing

FO d-DNNF Smooth FO d-DNNF Complicated rules for

  • atom counting
  • independent partial groundings
slide-79
SLIDE 79

18/07/11 First-Order Knowledge Compilation 79

Circuit Evaluation

  • Propagate weighted model

count to root node

  • Propagate
  • + for disjunction
  • * for conjunction
  • ...
  • for atom counting
  • Atom counting linear in domain

size, others independent of

slide-80
SLIDE 80

18/07/11 First-Order Knowledge Compilation 80