DTProbLog A Decision-Theoretic Probabilistic Prolog Guy Van den - - PowerPoint PPT Presentation

▶
dtproblog
SMART_READER_LITE
LIVE PREVIEW

DTProbLog A Decision-Theoretic Probabilistic Prolog Guy Van den - - PowerPoint PPT Presentation

Mar 23, 2023 368 likes •814 views

DTProbLog A Decision-Theoretic Probabilistic Prolog Guy Van den Broeck Ingo Thon Martijn van Otterlo Luc De Raedt Motivation Many real-world decision problems are relational and probabilistic Wildfire control Who to vaccinate for

slide-1
SLIDE 1

DTProbLog

A Decision-Theoretic Probabilistic Prolog

Guy Van den Broeck Ingo Thon Martijn van Otterlo Luc De Raedt

slide-2
SLIDE 2

07/14/10 DTProbLog 2

Motivation

  • Many real-world decision problems are relational

and probabilistic

  • Wildfire control
  • Who to vaccinate for swine flu
  • Viral marketing
  • ...
slide-3
SLIDE 3

07/14/10 DTProbLog 3

Motivation

  • Combination of relations, uncertainty and

decision theory largely ignored

some exceptions are MLDNs, DTLPs and FOMDPs

  • Relations with uncertainty

= Statistical Relational Learning

  • ProbLog is a simple probabilistic Prolog
  • DTProbLog is a decision-theoretic ProbLog
slide-4
SLIDE 4

07/14/10 DTProbLog 4

Outline

  • DTProbLog: the Language
  • DTProbLog: the Algorithms
  • Exact Solution Algorithm
  • Approximate Solution Algorithms
  • Experiments: Viral Marketing
  • Related Work & Conclusions
slide-5
SLIDE 5

07/14/10 DTProbLog 5

DTProbLog: the Language

slide-6
SLIDE 6

07/14/10 DTProbLog 6

Homer Marge Bart Lisa Lenny Apu Moe Seymour Ralph Maggie

slide-7
SLIDE 7

07/14/10 DTProbLog 7

Homer 1 Marge Bart Lisa Lenny Apu Moe Seymour Ralph Maggie

slide-8
SLIDE 8

07/14/10 DTProbLog 8

Homer 1 Marge 0.3 Bart 0.3 Lisa Lenny Apu Moe 0.3 Seymour Ralph Maggie

slide-9
SLIDE 9

07/14/10 DTProbLog 9

Homer 1 Marge 0.39 Bart 0.41 Lisa 0.25 Lenny 0.14 Apu 0.17 Moe 0.34 Seymour 0.25 Ralph 0.21 Maggie 0.17

slide-10
SLIDE 10

07/14/10 DTProbLog 10

Homer 1 Marge 0.39 Bart 0.41 Lisa 0.25 Lenny 0.14 Apu 0.17 Moe 0.34 Seymour 0.25 Ralph 0.21 Maggie 0.17

ProbLog

Probabilistic Facts

0.3 :: buy_trust(_,_).

Background Knowledge

buys(X) :- trusts(X,Y), buys(Y), buy_trust(X,Y).

slide-11
SLIDE 11

07/14/10 DTProbLog 11

Homer Marge Bart Lisa Lenny Apu Moe Seymour Ralph Maggie

slide-12
SLIDE 12

07/14/10 DTProbLog 12

Homer 0.20 Marge Bart Lisa Lenny Apu Moe Seymour Ralph Maggie

slide-13
SLIDE 13

07/14/10 DTProbLog 13

Homer 0.20 Marge 0.08 Bart 0.08 Lisa 0.05 Lenny 0.03 Apu 0.03 Moe 0.07 Seymour 0.05 Ralph 0.04 Maggie 0.03

slide-14
SLIDE 14

07/14/10 DTProbLog 14

Homer 0.20 Marge 0.08 Bart 0.08 Lisa 0.05 Lenny 0.03 Apu 0.03 Moe 0.07 Seymour 0.05 Ralph 0.04 Maggie 0.03

Probabilistic Facts

0.3 :: buy_trust(_,_). 0.2 :: buy_marketing(_).

Background Knowledge

buys(X) :- buys(X) :- trusts(X,Y), marketed(X), buys(Y), buy_marketing(X). buy_trust(X,Y).

ProbLog

slide-15
SLIDE 15

07/14/10 DTProbLog 15

Homer Marge Bart Lisa Lenny Apu Moe Seymour Ralph Maggie

slide-16
SLIDE 16

07/14/10 DTProbLog 16

Homer 0.25 Marge 0.11 Bart 0.12 Lisa 0.08 Lenny 0.10 Apu 0.11 Moe 0.25 Seymour 0.12 Ralph 0.08 Maggie 0.05

slide-17
SLIDE 17

07/14/10 DTProbLog 17

Homer Marge Bart Lisa Lenny Apu Moe Seymour Ralph Maggie

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

+$5

  • $3

Which strategy gives the maximum expected utility?

slide-18
SLIDE 18

07/14/10 DTProbLog 18

$ 1 . 1 2

Homer 0.12 Marge 0.13 Bart 0.27 Lisa 0.13 Lenny 0.06 Apu 0.11 Moe 0.12 Seymour 0.27 Ralph 0.16 Maggie 0.07

+$5

  • $3
slide-19
SLIDE 19

07/14/10 DTProbLog 19

$ 1 . 1 2

Homer 0.12 Marge 0.13 Bart 0.27 Lisa 0.13 Lenny 0.06 Apu 0.11 Moe 0.12 Seymour 0.27 Ralph 0.16 Maggie 0.07

Probabilistic Facts

...

Background Knowledge

...

Decisions

? :: marketed(P) :- person(P).

Utility Facts

buys(P) => 5 :- person(P). marketed(P) => -3 :- person(P).

DTProbLog

slide-20
SLIDE 20

07/14/10 DTProbLog 20

DTProbLog: the Algorithms

slide-21
SLIDE 21

07/14/10 DTProbLog 21

DTProbLog: the Algorithms

  • DTProbLog solves decision problems in

complex relational and uncertain environments.

  • Exact solution algorithm
  • Extends ProbLog's BDD-based inference
  • Efficient datastructures: BDD and ADD
  • Approximate algorithms
  • Local search
  • K-best proofs
slide-22
SLIDE 22

07/14/10 DTProbLog 22

Example:

Dressing for unpredictable weather

Decision Facts Probabilistic Facts

? :: umbrella. 0.3 :: rainy . ? :: raincoat. 0.5 :: windy .

Background Knowledge

dry :- rainy, umbrella, not(broken_umbrella). dry :- rainy, raincoat. dry :- not(rainy). broken_umbrella :- umbrella, rainy, windy .

Utility Facts

umbrella => -2. dry => 60. raincoat => -20. broken_umbrella => -40.

slide-23
SLIDE 23

07/14/10 DTProbLog 23

Exact Solution Algorithm

1

Find all proofs for each utility attribute (Prolog)

2 3 4 5

Binary decision diagrams from the proofs Algebraic decision diagrams for the probability

  • f each attribute

Algebraic decision diagrams for the utility of each attribute Algebraic decision diagram for the total utility

slide-24
SLIDE 24

07/14/10 DTProbLog 24

1

Find all proofs for each utility attribute (Prolog)

2

Binary decision diagrams from the proofs

dry => 60. broken_umbrella => -40. umbrella => -2. raincoat => -20.

  • Leafs indicate attribute true or false
  • Nodes are probabilistic facts or decisions

false true

slide-25
SLIDE 25

07/14/10 DTProbLog 25

3

Algebraic decision diagrams for the probability

  • f each attribute

dry => 60. broken_umbrella => -40. umbrella => -2. raincoat => -20.

  • Probabilistic facts are marginalized out
  • Nodes are decisions only
  • Leafs are probabilities
slide-26
SLIDE 26

07/14/10 DTProbLog 26

4

Algebraic decision diagrams for the utility of each attribute

dry => 60. broken_umbrella => -40. umbrella => -2. raincoat => -20.

  • Leafs are expected utilities
slide-27
SLIDE 27

07/14/10 DTProbLog 27

5

Algebraic decision diagram for the total utility

+ sound pruning (ADD not built entirely in memory)

∑

slide-28
SLIDE 28

07/14/10 DTProbLog 28

Approximate Solution: Local Search

1

Find all proofs for each utility attribute (Prolog)

2 3 4 5

Binary decision diagrams from the proofs Algebraic decision diagrams for the probability

  • f each attribute

Algebraic decision diagrams for the utility of each attribute Algebraic decision diagram for the total utility

slide-29
SLIDE 29

07/14/10 DTProbLog 29

Approximate Solution: Local Search

1

Find all proofs for each utility attribute (Prolog)

2 3 4 5

Binary decision diagrams from the proofs Algebraic decision diagrams for the probability

  • f each attribute

Algebraic decision diagrams for the utility of each attribute Algebraic decision diagram for the total utility

3

Greedy hillclimber search

slide-30
SLIDE 30

07/14/10 DTProbLog 30

Approximate Solution: K-best Proofs

1

Find all proofs for each utility attribute (Prolog)

2 3 4 5

Binary decision diagrams from the proofs Algebraic decision diagrams for the probability

  • f each attribute

Algebraic decision diagrams for the utility of each attribute Algebraic decision diagram for the total utility

slide-31
SLIDE 31

07/14/10 DTProbLog 31

Approximate Solution: K-best Proofs

1

Find all proofs for each utility attribute (Prolog)

2 3 4 5

Binary decision diagrams from the proofs Algebraic decision diagrams for the probability

  • f each attribute

Algebraic decision diagrams for the utility of each attribute Algebraic decision diagram for the total utility

1

Find the k most likely proofs for each utility attribute

slide-32
SLIDE 32

07/14/10 DTProbLog 32

Experiments: Viral Marketing

slide-33
SLIDE 33

07/14/10 DTProbLog 33

Experiments: Viral Marketing

  • Synthetic dataset

Random power law graphs of increasing size

  • Real-world Epinions dataset [Domingos02]
slide-34
SLIDE 34

07/14/10 DTProbLog 34

(Q1) Does the exact solution algorithm perform better than naively calculating the utility of all strategies?

slide-35
SLIDE 35

07/14/10 DTProbLog 35

(Q2) What is the difference in runtime and solution quality between exact and local search?

slide-36
SLIDE 36

07/14/10 DTProbLog 36

(Q3) What is the difference in runtime and solution quality between for different values of k in k-best.

slide-37
SLIDE 37

07/14/10 DTProbLog 37

(Q4) Do the algorithms scale?

Epinions social network

  • 75,000 people
  • 500,000 trust relations

YES

  • Local search
  • 17-best proofs
  • Solved in 16 hours
slide-38
SLIDE 38

07/14/10 DTProbLog 38

Related Work & Conclusions

slide-39
SLIDE 39

07/14/10 DTProbLog 39

Related Work

Representation Solution Evaluation

Relational Probabilities Global

  • ptimum

Local

  • ptimum

Exact inference Approximate inference

Influence Diagrams MLDNs [Nath] ICL [Poole] DTLPs [Chen] DTProbLog

slide-40
SLIDE 40

07/14/10 DTProbLog 40

Summary

  • DTProbLog, the programming language
  • Probabilistic Prolog
  • Decisions
  • Utilities: rewards or costs attached to goals
  • Solution algorithms
  • Exactly
  • Approximately
  • Experiments
  • Effective
  • Scale well
slide-41
SLIDE 41

07/14/10 DTProbLog 41

Ongoing and Future Work

  • Sequential decision problems
  • Easy to represented in DTProbLog
  • Bad fit for solution algorithms
  • Solvers
  • Integer linear programming
  • Bounded approximation
  • Monte-Carlo
  • Lifting (many BDDs have same structure)
  • Learning DTProbLog programs
  • Inverse reinforcement learning
slide-42
SLIDE 42

07/14/10 DTProbLog 42

Thank You!

slide-43
SLIDE 43

07/14/10 DTProbLog 43

Viral Marketing

Decisions

? :: marketed(P) :- person(P).

Probabilistic Facts

0.3 :: buy_trust(_,_). 0.2 :: buy_marketing(_).

Background Knowledge

buys(X) :- buys(X) :- trusts(X,Y), marketed(X), buys(Y), buy_marketing(X). buy_trust(X,Y).

Utility Facts

buys(P) => 5 :- person(P). marketed(P) => -3 :- person(P).