Approximate Knowledge Compilation by Online Collapsed Importance - - PowerPoint PPT Presentation

▶
approximate knowledge compilation by online collapsed
SMART_READER_LITE
LIVE PREVIEW

Approximate Knowledge Compilation by Online Collapsed Importance - - PowerPoint PPT Presentation

Apr 03, 2024 273 likes •636 views

Approximate Knowledge Compilation by Online Collapsed Importance Sampling Tal Friedman and Guy Van den Broeck Motivation Factor Graphs: 1 Motivation Factor Graphs: Great! But asking queries is hard 2 Motivation Factor Graphs: 3

slide-1
SLIDE 1

Approximate Knowledge Compilation by Online Collapsed Importance Sampling

Tal Friedman and Guy Van den Broeck

slide-2
SLIDE 2

Motivation

Factor Graphs:

1

slide-3
SLIDE 3

Motivation

Factor Graphs: Great! But asking queries is hard

2

slide-4
SLIDE 4

Motivation

Factor Graphs:

3

slide-5
SLIDE 5

Motivation: Arithmetic Circuit

  • Exact inference: Use Knowledge Compilation (e.g. BDD, SPN)

4

  • Tractable form: easy

queries + operations

  • Take advantage of

further independence properties, logical structure

slide-6
SLIDE 6

But they don’t scale!

5

slide-7
SLIDE 7

6

Exact Independence Properties Logical Structure Scalable Anytime Knowledge Compilation Sampling This work

slide-8
SLIDE 8

Collapsed Sampling (Rao-Blackwell)

Sampling on some variables, exact inference conditioned on sample

7

slide-9
SLIDE 9

Collapsed Sampling (Rao-Blackwell)

Sampling on some variables, exact inference conditioned on sample

8

slide-10
SLIDE 10

Collapsed Sampling (Rao-Blackwell)

Sampling on some variables, exact inference conditioned on sample

9

Sample A,B

slide-11
SLIDE 11

Collapsed Sampling (Rao-Blackwell)

Sampling on some variables, exact inference conditioned on sample

10

Observe sampled values

slide-12
SLIDE 12

Collapsed Sampling (Rao-Blackwell)

Sampling on some variables, exact inference conditioned on sample

11

Compute exactly P(C|A,B)

slide-13
SLIDE 13

What to Sample?

  • Is it even possible to pick a correct set a priori?
  • Consider a network of potential smokers, with friendships sampled

12

Sample 1 Sample 2

slide-14
SLIDE 14

Online Collapsed Sampling

Choose on-the-fly which variable to sample next, based on result of sampling previous variables Theorem: Still unbiased

13

slide-15
SLIDE 15

How?

  • 1. What/when do we sample?

14

slide-16
SLIDE 16

How?

  • 1. What/when do we sample?
  • 2. How do we sample?

15

slide-17
SLIDE 17

How do we Sample?

  • Importance Sampling
  • Need a proposal for any variable conditioned on any other variables

16

slide-18
SLIDE 18

How?

  • 1. What/when do we sample?
  • 2. How do we sample?
  • 3. How do we do exact inference?

17

slide-19
SLIDE 19

Exact Inference

How do we do exact inference conditioned on different variables?

18

slide-20
SLIDE 20

Exact Inference

  • How do we do exact inference conditioned on different variables?

19

slide-21
SLIDE 21

Collapsed Compilation

20

Exact Inference Sampling

Result: A circuit for factor graph with some sampled variables Big Circuit? Small Circuit?

slide-22
SLIDE 22

Collapsed Compilation

21

Exact Inference Sampling

Big Circuit? Small Circuit?

  • 1. What/when do we

sample?

  • 2. How do we sample?
  • 3. How do we do exact

inference?

slide-23
SLIDE 23

Collapsed Compilation

22

Exact Inference Sampling

Big Circuit? Small Circuit?

  • 1. What/when do we

sample?

  • 2. How do we sample?
  • 3. How do we do exact

inference?

slide-24
SLIDE 24

What/when do we sample?

When: Circuit too big What: Heuristic on current circuit

23

slide-25
SLIDE 25

Collapsed Compilation

24

Exact Inference Sampling

Big Circuit? Small Circuit?

  • 1. What/when do we

sample?

  • 2. How do we sample?
  • 3. How do we do exact

inference?

slide-26
SLIDE 26

Motivation: Arithmetic Circuit

  • Exact inference: Use Knowledge Compilation (e.g. BDD, SPN)

25

  • Tractable form: easy

queries + operations

  • Take advantage of

further independence properties, logical structure

slide-27
SLIDE 27

How do we sample?

Compute the marginal of the variable in the current circuit!

26

slide-28
SLIDE 28

Collapsed Compilation

27

Exact Inference Sampling

Big Circuit? Small Circuit?

  • 1. What/when do we

sample?

  • 2. How do we sample?
  • 3. How do we do exact

inference?

slide-29
SLIDE 29

Conditional Exact Inference

Result is a circuit: any joint can be computed efficiently & exactly

28

slide-30
SLIDE 30

Online Collapsed Importance Sampling

29

Approximate any query! 0.6 x 0.9 x 1.1 x

slide-31
SLIDE 31

Experiments

  • Approximate marginal in factor graph
  • Algorithmically limit exact inference

30

slide-32
SLIDE 32

Experiments

31

slide-33
SLIDE 33

32

Knowledge Compilation Sampling Collapsed Compilation Exact Independence Properties Logical Structure Scalable Anytime Scalable Anytime Independence Properties Logical Structure

slide-34
SLIDE 34

Thanks! Poster: Room 210 #5 Code: github.com/UCLA-StarAI/Collapsed-Compilation

33