Amortized Monte Carlo Integration Adam Goli ski*, Frank Wood, Tom - - PowerPoint PPT Presentation

Amortized Monte Carlo Integration

Adam Goliński*, Frank Wood, Tom Rainforth*

11/06/19

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

2

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

3

Ep(x|y)[f(x; θ)]

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

4

AMCI = novel estimator + amortization objectives

Ep(x|y)[f(x; θ)]

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

5

AMCI = novel estimator + amortization objectives

Ep(x|y)[f(x; θ)]

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

6

Importance Sampling (IS)

Eπ(x)[f(x)] ≈ b µ := 1 N

N

X

n=1

f (xn) wn where xn ∼ q(x), wn = π (xn) q (xn) E[b µ] = Eπ(x)[f(x)] Var[b µ] = Var[f(x1)w1] N

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

7

Importance Sampling (IS) When and yields an exact estimate using a single sample

q(x) ∝ π(x)f(x)

f(x) ≥ 0

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

8

Importance Sampling (IS) When and yields an exact estimate using a single sample

q(x) ∝ π(x)f(x)

f(x) ≥ 0

q∗(x) ∝ f(x)π(x) = ⇒ q∗(x) = f(x)π(x) R f(x)π(x)dx = f(x)π(x) Eπ(x)[f(x)] f(x1)w1 = f(x1) π(x1) q∗(x1) = f(x1) π(x1)

f(x1)π(x1) Eπ(x)[f(x)]

= Eπ(x)[f(x)]

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

9

Self-Normalized Importance Sampling (SNIS) Ep(x|y)[f(x)]

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

10

Self-Normalized Importance Sampling (SNIS) Ep(x|y)[f(x)] = Ep(x)[f(x)p(y|x)] Ep(x)[p(y|x)] ≈

1 N

PN

n=1 f (xn) wn 1 N

PN

n=1 wn

xn ∼ q(x) wn = p (xn, y) q (xn)

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

11

101 102 103 104 105

Number of samples

10−6 10−4 10−2 100

Relative Error

Traditional approach Its error lower bound AMCI

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

12

Ep(x|y)[f(x)] = Ep(x)[f(x)p(y|x)] Ep(x)[p(y|x)] ≈

1 N

PN

n=1 f (xn) wn 1 N

PN

n=1 wn

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

13

Ep(x|y)[f(x)] = Ep(x)[f(x)p(y|x)] Ep(x)[p(y|x)] ≈

1 N

PN

n=1 f (xn) wn 1 N

PN

n=1 wn

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

14

Ep(x|y)[f(x)] = Ep(x)[f(x)p(y|x)] Ep(x)[p(y|x)] ≈

1 N

PN

n=1 f (xn) wn 1 N

PN

n=1 wn

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

15

Ep(x|y)[f(x)] = Ep(x)[f(x)p(y|x)] Ep(x)[p(y|x)] ≈

1 N

PN

n=1 f (xn) wn 1 N

PN

n=1 wn

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

16

Ep(x|y)[f(x)] = Ep(x)[f(x)p(y|x)] Ep(x)[p(y|x)] ≈

1 N

PN

n=1 f (xn) wn 1 N

PN

n=1 wn

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

17

Ep(x|y)[f(x)] = Ep(x)[f(x)p(y|x)] Ep(x)[p(y|x)] ≈

1 N

PN

n=1 f (xn) wn 1 N

PN

n=1 wn

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

18

Ep(x|y)[f(x)] = Ep(x)[f(x)p(y|x)] Ep(x)[p(y|x)] ≈

1 N

PN

n=1 f (xn) wn 1 N

PN

n=1 wn

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

19

Ep(x|y)[f(x)] = Ep(x)[f(x)p(y|x)] Ep(x)[p(y|x)] ≈

1 N

PN

n=1 f (xn) wn 1 N

PN

n=1 wn

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

20

Ep(x|y)[f(x)] = Ep(x)[f(x)p(y|x)] Ep(x)[p(y|x)] ≈

1 N

PN

n=1 f (xn) wn 1 N

PN

n=1 wn

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

21

Ep(x|y)[f(x)] = Ep(x)[f(x)p(y|x)] Ep(x)[p(y|x)] ≈

1 N

PN

n=1 f (xn) wn 1 N

PN

n=1 wn

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

22

Ep(x|y)[f(x)] = Ep(x)[f(x)p(y|x)] Ep(x)[p(y|x)] ≈

1 N

PN

n=1 f (xn) wn 1 N

PN

n=1 wn

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

23

Self-Normalized Importance Sampling (SNIS)

Mean Squared Error ≥ 1 N

• Ep(x|y)

⇥ f(x) − Ep(x|y)[f(x)]

• ⇤2

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

24

Solution: Use Multiple Proposals Targeting Different Aspects of the Problem

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

25

The AMCI Estimator

Ep(x|y)[f(x; θ)] = Ep(x)[f(x; θ)p(y|x)] Ep(x)[p(y|x)] E

+

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

26

The AMCI Estimator

Ep(x|y)[f(x; θ)] = Ep(x)[f(x; θ)p(y|x)] Ep(x)[p(y|x)]

f +(x; θ) = max(f(x; θ), 0) f −(x; θ) = − min(f(x; θ), 0) f(x; θ) = f +(x; θ) − f −(x; θ)

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

27

The AMCI Estimator

Ep(x|y)[f(x; θ)] = Ep(x)[f(x; θ)p(y|x)] Ep(x)[p(y|x)] = Ep(x)[f +(x; θ)p(y|x)] − Ep(x)[f −(x; θ)p(y|x)] Ep(x)[p(y|x)]

f +(x; θ) = max(f(x; θ), 0) f −(x; θ) = − min(f(x; θ), 0) f(x; θ) = f +(x; θ) − f −(x; θ)

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

28

The AMCI Estimator

Ep(x|y)[f(x; θ)] = Ep(x)[f(x; θ)p(y|x)] Ep(x)[p(y|x)] = Ep(x)[f +(x; θ)p(y|x)] − Ep(x)[f −(x; θ)p(y|x)] Ep(x)[p(y|x)] = Eq+

1 (x;y,θ)[ f +(x;θ)p(y|x)

q+

1 (x;y,θ)

] − Eq−

1 (x;y,θ)[ f −(x;θ)p(y|x)

q−

1 (x;y,θ)

] Eq2(x;y)[ p(y|x)

q2(x;y)]

f +(x; θ) = max(f(x; θ), 0) f −(x; θ) = − min(f(x; θ), 0) f(x; θ) = f +(x; θ) − f −(x; θ)

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

29

The AMCI Estimator

Ep(x|y)[f(x; θ)] = Ep(x)[f(x; θ)p(y|x)] Ep(x)[p(y|x)] = Ep(x)[f +(x; θ)p(y|x)] − Ep(x)[f −(x; θ)p(y|x)] Ep(x)[p(y|x)] = Eq+

1 (x;y,θ)[ f +(x;θ)p(y|x)

q+

1 (x;y,θ)

] − Eq−

1 (x;y,θ)[ f −(x;θ)p(y|x)

q−

1 (x;y,θ)

] Eq2(x;y)[ p(y|x)

q2(x;y)]

=: E+

1 − E− 1

E2

f +(x; θ) = max(f(x; θ), 0) f −(x; θ) = − min(f(x; θ), 0) f(x; θ) = f +(x; θ) − f −(x; θ)

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

30

E2 = Ep(x) [p(y|x)]] ≈ ˆ E2 = 1 M

M

X

m=1

p(xm, y) q2(xm; y) E+

1 = Ep(x) [max(f(x; θ), 0)p(y|x)] ≈ ˆ

E+

1 = 1

N

N

X

n=1

f +(x+

n; θ)p(x+ n, y)

q+

1 (x+ n; y, θ)

E−

1 = Ep(x) [– min(f(x; θ), 0)p(y|x)] ≈ ˆ

E−

1 = 1

K

K

X

k=1

f −(x−

k ; θ)p(x− k , y)

q−

1 (x− k ; y, θ)

The AMCI Estimator

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

31

|

− E2 = Ep(x) [p(y|x)]] ≈ ˆ E2 = 1 M

M

X

m=1

p(xm, y) q2(xm; y) E+

1 = Ep(x) [max(f(x; θ), 0)p(y|x)] ≈ ˆ

E+

1 = 1

N

N

X

n=1

f +(x+

n; θ)p(x+ n, y)

q+

1 (x+ n; y, θ)

E−

1 = Ep(x) [– min(f(x; θ), 0)p(y|x)] ≈ ˆ

E−

1 = 1

K

K

X

k=1

f −(x−

k ; θ)p(x− k , y)

q−

1 (x− k ; y, θ)

The AMCI Estimator

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

32

X Theorem 1.If q2(x; y)∝p(x, y), q+

1 (x; y, θ)∝f +(x; θ)p(x, y), and

q−

1 (x; y, θ)∝f −(x; θ)p(x, y), then the AMCI estimator ( ˆ

E+

1 − ˆ

E−

1 )/ ˆ

E2 is an exact estimator for Ep(x|y)[f(x; θ)] even if N =K =M =1

AMCI Can Produce Perfect Estimates with a Single Sample from Each Proposal!

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

33

Amortized Inference

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

34

Amortized Inference

) [DKL[p(x|y)kq2(x; y, η)]]

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

35

Amortized Inference

• B. Paige, F

. Wood. Inference networks for SMC in graphical models. ICML, 2016.

Jq2(η) = Ep(y) [DKL[p(x|y)kq2(x; y, η)]]

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

36

Amortized Inference

• B. Paige, F

. Wood. Inference networks for SMC in graphical models. ICML, 2016.

Jq2(η) = Ep(y) [DKL[p(x|y)kq2(x; y, η)]] = Ep(x,y)[ log q2(x; y, η)] + const wrt η

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

37

Amortizing Monte Carlo Integration

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

38

To amortize over function parameters, we introduce a pseudo prior p(θ)

Amortizing Monte Carlo Integration

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

39

To amortize over function parameters, we introduce a pseudo prior p(θ)

Amortizing Monte Carlo Integration

∝ Jq±

1 (η) = Ep(x,y)p(θ)

⇥ −f ±(x; θ) log q±

1 (x; y, θ, η)

⇤ + const

q±

1 ∝ f ±(x; θ)p(x, y)

⇥ ⇤

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

40

Experiments: Gaussian Tail Integral

p(x) = N(x; 0, 1) p(y|x) = N(y; x, 1) f(x; θ) = 1x>θ

f(x; θ)p(x|y) p(x|y) θ q1(x; y, θ) q2(x; y)

Density

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

41

Experiments: Gaussian Tail Integral

101 102 103 104 105

Number of samples N

10−6 10−4 10−2 100

Relative Mean Squared Error

Traditional SNIS approach SNIS error lower bound AMCI

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

42

Test

f(x; θ)p(x|y) p(x|y) θ q1(x; y, θ) q2(x; y)

Test Density

When Does AMCI Work Well in Practice?

−5.0 −2.5 0.0 2.5 5.0

x Density

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

43

Test

f(x; θ)p(x|y) p(x|y) θ q1(x; y, θ) q2(x; y)

10−4 10−1 102

Test

AMCI SNIS q2

Density ReMSE

When Does AMCI Work Well in Practice?

−5.0 −2.5 0.0 2.5 5.0

x

101 102 103 104

Number of samples N ReMSE Density

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

44

Test

f(x; θ)p(x|y) p(x|y) θ q1(x; y, θ) q2(x; y)

10−4 10−1 102

Test

AMCI SNIS q2

−5.0 −2.5 0.0 2.5 5.0

x

101 102 103 104

Number of samples N

10−4 10−1 102

Density ReMSE

When Does AMCI Work Well in Practice?

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

45

Recap

I Amortized inference focuses on approximating the posterior p(x|y), later using this to estimate expectation(s) Ep(x|y)[f(x; θ)] I This pipeline is inefficient if f(x; θ) is known upfront I AMCI instead targets Ep(x|y)[f(x; θ)] directly, allowing amortization

• ver datasets y and/or function parameters θ

I It can give exact estimates for any expectation with only a single sample from each of three separate amortized proposals I It can empirically outperform the theoretically optimal self-normalized importance sampler, even in non-amortized settings

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

Adam Goliński @adam_golinski Frank Wood @frankdonaldwood Tom Rainforth @tom_rainforth

Come see us at poster #999 today at 6.30pm!

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

47

When Does AMCI Work Well in Practice?

∼ N · MSE ≈ σ2

2

E2

2

·

• (κ − Corr[ξ1, ξ2])2 + 1 − Corr[ξ1, ξ2]2

is a measure of relative performance of top and bottom estimators

• l κ, f

For AMCI, we can control , for SNIS we cannot

• l κ, f

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

48

Intractability in the naively adjusted objective

E1(y) := Ep(x)[f(x)p(y|x)] g(x|y) := f(x)p(x, y) E1(y) J 0

q1(η) = Ep(y) [DKL (g(x|y)kq1(x; y, η))]

= Ep(y) 

• Z

X

f(x)p(x, y) E1(y) log q1(x; y, η)dx

• + const

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

49

E1(y) := Ep(x)[f(x)p(y|x)] g(x|y) := f(x)p(x, y) E1(y) J 0

q1(η) = Ep(y) [DKL (g(x|y)kq1(x; y, η))]

= Ep(y) 

• Z

X

f(x)p(x, y) E1(y) log q1(x; y, η)dx

• + const

Intractability in the naively adjusted objective

 Z

X

• h(y) / p(y)E1(y)

Jq1(η) = Eh(y) [DKL (g(x|y)kq1(x; y, η))] = 1 constEp(x,y) [f(x) log q1(x; y, η)] + const

Amortized Monte Carlo Integration Adam Goliński*, Frank Wood, Tom Rainforth*

50

Experiments: Cancer Treatment Planning

101 102 103

Number of samples N

10−4 10−3 10−2 10−1 100

Relative Mean Squared Error

Traditional SNIS approach SNIS error lower bound AMCI