Hybrid Monte Carlo: Geometric Integration and Statistics Andrew - - PowerPoint PPT Presentation

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

Hybrid Monte Carlo: Geometric Integration and Statistics

Andrew Stuart1

1Mathematics Institute and

Centre for Scientific Computing University of Warwick

SCMS2010 Heriot-Watt, September 6th 2010 Funded by EPSRC, ONR

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

“Stable Periodic Bifurcations of an Explicit Discretization of a Nonlinear Partial Differential Equation in Reaction Diffusion”. D.F . Griffiths and A.R.Mitchell IMA J Numerical Analysis 8(1988), 435-454.

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

Outline

1

INVARIANT MEASURES AND DYNAMICAL SYSTEMS

2

MARKOV CHAIN MONTE CARLO

3

EXPLICIT DISCRETIZATIONS

4

IMPLICIT DISCRETIZATIONS

5

CONCLUSIONS

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

Outline

1

INVARIANT MEASURES AND DYNAMICAL SYSTEMS

2

MARKOV CHAIN MONTE CARLO

3

EXPLICIT DISCRETIZATIONS

4

IMPLICIT DISCRETIZATIONS

5

CONCLUSIONS

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

Goals of Work

To find numerical methods to sample a probability density function (pdf) π : Rn → R+. To analyze and develop methods in the cases of high dimensions n ≫ 1. Baisc building blocks are π−invariant dynamical systems and MCMC methodology.

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

Langevin Stochastic Dynamics

Let A be a positive-definite symmetric matrix. The Langevin SDE is ˙ x = A∇ log π(x) + √ 2A ˙ W. This equation is π− invariant: if x(0) ∼ π then x(t) ∼ π for all t > 0.

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

Hybrid Monte Carlo

(Duane et al 1987) Let A be a positive-definite symmetric matrix. Define the Hamiltonian H(x, p) = 1 2p, Ap − log π(x). Hamiltons equations are ˙ x = Ap, ˙ p = ∇

• log π(x)
• .

Assume that p(0) ∼ N(0, A−1). This equation is π− invariant: if x(0) ∼ π, then x(t) ∼ π for all t > 0.

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

Outline

1

INVARIANT MEASURES AND DYNAMICAL SYSTEMS

2

MARKOV CHAIN MONTE CARLO

3

EXPLICIT DISCRETIZATIONS

4

IMPLICIT DISCRETIZATIONS

5

CONCLUSIONS

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

Metropolis-Hastings Algorithm

Enforcing π−invariant dynamics via accept-reject:

• 1. Set k = 0 and choose x(0) ∈ Rn.
• 2. Propose y = G(x(k), ξ(k), ∆t),

ξ(k) ∼ N(0, 1).

• 3. Set x(k+1) = y with probability α; else x(k+1) = x(k).
• 4. Set k → k + 1 and goto 2.

Step 3. is the proposal: how do we choose it?

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

Choice of Parameters

We choose G to be a time-discretization of one of the invariant dynamical systems. We want largest ∆t compatible with O(1) average acceptance probability for n ≫ 1. Choose ∆t = n−γ Courant condition. γ0 = minγ≥0

• γ : lim infn→∞ Eα > 0
• .

Number of steps required to adequately sample π is then M(n) = O(∆t)−1 = O(nγ0).

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

Structure of the Target

IID Product in Rn π0(x) = Πn

i=1f(xi).

Change of Measure From Gaussian in Rn π(x) = exp

• −Φn(x)
• π0(x)

π0(x) ∝ exp

• −1

2x, C−1

0 x

• .

The covariance matrix C0 is assumed to have condition number O(n2k). The resulting Gaussian part is assumed to dominate Φn, uniformly in n.

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

Outline

1

INVARIANT MEASURES AND DYNAMICAL SYSTEMS

2

MARKOV CHAIN MONTE CARLO

3

EXPLICIT DISCRETIZATIONS

4

IMPLICIT DISCRETIZATIONS

5

CONCLUSIONS

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

Langevin 1

π0(x) = Πn

i=1f(xi).

Recall that the SDE ˙ x = ∇ log π0(x) + √ 2 ˙ W is π0− invariant. We use the following discretization as proposal: Proposal y − x ∆t = β∇ log π0(x) +

• 2

∆t ξ, ξ ∼ N(0, I). Theorem 1. (Roberts et al 97, Roberts/Rosenthal 98) β = 0 then M(n) = O(n1). β = 1 then M(n) = O(n1/3). Steepest Descents Impacts Cost

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

Langevin 2

π(x) = exp

• −Φn(x)
• π0(x) = exp
• −Φn(x) − 1

2x, C−1

0 x

• .

Proposal y − x ∆t = A∇ log π0(x) +

• 2A

∆t ξ, ξ ∼ N(0, I). Theorem 2. (Beskos, Roberts, Stuart 2009) A = I then M(n) = O(n(2k+1/3)). A = C0 then M(n) = O(n1/3). Preconditioning Impacts Cost

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

Hybrid Monte Carlo 1

The key dynamical system is: ˙ x = Ap, ˙ p = ∇

• log π(x)
• .

Volume preserving reversible integration is required to ensure that the acceptance probability α is tractable. This can be achieved by operator splitting (eg Verlet) based on the two dynamical systems ˙ x = Ap,

• ˙

x = 0, ˙ p = 0.

• ˙

p = ∇

• log π(x)
• .

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

Hybrid Monte Carlo 2

π0(x) = Πn

i=1f(xi).

Theorem 3. (Beskos, Pillai, Roberts, Sanz-Serna and Stuart 2010) For Verlet integration within Hybrid Monte Carlo we have M(n) = O(n1/4). Hamiltonian Formulation Impacts Cost

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

Outline

1

INVARIANT MEASURES AND DYNAMICAL SYSTEMS

2

MARKOV CHAIN MONTE CARLO

3

EXPLICIT DISCRETIZATIONS

4

IMPLICIT DISCRETIZATIONS

5

CONCLUSIONS

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

Langevin

π(x) = exp

• −Φn(x) − 1

2x, C−1

n x

• .

y − x ∆t + A

• θC−1

n y + (1 − θ)C−1 n x

• =
• 2A

∆t ξ, ξ ∼ N(0, I). Theorem 4. (Beskos, Roberts, Stuart 2009) θ = 1

2 and A = Cn then M(n) = O(n1/3).

θ = 1

2 and A = I, Cn then M(n) = O(1).

Implicitness Impacts Cost

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

Hybrid Monte Carlo 1

The key dynamical system is: ˙ x = Ap, ˙ p = −C−1

n x − ∇Φn(x).

Volume preserving reversible integration is required to ensure that the acceptance probability α is tractable. This can be achieved by operator splitting based on the two dynamical systems ˙ x = Ap,

• ˙

x = 0, ˙ p = −C−1

n x.

• ˙

p = −∇Φn(x).

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

Hybrid Monte Carlo 2

If we use a second-order (Strang-splitting) for this operator-split then we obtain: Theorem 4. (Beskos, Pinski, Sanz-Serna and Stuart 2010) If A = Cn then M(n) = O(1). Implicitness Impacts Cost

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

Outline

1

INVARIANT MEASURES AND DYNAMICAL SYSTEMS

2

MARKOV CHAIN MONTE CARLO

3

EXPLICIT DISCRETIZATIONS

4

IMPLICIT DISCRETIZATIONS

5

CONCLUSIONS

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

What We Have Shown

We have shown that the following ideas from numerical analysis have direct impact on MCMC based statistical sampling methods in high dimensions: Steepest descents Preconditioning Implicit integration for dissipative systems Geometric integration

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

References

For all papers see: http : //www.maths.warwick.ac.uk/ ∼ masdr/sample.html

• A. Beskos, N. Pillai, G.O. Roberts, J.-M. Sanz-Serna and

A.M. Stuart. “Optimal tuning of hybrid Monte Carlo”. Submitted.

• A. Beskos, F

. Pinski, J.-M. Sanz-Serna and A.M. Stuart. “Hybrid Monte Carlo on Hilbert spaces”. Submitted.

• A. Beskos, G.O. Roberts and A.M. Stuart. ”Optimal

scalings for local Metropolis-Hastings chains on non-product targets in high dimensions.” Ann. Appl. Prob. 19(2009), 863–898.

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

References (Continued)

• A. Beskos and A.M. Stuart. ”MCMC Methods for Sampling

Function Space”. To appear, proceedings of ICIAM 2007.

• M. Hairer, A.M.Stuart and J. Voss. ”Sampling the posterior:

an approach to non-Gaussian data assimilation.” PhysicaD, 230(2007), 50–64.

• S. Duane, A.D. Kennedy, B.J. Pendelton and D. Roweth.

“Hybrid Monte Carlo.” Physics Letters B, 195(1987), 216-222.

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT

References (Continued)

• A. Gelman, W.R. Gilks and G.O. Roberts, Weak

convergence and optimal scaling of random walk Metropolis algorithms. Ann. Appl. Prob. 7(1997), 110–120. G.O. Roberts and J. Rosenthal, Optimal scaling of discrete approximations to Langevin diffusions. JRSSB 60(1998), 255–268.

• M. Bédard, Weak Convergence of Metropolis Algorithms

for Non-iid Target Distributions. Ann. Appl. Probab. 17(2007), 1222-44.

• M. Bédard and J.S. Rosenthal, Optimal Scaling of

Metropolis Algorithms: Heading Towards General Target

• Distributions. To appear Can. J. Stat.