Probing the properties of soft matter by optimally designed - PowerPoint PPT Presentation

Probing the properties of soft matter by optimally designed nonequilibrium experiments Carsten Hartmann (FU Berlin) Newton Institute, Cambridge, UK, 1–4 December 2015

Predicting molecular flexibility ◮ Estimation of molecular properties in thermodynamic equilibrium , e.g. e − W � � F = − log E . (includes rates, statistical weights, etc.) ◮ Perturbation drives the system out of equilibrium with likelihood quotient ϕ = d µ 0 d µ . ◮ Experimental and numerical realization: AFM, SMD, TMD, Metadynamics, . . . [Schlitter, J Mol Graph, 1994], [Schulten & Park, JCP, 2004], [H. et al, Proc Comput Sci, 2010]

Set-up (estimation problem) Given an “equilibrium” diffusion process X = ( X t ) t ≥ 0 on R n , dX t = b ( X t ) dt + σ ( X t ) dB t , X 0 = x , we want to estimate path functionals of the form e − W ( X ) � � ψ ( x ) = E Example: mean passage time to a set C ⊂ R n Let W = ατ C . Then, for sufficiently small α > 0, − α − 1 log ψ = E [ τ C ] + O ( α )

Guiding example: bistable system 7 6 ◮ Overdamped Langevin equation 5 4 √ 3 V dX t = −∇ V ( X t ) dt + 2 ǫ dB t . 2 1 0 ◮ Small noise asymptotics (Kramers) − 1 − 2 − 1.5 − 1 − 0.5 0 0.5 1 1.5 x ǫ → 0 ǫ log E [ τ C ] = ∆ V . lim 6000 5000 4000 ◮ Standard MC estimator of ψ fails: time (ns) C � 3000 2000 E [ e − 2 ατ C ] ≫ ( E [ e − ατ C ]) 2 1000 0 − 1.5 − 1 − 0.5 0 0.5 1 1.5 x [Freidlin & Wentzell, 1984], [Berglund, Markov Processes Relat Fields 2013]

Outline Sampling of rare events based on optimal control Optimal controls from cross-entropy minimization High-dimensional problems and suboptimal controls

Sampling of rare events based on optimal control Optimal controls from cross-entropy minimization High-dimensional problems and suboptimal controls

Guiding example, cont’d � ◮ Mean first passage time for small ǫ � � E [ τ C ] ≍ exp(∆ V /ǫ ) � � � � ◮ Adaptive tilting of the potential � � � � � ��� � � � ��� � ��� � ��� � U ( x , t ) = V ( x ) − u t x 6000 5000 decreases the energy barrier. 4000 time (ns) ◮ Controlled Langevin equation 3000 2000 √ dX u t = ( u t − ∇ V ( X u t )) dt + 2 ǫ dB t . 1000 0 − 1.5 − 1 − 0.5 0 0.5 1 1.5 x

Can we systematically speed up the sampling while controlling the variance by tilting the energy landscape?

Estimation problem revisited Given a “nonequilibrium” (tilted) diffusion process X u = ( X u t ) t ≥ 0 , dX u t = ( b ( X u t ) + σ ( X u t ) u t ) dt + σ ( X u X u t ) dB t , 0 = x , estimate a reweigthed version of ψ : e − W ( X u ) ϕ ( X u ) e − W ( X ) � = E µ � � � E with equilibrium/nonequilibrium likelihood ratio ϕ = d µ 0 d µ . Remark: We allow for W ’s of the general form � τ W ( X ) = f ( X s , s ) ds + g ( X τ ) , 0 for suitable functions f , g and a stopping time τ < ∞ (a.s.).

Sufficient condition for optimal nonequilibrium forcing Theorem (H, 2012) Let u ∗ be a minimizer of the cost functional � τ u � W ( X u ) + 1 � | u s | 2 ds J ( u ) = E 4 0 under the nonequilibrium dynamics X u t , with X u 0 = x . Then, ψ ( x ) = e − W ( X u ∗ ) ϕ ( X u ∗ ) (a.s.) . Moreover, u ∗ is unique. Proof: Jensen’s inequality and Girsanov’s theorem. [H & Sch¨ utte, JSTAT, 2012], [H et al, Entropy, 2014]

Guiding example, cont’d ◮ Exit problem: f = α , g = 0, τ = τ C : � � τ u � � � C + 1 C | u s | 2 ds J ( u ∗ ) = min ατ u � u E 4 � 0 � � ◮ Recovering equilibrium statistics : � � � � � � ��� � � � ��� � ��� � ��� � d � � J ( u ∗ ) E [ τ C ] = � 6000 d α � α =0 5000 ◮ Optimally tilted potential 4000 time (ns) 3000 2000 U ∗ ( x , t ) = V ( x ) − u ∗ t x 1000 0 with stationary feedback u ∗ t = c ( X u ∗ − 1.5 − 1 − 0.5 0 0.5 1 1.5 t ). x

Yet, . . .

. . . there is a catch The optimal control is a feedback control in gradient form , u ∗ t = − 2 σ ( X u ∗ t ) T ∇ F ( X u ∗ t ) , with the bias potential being the value function F ( x ) = min u J ( u ) . NFL Theorem I: The bias potential is given by F = − log ψ . NFL Theorem II: F solves a nonlinear Hamilton-Jacobi-type PDE, − ∂ F x , F , ∇ F , ∇ 2 F � � ∂ t + H = 0 . (Remark: In some cases, F may be explicitly time-dependent.) [H & Sch¨ utte, JSTAT, 2012], [H et al, Entropy, 2014]; cf. [Fleming, SIAM J Control, 1978]

Sampling of rare events based on optimal control Optimal controls from cross-entropy minimization High-dimensional problems and suboptimal controls

Two key facts about our control problem

Fact #1 The optimal control is a feedback law of the form ∞ � u ∗ t = σ ( X u c i ∇ φ i ( X u t ) t ) , i =1 with coefficients c i ∈ R and suitable basis functions φ i ∈ C 1 ( R n ).

Fact #2 Letting µ denote the probability (path) measure on C ([0 , ∞ )) associated with the tilted dynamics X u , it holds that J ( u ) − J ( u ∗ ) = KL ( µ, µ ∗ ) with µ ∗ = µ ( u ∗ ) and � d µ � �  if µ ≪ µ ∗ log d µ  KL ( µ, µ ∗ ) = d µ ∗ ∞ otherwise  the Kullback-Leibler divergence between µ and µ ∗ .

Cross-entropy method for diffusions Idea: seek a minimizer of J among all controls of the form M � u t = σ ( X u α i ∇ φ i ( X u φ i ∈ C 1 ( X ) . ˆ i ) t ) , i =1 and minimize the Kullback-Leibler divergence S ( µ ) = KL ( µ, µ ∗ ) over all candidate probability measures of the form µ = µ (ˆ u ). Remark: unique minimizer is given by d µ ∗ = ψ − 1 e − W d µ 0 . cf. [Oberhofer & Dellago, CPC, 2008], [Aurell et al, PRL, 2011]

Unfortunately, . . .

Cross-entropy method for diffusions, cont’d . . . that doesn’t work without knowing the normalization factor ψ . Feasible cross-entropy minimization Minimization of the relaxed functional KL ( µ ∗ , · ) is equivalent to cross-entropy minimization : minimize � log µ d µ ∗ CE ( µ ) = − u ), with d µ ∗ ∝ e − W d µ 0 . over all admissible µ = µ (ˆ Note: KL ( µ, µ ∗ )=0 iff KL ( µ ∗ , µ ) = 0, which holds iff µ = µ ∗ . [Rubinstein & Kroese, Springer, 2004], [Zhang et al, SISC, 2014], [Badowski, PhD thesis, 2015]

Example I (guiding example)

Computing the mean first passage time ( n = 1) Minimize � τ C � ατ + 1 � | u t | 2 dt J ( u ; α ) = E 4 0 with τ C = inf { t > 0: X t ∈ [ − 1 . 1 , − 1] } and the dynamics t )) dt + 2 − 1 / 2 dB t dX u t = ( u t − ∇ V ( X u 2.5 160 140 2 120 100 1.5 E x ( τ ) V(x) 80 1 60 40 0.5 20 0 0 −1.5 −1 −0.5 0 0.5 1 1.5 −2 −1 0 1 2 x x Skew double-well potential V and MFPT of the set S = [ − 1 . 1 , − 1] (FEM reference solution).

Computing the mean first passage time, cont’d Cross-entropy minimization using a parametric ansatz 10 � c ( x ) = α i ∇ φ i ( x ) , φ i : equispaced Gaussians i =1 4 1.2 3.5 1 E x ( τ ) with opt. control 3 0.8 (V+U)(x) 160 2.5 140 0.6 120 2 100 E x ( τ ) 80 0.4 60 1.5 40 20 0.2 1 0 − 2 − 1 0 1 2 x 0.5 0 −1.5 −1 −0.5 0 0.5 1 1.5 − 1 − 0.5 0 0.5 1 1.5 x x Biasing potential V + 2 F and unbiased estimate of the limiting MFPT. cf. [H & Sch¨ utte, JSTAT, 2012]

Sampling of rare events based on optimal control Optimal controls from cross-entropy minimization High-dimensional problems and suboptimal controls

The bad news

The good news 20 15 10 Bound for the relative sampling error V ε (x) 5 for suboptimal controls ¯ u based on 0 -5 averaged equations of motion: -10 -6 -4 -2 0 2 4 6 x 9 � τ fast Theo. Value Fun.: ε = 0.3 8 � 1 / 8 Opt. Value Func.: Homogenized C 7 √ 6 δ rel ≤ . 5 τ slow V(x) N 4 3 2 ( N : sample size, C ≈ 1) 1 0 -5 -4 -3 -2 -1 0 1 2 x 9 Opt. Control: ε = 0.3 8 Opt. Control: Homogenized Opt. Control Correction: ε = 0.3 7 Remark: δ ∗ rel = 0 for u = u ∗ . 6 5 u(x) 4 3 2 1 0 -5 -4 -3 -2 -1 0 1 2 x [H et al, J Comp Dyn, 2014], [Zhang et al, Prob Theory Rel Fields, submitted]

The good news, cont’d 20 Averaged control problem : minimize 15 10 � τ v V ε (x) 5 � W ( ξ v ) + 1 � | v s | 2 ds ¯ 0 I ( v ) = E 4 -5 0 -10 -6 -4 -2 0 2 4 6 x 9 subject to the averaged dynamics Theo. Value Fun.: ε = 0.3 8 Opt. Value Func.: Homogenized 7 6 5 t = ( v t − ¯ V(x) d ξ u b ( ξ v σ ( ξ v t )) dt + ¯ t ) dB t 4 3 2 1 0 -5 -4 -3 -2 -1 0 1 2 x 9 Opt. Control: ε = 0.3 8 Opt. Control: Homogenized Opt. Control Correction: ε = 0.3 Control approximation strategy 7 6 5 u(x) 4 t ≈ c ( ξ ( X u ∗ t )) = ∇ ξ ( X u ∗ u ∗ t ) v ∗ 3 t 2 1 0 -5 -4 -3 -2 -1 0 1 2 x [H et al, Nonlinearity, submitted]; cf. [Legoll & Leli` evre, Nonlinearity, 2010]

Example II (suboptimal control)