Optimal mass transport and density flows Tryphon Georgiou - PowerPoint PPT Presentation

Optimal mass transport and density flows Tryphon Georgiou Mechanical & Aerospace Engineering University of California, Irvine Joint work with Yongxin Chen (MSKCC), M. Pavon (Padova) , A. Tannenbaum (Stony Brook), and Wilfrid Gangbo (UCLA) LCCC, Lund June 13, 2017 Supported by the NSF & AFOSR

Plan of the talk: – Nexus of ideas: Mass Transport ⇔ Schr¨ odinger bridges ⇔ Stochastic control with a bit on LQG, Riccati, etc. – Discrete-space counterpart: Markov chains and networks – Non-commutative counterpart: Quantum flows & non-commutative geometry

Density flows

Optimal Mass Transport (OMT) Gaspard Monge 1781 Leonid Kantorovich 1976 Work in early 1940’s, Nobel 1975 CIA file on Kantorovich (wikipedia)

Monge’s formulation Le m´ emoire sur les d´ eblais et les remblais Gaspard Monge 1781 � � 2 dµ ( x ) � x − T ( x ) inf T � �� � y where T # µ = ν

Kantorovich’s formulation �� � x − y � 2 dπ ( x, y ) 3 3 inf 2.5 2.5 π ∈ Π( ρ 0 ,ρ 1 ) 2 2 1.5 1.5 1 1 0.5 0.5 0 0 0 1 2 0 1 2 3 where Π( µ, ν ) are “couplings”: 6 5 4 � y π ( dx, dy ) = ρ 0 ( x ) dx = dµ ( x ) 3 2 1 0 � 0 1 2 3 x π ( dx, dy ) = ρ 1 ( y ) dy = dν ( y ) .

B&B’s fluid dynamic formulation Benamou and Brenier (2000): � 1 � � v ( x, t ) � 2 ρ ( x, t ) dtdx inf R n ( ρ,v ) 0 ∂ρ ∂t + ∇ · ( vρ ) = 0 ρ ( x, 0) = ρ 0 ( x ) , ρ ( y, 1) = ρ 1 ( y ) McCann, Gangbo, Otto, Villani, ...

Stochastic control formulation �� 1 � � v ( x, t ) � 2 dt inf v E ρ 0 x ( t ) = v ( x, t ) ˙ x (0) ∼ ρ 0 ( x ) dx x (1) ∼ ρ 1 ( y ) dy

OMT as a control problem – derivation � 1 � x − y � 2 = x � 2 dt, � ˙ inf x ∈X xy 0 X xy = { x ∈ C 1 | x(0) = x, x(1) = y } . Inf attained at constant speed geodesic x ∗ ( t ) = (1 − t ) x + ty

OMT as a control problem – Dirac marginals Also, Inf = any probabilistic average in X xy �� 1 � � x − y � 2 = inf x( t ) � 2 dt � ˙ , E P xy P xy 0 P xy ∈ D ( δ x , δ y ) : prob. measures on C 1 with delta marginals

OMT as a control problem – general marginals �� 1 � � R n × R n � x − y � 2 dπ ( x, y ) = x( t ) � 2 dt � ˙ inf inf . E P π ∈ Π( ρ 0 ,ρ 1 ) P ∈ D ( ρ 0 ,ρ 1 ) 0 ⇒ OMT ≃ stochastic control problem with atypical boundary constraints �� 1 � � v � 2 dt inf v E 0 x (0) ∼ ρ 0 dx, x (1) ∼ ρ 1 dy. x ( t ) = v ( x ( t ) , t ) , ˙ a . s .,

Schr¨ odinger’s Bridges ρ = Ψ ¯ Ψ Ψ t = U ( t )Ψ0 Erwin Schr¨ odinger Work in 1926, Nobel 1935 Bridges 1931/32

Schr¨ odinger’s Bridge Problem (SBP) – Cloud of N independent Brownian particles ( N large) – empirical distr. ρ 0 ( x ) dx and ρ 1 ( y ) dy at t = 0 and t = 1 , resp. – ρ 0 and ρ 1 not compatible with transition mechanism � 1 ρ 1 ( y ) � = p ( t 0 , x, t 1 , y ) ρ 0 ( x ) dx, 0 where � � −| x − y | 2 p ( s, y, t, x ) = [2 π ( t − s )] − n 2 exp , s < t 2( t − s ) Particles have been transported in an unlikely way Schr¨ odinger (1931): Of the many unlikely ways in which this could have happened, which one is the most likely?

Large deviations formulation of SBP � � log dQ Minimize H ( Q, W ) = E Q dW over Q ∈ D ( ρ 0 , ρ 1 ) distributions on paths with marginals ρ ’s H ( · , · ) : relative entropy F¨ ollmer 1988: This is a problem of large deviations of the empirical distribution on path space connected through Sanov’s theorem to a maximum entropy problem .

Relative entropy w.r.t. Wiener measure dX = vdt + dB Girsanov: � � � t � � log dQ 1 � v � 2 ds E Q = E Q dW 2 0 is a quadratic cost!!!

SBP as a stochastic control problem � 1 � � v ( x, t ) � 2 ρ ( x, t ) dtdx, inf R n ( ρ,v ) 0 ∂ρ ∂t + ∇ · ( vρ ) = 1 2∆ ρ ρ ( x, 0) = ρ 0 ( x ) , ρ ( y, 1) = ρ 1 ( y ) . Blaqui` ere, Dai Pra, ... compare with OMT: � 1 � 1 2 � v ( x, t ) � 2 ρ ( x, t ) dtdx inf ( ρ,v ) 0 R n ∂ρ ∂t + ∇ · ( vρ ) = 0 ρ ( x, 0) = ρ 0 ( x ) , ρ ( y, 1) = ρ 1 ( y )

Fluid-dynamic formulation of SBP (time-symmetric) � 1 � � � � v ( x, t ) � 2 + � 1 2 ∇ log ρ ( x, t ) � 2 inf ρ ( x, t ) dtdx, R n ( ρ,v ) 0 ∂ρ ∂t + ∇ · ( vρ ) = 0 , ρ (0 , x ) = ρ 0 ( x ) , ρ (1 , y ) = ρ 1 ( y ) . 2 ∇ log ρ ( x, t ) � 2 : Fisher information , Nelson’s osmotic power � 1 Chen-Georgiou-Pavon, On the relation between optimal transport and Schr¨ odinger bridges: A stochastic control viewpoint, J. Opt. Theory Appl. , 2015 Mikami 2004, Mikami-Thieullen 2006,2008, L´ eonard 2012

Erwin Schr¨ odinger’s insight on SBP the density factors into ρ ( x, t ) = ϕ ( x, t ) ˆ ϕ ( x, t ) where ϕ and ˆ ϕ solve (Schr¨ odinger’s system): � ϕ ( x, t ) = p ( t, x, 1 , y ) ϕ ( y, 1) dy, ϕ ( x, 0) ˆ ϕ ( x, 0) = ρ 0 ( x ) � ϕ ( x, t ) = ˆ p (0 , y, t, x ) ˆ ϕ ( y, 0) dy, ϕ ( x, 1) ˆ ϕ ( x, 1) = ρ 1 ( x ) . compare with Ψ ¯ Ψ = ρ Existence and uniqueness for Schr¨ odinger’s system: Fortet 1940, Beurling 1960, Jamison 1974/75, F¨ ollmer 1988. ∼ Sinkhorn iteration & Quantum version: Georgiou-Pavon 2015

SBP schematic

SBP schematic

SBP schematic

SBP schematic

Schr¨ odinger system − ∂ϕ ∂t ( t, x ) = 1 2 ∆ ϕ ( t, x ) ∂ ˆ ϕ ∂t ( t, x ) = 1 2 ∆ ˆ ϕ ( t, x ) ϕ (0 , x ) ˆ ϕ (0 , x ) = ρ 0 ( x ) ϕ (1 , x ) ˆ ϕ (1 , x ) = ρ 1 ( x )

Existence & uniqueness (Sinkhorn scaling) − ∂ϕ ∂t ( t, x ) = 1 2 ∆ ϕ ( t, x ) ∆ / 2 − − − − − − → ϕ ˆ ϕ ˆ ∂ ˆ ϕ ∂t ( t, x ) = 1 � ↑ 2 ∆ ˆ ϕ ( t, x ) � ρ 0 � ρ 1 � ↓ · · − ∆ / 2 ϕ (0 , x ) ˆ ϕ (0 , x ) = ρ 0 ( x ) ← − − − − − − − ϕ ϕ ϕ (1 , x ) ˆ ϕ (1 , x ) = ρ 1 ( x ) iteration is contractive in the Hilbert metric! d H ( p, q ) = log M ( p, q ) m ( p, q ) inf { λ | p ≤ λq } M ( p, q ) := sup { λ | λq ≤ p } m ( p, q ) := Chen-Georgiou-Pavon, Entropic and displacement interpolation: a computational approach using the Hilbert metric, SIAM J. Appl. Math.

OMT as limit to SBP: numerics in general OMT interpolation: Marginal distributions ρ t + ∇ · ρv = 0 ρ t + ∇ · ρv = ǫ ∆ ρ , varying ǫ

Applications: Image interpolation Interpolation of 2D images to a 3D model:

LQG - covariance control �� T � � u ( t ) � 2 dt min , E u 0 s.t. dX = AXdt + Budt + BdW X (0) ∼ N (0 , Σ 0 ) , X ( T ) ∼ N (0 , Σ 1 ) ⇐ these are the ρ ’s Beghi (1996), Grigoriadis- Skelton (1997) Brockett (2007, 2012), Vladimirov-Petersen (2010, 2015)

Bridges - LQG - covariance control in general �� T � � u ( t ) � 2 dt min , E u 0 s.t. dX = AXdt + Budt + B 1 dW X (0) ∼ N (0 , Σ 0 ) , X ( T ) ∼ N (0 , Σ 1 ) connection with SBP ⇒ φ ( t, x ) = exp( −� x � 2 Q ( t ) − 1 ) & Riccati’s Chen-Georgiou-Pavon (TAC 2016)

SBP Riccati’s – nonlinearly coupled Riccati equations ≡ Schr¨ odinger system Π = − A ′ Π − Π A + Π BB ′ Π ˙ H = − A ′ H − H A − H BB ′ H ˙ � (Π + H) . � BB ′ − B 1 B ′ + (Π + H) 1 Σ − 1 = Π(0) + H(0) 0 Σ − 1 = Π( T ) + H( T ) . T φ ) ⇔ Σ − 1 = Π + H log( ρ ) = log( φ ) + log( ˆ Chen-Georgiou-Pavon, Optimal steering of a linear stochastic system to a final probability distribution, IEEE Trans. Aut. Control , May 2016

stationary SBP When can Σ be a stationary state-covariance for dx ( t ) = ( A − BK ) x ( t ) dt + B 1 dw ( t )? i.e., when is Σ = Exx ′ , for suitable choice of K ? – not all Σ can be realized by state feedback

stationary SBP When can Σ be a stationary state-covariance for dx ( t ) = ( A − BK ) x ( t ) dt + B 1 dw ( t )? This is so iff � 0 � A Σ + Σ A ′ + B 1 B ′ � � 1 B B rank = rank . B 0 B 0 – Chen-Georgiou-Pavon, Optimal steering..., Part II IEEE TAC , May 2016 – Georgiou, Structure of state covariances... TAC 2002 – recent work with Mihailo Jovanovic etal. on inverse problems, etc., 2016, 2017

stationary SBP Assuming � 0 � A Σ + Σ A ′ + B 1 B ′ � � B B 1 rank = rank , B 0 B 0 find K so that for u = − Kx and dx = ( A − BK ) xdt + B 1 dw , we have: Σ = Exx ′ and J power ( u ) := E {� u � 2 } is minimal Via semidefinite programming: – Chen-Georgiou-Pavon, Optimal steering..., Part II IEEE TAC , May 2016.

Application: Cooling Efficient steering from initial condition ρ 0 to ρ 1 at finite time – Efficient stationary state of stochastic oscillators to desired ρ 1 – thermodynamic systems, controlling collective response – magnetization distribution in NMR spectroscopy,.. - Chen-Georgiou-Pavon Fast cooling for a system of stochastic oscillators, J. Math. Phys. Nov. 2015.

Cooling (cont’d) Nyquist-Johnson noise driven oscillator Ldi L ( t ) = v C ( t ) dt − v C ( t ) dt − Ri L ( t ) dt + u ( t ) dt + dw ( t ) RCdv C ( t ) =

Cooling & keeping it cool ! Inertial particles with stochastic excitation trajectories in phase space transparent tube: “ 3 σ region”

Application: OMT with dynamics via SBP Schr¨ odinger bridge with ǫ = 9 Schr¨ odinger bridge with ǫ = 0 . 01 Schr¨ odinger bridge with ǫ = 4 Optimal transport with prior