Controlled McKean-Vlasov Equations and Related Master Equations Cong Wu Department of Mathematics, University of Southern California March 24, 2017 Cong Wu Controlled McKean-Vlasov Equations and Related Master Equations
Introduction Consider n -player mean-field game: dX i t = b ( t , X i , µ n , α i t ) dt + σ ( t , X i , µ n , α i t ) dB i t � n with empirical distribution µ n = 1 j =1 δ X j . n Question: What kind of information should the controls α i use? Strong formulation: α i depend on the random noise B i . But W is usually unobservable in practice. Weak formulation: α i depend on state process X i , which is observable. Markov? non-Markov? Since players have freedom to use past information, non-Markov seems more reasonable in prictice. Cong Wu Controlled McKean-Vlasov Equations and Related Master Equations
Two different problems As n → ∞ , we get SDE of McKean-Vlasov type: dX t = b ( t , X ·∧ t , µ [0 , t ] , α t ) dt + σ ( t , X ·∧ t , µ [0 , t ] , α t ) dB t � T with objective J ( α, µ ) = E [ 0 f ( t , X ·∧ t , µ [0 , t ] , α t ) dt + g ( X , µ )] Mean field game problem: Find α ⋆ so that J ( α ⋆ , µ α ⋆ ) = sup α J ( α, µ α ⋆ ) Fixed point problem: µ → α ⋆ → µ α ⋆ Use fixed point to find approximate equilibrium of finite-player game (work done by Carmona, Lacker 2015) Stochastic control problem of McKean-Vlasov type: Find α ⋆ so that J ( α ⋆ , µ α ⋆ ) = sup α J ( α, µ α ) Non-standard control problem → our topic Cong Wu Controlled McKean-Vlasov Equations and Related Master Equations
Problem formulation Let Ω := C ([0 , T ] , R ) be endowed the uniform norm � · � ∞ , X t ( ω ) := ω t , F t := F X and t � � � � P 2 (Ω) := P ∈ P (Ω) � � X � ∞ is square integrable under P By DCT, it easy to see the following characterization: marginal µ t ’s are square integrable and � � n − 1 �� � 2 � � E µ � � � X · − X t i 1 [ t i , t i +1 ) ( · )+ X T 1 { T } ( · ) → 0 µ ∈ P 2 (Ω) ⇔ � ∞ i =0 as | p | → 0, for p : 0 = t 0 < t 1 < · · · < t n = T Cong Wu Controlled McKean-Vlasov Equations and Related Master Equations
Wasserstein metric The Wasserstein metric on P 2 (Ω) is � � 1 E P � X ′ − X ′′ � 2 W 2 ( µ, ν ) := inf 2 ∞ P ∈ Γ( µ,ν ) Let P [0 , t ] := P ◦ ( X ·∧ t ) − 1 , Λ := [0 , T ] × P 2 (Ω). Define pseudometric on Λ as � | t − s | + W 2 ( µ [0 , t ] , ν [0 , s ] ) 2 � 1 2 W 2 (( t , µ ) , ( s , ν )) := We say function F : Λ → R is adapted if F ( t , µ ) = F ( t , µ [0 , t ] ) Cong Wu Controlled McKean-Vlasov Equations and Related Master Equations
Control problem Consider the following simplified version of optimal control problem of McKean-Vlasov type (no dependence on state): g ( P t ,µ,α ) V ( t , µ ) := sup α ∈A t where P t ,µ,α is the law of solution of McKean-Vlasov equation � s � s X t ,µ,α = ξ t + b ( r , L X t ,µ,α , α r ) dr + σ ( r , L X t ,µ,α , α r ) dB r ( ⋆ ) s ·∧ r ·∧ r t t and X ·∧ t = ξ ·∧ t with process ξ ∼ µ . Above SDE is wellposed when the usual Lipschitz condition for b , σ holds and α is any fixed open loop control. Cong Wu Controlled McKean-Vlasov Equations and Related Master Equations
Strong formulation: a filtration issue When α is open-loop, we are in the strong formulation. There are two choices for admissible controls: In A 1 t , α s = α ( s , ( B r − B s ) t ≤ r ≤ s ) In A 2 α s = α ( s , ( B r ) 0 ≤ r ≤ s ) t , On one hand, we cannot establish a weak solution for the master equation from any of A 1 , A 2 alone. On the other hand, we don’t know if g ( P t ,µ,α ) ? g ( P t ,µ,α ) sup = sup α ∈A 1 α ∈A 2 t t Even the fact that the value function V ( t , µ ) is well defined under A 2 is nontrivial! Cong Wu Controlled McKean-Vlasov Equations and Related Master Equations
Weak formulation When α is closed-loop, SDE ( ⋆ ) may not be well posed even in the weak sense unless we assume regularity in α . Study of wellposedness of weak solution of MKV SDE ( ⋆ ) is difficult even if α is of feedback form. Existing works always assume no volatility control. See Carmona, Lacker (2015); Li, Hui (2016). However, if α ∈ F X is assumed to be piecewise constant, then ( ⋆ ) is well posed in the strong sense. If σ � = 0, then B ∈ F X . So X , B , P t ,µ,α can be constructed on Ω = C ([0 , T ]). Cong Wu Controlled McKean-Vlasov Equations and Related Master Equations
Motivating example Warning: When set A t is too small, continuity of V may fail. For example, let g ( L X ) = 1 dX t = (1 + α 2 3 E [ X 4 1 ] − ( E [ X 2 1 ]) 2 , t ∧ 1) dB t , and A t consists only of constant controls, then V (0 , 1 2( δ ε + δ − ε )) ≥ 9 4 � = 0 = V (0 , δ 0 ) lim ε → 0 Cong Wu Controlled McKean-Vlasov Equations and Related Master Equations
Main results Assumption b , σ, g are bounded, (uniformly) Lipschitz continuous, and σ > 0 . Theorem Under the above Assumption and let � � � n − 1 � � � A t := � α s ( X ) = h i ( X [0 , t i ] )1 [ t i , t i +1 ) ( s ) , h i ’s are bdd. meas. α , i =0 then V ( t , µ ) is Lipschitz continuous in µ , uniformly in t, under W 2 . Note it is implicit that functions h i could also depend on the (deterministic) law of X [0 , t i ] . Cong Wu Controlled McKean-Vlasov Equations and Related Master Equations
Technical lemma Lemma Fix µ, ν ∈ P 2 , π ∈ Γ( µ, ν ) . For any ε, δ > 0 and process ( η s ) 0 ≤ s ≤ T defined on a rich enough probability space with L η = ν and partition 0 ≤ t 1 < · · · < t m ≤ T, there exist another process ( ξ s ) 0 ≤ s ≤ T and Brownian motion ( B s ) 0 ≤ s ≤ δ such that: (i) L ξ = µ , (ii) η ⊥ B, (iii) ξ t i ∈ σ ( η t 1 , ··· , t m , B [0 ,δ ] ) and (iv) � � � 1 2 | ω ′ t j − ω ′′ t j | 2 d π ( ω ′ , ω ′′ ) W 2 ( L ξ t 1 , ··· , tm , L η t 1 , ··· , tm ) ≤ max + ε. j Ω × Ω The key part of this result is (iii); otherwise it is trivial. This result relies on the fact that any random vector can be constructed from i.i.d. U (0 , 1) random variables and its multivariate distribution function. Cong Wu Controlled McKean-Vlasov Equations and Related Master Equations
Change admissible controls Proposition All the following four cases define the same value function V ( t , µ ) : (i) α s ( X ) = h i ( X [0 , t i ] ) , h i ’s are bounded measurable; (ii) α s ( X ) = h i ( X [0 , t i ] ) , h i ’s are bounded continuous; (iii) α s ( X ) = h i ( X s 1 , ··· , s m , X [ t , t i ] ) , h i ’s are bounded measurable; (iv) α s ( X ) = h i ( X s 1 , ··· , s m , X [ t , t i ] ) , h i ’s are bounded continuous; Cong Wu Controlled McKean-Vlasov Equations and Related Master Equations
DPP Under weak formulation, DPP follows quite easily. Theorem (Dynamic Programming Principle) V ( s , P t ,µ,α ) , V ( t , µ ) = sup ∀ s > t α ∈A t By DPP, we immediately see that Proposition Value function V : Λ → R is Lipschitz continuous under W 2 . Question: What kind of master equation is satisfied by this continuous value function on [0 , T ] × P 2 (Ω)? Cong Wu Controlled McKean-Vlasov Equations and Related Master Equations
Differentiation Right now it is still not known to us how to define the proper derivatives in Λ, but here are several ideas from earlier works. In the Markovian case, V becomes a function on [0 , T ] × P 2 ( R ) and P.L. Lions studied how to define derivatives ∂ µ V through Frech´ et derivative of lifted function V on L 2 (Ω; R ). It turns out D � � V ( t , ξ ) = h ( t , µ, ξ ) , ξ ∼ µ for some deterministic function h defined on [0 , T ] × P 2 ( R ) × R . Generalized Itˆ o’s formula in this case was also proved by Carmona, Delarue (2014); Chassagneux, Crisan, Delarue (2014). Question: Does this form generalize to non-Markovian case? Since our value function V satisfies the adaptedness property, it’s also possible for us to borrow the idea of Functional Itˆ o Calculus. Cong Wu Controlled McKean-Vlasov Equations and Related Master Equations
Generalized Itˆ o’s formula: Markovian case Fact: Most functions on L 2 (Ω; R ) are not twice Frech´ et differentiable, e.g. f ( X ) = E [sin( X )], then Df ( X ) = cos( X ) is not Frech´ et differentiable. But for Itˆ o’s formula, we only need directional derivatives (i.e. Gˆ ateaux derivative) to exist. Theorem (Itˆ o’s formula) � T 0 | b t | 2 + | σ t | 4 dt ] ≤ ∞ Suppose dX t = b t dt + σ t dB t such that E [ and f ∈ C 2 b ( P 2 ( R )) , then � t � � ∂ µ f ( L X s , X s ) b s + 1 2 ∂ x ∂ µ f ( L X s , X s ) σ 2 f ( L X t ) = f ( L X 0 ) + ds E s 0 Note that derivative ∂ µ ∂ µ f is not involved in above formula, so we only need ”partial regularity” on f , see Chassagneux, Crisan, Delarue (2014). Cong Wu Controlled McKean-Vlasov Equations and Related Master Equations
Recommend
More recommend
