On the asymptotics of exit problem for controlled Markov diffusion - PowerPoint PPT Presentation

On the asymptotics of exit problem for controlled Markov diffusion processes with random jumps and vanishing diffusion terms CONTROL SYSTEMS AND THE QUEST FOR AUTONOMY A Symposium in Honor of Professor Panos J. Antsaklis University of Notre Dame October 27-28, 2018 Getachew K. Befekadu, Ph.D. Department of Electrical & Computer Engineering Morgan State University

Outline ◮ Introduction ◮ General objectives ◮ Part I - On the asymptotic estimates for exit probabilities ◮ Exit probabilities ◮ Connection with stochastic control problem ◮ Part II - Minimum exit rate problem for prescription opioid epidemic models ◮ On the minimum exit rate problem ◮ Connection with principal eigenvalue problem ◮ Further remarks

Introduction Consider the following n -dimensional process x ǫ ( t ) defined by dx ǫ ( t ) = F ( t , x ǫ ( t ) , y ǫ ( t )) dt (1) and an m -dimensional diffusion process y ǫ ( t ) obeying the following SDE dy ǫ ( t ) = f ( t , y ǫ ( t )) dt + √ ǫσ ( t , y ǫ ( t )) dw ( t ) , ( x ǫ ( s ) , y ǫ ( s )) = ( x , y ) , t ∈ [ s , T ] , (2) where ◮ ( x ǫ ( t ) , y ǫ ( t )) jointly defined an R ( n + m ) -valued Markov diffusion process, ◮ w ( t ) is a standard Wiener process in R m , ◮ the functions F and f are uniformly Lipschitz, with bounded first derivatives,

Introduction . . . ◮ σ ( t , y ) is an R m × m -valued Lipschitz continuous function such that a ( t , y ) = σ ( t , y ) σ T ( t , y ) is uniformly elliptic, i.e., a min | p | 2 < p · a ( t , y ) p < a max | p | 2 , p , y ∈ R m , ∀ t > 0 , for some a max > a min > 0, and ◮ ǫ is a small positive number representing the level of random perturbation. Remark (1) Note that the small random perturbation enters only in the second system and then passes to other system. As a result, the diffusion process ( x ǫ ( t ) , y ǫ ( t )) is degenerate, i.e., the associated backward operator is degenerate.

Introduction . . . Here, we distinguish two general problems: ◮ A direct problem : the study of asymptotic behavior for the diffusion process ( x ǫ ( t ) , y ǫ ( t )), as ǫ → 0, provided that some information about the deterministic coupled dynamical systems, i.e., x 0 ( t ) = F ( t , x 0 ( t ) , y 0 ( t )) , y 0 ( t ) = f ( t , y 0 ( t )) ˙ ˙ and the type of perturbation are known. ◮ An indirect problem : the study of the deterministic coupled dynamical systems, when the asymptotic behavior of the diffusion process ( x ǫ ( t ) , y ǫ ( t )) is known.

General objectives ◮ To provide a framework that exploits three way connections between: 1 (i) boundary value problems associated with certain second order linear PDEs, (ii) stochastic optimal control problems, and (iii) probabilistic interpretation of controlled principal eigenvalue problems. ◮ To provide additional results for stochastically perturbed dynamical systems with randomly varying intensities. Typical applications include : climate modeling [ Benzi et al. (1983); Berglund & Gentz (2002, 2006) ], electrical engineering [ Bobrovsky, Zakai & Zeitouni (1988); Zeitouni and Zakai (1992) ], molecular and cellular biology [ Holcman & Schuss (2015) ], mathematical finance [ Feng et al. (2010) ], and stochastic resonance [ H¨ aggi et al. (1998); Moss (1994) ]. General works include : [ Berglund & Gentz (2006); Freidlin & Wentzell (1998); Olivieri & Vares (2005) ]. 1 G. K. Befekadu & P. J. Antsaklis , On the asymptotic estimates for exit probabilities and minimum exit rates of diffusion processes pertaining to a chain of distributed control systems, SIAM J. Contr. Opt., 53 (2015) 2297-2318.

Part I - Asymptotic estimates for exit probabilities Let D ⊂ R n be a bounded open domain with smooth boundary ∂ D . Let τ ǫ D be the exit time for the process x ǫ ( t ) from D � � � x ǫ ( t ) ∈ ∂ D τ ǫ � D = inf t > s . For a given T > 0, define the exit probability as � � q ǫ � = P ǫ τ ǫ � s , x , y D ≤ T , s , x , y where the probability P ǫ s , x , y is conditioned on ( x , y ) ∈ D × R m . Important : Note that the solution q ǫ � � s , x , y , as ǫ → 0, strongly depends on the behavior of the trajectories for the corresponding deterministic coupled dynamical systems, i.e., x 0 ( t ) = F ( t , x 0 ( t ) , y 0 ( t )) ˙ y 0 ( t ) = f ( t , y 0 ( t )) , ( x 0 (0) , y 0 (0)) = ( x , y ) . ˙ ( a ) ( b ) ( c )

Exit probabilities . . . � x ǫ ( t ) , y ǫ ( t ) � The backward operator for the process , when applied � � to a certain smooth function ψ s , x , y , is given by + ǫ � �� + L ǫ ψ � � � � � ψ s � � � � � ψ s s , x , y s , x , y s , x , y 2 tr a s , y ψ yy s , x , y � � � � + � F s , x , y , ψ x s , x , y � � � � � + � f s , y , ψ y s , x , y � , (3) where L ǫ is a second-order elliptic operator, i.e., � ǫ � �� L ǫ � ▽ 2 � � � � � � � , ▽ x � �� � � � , ▽ y � �� · 2 tr a s , y · + F s , x , y · + f s , y · yy and a ( s , y ) = σ ( s , y ) σ T ( s , y ) .

Exit probabilities . . . Let Q be an open set given by Q = (0 , T ) × D × R m . Assumption (1) (a) The function F is a bounded C ∞ ( Q 0 ) -function, with bounded first derivative, where Q 0 = (0 , ∞ ) × R n × R m . Moreover, f , σ and σ − 1 are bounded C ∞ � (0 , ∞ ) × R m � -functions, with bounded first derivatives. (b) The backward operator in Eq (3) is hypoelliptic in C ∞ ( Q 0 ) (which is also related to an appropriate H¨ ormander condition). (c) Let n ( x ) be the outer normal vector to ∂ D. Furthermore, let Γ + and Γ 0 denote the sets of points ( t , x , y ) , with x ∈ ∂ D, is positive and zero, respectively. 2 � � such that F ( t , x , y ) , n ( x ) 2 Note that ∈ Γ + ∪ Γ 0 � �� � P ǫ τ ǫ D , x ǫ ( τ ǫ D ) , y ǫ ( τ ǫ � τ ǫ � D ) D < ∞ = 1 , ∀ s , x , y ∈ Q . � s , x , y

Exit probabilities . . . Consider the following boundary value problem � � + L ǫ ψ � � Q = (0 , T ) × D × R m  s , x , y s , x , y = 0 in ψ s  Γ + � � ψ s , x , y = 1 on  (4) T � � { T } × D × R m s , x , y = 0 on ψ �� � � 0 < s ≤ T where Γ + ∈ Γ + � � T = s , x , y . Then, we have the following result for the exit probability. Proposition (1) Suppose that the statements (a)–(c) in the above assumption (i.e., Assumption (1)) hold true. Then, the exit probability q ǫ ( s , x , y ) = P ǫ � τ ǫ � D ≤ T is a smooth solution to the above boundary s , x , y value problem in Eq (4) . Moreover, it is a continuous function on Q ∪ { T } × D × R m .

Exit probabilities . . . Proof : Involves introducing a non-degenerate diffusion process 3 √ dx ǫ,δ ( t ) = F ( t , x ǫ,δ ( t ) , y ǫ ( t )) dt + δ dV ( t ) dy ǫ ( t ) = f ( t , y ǫ ( t )) dt + √ ǫσ ( t , y ǫ ( t )) dw ( t ) , with V is a standard Wiener process in R n and independent to W . Then, using the following statements � �  � x ǫ,δ ( r ) − x ǫ ( r ) ( i ) sup � → 0 � �    s ≤ r ≤ T as δ → 0 , P − almost surely . , τ ǫ,δ → τ ǫ ( ii ) D D  x ǫ,δ ( τ ǫ,δ D ) → x ǫ ( τ ǫ  ( iii ) D )  and the hypoellipticity assumption. We can relate the exit probability of the process ( x ǫ,δ ( t ) , y ǫ ( t )) with the boundary value problem in Eq (4). 3 G. K. Befekadu & P. J. Antsaklis , On the asymptotic estimates for exit probabilities and minimum exit rates of diffusion processes pertaining to a chain of distributed control systems, SIAM J. Contr. Opt., vol. 53 (4), pp. 2297–2318, 2015.

Connection with stochastic control problems Consider the following boundary value problem g ǫ s + ǫ a g ǫ + � F , g ǫ x � + � f , g ǫ � � � 2 tr y � = 0 in Q yy (5) � � �� g ǫ = E ǫ − 1 exp ǫ Φ on ∂ ∗ Q s , x , y � � where Φ s , x , y is bounded, nonnegative Lipschitz such that ∈ Γ + � � � � Φ s , x , y = 0 , ∀ s , x , y T . Introduce the following logarithm transformation J ǫ � = − ǫ log g ǫ � � � s , x , y s , x , y . s Then, J ǫ � � s , x , y satisfies the following HJB equation s + ǫ � � + F T · J ǫ a J ǫ,ℓ 0 = J ǫ s , y , J ǫ � � 2 tr x + H in Q , (6) y x ǫ, 1 x ǫ, 1 where y − 1 T · a ( s , y ) J ǫ � s , y , J ǫ � = f T ( s , y ) · J ǫ 2 J ǫ H y . y y

Connection with stochastic control problems . . . Then, we see that J ǫ � � s , x , y is a solution for the DP equation in Eq (6), which is associated to the following stochastic control problem �� θ J ǫ � E ǫ s , y ǫ ( t ) , ˆ � � � s , x , y = inf L u ( t ) dt s , x , y u ∈ ˆ ˆ U ( s , x , y ) s � θ, x ǫ ( θ ) , y ǫ ( θ ) � � + Φ with the SDE dx ǫ ( t ) = F t , x ǫ ( t ) , y ǫ ( t ) � �  dt u ( t ) dt + √ ǫ σ  dy ǫ ( t ) = ˆ � t , y ǫ ( t ) � dW ( t ) ( x ǫ ( s ) , y ǫ ( s )) = ( x , y ) , s ≤ t ≤ T  where ˆ U ( s , x , y ) is a class of (non-anticipatory) continuous functions ∈ Γ + θ, x ǫ ( θ ) , y ǫ ( θ ) � � for which θ ≤ T and T .