Outline Outline Markov Processes Markov Processes Important - PowerPoint PPT Presentation

Outline Outline � � Markov Processes Markov Processes � Important Properties � Important Properties � Chapman � Chapman- -Kolmogorov Kolmogorov- -Equation Equation � Fokker � Fokker- -Planck Equation Planck Equation � Fokker � Fokker- -Planck Equation for Vector Planck Equation for Vector Markov Processes Markov Processes � Fokker � Fokker- -Planck Equation for Ito’s Planck Equation for Ito’s Equation Equation ME 529 - Stochastics G. Ahmadi ME 529 - Stochastics G. Ahmadi i) X(t) is also Markov in reverse: i) X(t) is also Markov in reverse: A stochastic process X(t) is called a Markov A stochastic process X(t) is called a Markov { ( ) ( ) } { ( ) ( ) } ≤ ≥ = ≤ P X t x | X t for all t t P X t x | X t process if for every n and any t process if for every n and any t 1 1 < t < t 2 2 < … < < … < t t n n , , 1 1 2 1 1 2 its conditional probability satisfies the its conditional probability satisfies the ii) The future is independent of the past under ii) The future is independent of the past under following condition following condition the given condition of the present. the given condition of the present. { ( ) ( ) ( ) ( ) } { ( ) ( ) } ≤ = ≤ P X t x | X t , X t ,..., X t P X t x | X t − − − iii) If for any t iii) If for any t 1 1 < t < t 2 2 , X(t , X(t 2 2 ) ) – – X(t X(t 1 1 ) is independent of ) is independent of n n n 1 n 2 1 n n n 1 ≤ t X(t) for t ≤ X(t) for t t 1 1 , the , the X(t X(t) is a Markov process. ) is a Markov process. or or Thus, independent increment processes Thus, independent increment processes . . { ( ) ( ) } { ( ) ( ) } ≤ ≤ = ≤ (Poisson, Wiener- (Poisson, Wiener -Levy) are Markov processes. Levy) are Markov processes. P X t x | X t for all t t P X t x | X t − − n n n 1 n n n 1 ME 529 - Stochastics G. Ahmadi ME 529 - Stochastics G. Ahmadi 1

{ ( ) ( ) ( ) } { ( ) ( ) } = iv) iv) E X t | X t ,..., X t E X t | X t vi) The following relation holds The following relation holds − − vi) n n 1 1 n n 1 ( ) ( ) ( ) ( ) st order equation, = ) is associated with a 1 st f x , t ; x , t ; x , t f x , t | x , t f x , t | x , t f x , t v) v) X(t X(t) is associated with a 1 order equation, X 1 1 2 2 3 3 X 3 3 2 2 X 2 2 1 1 X 1 1 dX ( ) ( ) ( ) dW ( ) ( ) ( ) ( ) = − β = = f x , t ;...; x , t f x , t | x , t ... f x , t | x , t f x , t X , t j t j t − − X 1 1 n n X n n . n 1 n 1 X 2 2 1 1 X 1 1 dt dt ( ) ( ) ( ) = = f x , t f x , t | x , 0 If then If then X 0 x ( ) ( ) ( ( ) ) ( ) with solution with solution t t ∫ ∫ = + β τ τ τ + τ τ X 1 1 X 1 1 o o X t X t X , d j d 0 t t A Markov Process is fully specified if: A Markov Process is fully specified if: 0 0 st order density and transition a) Given 1 st order density and transition a) Given 1 vi) Conditional probability density satisfies Conditional probability density satisfies vi) probability density; probability density; ( ) ( ) . = nd order density; f X x , t | x , t ;...; x , t ; x , t f x , t | x , t b) 2 nd order density; b) 2 − − − − n n n 1 n 1 2 2 1 1 X n n n 1 n 1 c) Transition density and X(0) c) Transition density and X(0) ME 529 - Stochastics G. Ahmadi ME 529 - Stochastics G. Ahmadi For Continuous Random Process For Continuous Random Process ∂ ∂ ∂ 2 f [ ] 1 [ ] ( ) ( ) = − α + α x , t f x , t f ∂ ∂ 1 ∂ 11 2 ( ) +∞ ( ) t x 2 x ∫ = f x , t | x , t f x , t ; x , t | x , t dx 0 0 1 1 0 0 1 − ∞ + ∞ ( ) ( ) ∫ = { } f x , t | x , t ; x , t f x , t | x , t dx ( ) 1 { ( ) ( ) } ( ) 1 [ ( ) ] ( ) α = = α = 2 = 1 1 0 0 1 1 0 0 1 x , t lim E dX t | X t x x , t lim E dX t | X t x − ∞ 1 11 → dt → dt dt 0 dt 0 For Markov Processes For Markov Processes Kolmogorov Equation Kolmogorov Equation ( ) ( ) = f x , t | x , t ; x , t f x , t | x , t ( ) α ∂ ∂ ∂ 1 1 0 0 1 1 2 f ( ) f x , t f + α + = 11 0 0 x , t 0 ∂ ∂ ∂ 1 0 0 2 t x 2 x ( ) +∞ ( ) ( ) ∫ 0 0 0 = f x , t | x , t f x , t | x , t f x , t | x , t dx 0 0 1 1 1 1 0 0 1 ( ) − ∞ f = f x , t | x 0 , t 0 ME 529 - Stochastics G. Ahmadi ME 529 - Stochastics G. Ahmadi 2

{ } { ( ) } ( ) = Recall Recall = ≥ Corresponding Fokker- Corresponding Fokker -Planck Equation Planck Equation 2 ⎧ ⎫ E W t 0 E W t 2 Dt 2 Dt t t ( ) = ⎨ 1 2 1 ⎬ R WW t , t ≥ 1 2 ⎩ 2 Dt t t ⎭ 2 1 2 ∂ ∂ ∂ ∂ ∂ ∂ 2 2 2 f 1 f f f ( ) ( ) { } = − α + α = = { ( ) ( ) } [ ( ) ( ) ] ( ) − = f f D D E W t W t 0 − 2 = − t > E W t W t 2 D t t t ∂ ∂ 1 ∂ 11 ∂ ∂ ∂ 2 2 2 2 1 2 1 2 1 t x 2 x x t W 2 1 ( ) ( ) = + ( ) ( ) Let Let t = = + − t t dt t dW W t dt W t = δ − For solution is For solution is f w , t | w , t w w 2 1 o o o o { } { } 2 = ( ) ( ) − 2 = w w E dW 0 E dW 2 Ddt − o ( ) − 4 D t t e o = f ( ) { } 1 { } 1 ( ) π − α = = α = 2 = 4 D t t lim E dW | W 0 lim E dW | W 2 D 1 11 o → dt → dt dt 0 dt 0 ME 529 - Stochastics G. Ahmadi ME 529 - Stochastics G. Ahmadi d X ( ) ( ) ( ) Ito’s Equation Ito’s Equation = + ⋅ , t , t t g X G x n or or ∂ ∂ [ ] ∂ [ ] 2 f n 1 ( ) ( ) dt ∑ ∑∑ = − α + α x , t f x , t f ∂ ∂ j ∂ ∂ ij t x 2 x x = j 1 i j j i j ( ) ( ) dW dX ( ) ( ) ( ) ∑ = + ⋅ = + i d , t dt , t g , t G , t n t X g X G X X X i ij j dt j { } 1 ( ) ( ) α = = Here n and W being vector white noise and lim E dX t | t Here n and W being vector white noise and X x j j → dt dt 0 Wiener processes with Wiener processes with { } { } { } ( ) ( ) ( ) { } = = + τ = δ τ 1 ( ) ( ) ( ) E n E dW 0 E n t n t 2 D α = = i i i j ij lim E dX t dX t | t X x ij i j → dt dt 0 { } = E dW dW 2 D dt i j ij ME 529 - Stochastics G. Ahmadi ME 529 - Stochastics G. Ahmadi 3

{ } 2 = ( ) dx ( ) ( ) dt dW Fokker- Fokker -Planck Equation for Ito’s Equation Planck Equation for Ito’s Equation Consider Consider E dW 2 Ddt = + g x , t G x , t dt { } { } 1 ( ) 1 = = = = X = a lim E dX | g x , t a lim E dX dX | X x x Fokker- -Planck Planck ∂ ∂ ∂ ( ) Fokker 2 f ( ) j j j ij i j → → dt dt = − + dt 0 dt 0 2 gf D G f ∂ ∂ ∂ 2 t x x ( ) ∑ ∑ ∑∑ = + + + 2 dX dX g g dt g G dW dt g G dW dt G G dW dW ( ) ⎡ ( ) ⎤ Stationary Stationary d d d i j i j i jk k j ik k ik j l k l − + = = − + = 2 2 gf D G f c 0 gf D G f 0 k k k l ⎢ ⎥ 1 ⎣ ⎦ dx dx dx ( ) ij = ∑∑ α = ⋅ ⋅ T 2 G G D 2 G D G ij ik j l k l Let = Let 2 dF g dF g G f F k l = = D F dx dx 2 F 2 G DG ∂ ∂ [ ] ∂ [ ( ) ] 2 f ∑ ( ) ∑∑ ( ) ( ) ⎧+ ⎫ ⎧ ⎫ = − + ⋅ ⋅ T g , t f f g x c g x x G D G x x ∫ ∫ = = ⎨ 1 ⎬ ⎨ 1 ⎬ ∂ ∂ j ∂ ∂ ij F C exp dx f 2 exp dx t x x x ( ) ( ) 1 1 2 2 ⎩ DG x ⎭ G ⎩ DG x ⎭ j j i j i j 0 0 1 1 ME 529 - Stochastics G. Ahmadi ME 529 - Stochastics G. Ahmadi Concluding Remarks Concluding Remarks � � Markov Processes Markov Processes � Important Properties � Important Properties � Chapman � Chapman- -Kolmogorov Kolmogorov- -Equation Equation � Fokker � Fokker- -Planck Equation Planck Equation � Fokker � Fokker- -Planck Equation for Vector Planck Equation for Vector Markov Processes Markov Processes � Fokker � Fokker- -Planck Equation for Ito’s Planck Equation for Ito’s Equation Equation ME 529 - Stochastics G. Ahmadi ME 529 - Stochastics G. Ahmadi 4