Stochastic Processes MATH5835, P. Del Moral UNSW, School of - PowerPoint PPT Presentation

Stochastic Processes MATH5835, P. Del Moral UNSW, School of Mathematics & Statistics Lectures Notes 2 Consultations (RC 5112): Wednesday 3.30 pm � 4.30 pm & Thursday 3.30 pm � 4.30 pm 1/34

2/34

Citation of the day As far as the laws of mathematics refer to reality, they are not certain, 3/34

Citation of the day As far as the laws of mathematics refer to reality, they are not certain, and as far as they are certain, they do not refer to reality. – Albert Einstein (1879-1955) 3/34

Citation of the day As far as the laws of mathematics refer to reality, they are not certain, and as far as they are certain, they do not refer to reality. – Albert Einstein (1879-1955) personal question X random variable ⇔ Law ( X ) = certain ?? 3/34

Citation of the day As far as the laws of mathematics refer to reality, they are not certain, and as far as they are certain, they do not refer to reality. – Albert Einstein (1879-1955) personal question X random variable ⇔ Law ( X ) = certain ?? Mathematics is a game played according to certain simple rules with meaningless marks on paper. – Hilbert, David (1862-1943) 3/34

Some basic notation 4/34

Some basic notation � ∀ 1 ≤ i ≤ d P ( Y = j ) = P ( X = i ) P ( Y = j | X = i ) � �� � � �� � � �� � 1 ≤ i ≤ d = p Y ( j ) = p X ( i ) = M ( i , j ) 4/34

Some basic notation � ∀ 1 ≤ i ≤ d P ( Y = j ) = P ( X = i ) P ( Y = j | X = i ) � �� � � �� � � �� � 1 ≤ i ≤ d = p Y ( j ) = p X ( i ) = M ( i , j ) � Matrix notation: p Y = [ P ( Y = 1) , . . . , P ( Y = d )]   P ( Y = 1 | X = 1) P ( Y = 2 | X = 1)) . . . P ( Y = d | X = 1)     P ( Y = 1 | X = 2) P ( Y = 2 | X = 2)) . . . P ( Y = d | X = 2)     = [ P ( X = 1) , . . . , P ( X = d )] × . . . .   � �� � . . . .   . . . .   = p X P ( Y = 1 | X = d ) P ( Y = 2 | X = d )) . . . P ( Y = d | X = d ) � �� � M =( M ( i , j )) i , j 4/34

Some basic notation � ∀ 1 ≤ i ≤ d P ( Y = j ) = P ( X = i ) P ( Y = j | X = i ) � �� � � �� � � �� � 1 ≤ i ≤ d = p Y ( j ) = p X ( i ) = M ( i , j ) � Matrix notation: p Y = [ P ( Y = 1) , . . . , P ( Y = d )]   P ( Y = 1 | X = 1) P ( Y = 2 | X = 1)) . . . P ( Y = d | X = 1)     P ( Y = 1 | X = 2) P ( Y = 2 | X = 2)) . . . P ( Y = d | X = 2)     = [ P ( X = 1) , . . . , P ( X = d )] × . . . .   � �� � . . . .   . . . .   = p X P ( Y = 1 | X = d ) P ( Y = 2 | X = d )) . . . P ( Y = d | X = d ) � �� � M =( M ( i , j )) i , j � Matrix synthetic notation: p Y = p X M 4/34

Some basic notation � E ( f ( Y ) | X = i ) = P ( Y = j | X = i ) f ( j ) � �� � 1 ≤ j ≤ d = M ( i , j ) 5/34

Some basic notation � E ( f ( Y ) | X = i ) = P ( Y = j | X = i ) f ( j ) � �� � 1 ≤ j ≤ d = M ( i , j ) � Matrix notation:       P ( Y = 1 | X = 1) P ( Y = 2 | X = 1)) . . . P ( Y = d | X = 1) f (1) E ( f ( Y ) | X = 1)             P ( Y = 1 | X = 2) P ( Y = 2 | X = 2)) P ( Y = d | X = 2) E ( f ( Y ) | X = 2)  . . .   f (2)          =  . . . .   .   .  . . . . . .       . . . . . .       P ( Y = 1 | X = d ) P ( Y = 2 | X = d )) . . . P ( Y = d | X = d ) f ( d ) E ( f ( Y ) | X = d ) � �� � � �� � � �� � M =( M ( i , j )) i , j = f = M ( f ) 5/34

Some basic notation � E ( f ( Y ) | X = i ) = P ( Y = j | X = i ) f ( j ) � �� � 1 ≤ j ≤ d = M ( i , j ) � Matrix notation:       P ( Y = 1 | X = 1) P ( Y = 2 | X = 1)) . . . P ( Y = d | X = 1) f (1) E ( f ( Y ) | X = 1)             P ( Y = 1 | X = 2) P ( Y = 2 | X = 2)) P ( Y = d | X = 2) E ( f ( Y ) | X = 2)  . . .   f (2)          =  . . . .   .   .  . . . . . .       . . . . . .       P ( Y = 1 | X = d ) P ( Y = 2 | X = d )) . . . P ( Y = d | X = d ) f ( d ) E ( f ( Y ) | X = d ) � �� � � �� � � �� � M =( M ( i , j )) i , j = f = M ( f ) � Matrix synthetic notation: E ( f ( Y ) | X = i ) = M ( f )( i ) 5/34

Some basic notation Markov chain = ”sequence of r.v.” X 0 � X 1 � . . . � X n − 1 � X n 6/34

Some basic notation Markov chain = ”sequence of r.v.” X 0 � X 1 � . . . � X n − 1 � X n � � P ( X n = j ) = P ( X n − 1 = i ) P ( X n = j | X n − 1 = i ) � �� � � �� � � �� � 1 ≤ i ≤ d = p Xn ( j ) = p Xn − 1 ( i ) = M n ( i , j ) 6/34

Some basic notation Markov chain = ”sequence of r.v.” X 0 � X 1 � . . . � X n − 1 � X n � � P ( X n = j ) = P ( X n − 1 = i ) P ( X n = j | X n − 1 = i ) � �� � � �� � � �� � 1 ≤ i ≤ d = p Xn ( j ) = p Xn − 1 ( i ) = M n ( i , j ) � Matrix synthetic notation: p X n = p X n − 1 M n = . . . = p X 0 M 1 M 2 . . . M n 6/34

Some basic notation   M n ( f )( X n − 1 ) � �� �   E ( f ( X n ) | X 0 = i ) = E E ( f ( X n ) | X n − 1 ) | X 0 = i   E ( M n ( f )( X n − 1 ) | X 0 = i ) = 7/34

Some basic notation   M n ( f )( X n − 1 ) � �� �   E ( f ( X n ) | X 0 = i ) = E E ( f ( X n ) | X n − 1 ) | X 0 = i   E ( M n ( f )( X n − 1 ) | X 0 = i ) = E ( M n − 1 ( M n ( f )) ( X n − 2 ) | X 0 = i ) = 7/34

Some basic notation   M n ( f )( X n − 1 ) � �� �   E ( f ( X n ) | X 0 = i ) = E E ( f ( X n ) | X n − 1 ) | X 0 = i   E ( M n ( f )( X n − 1 ) | X 0 = i ) = E ( M n − 1 ( M n ( f )) ( X n − 2 ) | X 0 = i ) = = . . . 7/34

Some basic notation   M n ( f )( X n − 1 ) � �� �   E ( f ( X n ) | X 0 = i ) = E E ( f ( X n ) | X n − 1 ) | X 0 = i   E ( M n ( f )( X n − 1 ) | X 0 = i ) = E ( M n − 1 ( M n ( f )) ( X n − 2 ) | X 0 = i ) = = . . . E (( M 1 . . . ( M n ( f ))) ( X 0 ) | X 0 = i ) = 7/34

Some basic notation   M n ( f )( X n − 1 ) � �� �   E ( f ( X n ) | X 0 = i ) = E E ( f ( X n ) | X n − 1 ) | X 0 = i   E ( M n ( f )( X n − 1 ) | X 0 = i ) = E ( M n − 1 ( M n ( f )) ( X n − 2 ) | X 0 = i ) = = . . . E (( M 1 . . . ( M n ( f ))) ( X 0 ) | X 0 = i ) = = ( M 1 M 2 . . . M n )( f )( i ) 7/34

Stabilizing populations - Migration processes ◮ 193 countries (UN report 2013) c i , i = 1 , . . . , 193. ◮ q n ( i ) = average-population of country c i at some time n (years/months/...). ◮ M n ( i , j ) = proportions of migrants from c i to c j at time n . Some questions: ◮ Stabilization ∃ ? q ∞ ( i ) invariant w.r.t. migration process ◮ Chance for two migrants to meet in some country? 8/34

Migration - Stochastic process individuals � �� � i , n , . . . , I m n ( i ) I 1 i , n , I 2 i , n , I 3 { } = Country c i at time n with pop. m n ( i ) i , n During the migration process i , n chooses the index � Each I k I k i , n = j of a country c j ∼ M n ( i , j ) Simulation? 9/34

Migration - Stochastic process individuals � �� � i , n , . . . , I m n ( i ) I 1 i , n , I 2 i , n , I 3 { } = Country c i at time n with pop. m n ( i ) i , n During the migration process i , n chooses the index � Each I k I k i , n = j of a country c j ∼ M n ( i , j ) Simulation? 9/34

Migration - Stochastic process individuals � �� � i , n , . . . , I m n ( i ) I 1 i , n , I 2 i , n , I 3 { } = Country c i at time n with pop. m n ( i ) i , n During the migration process i , n chooses the index � Each I k I k i , n = j of a country c j ∼ M n ( i , j ) Simulation? � � � � I k m ( n + 1 ) ( i , j ) = 1 j = Migrants i � j i , n 1 ≤ k ≤ m n ( i ) 9/34

Migration - Stochastic process individuals � �� � i , n , . . . , I m n ( i ) I 1 i , n , I 2 i , n , I 3 { } = Country c i at time n with pop. m n ( i ) i , n During the migration process i , n chooses the index � Each I k I k i , n = j of a country c j ∼ M n ( i , j ) Simulation? � � � � I k m ( n + 1 ) ( i , j ) = 1 j = Migrants i � j i , n 1 ≤ k ≤ m n ( i ) ⇓ � m ( n + 1 ) ( j ) = m ( n + 1 ) ( i , j ) 1 ≤ i ≤ 193 9/34

Migration - Stochastic process individuals � �� � i , n , . . . , I m n ( i ) I 1 i , n , I 2 i , n , I 3 { } = Country c i at time n with pop. m n ( i ) i , n During the migration process i , n chooses the index � Each I k I k i , n = j of a country c j ∼ M n ( i , j ) Simulation? � � � � I k m ( n + 1 ) ( i , j ) = 1 j = Migrants i � j i , n 1 ≤ k ≤ m n ( i ) ⇓ � m ( n + 1 ) ( j ) = m ( n + 1 ) ( i , j ) If no birth & death! 1 ≤ i ≤ 193 9/34