Mixing in Product Spaces Elchanan Mossel Elchanan Mossel Mixing in - PowerPoint PPT Presentation

Mixing in Product Spaces Elchanan Mossel Elchanan Mossel Mixing in Product Spaces

Poincar´ e Recurrence Theorem Theorem (Poincar´ e, 1890) Let f : X → X be a measure preserving transformation. Let E ⊂ X measurable. Then P [ x ∈ E : f n ( x ) / ∈ E , n > N ( x )] = 0 Elchanan Mossel Mixing in Product Spaces

Poincar´ e Recurrence Theorem Theorem (Poincar´ e, 1890) Let f : X → X be a measure preserving transformation. Let E ⊂ X measurable. Then P [ x ∈ E : f n ( x ) / ∈ E , n > N ( x )] = 0 One of the first results in Ergodic Theory. Long term mixing. This talk is about short term mixing. Elchanan Mossel Mixing in Product Spaces

Finite Markov Chains As a first example consider a Finite Markov chain . Let M be a k × k doubly stochastic symmetric matrix . Pick X 0 uniformly at random from 1 , . . . , k . Given X i = a , let X i + 1 = b with probability M a , b . Elchanan Mossel Mixing in Product Spaces

Finite Markov Chains As a first example consider a Finite Markov chain . Let M be a k × k doubly stochastic symmetric matrix . Pick X 0 uniformly at random from 1 , . . . , k . Given X i = a , let X i + 1 = b with probability M a , b . Theorem (Long Term Mixing for Markov Chains) Suppose that other than 1 , all eigenvalues λ i of M satisfy | λ i | ≤ λ < 1 . Then for any two sets A , B ⊂ [ k ] , it holds that � � � P [ X 0 ∈ A , X t ∈ B ] − P [ A ] P [ B ] � ≤ λ t � � Elchanan Mossel Mixing in Product Spaces

Short Term Mixing for Markov Chains Theorem � � � P [ X 0 ∈ A , X 1 ∈ B ] − P [ A ] P [ B ] � is upper bounded by � � � λ P [ A ]( 1 − P [ A ]) P [ B ]( 1 − P [ B ]) Shows: mixing in one step for large sets. Elchanan Mossel Mixing in Product Spaces

Short Term Mixing for Markov Chains Theorem � � � P [ X 0 ∈ A , X 1 ∈ B ] − P [ A ] P [ B ] � is upper bounded by � � � λ P [ A ]( 1 − P [ A ]) P [ B ]( 1 − P [ B ]) Shows: mixing in one step for large sets. Proof: 1 A = P [ A ] 1 + f , 1 B = P [ B ] 1 + g , where f , g ⊥ 1 Elchanan Mossel Mixing in Product Spaces

Short Term Mixing for Markov Chains Theorem � � � P [ X 0 ∈ A , X 1 ∈ B ] − P [ A ] P [ B ] � is upper bounded by � � � λ P [ A ]( 1 − P [ A ]) P [ B ]( 1 − P [ B ]) Shows: mixing in one step for large sets. Proof: 1 A = P [ A ] 1 + f , 1 B = P [ B ] 1 + g , where f , g ⊥ 1 1 P [ X 0 ∈ A , X 1 ∈ B ] = k ( P [ A ] 1 + f ) t M ( P [ B ] 1 + g ) P [ A ] P [ B ] + 1 k f t Mg , = 1 k | f t Mg | ≤ λ � f � 2 � g � 2 = λ � P [ A ]( 1 − P [ A ]) P [ B ]( 1 − P [ B ]) Elchanan Mossel Mixing in Product Spaces

Short Term Mixing for Markov Chains Theorem � � � P [ X 0 ∈ A , X 1 ∈ B ] − P [ A ] P [ B ] � is upper bounded by � � � λ P [ A ]( 1 − P [ A ]) P [ B ]( 1 − P [ B ]) Shows: mixing in one step for large sets. Proof: 1 A = P [ A ] 1 + f , 1 B = P [ B ] 1 + g , where f , g ⊥ 1 1 P [ X 0 ∈ A , X 1 ∈ B ] = k ( P [ A ] 1 + f ) t M ( P [ B ] 1 + g ) P [ A ] P [ B ] + 1 k f t Mg , = 1 k | f t Mg | ≤ λ � f � 2 � g � 2 = λ � P [ A ]( 1 − P [ A ]) P [ B ]( 1 − P [ B ]) Also called Expander Mixing Lemma . Used a lot in computer science, e.g. in (de)randomization. Elchanan Mossel Mixing in Product Spaces

The tensor property Consider ( Y 1 , Z 1 ) , . . . , ( Y n , Z n ) which are drawn independently from the distribution of ( X 0 , X 1 ) . Equivalently, the transition matrix from Y = ( Y 1 , . . . , Y n ) to Z = ( Z 1 , . . . , Z n ) is M ⊗ n . Elchanan Mossel Mixing in Product Spaces

The tensor property Consider ( Y 1 , Z 1 ) , . . . , ( Y n , Z n ) which are drawn independently from the distribution of ( X 0 , X 1 ) . Equivalently, the transition matrix from Y = ( Y 1 , . . . , Y n ) to Z = ( Z 1 , . . . , Z n ) is M ⊗ n . ⇒ that for any sets A , B ⊂ [ k ] n : Thm = � � � � P [ Y ∈ A , Z ∈ B ] − P [ A ] P [ B ] � ≤ λ P [ A ]( 1 − P [ A ]) P [ B ]( 1 − P [ B ]) � � Elchanan Mossel Mixing in Product Spaces

The tensor property Consider ( Y 1 , Z 1 ) , . . . , ( Y n , Z n ) which are drawn independently from the distribution of ( X 0 , X 1 ) . Equivalently, the transition matrix from Y = ( Y 1 , . . . , Y n ) to Z = ( Z 1 , . . . , Z n ) is M ⊗ n . ⇒ that for any sets A , B ⊂ [ k ] n : Thm = � � � � P [ Y ∈ A , Z ∈ B ] − P [ A ] P [ B ] � ≤ λ P [ A ]( 1 − P [ A ]) P [ B ]( 1 − P [ B ]) � � Follows immediately from tensorization of the spectrum. Elchanan Mossel Mixing in Product Spaces

Log Sobolev inequalities Entropy, Log Sobolev and hyper-contraction A similar story could be told using more sophisticated analytic tools. Easier to work with Markov semi-groups T t = e − tL . Elchanan Mossel Mixing in Product Spaces

Log Sobolev inequalities Entropy, Log Sobolev and hyper-contraction A similar story could be told using more sophisticated analytic tools. Easier to work with Markov semi-groups T t = e − tL . Entropy, Dirchelet Form Ent ( f ) = E ( f log f ) − E f · log E f � E ( f , g ) = E ( fLg ) = E ( gLf ) = E ( g , f ) = − d t = 0 . dt E fT t g � � Elchanan Mossel Mixing in Product Spaces

Log Sobolev inequalities Entropy, Log Sobolev and hyper-contraction A similar story could be told using more sophisticated analytic tools. Easier to work with Markov semi-groups T t = e − tL . Entropy, Dirchelet Form Ent ( f ) = E ( f log f ) − E f · log E f � E ( f , g ) = E ( fLg ) = E ( gLf ) = E ( g , f ) = − d t = 0 . dt E fT t g � � Definition of Log-Sob Cp 2 ⇒ ∀ f , Ent ( f p ) ≤ 4 ( p − 1 ) E ( f p − 1 , f ) ( p � = 0 , 1) p -logSob(C) ⇐ ⇒ ∀ f , Ent ( f ) ≤ C 1-logSob(C) ⇐ 4 E ( f , log f ) ⇒ ∀ f , Var (log f ) ≤ − C 0-logSob(C) ⇐ 2 E ( f , 1 / f ) Elchanan Mossel Mixing in Product Spaces

Log Sob. Inequalities and Hyper-Contraction Hyper-Contraction (Gross, Nelson 1960 ... ) r -logSob with constant C implies 4 log p − 1 t ≥ C 1 < p < q < r or r ′ < q < p � T t f � p ≤ � f � q , q − 1 , = ⇒ | E [ g ( X 0 ) f ( X t )] | = | E [ gT t f | ≤ � g � p ′ � Tf � p ≤ � g � p ′ � f � q If f = 1 A and g = 1 B , get: P [ X 0 ∈ A , X t ∈ B ] ≤ � 1 A � q � 1 B � p ′ = P [ A ] 1 / q P [ B ] 1 / p ′ , Now optimize over norms to get a better bound than CS. Elchanan Mossel Mixing in Product Spaces

Reverse-Hyper-Contraction Log-Sobolev and Rev. Hyper-Contraction(M-Oleszkiewicz-Sen-13) Let T t = e − tL be a general Markov semi-group satisfying 2-Logsob with constant C or 1-Logsob inequality with constant C . 4 log 1 − q Then for all q < p < 1, all positive f , g and all t ≥ C 1 − p it holds that � T t f � q ≥ � f � p = ⇒ E [ g ( X 0 ) f ( X t )] = E [ gT t f ] ≥ � g � q ′ � f � p Elchanan Mossel Mixing in Product Spaces

Short-Time Implications Theorem (M-Oleszkiewicz-Sen-13 ; Short-Time Implications) Let T t = e − tL , where L satisfy 1 or 2 -LogSob inequality with constant C . Let A , B ⊂ Ω n with P [ A ] ≥ ǫ and P [ B ] ≥ ǫ . Then: 2 P [ X ( 0 ) ∈ A , X ( t ) ∈ B ] ≥ ǫ 1 − e − 2 t / C Elchanan Mossel Mixing in Product Spaces

Short-Time Implications Theorem (M-Oleszkiewicz-Sen-13 ; Short-Time Implications) Let T t = e − tL , where L satisfy 1 or 2 -LogSob inequality with constant C . Let A , B ⊂ Ω n with P [ A ] ≥ ǫ and P [ B ] ≥ ǫ . Then: 2 P [ X ( 0 ) ∈ A , X ( t ) ∈ B ] ≥ ǫ 1 − e − 2 t / C Comments 1. Works for small sets too. 2. Tensorizes. 3. Some examples where it is (almost) tight. 4. Uses in social choice analysis, queuing theory. Elchanan Mossel Mixing in Product Spaces

Comment: typical application MCMC Long Time Behavior Log Sobolev inequalities play a major role in analyzing long term mixing of Markov chains, in particular in analysis of mixing times (Diaconis, Saloff-Coste etc.) Long Time Behavior The ε -total variation mixing time of a finite Markov chain is bounded by: 1 λ (log( 1 /π ∗ ) + log( 1 /ǫ )) 1 C (log log( 1 /π ∗ ) + log( 1 /ǫ )) for a continuous time Markov chain with spectral gap λ and 2-LogSob C . Elchanan Mossel Mixing in Product Spaces

Comment: typical application MCMC Long Time Behavior Log Sobolev inequalities play a major role in analyzing long term mixing of Markov chains, in particular in analysis of mixing times (Diaconis, Saloff-Coste etc.) Long Time Behavior The ε -total variation mixing time of a finite Markov chain is bounded by: 1 λ (log( 1 /π ∗ ) + log( 1 /ǫ )) 1 C (log log( 1 /π ∗ ) + log( 1 /ǫ )) for a continuous time Markov chain with spectral gap λ and 2-LogSob C . Elchanan Mossel Mixing in Product Spaces

What are these lectures about? High Dimensional Phenomena High dimensional mixing: mixing of product processes on product spaces Ω n with n large. Tight bounds For which processes, given measures a and b can we find precise upper/lower bounds for � � sup P [ X 0 ∈ A , X t ∈ B ] : P [ A ] = a , P [ B ] = b Interested in product space/processes of dimension n and answers as n → ∞ . Most important examples / techniques from probability / analysis. Elchanan Mossel Mixing in Product Spaces