Non asymptotic study of the singular values of some random - PowerPoint PPT Presentation

Non asymptotic study of the singular values of some random covariance matrices. Olivier Gu´ edon Universit´ e Paris-Est Marne-la-Vall´ ee High-dimensional problems and quantum physics. June 2015

The setting. Let A be a random matrix defined as A = ( X 1 . . . X N ) where X 1 , . . . , X N are independent random vectors in R n . What can we say about the singular values of A ? Study of   X T N 1 . M = AA T = ( X 1 . . . X N ) � . X i X T  =   . i  X T i = 1 N

The setting. Let A be a random matrix defined as A = ( X 1 . . . X N ) where X 1 , . . . , X N are independent random vectors in R n . What can we say about the singular values of A ? Study of   X T N 1 . M = AA T = ( X 1 . . . X N ) � . X i X T  =   . i  X T i = 1 N  N �  λ max ( AA T ) = sup � X i , a � 2     a ∈ S n − 1 i = 1 N �  λ min ( AA T ) = � X i , a � 2 inf    a ∈ S n − 1  i = 1

Random Matrix Theory. All X i ’s have identically independent random entries. M n = AA T A = ( a ij ) 1 ≤ i ≤ n , 1 ≤ j ≤ N , Little n and N go to infinity and N / n → c > 1 .

Random Matrix Theory. All X i ’s have identically independent random entries. M n = AA T A = ( a ij ) 1 ≤ i ≤ n , 1 ≤ j ≤ N , Little n and N go to infinity and N / n → c > 1 . Counting probability measure : n ν n , m = 1 � δ λ k ( M n / √ N ) n i = 1

Random Matrix Theory. Marchenko-Pastur ’67 (bulk of the spectrum) If E a 2 i , j = 1 , then with probability one, for any continuous bounded functon f : [ 0 , + ∞ ) → R , � � f d ν n , m = f d ν lim n → + ∞ where ν is the semi-circle law.

Random Matrix Theory. Marchenko-Pastur ’67 (bulk of the spectrum) If E a 2 i , j = 1 , then with probability one, for any continuous bounded functon f : [ 0 , + ∞ ) → R , � � f d ν n , m = f d ν lim n → + ∞ where ν is the semi-circle law. Bai-Yin ’93 (edge of the spectrum) If E a 2 i , j = 1 , E a i , j = 0 , and E a 4 i , j < + ∞ , then with probability one � n  n →∞ λ max ( 1 N AA T ) = 1 + lim    N � n n →∞ λ min ( 1 N AA T ) = 1 −  lim   N

Frame in Harmonic Analysis Take an orthonormal basis from R M and project it on R n (or on an n -dimensional subspace of R M ). You get v 1 , . . . , v M : M � ∀ x ∈ R n , | x | 2 c j � x , u j � 2 2 = j = 1 v j | v j | 2 and c j = | v j | 2 where u j = 2 . Define a random vector X in R n as X = √ n u j with proba c j n . Then for every vector θ ∈ R n M E � X , θ � 2 = c j � u j , θ � 2 = | θ | 2 � 2 j = 1

Frame in Harmonic Analysis Take an orthonormal basis from R M and project it on R n (or on an n -dimensional subspace of R M . You get v 1 , . . . , v M : M � ∀ x ∈ R n , | x | 2 c j � x , u j � 2 2 = j = 1 v j | v j | 2 and c j = | v j | 2 where u j = 2 . Define a random vector X in R n as X = √ n u j with proba c j n . Hence Σ = E X ⊗ X = Id

Frame in Harmonic Analysis Take an orthonormal basis from R M and project it on R n (or on an n -dimensional subspace of R M . You get v 1 , . . . , v M : M � ∀ x ∈ R n , | x | 2 c j � x , u j � 2 2 = j = 1 v j | v j | 2 and c j = | v j | 2 where u j = 2 . Define a random vector X in R n as X = √ n u j with proba c j n . Hence Σ = E X ⊗ X = Id Question : Find the size N ( ε ) of a sample such that N 2 ≤ 1 � X j , θ � 2 ≤ ( 1 + ε ) | θ | 2 � ∀ θ ∈ R n , ( 1 − ε ) | θ | 2 2 N j = 1

Frame in Harmonic Analysis Take an orthonormal basis from R M and project it on R n (or on an n -dimensional subspace of R M . You get v 1 , . . . , v M : M � ∀ x ∈ R n , | x | 2 c j � x , u j � 2 2 = j = 1 v j | v j | 2 and c j = | v j | 2 where u j = 2 . Define a random vector X in R n as X = √ n u j with proba c j n . Hence Σ = E X ⊗ X = Id Question : Find the size N ( ε ) of a sample such that N 2 ≤ 1 � X j , θ � 2 ≤ ( 1 + ε ) | θ | 2 � ∀ θ ∈ R n , ( 1 − ε ) | θ | 2 2 N j = 1 This gives a subset with a very particular structure.

Frame in Harmonic Analysis Theorem (Rudelson, ’97). If X is a random vector in R n such that | X | 2 ≤ K √ n a . s . and E � X , θ � 2 = | θ | 2 ∀ θ ∈ R n , : isotropy 2 then for N ≈ C K ( ε ) n log n , � � N 1 � � � X j X T E j − Id � ≤ ε � � � N � � j = 1

Frame in Harmonic Analysis Theorem (Rudelson, ’97). If X is a random vector in R n such that | X | 2 ≤ K √ n a . s . and E � X , θ � 2 = | θ | 2 ∀ θ ∈ R n , : isotropy 2 then for N ≈ C K ( ε ) n log n , � � N 1 � � � X j X T E j − Id � ≤ ε � � � N � � j = 1 The main assumption is | X | 2 ≤ K √ n , otherwise, there is no moment assumption of the entries.

Frame in Harmonic Analysis Theorem (Rudelson, ’97). If X is a random vector in R n such that | X | 2 ≤ K √ n a . s . and E � X , θ � 2 = | θ | 2 ∀ θ ∈ R n , : isotropy 2 then for N ≈ C K ( ε ) n log n , � � N 1 � � � X j X T E j − Id � ≤ ε � � � N � � j = 1 The main assumption is | X | 2 ≤ K √ n , otherwise, there is no moment assumption of the entries. You can not do better ! Coupon collector.

Computing the volume of a convex body K ⊂ R n is given by a separation oracle

Computing the volume of a convex body K ⊂ R n is given by a separation oracle Elekes (’86), B´ ar´ any-F¨ uredi (’86) : it is not possible to compute with a deterministic algorithm in polynomial time the volume of a convex body (even approximately)

Computing the volume of a convex body K ⊂ R n is given by a separation oracle Elekes (’86), B´ ar´ any-F¨ uredi (’86) : it is not possible to compute with a deterministic algorithm in polynomial time the volume of a convex body (even approximately) Randomization - Given ε and η , Dyer-Frieze-Kannan(’89) established randomized algorithms returning a non-negative number ζ such that ( 1 − ε ) ζ < Vol K < ( 1 + ε ) ζ with probability at least 1 − η . The running time of the algorithm is polynomial in n , 1 /ε and log ( 1 /η ) .

Computing the volume of a convex body K ⊂ R n is given by a separation oracle Elekes (’86), B´ ar´ any-F¨ uredi (’86) : it is not possible to compute with a deterministic algorithm in polynomial time the volume of a convex body (even approximately) Randomization - Given ε and η , Dyer-Frieze-Kannan(’89) established randomized algorithms returning a non-negative number ζ such that ( 1 − ε ) ζ < Vol K < ( 1 + ε ) ζ with probability at least 1 − η . The running time of the algorithm is polynomial in n , 1 /ε and log ( 1 /η ) . The number of oracle calls is a random variable and the bound is for example on its expected value.

Computing the volume of a convex body The randomized algorithm proposed by Kannan, Lov´ asz and Simonovits ’97 improves significantly the polynomial dependence.

Computing the volume of a convex body The randomized algorithm proposed by Kannan, Lov´ asz and Simonovits ’97 improves significantly the polynomial dependence. Rounding - Put the convex body in a position where B n 2 ⊂ K ⊂ d B n 2 where d ≤ n const .

Computing the volume of a convex body The randomized algorithm proposed by Kannan, Lov´ asz and Simonovits ’97 improves significantly the polynomial dependence. Rounding - Put the convex body in a position where B n 2 ⊂ K ⊂ d B n 2 where d ≤ n const . - John (’48) : d ≤ n ( or d ≤ √ n in the symmetric case). How to find an algorithm to do so ?

Computing the volume of a convex body The randomized algorithm proposed by Kannan, Lov´ asz and Simonovits ’97 improves significantly the polynomial dependence. Rounding - Put the convex body in a position where B n 2 ⊂ K ⊂ d B n 2 where d ≤ n const . - Idea : find an algorithm which produces in polynomial time a matrix A such that AK is in an approximate isotropic position. Conjecture 2 of KLS (’97) : solved in 2010 by Adamczak, Litvak, Pajor, Tomczak-Jaegermann

Computing the volume of a convex body The randomized algorithm proposed by Kannan, Lov´ asz and Simonovits ’97 improves significantly the polynomial dependence. Rounding - Put the convex body in a position where B n 2 ⊂ K ⊂ d B n 2 where d ≤ n const . - Idea : find an algorithm which produces in polynomial time a matrix A such that AK is in an approximate isotropic position. Conjecture 2 of KLS (’97) : solved in 2010 by Adamczak, Litvak, Pajor, Tomczak-Jaegermann Computing the volume - Monte Carlo algorithm, estimates of local conductance. Conjecture 1 of KLS (’95) : isoperimetric inequality - open !

Approximation of the covariance matrix. Question of KLS (’97) : let X be a vector uniformly distributed on a convex body K , X 1 , . . . , X N ind. copies of X , what is the smallest N such that � � N 1 � � � X j X ⊤ j − E X X ⊤ � � E X X ⊤ � � ≤ ε � � � � N � � j = 1 � · � is the operator norm

Approximation of the covariance matrix. Question of KLS (’97) : let X be a vector uniformly distributed on a convex body K , X 1 , . . . , X N ind. copies of X , what is the smallest N such that � � N 1 � � � X j X ⊤ j − Id � ≤ ε � � � N � � j = 1 Assume E X X ⊤ = Id ,