eigenvectors of some large sample covariance matrix
play

Eigenvectors of some large sample covariance matrix ensembles - PowerPoint PPT Presentation

Eigenvectors of some large sample covariance matrix ensembles Random Matrix Workshop, Tlcom ParisTech Monday, October 11 th 2010 Olivier Ledoit Sandrine P ech e oledoit@iew.uzh.ch - sandrine.peche@ujf-grenoble.fr University of Z


  1. Generalization of MP67/S95 Equation � ( S N − zI ) − 1 � 1 m F N ( z ) = N Tr Eigenvectors of some large sample covariance matrix ensembles – p. 10/23

  2. Generalization of MP67/S95 Equation � ( S N − zI ) − 1 � 1 m F N ( z ) = N Tr � � 1 ( S N − zI ) − 1 g (Σ N ) Θ g N ( z ) = N Tr Eigenvectors of some large sample covariance matrix ensembles – p. 10/23

  3. Generalization of MP67/S95 Equation Eigenvectors of some large sample covariance matrix ensembles – p. 11/23

  4. Generalization of MP67/S95 Equation ∃ Θ g : a . s . Θ g ∀ z ∈ C + → Θ g ( z ) N ( z ) − Eigenvectors of some large sample covariance matrix ensembles – p. 11/23

  5. Generalization of MP67/S95 Equation ∃ Θ g : a . s . Θ g ∀ z ∈ C + → Θ g ( z ) N ( z ) − � + ∞ 1 Θ g ( z ) = g ( τ ) × τ [1 − c − czm F ( z )] − zdH ( τ ) −∞ Eigenvectors of some large sample covariance matrix ensembles – p. 11/23

  6. Generalization of MP67/S95 Equation ∃ Θ g : a . s . Θ g ∀ z ∈ C + → Θ g ( z ) N ( z ) − � + ∞ 1 Θ g ( z ) = g ( τ ) × τ [1 − c − czm F ( z )] − zdH ( τ ) −∞ Same integration kernel! Eigenvectors of some large sample covariance matrix ensembles – p. 11/23

  7. Extension to the Real Line Eigenvectors of some large sample covariance matrix ensembles – p. 12/23

  8. Extension to the Real Line N N � � 1 1 i v j | 2 × g ( τ j ) Θ g | u ∗ N ( z ) = N λ i − z i =1 j =1 Eigenvectors of some large sample covariance matrix ensembles – p. 12/23

  9. Extension to the Real Line N N � � 1 1 i v j | 2 × g ( τ j ) Θ g | u ∗ N ( z ) = N λ i − z i =1 j =1 N N � � 1 i v j | 2 × g ( τ j ) Ω g | u ∗ N ( λ ) = 1 [ λ i , + ∞ ) ( λ ) N i =1 j =1 Eigenvectors of some large sample covariance matrix ensembles – p. 12/23

  10. Extension to the Real Line N N � � 1 1 i v j | 2 × g ( τ j ) Θ g | u ∗ N ( z ) = N λ i − z i =1 j =1 N N � � 1 i v j | 2 × g ( τ j ) Ω g | u ∗ N ( λ ) = 1 [ λ i , + ∞ ) ( λ ) N i =1 j =1 � λ 1 Im [Θ g ( l + iη )] dl a . s . Ω g → Ω g ( λ ) = lim N ( λ ) π η → 0 + −∞ wherever Ω g is continuous Eigenvectors of some large sample covariance matrix ensembles – p. 12/23

  11. Sample Eigenvectors Eigenvectors of some large sample covariance matrix ensembles – p. 13/23

  12. Sample Eigenvectors Fix τ and take g = 1 ( −∞ ,τ ) Eigenvectors of some large sample covariance matrix ensembles – p. 13/23

  13. Sample Eigenvectors Fix τ and take g = 1 ( −∞ ,τ ) N N � � N ( λ ) = 1 i v j | 2 1 [ λ i , + ∞ ) ( λ ) × 1 [ τ j , + ∞ ) ( τ ) Ω g | u ∗ N i =1 j =1 � λ � τ clt → | t [1 − c − clm F ( l )] − l | 2 dH ( t ) dF ( l ) −∞ −∞ Eigenvectors of some large sample covariance matrix ensembles – p. 13/23

  14. Sample Eigenvectors Fix τ and take g = 1 ( −∞ ,τ ) N N � � N ( λ ) = 1 i v j | 2 1 [ λ i , + ∞ ) ( λ ) × 1 [ τ j , + ∞ ) ( τ ) Ω g | u ∗ N i =1 j =1 � λ � τ clt → | t [1 − c − clm F ( l )] − l | 2 dH ( t ) dF ( l ) −∞ −∞ cλ i τ j i v j | 2 ≈ N | u ∗ m F ( λ i )] − λ i | 2 | τ j [1 − c − cλ i ˘ Eigenvectors of some large sample covariance matrix ensembles – p. 13/23

  15. Eigenvalues with Multiplicity Eigenvectors of some large sample covariance matrix ensembles – p. 14/23

  16. Eigenvalues with Multiplicity Σ N has K distinct eigenvalues t 1 , . . . , t K with multiplicities n 1 , . . . , n K Eigenvectors of some large sample covariance matrix ensembles – p. 14/23

  17. Eigenvalues with Multiplicity Σ N has K distinct eigenvalues t 1 , . . . , t K with multiplicities n 1 , . . . , n K P k = projection onto k th eigenspace Eigenvectors of some large sample covariance matrix ensembles – p. 14/23

  18. Eigenvalues with Multiplicity Σ N has K distinct eigenvalues t 1 , . . . , t K with multiplicities n 1 , . . . , n K P k = projection onto k th eigenspace n k cλ i t k | P k u i | 2 ≈ m F ( λ i )] − λ i | 2 N | t k [1 − c − cλ i ˘ Eigenvectors of some large sample covariance matrix ensembles – p. 14/23

  19. Estimating the Covariance Matrix (1) Eigenvectors of some large sample covariance matrix ensembles – p. 15/23

  20. Estimating the Covariance Matrix (1) � Tr ( AA ∗ ) Frobenius norm: � A � = Eigenvectors of some large sample covariance matrix ensembles – p. 15/23

  21. Estimating the Covariance Matrix (1) � Tr ( AA ∗ ) Frobenius norm: � A � = U N : matrix of eigenvectors of S N Eigenvectors of some large sample covariance matrix ensembles – p. 15/23

  22. Estimating the Covariance Matrix (1) � Tr ( AA ∗ ) Frobenius norm: � A � = U N : matrix of eigenvectors of S N Find matrix closest to Σ N among those that have eigenvectors U N Eigenvectors of some large sample covariance matrix ensembles – p. 15/23

  23. Estimating the Covariance Matrix (1) � Tr ( AA ∗ ) Frobenius norm: � A � = U N : matrix of eigenvectors of S N Find matrix closest to Σ N among those that have eigenvectors U N � U N D N U ∗ min N − Σ N � D N diagonal Eigenvectors of some large sample covariance matrix ensembles – p. 15/23

  24. Estimating the Covariance Matrix (1) � Tr ( AA ∗ ) Frobenius norm: � A � = U N : matrix of eigenvectors of S N Find matrix closest to Σ N among those that have eigenvectors U N � U N D N U ∗ min N − Σ N � D N diagonal Solution: D N = Diag ( � � d 1 , . . . , � � d i = u ∗ d N ) where i Σ N u i Eigenvectors of some large sample covariance matrix ensembles – p. 15/23

  25. Estimating the Covariance Matrix (2) Eigenvectors of some large sample covariance matrix ensembles – p. 16/23

  26. Estimating the Covariance Matrix (2) Take g ( τ ) = τ Eigenvectors of some large sample covariance matrix ensembles – p. 16/23

  27. Estimating the Covariance Matrix (2) Take g ( τ ) = τ N � N ( λ ) = 1 Ω g u ∗ i Σ N u i 1 [ λ i , + ∞ ) ( λ ) N i =1 � λ l → | 1 − c − clm F ( l ) | 2 dF ( l ) −∞ Eigenvectors of some large sample covariance matrix ensembles – p. 16/23

  28. Estimating the Covariance Matrix (2) Take g ( τ ) = τ N � N ( λ ) = 1 Ω g u ∗ i Σ N u i 1 [ λ i , + ∞ ) ( λ ) N i =1 � λ l → | 1 − c − clm F ( l ) | 2 dF ( l ) −∞ λ i u ∗ i Σ N u i ≈ m F ( λ i ) | 2 | 1 − c − cλ i ˘ Eigenvectors of some large sample covariance matrix ensembles – p. 16/23

  29. Oracle Estimator Eigenvectors of some large sample covariance matrix ensembles – p. 17/23

  30. Oracle Estimator Keep same eigenvectors as those of S n , Eigenvectors of some large sample covariance matrix ensembles – p. 17/23

  31. Oracle Estimator Keep same eigenvectors as those of S n , divide i th sample eigenvalue by | 1 − c − cλ i ˘ m F ( λ i ) | 2 Eigenvectors of some large sample covariance matrix ensembles – p. 17/23

  32. Oracle Estimator Keep same eigenvectors as those of S n , divide i th sample eigenvalue by | 1 − c − cλ i ˘ m F ( λ i ) | 2 → oracle estimator � − S N Eigenvectors of some large sample covariance matrix ensembles – p. 17/23

  33. Oracle Estimator Keep same eigenvectors as those of S n , divide i th sample eigenvalue by | 1 − c − cλ i ˘ m F ( λ i ) | 2 → oracle estimator � − S N Percentage Relative Improvement in Average Loss:   � � 2 � � �� S N − U N � D N U ∗ E � N   PRIAL = 100 ×  1 − � �  2 � � � S N − U N � D N U ∗ E � N Eigenvectors of some large sample covariance matrix ensembles – p. 17/23

  34. Monte-Carlo Simulations Eigenvectors of some large sample covariance matrix ensembles – p. 18/23

  35. Monte-Carlo Simulations 10,000 simulations Eigenvectors of some large sample covariance matrix ensembles – p. 18/23

  36. Monte-Carlo Simulations 10,000 simulations c=1/2 Eigenvectors of some large sample covariance matrix ensembles – p. 18/23

  37. Monte-Carlo Simulations 10,000 simulations c=1/2 Population eigenvalues: 20% equal to 1 40% equal to 3 40% equal to 10 Eigenvectors of some large sample covariance matrix ensembles – p. 18/23

  38. Monte-Carlo Simulations 10,000 simulations c=1/2 Population eigenvalues: 20% equal to 1 40% equal to 3 40% equal to 10 Compare with Ledoit-Wolf (2004) linear shrinkage estimator Eigenvectors of some large sample covariance matrix ensembles – p. 18/23

  39. Simulation Results Eigenvectors of some large sample covariance matrix ensembles – p. 19/23

  40. Simulation Results γ=2 100% Relative Improvement in Average Loss 90% 80% 70% 60% Optimal Nonlinear Shrinkage Optimal Linear Shrinkage 50% 10 20 40 60 80 100 120 140 160 180 200 Sample Size Eigenvectors of some large sample covariance matrix ensembles – p. 19/23

  41. Inverse of the Covariance Matrix (1) Eigenvectors of some large sample covariance matrix ensembles – p. 20/23

  42. Inverse of the Covariance Matrix (1) Find matrix closest to Σ − 1 N among those that have eigenvectors U N Eigenvectors of some large sample covariance matrix ensembles – p. 20/23

  43. Inverse of the Covariance Matrix (1) Find matrix closest to Σ − 1 N among those that have eigenvectors U N � U N ∆ N U ∗ N − Σ − 1 min N � ∆ N diagonal Eigenvectors of some large sample covariance matrix ensembles – p. 20/23

  44. Inverse of the Covariance Matrix (1) Find matrix closest to Σ − 1 N among those that have eigenvectors U N � U N ∆ N U ∗ N − Σ − 1 min N � ∆ N diagonal Solution: ∆ N = Diag ( � � δ 1 , . . . , � � δ i = u ∗ i Σ − 1 δ N ) where N u i Eigenvectors of some large sample covariance matrix ensembles – p. 20/23

  45. Inverse of the Covariance Matrix (1) Find matrix closest to Σ − 1 N among those that have eigenvectors U N � U N ∆ N U ∗ N − Σ − 1 min N � ∆ N diagonal Solution: ∆ N = Diag ( � � δ 1 , . . . , � � δ i = u ∗ i Σ − 1 δ N ) where N u i i Σ N u i ) − 1 u ∗ i Σ − 1 N u i ≥ ( u ∗ Eigenvectors of some large sample covariance matrix ensembles – p. 20/23

  46. Inverse of the Covariance Matrix (2) Eigenvectors of some large sample covariance matrix ensembles – p. 21/23

  47. Inverse of the Covariance Matrix (2) Take g ( τ ) = 1 τ Eigenvectors of some large sample covariance matrix ensembles – p. 21/23

  48. Inverse of the Covariance Matrix (2) Take g ( τ ) = 1 τ N � N ( λ ) = 1 Ω g u ∗ i Σ − 1 N u i 1 [ λ i , + ∞ ) ( λ ) N i =1 � λ 1 − c − 2 cl Re [ m F ( l )] → dF ( l ) l −∞ Eigenvectors of some large sample covariance matrix ensembles – p. 21/23

  49. Inverse of the Covariance Matrix (2) Take g ( τ ) = 1 τ N � N ( λ ) = 1 Ω g u ∗ i Σ − 1 N u i 1 [ λ i , + ∞ ) ( λ ) N i =1 � λ 1 − c − 2 cl Re [ m F ( l )] → dF ( l ) l −∞ N u i ≈ 1 − c − 2 cλ i Re [ ˘ m F ( λ i )] u ∗ i Σ − 1 λ i Eigenvectors of some large sample covariance matrix ensembles – p. 21/23

  50. Conclusion Eigenvectors of some large sample covariance matrix ensembles – p. 22/23

  51. Conclusion Generalization of the Marˇ cenko-Pastur (1967)/Silverstein (1995) Equation Eigenvectors of some large sample covariance matrix ensembles – p. 22/23

  52. Conclusion Generalization of the Marˇ cenko-Pastur (1967)/Silverstein (1995) Equation Gives location of sample eigenvectors relative to: Eigenvectors of some large sample covariance matrix ensembles – p. 22/23

  53. Conclusion Generalization of the Marˇ cenko-Pastur (1967)/Silverstein (1995) Equation Gives location of sample eigenvectors relative to: Population eigenvectors Eigenvectors of some large sample covariance matrix ensembles – p. 22/23

  54. Conclusion Generalization of the Marˇ cenko-Pastur (1967)/Silverstein (1995) Equation Gives location of sample eigenvectors relative to: Population eigenvectors Population covariance matrix as a whole Eigenvectors of some large sample covariance matrix ensembles – p. 22/23

  55. Conclusion Generalization of the Marˇ cenko-Pastur (1967)/Silverstein (1995) Equation Gives location of sample eigenvectors relative to: Population eigenvectors Population covariance matrix as a whole Inverse of population covariance matrix Eigenvectors of some large sample covariance matrix ensembles – p. 22/23

  56. Conclusion Generalization of the Marˇ cenko-Pastur (1967)/Silverstein (1995) Equation Gives location of sample eigenvectors relative to: Population eigenvectors Population covariance matrix as a whole Inverse of population covariance matrix We do for sample eigenvectors what MP67/S95 did for sample eigenvalues Eigenvectors of some large sample covariance matrix ensembles – p. 22/23

  57. Directions for Future Research Eigenvectors of some large sample covariance matrix ensembles – p. 23/23

  58. Directions for Future Research 1. Construct bona fide nonlinear shrinkage estimator of the covariance matrix Eigenvectors of some large sample covariance matrix ensembles – p. 23/23

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend


More recommend