analytical nonlinear shrinkage of large dimensional
play

Analytical Nonlinear Shrinkage of Large-Dimensional Covariance - PowerPoint PPT Presentation

May 21, 2023 •345 likes •1.99k views

Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Analytical Nonlinear Shrinkage of Large-Dimensional Covariance Matrices Olivier Ledoit 1 and Michael Wolf 1 1 Department of Economics

  1. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Comparison with Other Approaches (1) This is not the sparsity approach of Bickel and Levina (2008, AoS): 10 / 53

  2. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Comparison with Other Approaches (1) This is not the sparsity approach of Bickel and Levina (2008, AoS): they assume that the current orthormal basis is special in the sense that most of the p ( p − 1) / 2 covariances are zero 10 / 53

  3. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Comparison with Other Approaches (1) This is not the sparsity approach of Bickel and Levina (2008, AoS): they assume that the current orthormal basis is special in the sense that most of the p ( p − 1) / 2 covariances are zero this condition would not hold true in general after rotation 10 / 53

  4. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Comparison with Other Approaches (1) This is not the sparsity approach of Bickel and Levina (2008, AoS): they assume that the current orthormal basis is special in the sense that most of the p ( p − 1) / 2 covariances are zero this condition would not hold true in general after rotation it requires a priori information about the orientation of the eigenvectors of the population covariance matrix, which is unverifiable in practice. 10 / 53

  5. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Comparison with Other Approaches (1) This is not the sparsity approach of Bickel and Levina (2008, AoS): they assume that the current orthormal basis is special in the sense that most of the p ( p − 1) / 2 covariances are zero this condition would not hold true in general after rotation it requires a priori information about the orientation of the eigenvectors of the population covariance matrix, which is unverifiable in practice. (2) This is not the linear shrinkage of Ledoit and Wolf (2004, JMVA): 10 / 53

  6. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Comparison with Other Approaches (1) This is not the sparsity approach of Bickel and Levina (2008, AoS): they assume that the current orthormal basis is special in the sense that most of the p ( p − 1) / 2 covariances are zero this condition would not hold true in general after rotation it requires a priori information about the orientation of the eigenvectors of the population covariance matrix, which is unverifiable in practice. (2) This is not the linear shrinkage of Ledoit and Wolf (2004, JMVA): they assume the modified eigenvalues are linear functions of the � observed ones: ∀ i = 1 , . . . , p δ n , i = a n + b n λ n , i 10 / 53

  7. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Comparison with Other Approaches (1) This is not the sparsity approach of Bickel and Levina (2008, AoS): they assume that the current orthormal basis is special in the sense that most of the p ( p − 1) / 2 covariances are zero this condition would not hold true in general after rotation it requires a priori information about the orientation of the eigenvectors of the population covariance matrix, which is unverifiable in practice. (2) This is not the linear shrinkage of Ledoit and Wolf (2004, JMVA): they assume the modified eigenvalues are linear functions of the � observed ones: ∀ i = 1 , . . . , p δ n , i = a n + b n λ n , i they have only 2 degrees of freedom, whereas our class has p ≫ 2 degrees of freedom 10 / 53

  8. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Comparison with Other Approaches (1) This is not the sparsity approach of Bickel and Levina (2008, AoS): they assume that the current orthormal basis is special in the sense that most of the p ( p − 1) / 2 covariances are zero this condition would not hold true in general after rotation it requires a priori information about the orientation of the eigenvectors of the population covariance matrix, which is unverifiable in practice. (2) This is not the linear shrinkage of Ledoit and Wolf (2004, JMVA): they assume the modified eigenvalues are linear functions of the � observed ones: ∀ i = 1 , . . . , p δ n , i = a n + b n λ n , i they have only 2 degrees of freedom, whereas our class has p ≫ 2 degrees of freedom linear shrinkage is a good first-order approximation if optimal nonlinear shrinkage happens to be ‘almost’ linear, but in the general case it can be further improved 10 / 53

  9. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Loss Functions 11 / 53

  10. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Loss Functions �� � ��� � 2 � . = 1 L FR . p Tr Σ n − Σ n Frobenius: Σ n , Σ n n �� �� n Σ n � Σ − 1 Σ − 1 �� � Tr p 1 n L MV . Minimum Variance: Σ n , Σ n . = − � � � �� �� � 2 n Σ − 1 Σ − 1 Tr / p Tr p n n �� � � � � � �� . = 1 − 1 Σ n � Σ n � L IS Σ − 1 Σ − 1 . Inverse Stein: Σ n , Σ n p Tr p log det n n n �� � � � � � �� . = 1 − 1 n � n � L ST . Σ − 1 Σ − 1 Stein: Σ n , Σ n p Tr Σ n p log det Σ n n �� � ��� � 2 � . = 1 L IF Σ − 1 n − Σ − 1 . Inverse Frobenius: Σ n , Σ n p Tr n n �� � ��� � 2 Σ − 1 � . = 1 L WF . Weighted Frobenius: Σ n , Σ n p Tr Σ n − Σ n n n 11 / 53

  11. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Loss Functions �� � ��� � 2 � . = 1 L FR . p Tr Σ n − Σ n Frobenius: Σ n , Σ n n �� �� n Σ n � Σ − 1 Σ − 1 �� � Tr p 1 n L MV . Minimum Variance: Σ n , Σ n . = − � � � �� �� � 2 n Σ − 1 Σ − 1 Tr / p Tr p n n �� � � � � � �� . = 1 − 1 Σ n � Σ n � L IS Σ − 1 Σ − 1 . Inverse Stein: Σ n , Σ n p Tr p log det n n n �� � � � � � �� . = 1 − 1 n � n � L ST . Σ − 1 Σ − 1 Stein: Σ n , Σ n p Tr Σ n p log det Σ n n �� � ��� � 2 � . = 1 L IF Σ − 1 n − Σ − 1 . Inverse Frobenius: Σ n , Σ n p Tr n n �� � ��� � 2 Σ − 1 � . = 1 L WF . Weighted Frobenius: Σ n , Σ n p Tr Σ n − Σ n n n We use the Minimum-Variance Loss championed by Rob Engle, Olivier Ledoit and Michael Wolf (2019) 11 / 53

  12. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Finite-Sample Optimal (FSOPT) Estimator 12 / 53

  13. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Finite-Sample Optimal (FSOPT) Estimator Find rotation-equivariant estimator closest to Σ n according to MV loss 12 / 53

  14. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Finite-Sample Optimal (FSOPT) Estimator Find rotation-equivariant estimator closest to Σ n according to MV loss Optimization problem:   � p       � δ n , i · u n , i u ′ L MV   min n , i , Σ n     n � δ n , 1 , . . . , � δ n , p i = 1 12 / 53

  15. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Finite-Sample Optimal (FSOPT) Estimator Find rotation-equivariant estimator closest to Σ n according to MV loss Optimization problem:   � p       � δ n , i · u n , i u ′ L MV   min n , i , Σ n     n δ n , 1 , . . . , � � δ n , p i = 1 Solution: p � S ∗ d ∗ n , i · u n , i u ′ d ∗ . = u ′ . . . = where n , i Σ n u n , i (1) n n , i n , i i = 1 12 / 53

  16. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Finite-Sample Optimal (FSOPT) Estimator Find rotation-equivariant estimator closest to Σ n according to MV loss Optimization problem:   � p       � δ n , i · u n , i u ′ L MV   min n , i , Σ n     n δ n , 1 , . . . , � � δ n , p i = 1 Solution: p � S ∗ d ∗ n , i · u n , i u ′ d ∗ . = u ′ . . . = where n , i Σ n u n , i (1) n n , i n , i i = 1 Very intuitive: d ∗ n , i is the true variance of the linear combination of original variables weighted by eigenvector u n , i 12 / 53

  17. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Finite-Sample Optimal (FSOPT) Estimator Find rotation-equivariant estimator closest to Σ n according to MV loss Optimization problem:   � p       � δ n , i · u n , i u ′ L MV   min n , i , Σ n     n � δ n , 1 , . . . , � δ n , p i = 1 Solution: p � S ∗ d ∗ n , i · u n , i u ′ d ∗ . = u ′ . . . = where n , i Σ n u n , i (1) n n , i n , i i = 1 Very intuitive: d ∗ n , i is the true variance of the linear combination of original variables weighted by eigenvector u n , i By contrast, λ n , i is the sample variance of the linear combination of original variables weighted by eigenvector u n , i : overfitting! 12 / 53

  18. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Finite-Sample Optimal (FSOPT) Estimator Find rotation-equivariant estimator closest to Σ n according to MV loss Optimization problem:   � p       � δ n , i · u n , i u ′ L MV   min n , i , Σ n     n � δ n , 1 , . . . , � δ n , p i = 1 Solution: p � S ∗ d ∗ n , i · u n , i u ′ d ∗ . = u ′ . . . = where n , i Σ n u n , i (1) n n , i n , i i = 1 Very intuitive: d ∗ n , i is the true variance of the linear combination of original variables weighted by eigenvector u n , i By contrast, λ n , i is the sample variance of the linear combination of original variables weighted by eigenvector u n , i : overfitting! FSOPT is the unattainable ‘Gold Standard’ 12 / 53

  19. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion A Numerical Scheme: NERCOME 13 / 53

  20. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion A Numerical Scheme: NERCOME Proposed by Abadir et al. (2014) and Lam (2016, AoS) 13 / 53

  21. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion A Numerical Scheme: NERCOME Proposed by Abadir et al. (2014) and Lam (2016, AoS) Split the sample into two parts 13 / 53

  22. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion A Numerical Scheme: NERCOME Proposed by Abadir et al. (2014) and Lam (2016, AoS) Split the sample into two parts Estimate the eigenvectors from the first part 13 / 53

  23. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion A Numerical Scheme: NERCOME Proposed by Abadir et al. (2014) and Lam (2016, AoS) Split the sample into two parts Estimate the eigenvectors from the first part Estimate the d ∗ n , i ’s from the second part 13 / 53

  24. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion A Numerical Scheme: NERCOME Proposed by Abadir et al. (2014) and Lam (2016, AoS) Split the sample into two parts Estimate the eigenvectors from the first part Estimate the d ∗ n , i ’s from the second part Average over many di ff erent ways to split the sample 13 / 53

  25. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion A Numerical Scheme: NERCOME Proposed by Abadir et al. (2014) and Lam (2016, AoS) Split the sample into two parts Estimate the eigenvectors from the first part Estimate the d ∗ n , i ’s from the second part Average over many di ff erent ways to split the sample Gets around the overfitting problem 13 / 53

  26. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion A Numerical Scheme: NERCOME Proposed by Abadir et al. (2014) and Lam (2016, AoS) Split the sample into two parts Estimate the eigenvectors from the first part Estimate the d ∗ n , i ’s from the second part Average over many di ff erent ways to split the sample Gets around the overfitting problem Problems: 13 / 53

  27. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion A Numerical Scheme: NERCOME Proposed by Abadir et al. (2014) and Lam (2016, AoS) Split the sample into two parts Estimate the eigenvectors from the first part Estimate the d ∗ n , i ’s from the second part Average over many di ff erent ways to split the sample Gets around the overfitting problem Problems: Requires brute-force spectral decomposition of many matrices 13 / 53

  28. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion A Numerical Scheme: NERCOME Proposed by Abadir et al. (2014) and Lam (2016, AoS) Split the sample into two parts Estimate the eigenvectors from the first part Estimate the d ∗ n , i ’s from the second part Average over many di ff erent ways to split the sample Gets around the overfitting problem Problems: Requires brute-force spectral decomposition of many matrices Easy to code but slow to execute 13 / 53

  29. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion A Numerical Scheme: NERCOME Proposed by Abadir et al. (2014) and Lam (2016, AoS) Split the sample into two parts Estimate the eigenvectors from the first part Estimate the d ∗ n , i ’s from the second part Average over many di ff erent ways to split the sample Gets around the overfitting problem Problems: Requires brute-force spectral decomposition of many matrices Easy to code but slow to execute Cannot go much beyond dimension p = 1000 computationally 13 / 53

  30. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion A Numerical Scheme: NERCOME Proposed by Abadir et al. (2014) and Lam (2016, AoS) Split the sample into two parts Estimate the eigenvectors from the first part Estimate the d ∗ n , i ’s from the second part Average over many di ff erent ways to split the sample Gets around the overfitting problem Problems: Requires brute-force spectral decomposition of many matrices Easy to code but slow to execute Cannot go much beyond dimension p = 1000 computationally To get an analytical solution: need Random Matrix Theory 13 / 53

  31. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Outline Introduction 1 Finite Samples 2 Random Matrix Theory 3 Kernel Estimation 4 Monte Carlo 5 Application 6 Conclusion 7 14 / 53

  32. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Limiting Spectral Distribution 15 / 53

  33. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Limiting Spectral Distribution Assumption 3.1 p and n go to infinity with p / n → c ∈ (0 , 1) ‘concentration ratio’ 15 / 53

  34. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Limiting Spectral Distribution Assumption 3.1 p and n go to infinity with p / n → c ∈ (0 , 1) ‘concentration ratio’ population eigenvalues are τ n , 1 , . . . , τ n , p 15 / 53

  35. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Limiting Spectral Distribution Assumption 3.1 p and n go to infinity with p / n → c ∈ (0 , 1) ‘concentration ratio’ population eigenvalues are τ n , 1 , . . . , τ n , p . = p − 1 � p population spectral c.d.f. is H n ( x ) . i = 1 1 { x ≥ τ i } 15 / 53

  36. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Limiting Spectral Distribution Assumption 3.1 p and n go to infinity with p / n → c ∈ (0 , 1) ‘concentration ratio’ population eigenvalues are τ n , 1 , . . . , τ n , p . = p − 1 � p population spectral c.d.f. is H n ( x ) . i = 1 1 { x ≥ τ i } H n converges to some limit H 15 / 53

  37. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Limiting Spectral Distribution Assumption 3.1 p and n go to infinity with p / n → c ∈ (0 , 1) ‘concentration ratio’ population eigenvalues are τ n , 1 , . . . , τ n , p . = p − 1 � p population spectral c.d.f. is H n ( x ) . i = 1 1 { x ≥ τ i } H n converges to some limit H Remark 3.1 This is not the spiked model of Johnstone (2001, AoS), which assumes that, apart from a finite number r of ‘spikes’, the p − r population eigenvalues in the ‘bulk’ are equal to one another. By contrast, we can handle the general case with any shape(s) of bulk(s). 15 / 53

  38. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Limiting Spectral Distribution Assumption 3.1 p and n go to infinity with p / n → c ∈ (0 , 1) ‘concentration ratio’ population eigenvalues are τ n , 1 , . . . , τ n , p . = p − 1 � p population spectral c.d.f. is H n ( x ) . i = 1 1 { x ≥ τ i } H n converges to some limit H Remark 3.1 This is not the spiked model of Johnstone (2001, AoS), which assumes that, apart from a finite number r of ‘spikes’, the p − r population eigenvalues in the ‘bulk’ are equal to one another. By contrast, we can handle the general case with any shape(s) of bulk(s). Theorem 1 (Marˇ cenko and Pastur (1967)) There exists a unique F . . = F c , H such that the sample spectral c.d.f. . = p − 1 � p F n ( x ) . i = 1 1 { x ≥ λ n , i } converges to F ( x ) . 15 / 53

  39. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Σ n = Identity Matrix: Marˇ cenko-Pastur Law 16 / 53

  40. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Σ n = Identity Matrix: Marˇ cenko-Pastur Law � � � 2 ( a + − x )( x − a − ) √ f c , H ( x ) . where a ± . ∀ x ∈ [ a − , a + ] . = . = 1 ± c 2 π cx 16 / 53

  41. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Σ n = Identity Matrix: Marˇ cenko-Pastur Law � � � 2 ( a + − x )( x − a − ) √ f c , H ( x ) . where a ± . ∀ x ∈ [ a − , a + ] . = . = 1 ± c 2 π cx H(x) = 1 {x 1} 3.5 c = 0.01 3 2.5 Density 2 1.5 1 0.5 0 0 0.5 1 1.5 2 2.5 Sample Eigenvalues 16 / 53

  42. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Σ n = Identity Matrix: Marˇ cenko-Pastur Law � � � 2 ( a + − x )( x − a − ) √ f c , H ( x ) . where a ± . ∀ x ∈ [ a − , a + ] . = . = 1 ± c 2 π cx H(x) = 1 {x 1} 3.5 c = 0.01 c = 0.05 3 2.5 Density 2 1.5 1 0.5 0 0 0.5 1 1.5 2 2.5 Sample Eigenvalues 16 / 53

  43. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Σ n = Identity Matrix: Marˇ cenko-Pastur Law � � � 2 ( a + − x )( x − a − ) √ f c , H ( x ) . where a ± . ∀ x ∈ [ a − , a + ] . = . = 1 ± c 2 π cx H(x) = 1 {x 1} 3.5 c = 0.01 c = 0.05 3 c = 0.1 2.5 Density 2 1.5 1 0.5 0 0 0.5 1 1.5 2 2.5 Sample Eigenvalues 16 / 53

  44. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Σ n = Identity Matrix: Marˇ cenko-Pastur Law � � � 2 ( a + − x )( x − a − ) √ f c , H ( x ) . where a ± . ∀ x ∈ [ a − , a + ] . = . = 1 ± c 2 π cx H(x) = 1 {x 1} 3.5 c = 0.01 c = 0.05 3 c = 0.1 c = 0.15 2.5 Density 2 1.5 1 0.5 0 0 0.5 1 1.5 2 2.5 Sample Eigenvalues 16 / 53

  45. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Σ n = Identity Matrix: Marˇ cenko-Pastur Law � � � 2 ( a + − x )( x − a − ) √ f c , H ( x ) . where a ± . ∀ x ∈ [ a − , a + ] . = . = 1 ± c 2 π cx H(x) = 1 {x 1} 3.5 c = 0.01 c = 0.05 3 c = 0.1 c = 0.15 2.5 c = 0.2 Density 2 1.5 1 0.5 0 0 0.5 1 1.5 2 2.5 Sample Eigenvalues 16 / 53

  46. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Σ n = Identity Matrix: Marˇ cenko-Pastur Law � � � 2 ( a + − x )( x − a − ) √ f c , H ( x ) . where a ± . ∀ x ∈ [ a − , a + ] . = . = 1 ± c 2 π cx H(x) = 1 {x 1} 3.5 c = 0.01 c = 0.05 3 c = 0.1 c = 0.15 2.5 c = 0.2 c = 0.25 Density 2 1.5 1 0.5 0 0 0.5 1 1.5 2 2.5 Sample Eigenvalues 16 / 53

  47. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion General Case: True Covariance Matrix � Identity 17 / 53

  48. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion General Case: True Covariance Matrix � Identity Definition 2 (Stieltjes Transform) � m F ( z ) . ( λ − z ) − 1 dF ( λ ) The Stieltjes transform of F is . = for z ∈ C + : complex numbers with imaginary part > 0. 17 / 53

  49. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion General Case: True Covariance Matrix � Identity Definition 2 (Stieltjes Transform) � m F ( z ) . ( λ − z ) − 1 dF ( λ ) The Stieltjes transform of F is . = for z ∈ C + : complex numbers with imaginary part > 0. Theorem 3 (Silverstein and Bai (1995); Silverstein (1995)) m ≡ m F ( z ) is the unique solution in C + to � + ∞ dH ( τ ) m = τ [1 − c − c z m ] − z −∞ 17 / 53

  50. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion General Case: True Covariance Matrix � Identity Definition 2 (Stieltjes Transform) � m F ( z ) . ( λ − z ) − 1 dF ( λ ) The Stieltjes transform of F is . = for z ∈ C + : complex numbers with imaginary part > 0. Theorem 3 (Silverstein and Bai (1995); Silverstein (1995)) m ≡ m F ( z ) is the unique solution in C + to � + ∞ dH ( τ ) m = τ [1 − c − c z m ] − z −∞ Theorem 4 (Silverstein and Choi (1995)) m F ( x ) . m F admits a continuous extension to the real line ˘ . = lim z ∈ C + → x m F ( z ) , and the sample spectral density is f ( x ) . . = F ′ ( x ) = π − 1 Im [ ˘ m F ( x )] . 17 / 53

  51. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion General Case: True Covariance Matrix � Identity Definition 2 (Stieltjes Transform) � m F ( z ) . ( λ − z ) − 1 dF ( λ ) The Stieltjes transform of F is . = for z ∈ C + : complex numbers with imaginary part > 0. Theorem 3 (Silverstein and Bai (1995); Silverstein (1995)) m ≡ m F ( z ) is the unique solution in C + to � + ∞ dH ( τ ) m = τ [1 − c − c z m ] − z −∞ Theorem 4 (Silverstein and Choi (1995)) m F ( x ) . m F admits a continuous extension to the real line ˘ . = lim z ∈ C + → x m F ( z ) , and the sample spectral density is f ( x ) . . = F ′ ( x ) = π − 1 Im [ ˘ m F ( x )] . Integrate f , and this is how you go from ( c , H ) to F = F c , H . 17 / 53

  52. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion A Conjecture on the Inverse Problem when F = F c , H 18 / 53

  53. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion A Conjecture on the Inverse Problem when F = F c , H Conjecture 3.1 For every c ′ ≤ c, there exists a c.d.f. H ′ such that F c ′ , H ′ = F. 18 / 53

  54. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion A Conjecture on the Inverse Problem when F = F c , H Conjecture 3.1 For every c ′ ≤ c, there exists a c.d.f. H ′ such that F c ′ , H ′ = F. Mapping from ( c ', f ) to h' 1.6 c ' = 0 1.4 1.2 1 Density h' Limiting 0.8 Sample Spectral 0.6 Density 0.4 f 0.2 0 0 0.5 1 1.5 2 2.5 Population Eigenvalues 18 / 53

  55. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion A Conjecture on the Inverse Problem when F = F c , H Conjecture 3.1 For every c ′ ≤ c, there exists a c.d.f. H ′ such that F c ′ , H ′ = F. Mapping from ( c ', f ) to h' 1.6 c ' = 0 1.4 c ' = 0.05 1.2 1 Density h' Limiting Limiting 0.8 Sample Sample Spectral Spectral 0.6 Density Density 0.4 f f 0.2 0 0 0.5 1 1.5 2 2.5 Population Eigenvalues 18 / 53

  56. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion A Conjecture on the Inverse Problem when F = F c , H Conjecture 3.1 For every c ′ ≤ c, there exists a c.d.f. H ′ such that F c ′ , H ′ = F. Mapping from ( c ', f ) to h' 1.6 c ' = 0 1.4 c ' = 0.05 c ' = 0.1 1.2 1 Density h' Limiting Limiting Limiting 0.8 Sample Sample Sample Spectral Spectral Spectral 0.6 Density Density Density 0.4 f f f 0.2 0 0 0.5 1 1.5 2 2.5 Population Eigenvalues 18 / 53

  57. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion A Conjecture on the Inverse Problem when F = F c , H Conjecture 3.1 For every c ′ ≤ c, there exists a c.d.f. H ′ such that F c ′ , H ′ = F. Mapping from ( c ', f ) to h' 1.6 c ' = 0 1.4 c ' = 0.05 c ' = 0.1 c ' = 0.125 1.2 1 Density h' Limiting Limiting Limiting Limiting 0.8 Sample Sample Sample Sample Spectral Spectral Spectral Spectral 0.6 Density Density Density Density 0.4 f f f f 0.2 0 0 0.5 1 1.5 2 2.5 Population Eigenvalues 18 / 53

  58. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion A Conjecture on the Inverse Problem when F = F c , H Conjecture 3.1 For every c ′ ≤ c, there exists a c.d.f. H ′ such that F c ′ , H ′ = F. Mapping from ( c ', f ) to h' 1.6 c ' = 0 1.4 c ' = 0.05 c ' = 0.1 c ' = 0.125 1.2 c ' = 0.15 1 Density h' Limiting Limiting Limiting Limiting Limiting 0.8 Sample Sample Sample Sample Sample Spectral Spectral Spectral Spectral Spectral 0.6 Density Density Density Density Density 0.4 f f f f f 0.2 0 0 0.5 1 1.5 2 2.5 Population Eigenvalues 18 / 53

  59. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion A Conjecture on the Inverse Problem when F = F c , H Conjecture 3.1 For every c ′ ≤ c, there exists a c.d.f. H ′ such that F c ′ , H ′ = F. Mapping from ( c ', f ) to h' 1.6 c ' = 0 1.4 c ' = 0.05 c ' = 0.1 c ' = 0.125 1.2 c ' = 0.15 c ' = 0.16 1 Density h' Limiting Limiting Limiting Limiting Limiting Limiting 0.8 Sample Sample Sample Sample Sample Sample Spectral Spectral Spectral Spectral Spectral Spectral 0.6 Density Density Density Density Density Density 0.4 f f f f f f 0.2 0 0 0.5 1 1.5 2 2.5 Population Eigenvalues 18 / 53

  60. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion A Conjecture on the Inverse Problem when F = F c , H Conjecture 3.1 For every c ′ ≤ c, there exists a c.d.f. H ′ such that F c ′ , H ′ = F. Mapping from ( c ', f ) to h' 1.6 c ' = 0 1.4 c ' = 0.05 c ' = 0.1 c ' = 0.125 1.2 c ' = 0.15 c ' = 0.16 1 Density h' c ' = 0.17 Limiting Limiting Limiting Limiting Limiting Limiting Limiting 0.8 Sample Sample Sample Sample Sample Sample Sample Spectral Spectral Spectral Spectral Spectral Spectral Spectral 0.6 Density Density Density Density Density Density Density 0.4 f f f f f f f 0.2 0 0 0.5 1 1.5 2 2.5 Population Eigenvalues 18 / 53

  61. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion The Real Part of the Stieltjes Transform 19 / 53

  62. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion The Real Part of the Stieltjes Transform π − 1 Im [ ˘ m F ( x )] = f ( x ): the limiting sample spectral density 19 / 53

  63. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion The Real Part of the Stieltjes Transform π − 1 Im [ ˘ m F ( x )] = f ( x ): the limiting sample spectral density π − 1 Re [ ˘ m F ( x )] = H f ( x ): its Hilbert transform 19 / 53

  64. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion The Real Part of the Stieltjes Transform π − 1 Im [ ˘ m F ( x )] = f ( x ): the limiting sample spectral density π − 1 Re [ ˘ m F ( x )] = H f ( x ): its Hilbert transform Krantz (2009) The Hilbert transform is, without question, the most important operator in analysis. It arises in many di ff erent contexts, and all these contexts are intertwined in profound and influential ways. 19 / 53

  65. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion The Real Part of the Stieltjes Transform π − 1 Im [ ˘ m F ( x )] = f ( x ): the limiting sample spectral density π − 1 Re [ ˘ m F ( x )] = H f ( x ): its Hilbert transform Krantz (2009) The Hilbert transform is, without question, the most important operator in analysis. It arises in many di ff erent contexts, and all these contexts are intertwined in profound and influential ways. � Hilbert transform of some p.d.f. g : H g ( x ) . . = π − 1 PV ( t − x ) − 1 g ( t ) dt 19 / 53

  66. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion The Real Part of the Stieltjes Transform π − 1 Im [ ˘ m F ( x )] = f ( x ): the limiting sample spectral density π − 1 Re [ ˘ m F ( x )] = H f ( x ): its Hilbert transform Krantz (2009) The Hilbert transform is, without question, the most important operator in analysis. It arises in many di ff erent contexts, and all these contexts are intertwined in profound and influential ways. � Hilbert transform of some p.d.f. g : H g ( x ) . . = π − 1 PV ( t − x ) − 1 g ( t ) dt Cauchy Principal Value: � � � + ∞ � x − ε � −∞ g ( t ) g ( t ) g ( t ) t − xdt . PV . = lim PV t − xdt + PV t − x ε ց 0 −∞ −∞ x − ε 19 / 53

  67. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion The Real Part of the Stieltjes Transform π − 1 Im [ ˘ m F ( x )] = f ( x ): the limiting sample spectral density π − 1 Re [ ˘ m F ( x )] = H f ( x ): its Hilbert transform Krantz (2009) The Hilbert transform is, without question, the most important operator in analysis. It arises in many di ff erent contexts, and all these contexts are intertwined in profound and influential ways. � Hilbert transform of some p.d.f. g : H g ( x ) . . = π − 1 PV ( t − x ) − 1 g ( t ) dt Cauchy Principal Value: � � � + ∞ � x − ε � −∞ g ( t ) g ( t ) g ( t ) t − xdt . PV . = lim PV t − xdt + PV t − x ε ց 0 −∞ −∞ x − ε Highly positive just below the center of mass of the density g 19 / 53

  68. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion The Real Part of the Stieltjes Transform π − 1 Im [ ˘ m F ( x )] = f ( x ): the limiting sample spectral density π − 1 Re [ ˘ m F ( x )] = H f ( x ): its Hilbert transform Krantz (2009) The Hilbert transform is, without question, the most important operator in analysis. It arises in many di ff erent contexts, and all these contexts are intertwined in profound and influential ways. � Hilbert transform of some p.d.f. g : H g ( x ) . . = π − 1 PV ( t − x ) − 1 g ( t ) dt Cauchy Principal Value: � � � + ∞ � x − ε � −∞ g ( t ) g ( t ) g ( t ) t − xdt . PV . = lim PV t − xdt + PV t − x ε ց 0 −∞ −∞ x − ε Highly positive just below the center of mass of the density g Highly negative just above the center of mass of the density g 19 / 53

  69. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion The Real Part of the Stieltjes Transform π − 1 Im [ ˘ m F ( x )] = f ( x ): the limiting sample spectral density π − 1 Re [ ˘ m F ( x )] = H f ( x ): its Hilbert transform Krantz (2009) The Hilbert transform is, without question, the most important operator in analysis. It arises in many di ff erent contexts, and all these contexts are intertwined in profound and influential ways. � Hilbert transform of some p.d.f. g : H g ( x ) . . = π − 1 PV ( t − x ) − 1 g ( t ) dt Cauchy Principal Value: � � � + ∞ � x − ε � −∞ g ( t ) g ( t ) g ( t ) t − xdt . PV . = lim PV t − xdt + PV t − x ε ց 0 −∞ −∞ x − ε Highly positive just below the center of mass of the density g Highly negative just above the center of mass of the density g Fades to zero away from center of mass 19 / 53

  70. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Four Examples of Hilbert Transforms 20 / 53

  71. Introduction Finite Samples Random Matrix Theory Kernel Estimation Monte Carlo Application Conclusion Four Examples of Hilbert Transforms Gaussian Semicircle 0.5 0.5 0 0 -0.5 -0.5 -4 -2 0 2 4 -4 -2 0 2 4 Density Hilbert Transform Epanechnikov Triangular 0.5 0.5 0 0 -0.5 -0.5 -4 -2 0 2 4 -4 -2 0 2 4 Works like a local attraction force 20 / 53

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Nonlinear Methods Data often lies on or near a nonlinear low-dimensional curve aka manifold. 27

Nonlinear Methods Data often lies on or near a nonlinear low-dimensional curve aka manifold. 27

Nonlinear Methods Data often lies on or near a nonlinear low-dimensional curve aka manifold. 27 Laplacian Eigenmaps Linear methods Lower-dimensional linear projection that preserves distances between all points Laplacian Eigenmaps (key idea)

533 views • 14 slides
Nonlinear Control Lecture # 3 Two-Dimensional Systems Nonlinear Control Lecture # 3

Nonlinear Control Lecture # 3 Two-Dimensional Systems Nonlinear Control Lecture # 3

Nonlinear Control Lecture # 3 Two-Dimensional Systems Nonlinear Control Lecture # 3 Two-Dimensional Systems 1 Multiple Equilibria Example 2.2: Tunnel-diode circuit L i L i,mA X X s v + L 1

430 views • 17 slides
Analytical and Numerical Methods for Nonlinear Dynamics Tim Zolkin 1 1 Fermi National Accelerator

Analytical and Numerical Methods for Nonlinear Dynamics Tim Zolkin 1 1 Fermi National Accelerator

Analytical and Numerical Methods for Nonlinear Dynamics Tim Zolkin 1 1 Fermi National Accelerator Laboratory July 24, 2019 Tim Zolkin Analytical and Numerical Methods for Nonlinear Dynamics 0 . INTRODUCTION Tim Zolkin Analytical and Numerical

584 views • 45 slides
High-Dimensional Multivariate Bayesian Linear Regression with Shrinkage Priors Ray Bai

High-Dimensional Multivariate Bayesian Linear Regression with Shrinkage Priors Ray Bai

High-Dimensional Multivariate Bayesian Linear Regression with Shrinkage Priors Ray Bai Department of Statistics, University of Florida Joint work with Dr. Malay Ghosh March 20, 2018 Ray Bai (University of Florida) MBSP March 20, 2018 1 / 48

963 views • 48 slides
Nonlinear propagation of one-dimensional waves: some recent experimental results Stphane

Nonlinear propagation of one-dimensional waves: some recent experimental results Stphane

Nonlinear propagation of one-dimensional waves: some recent experimental results Stphane Randoux Laboratoire de Physique des Lasers, Atomes et Molcules Universit de Lille , 59 655 Villeneuve d'Ascq, France Nonlinear propagation of

236 views • 21 slides
Nonlinear Control Lecture # 1 Introduction & Two-Dimensional Systems Nonlinear Control

Nonlinear Control Lecture # 1 Introduction & Two-Dimensional Systems Nonlinear Control

Nonlinear Control Lecture # 1 Introduction & Two-Dimensional Systems Nonlinear Control Lecture # 1 Introduction & Two-Dimensional Systems Nonlinear State Model x 1 = f 1 ( t, x 1 , . . . , x n , u 1 , . . . , u m ) x 2 = f 2 (

548 views • 35 slides
High Dimensional Data, Covariance Matrices High Dimensional Data Examples and Application to

High Dimensional Data, Covariance Matrices High Dimensional Data Examples and Application to

Outline Motivation High Dimensional Data, Covariance Matrices High Dimensional Data Examples and Application to Genetics Theoretical Underpinnings Random Matrices Shrinkage Estimation Decision Theory Samprit Banerjee, Ph.D Bayesian

1.34k views • 80 slides
Justification of the nonlinear Sch odinger equation for two-dimensional gravity driven water

Justification of the nonlinear Sch odinger equation for two-dimensional gravity driven water

Justification of the nonlinear Sch odinger equation for two-dimensional gravity driven water waves C. E. Wayne January 12, 2014 Fields Institute, Jan. 2013 Water Waves and NLS Abstract Abstract: In 1968 V.E. Zakharov derived the Nonlinear

861 views • 44 slides
Using Stata to estimate nonlinear models with fixed effects Paulo high-dimensional fixed effects

Using Stata to estimate nonlinear models with fixed effects Paulo high-dimensional fixed effects

motivation nonlinear models generalized linear models other models final considerations Using Stata to estimate nonlinear models with high- dimensional Using Stata to estimate nonlinear models with fixed effects Paulo high-dimensional

277 views • 11 slides
Operator-theoretic methods for prediction and control of nonlinear dynamical systems Milan Milan

Operator-theoretic methods for prediction and control of nonlinear dynamical systems Milan Milan

Operator-theoretic methods for prediction and control of nonlinear dynamical systems Milan Milan Kor Korda da Big picture Infinite-dimensional Lifting Approximation Linear object Difficult to handle Finite-dimensional Nonlinear object

1.06k views • 90 slides
High-dimensional estimation of nonlinear transformations for Bayesian filtering Ricardo Baptista,

High-dimensional estimation of nonlinear transformations for Bayesian filtering Ricardo Baptista,

High-dimensional estimation of nonlinear transformations for Bayesian filtering Ricardo Baptista, Daniele Bigoni, Alessio Spantini, Youssef Marzouk Massachusetts Institute of Technology Department of Aeronautics & Astronautics 7th

610 views • 17 slides
Nonlinear Dimensionality Visualize all dimensions Visualize the intrinsic low-dimensional

Nonlinear Dimensionality Visualize all dimensions Visualize the intrinsic low-dimensional

Overview Direct visualization Dimensionality reduction Nonlinear Dimensionality Visualize all dimensions Visualize the intrinsic low-dimensional structure Reduction Direct visualization vs. within a high-dimensional data space

328 views • 3 slides
Visualization ( Nonlinear dimensionality reduction ) Fei Sha Yahoo! Research

Visualization ( Nonlinear dimensionality reduction ) Fei Sha Yahoo! Research

Visualization ( Nonlinear dimensionality reduction ) Fei Sha Yahoo! Research feisha@yahoo-inc.com CS294 March 18, 2008 Dimensionality reduction Question: How can we detect low dimensional structure in high dimensional data?

713 views • 59 slides
Nonlinear Manifold Learning Part One: Background, LLE, IsoMap 6.454 Area One Seminar October 8

Nonlinear Manifold Learning Part One: Background, LLE, IsoMap 6.454 Area One Seminar October 8

Nonlinear Manifold Learning Part One: Background, LLE, IsoMap 6.454 Area One Seminar October 8 th 2003 Alexander Ihler Introduction Motivation Observe high-dimensional data Hopefully, a low-dimensional (simple) underlying process

426 views • 28 slides
Nonlinear Control Lecture # 2 Two-Dimensional Systems Nonlinear Control Lecture # 2

Nonlinear Control Lecture # 2 Two-Dimensional Systems Nonlinear Control Lecture # 2

Nonlinear Control Lecture # 2 Two-Dimensional Systems Nonlinear Control Lecture # 2 Two-Dimensional Systems x 1 = f 1 ( x 1 , x 2 ) = f 1 ( x ) x 2 = f 2 ( x 1 , x 2 ) = f 2 ( x ) Let x ( t ) = ( x 1 ( t ) , x 2 ( t )) be a solution that

426 views • 21 slides
Predictor-corrector ensemble filters for high-dimensional nonlinear systems and sparse data and

Predictor-corrector ensemble filters for high-dimensional nonlinear systems and sparse data and

Predictor-corrector ensemble filters for high-dimensional nonlinear systems and sparse data and regularized ensemble Kalman filter for a wildfire model Jan Mandel University of Colorado at Denver and Health Sciences Center NCAR MMM and IMAGe

982 views • 71 slides
3rd High Lift Prediction Workshop R. Rudnik, S. Melber-Wilkending DLR, Institute of Aerodynamics

3rd High Lift Prediction Workshop R. Rudnik, S. Melber-Wilkending DLR, Institute of Aerodynamics

TAU-SOLAR Contributions to the 3rd High Lift Prediction Workshop R. Rudnik, S. Melber-Wilkending DLR, Institute of Aerodynamics and Flow Technology, Braunschweig, Germany P. Risley-Settle ARA, Aircraft Research Association, Bedford MK41 7PF,

334 views • 31 slides
Symmetric Cauchy stresses do not imply symmetric Biot strains in weak formulations of isotropic

Symmetric Cauchy stresses do not imply symmetric Biot strains in weak formulations of isotropic

Symmetric Cauchy stresses do not imply symmetric Biot strains in weak formulations of isotropic hyperelasticity with rotational degrees of freedom. Andreas Fischle , Patrizio Neff and Ingo Mnch Universitt Duisburg-Essen ,

515 views • 38 slides
Routing, Geometric and Regulations Constraints Nicolas Beldiceanu SICS Lgerhyddsvgen 5

Routing, Geometric and Regulations Constraints Nicolas Beldiceanu SICS Lgerhyddsvgen 5

Routing, Geometric and Regulations Constraints Nicolas Beldiceanu SICS Lgerhyddsvgen 5 SE-75237 Uppsala, Sweden (nicolas@sics.se) The cycle constraint

1.13k views • 86 slides
CMU-Q 15-381 Lecture 4: Path Planning Teacher: Gianni A. Di Caro A PPLICATION : M OTION P

CMU-Q 15-381 Lecture 4: Path Planning Teacher: Gianni A. Di Caro A PPLICATION : M OTION P

CMU-Q 15-381 Lecture 4: Path Planning Teacher: Gianni A. Di Caro A PPLICATION : M OTION P LANNING Path planning: computing a continuous sequence (a path) of configurations (states) between an initial configuration (start) and a final

367 views • 32 slides
Review on the R&D Activities within the RD51 Collaboration Kondo Gnanvo on Behalf of the RD5

Review on the R&D Activities within the RD51 Collaboration Kondo Gnanvo on Behalf of the RD5

INSTR20 - February 24-28, BINP Novosibirsk, Review on the R&D Activities within the RD51 Collaboration Kondo Gnanvo on Behalf of the RD5 Collaboration Outline Introduction to the RD51 Collaboration INSTR20 - February 24-28, BINP

337 views • 32 slides
CONFINEMENT IN MULTI-PARTON SECTORS OF TWO DIMENSIONAL GAUGE THEORIES Daniele Dorigoni, Gabriele

CONFINEMENT IN MULTI-PARTON SECTORS OF TWO DIMENSIONAL GAUGE THEORIES Daniele Dorigoni, Gabriele

CONFINEMENT IN MULTI-PARTON SECTORS OF TWO DIMENSIONAL GAUGE THEORIES Daniele Dorigoni, Gabriele Veneziano, J W 1 How to calculate masses of particles ? Lattice Diagonalize Hamiltonian Light Cone Discretization QCD equations:

461 views • 33 slides
Synthesis of Data-Parallel GPU Software into FPGA Hardware Satnam Singh Microsoft Corporation

Synthesis of Data-Parallel GPU Software into FPGA Hardware Satnam Singh Microsoft Corporation

Synthesis of Data-Parallel GPU Software into FPGA Hardware Satnam Singh Microsoft Corporation Alchemy Project Kiwi: concurrent shape analysis: Accelerator/FPGA: C# programs for synthesis of synthesis of data control-oriented dynamic data

901 views • 59 slides
CTP431- Music and Audio Computing Digital Audio Effects Graduate School of Culture Technology

CTP431- Music and Audio Computing Digital Audio Effects Graduate School of Culture Technology

CTP431- Music and Audio Computing Digital Audio Effects Graduate School of Culture Technology KAIST Juhan Nam 1 Introduction Amplitude Gain, fade in/out, automation curve, compressor Timbre Filters, EQ, distortion, modulation,

1.06k views • 51 slides

More recommend