For matrix A ( p p ) with real eigenvalues, define F A , the - PDF document

For matrix A ( p × p ) with real eigenvalues, define F A , the empirical distribution function of the eigenvalues of A , to be F A ( x ) ≡ (1 /p ) · (number of eigenvalues of A ≤ x ) . For and p.d.f. G the Stieltjes transform of G is defined as 1 � z ∈ C + ≡ { z ∈ C : ℑ z > 0 } . m G ( z ) ≡ λ − z dG ( λ ) , Inversion formula � b G { [ a, b ] } = (1 /π ) lim ℑ m G ( ξ + iη ) dξ η → 0 + a ( a, b continuity points of G). Notice m F A ( z ) = (1 /p ) tr ( A − zI ) − 1 . 1

Theorem [S. (1995)]. Assume a) For n = 1 , 2 , . . . X n = ( X n ij ), n × N , X n ij ∈ C , i.d. for all n, i, j, 1 1 | 2 = 1. independent across i, j for each n , E | X 1 1 1 − E X 1 b) N = N ( n ) with n/N → c > 0 as n → ∞ . c) T n n × n random Hermitian nonnegative definite, with F T n con- verging almost surely in distribution to a p.d.f. H on [0 , ∞ ) as n → ∞ . d) X n and T n are independent. Let T 1 / 2 be the Hermitian nonnegative square root of T n , and n let B n = (1 /N ) T 1 / 2 n T 1 / 2 (obviously F B n = F (1 /N ) X n X ∗ n T n ). X n X ∗ n n Then, almost surely, F B n converges in distribution, as n → ∞ , to a (nonrandom) p.d.f. F , whose Stieltjes transform m ( z ) ( z ∈ C + ) satisfies � 1 ( ∗ ) m = t (1 − c − czm ) − z dH ( t ) , in the sense that, for each z ∈ C + , m = m ( z ) is the unique solution to ( ∗ ) in { m ∈ C : − 1 − c + cm ∈ C + } . z 2

We have F (1 /N ) X ∗ T X = (1 − n N ) I [0 , ∞ ) + n N F (1 /N ) XX ∗ T a.s. − → (1 − c ) I [0 , ∞ ) + cF ≡ F. Notice m F and m F satisfy 1 − c + 1 � 1 c m F ( z ) = m F ( z ) = − zm F t − z dH ( t ) . cz Therefore, m = m F solves z = − 1 t � m + c 1 + tmdH ( t ) . 3

Facts on F: 1. The endpoints of the connected components (away from 0) of the support of F are given by the extrema of � f ( m ) = − 1 t m + c 1 + tmdH ( t ) m ∈ C [Marˇ cenko and Pastur (1967), S. and Choi (1995)]. 2. F has a continuous density away from the origin given by 1 cπ ℑ m ( x ) 0 < x ∈ support of F where m ( x ) = z ∈ C + → x m F ( z ) lim solves � x = − 1 t m + c 1 + tmdH ( t ) . (S. and Choi 1995). 3. F ′ is analytic inside its support, and when H is discrete, has infinite slopes at boundaries of its support [S. and Choi (1995)]. 4. c and F uniquely determine H . D a.s. 5. F − → H as c → 0 (complements B n − → T n as N → ∞ , n fixed). 4

0.5 0.4 0.3 0,2 0 . 1 0.0; f '1/3 ,'

o I P I I I o + (j UI N O r o [ " " N I J + ( d = ll I \ ) . t s o + o O

I O O r r) tl . N F \ A L i l tl N ' t s o F o o

T n = I n = ⇒ F = F c , where, for 0 < c ≤ 1, F ′ c ( x ) = f c ( x ) = 1 � ( x − b 1 )( b 2 − x ) b 1 < x < b 2 , 2 πcx 0 otherwise, where b 1 = (1 − √ c ) 2 and b 2 = (1 + √ c ) 2 , and for 1 < c < ∞ , � x F c ( x ) = (1 − (1 /c )) I [0 , ∞ ) ( x ) + f c ( t ) dt. b 1 Marˇ cenko and Pastur (1967) Grenander and S. (1977) 8

Let, for any d > 0 and d.f. G , F d,G denote the limiting spectral d.f. of (1 /N ) X ∗ TX corresponding to limiting ratio d and limiting F T n G . Theorem [Bai and S. (1998)]. Assume: a) X ij , i, j = 1 , 2 , ... are i.i.d. random variables in C with E X 11 = 0, E | X 11 | 2 = 1, and E | X 11 | 4 < ∞ . b) N = N ( n ) with c n = n/N → c > 0 as n → ∞ . c) For each n T n is an n × n Hermitian nonnegative definite satisfying D H n ≡ F T n − → H , a p.d.f. d) � T n � , the spectral norm of T n is bounded in n . e) B n = (1 /N ) T 1 / 2 n T 1 / 2 , T 1 / 2 X n X ∗ any Hermitian square root of n n n T n , B n = (1 /N ) X ∗ n T n X n , where X n = ( X ij ), i = 1 , 2 , . . . , n , j = 1 , 2 , . . . , N . f) The interval [ a, b ] with a > 0 lies in an open interval outside the support of F c n ,H n for all large n . Then P ( no eigenvalue of B n appears in [ a, b ] for all large n ) = 1 . 9

Theorem [Bai and S. (1999)]. Assume (a)–(f) of the previous the- orem. 1) If c [1 − H (0)] > 1, then x 0 , the smallest value in the support of F c,H , is positive, and with probability one λ B n → x 0 as n → ∞ . N The number x 0 is the maximum value of the function z ( m ) = − 1 � t m + c 1 + tmdH ( t ) for m ∈ R + . 2) If c [1 − H (0)] ≤ 1, or c [1 − H (0)] > 1 but [ a, b ] is not contained in [0 , x 0 ] then m F c,H ( b ) < 0. Let for large n integer i n ≥ 0 be such that λ T n λ T n i n > − 1 /m F c,H ( b ) and i n +1 < − 1 /m F c,H ( a ) (eigenvalues arranged in non-increasing order). Then P ( λ B n λ B n i n > b i n +1 < a for all large n ) = 1 . and 10

Theorem (Baik and S. (2006)) Assume the conditions in Bai and S. (1998). Suppose a fixed number of the eigenvalues of T n are different than 1, positive, and remain the same value for all n large. n largest eigenvalue, λ T n Assume the i th i n , is one of these numbers, say α . Then with probability one  if | α − 1 | > √ c cα α +   α − 1 (1 + √ c ) 2 if 1 < α ≤ 1 + √ c λ B n i n →  (1 − √ c ) 2 if 1 − √ c ≤ α < 1 .  11

Theorem (Baik and S.). Assume when X 1 1 is complex, E X 2 1 1 = 0. Suppose α > 0 is an eigenvalue of T n with multiplicity k satisfying | α − 1 | > √ c ( ∗∗ ) and eigenspace formed from k elements of the canonical basis set B n = { (1 , 0 , . . . , 0) ∗ , (0 , 1 , . . . , 0) ∗ , . . . , (0 , . . . , 1) ∗ } in R n . Let λ i , i = 1 , . . . , k be the eigenvalues of B n corresponding to α . Let λ n = α + n α α − 1 . N Then √ n ( λ i − λ n ) i = 1 , . . . , k converge weakly to the eigenvalues of a mean zero Gaussian k × k Hermitian matrix containing inde- pendent random variables on and above the diagonal. The diagonal entries have variance = ( E | x 1 | 4 − 1 − t/ 2) α 2 (( α − 1) 2 − c ) 2 + ( t/ 2) α 2 (( α − 1) 2 − c ) ( α − 1) 4 ( α − 1) 2 where t = 4 when X 1 1 is real, t = 2 when X 1 1 is complex. In the complex case the real and imaginary parts of the off-diagonal entries are i.i.d. The real part of the off-diagonal elements (in either case) has variance ( t/ 4) α 2 (( α − 1) 2 − c ) . ( α − 1) 2 Moreover, the weak limit of eigenvalues corresponding to different positive α ’s satisfying ( ∗∗ ) with eigenvectors in B n are independent. 12

Theorem (Rao and S.). Assume the conditions in Bai and S. (1998), and additionally a) There are r positive eigenvalues of T n all converging uniformly to t ′ , a positive number. Denote by ˆ H n the e.d.f. of the n − r other eigenvalues of T n . b) There exists positive t a < t b contained in an interval ( α, β ) with α > 0 which is outside the support of ˆ H n for all large n , such that for these n � λ 2 n ( λ − t ) 2 d ˆ H n ( λ ) ≤ 1 N for t = t a , t b . c) t ′ ∈ ( t a , t b ). Suppose λ T n i n , . . . , λ T n i n + r − 1 are the eigenvalues stated in a). Then, with probability one n →∞ λ B n n →∞ λ B n i n + r − 1 = z ( − 1 /t ′ ) lim i n = · · · = lim � � � λ = t ′ 1 + c t ′ − λdH ( λ ) . 13

Application to array signal processing : Consider m i.i.d. samples of an n dimensional Gaussian (real or complex) distributed random vectors x 1 , . . . , x N , modeling the recordings ( N “snapshots”) taken from a bank of n antennas due to signals emitting from an unknown number, k , of sources, and through a noise-filled environment. It is typically assumed that x 1 is mean zero, with covariance matrix R = Ψ + Σ, where Ψ is the covariance matrix attributed to the signals and would be of rank k , and Σ is the covariance of the additive noise, assumed to be of full rank. Then the matrix R Σ = Σ − 1 R = Σ − 1 Ψ + I would have n − k eigenvalues, attributed to the noise, equal to 1, the remaining k eigenvalues all strictly greater than 1. 14

When it is possible to sample the purely additive noise portion along the antennas, say z 1 , . . . , z N ′ with N ′ > n , then one would Σ − 1 � estimate R Σ with � Σ ≡ � R � R , where N ′ N � � R = 1 Σ = 1 � � x i x ∗ Σ z j z ∗ and Σ j , i N N ′ i =1 j =1 and seek to identify eigenvalues of � R � Σ which are near one, the number of the remaining eigenvalues (presumably all greater than these) would be an estimate of k , known in the literature as the detection problem. 15

Theorem (Rao and S.) Denote the eigenvalues of R Σ by λ 1 ≥ λ 2 > . . . ≥ λ k > λ k +1 = . . . λ n = 1. Let l j denote the j -th largest eigenvalue of � R � Σ . Then as n, N ( n ) , N ′ ( n ) → ∞ , c n = n/N → c > N ′ = n/N ′ → c 1 < 1, with probability 1, l j converges to: 0, c 1 �   2 − 2 c 1 λ j 2 − 2 c 1 λ j + λ j 2 − 2 λ j +1 c 12 λ j − c 1 λ j − λ j +1+   λ j 1 − c − c 2 c 1 λ j if λ j > τ ( c, c 1 ) , � � 2 � 1 + 1 − (1 − c )(1 − c 1 ) 1 − c 1 if λ j ≤ τ ( c, c 1 ) for j = 1 , . . . , k . The threshold τ ( c, c 1 ) = √ c + c 1 − c 1 c −√ c + c 1 − c 1 c − 1) c − c 12 − 2 c 1 √ c + c 1 − c 1 c − c 1 ( c 12 + c 1 . (( c 1 − 1) c − c 1 ) (1 − c 1 ) 2 16