Spectral distributions of high-dimensional sample correlation - - PowerPoint PPT Presentation

▶
spectral distributions of high dimensional sample
SMART_READER_LITE
LIVE PREVIEW

Spectral distributions of high-dimensional sample correlation - - PowerPoint PPT Presentation

Dec 02, 2022 218 likes •692 views

Spectral distributions of high-dimensional sample correlation matrices under infinite variance Johannes Heiny Ruhr-University Bochum Joint work with Jianfeng Yao (HKU), Thomas Mikosch and Jorge Yslas (Copenhagen). Random Matrices and Complex

slide-1
SLIDE 1

Spectral distributions of high-dimensional sample correlation matrices under infinite variance

Johannes Heiny

Ruhr-University Bochum

Joint work with Jianfeng Yao (HKU), Thomas Mikosch and Jorge Yslas (Copenhagen). Random Matrices and Complex Data Analysis Workshop, December 10-12, 2019, Shanghai

  • J. Heiny

Sample correlation & off-diagonal 1 / 30

slide-2
SLIDE 2

1 2 3 4 5 6 7 8 9 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Normalized histogram of eigenvalues and MP density x y Histogram of eigenvalues y = fγ(x)

Figure: These are NOT spikes!

  • J. Heiny

Sample correlation & off-diagonal 2 / 30

slide-3
SLIDE 3

Setup for the picture

Data matrix X = Xn: p × n matrix with iid centered entries and generic variable X d = X11. X = (Xit)i=1,...,p;t=1,...,n Sample covariance matrix S = 1

nXX′

Ordered eigenvalues of S λ1(S) ≥ λ2(S) ≥ · · · ≥ λp(S) Sample correlation matrix R = (diag(S))−1/2S (diag(S))−1/2 .

  • J. Heiny

Sample correlation & off-diagonal 3 / 30

slide-4
SLIDE 4

Regular variation

Regular variation with index α > 0: P(|X| > x) = x−αL(x), where L is a slowly varying function. This implies E[|X|α+ε] = ∞ for any ε > 0. Normalizing sequence (a2

np) such that

np P(X2 > a2

npx) → x−α/2,

as n → ∞ for x > 0. Then anp = (np)

1/αℓ(np) for a slowly varying function ℓ.

  • J. Heiny

Sample correlation & off-diagonal 4 / 30

slide-5
SLIDE 5

Reduction to Diagonal

Diagonal X with iid regularly varying entries α ∈ (0, 4) and p = nβ with β ∈ [0, 1]. We have a−2

np XX′ − diag(XX′) P

→ 0 , where · denotes the spectral norm. (XX′)ij =

n

  • t=1

XitXjt.

  • J. Heiny

Sample correlation & off-diagonal 5 / 30

slide-6
SLIDE 6

Eigenvalues

Weyl’s inequality max

i=1,...,p

  • λi(A + B) − λi(A)
  • ≤ B .

Choose A + B = XX′ and A = diag(XX′) to obtain a−2

np

max

i=1,...,p

  • λi(XX′) − λi(diag(XX′))
  • P

→ 0 , n → ∞ . Note: Limit theory for (λi(S)) reduced to (Sii).

  • J. Heiny

Sample correlation & off-diagonal 6 / 30

slide-7
SLIDE 7

Heavy-tailed case

Theorem (Heiny and Mikosch, 2016) X with iid regularly varying entries α ∈ (0, 4) and pn = nβℓ(n) with β ∈ [0, 1].

1 If β ∈ [0, 1], then

a−2

np

max

i=1,...,p

  • λi(XX′) − λi(diag(XX′))
  • P

→ 0 .

2 If β ∈ ((α/2 − 1)+, 1], then

a−2

np

max

i=1,...,p

  • λi(XX′) − X2

(i),np

  • P

→ 0 .

  • J. Heiny

Sample correlation & off-diagonal 7 / 30

slide-8
SLIDE 8

Example: Eigenvalues

Figure: Smoothed histogram based on 20000 simulations of the approximation error for the normalized eigenvalue a−2

np λ1(S) for entries

Xit with α = 1.6, β = 1, n = 1000 and p = 200.

  • J. Heiny

Sample correlation & off-diagonal 8 / 30

slide-9
SLIDE 9

Eigenvectors

vk unit eigenvector of S associated to λk(S) Unit eigenvectors of diag(S) are canonical basisvectors ej. Eigenvectors X with iid regularly varying entries with index α ∈ (0, 4) and pn = nβℓ(n) with β ∈ [0, 1]. Then for any fixed k ≥ 1, vk − eLkℓ2

P

→ 0 , n → ∞ .

  • J. Heiny

Sample correlation & off-diagonal 9 / 30

slide-10
SLIDE 10

Localization vs. Delocalization

  • 50

100 150 200 0.0 0.2 0.4 0.6 0.8 1.0

Pareto data

Indices of components Size of components

Figure: X ∼ Pareto(0.8)

  • 50

100 150 200 −0.15 −0.10 −0.05 0.00 0.05 0.10 0.15

Normal Data

Indices of Components Size of Components

Figure: X ∼ N(0, 1)

Components of eigenvector v1. p = 200, n = 1000.

  • J. Heiny

Sample correlation & off-diagonal 10 / 30

slide-11
SLIDE 11

Point Process of Normalized Eigenvalues

Point process convergence Nn =

p

  • i=1

δa−2

np λi(XX′)

d

→

∞

  • i=1

δΓ−2/α

i

= N The limit is a PRM on (0, ∞) with mean measure µ(x, ∞) = x−α/2, x > 0, and Γi = E1 + · · · + Ei , (Ei) iid standard exponential.

  • J. Heiny

Sample correlation & off-diagonal 11 / 30

slide-12
SLIDE 12

Point Process of Normalized Eigenvalues

Limiting distribution: For k ≥ 1, lim

n→∞ P(a−2 np λk ≤ x) = lim n→∞ P(Nn(x, ∞) < k) = P(N(x, ∞) < k)

=

k−1

  • s=0
  • x−α/2s

s! e−x−α/2, x > 0 .

  • J. Heiny

Sample correlation & off-diagonal 12 / 30

slide-13
SLIDE 13

Point Process of Normalized Eigenvalues

Limiting distribution: For k ≥ 1, lim

n→∞ P(a−2 np λk ≤ x) = lim n→∞ P(Nn(x, ∞) < k) = P(N(x, ∞) < k)

=

k−1

  • s=0
  • x−α/2s

s! e−x−α/2, x > 0 . Largest eigenvalue n a2

np

λ1(S) d → Γ−α/2

1

, where the limit has a Fr´ echet distribution with parameter α/2.

Soshnikov (2006), Auffinger et al. (2009), Auffinger and Tang (2016), Davis et al. (2014, 20162), JH and Mikosch (2016)

  • J. Heiny

Sample correlation & off-diagonal 12 / 30

slide-14
SLIDE 14

α = 3.99

α = 3.99, n = 2000, p = 1000

  • J. Heiny

Sample correlation & off-diagonal 13 / 30

slide-15
SLIDE 15

α = 3

α = 3, n = 2000, p = 1000

  • J. Heiny

Sample correlation & off-diagonal 14 / 30

slide-16
SLIDE 16

α = 2.1

α = 2.1, n = 10000, p = 1000

  • J. Heiny

Sample correlation & off-diagonal 15 / 30

slide-17
SLIDE 17

Infinite variance, α < 2

Limiting spectral distribution of (XX′) under E[X2] = ∞: Regular variation with α < 2: Fa−2

n+pXX′ → Gγ

α weakly,

whose density gγ

α satisfies

gγ

α(x) ∼ c x−1−α/2 ,

x → ∞ . Ben Arous and Guionnet (2008), Belinschi et al. (2009)

  • J. Heiny

Sample correlation & off-diagonal 16 / 30

slide-18
SLIDE 18

Moments of LSD

Assumption: X symmetric and regularly varying with index α ∈ (0, 2). Goal: For k ≥ 1, find the limit of E xkFR(dx)

  • = 1

pE[tr(Rk)]

  • J. Heiny

Sample correlation & off-diagonal 17 / 30

slide-19
SLIDE 19

Moments of LSD

One has E[tr(Rk)] =

p

  • i1,...,ik=1

n

  • t1,...,tk=1

E[Yi1t1Yi2t1 · · · YiktkYi1tk]

  • :=F(i1,...,ik)

. Assumption: X symmetric ⇒ Yij symmetric Yij =

Xij

√n

t=1 X2 it

  • J. Heiny

Sample correlation & off-diagonal 18 / 30

slide-20
SLIDE 20

Moments of LSD

1 pE[tr(Rk)] → βk(γ) + 2 α

k−2

  • r=2

γr−1

r−2

  • q=0

(Γ(1 − α/2))−r+q+1

  • I∈C(q)

r,k

t⋆( I)

  • s=1
  • α/2

Γ(1 − α/2) s

  • T∈Cs,|

I|(

I)

r−q

  • i=1

Γ(di( I, T)) Γ(Ni( I))

  • (i,t)∈∆(

I,T)

Γ mit( I, T) − α 2

  • .
  • J. Heiny

Sample correlation & off-diagonal 19 / 30

slide-21
SLIDE 21
  • J. Heiny

Sample correlation & off-diagonal 20 / 30

slide-22
SLIDE 22
  • J. Heiny

Sample correlation & off-diagonal 21 / 30

slide-23
SLIDE 23

Motivation

Random walk Sn = X1 + · · · + Xn , n ≥ 1 .

1

(Xi) are iid random variables with generic element X.

2

E[X] = 0 and E[X2] = 1.

Dimension p = pn → ∞ Consider iid copies (S(i)

n )i≤p of Sn and define the point

process Nn =

p

  • i=1

δdp(S(i)

n /√n−dp) .

  • J. Heiny

Sample correlation & off-diagonal 22 / 30

slide-24
SLIDE 24

We want to prove: Nn =

p

  • i=1

δdp(S(i)

n /√n−dp)

d

→ N , n → ∞ , where N is a Poisson random measure with mean measure µ(x, ∞) = e −x, x ∈ R, and dp =

  • 2 log p − log log p + log 4π

2(2 log p)1/2 .

  • J. Heiny

Sample correlation & off-diagonal 23 / 30

slide-25
SLIDE 25

We want to prove: Nn =

p

  • i=1

δdp(S(i)

n /√n−dp)

d

→ N , n → ∞ , where N is a Poisson random measure with mean measure µ(x, ∞) = e −x, x ∈ R, and dp =

  • 2 log p − log log p + log 4π

2(2 log p)1/2 . Note: dp is the centering and normalizing sequence for the maximum of p iid standard normals. By Resnick (2007), this is equivalent to p P

  • dp (Sn/√n − dp) > x
  • → e −x ,

x ∈ R .

  • J. Heiny

Sample correlation & off-diagonal 23 / 30

slide-26
SLIDE 26

H., Mikosch, Yslas (2019+) Assume that the sequence (pn) satisfies the following conditions: (C1) p = O(n(s−2)/2) for s > 2 if E[|X|s] < ∞. (C2) p = exp(o(n1/3)) if E[exp(h |X|)] < ∞ for some h > 0. Then p P

  • dp (Sn/√n − dp) > x
  • → e −x ,

x ∈ R .

  • J. Heiny

Sample correlation & off-diagonal 24 / 30

slide-27
SLIDE 27

H., Mikosch, Yslas (2019+) Assume that the sequence (pn) satisfies the following conditions: (C1) p = O(n(s−2)/2) for s > 2 if E[|X|s] < ∞. (C2) p = exp(o(n1/3)) if E[exp(h |X|)] < ∞ for some h > 0. Then p P

  • dp (Sn/√n − dp) > x
  • → e −x ,

x ∈ R . Precise large deviation bounds of the type sup

0≤y≤γn

  • P(Sn/√n > y)

Φ(y) − 1

  • → 0 ,

n → ∞ , Under (C1): γn =

  • (s − 2) log n, Michel (1974)

Under (C2): γn = o(n1/6), Petrov (1972)

  • J. Heiny

Sample correlation & off-diagonal 24 / 30

slide-28
SLIDE 28

Data matrix X = Xn: p × n matrix with iid entries with generic element X. X = (Xit)i=1,...,p;t=1,...,n Sample covariance matrix S = XX′ Dependent random walks Sij =

n

  • t=1

XitXjt , i < j . Off-diagonal point process: NS

n =

  • 1≤i<j≤p

δ

dp(Sij/√n− dp) ,

where dp = dp(p−1)/2.

  • J. Heiny

Sample correlation & off-diagonal 25 / 30

slide-29
SLIDE 29

Off-diagonal point process

NS

n =

  • 1≤i<j≤p

δ

dp(Sij/√n− dp)

Theorem: H., Mikosch, Yslas (2019+) Assume that the sequence (pn) satisfies: p = O(n(s−2)/4) for s > 2 if E[|X|s] < ∞. p = exp(o(n1/3)) if E[exp(h |X11X12|)] < ∞ for some h > 0. Then NS

n d

→ N . Remark: Entries of X do not have to be identically distributed.

  • J. Heiny

Sample correlation & off-diagonal 26 / 30

slide-30
SLIDE 30

Note that N =

∞

  • i=1

δ− log Γi , where Γi = E1 + · · · + Ei, i ≥ 1, and (Ei) is iid standard exponential.

  • J. Heiny

Sample correlation & off-diagonal 27 / 30

slide-31
SLIDE 31

Note that N =

∞

  • i=1

δ− log Γi , where Γi = E1 + · · · + Ei, i ≥ 1, and (Ei) is iid standard exponential. For fixed k,

  • dp
  • S(i)/√n −

dp

  • i=1,...,k

d

→ (− log Γi)i=1,...,k .

  • J. Heiny

Sample correlation & off-diagonal 27 / 30

slide-32
SLIDE 32

In particular, Jiang (2004) lim

n→∞ P

  • dp
  • S(1)/√n −

dp

  • ≤ x
  • = exp(−e −x) .

Fang Han’s talk on Tuesday, Songxi Chen’s talk on Wednesday

  • J. Heiny

Sample correlation & off-diagonal 28 / 30

slide-33
SLIDE 33

Extension to sample correlation matrices

Sample correlation matrix R = (diag(S))−1/2S(diag(S))−1/2

  • J. Heiny

Sample correlation & off-diagonal 29 / 30

slide-34
SLIDE 34

Extension to sample correlation matrices

Sample correlation matrix R = (diag(S))−1/2S(diag(S))−1/2 Theorem: H., Mikosch, Yslas (2019+) Assume that the sequence (pn) satisfies: p = O(n(s−2)/4) for s > 2 if E[|X|s] < ∞. p = exp(o(n1/3)) if E[exp(h |X11X12|)] < ∞ for some h > 0. Then NR

n =

  • 1≤i<j≤p

δ

dp(√nRij− dp) d

→ N .

  • J. Heiny

Sample correlation & off-diagonal 29 / 30

slide-35
SLIDE 35

Thank you!

  • J. Heiny

Sample correlation & off-diagonal 30 / 30

slide-36
SLIDE 36

Heavy Tails and Dependence

(Zit): iid field of regularly varying random variables. Stochastic volatility model: X =

  • Zit σ(n)

it

  • p×n
  • J. Heiny

Sample correlation & off-diagonal 31 / 30

slide-37
SLIDE 37

Heavy Tails and Dependence

(Zit): iid field of regularly varying random variables. Stochastic volatility model: X =

  • Zit σ(n)

it

  • p×n

Generate deterministic covariance structure A: X = A1/2Z Davis et al. (2014)

  • J. Heiny

Sample correlation & off-diagonal 31 / 30

slide-38
SLIDE 38

Heavy Tails and Dependence

(Zit): iid field of regularly varying random variables. Dependence among rows and columns: Xit =

∞

  • l=0

∞

  • k=0

hklZi−k,t−l with some constants hkl. Davis et al. (2016)

  • J. Heiny

Sample correlation & off-diagonal 32 / 30

slide-39
SLIDE 39

Heavy Tails and Dependence

(Zit): iid field of regularly varying random variables. Dependence among rows and columns: Xit =

∞

  • l=0

∞

  • k=0

hklZi−k,t−l with some constants hkl. Davis et al. (2016) Relation to iid case: XX′ =

∞

  • l1,l2=0

∞

  • k1,k2=0

hk1l1hk2l2Z(k1, l1)Z′(k2, l2) , where Z(k, l) = (Zi−k,t−l)i=1,...,p;t=1,...,n , l, k ∈ Z .

  • J. Heiny

Sample correlation & off-diagonal 32 / 30

slide-40
SLIDE 40

Heavy Tails and Dependence

(Zit): iid field of regularly varying random variables. Dependence among rows and columns: Xit =

∞

  • l=0

∞

  • k=0

hklZi−k,t−l with some constants hkl. Davis et al. (2016) Relation to iid case: XX′ =

∞

  • l1,l2=0

∞

  • k1,k2=0

hk1l1hk2l2Z(k1, l1)Z′(k2, l2) , where Z(k, l) = (Zi−k,t−l)i=1,...,p;t=1,...,n , l, k ∈ Z . Location of squares: M ij =

  • l∈Z

hilhjl, i, j ∈ Z .

  • J. Heiny

Sample correlation & off-diagonal 32 / 30

slide-41
SLIDE 41

Autocovariance Matrices

For s ≥ 0, Xn(s) = (Xi,t+s)i=1,...,p; t=1,...,n , n ≥ 1 . Then Xn = Xn(0). Autocovariance matrix for lag s Xn(0)Xn(s)′ Limit theory for singular values of such matrices.

  • J. Heiny

Sample correlation & off-diagonal 33 / 30

slide-42
SLIDE 42

Autocovariance Matrices

Autocovariance matrix for lag s Cn(s) = Xn(0)Xn(s)′ , if α < 2(1 + β), Xn(0)Xn(s)′ − E[Xn(0)Xn(s)′] , if α > 2(1 + β), Consider Pn(s1, s2) =

s2

  • s=s1

Cn(s)Cn(s)′ for fixed 0 ≤ s1 ≤ s2.

  • J. Heiny

Sample correlation & off-diagonal 34 / 30

slide-43
SLIDE 43

Autocovariance Matrices

(M(s))ij =

  • l∈Z

hi,lhj,l+s, i, j ∈ Z. For 0 ≤ s1 ≤ s2 < ∞, we define the positive semi-definite matrix K(s1, s2) =

s2

  • s=s1

M(s)M(s)′ Eigenvector approximation yi(s1, s2) − ua(i)

b(i)(s1, s2)ℓ2 P

→ 0 , n → ∞ .

  • J. Heiny

Sample correlation & off-diagonal 35 / 30

slide-44
SLIDE 44

Autocovariance eigenvectors

  • J. Heiny

Sample correlation & off-diagonal 36 / 30

slide-45
SLIDE 45

Autocovariance eigenvectors

Eigenvectors of K(0,0) 1

  • 0.5

0.5 1st eigenvector of P(0,0) 878 880 882 884 886 888 890 892

  • 0.5

0.5 2nd eigenvector of P(0,0) 878 880 882 884 886 888 890 892 Number of coordinate

  • 0.5

0.5 Coordinates of eigenvector 3rd eigenvector of P(0,0) 78 80 82 84 86 88 90 92

  • 0.5

0.5 4th eigenvector of P(0,0) 878 880 882 884 886 888 890 892

  • 0.5

0.5 5th eigenvector of P(0,0) 395 400 405 410

  • 0.5

0.5

  • J. Heiny

Sample correlation & off-diagonal 37 / 30