Germain Van Bever (joint work with B. Li, H. Oja, F. Critchley and R. - - PowerPoint PPT Presentation

Germain Van Bever (joint work with B. Li, H. Oja, F. Critchley and R. Sabolova) Université de Namur IWAFDA (III), May 24th, 2019

The cocktail party effect

1 / 21

Departing from elliptical symmetry

PCA fails to uncover hidden structure... since emphasis is on another dispersion measure.

2 / 21

Departing from elliptical symmetry

PCA fails to uncover hidden structure... since emphasis is on another dispersion measure.

3 / 21

Departing from elliptical symmetry

PCA fails to uncover hidden structure... since emphasis is on another dispersion measure.

4 / 21

Functional example: Australian weather data

100 200 300 50 100 150 day Precipitations (mm)

Figure: Australian weather data. Daily measurements in 190 weather stations from 1840 to 1990 (length of records vary from one station to the other).

5 / 21

Australian weather data: principal components

PCA eigenfunctions 90 180 270 360

• 0.10
• 0.05

0.00 0.05 0.10 PC1 scores Frequency

• 1

1 2 3 4 5 10 20 30 40 50

Figure: Left: First four (from thickest to thinnest) principal eigenfunctions. Right: Histogram of the first principal score.

6 / 21

Australian weather data: principal clustering

7 / 21

The IC model

◮ X follows an Independent Component Model (X ∼ IC(Z)) if

X = ΩZ, for Z = (Z1, . . . , Zp)T with independent marginals and Ω a nonsin- gular p × p mixing matrix.

◮ IC models ⊂ BSS models ◮ Independent component analysis (ICA): Find (one) unmixing matrix

Γ such that ΓX has independent marginals.

◮ If Z has at most one gaussian marginal, then Γ = Ω−1 up to permu-

tation and multiplication by a diagonal matrix.

8 / 21

Whitening

Proposition

Let X ∼ IC(Z) and let Σ = Var (X) . Write Xst = Σ−1/2X, where Σ−1/2 is the symmetric inverse square root of Σ. Then, Z = U T Xst = U T Σ−1/2X for some orthogonal matrix U = (u1 · · · up).

◮ ICA problems are thus often reduced to the estimation of an orthog-

• nal matrix after whitening the distribution.

9 / 21

The FOBI procedure in a nutshell

Let Σ0(X) = Var(X) and let Σ1(X) = E

• (XT X)XXT

. If Z has inde- pendent components, then Σ1(Z) is diagonal.

Theorem

Let X ∼ IC (Z). Assume the components of Z have finite and dis- tinct kurtoses. Let V ΛV T be the eigendecomposition of Σ1(Xst). Then, V T Xst = V T Σ−1/2 X has independent components.

10 / 21

Functional data

Observations are typically assumed to belong to H = L2(I) allows general treatment of FOBI in a Hilbert space H. Extension to a generic H faces three hurdles:

◮ No notion of marginals in general. ◮ No standardization possible: Σ−1/2

X does not exist.

◮ Intrinsic limit to knowledge requires regularization.

11 / 21

Functional IC model

◮ Assume X(n) resides in a finite-dimensional subspace Hn of H. ◮ The dimension mn of Hn is thought to increase to infinity with n. ◮ The subspaces Hn are assumed to be nested in n.

Definition

Suppose E||X||2

Hn < ∞ and let {fi : i ∈ N0} be a Σ0-ONB. We say that

X follows a functional independent component model with respect to a functional operator Γ ∈ B(Hn) if the sequence of random variables {ΓX, fi : i ∈ N0} is independent. If this condition is satisfied then we write X ∼ FIC(Γ).

12 / 21

FOBI operator

Define the FOBI operator Σ1(X) as E

• (X ⊗ X)2

.

Theorem

If EX4

Hn < ∞, then Σ1(X) is a trace-class operator and is unitarily

equivariant, that is Σ1(UX) = UΣ1(X)U ∗ for any operator satisfying UU ∗ = U ∗U = I.

13 / 21

Fisher consistency

Assume (a) X ∈ Hn with EX4 < ∞ and {fi : 1, · · · , mn} is a Σ0-ONB; (b) X ∼ FIC(Γ) for some Γ ∈ B(Hn); and (c) kurt(ΓX, fi), i = 1, . . . , mn are all distinct. Let mn

i=1 τi(hi ⊗ hi) be the spectral decomposition of Σ1(Σ−1/2

X). Let V =

mn

• i=1

(hi ⊗ fi). Then {V ∗Σ−1/2 X, fiHn : i = 1, . . . , mn} is independent. Remark: Σ−1/2 denotes the Moore-Penrose inverse operator of Σ0.

14 / 21

Karhunen-Loève revisited

◮ Classical KL expansion: X = X, fiHfi. ◮ Alternative expansion: X = V ∗Σ−1/2

X, giHgi.

◮ In the latter, the (FOBI) coefficients are independent rather than

• nly uncorrelated.

◮ The resulting transformation X → V ∗Σ−1/2

X is similarly denoted FOBI(X).

◮ X → FOBI(X) is affine-invariant

15 / 21

Consistency

Let λmn denote the smallest eigenvalue of Σ0. Let Fn,X denote the oper- ator Σ1(Σ−1/2 X) and ˆ Fn,X denote its empirical version.

Theorem

If lim supnEX8

Hn < ∞, X1, . . . , Xn are i.i.d X, and n−1/7 ≺ λmn 1,

then ˆ Fn,X − Fn,XOP = OP (n−1/2λ−7/2

mn ).

16 / 21

Functional example: Australian weather data

100 200 300 50 100 150 day Precipitations (mm)

Figure: Australian weather data. Daily measurements in 190 weather stations from 1840 to 1990 (length of records vary from one station to the other).

17 / 21

Australian weather data: principal clustering

18 / 21

Australian weather data: FOBI eigenfunctions

FOBI eigenfunctions 90 180 270 360

• 0.10
• 0.05

0.00 0.05 0.10 FFOBI 4 scores Frequency

• 3
• 2
• 1

1 2 5 10 15 20

Figure: Left: Four functional FOBI eigenfunctions. Right: Histogram of the fourth FOBI score.

19 / 21

Australian weather data: FOBI clustering

20 / 21

References

◮ Cardoso, J. F. (1989), Source separation using higher order moments,

Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, Glasgow, UK, 2109-2112.

◮ Li, B., Van Bever, G., Oja, H., Sabolova, R. and Critchley, F., Func-

tional independent component analysis: an extension of fourth order blind identification. Submitted.

◮ Ramsay, J. and Silverman, B. (2005), Functional Data Analysis, 2nd

Ed., Springer-Verlag.

◮ Tyler, D., Critchley, F., Dümbgen, L. and Oja, H. (2009), Invariant

coordinate selection. Journal of Royal Statistical Society B, 71, 549- 592. Thank you for your attention.

21 / 21