Germain Van Bever (joint work with B. Li, H. Oja, F. Critchley and R. - PowerPoint PPT Presentation

Germain Van Bever (joint work with B. Li, H. Oja, F. Critchley and R. Sabolova) Université de Namur IWAFDA (III), May 24th, 2019

The cocktail party effect 1 / 21

Departing from elliptical symmetry PCA fails to uncover hidden structure... since emphasis is on another dispersion measure. 2 / 21

Departing from elliptical symmetry PCA fails to uncover hidden structure... since emphasis is on another dispersion measure. 3 / 21

Departing from elliptical symmetry PCA fails to uncover hidden structure... since emphasis is on another dispersion measure. 4 / 21

Functional example: Australian weather data 150 Precipitations (mm) 100 50 0 0 100 200 300 day Figure: Australian weather data. Daily measurements in 190 weather stations from 1840 to 1990 (length of records vary from one station to the other). 5 / 21

Australian weather data: principal components 50 0.10 40 0.05 PCA eigenfunctions Frequency 30 0.00 20 -0.05 10 -0.10 0 0 90 180 270 360 -1 0 1 2 3 4 5 PC1 scores Figure: Left: First four (from thickest to thinnest) principal eigenfunctions. Right: Histogram of the first principal score. 6 / 21

Australian weather data: principal clustering 7 / 21

The IC model ◮ X follows an Independent Component Model ( X ∼ IC ( Z ) ) if X = Ω Z, for Z = ( Z 1 , . . . , Z p ) T with independent marginals and Ω a nonsin- gular p × p mixing matrix. ◮ IC models ⊂ BSS models ◮ Independent component analysis (ICA): Find (one) unmixing matrix Γ such that Γ X has independent marginals. ◮ If Z has at most one gaussian marginal, then Γ = Ω − 1 up to permu- tation and multiplication by a diagonal matrix. 8 / 21

Whitening Proposition . Write X st = Σ − 1 / 2 X, where Σ − 1 / 2 Let X ∼ IC(Z) and let Σ = Var ( X ) is the symmetric inverse square root of Σ . Then, Z = U T X st = U T Σ − 1 / 2 X for some orthogonal matrix U = ( u 1 · · · u p ) . ◮ ICA problems are thus often reduced to the estimation of an orthog- onal matrix after whitening the distribution. 9 / 21

The FOBI procedure in a nutshell ( X T X ) XX T � � Let Σ 0 ( X ) = Var ( X ) and let Σ 1 ( X ) = E . If Z has inde- pendent components, then Σ 1 ( Z ) is diagonal. Theorem Let X ∼ IC ( Z ) . Assume the components of Z have finite and dis- tinct kurtoses. Let V Λ V T be the eigendecomposition of Σ 1 ( X st ) . Then, V T X st = V T Σ − 1 / 2 X has independent components. 0 10 / 21

Functional data Observations are typically assumed to belong to H = L 2 ( I ) � allows general treatment of FOBI in a Hilbert space H . Extension to a generic H faces three hurdles: ◮ No notion of marginals in general. ◮ No standardization possible: Σ − 1 / 2 X does not exist. 0 ◮ Intrinsic limit to knowledge requires regularization. 11 / 21

Functional IC model ◮ Assume X ( n ) resides in a finite-dimensional subspace H n of H . ◮ The dimension m n of H n is thought to increase to infinity with n . ◮ The subspaces H n are assumed to be nested in n . Definition Suppose E || X || 2 H n < ∞ and let { f i : i ∈ N 0 } be a Σ 0 -ONB. We say that X follows a functional independent component model with respect to a functional operator Γ ∈ B ( H n ) if the sequence of random variables {� Γ X, f i � : i ∈ N 0 } is independent. If this condition is satisfied then we write X ∼ FIC (Γ) . 12 / 21

FOBI operator ( X ⊗ X ) 2 � � Define the FOBI operator Σ 1 ( X ) as E . Theorem If E � X � 4 H n < ∞ , then Σ 1 ( X ) is a trace-class operator and is unitarily equivariant, that is Σ 1 ( UX ) = U Σ 1 ( X ) U ∗ for any operator satisfying UU ∗ = U ∗ U = I . 13 / 21

Fisher consistency Assume (a) X ∈ H n with E � X � 4 < ∞ and { f i : 1 , · · · , m n } is a Σ 0 -ONB; (b) X ∼ FIC (Γ) for some Γ ∈ B ( H n ) ; and (c) kurt ( � Γ X, f i � ) , i = 1 , . . . , m n are all distinct. i =1 τ i ( h i ⊗ h i ) be the spectral decomposition of Σ 1 (Σ − 1 / 2 Let � m n X ) . Let 0 m n � V = ( h i ⊗ f i ) . i =1 Then {� V ∗ Σ − 1 / 2 X, f i � H n : i = 1 , . . . , m n } is independent. 0 Remark: Σ − 1 / 2 denotes the Moore-Penrose inverse operator of Σ 0 . 0 14 / 21

Karhunen-Loève revisited ◮ Classical KL expansion: X = � � X, f i � H f i . ◮ Alternative expansion: X = � � V ∗ Σ − 1 / 2 X, g i � H g i . 0 ◮ In the latter, the (FOBI) coefficients are independent rather than only uncorrelated. ◮ The resulting transformation X �→ V ∗ Σ − 1 / 2 X is similarly denoted 0 FOBI ( X ) . ◮ X �→ FOBI ( X ) is affine-invariant 15 / 21

Consistency Let λ m n denote the smallest eigenvalue of Σ 0 . Let F n,X denote the oper- ator Σ 1 (Σ − 1 / 2 X ) and ˆ F n,X denote its empirical version. 0 Theorem H n < ∞ , X 1 , . . . , X n are i.i.d X , and n − 1 / 7 ≺ λ m n � 1 , If lim sup n E � X � 8 then � ˆ F n,X − F n,X � OP = O P ( n − 1 / 2 λ − 7 / 2 m n ) . 16 / 21

Functional example: Australian weather data 150 Precipitations (mm) 100 50 0 0 100 200 300 day Figure: Australian weather data. Daily measurements in 190 weather stations from 1840 to 1990 (length of records vary from one station to the other). 17 / 21

Australian weather data: principal clustering 18 / 21

Australian weather data: FOBI eigenfunctions 0.10 20 FOBI eigenfunctions 0.05 15 Frequency 0.00 10 -0.05 5 -0.10 0 0 90 180 270 360 -3 -2 -1 0 1 2 FFOBI 4 scores Figure: Left: Four functional FOBI eigenfunctions. Right: Histogram of the fourth FOBI score. 19 / 21

Australian weather data: FOBI clustering 20 / 21

References ◮ Cardoso, J. F. (1989), Source separation using higher order moments, Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing , Glasgow, UK, 2109-2112. ◮ Li, B., Van Bever, G., Oja, H., Sabolova, R. and Critchley, F., Func- tional independent component analysis: an extension of fourth order blind identification. Submitted. ◮ Ramsay, J. and Silverman, B. (2005), Functional Data Analysis , 2nd Ed., Springer-Verlag. ◮ Tyler, D., Critchley, F., Dümbgen, L. and Oja, H. (2009), Invariant coordinate selection. Journal of Royal Statistical Society B, 71 , 549- 592. Thank you for your attention. 21 / 21