When tensor decomposition meets compressed sensing Pierre Comon - PowerPoint PPT Presentation

Intro General Exact CP Approx CP NonNeg Coherence AAP END When tensor decomposition meets compressed sensing Pierre Comon I3S, CNRS, University of Nice - Sophia Antipolis, France Collaborator: Lek-Heng Lim Sept. 27-30, 2010 Pierre Comon LVA/ICA – Sept. 2010 1 / 43

Intro General Exact CP Approx CP NonNeg Coherence AAP END Sparse representation & BSS Blind Source Separation & Sparse Representation x = H s H is K × P , underdetermined : K < P Sparse representation : Columns h n ∈ D , known dictionary BSS : H unknown • s sparse • s not sparse Pierre Comon LVA/ICA – Sept. 2010 2 / 43

Intro General Exact CP Approx CP NonNeg Coherence AAP END Sparse representation & BSS Blind Source Separation & Sparse Representation x = H s H is K × P , underdetermined : K < P Sparse representation : Columns h n ∈ D , known dictionary BSS : H unknown • s sparse • s not sparse Pierre Comon LVA/ICA – Sept. 2010 2 / 43

Intro General Exact CP Approx CP NonNeg Coherence AAP END Sparse representation & BSS Blind Source Separation & Sparse Representation x = H s H is K × P , underdetermined : K < P Sparse representation : Columns h n ∈ D , known dictionary BSS : H unknown • s sparse • s not sparse Pierre Comon LVA/ICA – Sept. 2010 2 / 43

Intro General Exact CP Approx CP NonNeg Coherence AAP END Sparse representation & BSS Blind Source Separation & Sparse Representation x = H s H is K × P , underdetermined : K < P Sparse representation : Columns h n ∈ D , known dictionary BSS : H unknown • s sparse • s not sparse Pierre Comon LVA/ICA – Sept. 2010 2 / 43

Intro General Exact CP Approx CP NonNeg Coherence AAP END Sparse representation & BSS Blind Source Separation & Sparse Representation x = H s H is K × P , underdetermined : K < P Sparse representation : Columns h n ∈ D , known dictionary BSS : H unknown • s sparse • s not sparse Pierre Comon LVA/ICA – Sept. 2010 2 / 43

Intro General Exact CP Approx CP NonNeg Coherence AAP END Statistical approach Blind identification of linear mixtures Linear mixtures: x = H s If s ℓ statistically independent , we have the core equation : P � ϕ ℓ ( u T H ) Ψ( u ) = ℓ =1 Take 3rd derivatives at point u : P � T ijk ( u ) = H i ℓ H j ℓ H k ℓ C ℓℓℓ ( u ) (1) ℓ =1 At u = 0 ➽ symmetric decomposition of T ijk [HOS] At u � = 0 ➽ [Taleb, Comon-Rajih, Yeredor] Pierre Comon LVA/ICA – Sept. 2010 3 / 43

Intro General Exact CP Approx CP NonNeg Coherence AAP END Statistical approach Blind identification of linear mixtures Linear mixtures: x = H s If s ℓ statistically independent , we have the core equation : P � ϕ ℓ ( u T H ) Ψ( u ) = ℓ =1 Take 3rd derivatives at point u : P � T ijk ( u ) = H i ℓ H j ℓ H k ℓ C ℓℓℓ ( u ) (1) ℓ =1 At u = 0 ➽ symmetric decomposition of T ijk [HOS] At u � = 0 ➽ [Taleb, Comon-Rajih, Yeredor] Pierre Comon LVA/ICA – Sept. 2010 3 / 43

Intro General Exact CP Approx CP NonNeg Coherence AAP END Statistical approach Blind identification of linear mixtures Linear mixtures: x = H s If s ℓ statistically independent , we have the core equation : P � ϕ ℓ ( u T H ) Ψ( u ) = ℓ =1 Take 3rd derivatives at point u : P � T ijk ( u ) = H i ℓ H j ℓ H k ℓ C ℓℓℓ ( u ) (1) ℓ =1 At u = 0 ➽ symmetric decomposition of T ijk [HOS] At u � = 0 ➽ [Taleb, Comon-Rajih, Yeredor] Pierre Comon LVA/ICA – Sept. 2010 3 / 43

Intro General Exact CP Approx CP NonNeg Coherence AAP END Statistical approach Blind identification of linear mixtures Linear mixtures: x = H s If s ℓ statistically independent , we have the core equation : P � ϕ ℓ ( u T H ) Ψ( u ) = ℓ =1 Take 3rd derivatives at point u : P � T ijk ( u ) = H i ℓ H j ℓ H k ℓ C ℓℓℓ ( u ) (1) ℓ =1 At u = 0 ➽ symmetric decomposition of T ijk [HOS] At u � = 0 ➽ [Taleb, Comon-Rajih, Yeredor] Pierre Comon LVA/ICA – Sept. 2010 3 / 43

Intro General Exact CP Approx CP NonNeg Coherence AAP END Statistical approach Blind identification of linear mixtures Linear mixtures: x = H s If s ℓ statistically independent , we have the core equation : P � ϕ ℓ ( u T H ) Ψ( u ) = ℓ =1 Take 3rd derivatives at point u : P � T ijk ( u ) = H i ℓ H j ℓ H k ℓ C ℓℓℓ ( u ) (1) ℓ =1 At u = 0 ➽ symmetric decomposition of T ijk [HOS] At u � = 0 ➽ [Taleb, Comon-Rajih, Yeredor] Pierre Comon LVA/ICA – Sept. 2010 3 / 43

Intro General Exact CP Approx CP NonNeg Coherence AAP END Tensors: General items Pierre Comon LVA/ICA – Sept. 2010 4 / 43

Intro General Exact CP Approx CP NonNeg Coherence AAP END Notation Arrays and Multi-linearity A tensor of order d is a multi-linear map: S ∗ ⊗ S ∗ ⊗ . . . ⊗ m → S m +1 ⊗ ⊗ . . . ⊗ ⊗ S d 1 ⊗ ⊗ ⊗ ⊗ ⊗ Once bases of spaces S ℓ are fixed, they can be represented by d-way arrays of coordinates ➽ bilinear form, or linear operator: represented by a matrix ➽ trilinear form, or bilinear operator: by a 3rd order tensor . Pierre Comon LVA/ICA – Sept. 2010 5 / 43

Intro General Exact CP Approx CP NonNeg Coherence AAP END Notation Arrays and Multi-linearity A tensor of order d is a multi-linear map: S ∗ ⊗ S ∗ ⊗ . . . ⊗ m → S m +1 ⊗ ⊗ . . . ⊗ ⊗ S d 1 ⊗ ⊗ ⊗ ⊗ ⊗ Once bases of spaces S ℓ are fixed, they can be represented by d-way arrays of coordinates ➽ bilinear form, or linear operator: represented by a matrix ➽ trilinear form, or bilinear operator: by a 3rd order tensor . Pierre Comon LVA/ICA – Sept. 2010 5 / 43

Intro General Exact CP Approx CP NonNeg Coherence AAP END Notation Multi-linearity Compact notation Linear change in contravariant spaces: T ′ � = A in B jp C kq T npq ijk npq Denoted compactly T ′ = ( A , B , C ) · T (2) Example : covariance matrix z = Ax ⇒ R z = ( A , A ) · R x = A R x A T Pierre Comon LVA/ICA – Sept. 2010 6 / 43

Intro General Exact CP Approx CP NonNeg Coherence AAP END Definitions Decomposable tensor A d th order “ decomposable ” tensor is the tensor product of d vectors: T = u ⊗ ⊗ v ⊗ ⊗ w ⊗ . . . ⊗ ⊗ ⊗ ⊗ and has coordinates T ij ... k = u i v j . . . w k . may be seen as a discretization of multivariate functions whose variables separate : t ( x , y , . . . , z ) = u ( x ) v ( y ) . . . w ( z ) Nothing else but rank-1 tensors, with forthcoming definition Pierre Comon LVA/ICA – Sept. 2010 7 / 43

Intro General Exact CP Approx CP NonNeg Coherence AAP END Definitions Decomposable tensor A d th order “ decomposable ” tensor is the tensor product of d vectors: T = u ⊗ ⊗ v ⊗ ⊗ w ⊗ . . . ⊗ ⊗ ⊗ ⊗ and has coordinates T ij ... k = u i v j . . . w k . may be seen as a discretization of multivariate functions whose variables separate : t ( x , y , . . . , z ) = u ( x ) v ( y ) . . . w ( z ) Nothing else but rank-1 tensors, with forthcoming definition Pierre Comon LVA/ICA – Sept. 2010 7 / 43

Intro General Exact CP Approx CP NonNeg Coherence AAP END Definitions Decomposable tensor A d th order “ decomposable ” tensor is the tensor product of d vectors: T = u ⊗ ⊗ v ⊗ ⊗ w ⊗ . . . ⊗ ⊗ ⊗ ⊗ and has coordinates T ij ... k = u i v j . . . w k . may be seen as a discretization of multivariate functions whose variables separate : t ( x , y , . . . , z ) = u ( x ) v ( y ) . . . w ( z ) Nothing else but rank-1 tensors, with forthcoming definition Pierre Comon LVA/ICA – Sept. 2010 7 / 43

Intro General Exact CP Approx CP NonNeg Coherence AAP END Definitions Example � � 1 Take v = − 1 � � 1 − 1 − 1 1 ⊗ 3 def Then v ⊗ ⊗ = v ⊗ ⊗ v ⊗ ⊗ v = ⊗ ⊗ − 1 1 1 − 1 This is a “decomposable” symmetric tensor → rank-1 ✇ ✇ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ✇ . . . ✇ . . . . . . . . . . . . . . . . . . . ✇ ✇ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ✇ . . . ✇ . . . . . . . . . . . . . . . . . . . blue bullets = 1, red bullets = − 1. Pierre Comon LVA/ICA – Sept. 2010 8 / 43

Intro General Exact CP Approx CP NonNeg Coherence AAP END Definitions Example � � 1 Take v = − 1 � � 1 − 1 − 1 1 ⊗ 3 def Then v ⊗ ⊗ = v ⊗ ⊗ v ⊗ ⊗ v = ⊗ ⊗ − 1 1 1 − 1 This is a “decomposable” symmetric tensor → rank-1 ✇ ✇ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ✇ . . . ✇ . . . . . . . . . . . . . . . . . . . ✇ ✇ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ✇ . . . ✇ . . . . . . . . . . . . . . . . . . . blue bullets = 1, red bullets = − 1. Pierre Comon LVA/ICA – Sept. 2010 8 / 43