Introduction The method Summary Structure-Preserving Method for Dimension Reduction Ewa Nowakowska Institute of Computer Science, Polish Academy of Sciences Joint Statistical Meetings, San Diego, CA 29 July 2012 Ewa Nowakowska Institute of Computer Science, Polish Academy of Sciences Structure-Preserving Method for Dimension Reduction
Introduction The method Summary Outline Introduction 1 Model and notation Basic definitions Concept The method 2 Isotropic transformation Weighting Algorithm Summary 3 Ewa Nowakowska Institute of Computer Science, Polish Academy of Sciences Structure-Preserving Method for Dimension Reduction
Introduction The method Summary Outline Introduction 1 Model and notation Basic definitions Concept The method 2 Isotropic transformation Weighting Algorithm Summary 3 Ewa Nowakowska Institute of Computer Science, Polish Academy of Sciences Structure-Preserving Method for Dimension Reduction
Introduction The method Summary Model and notation Model and notation data: X = ( x 1 , . . . , x n ) T , X ∈ R n × d model: f ( x ) = π 1 f 1 ( µ 1 , Σ 1 )( x ) + . . . + π k f k ( µ k , Σ k )( x ) , where 1 2 ( x − µ l ) T Σ − 1 e − 1 ( x − µ l ) . f l ( µ l , Σ l )( x ) = √ 2 π ) d √ det Σ l l ( additional assumptions: equal mixing factors π 1 = · · · = π k = 1 k heterogeneity Σ l 1 � = Σ l 2 large space dimension d > k − 1 large sample size n ≫ d number of components k known Ewa Nowakowska Institute of Computer Science, Polish Academy of Sciences Structure-Preserving Method for Dimension Reduction
Introduction The method Summary Model and notation Model and notation data: X = ( x 1 , . . . , x n ) T , X ∈ R n × d model: f ( x ) = π 1 f 1 ( µ 1 , Σ 1 )( x ) + . . . + π k f k ( µ k , Σ k )( x ) , where 1 2 ( x − µ l ) T Σ − 1 e − 1 ( x − µ l ) . f l ( µ l , Σ l )( x ) = √ 2 π ) d √ det Σ l l ( additional assumptions: equal mixing factors π 1 = · · · = π k = 1 k heterogeneity Σ l 1 � = Σ l 2 large space dimension d > k − 1 large sample size n ≫ d number of components k known Ewa Nowakowska Institute of Computer Science, Polish Academy of Sciences Structure-Preserving Method for Dimension Reduction
Introduction The method Summary Model and notation Model and notation data: X = ( x 1 , . . . , x n ) T , X ∈ R n × d model: f ( x ) = π 1 f 1 ( µ 1 , Σ 1 )( x ) + . . . + π k f k ( µ k , Σ k )( x ) , where 1 2 ( x − µ l ) T Σ − 1 e − 1 ( x − µ l ) . f l ( µ l , Σ l )( x ) = √ 2 π ) d √ det Σ l l ( additional assumptions: equal mixing factors π 1 = · · · = π k = 1 k heterogeneity Σ l 1 � = Σ l 2 large space dimension d > k − 1 large sample size n ≫ d number of components k known Ewa Nowakowska Institute of Computer Science, Polish Academy of Sciences Structure-Preserving Method for Dimension Reduction
Introduction The method Summary Basic definitions Basic facts and definitions Let µ X and Σ X be the empirical estimates of µ and Σ . Definition (Scatter decomposition) Let T X = n Σ X . Then T X = W X + B X constitutes the total scatter decomposition to its between and within cluster component. Definition (Isotropic position) We say that data is in isotropic position if µ X = 0 and T X = I . Definition (Principal component subspace PC ( k − 1 ) ) By principal component subspace PC ( k − 1 ) we understand the subspace spanned by the first k − 1 principal components. Ewa Nowakowska Institute of Computer Science, Polish Academy of Sciences Structure-Preserving Method for Dimension Reduction
Introduction The method Summary Basic definitions Basic facts and definitions Let µ X and Σ X be the empirical estimates of µ and Σ . Definition (Scatter decomposition) Let T X = n Σ X . Then T X = W X + B X constitutes the total scatter decomposition to its between and within cluster component. Definition (Isotropic position) We say that data is in isotropic position if µ X = 0 and T X = I . Definition (Principal component subspace PC ( k − 1 ) ) By principal component subspace PC ( k − 1 ) we understand the subspace spanned by the first k − 1 principal components. Ewa Nowakowska Institute of Computer Science, Polish Academy of Sciences Structure-Preserving Method for Dimension Reduction
Introduction The method Summary Basic definitions Basic facts and definitions Let µ X and Σ X be the empirical estimates of µ and Σ . Definition (Scatter decomposition) Let T X = n Σ X . Then T X = W X + B X constitutes the total scatter decomposition to its between and within cluster component. Definition (Isotropic position) We say that data is in isotropic position if µ X = 0 and T X = I . Definition (Principal component subspace PC ( k − 1 ) ) By principal component subspace PC ( k − 1 ) we understand the subspace spanned by the first k − 1 principal components. Ewa Nowakowska Institute of Computer Science, Polish Academy of Sciences Structure-Preserving Method for Dimension Reduction
Introduction The method Summary Basic definitions Basic facts and definitions Let µ X and Σ X be the empirical estimates of µ and Σ . Definition (Scatter decomposition) Let T X = n Σ X . Then T X = W X + B X constitutes the total scatter decomposition to its between and within cluster component. Definition (Isotropic position) We say that data is in isotropic position if µ X = 0 and T X = I . Definition (Principal component subspace PC ( k − 1 ) ) By principal component subspace PC ( k − 1 ) we understand the subspace spanned by the first k − 1 principal components. Ewa Nowakowska Institute of Computer Science, Polish Academy of Sciences Structure-Preserving Method for Dimension Reduction
Introduction The method Summary Basic definitions Basic facts and definitions Definition (Fisher’s subspace S ∗ ) We define Fisher’s subspace (Fisher’s discriminant) as � k − 1 j = 1 v T j B X v j S ∗ = argmax , � k − 1 j = 1 v T j T X v j S ⊂ R d dim ( S )= k − 1 where v 1 , . . . , v k − 1 is the orthonormal basis for S . Equivalently , S ∗ – solution to an eigenproblem with T − 1 X B X . Definition (Structure distinctness coefficient ¯ λ X ) k − 1 1 T − 1 λ X = B X ¯ � λ X j k − 1 j = 1 Ewa Nowakowska Institute of Computer Science, Polish Academy of Sciences Structure-Preserving Method for Dimension Reduction
Introduction The method Summary Basic definitions Basic facts and definitions Definition (Fisher’s subspace S ∗ ) We define Fisher’s subspace (Fisher’s discriminant) as � k − 1 j = 1 v T j B X v j S ∗ = argmax , � k − 1 j = 1 v T j T X v j S ⊂ R d dim ( S )= k − 1 where v 1 , . . . , v k − 1 is the orthonormal basis for S . Equivalently , S ∗ – solution to an eigenproblem with T − 1 X B X . Definition (Structure distinctness coefficient ¯ λ X ) k − 1 1 T − 1 λ X = B X ¯ � λ X j k − 1 j = 1 Ewa Nowakowska Institute of Computer Science, Polish Academy of Sciences Structure-Preserving Method for Dimension Reduction
Introduction The method Summary Basic definitions Basic facts and definitions Definition (Fisher’s subspace S ∗ ) We define Fisher’s subspace (Fisher’s discriminant) as � k − 1 j = 1 v T j B X v j S ∗ = argmax , � k − 1 j = 1 v T j T X v j S ⊂ R d dim ( S )= k − 1 where v 1 , . . . , v k − 1 is the orthonormal basis for S . Equivalently , S ∗ – solution to an eigenproblem with T − 1 X B X . Definition (Structure distinctness coefficient ¯ λ X ) k − 1 1 T − 1 λ X = B X ¯ � λ X j k − 1 j = 1 Ewa Nowakowska Institute of Computer Science, Polish Academy of Sciences Structure-Preserving Method for Dimension Reduction
Recommend
More recommend
