Principal Component Analysis (PCA) Dr. Veselina Kalinova Max - - PowerPoint PPT Presentation
PCA PCA Principal Component Analysis (PCA) Dr. Veselina Kalinova Max Planck Institute for Radioastronomy 2-nd lecture from the course Introduction to Machine learning: the elegant way to extract information from data, Bonn, MPIfR,14th
Dr. Veselina Kalinova Max Planck Institute for Radioastronomy
2-nd lecture from the course “Introduction to Machine learning: the elegant way to extract information from data”, Bonn, MPIfR,14th of February, 2017
PCA PCA PCA PCA
credit: IBM Data Science Experience http://datascience.ibm.com/blog/the-mathematics-of-machine-learning/
Machine Learning - the elegant way to extract information from complex and multi-dimensional data
PCA PCA PCA PCA
PCA It’s the projection that maximises the area of the shadow and an equivalent measurement is the sums of squares of the distances between points in the projection, we want to see as much of the variation as possible, that’s what PCA does. credit: http://web.stanford.edu/class/bios221/PCA_Slides.html Which projection do you think is better to get more information from the data? PCA PCA PCA
Principal Component Analysis (PCA) Definition
Principal component analysis (PCA) is a statistical procedure that uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called principal components. The number of principal components is less than or equal to the number of original variables. This transformation is defined in such a way that the first principal component has the largest possible variance (that is, accounts for as much
each succeeding component in turn has the highest variance possible under the constraint that it is orthogonal to the preceding components. The resulting vectors are an uncorrelated
credit: Wikipedia PCA PCA PCA PCA
Gaussian distribution centered at (1,3) with a standard deviation of 3 in roughly the (0.866, 0.5) direction and of 1 in the
shown are the eigenvectors of the covariance matrix scaled by the square root of the corresponding eigenvalue, and shifted so their tails are at the mean.
Karl Pearson (1857 - 1936), English mathematician and biostatistician, inventor of PCA in 1901 year.
PC1 captures the direction of the most variation PC2 captures the direction of the 2nd most variation PCn captures the direction of the nnd most variation … (n=100 for our sample. However, we need only n=2 to reconstruct 99 % of the Vc data for all galaxies. )
credit: http://stats.stackexchange.com/questions/2691/making-sense-of-principal-component-analysis-eigenvectors-eigenvalues
red lines represent the eigenvectors’ axes, i.e. PC axes
Checks if the image is a face using PC space
credit: http://archive.cnx.org/contents/ce6cf0ed-4c63-4237-b151-2f4eff8a7b8c@6/facial-recognition- using-eigenfaces-obtaining-eigenfaces
Recognising emotions using PCA (eigenfaces)
credit: Barnabás Póczos
Recognising emotions using PCA (eigenfaces)
Barnabás Póczos
credit:
Representative male face per country
Representative female face per country
36
credit: Barnabás Póczos
credit: Barnabás Póczos
PCA compression: 144D ) 16D
16 most important eigenvectors
2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2PCA compression: 144D ) 3D
2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 2 2 4 6 8 1 0 1 23 most important eigenvectors
Barnabás Póczos
credit:
credit: Barnabás Póczos
Application of PCA to Astronomy
Different CVC due to different potential
Kalinova et al., 2017, MNRAS, submitted
We compare the shapes of the rotation curves in each cell of the radius (x-axis).
Kalinova et al., 2017, MNRAS, submitted
We compare the shape of the rotation curves in both axes - radius and velocity amplitude.
Kalinova et al., 2017, MNRAS, submitted
Vc,rec = (PC1u1 +PC2u2 +PC3u3 +PC4u4 +PC5u5)+Vc.
reconstructed mean velocity
P C1 = +0.79, P C2 = −1.86, PC3 = −1.98, PC4 = −1.90, PC5 = +1.82.
Main PC Eigenvectors of Vc
0.2 0.4 0.6 0.8 1.0 1.2 1.4 R/Re
20 40 60 80 100 120 Eigenvectors [km s-1] u1 (93.33%) u2 (5.41%) u3 (0.87%) u4 (0.29%) u5 (0.07%)
Reconstructed Vc via PCA
0.2 0.4 0.6 0.8 1.0 1.2 1.4 R/Re 150 200 250 300 350 400 Vc [kms-1] NGC7671 PC1= 1.29 PC2=-2.50 PC3=-1.92 PC4=-2.41 PC5= 0.84
Kalinova et al., 2017, MNRAS, submitted
Kalinova et al., 2017, MNRAS, submitted
PCA analysis
Kalinova et al., 2017, MNRAS, submitted
credit:
PCA analysis K-means analysis
Kalinova et al., 2017, MNRAS, submitted
Kalinova et al., 2017, MNRAS, submitted
Kalinova et al., 2017, MNRAS, submitted
Kalinova et al., 2017, MNRAS, submitted
Kalinova et al., 2017, MNRAS, submitted
Distribution of some galaxy properties by the established 4 dynamical classes Box-and-Whisker representation: boxes contains 50 %
the extended dashed line corresponds to 3-σ
distribution.
Removing 20 galaxies on each run and estimate the stability
Kalinova et al., 2017, MNRAS, submitted
“A tutorial on Principal Component Analysis”, Lindsay Smith, 2002, http://faculty.iiit.ac.in/~mkrishna/PrincipalComponents.pdf “The Elements of Statistical Learning”, Trevor Hastie, Robert Tibshirani, Jerome Friedman, Springer, 2nd edition, 2009 (Chapter 14.5) http://statweb.stanford.edu/~tibs/ElemStatLearn/ Presentation “Principal Component Analysis”, Barnabos Poszos, University of Alberta, 2009 https://www.cs.cmu.edu/~bapoczos/other_presentations/PCA_24_10_2009.pdf “A tutorial on Principal Component Analysis: derivation, discussion and singular value decomposition”, Jonathon Shlens https://arxiv.org/pdf/1404.1100.pdf