multivariate analysis multivariate analysis

MultivariateAnalysis MultivariateAnalysis AUnifiedPerspective - PowerPoint PPT Presentation

MultivariateAnalysis MultivariateAnalysis AUnifiedPerspective AUnifiedPerspective HarrisonB.Prosper FloridaStateUniversity AdvancedStatisticalTechniquesinParticlePhysics


  1. Multivariate�Analysis Multivariate�Analysis A�Unified�Perspective A�Unified�Perspective Harrison�B.�Prosper Florida�State�University Advanced�Statistical�Techniques�in�Particle�Physics Durham,�UK,�20�March�2002 Multivariate�Analysis�����Harrison�B.�Prosper������Durham,�UK�2002 1

  2. Outline Outline � Introduction� � Some�Multivariate�Methods � Fisher�Linear�Discriminant (FLD) � Principal�Component�Analysis� (PCA) � Independent�Component�Analysis� (ICA) � Self�Organizing�Map� (SOM) � Random�Grid�Search (RGS) � Probability�Density�Estimation (PDE) � Artificial�Neural�Network (ANN) � Support�Vector�Machine� (SVM) � Comments� � Summary Multivariate�Analysis�����Harrison�B.�Prosper������Durham,�UK�2002 2

  3. Introduction�– – i� i� Introduction� � Multivariate�analysis�is�hard! � Our�mathematical�intuition�based�on�analysis�in�one� dimension�often�fails�rather�badly�for�spaces�of�very� high�dimension. � One�should�distinguish�the�problem�to�be�solved�from�the� algorithm�to�solve�it.� � Typically,�the�problems�to�be�solved,�when�viewed�with� sufficient�detachment,�are�relatively�few�in�number� whereas�algorithms�to�solve�them�are�invented�every�day.� Multivariate�Analysis�����Harrison�B.�Prosper������Durham,�UK�2002 3

  4. Introduction�– – ii� ii� Introduction� � So�why�bother�with�multivariate�analysis? � Because: � The�variables�we�use�to�describe�events�are� usually� statistically�dependent . � Therefore,�the�N-d�density�of�the�variables� contains�more�information�than�is�contained�in� the�set�of�1-d�marginal�densities�f i (x i ).� � This�extra�information�may�be�useful Multivariate�Analysis�����Harrison�B.�Prosper������Durham,�UK�2002 4

  5. ✄ ☎ ✟ ✍ ✡ ✄ ✝✞ ✄ ✆ ☎ ✁ ✄ ✆ ✆ ☎ ✝✞ ✏ ✆ → → + p p t t l jets - -1 -1 �✂✁ 105 7 tt 0.3 Dzero�1995 0.2 Top Discovery� 0.1 Aplanarity 0 -1 -1 ✝✂✠ ☛☞✂✌ 700 ✎☞✂✌ 385 W 0.3 0.2 0.1 0 0 100 200 300 400 0 100 200 300 400 (GeV) H T Multivariate�Analysis�����Harrison�B.�Prosper������Durham,�UK�2002 5

  6. Introduction�- - iii iii Introduction� � Problems�that�may�benefit�from�multivariate� analysis:� � Signal�to�background�discrimination � Variable�selection�(e.g.,�to�give�maximum� signal/background�discrimination) � Dimensionality�reduction�of�the� feature space � Finding� regions�of�interest in�the�data N → ℜ 1 � Simplifying�optimization�(by����������������������) : f U � Model�comparison � Measuring�stuff�(e.g.,�tan β in�SUSY) Multivariate�Analysis�����Harrison�B.�Prosper������Durham,�UK�2002 6

  7. Fisher�Linear�Discriminant Discriminant Fisher�Linear� � Purpose � Signal/background�discrimination ⋅ + > ( ) 0 w x b µ Σ | , g x 1 log ( ) µ Σ | , g x 2 → χ µ − χ µ 2 2 ( ) ( ) 2 1 → ⋅ + w x b g�is�a�Gaussian w ⋅ + < 0 w x b Multivariate�Analysis�����Harrison�B.�Prosper������Durham,�UK�2002 7

  8. Principal�Component�Analysis Principal�Component�Analysis � Purpose � Reduce�dimensionality� of�data 1 st principal�axis K = 2 x 2 arg max ( ) w d i w � 1 = 1 i = 1 w x � 2 nd principal�axis i w d i K = ⋅ − 2 arg max [ ( ( ))] � w w x w d w = ⋅ x 1 � 2 1 1 � i i d w x = i i 1 i Multivariate�Analysis�����Harrison�B.�Prosper������Durham,�UK�2002 8

  9. PCA�algorithm�in�practice PCA�algorithm�in�practice � Transform�from� X�=�(x 1 ,..x N ) T to� U�=�(u 1 ,..u N ) T in� which�lowest�order�correlations�are�absent. � Compute� Cov(X) � Compute�its�eigenvalues λ λ λ i λ λ λ λ λ i and�eigenvectors� v i � Construct�matrix� T�=�Col(v i ) T � U�=�TX � Typically,�one�eliminates�u i with�smallest�amount� of�variation Multivariate�Analysis�����Harrison�B.�Prosper������Durham,�UK�2002 9

  10. Independent�Component�Analysis Independent�Component�Analysis � Purpose � Find�statistically�independent�variables.� � Dimensionality�reduction � Basic�Idea � Assume� X =� (x 1 ,..,x N ) T is�a�linear�sum� X�=�AS of�independent�sources� S�=�(s 1 ,..,s N ) T .�Both� A ,� the� mixing� matrix,�and� S are� unknown. � Find�a� de-mixing matrix� T such�that�the� components�of� U�=�TX are� statistically� independent Multivariate�Analysis�����Harrison�B.�Prosper������Durham,�UK�2002 10

  11. ICA- -Algorithm Algorithm ICA Given�two�densities� f(U) and� g(U) one�measure�of�their�“closeness” is�the�Kullback-Leibler�divergence ( ) f U � � ≡ � ≥ ( | ) ( ) log 0 K f g f U dU � � � � ( ) g U � � which�is� zero if,�and�only�if,�� f(U)�=�g(U) .� We�set ∏ = ( ) ( ) g U f i u i i and�minimize� K (� f�|�g ) (now�called�the� mutual� information )�with�respect�to�the�de-mixing�matrix� T .� Multivariate�Analysis�����Harrison�B.�Prosper������Durham,�UK�2002 11

  12. Self�Organizing�Map Self�Organizing�Map � Purpose � Find�regions�of�interest�in�data;�that�is,�clusters. � Summarize�data� � Basic�Idea�(Kohonen,�1988) � Map�each�of� K feature�vectors� X =� (x 1 ,..,x N ) T into�one�of� M� regions�of�interest�defined�by�the� vector� w m so�that�all� X mapped�to�a�given� w m are�closer�to�it�than�to�all�remaining� w m .� � Basically,�perform�a�coarse-graining�of�the� feature�space.� Multivariate�Analysis�����Harrison�B.�Prosper������Durham,�UK�2002 12

  13. Grid�Search� Grid�Search� Purpose:�Signal/Background�discrimination Apply�cuts�at� Apply�cuts�at� each�grid�point� each�grid�point� > x x y i > y y i ( , ) x y We�refer�to� We�refer�to� i i as�a� cut cut- -point point as�a� cut-point x Ndim Number�of�cut-points�~ N bin Ndim Number�of�cut-points�~ N bin Multivariate�Analysis�����Harrison�B.�Prosper������Durham,�UK�2002 13

  14. Random�Grid�Search Random�Grid�Search Take�each�point� each�point�of Take�each�point�of the�signal�class�as� 1 the�signal�class�as� a�cut- -point point a�cut a�cut-point Signal�fraction > x x i > y y i 0 y 1 0 Background�fraction N tot =�#�events�before�cuts N tot =�#�events�before�cuts N cut =�#�events�after�cuts N cut =�#�events�after�cuts Fraction� =�N cut /N tot Fraction� =�N cut /N tot H.B.P.�et�al,�Proceedings,�CHEP�1995 x Multivariate�Analysis�����Harrison�B.�Prosper������Durham,�UK�2002 14

  15. Probability�Density�Estimation Probability�Density�Estimation � Purpose � Signal/background�discrimination � Parameter�estimation � Basic�Idea� � Parzen Estimation�(1960s) − 1 1 x x = ϕ � � ≤ ≤ n ( ) 1 p x n N � � � d N h h � � n � Mixtures = � ϕ << ( ) ( | ) ( ) p x x j q j j N j Multivariate�Analysis�����Harrison�B.�Prosper������Durham,�UK�2002 15

  16. Artificial�Neural�Networks Artificial�Neural�Networks � Purpose � Signal/background�discrimination � Parameter�estimation � Function�estimation � Density�estimation � Basic�Idea � Encode�mapping��(Kolmogorov,�1950s).� → = ϕ ϕ N M : ( ) [ ,.., ] f U U f x F 1 K � Using�a�set�of�1-D�functions. Multivariate�Analysis�����Harrison�B.�Prosper������Durham,�UK�2002 16

  17. Feedforward Networks Networks Feedforward θ θ i 2 � = � + θ → ( ) a w x f a i ij j i i x = 1 j � 1 5 w = � + θ ( , ) ( ( ) ) n x w f w f a ij � � i i x = 1 i 2 f(a) � w i � a Input�nodes Hidden�nodes Output�node Multivariate�Analysis�����Harrison�B.�Prosper������Durham,�UK�2002 17

Recommend


More recommend