Eigenvalues and Eigenvectors Xo --tdn Power Iteration Ann - AXK - PowerPoint PPT Presentation

Eigenvalues and Eigenvectors

Xo --tdn Power Iteration Ann - AXK XKH b 2 2 # $ # & ! 2 = # " 2 $ " % " + $ $ % $ + ⋯ + $ & % & # " # " - Assume that $ " ≠ 0 , the term $ " % " dominates the others when * is very large. 2 Since |# " > |# $ , we have 3 ! ≪ 1 when * is large 3 " I Hence, as * increases, ! 2 converges to a multiple of the first > Hnl eigenvector % " , i.e., 1412122121231 . . - large K → o → u , , × ,

How can we now get the eigenvalues? vector If ! is an eigenvector of / such that rectory / ! = # ! then how can we evaluate the corresponding eigenvalue # ? x. AX or i ↳ ¥¥J = X x . x x = coefficient Rayleigh

Power Iteration Xo - - X = Xo for i. =L , 2 , . . - → HH → growing X. = AX X=xtAx Ex

Normalized Power Iteration - # # + " + $ ) # = + ! # , ! - ! + , " - " + ⋯ + , $ - $ + ! + ! ! & = arbitrary nonzero vector ! & ! & = ! & for * = 1,2, … A 2 = / ! 21" ① ④ is normalized ( # ! 2 = ( # X = ITAI XI Xia

⇒ * d , € 0 Normalized Power Iteration * finite calculation 2 2 # $ # & ! 2 = # " 2 $ " % " + $ $ % $ + ⋯ + $ & % & # " # " - t an Un /A=o Xo = t - - What if the starting vector ) % have no component in the dominant eigenvector - ! ( , ! = 0 )? i tan ( tf ) Kun ) He = da XE Uz t Xie [ as Us t - - = - " ¥liniiu × , k →→ af±* → → power iteration will converge 42 Inpractice wine a ! .

I X , I 71121 Normalized Power Iteration 2 2 # $ # & ! 2 = # " 2 $ " % " + $ $ % $ + ⋯ + $ & % & # " # " What if the first two largest eigenvalues (in magnitude) are the same, |+ ! = |+ " ? 1) + ! and + " both positives Xr → Xk QU , t Xkdz Uz X , = Az = X ⇒ r O → L . combination of y , ye tf → HU , t a uz = 12 = X TAX → X = X ,

Normalized Power Iteration 2 2 # $ # & ! 2 = # " 2 $ " % " + $ $ % $ + ⋯ + $ & % & # " # " What if the first two largest eigenvalues (in magnitude) are the same, |+ ! = |+ " ? odd 2) + ! and + " both negative KT even xr → IN } ( dit da Uz ) . ① , ④ flipped signs → converge - C L in - Xr + " / lat - hi , that -

Normalized Power Iteration 2 2 # $ # & ! 2 = # " 2 $ " % " + $ $ % $ + ⋯ + $ & % & # " # " What if the first two largest eigenvalues (in magnitude) are the same, |+ ! = |+ " ? ① Xz do X , 20 3) + ! and + " opposite signs , Xn → did , Uz t attack kisodd kiseueu - delta ) ITCHY 744.4 take ) a F → no longer have convergence have longer no

Potential pitfalls O Starting vector ! ! may have no component in the dominant eigenvector " " ($ " = 1. o 0) . This is usually unlikely to happen if ! ! is chosen randomly, and in practice not a problem because rounding will usually introduce such component. 2. Risk of eventual overflow (or underflow): in practice the approximated eigenvector is → normalized at each iteration (Normalized Power Iteration) o First two largest eigenvalues (in magnitude) may be the same: |) " | = |) # | . In this 3. case, power iteration will give a vector that is a linear combination of the corresponding eigenvectors: • If signs are the same, the method will converge to correct magnitude of the eigenvalue. If the signs are different, the method will not converge. • This is a “real” problem that cannot be discounted in practice. ÷

ien=oH÷ exact " Error 2 2 # $ # & ! 2 = # " 2 $ " % " + $ $ % $ + ⋯ + $ & % & # " # " × Y÷=4tf¥YUzi Ust -- ¥2,2 - U , t error ¥ntne¥ K → 1K¥ # I Hull ⇐ ÷

Convergence and error - o ( ⇐ f) Henk - "f¥Y÷÷y ⇒ - constant §nH=l¥I%e = / Iff Henk → linear convergence Heath H

A- HI ! Example - X , → Ui ① Xo - → HX - XoH=O -3 ¥ ( " " " - Hx - x. 11=0.3 " " He .lt ' -- Ax - all - fY÷ µ eoH - silos ) 11411 - - ¥130.3 - 17,411194 HEH - . # me . " 11911=0.1536 , " en

Normalized ) ( - DAK power iteration - Xun - of eigenvector ④ multiple to converges → largest eigenvalue A to corresponding in magnitude = = TAI coefficient : X Rayleigh XT x what if I want another eigenvalue ? > I In 1 > I Iz I > I 1 , I - - -

Suppose ! is an eigenvector of / such that I , x → A / ! = # ! - What is an eigenvalue of / 1" ? Ax = Xx Ax ÷ x ④ = ¥isaneiegenvakeeof = the largest ④ - is ? - smallest ¥

- smallest Inverse Power Method eigenvalue Previously we learned that we can use the Power Method to obtain the largest eigenvalue and corresponding eigenvector, by using the update 00 ! 2>" = / ! 2 Suppose there is a single smallest eigenvalue of / . With the previous ordering |# " | > |# $ | ≥ |# ? | ≥ ⋯ > |# & | :÷÷÷ : ¥1 . powdwi# ftp.//Xr+i=AT will converge

Think a about t this q question… Which code snippet is the best option to compute the smallest eigenvalue of the matrix / ? B) A) C) D) E) I have no idea!

Inverse Power Method Xk - /AXm=XI " Xian = A solve ! → XKH la - IHA ) ① factorize A =P LU = Xk - Ochs ) PL U Xrt , - . → Off ) = Ptxr - solve for y - y § ② Ly solve for Kai - Ogi ) ③ U Xan

Cost of computing eigenvalues using = Xk inverse power iteration A Xpet , " ) # 2/0 OCR ) OW ) ' ) goin # Out ) yolk ) g- ④

DI Suppose ! is an eigenvector of / such that / ! = # 8 ! and also ! is an eigenvector of C such that C ! = # 9 ! . What is an eigenvalue of A → Xlix What is an eigenvalue of (/ + 8 9 C) 1" ? B - I ,x ( AtIBjx=X × fAt → - Catfish f- × ÷X=AttzBX * :* :¥ :

Suppose ! is an eigenvector of / such that / ! = # 8 ! and also ! is an eigenvector of C such that C ! = # 9 ! . What is an eigenvalue of What is an eigenvalue of / $ + FC ? X. F - A Xx - B CAITB )x - XX 1x=xi+rI ✓ ° ht - ATTB ? AItTBx= Xx T¥toTx - xx

Eigenvalues of a Shifted Inverse Matrix Suppose the eigenpairs !, # satisfy /! = # ! . - A " is I ? What - TI ) ( x ,I ) - CA ? a . i¥¥ ( A - JIT 'x=5x ¥ - a - r - - oryx - CA It . ± . #¥

Eigenvalues of a Shifted Inverse Matrix 15--171 me then - Power iteration = Xk → Inverse - TI ) Xa , ( A - TI ) ( A ltosmake.tk#J eigenvectors converge to , ÷ x → T to eigenvalue of A which is → converging to T closer

( A - JI ) Yeti He T define = • - - X = % × • random Io / . normalize . y - la .lu CA - TIN /X=xIE' OGE ) = ( A - TI ) • B . P.hu - solve for y four ) for i = I , 2 . . - - - = Ex - Ly solve for + new = y → UX new = Knew 111 X new H X which is the rig - A x : eigenvector corresponding X to T closer

Convergence summary Minear oui ) get ✓ ( Method Cost Convergence : &(( / : & 00h48 & " D $ % " ! !+1 = # ! ! Power Method & ! D D= % ) + $ % " O & # Inverse Power # ! !+1 = ! ! ① Method ④ & #*! 01ns ) Ochs ) ③ & + − * % ) + $ % " Shifted Inverse (# − *+)! !+1 = ! ! Power Method & +" − * closer too 0¥ ↳ Xc → eigenvalue + ! : largest eigenvalue (in magnitude) + " : second largest eigenvalue (in magnitude) + $ : smallest eigenvalue (in magnitude) + $*! : second smallest eigenvalue (in magnitude) + + : closest eigenvalue to < + +" : second closest eigenvalue to <