Matrix Calculations: Eigenvalues and Eigenvectors H. Geuvers (and A. - PowerPoint PPT Presentation

Eigenvalues and Eigenvectors Radboud University Nijmegen Applications of Eigenvalues and Eigenvectors Matrix Calculations: Eigenvalues and Eigenvectors H. Geuvers (and A. Kissinger) Institute for Computing and Information Sciences Radboud University Nijmegen Version: spring 2016 H. Geuvers (and A. Kissinger) Version: spring 2016 Matrix Calculations 1 / 37

Eigenvalues and Eigenvectors Radboud University Nijmegen Applications of Eigenvalues and Eigenvectors Outline Eigenvalues and Eigenvectors Applications of Eigenvalues and Eigenvectors H. Geuvers (and A. Kissinger) Version: spring 2016 Matrix Calculations 2 / 37

Eigenvalues and Eigenvectors Radboud University Nijmegen Applications of Eigenvalues and Eigenvectors Political swingers re-revisited, part I • Recall the political transisition matrix � 0 . 8 0 . 1 � � 8 1 � = 1 P = 10 0 . 2 0 . 9 2 9 • with some iterations: � 91 . 5 � 89 . 05 � � � � � 100 � � � 100 � � 100 95 P 2 · P 3 · P · = = = · · · 150 155 150 158 . 5 150 160 . 95 • Does this converge to a stable lefty-righty division? If so, what is a stable division? � � � � 83 1 83 1 3 3 • Check for yourself: P · = 166 2 166 2 3 3 H. Geuvers (and A. Kissinger) Version: spring 2016 Matrix Calculations 4 / 37

Eigenvalues and Eigenvectors Radboud University Nijmegen Applications of Eigenvalues and Eigenvectors Political swingers re-revisited, part II � x � � x � • When do we have P · = ? y y • This involves: � 0 . 8 x + 0 . 1 y = x � (0 . 8 − 1) x + 0 . 1 y = 0 so 0 . 2 x + 0 . 9 y = y 0 . 2 x − (0 . 9 − 1) y = 0 � − 0 . 2 x + 0 . 1 y = 0 � − 2 x + y = 0 ie. thus so y = 2 x 0 . 2 x − 0 . 1 y = 0 2 x − y = 0 � x � x � � • Indeed, P · = Twice as many righties is stable! 2 x 2 x • We found it by solving (homogeneous) equations given by the matrix: � − 0 . 2 � � − 2 � 0 . 1 1 1 P − I 2 = = 10 0 . 2 − 0 . 1 2 − 1 H. Geuvers (and A. Kissinger) Version: spring 2016 Matrix Calculations 5 / 37

Eigenvalues and Eigenvectors Radboud University Nijmegen Applications of Eigenvalues and Eigenvectors Eigenvector and eigenvalues Definition Assume an n × n matrix A . An eigenvector for A is a non-null vector v � = 0 for which there is an eigenvalue λ ∈ R with: A · v = λ · v Example � 100 � � 8 1 � is an eigenvector for P = 1 with eigenvalue λ = 1. 200 10 2 9 H. Geuvers (and A. Kissinger) Version: spring 2016 Matrix Calculations 6 / 37

Eigenvalues and Eigenvectors Radboud University Nijmegen Applications of Eigenvalues and Eigenvectors Two basic results Lemma An eigenvector has at most one eigenvalue Proof : Assume A · v = λ 1 v and A · v = λ 2 v . Then: 0 = A · v − A · v = λ 1 v − λ 2 v = ( λ 1 − λ 2 ) v Since v � = 0 we must have λ 1 − λ 2 = 0, and thus λ 1 = λ 2 . - Lemma If v is an eigenvector, then so is a v , for each a � = 0 . Proof : If A · v = λ v , then: A · ( a v ) = a ( A · v ) since matrix application is linear = a ( λ v ) = ( a λ ) v = ( λ a ) v = λ ( a v ) . - H. Geuvers (and A. Kissinger) Version: spring 2016 Matrix Calculations 7 / 37

Eigenvalues and Eigenvectors Radboud University Nijmegen Applications of Eigenvalues and Eigenvectors Finding eigenvectors and eigenvalues • We seek a eigenvector v and eigenvalue λ ∈ R with A · v = λ v • That is: λ and v ( v � = 0) such that ( A − λ · I ) · v = 0 • Thus, we seek λ for which the system of equations corresponding to the matrix A − λ · I has a non-zero solution • Hence we seek λ ∈ R for which the matrix A − λ · I does not have n pivots in its echelon form • This means: we seek λ ∈ R such that A − λ · I is not-invertible • So we need: det( A − λ · I ) = 0 • This can be seen as an equation, with λ as variable • This det( A − λ · I ) is called the characteristic polynomial of the matrix A H. Geuvers (and A. Kissinger) Version: spring 2016 Matrix Calculations 8 / 37

Eigenvalues and Eigenvectors Radboud University Nijmegen Applications of Eigenvalues and Eigenvectors Eigenvalue example I � 1 5 � • Task : find eigenvalues of matrix A = 3 3 � 1 5 � � λ 0 � � 1 − λ � 5 • Note: A − λ · I = − = 3 3 0 λ 3 3 − λ • Thus: � � 1 − λ 5 � � det( A − λ · I ) = 0 ⇐ ⇒ � = 0 � � � � 3 3 − λ � ⇐ ⇒ (1 − λ )(3 − λ ) − 5 · 3 = 0 ⇒ λ 2 − 4 λ − 12 = 0 ⇐ ⇐ ⇒ ( λ − 6)( λ + 2) = 0 ⇐ ⇒ λ = 6 or λ = − 2 . H. Geuvers (and A. Kissinger) Version: spring 2016 Matrix Calculations 9 / 37

Eigenvalues and Eigenvectors Radboud University Nijmegen Applications of Eigenvalues and Eigenvectors Recall: abc-formula • Consider a second-degree (quadratic) equation ax 2 + bx + c = 0 (for a � = 0) • Its solutions are: √ b 2 − 4 ac s 1 , 2 = − b ± 2 a • These solutions coincide (ie. s 1 = s 2 ) if b 2 − 4 ac = 0 • Real solutions do not exist if b 2 − 4 ac < 0 (But “complex number” solutions do exist in this case.) • [ Recall, if s 1 and s 2 are solutions of ax 2 + bx + c = 0, then we can write ax 2 + bx + c = a ( x − s 1 )( x − s 2 ) ] H. Geuvers (and A. Kissinger) Version: spring 2016 Matrix Calculations 10 / 37

Eigenvalues and Eigenvectors Radboud University Nijmegen Applications of Eigenvalues and Eigenvectors Higher degree polynomial equations • For third and fourth degree polynomial equations there are (complicated) formulas for the solutions. • For degree ≥ 5 no such formulas exist (proved by Abel) • In those cases one can at most use approximations. • In the examples in this course the solutions will typically be “obvious”. H. Geuvers (and A. Kissinger) Version: spring 2016 Matrix Calculations 11 / 37

Eigenvalues and Eigenvectors Radboud University Nijmegen Applications of Eigenvalues and Eigenvectors Eigenvalue example II   3 − 1 − 1 • Task : find eigenvalues of A = − 12 0 5   4 − 2 − 1 � � 3 − λ − 1 − 1 � � � � • Characteristic polynomial is − 12 − λ 5 � � � � 4 − 2 − 1 − λ � � � � � � � � − λ 5 − 1 − 1 − 1 − 1 � � � � � � = (3 − λ ) � + 12 � + 4 � � � � � � − 2 − 1 − λ − 2 − 1 − λ − λ 5 � � � � � � � � � � � � � � � � = (3 − λ ) λ (1 + λ ) + 10 + 12 1 + λ − 2 + 4 − 5 − λ = (3 − λ )( λ 2 + λ + 10) + 12( λ − 1) − 20 − 4 λ = 3 λ 2 + 3 λ + 30 − λ 3 − λ 2 − 10 λ + 12 λ − 12 − 20 − 4 λ = − λ 3 + 2 λ 2 + λ − 2 . H. Geuvers (and A. Kissinger) Version: spring 2016 Matrix Calculations 12 / 37

Eigenvalues and Eigenvectors Radboud University Nijmegen Applications of Eigenvalues and Eigenvectors Eigenvalue example II (cntd) • We need to solve − λ 3 + 2 λ 2 + λ − 2 = 0 • We try a few “obvious” values: λ = 1 YES! • Reduce from degree 3 to 2, by separating ( λ − 1) in: − λ 3 + 2 λ 2 + λ − 2 = ( λ − 1)( a λ 2 + b λ + c ) = a λ 3 + ( b − a ) λ 2 + ( c − b ) λ − c • This works for a = − 1, b = 1, c = 2 • Now we use “abc” for the equation − λ 2 + λ + 2 = 0 • Solutions: λ = − 1 ± √ 1 + 4 · 2 = − 1 ± 3 giving λ = 2 , − 1 − 2 − 2 • All three eigenvalues: λ = 1 , λ = − 1 , λ = 2 H. Geuvers (and A. Kissinger) Version: spring 2016 Matrix Calculations 13 / 37

Eigenvalues and Eigenvectors Radboud University Nijmegen Applications of Eigenvalues and Eigenvectors Getting eigenvectors • Once we have eigenvalues λ i for a matrix A we can find corresponding eigenvectors v i , with A · v i = λ i v i • These v i appear as the solutions of ( A − λ i · I ) · v = 0 • We can make a convenient choice, using that scalar multiplications a · v i are also a solution • We use standard techniqes for solving such equations. H. Geuvers (and A. Kissinger) Version: spring 2016 Matrix Calculations 14 / 37

Eigenvalues and Eigenvectors Radboud University Nijmegen Applications of Eigenvalues and Eigenvectors Eigenvector example I � 1 5 � Recall the eigenvalues λ = − 2 , λ = 6 for A = 3 3 � 1 + 2 � � 3 5 � 5 λ = − 2 gives matrix A − λ I = = 3 3 + 2 3 5 � 3 x + 5 y = 0 • Corresponding system of equations 3 x + 5 y = 0 • Solution choice x = − 5 , y = 3, so ( − 5 , 3) is eigenvector (of matrix A with eigenvalue λ = − 2) • Check: � � � 1 5 � � − 5 � � − 5 + 15 � � 10 � � − 5 · = = = − 2 3 3 3 − 15 + 9 − 6 3 H. Geuvers (and A. Kissinger) Version: spring 2016 Matrix Calculations 15 / 37

Eigenvalues and Eigenvectors Radboud University Nijmegen Applications of Eigenvalues and Eigenvectors Eigenvector example I (cntd) � 1 − 6 � � − 5 � 5 5 λ = 6 gives matrix A − λ I = = 3 3 − 6 3 − 3 � − 5 x + 5 y = 0 • Corresponding system of equations 3 x − 3 y = 0 • Solution choice x = 1 , y = 1, so (1 , 1) is eigenvector (of matrix A with eigenvalue λ = 6) • Check: � 1 5 � � � 1 � � 1 + 5 � � 6 � � 1 � · = = = 6 3 3 1 3 + 3 6 1 H. Geuvers (and A. Kissinger) Version: spring 2016 Matrix Calculations 16 / 37