[PPT] - Eigenvalues and Eigenvectors Xo --tdn Power Iteration Ann - AXK PowerPoint Presentation

SLIDE 1

Eigenvalues and Eigenvectors

SLIDE 2

Power Iteration

!2 = #" 2 $"%" + $$ #$ #"

2

%$ + ⋯ + $& #& #"

2

%& Assume that $" ≠ 0, the term $"%" dominates the others when * is very large. Since |#" > |#$ , we have 3!

3" 2

≪ 1 when * is large Hence, as * increases, !2 converges to a multiple of the first eigenvector %", i.e.,

Xo --tdn XKH

AXK

Ann

b

I

1412122121231

. . > Hnl

K→o→

u, ,×,

large

SLIDE 3

How can we now get the eigenvalues?

If ! is an eigenvector of / such that / ! = # ! then how can we evaluate the corresponding eigenvalue #?

rectory

vector

x. AX

= X x . x

x =

r i↳¥¥J

Rayleigh

coefficient

SLIDE 4

Power Iteration

Xo

-

X = Xo

for

i. =L, 2,
.

.

X. = AX

→ HH → growing

X=xtAx

Ex

SLIDE 5

Normalized Power Iteration

!& = arbitrary nonzero vector !& = !& !& for * = 1,2, … A2 = / !21" !2 =

(# (# )# = +! # ,!-! + ," +" +! #

" + ⋯ + ,$

+$ +! #

$
① ④ is normalized

X =ITAI

XI Xia

SLIDE 6

Normalized Power Iteration

!2 = #" 2 $"%" + $$ #$ #"

2

%$ + ⋯ + $& #& #"

2

%&

What if the starting vector )% have no component in the dominant eigenvector -! (,! = 0)?

*d,€0

* finite calculation Xo =

t

t anUn /A=o

He = da XE Uz t Xie [as

Us t

i tan (tf)

Kun)

= -

×,k→→af±*→ "¥liniiu

→ power iteration will

converge 42

Inpractice

.

⇒

wine a !

SLIDE 7

Normalized Power Iteration

!2 = #" 2 $"%" + $$ #$ #"

2

%$ + ⋯ + $& #& #"

2

%&

What if the first two largest eigenvalues (in magnitude) are the same, |+! = |+" ? 1) +! and +" both positives

I X,I 71121

Xr → Xk QU, t XkdzUz

X, = Az = X

tf

→ HU, t a uz

→ L . combination of y ,ye

⇒ r O

→ X = X,

= 12 = XTAX

SLIDE 8

Normalized Power Iteration

!2 = #" 2 $"%" + $$ #$ #"

2

%$ + ⋯ + $& #& #"

2

%&

What if the first two largest eigenvalues (in magnitude) are the same, |+! = |+" ? 2) +! and +" both negative

dd

xr →IN} ( dit da Uz)

KT even

in

→ converge

L

C

. ① ,

④ flipped signs

Xr

/

lat - hi, that

+ "

SLIDE 9

Normalized Power Iteration

!2 = #" 2 $"%" + $$ #$ #"

2

%$ + ⋯ + $& #& #"

2

%&

What if the first two largest eigenvalues (in magnitude) are the same, |+! = |+" ? 3) +! and +" opposite signs

①

X, 20

,

Xz do

Xn → did, Uz t attack

kiseueu

kisodd

744.4 take)

ITCHY

delta)

→ no longer

have convergence

no

longer

have

a F

SLIDE 10

Potential pitfalls

1. Starting vector !! may have no component in the dominant eigenvector "" ($" = 0). This is usually unlikely to happen if !! is chosen randomly, and in practice not a problem because rounding will usually introduce such component. 2. Risk of eventual overflow (or underflow): in practice the approximated eigenvector is normalized at each iteration (Normalized Power Iteration) 3. First two largest eigenvalues (in magnitude) may be the same: |)"| = |)#|. In this case, power iteration will give a vector that is a linear combination of the corresponding eigenvectors:

If signs are the same, the method will converge to correct magnitude of the
eigenvalue. If the signs are different, the method will not converge.
This is a “real” problem that cannot be discounted in practice.
O

→

÷

SLIDE 11

Error

!2 = #" 2 $"%" + $$ #$ #"

2

%$ + ⋯ + $& #& #"

2

%& exact "

ien=oH÷

×Y÷=4tf¥YUzi

error

-¥2,2
U,

t Ust

K→

¥ntne¥

⇐ ÷ 1K¥ # I

Hull

SLIDE 12

Convergence and error

Henk

o(⇐f)

"f¥Y÷÷y⇒

constant

Heath H

= /Iff Henk

→ linear convergence

§nH=l¥I%e

SLIDE 13

Example

A- HI!

X, → Ui

①

Xo -→ HX -XoH=O -3

He.lt

Hx -x. 11=0.3

11411

- Ax - all - fY÷µeoH
silos)

"en

. #me."

(

"""

'

¥

""

HEH

17,411194
¥130.3

11911=0.1536,

SLIDE 14

(

Normalized)

power iteration - Xun

DAK

converges

to

multiple

f eigenvector④

corresponding

to

A

→ largest eigenvalue

=

in magnitude

Rayleigh

coefficient

: X = TAI

XT x

what if I want another eigenvalue ?

I 1, I

> I Iz I >

> I In 1

SLIDE 15

Suppose ! is an eigenvector of / such that / ! = # ! What is an eigenvalue of /1"?

I , x → A

Ax

= Xx

÷

Ax

=

x

④= ¥isaneiegenvakeeof

④ - is

the largest

¥

? - smallest

SLIDE 16

Inverse Power Method

Previously we learned that we can use the Power Method to obtain the largest eigenvalue and corresponding eigenvector, by using the update !2>" = / !2 Suppose there is a single smallest eigenvalue of /. With the previous

rdering

|#"| > |#$| ≥ |#?| ≥ ⋯ > |#&|

smallest

eigenvalue

00

:÷÷÷:

¥1

.

powdwi#

will

converge

ftp.//Xr+i=AT

SLIDE 17

Think a about t this q question…

Which code snippet is the best option to compute the smallest eigenvalue of the matrix /?

A) B) D) E) I have no idea! C)

SLIDE 18

Inverse Power Method

Xian = A

"

Xk - /AXm=XI

solve ! → XKH

① factorize

A =PLU

la -IHA)

PL U Xrt,

= Xk - Ochs)

.

②

Ly

= Ptxr - solve for y → Off)

③

U Xan

y §

solve for Kai -Ogi)

SLIDE 19

Cost of computing eigenvalues using inverse power iteration

A Xpet,

= Xk

2/0

") #

OCR)

#

OW)

goin

')

g-

Out)

yolk

)

④

SLIDE 20

Suppose ! is an eigenvector of / such that / ! = #8 ! and also ! is an eigenvector of C such that C ! = #9 !. What is an eigenvalue of What is an eigenvalue of (/ + 8

9 C)1"?

DI

A → Xlix

(AtIBjx=X×

B - I,x

f-

×

Catfish

fAt→

÷X=AttzBX

*:*:¥:

SLIDE 21

Suppose ! is an eigenvector of / such that / ! = #8 ! and also ! is an eigenvector of C such that C ! = #9 !. What is an eigenvalue of What is an eigenvalue of /$ + FC?

X. F - A

CAITB

)x - XX

Xx - B

AItTBx= Xx

ht - ATTB ?

T¥toTx

xx

1x=xi+rI✓°

SLIDE 22

Eigenvalues of a Shifted Inverse Matrix

Suppose the eigenpairs !, # satisfy /! = # !.

A

(x,I) - CA

TI)

" ?

What

is I ?

(A -JIT'x=5x

It

CA
oryx

¥

a - r

±

.

a. i¥¥

.

#¥

SLIDE 23

Eigenvalues of a Shifted Inverse Matrix

then -

me

15--171

(

A

TI) Xa,

= Xk → Inverse

Power iteration

(A

TI )

converge to

eigenvectors

,÷

ltosmake.tk#J

x → T

→ converging

to eigenvalue of

A which is

closer to T

SLIDE 24

(A -JI)Yeti He

define

T

=

random Io / . normalize

X = %×

. y

B

= ( A -TI)

OGE)

. P.hu

la.lu CA -TIN /X=xIE'

for i = I , 2 .

.

Ly

= Ex

solve for y four)

UXnew

= y

→

solve for +new

X

= Knew 111 Xnew H

x : eigenvector corresponding X

which is the rig

A

closer

to T

SLIDE 25

Convergence summary

Method Cost Convergence :&(( / :& Power Method

!!+1 = # !! $ %" &" &!

Inverse Power Method

# !!+1 = !! %) + $ %" &# &#*!

Shifted Inverse Power Method

(# − *+)!!+1= !! %) + $ %" &+ − * &+" − *

+!: largest eigenvalue (in magnitude) +": second largest eigenvalue (in magnitude) +$: smallest eigenvalue (in magnitude) +$*!: second smallest eigenvalue (in magnitude) ++: closest eigenvalue to < ++": second closest eigenvalue to <

✓

get

(

ui)

Minear

D

00h48

④

D D=

O

③

Ochs)

01ns)

①

↳ Xc → eigenvalue

closer too 0¥