Chapter 12: Iterative Methods ES 240: Scientific and Engineering - - PowerPoint PPT Presentation

ES 240: Scientific and Engineering Computation. Iterative Methods and Eigenvalues

Chapter 12: Iterative Methods

Uchechukwu Ofoegbu Temple University

ES 240: Scientific and Engineering Computation. Iterative Methods and Eigenvalues

Gauss Gauss-

• Seidel Method

Seidel Method

The Gauss-Seidel method is the most commonly used iterative method

for solving linear algebraic equations [A]{x}={b}.

The method solves each equation in a system for a particular variable,

and then uses that value in later equations to solve later variables. For a 3x3 system with nonzero elements along the diagonal, for example, the jth iteration values are found from the j-1th iteration using:

x1

j = b1 − a12x2 j−1 − a13x3 j−1

a11 x2

j = b2 − a21x1 j − a23x3 j−1

a22 x3

j = b3 − a31x1 j − a32x2 j

a33

ES 240: Scientific and Engineering Computation. Iterative Methods and Eigenvalues

Gauss Gauss-

• Seidel Method

Seidel Method

The Gauss-Seidel method is the most commonly used iterative method

for solving linear algebraic equations [A]{x}={b}.

The method solves each equation in a system for a particular variable,

and then uses that value in later equations to solve later variables. For a 3x3 system with nonzero elements along the diagonal, for example, the jth iteration values are found from the j-1th iteration using:

3 3 33 2 32 1 31 2 3 23 2 22 1 21 1 3 13 2 12 1 11

c x a x a x a c x a x a x a c x a x a x a = + + = + + = + +

33 2 32 1 31 3 3 22 3 23 1 21 2 2 11 3 13 2 12 1 1

/ ) ( / ) ( / ) ( a x a x a c x a x a x a c x a x a x a c x − − = − − = − − =

ES 240: Scientific and Engineering Computation. Iterative Methods and Eigenvalues

Applying the Gauss Applying the Gauss-

• Seidel Method

Seidel Method

1.

Select an initial guess:

2.

Update each variable

3.

Continue to update the variables

4.

Stop when the tolerance is met

] , , [

3 2

1

x x x

33 2 32 1 31 3 3 22 3 23 1 21 2 2 11 3 13 2 12 1 1

/ ) ( / ) ( / ) ( a x a x a c x a x a x a c x a x a x a c x − − = − − = − − =

s t i t i t i i a

x x x ε ε < − =

−

% 100

) ( ) 1 ( ) ( ,

ES 240: Scientific and Engineering Computation. Iterative Methods and Eigenvalues

Convergence Convergence

1.

Sometimes this method diverges

2.

If the magnitude of each diagonal term is greater than the sum of the magnitudes of the other terms in the same row, the method will certainly converge

3.

Rows could be rearranged to ensure convergence

4.

Sometimes convergence is attained even when the condition is not met

∑

≠ =

≥

n i j i ij ii

a a

, 1

ES 240: Scientific and Engineering Computation. Iterative Methods and Eigenvalues

Example Example

Solve the following using the Gauss Seidel method Now use the GaussSeidel function

14 15 3 2 15 3 2 10 11 8

3 2 1 3 2 1 3 2 1

− = + − = + − = − + x x x x x x x x x

ES 240: Scientific and Engineering Computation. Iterative Methods and Eigenvalues

Jacobi Iteration Jacobi Iteration

• The Jacobi iteration is similar to the Gauss-

Seidel method, except the j-1th information is used to update all variables in the jth iteration:

a) Gauss-Seidel b) Jacobi

ES 240: Scientific and Engineering Computation. Iterative Methods and Eigenvalues

Example Example

Solve the following using the Jacobi Iteration method Compare with GaussSeidel

14 15 3 2 15 3 2 10 11 8

3 2 1 3 2 1 3 2 1

− = + − = + − = − + x x x x x x x x x

ES 240: Scientific and Engineering Computation. Iterative Methods and Eigenvalues

Eigenvalues Eigenvalues and Eigenvectors and Eigenvectors

• Nonhomogeneous system: [A]{x} = {b}
• Homogeneous system: [A]{x} = 0
• Nontrivial solutions exist but are not unique
• The solution of A satisfies the eigenvalue equation:

Ax=λx [A- λ I] x = 0

• λ = eigenvalue of A
• x = eigenvector of A
• Nontrivial solutions exist iff characteristic polynomial,

det(A - λI) = 0

• The size of A determines the number of eigenvalues

ES 240: Scientific and Engineering Computation. Iterative Methods and Eigenvalues

Polynomial method for Polynomial method for eigenvalues eigenvalues

Root of the characteristic polynomial = λ

– Example: x + 2x = 0 3x + 4x = 0

• A =
• Characteristic poly = f(λ) = det

=

• roots of f(λ) = -0.8541, 5.8541

In Matlab: p = ploy(A); e = roots(p)

⎥ ⎦ ⎤ ⎢ ⎣ ⎡ 4 3 3 1 ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ − − λ λ 4 3 3 1

5 5

2

− − λ λ

ES 240: Scientific and Engineering Computation. Iterative Methods and Eigenvalues

Power method for Power method for eigenvalues eigenvalues and eigenvectors and eigenvectors

Iterative method Determines the largest eigenvalue most of the time Method

– Set Ax = λx – Choose initial value for x – Factor out highest element as λ – Continue until desired accuracy

– Example:

• A =

⎥ ⎦ ⎤ ⎢ ⎣ ⎡ 4 3 3 1

ES 240: Scientific and Engineering Computation. Iterative Methods and Eigenvalues

Matlab Matlab eigenvalues eigenvalues and eigenvectors and eigenvectors

• Eig(A)
• [v e] = eig(A)
• How would you find the minimum eigenvalue using the power method?

ES 240: Scientific and Engineering Computation. Iterative Methods and Eigenvalues

Lab Lab

• Problem 12.2, 12.3
• Use the power method to find the minimum eigenvalue of the matrix
• B = [1 2; 2 2];