2 Forward Elimination Factored Portion Factored Portion - - PowerPoint PPT Presentation

SLIDE 1

Linear Systems Linear Systems

Pivoting in Gaussian Elim.

CSE 541 Roger Crawfis

Limitations of Gaussian Elimation

The naïve implementation of Gaussian

Elimation

The naïve implementation of Gaussian

Elimination is not robust and can suffer from severe round-off errors due to: from severe round off errors due to:

Dividing by zero Dividing by small numbers and adding Dividing by small numbers and adding.

Both can be solved with pivoting

Partial Pivoting

• What if at step i

Aii = 0?

g

• What if at step i, Aii = 0?

Si l Fi

⎡ ⎤ ⎢ ⎥

Factored Portion

• Simple Fix:

If Aii = 0 Fi d A 0 j i

⎢ ⎥ ⎢ ⎥

Factored Portion

ii

A

Row i

Find Aji ≠ 0 j > i Swap Row j with i

⎢ ⎥ ⎢ ⎥ ⎣ ⎦

ji

A

Row j

Example – Partial Pivoting

⎤ ⎡ ⎤ ⎡ ⎥ ⎤ ⎢ ⎡ ⋅

−

25 . 6 25 . 1 10 25 . 1

1 4

x

p g

⎥ ⎦ ⎤ ⎢ ⎣ ⎡ = ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ 75 5 . 12 5 . 12 5 . 5 .

2 1

x Forward Elimination ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ ⋅ − = ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ ⋅ − ⋅

− 5 2 1 5 4

10 25 . 6 75 25 . 6 10 25 . 1 5 . 12 25 . 1 10 25 . 1 x x ⎦ ⎣ ⎦ ⎣ ⎦ ⎣

2

10 25 . 6 75 10 25 . 1 5 . 12 x ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ = ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ 9999 . 4 0001 . 1

5 2 1 digits

x x

SLIDE 2

Example – Partial Pivoting p g

⎥ ⎤ ⎢ ⎡ = ⎥ ⎤ ⎢ ⎡ ⎥ ⎤ ⎢ ⎡ ⋅

−

25 . 6 25 . 1 10 25 . 1

1 4

x

⎤ ⎡ ⎤ ⎡ 0001 1 x

⎥ ⎦ ⎢ ⎣ = ⎥ ⎦ ⎢ ⎣ ⎥ ⎦ ⎢ ⎣ 75 5 . 12 5 . 12

2

x

Forward Elimination

⎤ ⎡ ⎤ ⎡ ⎤ ⎡

4

⎥ ⎦ ⎤ ⎢ ⎣ ⎡ = ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ 9999 . 4 0001 . 1

5 2 1 digits

x x

⎥ ⎦ ⎤ ⎢ ⎣ ⎡ ⋅ − = ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ ⋅ − ⋅

− 5 2 1 5 4

10 25 . 6 75 25 . 6 10 25 . 1 5 . 12 25 . 1 10 25 . 1 x x

⎥ ⎤ ⎢ ⎡ = ⎥ ⎤ ⎢ ⎡ ⎥ ⎤ ⎢ ⎡ ⋅

− 1 4

25 . 6 25 . 1 10 25 . 1 x

Rounded to 3 digits

⎥ ⎦ ⎢ ⎣ ⋅ − = ⎥ ⎦ ⎢ ⎣ ⎥ ⎦ ⎢ ⎣ ⋅ −

5 2 5

10 25 . 6 10 25 . 1 x

⎥ ⎦ ⎤ ⎢ ⎣ ⎡ = ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ 5

1

x ⎥ ⎦ ⎢ ⎣ ⎥ ⎦ ⎢ ⎣ 5

3 2 digits

x

Better Pivoting

Partial Pivoting to mitigate round-off error

g

Partial Pivoting to mitigate round off error

If | | < max | |

ii ji j i

A A Swap row with arg (max | |)

j j i ij j i

i A

> > j

Avoids Small Multipliers

Adds an O(n) search.

Multipliers

Partial Pivoting

⎥ ⎤ ⎢ ⎡ ⎥ ⎤ ⎢ ⎡ ⎥ ⎤ ⎢ ⎡ ⋅

−

25 . 6 25 . 1 10 25 . 1

1 4

x

swap

g

Forward Elimination

⎥ ⎦ ⎢ ⎣ = ⎥ ⎦ ⎢ ⎣ ⎥ ⎦ ⎢ ⎣ 75 5 . 12 5 . 12

2 1

x

⎥ ⎦ ⎤ ⎢ ⎣ ⎡ = ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ ⋅

−

25 . 6 75 25 . 1 10 25 . 1 5 . 12 5 . 12

2 1 4

x x

Forward Elimination

⎥ ⎦ ⎤ ⎢ ⎣ ⎡ ⋅ − = ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ ⋅ −

− − 5 2 1 5

10 75 25 . 6 75 10 5 . 12 25 . 1 5 . 12 5 . 12 x x ⎦ ⎣ ⎦ ⎣ ⎦ ⎣

2

⎤ ⎡ ⎤ ⎡ ⎤ ⎡ 75 5 12 5 12 x

Rounded to 3 digits

⎤ ⎡ ⎤ ⎡ 1 ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ = ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ 25 . 6 75 25 . 1 5 . 12 5 . 12

2 1

x x ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ = ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ 5 1

3 2 1 digits

x x

Pivoting strategies g g

k

Partial Pivoting:

k

Partial Pivoting:

Only row interchange

Complete (Full) Pivoting

k

Complete (Full) Pivoting

Row and Column interchange

k

Threshold Pivoting

Only if prospective pivot is found

to be smaller than a certain threshold

SLIDE 3

Pivoting With Permutations g

Adding permutation matrices in the mix: Adding permutation matrices in the mix:

H i G i Eli i ti ill

b P M P M P M Ax P M P M P M

n n n n n n n n 1 1 2 2 1 1 1 1 2 2 1 1

L L

− − − − − − − −

=

However, in Gaussian Elimination we will

• nly swap rows or columns below the

t i t i t Thi i li l b l current pivot point. This implies a global reordering of the equations will work:

b A M MPb MPAx ′ = ′ =

Pivoting

Again the pivoting is strictly a function of Again, the pivoting is strictly a function of

the matrix A, so once we determine P it is trivial to apply it to many problems bk is trivial to apply it to many problems bk.

For LU factorization we have:

LU PA

LU = PA Ly = Pb

U

Ux = y