Linear Systems Linear Systems Transform Ax = b into an equivalent - PowerPoint PPT Presentation

Solving Linear Systems g y Linear Systems Linear Systems � Transform Ax = b into an equivalent but � Transform Ax b into an equivalent but simpler system. Gaussian Elimination � Multiply on the left by a nonsingular � Multiply on the left by a nonsingular matrix: MAx = Mb : CSE 541 − − − − = = = 1 1 1 1 1 1 1 1 x ( ( MA MA ) ) Mb Mb A M A M Mb Mb A b A b Roger Crawfis � Mathematically equivalent but may � Mathematically equivalent, but may change rounding errors Gaussian Elimination Gaussian Elimination � Finding inverses of matrices is expensive � Finding inverses of matrices is expensive � � Consists of 2 steps Consists of 2 steps Forward Elimination of Unknowns. � Inverses are not necessary to solve a 1. ⎡ ⎡ ⎤ ⎤ ⎡ ⎡ ⎤ ⎤ linear system linear system. 25 25 5 5 1 1 25 25 5 5 1 1 ⎢ ⎥ ⎢ ⎥ → − − � Some system are much easier to solve: 64 8 1 0 4 . 8 1 . 56 ⎢ ⎥ ⎢ ⎢ ⎥ ⎥ ⎢ ⎢ ⎥ ⎥ ⎢ ⎥ � Diagonal matrices ⎣ ⎦ 144 12 1 ⎣ ⎦ 0 0 0 . 7 � Triangular matrices Back Substitution B k S b tit ti 2. � Gaussian Elimination transforms the problem into a triangular system p g y

Gaussian Elimination Gaussian Elimination � Systematically eliminate unknowns from the � Uses these elementary row operations � Uses these elementary row operations equations until only a equation with only one unknown is left. � Adding a multiple of one row to another � This is accomplished using three operations p g p � Doesn t change the equality of the equation � Doesn’t change the equality of the equation applied to the linear system of equations: � Hence the solution does not change. � A given equation can be multiplied by a non-zero constant and the result substituted for the original g � The sub diagonal elements are zeroed � The sub-diagonal elements are zeroed- equation, out through elementary row operations � A given equation can be added to a second equation, and the result substituted for the original q g � In a specific order (next slide) I ifi d ( t lid ) equation, � Two equations can be transposed in order. Order of Elimination Gaussian Elimination in 3D + − = 2 x 4 y 2 z 2 ? ? ? ? ? + + − = = 4 4 x x 9 9 y y 3 3 z z 8 8 1 ? ? ? ? − − + = = ? 2 x 3 y 7 z 10 2 4 ? ? � Using the first equation to eliminate x f from the next two equations th t t ti 3 5 6 ? ?

Gaussian Elimination in 3D Gaussian Elimination in 3D + − = + − = 2 x 4 y 2 z 2 2 x 4 y 2 z 2 + + = = + + = = y y z z 4 4 y y z z 4 4 + = = y 5 z 12 4 z 8 � Using the second equation to eliminate � Using the second equation to eliminate y from the third equation f th thi d ti y from the third equation f th thi d ti Solving Triangular Systems g g y Solving Triangular Systems g g y � We now have a triangular system which � We now have a triangular system which � If A is upper triangular, we can solve � If A is upper triangular we can solve is easily solved using a technique called Ax = b by: Backward-Substitution . Backward-Substitution = x b / A + − = n n nn 2 x 4 y 2 z 2 ⎛ ⎛ ⎞ ⎞ n n ∑ + = = − = − ⎜ ⎟ K x b A x / A , i n 1, ,1 y z 4 i i ij j ii ⎝ ⎝ ⎠ ⎠ = + j i 1 = 4 4 z 8 8

Backward Substitution Backward Substitution � From the previous work we have � From the previous work, we have + − = 2 x 4 y 4 2 + − = 2 x 4 y 2 z 2 + + = = y y 2 2 4 4 + = y z 4 = z 2 = = z z 2 2 � We can solve y � And substitute z in the first two equations � And substitute z in the first two equations Backward Substitution Backward Substitution + − = + − = 2 x 4 y 4 2 2 x 8 4 2 = = = = y y 2 2 y y 2 2 = = z 2 z 2 � Substitute to the first equation � We can solve the first equation

Backward Substitution Robustness of Solution � We can measure the precision or � We can measure the precision or = − x 1 accuracy of our solution by calculating = = the residual : y y 2 2 � Calling our computed solution x*… = z 2 � Calculate the distance Ax* is from b � |Ax* – b| � Some matrices are ill-conditioned � A tiny change in the input (the coefficients in A) drastically changes the output (x*) C# Implementation p Computational Complexity p p y //convert to upper triangular form � Forward Elimination // back substitution for (int k=0; k<n-1; k++) { f (i t k 0 k< 1 k++) { b[n-1]=b[n-1] / a[n-1,n-1]; For i = 1 to n-1 { // for each equation try { for (int i=n-2; i>=0; i--) { For j = i+1 to n { // for each target equation below the current for (int i=k+1; i<n; i++) { sum = b[i]; float s = a[i,k] / a[k,k]; [ , ] [ , ]; A A − 2 n n 1 1 = = ∑ n n ji ji − = M , A 0 divisions for(int j=k+1; j<n; j++) for ( int j=i+1; j<n; j++) ( n i ) ji ji A 2 a[i,j] -= a[k,j] * s; = sum -= a[i,j] * x[j]; ii 1 i b[i]=b[i]-b[k] * s; For k = i+1 to n { // for each element beyond pivot column x[i] = sum / a[i,i]; } } } ← − } A A M A − n 1 ∑ 2 jk jk ji ik catch ( DivideByZeroException e) − ≈ 2 3 ( ) n i n { 3 3 } } i = Console.WriteLine(e.Message); 1 } } O(n 3 ) multiply-add’s } }

Computational Complexity p p y � Backward Substitution � Backward Substitution For i = n-1 to 1 { // for each equation For j = n to i+1 { // for each known variable sum = sum – A ij * x j } − } } 2 n 1 ∑ ∑ n n − = ( n i ) multiply-add’s 2 = i 1 O(n 2 )