Chapter 6 Direct Methods for Solving Linear Systems Per-Olof - PowerPoint PPT Presentation

Chapter 6 Direct Methods for Solving Linear Systems Per-Olof Persson persson@berkeley.edu Department of Mathematics University of California, Berkeley Math 128A Numerical Analysis

Direct Methods for Linear Systems Consider solving a linear system of the form: E 1 : a 11 x 1 + a 12 x 2 + · · · + a 1 n x n = b 1 , E 2 : a 21 x 1 + a 22 x 2 + · · · + a 2 n x n = b 2 , . . . E n : a n 1 x 1 + a n 2 x 2 + · · · + a nn x n = b n , for x 1 , . . . , x n . Direct methods give an answer in a fixed number of steps, subject only to round-off errors. We use three row operations to simplify the linear system: 1 Multiply Eq. E i by λ � = 0 : ( λE i ) → ( E i ) 2 Multiply Eq. E j by λ and add to Eq. E i : ( E i + λE j ) → ( E i ) 3 Exchange Eq. E i and Eq. E j : ( E i ) ↔ ( E j )

Gaussian Elimination Gaussian Elimination with Backward Substitution Reduce a linear system to triangular form by introducing zeros using the row operations ( E i + λE j ) → ( E i ) Solve the triangular form using backward-substitution Row Exchanges If a pivot element on the diagonal is zero, the reduction to triangular form fails Find a nonzero element below the diagonal and exchange the two rows Definition An n × m matrix is a rectangular array of elements with n rows and m columns in which both value and position of an element is important

Operation Counts Count the number of arithmetic operations performed Use the formulas m m j = m ( m + 1) j 2 = m ( m + 1)(2 m + 1) � � , 2 6 j =1 j =1 Reduction to Triangular Form Multiplications/divisions: n − 1 ( n − i )( n − i + 2) = · · · = 2 n 3 + 3 n 2 − 5 n � 6 i =1 Additions/subtractions: n − 1 ( n − i )( n − i + 1) = · · · = n 3 + 3 n 2 − 5 n � 6 i =1

Operation Counts Backward Substitution Multiplications/divisions: n − 1 (( n − i ) + 1) = n 2 + n � 1 + 2 i =1 Additions/subtractions: n − 1 (( n − i − 1) + 1) = n 2 − n � 2 i =1

Operation Counts Gaussian Elimination Total Operation Count Multiplications/divisions: n 3 3 + n 2 − n 3 Additions/subtractions: n 3 3 + n 2 2 − 5 n 6

Partial Pivoting In Gaussian elimination, if a pivot element a ( k ) kk is small compared to an element a ( k ) jk below, the multiplier a ( k ) jk m jk = a ( k ) kk will be large, resulting in round-off errors Partial pivoting finds the smallest p ≥ k such that | a ( k ) k ≤ i ≤ n | a ( k ) pk | = max ik | and interchanges the rows ( E k ) ↔ ( E p )

Scaled Partial Pivoting If there are large variations in magnitude of the elements within a row, scaled partial pivoting can be used Define a scale factor s i for each row s i = max 1 ≤ j ≤ n | a ij | At step i , find p such that | a pi | | a ki | = max s p s k i ≤ k ≤ n and interchange the rows ( E i ) ↔ ( E p )

Linear Algebra Definition Two matrices A and B are equal if they have the same number of rows and columns n × m and if a ij = b ij . Definition If A and B are n × m matrices, the sum A + B is the n × m matrix with entries a ij + b ij . Definition If A is n × m and λ a real number, the scalar multiplication λA is the n × m matrix with entries λa ij .

Properties Theorem Let A, B, C be n × m matrices, λ, µ real numbers. (a) A + B = B + A (b) ( A + B ) + C = A + ( B + C ) (c) A + 0 = 0 + A = A (d) A + ( − A ) = − A + A = 0 (e) λ ( A + B ) = λA + λB (f) ( λ + µ ) A = λA + µA (g) λ ( µA ) = ( λµ ) A (h) 1 A = A

Matrix Multiplication Definition Let A be n × m and B be m × p . The matrix product C = AB is the n × p matrix with entries m � c ij = a ik b kj = a i 1 b 1 j + a i 2 b 2 j + · · · + a im b mj k =1

Special Matrices Definition A square matrix has m = n A diagonal matrix D = [ d ij ] is square with d ij = 0 when i � = j The identity matrix of order n , I n = [ δ ij ] , is diagonal with � 1 , if i = j, δ ij = 0 , if i � = j. Definition An upper-triangular n × n matrix U = [ u ij ] has u ij = 0 , if i = j + 1 , . . . , n. A lower-triangular n × n matrix L = [ l ij ] has l ij = 0 , if i = 1 , . . . , j − 1 .

Properties Theorem Let A be n × m , B be m × k , C be k × p , D be m × k , and λ a real number. (a) A ( BC ) = ( AB ) C (b) A ( B + D ) = AB + AD (c) I m B = B and BI k = B (d) λ ( AB ) = ( λA ) B = A ( λB )

Matrix Inversion Definition An n × n matrix A is nonsingular or invertible if n × n A − 1 exists with AA − 1 = A − 1 A = I The matrix A − 1 is called the inverse of A A matrix without an inverse is called singular or noninvertible Theorem For any nonsingular n × n matrix A , (a) A − 1 is unique (b) A − 1 is nonsingular and ( A − 1 ) − 1 = A (c) If B is nonsingular n × n , then ( AB ) − 1 = B − 1 A − 1

Matrix Transpose Definition The transpose of n × m A = [ a ij ] is m × n A t = [ a ji ] A square matrix A is called symmetric if A = A t Theorem (a) ( A t ) t = A (b) ( A + B ) t = A t + B t (c) ( AB ) t = B t A t (d) if A − 1 exists, then ( A − 1 ) t = ( A t ) − 1

Determinants Definition (a) If A = [ a ] is a 1 × 1 matrix, then det A = a (b) If A is n × n , the minor M ij is the determinant of the ( n − 1) × ( n − 1) submatrix deleting row i and column j of A (c) The cofactor A ij associated with M ij is A ij = ( − 1) i + j M ij (d) The determinant of n × n matrix A for n > 1 is n n � � ( − 1) i + j a ij M ij det A = a ij A ij = j =1 j =1 or n n � � ( − 1) i + j a ij M ij det A = a ij A ij = i =1 i =1

Properties Theorem (a) If any row or column of A has all zeros, then det A = 0 (b) If A has two rows or two columns equal, then det A = 0 (c) If ˜ A comes from ( E i ) ↔ ( E j ) on A , then det ˜ A = − det A (d) If ˜ A comes from ( λE i ) ↔ ( E i ) on A , then det ˜ A = λ det A (e) If ˜ A comes from ( E i + λE j ) ↔ ( E i ) on A , with i � = j , then det ˜ A = det A (f) If B is also n × n , then det AB = det A det B (g) det A t = det A (h) When A − 1 exists, det A − 1 = (det A ) − 1 (i) If A is upper/lower triangular or diagonal, then det A = � n i =1 a ii

Linear Systems and Determinants Theorem The following statements are equivalent for any n × n matrix A : (a) The equation A x = 0 has the unique solution x = 0 (b) The system A x = b has a unique solution for any b (c) The matrix A is nonsingular; that is, A − 1 exists (d) det A � = 0 (e) Gaussian elimination with row interchanges can be performed on the system A x = b for any b

LU Factorization The k th Gaussian transformation matrix is defined by  1 0 · · · · · · 0  . ... ... .  0 .    . . ... ... ...   . . . .     . . ... ...  . .  . 0 .   M ( k ) =   . . . ... ... . . .   . . − m k +1 ,k .     . . . . ... . . . .   . . . 0 .     . . . . ... ... . . . .   . . . . 0   0 · · · 0 − m n,k 0 · · · 0 1

LU Factorization Gaussian elimination can be written as a (1) a (1) a (1)   · · · 11 12 1 n . ... a (2) .   0 . A ( n ) = M ( n − 1) · · · M (1) A =  22  .  ... ...  . a ( n − 1)   . n − 1 ,n   a ( n ) 0 · · · 0 nn

LU Factorization Reversing the elimination steps gives the inverses: 1 0 · · · · · · 0   . ... ... . 0 .     . . ... ... ...   . . . .     . . ... ...   . . . 0 . L ( k ) = [ M ( k ) ] − 1 =     . . . ... ...  . . .  . . m k +1 ,k .     . . . . ... . . . .   . . . 0 .     . . . . ... ... . . . .   . . . . 0   0 · · · 0 m n,k 0 · · · 0 1 and we have LU = L (1) · · · L ( n − 1) · · · M ( n − 1) · · · M (1) A = [ M (1) ] − 1 · · · [ M ( n − 1) ] − 1 · · · M ( n − 1) · · · M (1) A = A

LU Factorization Theorem If Gaussian elimination can be performed on the linear system A x = b without row interchanges, A can be factored into the product of lower-triangular L and upper-triangular U as A = LU , where m ji = a ( i ) ji /a ( i ) ii : a (1) a (1) a (1)     · · · 1 0 · · · 0 11 12 1 n . . ... ... a (2) . .     0 . m 21 1 . 22     U = , L = . .  ... ...   ... ...  . a ( n − 1) .  .   . 0  n − 1 ,n     a ( n ) m n 1 · · · m n,n − 1 1 0 · · · 0 nn

Permutation Matrices Suppose k 1 , . . . , k n is a permutation of 1 , . . . , n . The permutation matrix P = ( p ij ) is defined by � 1 , if j = k i , p ij = 0 , otherwise. (i) PA permutes the rows of A :   a k 1 1 · · · a k 1 n . . ... . . PA =   . .   a k n 1 · · · a k n n (ii) P − 1 exists and P − 1 = P t Gaussian elimination with row interchanges then becomes: A = P − 1 LU = ( P t L ) U

Diagonally Dominant Matrices Definition The n × n matrix A is said to be strictly diagonally dominant when � | a ii | > | a ij | j � = i Theorem A strictly diagonally dominant matrix A is nonsingular, Gaussian elimination can be performed on A x = b without row interchanges, and the computations will be stable.