Linear Algebra in A Nutshell Gilbert Strang Computational Science - PowerPoint PPT Presentation

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis Linear Algebra in A Nutshell Gilbert Strang Computational Science and Engineering Wellesley-Cambridge Press. 2007.

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis Outline Matrix Singularity 1 Matrix Multiplication by Columns or Rows 2 Rank and nullspace Column space and solutions to linear equations The Four Fundamental Subspaces 3 Dimension and Basis 4

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis Outline Matrix Singularity 1 Matrix Multiplication by Columns or Rows 2 Rank and nullspace Column space and solutions to linear equations The Four Fundamental Subspaces 3 Dimension and Basis 4

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis Invertibility of an n -by- n matrix A is invertible A is not invertible The columns are independent The columns are dependent The rows are independent The rows are dependent The determinant is not zero The determinant is zero Ax = 0 has one solution x = 0 Ax = 0 has infinitely many solutions Ax = b has one solution A − 1 b Ax = b has no solution or infinitely many A has n (nonzero) pivots A has r < n pivots A has full rank A has rank r < n

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis Invertibility of an n -by- n matrix (cont.) The reduced row echelon form R has at least one zero row is R = I The column space is all of R n The column space has dimension r < n The row space is all of R n The row space has dimension r < n All eigenvalues are nonzero Zero is an eigenvalue of A A T A is symmetric positive A T A is only semidefinite definite A has n (positive) singular A has r < n nonzero (positive) values singular values

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis Outline Matrix Singularity 1 Matrix Multiplication by Columns or Rows 2 Rank and nullspace Column space and solutions to linear equations The Four Fundamental Subspaces 3 Dimension and Basis 4

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis Think of Ax a column at time Instead of thinking of Ax inner products, think of Ax a linear combination of columns of A : � 1 � � C � 1 � 2 � � � 2 = C + D 3 6 D 3 6

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis Think of Ax a column at time Instead of thinking of Ax inner products, think of Ax a linear combination of columns of A : � 1 � � C � 1 � 2 � � � 2 = C + D 3 6 D 3 6 In particular, � 1 � � 1 � 1 � � 0 � � 2 2 = first column = last column 3 6 0 3 6 1

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis In general matrix-vector multiplication: y = Ax

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis In general matrix-vector multiplication: y = Ax column version y = zeros(m,1); for j=1:n y = y + x(j)*A(:,j); endfor

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis In general matrix-vector multiplication: y = Ax column version y = zeros(m,1); for j=1:n y = y + x(j)*A(:,j); endfor matrix-matrix multiplication: C = AB

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis In general matrix-vector multiplication: y = Ax column version y = zeros(m,1); for j=1:n y = y + x(j)*A(:,j); endfor matrix-matrix multiplication: C = AB column version (Fortran, step 1) C(:,j) = A*B(:,j)

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis Row version vector-matrix multiplication: v T = u T A

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis Row version vector-matrix multiplication: v T = u T A row version v = zeros(1,n); for i=1:m v = v + u(i)*A(i,:); endfor

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis Row version vector-matrix multiplication: v T = u T A row version v = zeros(1,n); for i=1:m v = v + u(i)*A(i,:); endfor matrix-matrix multiplication: C = AB

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis Row version vector-matrix multiplication: v T = u T A row version v = zeros(1,n); for i=1:m v = v + u(i)*A(i,:); endfor matrix-matrix multiplication: C = AB row version (C) C(i,:) = A(i,:)*B

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis Rank and nullspace Suppose A is an m -by- n matrix, Ax = 0 has at least one (trivial) solution, namely x = 0. There are other (nontrivial) solutions in case n > m . Even if m = n , there might be nonzero solutions to Ax = 0 when A is not invertible. It is the number r of independent rows or columns that counts.

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis Rank and nullspace (cont.) Rank The number r of independent rows or columns is the rank of A ( r ≤ m and r ≤ n , that is, r ≤ min ( m , n ) ).

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis Rank and nullspace (cont.) Rank The number r of independent rows or columns is the rank of A ( r ≤ m and r ≤ n , that is, r ≤ min ( m , n ) ). Null space The null space of A is the set of all solutions x to Ax = 0. x 1 ( column 1 ) + · · · + x n ( column n ) = 0 x in nullspace

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis Rank and nullspace (cont.) Rank The number r of independent rows or columns is the rank of A ( r ≤ m and r ≤ n , that is, r ≤ min ( m , n ) ). Null space The null space of A is the set of all solutions x to Ax = 0. x 1 ( column 1 ) + · · · + x n ( column n ) = 0 x in nullspace This nullspace N ( A ) contains only x = 0 when the columns of A are independent . In that case A is of full column rank r = n .

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis Rank and nullspace (cont.) � 1 � 2 Example. The nullspace of is a line. 3 6 Question Find the line.

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis Rank and nullspace (cont.) � 1 � 2 Example. The nullspace of is a line. 3 6 Question Find the line. We often require that A is of full column rank. In that case, A T A , n -by- n , is invertible, and symmetric and positive definite.

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis Column (range) space Column (range) space The column (range) space contains all combinations of the columns. � 1 � 2 Example. The column space of is always through 3 6 � 1 � . 3

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis Column (range) space (cont.) In other words, the column space C ( A ) contains all possible products Ax , thus also called the range space R ( A ) . For an m -by- n matrix, the column space is in m -dimensional space. The word “space” indicates: Any combination of vectors in the space stays in the space. The zero combination is allowed, so x = 0 is in every space.

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis Solution to linear equations A solution to Ax = b calls for a linear combination of the columns that equals b . Thus, if b is in R ( A ) , there is a solution to Ax = b , otherwise, Ax = b has no solution.

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis Solution to linear equations A solution to Ax = b calls for a linear combination of the columns that equals b . Thus, if b is in R ( A ) , there is a solution to Ax = b , otherwise, Ax = b has no solution. How do we write down all solutions, when b ∈ R ( A ) ?

Matrix Singularity Matrix Multiplication by Columns or Rows The Four Fundamental Subspaces Dimension and Basis Solution to linear equations A solution to Ax = b calls for a linear combination of the columns that equals b . Thus, if b is in R ( A ) , there is a solution to Ax = b , otherwise, Ax = b has no solution. How do we write down all solutions, when b ∈ R ( A ) ? Suppose x p is a particular solution to Ax = b . Any vector x n in the nullspace solves Ax = 0. The complete solution to Ax = b has the form: x = ( one x p ) + ( all x n ) .