Matrix invertibility Rank-Nullity Theorem: For any n -column matrix A - - PowerPoint PPT Presentation

Matrix invertibility

Rank-Nullity Theorem: For any n-column matrix A, nullity A + rank A = n Corollary: Let A be an R × C matrix. Then A is invertible if and only if |R| = |C| and the columns of A are linearly independent. Proof: Let F be the field. Define f : FC − → FR by f (x) = Ax. Then A is an invertible matrix if and only if f is an invertible function. The function f is invertible iff dim Ker f = 0 and dim FC = dim FR iff nullity A = 0 and |C| = |R|. nullity A = 0 iff dim Null A = 0 iff Null A = {0} iff the only vector x such that Ax = 0 is x = 0 iff the columns of A are linearly independent. QED

Matrix invertibility examples

 1 2 3 4 5 6

• is not square so cannot be invertible.

 1 2 3 4

• is square and its columns are linearly independent so it is invertible.

2 4 1 1 2 2 1 3 3 1 4 3 5 is square but columns not linearly independent so it is not invertible.

Transpose of invertible matrix is invertible

Theorem: The transpose of an invertible matrix is invertible. A = 2 4 v1 · · ·

vn

3 5 = 2 6 4

a1

. . .

an

3 7 5 AT = 2 4 a1 · · ·

an

3 5 Proof: Suppose A is invertible. Then A is square and its columns are linearly independent. Let n be the number of columns. Then rank A = n. Because A is square, it has n rows. By Rank Theorem, rows are linearly independent. Columns of transpose AT are rows of A, so columns of AT are linearly independent. Since AT is square and columns are linearly independent, AT is invertible. QED

More matrix invertibility

Earlier we proved: If A has an inverse A−1 then AA−1 is identity matrix Converse: If BA is identity matrix then A and B are inverses? Not always true. Theorem: Suppose A and B are square matrices such that BA is an identity matrix 1. Then A and B are inverses of each other. Proof: To show that A is invertible, need to show its columns are linearly independent. Let u be any vector such that Au = 0. Then B(Au) = B0 = 0. On the other hand, (BA)u = 1u = u, so u = 0. This shows A has an inverse A−1. Now must show B = A−1. We know AA−1 = 1. BA = 1 (BA)A−1 = 1A−1 by multiplying on the right by A−1 (BA)A−1 = A−1 B(AA−1) = A−1 by associativity of matrix-matrix mult B 1 = A−1 B = A−1 QED

Representations of vector spaces

Two important ways to represent a vector space: As the solution set of homogeneous linear system

a1 · x = 0, . . . , am · x = 0

Equivalently, Null 2 6 4

a1

. . .

am

3 7 5 As Span {b1, . . . , bk} Equivalently, Col 2 6 6 6 6 4

b1

· · ·

bk

3 7 7 7 7 5

Conversions between the two representations

{[x, y, z] : [4, −1, 1] · [x, y, z] = 0, [0, 1, 1] · [x, y, z] = 0} Span {[1, 2, −2]} Span {[4, −1, 1], [0, 1, 1]} {[x, y, z] : [1, 2, −2] · [x, y, z] = 0}

Conversions for affine spaces?

I From representation as solution set of linear system to representation as affine hull I From representation as affine hull to representation as solution set of linear system

Conversions for affine spaces?

From representation as solution set of linear system to representation as affine hull

I input: linear system Ax = b I output: vectors whose affine hull is the solution set of the linear system.

 1 2 1 −1 2 2 2 4 x y z 3 5 =  1 2

• 1

Let u be one solution to the linear system.

u = [−0.5, 0.75, 0]

2 Consider the corresponding homogeneous system Ax = 0.  1 2 1 −1 2 2 2 4 x y z 3 5 =  0

• Its solution set, the null space of A, is a vector space V.

3 Let b1, . . . , bk be generators for V.

b1 = [2, −3, 4]

4 Then the solution set of the original linear system is the affine hull of u, b1 + u, b2 + u, . . . , bk + u. [−0.5, −.75, 0] and [−0.5, −.75, 0] + [2, −3, 4]

From representation as solution set to representation as affine hull

One solution to equation  1 2 1 −1 2 2 2 4 x y z 3 5 =  1 2

• is u = [−0.5, 0.75, 0]

Null space of  1 2 1 −1 2 2

• is Span {b1}:

b1

Solution set of equation is u + Span {b1}, i.e. the affine hull of u and u + b1

u+b1 u

Representations of vector spaces

Two important ways to represent a vector space: As the solution set of homogeneous linear system

a1 · x = 0, . . . , am · x = 0

Equivalently, Null 2 6 4

a1

. . .

am

3 7 5 As Span {b1, . . . , bk} Equivalently, Col 2 6 6 6 6 4

b1

· · ·

bk

3 7 7 7 7 5

Representations of vector spaces

Two important ways to represent a vector space: As the solution set of homogeneous linear system

a1 · x = 0, . . . , am · x = 0

Equivalently, Null 2 6 4

a1

. . .

am

3 7 5 As Span {b1, . . . , bk} Equivalently, Col 2 6 6 6 6 4

b1

· · ·

bk

3 7 7 7 7 5 How to transform between these two representations? Problem 1 (From left to right):

I input: homogeneous linear system a1 · x = 0, . . . , am · x = 0, I output: basis b1, . . . , bk for solution set

Problem 2 (From right to left):

I input: independent vectors b1, . . . , bk, I output: homogeneous linear system a1 · x = 0, . . . , am · x = 0 whose solution set equals

Reformulating Problem 1

I input: homogeneous linear system a1 · x = 0, . . . , am · x = 0, I output: basis b1, . . . , bk for solution set

Let’s express this in the language of matrices:

I input: matrix A =

2 6 4

a1

. . .

am

3 7 5

I output: matrix B =

2 4 b1 · · ·

bk

3 5 such that Col B = Null A Can require the rows of the input matrix A to be linearly independent. (Discarding a superfluous row does not change the null space of A.)

Reformulating Problem 1

I input: homogeneous linear system a1 · x = 0, . . . , am · x = 0, I output: basis b1, . . . , bk for solution set

Let’s express this in the language of matrices:

I input: matrix A =

2 6 4

a1

. . .

am

3 7 5

I output: matrix B =

2 4 b1 · · ·

bk

3 5 with independent columns such that Col B = Null A Can require the rows of the input matrix A to be linearly independent. (Discarding a superfluous row does not change the null space of A.)

Reformulating Problem 1

I input: homogeneous linear system a1 · x = 0, . . . , am · x = 0, I output: basis b1, . . . , bk for solution set

Let’s express this in the language of matrices:

I input: matrix A =

2 6 4

a1

. . .

am

3 7 5 with independent rows

I output: matrix B =

2 4 b1 · · ·

bk

3 5 with independent columns such that Col B = Null A Can require the rows of the input matrix A to be linearly independent. (Discarding a superfluous row does not change the null space of A.)

Reformulating the reformulation of Problem 1

I input: matrix A with independent rows I output: matrix B with independent columns such that Col B = Null A

By Rank-Nullity Theorem, rank A + nullity A = n Because rows of A are linearly independent, rank A = m, so m + nullity A = n Requiring Col B = Null A is the same as requiring (i) Col B is a subspace of Null A (ii) dim Col B = nullity A

Reformulating the reformulation of Problem 1

I input: matrix A with independent rows I output: matrix B with independent columns such that Col B = Null A

By Rank-Nullity Theorem, rank A + nullity A = n Because rows of A are linearly independent, rank A = m, so m + nullity A = n Requiring Col B = Null A is the same as requiring (i) Col B is a subspace of Null A = ⇒ same as requiring AB = 2 6 4 · · · . . . . . . · · · 3 7 5 (ii) dim Col B = nullity A

Reformulating the reformulation of Problem 1

I input: matrix A with independent rows I output: matrix B with independent columns such that Col B = Null A

By Rank-Nullity Theorem, rank A + nullity A = n Because rows of A are linearly independent, rank A = m, so m + nullity A = n Requiring Col B = Null A is the same as requiring (i) Col B is a subspace of Null A = ⇒ same as requiring AB = 2 6 4 · · · . . . . . . · · · 3 7 5 (ii) dim Col B = nullity A = ⇒ same as requiring number of columns of B = nullity A same as requiring number of columns of B = n − m

Reformulating the reformulation of Problem 1

I input: matrix A with independent rows I output: matrix B with independent columns such that Col B = Null A

By Rank-Nullity Theorem, rank A + nullity A = n Because rows of A are linearly independent, rank A = m, so m + nullity A = n Requiring Col B = Null A is the same as requiring (i) Col B is a subspace of Null A = ⇒ same as requiring AB = 2 6 4 · · · . . . . . . · · · 3 7 5 (ii) dim Col B = nullity A = ⇒ same as requiring number of columns of B = nullity A same as requiring number of columns of B = n − m

I input: m × n matrix A with independent rows I output: matrix B with n − m independent columns such that AB =

h 0 i

Hypothesize a procedure for reformulation of Problem 1

Problem 1:

I input: m × n matrix A with independent rows I output: matrix B with n − m independent columns such that AB =

h 0 i Define procedure null space basis(M) with this spec:

I input: r × n matrix M with independent rows I output: matrix C with n − r independent columns such that MC =

h 0 i

Reformulating Problem 2

I input: independent vectors b1, . . . , bk, I output: homogeneous linear system a1 · x = 0, . . . , am · x = 0 whose solution set

equals Span {b1, . . . , bk} Let’s express this in the language of matrices:

I input: n × k matrix B with independent columns I output: matrix A with independent rows such that Null A = Col B

As before, Rank-Nullity Theorem implies number of rows of A + nullity A = number of columns of A As before, requiring Null A = Col B is the same as requiring (i) AB = h 0 i (ii) number of rows of A = n − k

I input: n × k matrix B with independent rows I output: matrix A with n − k independent rows such that AB =

h 0 i

Solving Problem 2 with the procedure for Problem 1

Problem 1:

I input: m × n matrix A with independent rows I output: matrix B with n − m independent columns such that AB =

h 0 i Define procedure null space basis(M)

I input: r × n matrix M with independent rows I output: matrix C with n − r independent columns such that MC =

h 0 i Problem 2:

I input: n × k matrix B with independent rows I output: matrix A with n − k independent rows such that AB =

h 0 i To solve Problem 2, call null space basis(BT). Returns matrix AT with independent columns such that BTAT = h 0 i Since BT is k × n matrix, AT has n − k columns. Therefore AB = h 0 i and A has n − k independent rows. Therefore A is solution to Problem