CS475/CS675 Lecture 1: May 3, 2016 Basic Theory of Linear Algebra - - PowerPoint PPT Presentation

▶

Dec 27, 2023 277 likes •448 views

CS475/CS675 Lecture 1: May 3, 2016 Basic Theory of Linear Algebra Reading: [TB] chapt 1 (p. 110), chapt 20 (p. 147152) CS475/CS675 (c) 2016 P. Poupart & J. Wan 1 Range Definition: the range of A is defined as

SLIDE 1

CS475/CS675 Lecture 1: May 3, 2016

Basic Theory of Linear Algebra Reading: [TB] chapt 1 (p. 1‐10), chapt 20 (p. 147‐152)

SLIDE 2

Range

Definition: the range of A is defined as

– : for some

Theorem:

– space spanned by the columns of … ⋯ ∑

Thus

is also called the column space of

SLIDE 3

Rank

Definition:

– Column rank = dimension of column space – Row rank = dimension of row space

Theorem: column rank = row rank
Thus we simply call it the rank of ,

SLIDE 4

Full Rank

Definition: An

matrix is of full rank if

Thus, if

, , then a full rank matrix has n independent column vectors

Definition: A nonsingular (invertible) matrix is a

square matrix of full rank

Definition: The null space of ,

is defined as

SLIDE 5

Matrix inverse

Matrix inverse
Interesting identity:

Proof:

SLIDE 6

Updating Matrix Inverses

Sherman‐Morrison‐Woodbury formula

A UV

where

(

n ).

Thus a rank

correction to results in a rank correction of the inverse k

SLIDE 7

Example

SLIDE 8

How to compute ?

In numerical linear algebra, NEVER compute

and

then

We always consider as the solution of the eqn:
We compute by solving the equation by Gaussian

Elimination

SLIDE 9

Gaussian Elimination (GE)

Big picture of GE:

SLIDE 10

GE algorithm

For 1,2, … , 1 For 1, … , / ( for 1, … , end end End At the end, is solved by back substitution Update Update RHS

SLIDE 11

LU Factorization

Theorem:

where

lower , unit diagonal upper

mult

1 1

. . .

SLIDE 12

Solve

, then we have

– Solve by forward solve – Solve by back solve

SLIDE 13

Forward solve algorithm

For 1,2, … , For 1,2, … , 1 end end

CS475/CS675 (c) 2016 P. Poupart & J. Wan

SLIDE 14

Complexity

1 flop = + / ‐ / x /
Consider forward solve

– For each , the ‐loop performs 2 1 flops – Total flops ∑ 2 2

2 ∑

∑ 2

SLIDE 15

Complexity (continued)

(exercise)
For large , factorization is more expensive than

CS475/CS675 Lecture 1: May 3, 2016

Basic Theory of Linear Algebra Reading: [TB] chapt 1 (p. 1‐10), chapt 20 (p. 147‐152)

Range

– : for some

– space spanned by the columns of … ⋯ ∑

is also called the column space of

Rank

– Column rank = dimension of column space – Row rank = dimension of row space

Full Rank

matrix is of full rank if

, , then a full rank matrix has n independent column vectors

square matrix of full rank

is defined as

Matrix inverse

Proof:

Updating Matrix Inverses

A UV

where

n ).

correction to results in a rank correction of the inverse k

Example

How to compute ?

then

Elimination

Gaussian Elimination (GE)

GE algorithm

For 1,2, … , 1 For 1, … , / ( for 1, … , end end End At the end, is solved by back substitution Update Update RHS

LU Factorization

where

lower , unit diagonal upper

. . .

Solve

, then we have

– Solve by forward solve – Solve by back solve

Forward solve algorithm

For 1,2, … , For 1,2, … , 1 end end

Complexity

– For each , the ‐loop performs 2 1 flops – Total flops ∑ 2 2

∑ 2

Complexity (continued)

forward or backward solves