CS475/CS675 Lecture 1: May 3, 2016 Basic Theory of Linear Algebra - PowerPoint PPT Presentation

CS475/CS675 Lecture 1: May 3, 2016 Basic Theory of Linear Algebra Reading: [TB] chapt 1 (p. 1‐10), chapt 20 (p. 147‐152) CS475/CS675 (c) 2016 P. Poupart & J. Wan 1

Range • Definition: the range of A is defined as – ����� � � �: � � �� for some � • Theorem: – ����� � � space spanned by the columns of � � � � � � … � � � � �� � � � � � � � � � � � ⋯ � � � � � � ∑ � � � � � ��� • Thus is also called the column space of CS475/CS675 (c) 2016 P. Poupart & J. Wan 2

Rank • Definition: – Column rank = dimension of column space – Row rank = dimension of row space • Theorem: column rank = row rank • Thus we simply call it the rank of , CS475/CS675 (c) 2016 P. Poupart & J. Wan 3

Full Rank • Definition: An matrix is of full rank if • Thus, if , , then a full rank matrix has n independent column vectors • Definition: A nonsingular (invertible) matrix is a square matrix of full rank • Definition: The null space of , is defined as CS475/CS675 (c) 2016 P. Poupart & J. Wan 4

Matrix inverse • Matrix inverse �� �� � � �� � �� � �� � � � � �� � � �� • Interesting identity: � �� � � �� � � �� � � � � �� Proof: � � �� � � �� � � � � �� � �� �� � � � � � �� � �� �� � �� �� � � � � CS475/CS675 (c) 2016 P. Poupart & J. Wan 5

Updating Matrix Inverses • Sherman‐Morrison‐Woodbury formula A � UV � �� � � �� � � �� � � � � � � �� � �� � � � �� ��� ( where n ). k • Thus a rank correction to results in a rank correction of the inverse CS475/CS675 (c) 2016 P. Poupart & J. Wan 6

Example CS475/CS675 (c) 2016 P. Poupart & J. Wan 7

How to compute ? �� and • In numerical linear algebra, NEVER compute �� . then • We always consider as the solution of the eqn: • We compute by solving the equation by Gaussian Elimination CS475/CS675 (c) 2016 P. Poupart & J. Wan 8

Gaussian Elimination (GE) • Big picture of GE: CS475/CS675 (c) 2016 P. Poupart & J. Wan 9

GE algorithm For � � 1,2, … , � � 1 For � � � � 1, … , � ���� � � �� /� �� ( � �� � ����� for � � � � 1, … , � Update ������ � �� � � �� � ���� � � �� end � � � � � � ���� � � � Update RHS end End At the end, � ��� � � � ��� is solved by back substitution CS475/CS675 (c) 2016 P. Poupart & J. Wan 10

LU Factorization • Theorem: where lower , unit diagonal . 1 0 . . mult 1 ��� upper CS475/CS675 (c) 2016 P. Poupart & J. Wan 11

Solve • • Let , then we have – Solve by forward solve – Solve by back solve CS475/CS675 (c) 2016 P. Poupart & J. Wan 12

Forward solve algorithm For � � 1,2, … , � � � � � � For � � 1,2, … , � � 1 ��� � � � � � � � �� � � � � � � � � � � � �� � � ��� end end CS475/CS675 (c) 2016 P. Poupart & J. Wan 13

Complexity • 1 flop = + / ‐ / x / • Consider forward solve – For each � , the � ‐loop performs 2�� � 1� flops � – Total flops � ∑ �2� � 2� ��� � � � 2 ∑ � � ∑ 2 ��� ��� � ��� � 2 � 2� � � � � � � � ��� � � CS475/CS675 (c) 2016 P. Poupart & J. Wan 14

Complexity (continued) � • (exercise) � � � • � • For large , factorization is more expensive than forward or backward solves CS475/CS675 (c) 2016 P. Poupart & J. Wan 15