ECS130 Introduction Monday, January 7, 2019 About Course: ECS130 - PowerPoint PPT Presentation

ECS130 Introduction Monday, January 7, 2019

About Course: ECS130 Scientific Computing Professor: Zhaojun Bai Webpage: http://web.cs.ucdavis.edu/~bai/ECS130/

Today’s Agenda Mathematics Review: Linear Algebra

Vector spaces over R Denote a (abstract) vector by � v . A vector space V = { a collection of vectors � v } which satisfies ◮ All � v, � w ∈ V can be added and multiplied by a ∈ R : w ∈ V , a · � v ∈ V v + � � ◮ The operations ‘+ , · ’ must satisfy the axioms : For arbitrary � u,� v, � w ∈ V , 1. ‘+’ commutativity and associativity: � v + � w = � w + � v , ( � u + � v ) + � w = � u + ( � v + � w ). 2. Distributivity: a ( � v + � w ) = a� v + a� w , ( a + b ) � v = a� v + b� v , for all a, b ∈ R . 3. ‘+’ identity: there exists � 0 ∈ V with � 0 + � v = � v . 4. ‘+’ inverse: for any � v ∈ V , there exists � w ∈ V with � v + � w = 0. 5. ‘ · ’ identity: 1 · � v = � v . 6. ‘ · ’ compatibility: for all a, b ∈ R , ( ab ) · � v = a · ( b · � v ).

Example ◮ Euclidean space: � � R n = � a ≡ ( a 1 , a 2 , . . . , a n ): a i ∈ R . ◮ Addition: ( a 1 , . . . , a n ) + ( b 1 , . . . , b n ) = ( a 1 + b 1 , . . . , a n + b n ) ◮ Multiplication: c · ( a 1 , . . . , a n ) = ( ca 1 , . . . , ca n ) ◮ Illustration in R 2 : a + � � b � 2 � a b � a � a

Example ◮ Polynomials: � � � a i x i : a i ∈ R R [ x ] = p ( x ) = . i ◮ Addition and multiplication in the usual way, e.g. p ( x ) = a 0 + a 1 x + a 2 x 2 , q ( x ) = b 1 x : ◮ Addition: p ( x ) + q ( x ) = a 0 + ( a 1 + b 1 ) x + a 2 x 2 . ◮ Multiplication: 2 p ( x ) = 2 a 0 + 2 a 1 x + 2 a 2 x 2 .

Span of vectors ◮ Start with � v 1 , . . . ,� v n ∈ V , and a i ∈ R , we can define n � � v ≡ a i � v i = a 1 � v 1 + a 2 � v 2 + · · · + a n � v n , i =1 Such a � v is called a linear combination of � v n . v 1 , . . . ,� ◮ For a set of vectors S = { � v i : i ∈ I} , all its linear combinations define � � � span S ≡ a i � v i : � v i ∈ S and a i ∈ R i

Example in R 2 ◮ Observation from (c): adding a new vector does not always increase the span.

Linear dependence ◮ A set S of vectors is linearly dependent if it contains a vector k � for some v i ∈ S \{ � v } and nonzero c i ∈ R . � v = c i � v i , i =1 ◮ Otherwise, S is called linearly independent . ◮ Two other equivalent defs. of linear dependence: v k } ⊂ S \{ � ◮ There exists { � v 1 , . . . ,� 0 } such that k � where c i � = 0 for all i . c i � v i = 0 i =1 ◮ There exists � v ∈ S such that span S = span( S \{ � v } ) .

Dimension and basis ◮ Given a vector space V , it is natural to build a finite set of linearly independent vectors: { � v n } ⊂ V . v 1 , . . . ,� ◮ The max number n of such vectors defines the dimension of V . ◮ Any set S of such vectors is a basis of V , and satisfies span S = V .

Examples ◮ The standard basis for R n is given by the n vectors e i = (0 , . . . , 0 , 1 , 0 , . . . , 0 ) for i = 1 , . . . , n � � �� � � �� � i − 1 n − i Since ◮ � e i is not linear combination of the rest of vectors. c = � n ◮ For all � c ∈ R n , we have � e i . i =1 c i � Hence, the dimension of R n is n . ◮ A basis of polynomials R [ x ] is given by monomials { 1 , x, x 2 , . . . } . The dimension of R [ x ] is ∞ .

More about R n a = ( a 1 , . . . , a n ) ,� ◮ Dot product: for � b = ( b 1 , . . . , b n ) ∈ R n n � a · � b = � a i b i . i =1 ◮ Length of a vector √ � a 2 1 + · · · + a 2 � a � 2 = n = � a · � a. ◮ Angle between two vectors a · � � b θ = arccos . � a � 2 � b � 2 a · � (*Motivating trigonometric in R 3 : � b = � a � 2 � b � 2 cos θ .) a,� a · � ◮ Vectors � b are orthogonal if � b = 0 = cos 90 ◦ .

Linear function ◮ Given two vector spaces V , V ′ , a function L : V → V ′ is linear , if it preserves linearity . ◮ Namely, for all � v 1 ,� v 2 ∈ V and c ∈ R , ◮ L [ � v 1 + � v 2 ] = L [ � v 1 ] + L [ � v 2 ]. ◮ L [ c� v 1 ] = c L [ � v 1 ]. ◮ L is completely defined by its action on a basis of V : � L [ � v ] = c i L [ � v i ] , i v = � where � v i and { � v 2 , . . . } is a basis of V . i c i � v 1 ,�

Examples ◮ Linear map in R n : L : R 2 → R 3 defined by L [( x, y )] = (3 x, 2 x + y, − y ) . ◮ Integration operator: linear map L : R [ x ] → R [ x ] defined by � 1 L [ p ( x )] = p ( x ) dx. 0

Matrix ◮ Write vectors in R m in ‘ column forms ’, e.g.,       v 11 v 12 v 1 n . . .       . . . v 1 = v 2 = v n = �  ,�  , . . . ,�  . . . .    v m 1 v m 2 v mn ◮ Put n columns together we obtain an m × n matrix   v 11 v 12 . . . v 1 n   | | |  v 21 v 22 . . . v 2 n   =   V ≡ � v 1 � v 2 . . . � v n . . . .   . . . .  . . . .   | | | v m 1 v m 2 . . . v mn ◮ The space of all such matrices is denoted by R m × n .

Unified notation: Scalars, Vectors, and Matrices ◮ A scalar c ∈ R is viewed as a 1 × 1 matrix c ∈ R 1 × 1 . v ∈ R n is viewed as an n × 1 matrix ◮ A column vector � v ∈ R n × 1 . �

Matrix vector multiplication ◮ A matrix V ∈ R m × n can be multiplied by a vector � c ∈ R n :     c 1 | | | .  .  v 1 � � v 2 . . . � v n  = c 1 � v 1 + c 2 � v 2 + · · · + c n � v n .   .  | | | c n ◮ Elementwisely, we have       v 11 v 12 . . . v 1 n c 1 c 1 v 11 + c 2 v 12 + · · · + c n v 1 n c 1 v 21 + c 2 v 22 + · · · + c n v 2 n v 21 v 22 . . . v 2 n c 2              =  . . . . . . .       . . . . . . . . . . . .     c 1 v m 1 + c 2 v m 2 + · · · + c n v mn v m 1 v m 2 . . . v mn c n

Using matrix notation ◮ Matrix vector multiplication can be denoted by � = A � x b . ���� ���� ���� R m × n R n R m ◮ M ∈ R m × n multiplied by another matrix in R n × k can be defined as M [ � c 1 , . . . ,� c k ] ≡ [ M� c 1 , . . . , M� c k ] .

Example ◮ Identity matrix   1 0 0 . . .   | | | . ... .   0 1 .  =   I n ≡ � e 1 � e 2 . . . e n �  .  .  ... ...  . . 0  | | | 0 0 1 . . . It holds c ∈ R n . I n � c = � c for all �

Example ◮ Linear map L [( x, y )] = (3 x, 2 x + y, − y ) satisfies     3 0 3 x � x � L [( x, y )] = 2 1 · = 2 x + y .     y − 1 − y 0 ���� � �� � � �� � R 2 R 3 × 2 R 3 ◮ All linear maps L : R n → R m can be expressed as L [ � x ] = A� x, for some matrix A ∈ R m × n .

Matrix transpose ◮ Use A ij to denote the element of A at row i column j . ◮ The transpose of A ∈ R m × n is defined as A T ∈ R n × m ( A T ) ij = A ji . Example:   1 2 � 1 � 3 5 A T = A = 3 4 ⇒ .   2 4 6 5 6 ◮ Basic identities: ( A T ) T = A, ( A + B ) T = A T + B T , ( AB ) T = B T A T .

Examples: Matrix operations with transpose a,� ◮ Dot product of � b ∈ R n :   b 1 n � � � . a · � a T �   . b = a i b i = a 1 . . . a n  = � � b. .  i =1 b n x − � ◮ Residual norms of � r = A� b : x − � x − � x − � b � 2 b ) T ( A� � A� 2 = ( A� b ) x T A T − � x − � b T )( A� = ( � b ) = � b T � b − � x T A T � b T A� x T A T A� x − � b + � x b T A� x T A T � = � � 2 − 2 � (by � b � 2 b T A� x � 2 x + � A� x = � b ) 2 .

Computation aspects ◮ Storage of matrices in memory:    Row-major: 1 2 3 4 5 6 1 2    ⇒ 3 4   5 6  Column-major: 1 3 5 2 4 6 x for A ∈ R m × n and � ◮ Multiplication � x ∈ R n : b = A� Access A row-by-row: Access column-by-column: 1: � 1: � b = 0 b = 0 2: for i = 1 , . . . , m do 2: for j = 1 , . . . , n do for j = 1 , . . . , n do for i = 1 , . . . , m do 3: 3: b i = b i + A ij x j b i = b i + A ij x j 4: 4: end for end for 5: 5: 6: end for 6: end for

Linear systems of equations in matrix form ◮ Example : find ( x, y, z ) satisfying       3 x + 2 y + 5 z = 0 3 2 5 x 0  = − 4 x + 9 y − 3 z = − 7 ⇒ − 4 9 − 3 − 7 y      2 − 3 − 3 z 1 2 x − 3 y − 3 z = 1 . a n ] ∈ R m × n , � ◮ Given A = [ � b ∈ R m , find � x ∈ R n : a 1 , . . . ,� x = � A� b. ◮ Solution exists if � b is in column space of A : � n � � � x ∈ R n } = b ∈ col A ≡ { A� a i : x i ∈ R x : � x i � . i =1 The dimension of col A is defined as the rank of A .