Linear algebra explained in four pages Excerpt from the N O BULLSHIT - PDF document

Linear algebra explained in four pages Excerpt from the N O BULLSHIT GUIDE TO LINEAR ALGEBRA by Ivan Savov Abstract —This document will review the fundamental ideas of linear algebra. B. Matrix operations We will learn about matrices, matrix operations, linear transformations and We denote by A the matrix as a whole and refer to its entries as a ij . discuss both the theoretical and computational aspects of linear algebra. The The mathematical operations defined for matrices are the following: tools of linear algebra open the gateway to the study of more advanced mathematics. A lot of knowledge buzz awaits you if you choose to follow the • addition (denoted + ) path of understanding , instead of trying to memorize a bunch of formulas. C = A + B ⇔ c ij = a ij + b ij . I. I NTRODUCTION • subtraction (the inverse of addition) • matrix product. The product of matrices A ∈ R m × n and B ∈ R n × ℓ Linear algebra is the math of vectors and matrices. Let n be a positive integer and let R denote the set of real numbers, then R n is the set of all is another matrix C ∈ R m × ℓ given by the formula v ∈ R n is an n -tuple of real numbers. n -tuples of real numbers. A vector � n The notation “ ∈ S ” is read “element of S .” For example, consider a vector � C = AB ⇔ c ij = a ik b kj , that has three components: k =1 v = ( v 1 , v 2 , v 3 ) ∈ ( R , R , R ) ≡ R 3 .     a 11 a 12 a 11 b 11 + a 12 b 21 a 11 b 12 + a 12 b 22 � � b 11 � b 12 a 21 a 22 = a 21 b 11 + a 22 b 21 a 21 b 12 + a 22 b 22 A matrix A ∈ R m × n is a rectangular array of real numbers with m rows     b 21 b 22 a 31 a 32 a 31 b 11 + a 32 b 21 a 31 b 12 + a 32 b 22 and n columns. For example, a 3 × 2 matrix looks like this: • matrix inverse (denoted A − 1 )     a 11 a 12 R R • matrix transpose (denoted T ):  ∈  ≡ R 3 × 2 . A = a 21 a 22 R R   a 31 a 32 R R   α 1 β 1 � T � α 1 α 2 α 3  . = α 2 β 2 The purpose of this document is to introduce you to the mathematical  β 1 β 2 β 3 α 3 β 3 operations that we can perform on vectors and matrices and to give you a feel of the power of linear algebra. Many problems in science, business, • matrix trace: Tr [ A ] ≡ � n i =1 a ii and technology can be described in terms of vectors and matrices so it is • determinant (denoted det ( A ) or | A | ) important that you understand how to work with these. Note that the matrix product is not a commutative operation: AB � = BA . Prerequisites C. Matrix-vector product The only prerequisite for this tutorial is a basic understanding of high school math concepts 1 like numbers, variables, equations, and the fundamental The matrix-vector product is an important special case of the matrix- arithmetic operations on real numbers: addition (denoted + ), subtraction matrix product. The product of a 3 × 2 matrix A and the 2 × 1 column (denoted − ), multiplication (denoted implicitly), and division (fractions). x results in a 3 × 1 vector � vector � y given by: You should also be familiar with functions that take real numbers as       y 1 a 11 a 12 a 11 x 1 + a 12 x 2 � x 1 � inputs and give real numbers as outputs, f : R → R . Recall that, by � y = A� x ⇔ y 2  = a 21 a 22 = a 21 x 1 + a 22 x 2 definition, the inverse function f − 1 undoes the effect of f . If you are      x 2 y 3 a 31 a 32 a 31 x 1 + a 32 x 2 given f ( x ) and you want to find x , you can use the inverse function as follows: f − 1 ( f ( x )) = x . For example, the function f ( x ) = ln( x ) has the     a 11 a 12 inverse f − 1 ( x ) = e x , and the inverse of g ( x ) = √ x is g − 1 ( x ) = x 2 . = x 1 a 21  + x 2 a 22 (C)    a 31 a 32 II. D EFINITIONS  ( a 11 , a 12 ) · �  x A. Vector operations ( a 21 , a 22 ) · � = x . (R)   ( a 31 , a 32 ) · � x We now define the math operations for vectors. The operations we can u = ( u 1 , u 2 , u 3 ) and � v = ( v 1 , v 2 , v 3 ) are: addition, perform on vectors � There are two 2 fundamentally different yet equivalent ways to interpret the subtraction, scaling, norm (length), dot product, and cross product: matrix-vector product. In the column picture, (C) , the multiplication of the u + � v = ( u 1 + v 1 , u 2 + v 2 , u 3 + v 3 ) � matrix A by the vector � x produces a linear combination of the columns of the matrix : � y = A� x = x 1 A [: , 1] + x 2 A [: , 2] , where A [: , 1] and A [: , 2] are u − � v = ( u 1 − v 1 , u 2 − v 2 , u 3 − v 3 ) � the first and second columns of the matrix A . α� u = ( αu 1 , αu 2 , αu 3 ) In the row picture, (R) , multiplication of the matrix A by the vector � x � || � u || = u 2 1 + u 2 2 + u 2 produces a column vector with coefficients equal to the dot products of 3 rows of the matrix with the vector � x . u · � � v = u 1 v 1 + u 2 v 2 + u 3 v 3 � u × � v =( u 2 v 3 − u 3 v 2 , u 3 v 1 − u 1 v 3 , u 1 v 2 − u 2 v 1 ) D. Linear transformations The dot product and the cross product of two vectors can also be described The matrix-vector product is used to define the notion of a linear in terms of the angle θ between the two vectors. The formula for the dot transformation , which is one of the key notions in the study of linear product of the vectors is � u · � v = � � u �� � v � cos θ . We say two vectors � u and algebra. Multiplication by a matrix A ∈ R m × n can be thought of as � v are orthogonal if the angle between them is 90 ◦ . The dot product of computing a linear transformation T A that takes n -vectors as inputs and orthogonal vectors is zero: � u · � v = � � u �� � v � cos(90 ◦ ) = 0 . produces m -vectors as outputs: The norm of the cross product is given by � � u × � v � = � � u �� � v � sin θ . The T A : R n → R m . u × � v � = � v × � u × � v = − � v × � cross product is not commutative: � u , in fact � u . 1 A good textbook to (re)learn high school math is minireference.com 2 For more info see the video of Prof. Strang’s MIT lecture: bit.ly/10vmKcL 1