Matrix Arithmetic (Multidimensional Math) Shaina Race, PhD Institute for Advanced Analytics.
Element-wise Operations T ABLE OF C ONTENTS Linear Combinations of Matrices and Vectors. Vector Multiplication Inner products and Matrix-Vector Multiplication Matrix Multiplication Inner product and linear combination viewpoint Vector Multiplication The Outer Product
Matrix Addition/Subtraction ‣ Two matrices/vectors can be added/subtracted if and only if they have the same size ‣ Then simply add/subtract corresponding elements
Matrix Addition/Subtraction = ( ( ( ( ( ( ( ( ( ( + + + + + + ( A + B ) ij + A ij B ij + + + + + + + + + ! # # " ## $ ! # # " ## $ ! # # " ## $ B A+B A ( A + B ) ij = A ij + B ij (Element-wise)
Example: Matrix Addition/Subtraction
Scalar Multiplication ( ( ( ( α α α α α α α α α α = α α α α α α ! # # " ## $ ! # # " ## $ α M M ( α M ) ij = α M ij (Element-wise)
Geometric Look Vector addition and scalar multiplication
Points <—> Vectors Vectors have both ⎛ ⎞ 1 a = direction and magnitude ⎜ ⎟ ⎝ ⎠ 2 a Direction arrow points from Magnitude is the length of origin to the coordinate point that arrow #pythagoras
Scalar Multiplication (Geometrically) 2a a -0.5a
Vector Addition (Geometrically) b a a a + b b addition is still commutative
Example: Centering the data x 2 x 1 Average/Mean (Centroid) ( x 1 , x 2 )
Example: Centering the data x 2 x 1
Example: Centering the data x 2 x 1 New mean is the origin (0,0)
Linear Combinations
Linear Combinations A linear combination of vectors is a just weighted sum: α 1 v 1 + α 2 v 2 +…+ α p v p Vectors v i Scalar Coe ffi cients ⍺ i
Elementary Linear Combinations ‣ The simplest linear combination might involve columns of the identity matrix (elementary vectors): ‣ Picture this linear combination as a “breakdown into parts” where the parts give directions along the 3 coordinate axes.
Linear Combinations (Geometrically) - 3 b a a - 3b
Linear Combinations (Geometrically) (axis 3) (axis 1) (axis 2)
Example: Linear Combination of Matrices
Element-wise Operations T ABLE OF C ONTENTS Linear Combinations of Matrices and Vectors. Vector Multiplication Inner products and Matrix-Vector Multiplication Matrix Multiplication Inner product and linear combination viewpoint Vector Multiplication The Outer Product
Notation: Column vs. Row Vectors ‣ Throughout this course, unless otherwise specified, all vectors are assumed to be columns. ‣ Simplifies notation because if x is a column vector: ⎛ ⎞ x 1 ⎜ ⎟ x 2 ⎜ ⎟ x = ⎜ ⎟ ! ⎜ ⎟ x n ⎝ ⎠ then we can automatically assume that x T is a row vector: x T = ( x 1 x 2 … x n )
Vector Inner Product ‣ The vector inner product is the multiplication of a row vector times a column vector. ‣ It is known across broader sciences as the ‘dot product’. ‣ The result of this product is a scalar. =
Inner Product (row x column) ( ( 1 2 ( ( 3 = ! # # " ## $ 4 5 a T n 6 ∑ a i b i a T b = ! # # " ## $ b i = 1
Inner Product (row x column) 3 1 2 4 5 6 ( ( * * * * * * + + + + + = a and b must have the same number of elements.
Examples: Inner Product
Examples: Inner Product
Check your Understanding
Check your Understanding S OLUTION
Matrix-Vector Multiplication Inner Product View (I-P View)
Matrix-Vector Multiplication (I-P view) ( ( ( ( 1 2 3
Matrix-Vector Multiplication (I-P view) ( 2 3 1 ( ( ( Sizes must match up!
Matrix-Vector Multiplication (I-P view) ( ( 2 3 1
Matrix-Vector Multiplication (I-P view) ( ( 2 3 1
Matrix-Vector Multiplication (I-P view) ( ( ( ( ( ( =
Example: Matrix-Vector Products
Matrix-Vector Multiplication Linear Combination View (L-C View)
Matrix-Vector Multiplication (L-C view) ( ( ( ( 1 2 3
Matrix-Vector Multiplication (L-C view) = ( ( ( ( ( ( ( ( + + 1 3 2
Example: Linear Combination View ⎛ ⎞ ⎛ ⎞ 3 2 ⎛ ⎞ 3 ⎜ ⎟ ⎜ ⎟ − 1 4 ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎝ 2 ⎠ ⎜ ⎟ ⎜ ⎟ 1 ⎝ ⎠ 5 ⎝ ⎠
Example: Linear Combination View ⎛ ⎞ ⎛ ⎞ 2 3 ⎜ ⎟ ⎜ ⎟ − 1 = 3 + 2 4 ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ 1 ⎝ ⎠ 5 ⎝ ⎠
Element-wise Operations T ABLE OF C ONTENTS Linear Combinations of Matrices and Vectors. Vector Multiplication Inner products and Matrix-Vector Multiplication Matrix Multiplication Inner product and linear combination viewpoint Vector Multiplication The Outer Product
Matrix-Matrix Multiplication • Matrix multiplication is NOT commutative. AB ≠ BA • Matrix multiplication is only defined for dimension- compatible matrices
Matrix-Matrix Multiplication (I-P View) • If A and B are dimension compatible, then we compute the product AB by multiplying every row of A by every column of B (inner products) . • The ( i,j ) th entry of the product AB is the i th row of A multiplied by the j th column of B
Matrix-Matrix Multiplication (I-P View) A and B are dimension compatible for the product AB if the number of columns in A is equal to the number of rows in B ( ( ( ( 0 1 6 9 ( ( 0 1 6 9 A 2 5 8 = A 5 8 2 S 3 4 7 S 3 4 7 S 3 4 7 ! # # " ## $ ! # # " ## $ ! # # " ## $ B (AB) A 4 x 3 3 x 4 4 x 4
Matrix-Matrix Multiplication (I-P View) ( ( ( ( 0 6 9 1 ( ( A 2 ! 0 6 9 1 ( AB ) 23 A 2 5 8 = B ! 3 A 2 5 8 S 3 4 7 S 4 7 3 S 3 4 7 ( AB ) ij = A i ! B ! j
Example: Matrix-Matrix Multiplication
Check Your Understanding
Check your Understanding S OLUTION
Check Your Understanding
Check your Understanding S OLUTION
NOT Commutative ‣ Very important to remember that Matrix multiplication is NOT commutative! ‣ As we see in previous exercise, common to be able to compute product AB when the reverse product, BA, is not even defined. ‣ Even when both products are possible, almost never the case that AB = BA.
Diagonal Scaling Multiplication by a diagonal matrix
Multiplication by a diagonal matrix The net e ff ect is that the rows of A are scaled by the corresponding diagonal element of D
Multiplication by a diagonal matrix Rather than computing DA, what if we instead put the diagonal matrix on the right hand side and compute AD? AD = (Exercise)
Matrix-Matrix Multiplication As a Collection of Linear Combinations (L-C View)
Matrix-Matrix Multiplication (L-C View) ‣ Just a collection of matrix-vector products (linear combinations) with di ff erent coe ffi cients. ‣ Each linear combination involves the same set of vectors (the green columns) with di ff erent coe ffi cients (the purple columns).
Matrix-Matrix Multiplication (L-C View) ( ( ( ( 0 6 9 1 ( ( 0 6 9 1 A 2 5 8 = A 2 5 8 S 4 7 3 S 4 7 3 S 3 4 7
Matrix-Matrix Multiplication (L-C View) ( ( 6 9 1 ( ( 6 9 1 2 5 8 = 2 5 8 4 7 3 4 7 3 3 4 7 ( ( ( ( ( 0 ( 0 A A = S S S
Matrix-Matrix Multiplication (L-C View) ( ( 6 9 1 ( ( 6 9 1 2 5 8 = 2 5 8 4 7 3 4 7 3 3 4 7 ⎛ ⎞ a 11 ⎜ ⎟ ( ( n ( ( ∑ ( ( ⎜ ⎟ ( ( ⎜ ⎟ i = 1 ⎜ ⎟ ⎝ ⎠ 0 A + + S 0 A = S S
Matrix-Matrix Multiplication (L-C View) ( ( 9 0 1 ( ( 9 0 1 2 8 A = A 2 8 7 S 3 7 S 3 3 7 S ( ( ( ( ( ( ( ( 6 5 + + 6 4 5 = 4 4
Matrix-Matrix Multiplication (L-C View) ( ( 6 0 1 ( ( 6 0 1 2 5 A = A 2 5 4 S 3 4 S 3 3 4 S ( ( ( ( ( ( ( ( 9 8 + + 9 7 8 = 7 7
Matrix-Matrix Multiplication (L-C View) ‣ Just a collection of matrix-vector products (linear combinations) with di ff erent coe ffi cients. ‣ Each linear combination involves the same set of vectors (the green columns) with di ff erent coe ffi cients (the purple columns). ‣ This has important implications!
Element-wise Operations T ABLE OF C ONTENTS Linear Combinations of Matrices and Vectors. Vector Multiplication Inner products and Matrix-Vector Multiplication Matrix Multiplication Inner product and linear combination viewpoint Vector Multiplication The Outer Product
Vector Outer Product ‣ The vector outer product is the multiplication of a column vector times a row vector. ‣ For any column/row this product is possible ‣ The result of this product is a matrix ! = (1 x n ) (m x 1) (m x n )
Outer Product (column x row) ( ( ( = ( 1 2 ( ( 3 ! # # " ## $ 4 a T 5 ! # # " ## $ ! # # " ## $ b ba T
Outer Product (column x row) ( ( ( ( ( ( = ! # # " ## $ a T ! # # " ## $ ! # # " ## $ b ba T
Recommend
More recommend
