Arrays: computing with many numbers Some perspective We have so far - PowerPoint PPT Presentation

Arrays: computing with many numbers

Some perspective — We have so far (mostly) looked at what we can do with single numbers (and functions that return single numbers). — Things can get much more interesting once we allow not just one, but many numbers together. — It is natural to view an array of numbers as one object with its own rules. — The simplest such set of rules is that of a vector. Demo “Sound as Vectors”

Vectors A vector is an element of a Vector Space 8 9 x 1 > > > > x 2 > > < = 𝑜 -vector: x 2 · · · x n ] T x = = [ x 1 . . . > > > > > > x n : ; Vector space 𝓦 : A vector space is a set 𝒲 of vectors and a field ℱ of scalars with two operations: 1) addition: 𝑣 + 𝑤 ∈ 𝒲 , and 𝑣, 𝑤 ∈ 𝒲 2) multiplication : α * 𝑣 ∈ 𝒲 , and 𝑣 ∈ 𝒲 , α ∈ ℱ

Vector Space The addition and multiplication operations must satisfy: (for 𝛽 , 𝛾 ∈ ℱ and 𝑣 , 𝑤 ∈ 𝒲 ) Associativity: 𝑣 + 𝑤 + 𝑥 = 𝑣 + 𝑤 + 𝑥 Commutativity: 𝑣 + 𝑤 = 𝑤 + 𝑣 Additive identity: 𝑤 + 0 = 𝑤 Additive inverse: 𝑤 + −𝑤 = 0 Associativity wrt scalar multiplication: 𝛽 * 𝛾 * 𝑤 = 𝛽 * 𝛾 * 𝑤 Distributive wrt scalar addition: 𝛽 + 𝛾 * 𝑤 = 𝛽 * 𝑤 + 𝛾 * 𝑤 Distributive wrt vector addition: 𝛽 * 𝑣 + 𝑤 = 𝛽 * 𝑣 + 𝛽 * 𝑤 Scalar multiplication identity: 1 * 𝑣 = 𝑣

Linear Functions Function: set 𝒵 set 𝒴 (“output data”) (“input data”) The function 𝑔 takes vectors 𝒚 ∈ 𝒴 and transforms into vectors 𝒛 ∈ 𝒵 A function f is a linear function if (1) f ( u + v ) = f ( u ) + f ( v ) (2) f ( a u ) = a f ( u ) for any scalar a

Iclicker question 1) Is 𝑔 𝑦 = |"| " , 𝑔: ℛ → ℛ a linear function? A) YES B) NO 2) Is 𝑔 𝑦 = 𝑏 𝑦 + 𝑐 , 𝑔: ℛ → ℛ , 𝑏, 𝑐 ∈ ℛ and 𝑏, 𝑐 ≠ 0 a linear function? A) YES B) NO

Matrices 𝐵 ## 𝐵 #$ … 𝐵 #% 𝐵 $# 𝐵 $$ … 𝐵 $% • 𝑛 ×𝑜 -matrix 𝑩 = ⋮ ⋮ ⋱ ⋮ 𝐵 &# 𝐵 &$ … 𝐵 &% • Linear functions 𝑔(𝒚) can be represented by a Matrix-Vector multiplication. • Think of a matrix 𝑩 as a linear function that takes vectors 𝒚 and transforms them into vectors 𝒛 𝒛 = 𝑔(𝒚) → 𝒛 = 𝑩 𝒚 • Hence we have: 𝑩 𝒗 + 𝒘 = 𝑩 𝒗 + 𝑩 𝒘 𝑩 𝛽 𝒗 = 𝛽 𝑩 𝒗

Matrix-Vector multiplication Recall summation notation for matrix-vector multiplication 𝒛 = 𝑩 𝒚 • • You can think about matrix-vector multiplication as: Linear combination of 𝒛 = 𝑦 ! 𝐁 : , 1 + 𝑦 " 𝐁 : , 2 + ⋯ + 𝑦 # 𝐁[: , 𝑜] column vectors of A 𝐁 1, : 4 𝒚 Dot product of 𝒚 with 𝒛 = ⋮ rows of A 𝐁 𝑛, : 4 𝒚

Matrices operating on data Data set: 𝒚 Data set: 𝒛 Rotation 𝒛 = 𝒈 𝒚 or 𝒛 = 𝑩 𝒚

Demo “Matrix transformations” Example: Shear operator Matrix-vector multiplication for each vector (point representation in 2D): 𝑦 ( 𝑧 # 𝑦 # 𝑧 $ = 1 𝑐 𝑦 $ 𝑏 1 1 1 0.25 = 1.25 0 1 1 1 1 0.25 0 = 0.25 1 0 1 1 𝑦 ' 1 0.25 1 = 1 0 0 0 1 We can do this better… 𝑧 "! ⋯ 𝑧 !# 𝑧 !! 𝑦 !! 𝑦 "! … 𝑦 !# = 1 0.25 𝒁 = 𝑩 𝒀 𝑧 "# 𝑦 "# 0 1 (2×𝑜) (2×𝑜) (2×2) 𝑜 : number of data points in the set

Matrices as operators • Data : grid of 2D points • Transform the data using matrix multiply What can matrices do? 1. Shear 2. Rotate 3. Scale 4. Reflect 5. Can they translate?

Demo “Matrix transformations” Rotation operator 𝑧 # 𝑦 # 𝑧 $ = cos(𝜄) −sin(𝜄) 𝑦 $ sin(𝜄) cos(𝜄) 𝜄 = 𝜌/6

Demo “Matrix transformations” Scale operator 𝑧 # 𝑦 # 𝑧 $ = 𝑏 0 𝑦 $ 0 𝑐 3/2 0 0 3/2 3/2 0 0 1 3/2 0 0 2/3

Demo “Matrix transformations” Reflection operator 𝑧 # 𝑦 # 𝑧 $ = −𝑏 0 𝑦 $ 0 −𝑐 −1 0 −1 0 0 −1 0 1 Reflect about y-axis Reflect about x and y-axis

Demo “Matrix transformations” Translation (shift) 𝑧 # 𝑦 # + 𝑏 𝑧 $ = 1 0 𝑦 $ 𝑐 0 1 𝑏 = 0.6; 𝑐 = 1.1

Matrices operating on data Data set: 𝑩 Data set: 𝑪 Rotation Demo “Matrices for geometry transformation”

Iclicker question A triangle has vertices with coordinates (1,1), (2,1) and (2,4). The triangle is transformed by the matrix 1.5 0 0 0.75 What is the operation defined by the matrix above? a) Expand b) Shrink c) Scale d) Rotate e) Reflect What are the coordinates of the transformed triangle? a) (0.75,1.5), (0.75,3), (3,3) b) (1.5,0.75), (3,0.75), (3,3) c) (1.5,0.75), (1.5,1.5), (6,1.5) d) (0.75,1.5), (1.5,1.5), (1.5,6)

6 Notation and special matrices � Square matrix: 𝑛 = 𝑜 • ⇢ 1 i = j δ ij = 0 i 6 = j A ij = 0 • Zero matrix: ⇢ [ I ] = [ δ ij ] • Identity matrix [ A ] = [ A ] T • Symmetric matrix: A ij = A ji ⇢ • Permutation matrix: 𝑏 𝑑 0 0 1 𝑐 𝑏 = 1 0 0 • Permutation of the identity matrix 𝑑 𝑐 0 1 0 • Permutes (swaps) rows Diagonal matrix: 𝐵 $% = 0, ∀𝑗, 𝑘 | 𝑗 ≠ 𝑘 • • Triangular matrix: Upper triangular: 𝑉 $% = ?𝑉 $% , 𝑗 ≤ 𝑘 Lower triangular: 𝑀 $% = ?𝑀 $% , 𝑗 ≥ 𝑘 0, 𝑗 > 𝑘 0, 𝑗 < 𝑘

More about matrices Rank: the rank of a matrix 𝑩 is the dimension of the vector space generated • by its columns, which is equivalent to the number of linearly independent columns of the matrix. Suppose 𝑩 has shape 𝑛×𝑜: • 𝑠𝑏𝑜𝑙 𝑩 ≤ min(𝑛, 𝑜) • Matrix 𝑩 is full rank : 𝑠𝑏𝑜𝑙 𝑩 = min(𝑛, 𝑜) . Otherwise, matrix 𝑩 is • rank deficient . Singular matrix: a square matrix 𝑩 is invertible if there exists a square matrix • 𝐂 such that 𝑩𝑪 = 𝑪𝑩 = 𝑱 . If the matrix is not invertible, it is called singular.

Norms

Demo “Vector Norms” Example of Norms

Unit Ball: Set of vectors 𝒚 with norm 𝒚 = 1 Demo “Vector Norms”

— Why should be 𝑞 ≥ 1 when calculating the p-norm? 𝑦 ( (0,1) 𝑦 ' (1,0)

Iclicker question a) b) c) d) e)

Norms and Errors

Absolute and Relative Errors Absolute error: Relative error: a) 2.59×10 ;< a) 0.2240 b) 2.81×10 ;< b) 0.3380 c) 0.2513

Matrix Norms

Matrix Norms

Matrix Norms

Induced Matrix Norms % 𝑩 # = max X 𝐵 21 Maximum absolute column sum of the matrix 𝑩 1 23# % 𝑩 4 = max X 𝐵 21 Maximum absolute row sum of the matrix 𝑩 2 13# 𝑩 $ = max 𝜏 5 5 𝜏 $ are the singular value of the matrix 𝑩

Properties of Matrix Norms Demo “Matrix Norms”

Iclicker question Determine the norm of the following matrices: 1) a) 3 b) 4 c) 5 2) d) 6 e) 7

Iclicker question a) 90 b) 30 c) 20 d) 10 e) 5