Graphics 2014 Linear Algebra II Linear Maps & Matrices Linear - PowerPoint PPT Presentation

Invertible Matrices Rank  Number of linearly independent columns  Dimension of span{𝐝𝐩𝐦𝐯𝐧𝐨_𝐰𝐟𝐝𝐮𝐩𝐬𝐭} Theorem  Rank = number of linearly independent rows Full rank (𝑊)  rank(𝐍) = dim  Then: 𝐍 is invertible

Linear Systems of Equations First consider simpler case  Say, we know that 𝐍 ⋅ 𝐲 = 𝐳  Square matrix 𝐍 ∈ ℝ 𝑒×𝑒  Vectors 𝐲, 𝐳 ∈ ℝ 𝑒×𝑒 Knowns & Unknowns  We are given 𝐍 , 𝐳  We should compute 𝐲  Linear system of equations

Linear Systems of Equations Linear System of Equations 𝐍 ⋅ 𝐲 = 𝐳 ⇔ 𝑛 1,1 ⋯ 𝑛 1,𝑒 𝑦 1 𝑧 1 ⋮ ⋮ ⋮ ⋮ ⋅ = 𝑛 𝑒,1 ⋯ 𝑛 𝑒,𝑒 𝑦 𝑒 𝑧 𝑒 ⇔ 𝑛 1,1 𝑦 1 + ⋯ + 𝑛 1,𝑒 𝑦 𝑒 = 𝑧 1 𝑛 2,1 𝑦 1 + ⋯ + 𝑛 2,𝑒 𝑦 𝑒 = 𝑧 2 and ⋮ 𝑛 𝑒,1 𝑦 1 + ⋯ + 𝑛 𝑒,𝑒 𝑦 𝑒 = 𝑧 𝑒 and

Gaussian Elimination Linear System ∧ 𝑛 1,1 𝑦 1 + ⋯ + 𝑛 1,𝑒 𝑦 𝑒 = 𝑧 1 ∧ 𝑛 2,1 𝑦 1 + ⋯ + 𝑛 2,𝑒 𝑦 𝑒 = 𝑧 2 ⋮ ∧ 𝑛 𝑒,1 𝑦 1 + ⋯ + 𝑛 𝑒,𝑒 𝑦 𝑒 = 𝑧 𝑒 Row Operations  Swap rows 𝑠 𝑗 , 𝑠 𝑘  Scale row 𝑠 𝑗 by factor 𝜇 ≠ 0  Add multiple of row 𝑠 𝑗 to row 𝑠 𝑘 , 𝑗 ≠ 𝑘 (i.e., 𝑠 𝑘 ) 𝑗 += 𝜇𝑠

Convert to Upper Triangle Matrix 𝐳 𝐍 𝐲 0 = = 0 0 0 0 0 0 = 0 0 0 = 0 0 0 0 (use row-operations)

Convert to Diagonal Matrix 𝐳 𝐍 𝐲 0 = = 0 0 0 0 0 ′ 0 0 0 𝑛 1,1 𝑦 1 ′ 𝑧 1 ′ 0 0 𝑦 2 ′ 0 0 𝑛 2,2 𝑧 2 = = ′ 0 𝑦 3 ′ 0 0 0 𝑧 3 𝑛 3,3 ′ 𝑦 4 ′ 0 0 0 0 𝑛 4,4 𝑧 4 ′ 0 0 0 1 𝑦 1 ′ /m 1,1 𝑧 1 ′ 0 0 0 0 𝑦 2 ′ /m 2,2 1 𝑧 2 = = ′ 0 0 0 0 0 𝑦 3 ′ /m 3,3 1 𝑧 3 ′ 0 0 0 0 0 𝑦 4 ′ /m 4,4 1 𝑧 4 (use row-operations)

Gauss-Algorithm Gauss-Algorithm  Substract rows to cancel front-coefficient  Create upper triangle matrix first  Then create diagonal matrix  If current row starts with 0  Swap with another row  If all rows start with 0: matrix not invertible  Diagonal form: Solution can be read-off  Data structure  Modify matrix M , “right -hand- side” y .  x remains unknown (no change)

Matrix Inverse Solve for 1 0 0 0 1 0 𝐍 ⋅ 𝐲 1 = , 𝐍 ⋅ 𝐲 2 = , … , 𝐍 ⋅ 𝐲 𝑒 = ⋮ ⋮ ⋮ 0 0 1  The resulting 𝐲 1 , 𝐲 2 , … , 𝐲 𝑒 are the columns of 𝐍 −1 : | | 𝐍 −1 = 𝐲 1 ⋯ 𝐲 𝑒 | |

Matrix Inverse Algorithm  Simultaneous Gaussian elimination  Start as follows: 𝐍 𝐲 𝐉 0 0 0 1 0 0 0 1 = 0 0 0 1 0 0 0 1  Handle all right-hand sides simultaneously  After Gauss-algorithm, the right-hand matrix is the inverse

Alternative: Kramer’s Rule Small Matrices  Direct formula based on determinants  “Kramer’s rule”  (more later)  Naive implementation has run-time 𝒫(𝑒!) – Gauss: 𝒫(𝑒 3 )  Not advised for 𝑒 > 3

More Vector Operations: Scalar Products BASIC basic topics study completely

Additional Vector Operations 𝐰 2 𝐰 1 𝐰 1 = 𝟑. 𝟒cm 𝐰 2 = 𝟓. 𝟑cm Length of Vectors “length” or “norm” ‖𝐰‖ yields real number ≥ 0

Additional Vector Operations 𝐰 2 𝐰 1 𝛽 𝛽 = ∠ 𝐰 1 , 𝐰 2 = 𝟒𝟒° Angle between Vectors angle ∠ 𝐰 1 , 𝐰 2 yields real number 0, … , 2𝜌 = [0, … , 360°)

Additional Vector Operations 𝐰 2 𝐰 1 90° right angles Angle between Vectors

Additional Vector Operations 𝐰 𝐱 90° 𝐰 prj on 𝐱 Projection Projection: determine length of 𝐰 along direction of 𝐱

Additional Vector Operations 𝐰 𝐱 90° Scalar Product *) 𝐰 ⋅ 𝐱 = 𝐰 ⋅ 𝐱 ⋅ cos ∠(𝐰, 𝐱) also: 𝐰, 𝐱 *) also known as inner product or dot-product

Signature in in operato tor ∗ out 42.0 Scalar Product (dot product, inner-product)

Additional Vector Operations 𝐰 𝐱 90° Scalar Product *) 𝐰 ⋅ 𝐱 = 𝐰 ⋅ 𝐱 ⋅ cos ∠(𝐰, 𝐱) also: 𝐰, 𝐱 *) also known as inner product or dot-product

Additional Vector Operations 𝐰 𝐱 90° Scalar Product *) 𝐰 ⋅ 𝐱 = 𝐰 ⋅ 𝐱 ⋅ cos ∠(𝐰, 𝐱) Comprises: length, projection, angles *) also known as inner product or dot-product

Additional Vector Operations Length: 𝐰 = 𝐰 ⋅ 𝐰 Angle: ∠ 𝐰, 𝐱 = arccos 𝐰 ⋅ 𝐱 Projection : „ 𝐰 prj on 𝐱 ” = 𝐰⋅𝐱 𝐱 𝐰 ⋅ 𝐱 = 𝐰 ⋅ 𝐱 ⋅ cos ∠(𝐰, 𝐱) Comprises: length, projection, angles

Algebraic Representation (Implementation) BASIC basic topics study completely

Scalar Product 𝐰 𝐱 90° Scalar Product *) 𝐰 ⋅ 𝐱 = 𝑤 1 𝑤 2 ⋅ 𝑥 1 ≔ 𝑤 1 ⋅ 𝑥 1 + 𝑤 2 ⋅ 𝑥 2 𝑥 2 Theorem: 𝐰 ⋅ 𝐱 = 𝐰 ⋅ 𝐱 ⋅ cos ∠(𝐰, 𝐱)

Scalar Product 𝐱 𝐰 = 3 𝐰 2 𝐱 = 1 2 Scalar product 𝐰 ⋅ 𝐱 = 𝑤 1 𝑤 2 ⋅ 𝑥 1 𝑥 2

Scalar Product 2D Scalar product 𝐰 ⋅ 𝐱 = 𝑤 1 𝑤 2 ⋅ 𝑥 1 ≔ 𝑤 1 ⋅ 𝑥 1 + 𝑤 2 ⋅ 𝑥 2 𝑥 2 d -dim scalar product 𝑤 1 𝑥 1 ≔ 𝑤 1 ⋅ 𝑥 1 + ⋯ + 𝑤 𝑒 ⋅ 𝑥 𝑒 𝐰 ⋅ 𝐱 = ⋅ ⋮ ⋮ 𝑤 𝑒 𝑥 𝑒

Algebraic Properties Settings Properties 𝜇 ∈ ℝ  Symmetry (commutativity) 𝐯, 𝐰, 𝐱 ∈ ℝ 𝑒 𝐯, 𝐰 = 𝐰, 𝐯  Bilinearity 𝜇𝐰, 𝐱 = 𝜇 𝐰, 𝐱 = 𝐰, 𝜇𝐱 𝐯 + 𝐰, 𝐱 = 𝐯, 𝐱 + 𝐰, 𝐱 (symmetry: same for second argument)  Positive definite 𝐯, 𝐯 = 𝟏 ⇒ 𝐯 = 𝟏 𝐯, 𝐯 ≥ 0, These three: axiomatic definition

Attention! Do not mix  Scalar-vector product  Inner (scalar) product In general 𝐲, 𝐳 ⋅ 𝐴 ≠ 𝐲 ⋅ 𝐳, 𝐴 Beware of notation: 𝐲 ⋅ 𝐳 ⋅ 𝐴 ≠ 𝐲 ⋅ 𝐳 ⋅ 𝐴 (no violation of associativity: different operations; details later)

Applications of the Scalar Product CORE core topics important

Applications Obvious applications  Measuring length  Measuring angles  Projections More complex applications  Creating orthogonal (90°) pairs of vectors  Creating orthogonal bases

Projection 𝐰 𝐱 90° Scalar Product *) 𝐰 ⋅ 𝐱 = 𝐰 ⋅ 𝐱 ⋅ cos ∠(𝐰, 𝐱)

Projection 𝐱 𝐱 𝐰 𝐱 n = = 𝐱 𝐱, 𝐱 𝐱 90° 𝐱 n prj.-vector projection Scalar Product *) 𝐱 Prj.-Vector: 𝐰, Projection: 𝐰 ⋅ 𝐱⋅𝐱 𝐱 𝐱 𝐱,𝐱 ⋅ 𝐱,𝐱 𝐱 = 𝐰, 𝐱 ⋅ 𝐱,𝐱

Orthogonalization 𝐱 𝐱 𝐱 n = = 𝐰 𝐱 𝐱, 𝐱 𝐰 ′ 𝐱 90° prj.-vector projection Scalar Product *) Orthogonalize 𝐰 wrt. 𝐱 : 𝐱 𝐰 ′ = 𝐰 − 𝐰, 𝐱 ⋅ 𝐱, 𝐱

Orthogonalization 𝐰 𝐰 ′ 𝐱 90° Scalar Product *) Orthogonalize 𝐰 wrt. 𝐱 : 𝐱 𝐰 ′ = 𝐰 − 𝐰, 𝐱 ⋅ 𝐱, 𝐱

Gram-Schmidt Orthogonalization Orthogonal basis  All vectors in 90° angle to each other 𝐜 𝑗 , 𝐜 𝑘 = 0 for 𝑗 ≠ 𝑘 Create orthogonal bases  Start with arbitrary one  Orthogonalize 𝐜 2 by 𝐜 1  Orthogonalize 𝐜 3 by 𝐜 1 , then by 𝐜 2  Orthogonalize 𝐜 4 by 𝐜 1 , then by 𝐜 2 , then by 𝐜 3  ...

Orthonormal Basis Orthonormal bases  Orthogonal and all vectors have unit length Computation  Orthogonalize first  Then scale each vector 𝐜 𝑗 by 1/ 𝐜 𝑗 .

Matrices Orthogonal Matrices  A matrix with orthonormal columns is called orthogonal matrix  Yes, this terminology is not quite logical... Orthogonal Matrices are always  Rotation matrices  Or reflection matrices  Or products of the two

Further Operations CORE core topics important

Cross Product Cross-Product: Exists Only For 3D Vectors!  𝐲, 𝐳 ∈ ℝ 3 𝑦 1 𝑧 1 𝑦 2 𝑧 3 − 𝑦 3 𝑧 2 𝑧 2 𝑦 2 𝑦 3 𝑧 1 − 𝑦 1 𝑧 3  𝐲 × 𝐳 = × ≔ 𝑦 3 𝑧 3 𝑦 1 𝑧 2 − 𝑦 2 𝑧 1 Geometrically: Theorem x  y  𝐲 × 𝐳 orthogonal to 𝐲, 𝐳 y  Right-handed system 𝐲, 𝐳, 𝐲 × 𝐳 ‖ x  y ‖ = 𝐲 ⋅ 𝐳 ⋅ sin∠ 𝐲, 𝐳 𝐲 × 𝐳 x 

Cross-Product Properties Bilinearity  Distributive: 𝐯 × 𝐰 + 𝐱 = 𝐯 × 𝐰 + 𝐯 × 𝐱  Scalar-Mult.: 𝜇𝐯 × 𝐰 = 𝐯 × 𝜇𝐰 = 𝜇 𝐯 × 𝐰 But beware of  Anti -Commutative: 𝐯 × 𝐰 = −𝐰 × 𝐯  Not associative; we can have 𝐯 × 𝐰 × 𝐱 ≠ 𝐯 × 𝐰 × 𝐱

Determinants det 𝐍 | | | v 1 𝐰 1 𝐰 2 𝐰 3 𝐍 = | | | v 3 v 2 Determinants  Square matrix M  det( M ) = |M| = volume of parallelepiped of column vectors

Determinants det 𝐍 > 0 | | | v 1 𝐰 1 𝐰 2 𝐰 3 𝐍 = | | | v 3 v 2 det 𝐍 ′ < 0 | | | 𝐍 ′ = v 2 𝐰 2 𝐰 1 𝐰 3 | | | v 3 negative determinant Sign: → map v 1 contains  Positive for right handed coordinates reflection  Negative for left-handed coordinates

Properties A few properties: sign flips! → reflections  det( A ) det( B ) = det( A ⋅ B ) cancel each  det( 𝜇 A ) = 𝜇 d det( A ) ( d  d matrix A ) other (parity)  det( A -1 ) = det( A ) -1  det( A T ) = det( A )  det 𝐁 ≠ 0 ⇔ 𝐁 invertible  Efficient computation using Gaussian elimination

Computing Determinants + − + 𝑏 𝑐 𝑑 𝑗 − 𝑐 𝑒 𝑔 𝑗 + 𝑑 𝑒 𝑓 = +𝑏 𝑓 𝑔 ℎ 𝑒 𝑓 𝑔 𝑕 𝑕 ℎ 𝑕 ℎ 𝑗 signs +𝑏 −𝑐 +𝑑 Recursive Formula 𝑒 𝑓 𝑔 𝑕 ℎ 𝑗  Sum over first row subdeterminants  Multiply element there 𝑏 𝑐 𝑑 with subdeterminant 𝑒 𝑔 𝑒 𝑓 𝑔 𝑗 𝑕 𝑕 ℎ 𝑗  Subdeterminant : Leave out row and column | a |= a of selected element  Recursion ends with | a |= a Beware of 𝒫 𝑒𝑗𝑛!  Alternate signs +/−/+/−/ … complexity

Computing Determinants Result in 3D Case 𝑏 𝑐 𝑑 = 𝑏𝑓𝑗 + 𝑐𝑔𝑕 + 𝑑𝑒ℎ − 𝑑𝑓𝑕 − 𝑐𝑒𝑗 − 𝑏𝑔ℎ 𝑒 𝑓 𝑔 det 𝑕 ℎ 𝑗

Solving Linear Systems Consider 𝐁 ⋅ 𝐲 = 𝐜  Invertible matrix 𝐁 ∈ ℝ 𝑒×𝑒  Known vector 𝐜 ∈ ℝ 𝑒  Unknown vector 𝐲 ∈ ℝ 𝑒 Solution with Determinants ( Cramar’s rule): | | | 𝑦 𝑗 = det 𝐁 𝑗 det 𝐁 𝐰 1 ⋯ 𝐜 ⋯ 𝐰 3 𝐁 𝑗 = | | | column 𝑗

Addendum Matrix Algebra ADV advanced topics main ideas

Matrix Algebra Define three operations  Matrix addition 𝑏 1,1 ⋯ 𝑏 1,𝑜 𝑐 1,1 ⋯ 𝑐 1,𝑜 𝑏 1,1 + 𝑐 1,1 ⋯ 𝑏 1,𝑜 + 𝑐 1,𝑜 ⋮ ⋱ ⋮ ⋮ ⋱ ⋮ ⋮ ⋱ ⋮ + = 𝑏 𝑛,1 ⋯ 𝑏 𝑛,𝑜 𝑐 𝑛,1 ⋯ 𝑐 𝑛,𝑜 𝑏 𝑛,1 + 𝑐 𝑛,1 ⋯ 𝑏 𝑛,𝑜 + 𝑐 𝑛,𝑜  Scalar matrix multiplication 𝑏 1,1 ⋯ 𝑏 1,𝑜 𝜇 ⋅ 𝑏 1,1 ⋯ 𝜇 ⋅ 𝑏 1,𝑜 ⋮ ⋱ ⋮ ⋮ ⋱ ⋮ 𝜇 ⋅ = 𝑏 𝑛,1 ⋯ 𝑏 𝑛,𝑜 𝜇 ⋅ 𝑏 𝑛,1 ⋯ 𝜇 ⋅ 𝑏 𝑛,𝑜  Matrix-matrix multiplication ⋱ ⋰ 𝑏 1,1 ⋯ 𝑏 1,𝑜 𝑐 1,1 ⋯ 𝑐 1,𝑛 𝑙 ⋮ ⋱ ⋮ ⋮ ⋱ ⋮ ⋅ = 𝑏 𝑟,𝑘 ⋅ 𝑐 𝑗,𝑟 𝑏 𝑛,1 ⋯ 𝑏 𝑛,𝑜 𝑐 𝑙,1 ⋯ 𝑐 𝑙,𝑛 𝑟=1 ⋰ ⋱

Transposition T = Matrix Transposition  Swap rows and columns  Formally: T ⋱ ⋅ ⋰ ⋅ ⋅ ⋅ ⋱ ⋅ ⋅ ⋅ ⋰ 𝑏 𝑗,𝑘 𝑏 𝑘,𝑗 ⋅ ⋅ = ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋰ ⋅ ⋅ ⋅ ⋱ ⋰ ⋅ ⋱

Vectors Vectors  Column matrices 𝐲 ∈ ℝ 𝑒  Matrix-Vector product consistent Co-Vectors 𝐳 T ∈ ℝ 𝑒  “projectors”, “dual vectors”, “linear forms”, “row vectors”  Vectors to be projected on Transposition  Convert vectors into projectors and vice versa

Vectors 𝐲 T ⋅ 𝐳 → ℝ Inner product (as a generalized “projection”)  Matrix-product 𝐝𝐩𝐦𝐯𝐧𝐨 ⋅ 𝐬𝐩𝐱 „ 𝐲 ⋅ 𝐳 “ = 𝐲, 𝐳 = 𝐲 T ⋅ 𝐳  People use all three notations  Meaning of “ ⋅ ” clear from context

Matrix-Vector Products 𝐲 𝐍 ⋅ 𝐲 = 𝐳 ⋅ + ⋅ + ⋅ + ⋅ ° ⋅ ⋅ ° ⋅ = ° ⋅ 𝐍 𝐳 ° ⋅ ° Two Interpretations  Linear combination of column vectors  Projection on row (co-)vectors

Matrix Algebra We can add and scalar multiply  Matrices and vectors (special case) We can matrix-multiply  Matrices with other matrices (execute one-after-another)  Vectors in certain cases (next) We can “divide” by some (not all) matrices  Determine inverse matrix  Full-rank, square matrices only

Algebraic Rules: Addition Settings Addition: like real numbers 𝐁, 𝐂, 𝐃 ∈ ℝ 𝑜×𝑛 (“commutative group”) (matrices, same size)  Prerequisites:  Number of rows match  Number of columns match  Associative: 𝐁 + 𝐂 + 𝐃 = 𝐁 + 𝐂 + 𝐃  Commutative: 𝐁 + 𝐂 = 𝐂 + 𝐁  Subtraction: 𝐁 + −𝐁 = 𝟏  Neutral Op.: 𝐁 + 𝟏 = 𝐁

Algebraic Rules: Scalar Multiplication Scalar Multiplication: Vector space  Prerequisites: Settings 𝜇 ∈ ℝ  Always possible 𝐁, 𝐂 ∈ ℝ 𝑜×𝑛  Repeated Scaling: 𝜇 𝜈𝐁 = 𝜇𝜈 𝐁 (same size)  Neutral Operation: 1 ⋅ 𝐁 = 𝐁  Distributivity 1: 𝜇(𝐁 + 𝐂) = 𝜇𝐁 + 𝜇𝐂  Distributivity 2: 𝜇 + 𝜈 𝐁 = 𝜇𝐁 + 𝜈𝐁 So far:  Matrices form vector space  Just different notation, same semantics!

Algebraic Rules: Multiplication Multiplication: Non-Commutative Ring / Group  Prerequisites:  Number of columns right = number of rows left  Associative: 𝐁 ⋅ 𝐂 ⋅ 𝐃 = 𝐁 ⋅ 𝐂 ⋅ 𝐃  Not commutative: often 𝐁 ⋅ 𝐂 ≠ 𝐂 ⋅ 𝐁  Neutral Op.: 𝐁 ⋅ 𝐉 = 𝐁 𝐁 ⋅ 𝐁 −1 = 𝐉  Inverse: Set of invertible matrices:  Additional prerequisite: – Matrix must be square! 𝐻𝑀 𝑒 ⊂ ℝ 𝑒×𝑒 – Matrix must have full rank “general linear group”

Algebraic Rules: Multiplication Settings Multiplication: Non-Commutative Ring / Group 𝐁 ∈ ℝ 𝑜×𝑛  Prerequisites: 𝐂 ∈ ℝ 𝑛×𝑙  Number of columns right 𝐃 ∈ ℝ 𝑙×𝑚 = number of rows left  Associative: 𝐁 ⋅ 𝐂 ⋅ 𝐃 = 𝐁 ⋅ 𝐂 ⋅ 𝐃  Not commutative: often 𝐁 ⋅ 𝐂 ≠ 𝐂 ⋅ 𝐁  Neutral Op.: 𝐁 ⋅ 𝐉 = 𝐁 𝐁 ⋅ 𝐁 −1 = 𝐉  Inverse: Set of invertible matrices:  Additional prerequisite: – Matrix must be square! 𝐻𝑀 𝑒 ⊂ ℝ 𝑒×𝑒 – Matrix must have full rank “general linear group”

Transposition Rules Transposition 𝐁 + 𝐂 T = 𝐁 T + 𝐂 T = 𝐂 T + 𝐁 T  Addition: 𝜇𝐁 T = 𝜇𝐁 T  Scalar-mult.: 𝐁 ⋅ 𝐂 T = 𝐂 T ⋅ 𝐁 T  Multiplication: 𝐁 T T = 𝐁  Self-inverse: 𝐁 ⋅ 𝐂 −1 = 𝐂 −1 ⋅ 𝐁 −1  (Inversion:) 𝐁 T −1 = 𝐁 −1 T  Inverse-transp.: 𝐁 T = 𝐁 −1 ⇔ 𝐁 is orthogonal  Othogonality:

Matrix Multiplication Matrix Multiplication 𝐁 ⋅ 𝐂 − 𝐛 1 − | | ⋮ 𝐜 1 ⋯ 𝐜 𝑒 = ⋅ − 𝐛 𝑒 − | | ⋱ ⋰ = 𝐛 𝑗 , 𝐜 𝑘 ⋰ ⋱  Scalar products of rows and columns

Orthogonal Matrices Othogonal Matrices  (i.e., column vectors ortho normal ) 𝐍 𝑈 = 𝐍 −1  Proof: previous slide.