Matrices A brief introduction Basilio Bona DAUIN Politecnico di - - PowerPoint PPT Presentation

Matrices A brief introduction

Basilio Bona

DAUIN – Politecnico di Torino

Semester 1, 2016-17

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 1 / 41

Definitions

Definition

A matrix is a set of N real or complex numbers organized in m rows and n columns, with N = mn A =    a11 a12 ··· a1n a21 a22 ··· a2n ··· ··· aij ··· am1 am2 ··· amn    ≡ aij

• i = 1,...,m

j = 1,...,n A matrix is always written as a boldface capital letter, e.g., A. To indicate matrix dimensions we use the following symbols Am×n Am×n A ∈ Fm×n A ∈ Fm×n where F = R for real elements and F = C for complex elements.

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 2 / 41

Transpose matrix

Given a matrix Am×n the transpose matrix is the matrix obtained exchanging rows and columns AT

n×m =

    a11 a21 ··· am1 a12 a22 ··· am2 . . . . . . ... . . . a1n a2n ··· amn     The following property holds (AT)T = A

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 3 / 41

Square matrix

A matrix is said to be square when m = n A square n ×n matrix is upper triangular when aij = 0, ∀i > j An×n =     a11 a12 ··· a1n a22 ··· a2n . . . . . . ... . . . ··· ann     If a square matrix is upper triangular its transpose is lower triangular and viceversa AT

n×n =

    a11 ··· a12 a22 ··· . . . . . . ... . . . a1n a2n ··· ann    

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 4 / 41

Symmetric matrix

A real square matrix is said to be symmetric if A = AT, or A−AT = O In a real symmetric matrix there are at least n(n +1) 2 independent elements. If a matrix K has complex elements kij = aij +jbij (where j = √−1) its conjugate is K with elements kij = aij −jbij. Given a complex matrix K, its adjoint matrix K∗ is the conjugate transpose K∗ = K

T = KT

A complex matrix is called self-adjoint or hermitian when K = K∗. Some textbooks indicate this matrix as K† or KH

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 5 / 41

Diagonal matrix

A square matrix is diagonal if aij = 0 for i = j An×n = diag(ai) =     a1 ··· a2 ··· . . . . . . ... . . . ··· an     A diagonal matrix is always symmetric.

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 6 / 41

Matrix Algebra

Matrices form an algebra, i.e., a vector space endowed with the product

• perator. The main operations are: product by a scalar, sum, matrix

product

Product by a scalar c

cA = c     a11 a12 ··· a1n a21 a22 ··· a2n . . . . . . ... . . . am1 am2 ··· amn     =     ca11 ca12 ··· ca1n ca21 ca22 ··· ca2n . . . . . . ... . . . cam1 cam2 ··· camn    

Sum

A+B =     a11 +b11 a12 +b12 ··· a1n +b1n a21 +b21 a22 +b22 ··· a2n +b2n . . . . . . ... . . . am1 +bm1 am2 +bm2 ··· amn +bmn    

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 7 / 41

Sum

Properties

A+O = A A+B = B+A (A+B)+C = A+(B+C) (A+B)T = AT +BT The neutral element O is called null or zero matrix. The matrix difference is defined introducing the scalar α = −1: A−B = A+(−1)B.

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 8 / 41

Matrix Product

Matrix product

The operation follows the rule “row by column”: the generic cij element

• f the product matrix Cm×p = Am×n ·Bn×p is

cij =

n

∑

k=1

aikbkj The following identity holds: α(A·B) = (αA)·B = A·(αB)

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 9 / 41

Product

Properties

A·B·C = (A·B)·C = A·(B·C) A·(B+C) = A·B+A·C (A+B)·C = A·C+B·C (A·B)T = BT ·AT In general: the matrix product is NOT commutative: A·B = B·A, except some particular case; A·B = A·C does not imply B = C, except some particular case; A·B = O does not imply A = O or B = O, except some particular case.

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 10 / 41

Identity Matrix

The neutral element wrt the matrix product is called identity matrix usually written as In or I when there are no ambiguities on the dimension.

Identity matrix

I =     1 ··· ··· ··· . . . . . . ... . . . ··· 1     Given a rectangular matrix Am×n the following relations hold Am×n = ImAm×n = Am×nIn

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 11 / 41

Matrix Power

Given a square matric A ∈ Rn×n, the k-th power is Ak =

k

∏

ℓ=1

A One matrix is said to be idempotent iff A2 = A → Ak = A.

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 12 / 41

Matrix Trace

Trace

The trace of a square matrix An×n is the sum of its diagonal elements tr (A) =

n

∑

k=1

akk Trace satisfy the following properties tr (aA+bB) = a tr (A)+b tr (B) tr (AB) = tr (BA) tr (A) = tr (AT) tr (A) = tr (T−1AT) for T non-singular (see below for explanation)

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 13 / 41

Row/column cancellation

Given the square matrix A ∈ Rn×n, we call A(ij) ∈ R(n−1)×(n−1) the matrix

• btained deleting the la i-the row and the j-the columns of A.

Example: given A =     1 −5 3 2

• 6

4 9

• 7

7 −4

• 8

2 −9

• 2

−3     deleting row 2, column 3 we obtain A(23) =   1 −5 2 7 −4 2 −9 −3  

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 14 / 41

Minors and Determinant

A minor of order p of a generic matrix Am×n is defined as the determinant Dp of a square sub-matrix obtained selecting any p rows and p columns of Am×n There exist as many minors as the possible choices of p on m rows and p

• n n columns

The definition of determinant will be given soon. Given a matrix Am×n its principal minors of order k are the determinants Dk, with k = 1,··· ,min{m,n}, obtained selecting the first k rows and k columns of Am×n.

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 15 / 41

Example

Given the 4×3 matrix A =    1 −3 5 7 2 4 −1 3 2 8 −1 6    we compute a generic minor D2, for example that obtained selecting the first and rows 1 and 3 and columns 1 and 2 (in red). First we form the submatrix D =

• 1

−3 −1 3

• and then we compute the determinant

D2 = det(D) = 3×1−(−3×−1) = 0

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 16 / 41

Example

Given the 4×3 matrix A =    1 −3 5 7 2 4 −1 3 2 8 −1 6    we compute the principal minors minors Dk,k = 1,2,3, D1 = 1 D2 = det

• 1

−3 7 2

• = 23

D3 = det   1 −3 5 7 2 4 −1 3 2   = 161

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 17 / 41

Complement

We call the complement Crc of a generic (r,c) element of a square matrix An×n the determinant of the matrix obtained deleting its r-the row and the c-th column, i.e., detA(rc) Drc = detA(rc). The cofactor of the (r,c) element of a square matrix An×n is the signed product Crc = (−1)r+cDrc

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 18 / 41

Example

Given the 3×3 matrix A =   1 2 3 4 5 6 7 8 9   some of the cofactors are C11 = (−1)2(45−48) = −3 C12 = (−1)3(36−42) = 6 C31 = (−1)4(12−15) = −3

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 19 / 41

Adjugate/Adjunct/Adjoint

The cofactor matrix of A is the n ×n matrix C whose (i,j) entry Cij is the (i,j) cofactor of A Cij = (−1)i+jDij The adjugate or adjunct or adjoint of a square matrix A is the transpose

• f C, that is, the n ×n matrix whose (i,j) entry is the (j,i) cofactor of A,

Aadj

ij

= Cji = (−1)i+jDji The adjoint matrix of A is the matrix X that satisfies the following equality AX = XA = det(A)I

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 20 / 41

Example

Given the 3×3 matrix A =   1 3 2 4 6 5 7 9 8   its adjoint is Aadj =   3 −6 3 3 −6 3 −6 12 −6  

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 21 / 41

Determinant

The determinant of a square matrix Ax×n can be computed in different ways. Choosing any row i, the definition “by row” is: det(A) =

n

∑

k=1

aik(−1)i+k det(A(ik)) =

n

∑

k=1

aikAik Choosing any column j, the definition “by column” is:: det(A) =

n

∑

k=1

akj(−1)k+j det(A(kj)) =

n

∑

k=1

akjAkj Since these definitions are recursive, involving the determinants of increasingly smaller minors, we define the determinant of a 1×1 matrix A = a, simply as detA = a.

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 22 / 41

Properties

The determinant has the following properties: det(A·B) = det(A)det(B) det(AT) = det(A) det(kA) = kn det(A) if one exchanges s rows or columns of A, obtaining As, then det(As) = (−1)s det(A) if A has two or more rows/columns equal or proportional, then det(A) = 0 if A has a row/column that can be obtained as a linear combination

• f other rows/columns, then det(A) = 0

if A is triangular, then det(A) = ∏n

i=1 aii

if A is block-triangular, with p blocks Aii on the diagonal, then det(A) = ∏p

i=1 detAii

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 23 / 41

Rank and Singularity

A matrix A is singular if det(A) = 0. The rank (or characteristic) of a matrix Am×n is the number ρ(Am×n), defined as the largest integer p for which at least a minor Dp is non-zero. The following properties hold: ρ(A) ≤ min{m,n} if ρ(A) = min{m,n}, A is said to be full rank if ρ(A) < min{m,n}, the rank of the matrix is said to drop if An×n and detA < n the matrix is not full rank, or is rank deficient ρ(A·B) ≤ min{ρ(A),ρ(B)} ρ(A) = ρ(AT) ρ(A·AT) = ρ(AT ·A) = ρ(A)

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 24 / 41

Invertible Matrix

A square matrix An×n it is said to be invertible or non singular if the inverse A−1

n×n exists, such that

An×nA−1

n×n = A−1 n×nAn×n = In

A square matrix A is invertible iff ρ(A) = n, i.e., it is full-rank; this is equivalent to have a non zero determinant det(A) = 0. The inverse is computed as: A−1 = 1 det(A)Aadj The following properties hold: (A−1)−1 = A; (AT)−1 = (A−1)T. The inverse, if exists, allows to solve the following matrix equation y = Ax with respect to the unknown x, as x = A−1y.

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 25 / 41

Matrix derivative

If a square matrix An×n(t) has elements function of a variable (e.g., the time t) aij(t), then the matrix derivative is d dt A(t) = ˙ A(t) = d dt aij(t)

• = [ ˙

aij(t)] If A(t) rank is full, ρ(A(t)) = n for every t, then the derivative of the inverse is d dt A(t)−1 = −A−1(t) ˙ A(t)A(t)−1 Notice that, since the inverse is a nonlinear function, the derivative of the inverse is in general different from the inverse of the derivative. dA(t) dt −1 = d dt

• A(t)−1
• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 26 / 41

Example

Given the square matrix A(t) =

• cosθ(t)

−sinθ(t) sinθ(t) cosθ(t)

• we have

d dt A(t) = ˙ A(t) =

• −sinθ(t)

−cosθ(t) cosθ(t) −sinθ(t)

• ˙

θ(t) The inverse of A is A(t)−1 =

• cosθ(t)

sinθ(t) −sinθ(t) cosθ(t)

• = A(t)T

and in this particular case the two inverses are equal dA(t) dt −1 = d dt

• A(t)−1

=

• cosθ(t)

sinθ(t) −sinθ(t) cosθ(t)

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 27 / 41

Matrix Decomposition

Given a real matrix A ∈ Rm×n, the following products give symmetric matrices ATA ∈ Rn×n AAT ∈ Rm×m Given a square matrix A, it is always possible to decompose it in a sum of two matrices A = As +Ass where As = 1 2(A+AT) is symmetric, and Ass = 1 2(A−AT) is skew-symmetric.

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 28 / 41

Similarity Transformation

Similarity transformation

Given a square matrix A ∈ Rn×n and a square nonsingular matrix T ∈ Rn×n, the matrix B ∈ Rn×n obtained as B = T−1AT

• r

B = TAT−1 is called similar to A, and the transformation T is called similarity transformation.

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 29 / 41

Eigenvalues and Eigenvectors

If it is possible to find a nonsingular matrix U such that A is similar to the diagonal matrix Λ = diag(λi) A = UΛU−1 → AU = UΛ and if we call ui the i-th column of U, U = u1 u2 ··· un

• we have

Aui = λiui This relation is the well known formula defining eigenvectors and eigenvalues of A. The λi are the eigenvalues of A and the ui are the eigenvectors of A.

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 30 / 41

Eigenvalues and Eigenvectors

Eigenvalues and eigenvectors

Given a square matrix An×n, the matrix eigenvalues λi are the (real or complex) solutions of the characteristic equation det(λI−A) = 0 det(λI−A) is a polynomial in λ, called the characteristic polynomial of A. If the eigenvalues are all distinct, we call eigenvectors the vectors ui that satisfy the following identity Aui = λiui If the eigenvalues are not all distinct, we obtain the generalized eigenvectors, whose characterization goes beyond the scope of this presentation.

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 31 / 41

Geometrical interpretation

From a geometrical point of view, the eigenvectors represent those particular “directions” in the Rn space (i.e., the domain of the linear transformation represented by A), that remain invariant under the transformation, while the eigenvalues give the scaling constants along these same directions. The set of the matrix eigenvalues is usually indicated as Λ(A) or {λi(A)}; the set of the matrix eigenvectors is indicated as {ui(A)}. In general, they are normalized, i.e., {ui(A)} = 1

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 32 / 41

Eigenvalues Properties

Given a square matrix A and its eigenvalues, {λi(A)}, the following properties hold true {λi(A+cI)} = {(λi(A)+c)} {λi(cA)} = {(cλi(A)} Given a triangular matrix     a11 a12 ··· a1n a22 ··· a2n . . . . . . ... . . . ··· ann    ,     a11 ··· a21 a22 ··· . . . . . . ... . . . an1 an2 ··· ann     its eigenvalues are the elements on the main diagonal {λi(A)} = {aii}; the same is true for a diagonal matrix.

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 33 / 41

Other properties

Given a square matrix An×n and its eigenvalues {λi(A)}, the following hold true det(A) =

n

∏

i=1

λi and tr (A) =

n

∑

i=1

λi So, the determinant is the product of the eigenvalues, and the trace is the sum of the eigenvalues. Given any invertible similarity transformation T, B = T−1AT the eigenvalues of A are invariant to it, i.e., {λi(B)} = {λi(A)}

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 34 / 41

Modal matrix

If we build a matrix M whose columns are the normalized eigenvectors ui(A) M = u1 ··· un

• then the similarity transformation with respect to M results in the diagonal

matrix Λ =     λ1 ··· λ2 ··· . . . . . . ... . . . ··· λn     = M−1AM M is the modal matrix. If A is symmetric, all its eigenvalues are real and we have Λ = MTAM In this case M is orthonormal.

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 35 / 41

Singular Value decomposition (SVD)

Given a matrix A ∈ Rm×n, having rank r = ρ(A) ≤ s, with s = min{m,n}, it can be decomposed (factored) in the following way: A = UΣVT =

s

∑

i=1

σiuivT

i

(1) The decomposition is characterized by three elements: σi ui vi as follows.

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 36 / 41

SVD Characterization

σi(A) ≥ 0 are the singular values and are equal to the non-negative square roots of the eigenvalues of the symmetric matrix ATA: {σi(A)} = {

• λi(ATA)}

σi ≥ 0

• rdered in decreasing order

σ1 ≥ σ2 ≥ ··· ≥ σs ≥ 0 if r < s there are r positive singular values; the remaining ones are zero σ1 ≥ σ2 ≥ ··· ≥ σr > 0; σr+1 = ··· = σs = 0 U is a orthonormal (m ×m) matrix U = u1 u2 ··· um

• containing the eigenvectors ui of AAT
• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 37 / 41

SVD Characterization

V is a (n ×n) orthonormal matrix V = v1 v2 ··· vn

• whose columns are the eigenvectors vi of the matrix ATA

Σ is a (m ×n) matrix, with the following structure if m < n Σ = Σs O if m = n Σ = Σs if m > n Σ =

• Σs

O

• .

where Σs = diag(σi) is diagonal with dimension s ×s, having the singular values on the diagonal.

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 38 / 41

Example

Given A =

• 1

3 2 4 6 5

• ρ(A) = 2

its SVD is A = UΣVT where U =

• −0.3863

−0.9224 −0.9224 0.3863

• Σ =
• 9.5080

0.7729

• V =

  −0.4287 0.8060 −0.4082 −0.7039 −0.5812 −0.4082 −0.5663 0.1124 0.8165  

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 39 / 41

Alternative SVD Decomposition

Alternately, we can decompose the A matrix as follows: A =

• P

¯ P

• U
• Σr

O O O

• Σ
• QT

¯ QT

• VT

= PΣrQT (2) where P is an orthonormal m ×r matrix, ¯ P is an orthonormal m ×(m −r) matrix; Q is an orthonormal n ×r matrix, ¯ QT is an orthonormal n ×(n −r) matrix; Σr is an diagonal r ×r matrix, whose diagonal elements are the positive singular values σi > 0, i = 1,··· ,r.

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 40 / 41

Rank

The rank r = ρ(A) of A is equal to the number r ≤ s of nonzero singular values. Given any matrix A ∈ Rm×n, both ATA and AAT are symmetric, with identical positive singular values and differ only for the number of zero singular values.

• B. Bona (DAUIN)

Matrices Semester 1, 2016-17 41 / 41