Covariance Matrices and Covariance Operators in Machine Learning and - - PowerPoint PPT Presentation

▶
covariance matrices and covariance operators in machine
SMART_READER_LITE
LIVE PREVIEW

Covariance Matrices and Covariance Operators in Machine Learning and - - PowerPoint PPT Presentation

May 20, 2023 671 likes •1.72k views

Covariance Matrices and Covariance Operators in Machine Learning and Pattern Recognition A geometrical framework H` a Quang Minh Pattern Analysis and Computer Vision (PAVIS) Istituto Italiano di Tecnologia, ITALY November 13, 2017 H.Q. Minh

slide-1
SLIDE 1

Covariance Matrices and Covariance Operators in Machine Learning and Pattern Recognition

A geometrical framework H` a Quang Minh

Pattern Analysis and Computer Vision (PAVIS) Istituto Italiano di Tecnologia, ITALY

November 13, 2017

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 1 / 103

slide-2
SLIDE 2

Acknowledgment

Collaborators Marco San Biagio Vittorio Murino

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 2 / 103

slide-3
SLIDE 3

Introduction and motivation

Symmetric Positive Definite (SPD) matrices Sym++(n) = set of n × n SPD matrices Have been studied extensively mathematically Numerous practical applications

Brain imaging (Arsigny et al 2005, Dryden et al 2009, Qiu et al 2015) Computer vision: object detection (Tuzel et al 2008, Tosato et al 2013), image retrieval (Cherian et al 2013), visual recognition (Jayasumana et al 2015), many more Radar signal processing: Barbaresco (2013), Formont et al 2013 Machine learning: kernel learning (Kulis et al 2009)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 3 / 103

slide-4
SLIDE 4

Outline

Covariance matrices

Covariance matrix representation in computer vision Geometry of SPD matrices Kernel methods on covariance matrices

Covariance operators

Covariance operator representation in computer vision Geometry of covariance operators Kernel methods on covariance operators

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 4 / 103

slide-5
SLIDE 5

Outline

Covariance matrices Covariance matrix representation in computer vision Geometry of SPD matrices Kernel methods on covariance matrices

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 5 / 103

slide-6
SLIDE 6

Covariance matrix representation of images

Tuzel, Porikli, Meer (ECCV 2006, CVPR 2006): covariance matrices as region descriptors for images (covariance descriptors) Given an image F (or a patch in F), at each pixel, extract a feature vector (e.g. intensity, colors, filter responses etc) Each image corresponds to a data matrix X X = [x1, . . . , xm] = n × m matrix where

m = number of pixels n = number of features at each pixel

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 6 / 103

slide-7
SLIDE 7

Covariance matrix representation of images

X = [x1, . . . , xm] = data matrix of size n × m, with m observations Empirical mean vector µX = 1 m

m

  • i=1

xi = 1 mX1m, 1m = (1, . . . , 1)T ∈ Rm Empirical covariance matrix CX = 1 m

m

  • i=1

(xi − µX)(xi − µX)T = 1 mXJmXT Jm = Im − 1 m1m1T

m =

centering matrix

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 7 / 103

slide-8
SLIDE 8

Covariance matrix representation of images

Image F ⇒ Data matrix X ⇒ Covariance matrix CX Each image is represented by a covariance matrix Example of image features

f(x, y) =

  • I(x, y), R(x, y), G(x, y), B(x, y), |∂R

∂x |, |∂R ∂y |, |∂G ∂x |, |∂G ∂y |, |∂B ∂x |, |∂B ∂y |

  • at pixel location (x, y)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 8 / 103

slide-9
SLIDE 9

Example

Figure: An example of the covariance descriptor. At each pixel (x, y), a

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 9 / 103

slide-10
SLIDE 10

Covariance matrix representation - Properties

Encode linear correlations (second order statistics) between image features Flexible, allowing the fusion of multiple and different features

Handcrafted features, e.g. colors and SIFT Convolutional features

Compact Robust to noise

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 10 / 103

slide-11
SLIDE 11

Covariance matrix representation - generalization

Covariance representation for video: e.g. Guo et al (AVSS 2010), Sanin et al (WACV 2013)

Employ features that capture temporal information, e.g. optical flow

Covariance representation for 3D point clouds and 3D shapes: e.g. Fehr et al (ICRA 2012, ICRA 2014), Tabias et al (CVPR 2014), Hariri et al (Pattern Recognition Letters 2016)

Employ geometric features e.g. curvature, surface normal vectors

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 11 / 103

slide-12
SLIDE 12

Statistical interpretation

Representing an image by a covariance matrix is essentially equivalent to Representing an image by a Gaussian probability density ρ in Rn with mean zero Features extracted are random observations of a n-dimensional random vector with probability density ρ

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 12 / 103

slide-13
SLIDE 13

Statistical interpretation

X = (X1, . . . , Xn) = random vector in Rn with probability density function ρ Mean vector µ = E(X) Covariance matrix C = E[(X − µ)(X − µ)T]

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 13 / 103

slide-14
SLIDE 14

Statistical interpretation

Variance of the random variable Xi Cii = E[(Xi − µi)2] Covariance between the random variables Xi and Xj Cij = cov(Xi, Xj) = E[(Xi − µi)(Xj − µj)] Correlation between Xi and Xj corr(Xi, Xj) =

  • Cij

√

CiiCjj ,

i = j 1, i = j

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 14 / 103

slide-15
SLIDE 15

Statistical interpretation

Multivariate Gaussian probability density ρ = N(µ, C), with covariance matrix C ∈ Sym++(n) ρ(x) = 1

  • (2π)n det(C)

exp

  • −1

2(x − µ)TC−1(x − µ)

  • If µ = 0, then ρ is completely determined by the covariance matrix

C X = [x1, . . . , xm] = IID observations sampled according to ρ (µX, CX) = MLE estimates of (µ, C) Unbiased estimate of C ˜ CX = 1 m − 1XJmXT

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 15 / 103

slide-16
SLIDE 16

Empirical covariance matrices - Regularization

CX is only guaranteed to be positive semi-definite CX can be ill-conditioned For CX to be positive definite/well-conditioned, need to use regularization in general Diagonal loading, widely used, readily generalizable to infinite-dimensional setting CX + γI γ > 0 More generally, shrinkage estimators, Ledoit and Wolf (Journal of Multivariate Analysis, 2004)) ˆ CX = (1 − ρ)CX + ρνI, 0 ≤ ρ ≤ 1, ν > 0

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 16 / 103

slide-17
SLIDE 17

Outline

Covariance matrices Covariance matrix representation in computer vision Geometry of SPD matrices Kernel methods on covariance matrices

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 17 / 103

slide-18
SLIDE 18

Geometry of SPD matrices

Euclidean metric Set of SPD matrices viewed as a Riemannian manifold

Affine-invariant Riemannian metric Log-Euclidean metric

Set of SPD matrices viewed as a convex cone

Log-Determinant divergences (symmetric Stein divergence)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 18 / 103

slide-19
SLIDE 19

Euclidean metric

Frobenius inner product between A = (aij)n

i,j=1, B = (bij)n i,j=1

A, BF = tr(ATB) =

  • i,j=1n

aijbij Frobenius norm of an n × n matrix A = (aij)n

i,j=1

||A||2

F = tr(ATA) = n

  • i,j=1

a2

ij

Euclidean (Frobenius) distance between two n × n matrices A, B dE(A, B) = ||A − B||F

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 19 / 103

slide-20
SLIDE 20

Euclidean metric

In the Euclidean metric, each n × n matrix A is equivalent to its vectorized version vec(A) ∈ Rn2 A, BF = vec(A), vec(B) ||A||F = ||vec(A))|| dE(A, B) = ||A − B||F = ||vec(A) − vec(B)||

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 20 / 103

slide-21
SLIDE 21

Euclidean metric - Properties

Simple to implement and efficient to compute Essentially treats matrices as vectors, without taking into account their inherent structures Not intrinsic to Sym++(n) May lead to swelling effect (mean of a set of SPD matrices might have larger determinant than the component matrices, Arsigny et al 2007) (Sym++(n), dE) is an incomplete metric space: Cauchy sequences, {An}, ||An − Am||F arbitrarily small, may not converge Tend to be suboptimal in practical applications

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 21 / 103

slide-22
SLIDE 22

Euclidean metric - Properties

Unitary (orthogonal) invariance: DDT = I ⇐ ⇒ D−1 = DT dE(DADT, DBDT) = dE(A, B) Corresponds to e.g. rotation invariance of Euclidean distance in Rn X → DX CX = 1 mXJmXT → DCXDT

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 22 / 103

slide-23
SLIDE 23

Geometry of SPD matrices

Euclidean metric Set of SPD matrices viewed as a Riemannian manifold

Affine-invariant Riemannian metric Log-Euclidean metric

Set of SPD matrices viewed as a convex cone

Log-Determinant divergences (symmetric Stein divergence)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 23 / 103

slide-24
SLIDE 24

Affine-invariant Riemannian metric

Has been studied extensively in mathematics Siegel (1943), Mostow (1955), Pennec et al (2006), Bhatia (2007), Moakher and Z´ era¨ ı (2011), Bini and Iannazzo (2013)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 24 / 103

slide-25
SLIDE 25

Affine-invariant Riemannian metric

Riemannian metric: On the tangent space TP(Sym++(n)) ∼ = Sym(n), the inner product , P is V, WP = P−1/2VP−1/2, P−1/2WP−1/2F = tr(P−1VP−1W) P ∈ Sym++(n), V, W ∈ Sym(n)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 25 / 103

slide-26
SLIDE 26

Affine-invariant Riemannian metric

Geodesically complete Riemannian manifold, nonpositive curvature Unique geodesic joining A, B ∈ Sym++(n) γAB(t) = A1/2(A−1/2BA−1/2)tA1/2 γAB(0) = A, γAB(1) = B Riemannian (geodesic) distance daiE(A, B) = || log(A−1/2BA−1/2)||F where log(A) is the principal logarithm of A A = UDUT = Udiag(λ1, . . . , λn)UT log(A) = U log(D)UT = Udiag(log λ1, . . . , log λn)UT

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 26 / 103

slide-27
SLIDE 27

Affine-invariant Riemannian distance - Properties

Affine-invariance daiE(CACT, CBCT) = daiE(A, B), any C invertible Scale invariance: C = √sI, s > 0, daiE(sA, sB) = daiE(A, B) Unitary (orthogonal) invariance: CCT = I ⇐ ⇒ C−1 = CT daiE(CAC−1, CBC−1) = daiE(A, B)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 27 / 103

slide-28
SLIDE 28

Affine-invariant Riemannian distance - Properties

Invariance under inversion daiE(A−1, B−1) = daiE(A, B) (Sym++(n), daiE) is a complete metric space

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 28 / 103

slide-29
SLIDE 29

Connection with Fisher-Rao metric

Close connection with Fisher-Rao metric in information geometry (e.g. Amari 1985, 2016) For two multivariate Gaussian probability densities ρ1 ∼ N(µ, C1), ρ2 ∼ N(µ, C2) daiE(C1, C2) = 2(Fisher-Rao distance between ρ1 and ρ2 )

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 29 / 103

slide-30
SLIDE 30

Affine-invariant Riemannian distance - Complexity

For two matrices A, B ∈ Sym++(n) d2

aiE(A, B) = || log(A−1/2BA−1/2)||2 F = n

  • k=1

(log λk)2 where {λk}n

k=1 are the eigenvalues of

A−1/2BA−1/2

  • r equivalently

A−1B Matrix inversion, SVD, eigenvalue computation all have computational complexity O(n3) Therefore daiE(A, B) has computational complexity O(n3)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 30 / 103

slide-31
SLIDE 31

Affine-invariant Riemannian distance - Complexity

For a set {Ai}N

i=1 of N SPD matrices, consider computing all the

pairwise distances daiE(Ai, Aj) = || log(A−1/2

i

AjA−1/2

i

)||F, 1 ≤ i, j ≤ N The matrices Ai, Aj are all coupled together The computational complexity required is O(N2n3) This is very large when N is large

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 31 / 103

slide-32
SLIDE 32

Log-Euclidean metric

Arsigny, Fillard, Pennec, Ayache (SIAM Journal on Matrix Analysis and Applications 2007) Another Riemannian metric on Sym++(n) Much faster to compute than the affine-invariant Riemannian distance on large sets of matrices Can be used to define many positive definite kernels on Sym++(n)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 32 / 103

slide-33
SLIDE 33

Log-Euclidean metric

Riemannian metric: On the tangent space TP(Sym++(n)) V, WP = D log(P)(V), D log(P)(W)F P ∈ Sym++(n), V, W ∈ Sym(n) where D log is the Fr´ echet derivative of the function log : Sym++(n) → Sym(n) D log(P) : Sym(n) → Sym(n) is a linear map Explicit knowledge of , P is not necessary for computing geodesics and Riemannian distances

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 33 / 103

slide-34
SLIDE 34

Log-Euclidean metric

Unique geodesic joining A, B ∈ Sym++(n) γAB(t) = exp[(1 − t) log(A) + t log(B)] Riemannian (geodesic) distance dlogE(A, B) = || log(A) − log(B)||F

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 34 / 103

slide-35
SLIDE 35

Log-Euclidean distance - Complexity

For two matrices A, B ∈ Sym++(n) dlogE(A, B) = || log(A) − log(B)||F The computation of the log function, requiring an SVD, has computational complexity O(n3) Therefore dlogE(A, B) has computational complexity O(n3)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 35 / 103

slide-36
SLIDE 36

Log-Euclidean distance - Complexity

For a set {Ai}N

i=1 of N SPD matrices, consider computing all the

pairwise distances dlogE(Ai, Aj) = || log(Ai) − log(Aj)||F, 1 ≤ i, j ≤ N The matrices Ai, Aj are all uncoupled The computational complexity required is O(Nn3) This is much faster than the affine-invariant Riemannian distance daiE when N is large

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 36 / 103

slide-37
SLIDE 37

Log-Euclidean vector space

Arsigny et al (2007): Log-Euclidean metric is a bi-invariant Riemannian metric associated with the Lie group operation ⊙ : Sym++(n) × Sym++(n) → Sym++(n) A ⊙ B = exp(log(A) + log(B)) = B ⊙ A Bi-invariance: for any C ∈ Sym++(n) dlogE[(A ⊙ C), (B ⊙ C)] = dlogE[(C ⊙ A), (C ⊙ B)] = dlogE(A, B)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 37 / 103

slide-38
SLIDE 38

Log-Euclidean vector space

Arsigny et al (2007): scalar multiplication operation ⊛ : R × Sym++(n) → Sym++(n) λ ⊛ A = exp(λ log(A)) = Aλ (Sym++(n), ⊙, ⊛) is a vector space, with ⊙ acting as vector addition and ⊛ acting as scalar multiplication Sym++(n) under the Log-Euclidean metric is a Riemannian manifold with zero curvature

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 38 / 103

slide-39
SLIDE 39

Log-Euclidean vector space

Vector space isomorphism log : (Sym++(n), ⊙, ⊛) → (Sym(n), +, ·) A → log(A) The vector space (Sym++(n), ⊙, ⊛) is not a subspace of the Euclidean vector space (Sym(n), +, ·)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 39 / 103

slide-40
SLIDE 40

Log-Euclidean inner product space

Log-Euclidean inner product (Li, Wang, Zuo, Zhang, ICCV 2013) A, BlogE = log(A), log(B)F ||A||logE = || log(A)||F Log-Euclidean inner product space (Sym++(n), ⊙, ⊛, , logE) Log-Euclidean distance dlogE(A, B) = || log(A) − log(B)||F = ||(A ⊙ B−1)||logE

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 40 / 103

slide-41
SLIDE 41

Log-Euclidean vs. Euclidean

Unitary (orthogonal) invariance CCT = I ⇐ ⇒ CT = C−1 Euclidean distance dE(CAC−1, CBC−1) = ||CAC−1 − CBC−1||F = ||A − B||F = dE(A, B) Log-Euclidean distance dlogE(CAC−1, CBC−1) = || log(CAC−1) − log(CBC−1)||F = || log(A) − log(B)||F = dlogE(A, B)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 41 / 103

slide-42
SLIDE 42

Log-Euclidean vs. Euclidean

Log-Euclidean distance is scale-invariant dlogE(sA, sB) = || log(sA) − log(sB)||F = || log(A) − log(B)||F = dlogE(A, B) Euclidean distance is not scale-invariant dE(sA, sB) = s||A − B||F = sdE(A, B)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 42 / 103

slide-43
SLIDE 43

Log-Euclidean vs. Euclidean

Log-Euclidean distance is inversion-invariant dlogE(A−1, B−1) = || log(A−1) − log(B−1)|| = || − log(A) + log(B)||F = dlogE(A, B) Euclidean distance is not inversion-invariant dE(A−1, B−1) = ||A−1 − B−1||F = ||A − B||F = dE(A, B)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 43 / 103

slide-44
SLIDE 44

Log-Euclidean vs. Euclidean

As metric spaces (Sym++(n), dE) is incomplete (Sym++(n), dlogE) is complete

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 44 / 103

slide-45
SLIDE 45

Log-Euclidean vs. Euclidean

Summary of comparison The two metrics are fundamentally different Euclidean metric is extrinsic to Sym++(n) Log-Euclidean metric is intrinsic to Sym++(n) The vector space structures are fundamentally different They have different invariance properties

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 45 / 103

slide-46
SLIDE 46

Geometry of SPD matrices

Euclidean metric Set of SPD matrices viewed as a Riemannian manifold

Affine-invariant Riemannian metric Log-Euclidean metric

Set of SPD matrices viewed as a convex cone

Log-Determinant divergences (symmetric Stein divergence)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 46 / 103

slide-47
SLIDE 47

Alpha Log-Determinant divergences

Chebbi and Moakher (Linear Algebra and Its Applications 2012) Ω = Sym++(n), φ(X) = − log det(X) dα

logdet(A, B) =

4 1 − α2 log det( 1−α

2 A + 1+α 2 B)

det(A)

1−α 2

det(B)

1+α 2

−1 < α < 1 Limiting cases d1

logdet(A, B) = lim α→1 dα logdet(A, B) = tr(B−1A − I) − log det(B−1A)

(Burg divergence) d−1

logdet(A, B) = lim α→−1 dα logdet(A, B) = tr(A−1B − I) − log det(A−1B)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 47 / 103

slide-48
SLIDE 48

Alpha Log-Determinant divergences

α = 0: Symmetric Stein divergence (also called S-divergence) d0

logdet(A, B) = 4

  • log

A + B 2

  • − 1

2 log det(AB)

  • = 4d2

stein(A, B)

Sra (NIPS 2012): dstein(A, B) =

  • log

A + B 2

  • − 1

2 log det(AB) is a metric (satisfying positivity, symmetry, and triangle inequality)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 48 / 103

slide-49
SLIDE 49

Outline

Covariance matrices Covariance matrix representation in computer vision Geometry of SPD matrices Kernel methods on covariance matrices

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 49 / 103

slide-50
SLIDE 50

Positive Definite Kernels

X any nonempty set K : X × X → R is a (real-valued) positive definite kernel if it is symmetric and

N

  • i,j=1

aiajK(xi, xj) ≥ 0 for any finite set of points {xi}N

i=1 ∈ X and real numbers

{ai}N

i=1 ∈ R.

[K(xi, xj)]N

i,j=1 is symmetric positive semi-definite

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 50 / 103

slide-51
SLIDE 51

Reproducing Kernel Hilbert Spaces

K a positive definite kernel on X × X. For each x ∈ X, there is a function Kx : X → R, with Kx(t) = K(x, t). HK = {

N

  • i=1

aiKxi : N ∈ N} with inner product

  • i

aiKxi,

  • j

bjKyjHK =

  • i,j

aibjK(xi, yj) HK = RKHS associated with K (unique).

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 51 / 103

slide-52
SLIDE 52

Reproducing Kernel Hilbert Spaces

Reproducing property: for each f ∈ HK, for every x ∈ X f(x) = f, KxHK Abstract theory due to Aronszajn (1950) Numerous applications in machine learning (kernel methods)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 52 / 103

slide-53
SLIDE 53

Examples: RKHS

Polynomial kernels K(x, y) = (x, y + c)d, c ≥ 0, d ∈ N, x, y ∈ Rn The Gaussian kernel K(x, y) = exp(− |x−y|2

σ2

) on Rn induces the space HK = {||f||2

HK =

1 (2π)n(σ√π)n

  • Rn e

σ2|ξ|2 4

| f(ξ)|2dξ < ∞}.

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 53 / 103

slide-54
SLIDE 54

Kernels with Log-Euclidean metric

Positive definite kernels on Sym++(n) defined with the Log-Euclidean inner product , logE and norm || ||logE Polynomial kernels K(A, B) = (A, BlogE + c)d = (log(A), log(B)F + c)d, d ∈ N, c ≥ 0 Gaussian and Gaussian-like kernels K(A, B) = exp(− 1 σ2 ||(A ⊙ B−1)||p

logE),

0 < p ≤ 2 = exp(− 1 σ2 || log(A) − log(B)||p

F)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 54 / 103

slide-55
SLIDE 55

Kernel methods with Log-Euclidean metric

  • S. Jayasumana, R. Hartley, M. Salzmann, H. Li, and M. Harandi.

Kernel methods on the Riemannian manifold of symmetric positive definite matrices. CVPR 2013.

  • S. Jayasumana, R. Hartley, M. Salzmann, H. Li, and M. Harandi.

Kernel methods on Riemannian manifolds with Gaussian RBF kernels, PAMI 2015. P . Li, Q. Wang, W. Zuo, and L. Zhang. Log-Euclidean kernels for sparse representation and dictionary learning, ICCV 2013

  • D. Tosato, M. Spera, M. Cristani, and V. Murino. Characterizing

humans on Riemannian manifolds, PAMI 2013

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 55 / 103

slide-56
SLIDE 56

Kernel methods with Log-Euclidean metric for image classification

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 56 / 103

slide-57
SLIDE 57

Material classification

Example: KTH-TIPS2b data set f(x, y) =

  • R(x, y), G(x, y), B(x, y),
  • G0,0(x, y)
  • , . . .
  • G3,4(x, y)
  • H.Q. Minh (IIT)

Covariance matrices & covariance operators November 13, 2017 57 / 103

slide-58
SLIDE 58

Object recognition

Example: ETH-80 data set f(x, y) = [x, y, I(x, y), |Ix|, |Iy|]

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 58 / 103

slide-59
SLIDE 59

Numerical results

Better results with covariance operators (Part II)! Method KTH-TIPS2b ETH-80 E 55.3% 64.4% (±7.6%) (±0.9%) Stein 73.1% 67.5% (±8.0%) (±0.4%) Log-E 74.1 % 71.1% (±7.4%) (±1.0%)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 59 / 103

slide-60
SLIDE 60

Comparison of metrics

Results from Cherian et al (PAMI 2013) using Nearest Neighbor Method Texture Activity Affine-invariant 85.5% 99.5% Stein 85.5% 99.5% Log-E 82.0% 96.5% Texture: images from Brodatz and CURET datasets Activity: videos from Weizmann, KTH, and UT Tower datasets

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 60 / 103

slide-61
SLIDE 61

Outline

Covariance operators Covariance operator representation in computer vision Geometry of covariance operators Kernel methods on covariance operators

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 61 / 103

slide-62
SLIDE 62

Covariance operator representation - Motivation

Covariance matrices encode linear correlations of input features Nonlinearization

1

Map original input features into a high (generally infinite) dimensional feature space (via kernels)

2

Covariance operators: covariance matrices of infinite-dimensional features

3

Encode nonlinear correlations of input features

4

Provide a richer, more expressive representation of the data

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 62 / 103

slide-63
SLIDE 63

Covariance operator representation

S.K. Zhou and R. Chellappa. From sample similarity to ensemble similarity: Probabilistic distance measures in reproducing kernel Hilbert space, PAMI 2006

  • M. Harandi, M. Salzmann, and F

. Porikli. Bregman divergences for infinite-dimensional covariance matrices, CVPR 2014 H.Q.Minh, M. San Biagio, V. Murino. Log-Hilbert-Schmidt metric between positive definite operators on Hilbert spaces, NIPS 2014 H.Q.Minh, M. San Biagio, L. Bazzani, V. Murino. Approximate Log-Hilbert-Schmidt distances between covariance operators for image classification, CVPR 2016

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 63 / 103

slide-64
SLIDE 64

Positive definite kernels, feature map, and feature space

K = positive definite kernels on X × X HK = corresponding RKHS Geometric viewpoint from machine learning Positive definite kernel K on X × X induces feature map Φ : X → HK Φ(x) = Kx ∈ HK, HK = feature space Φ(x), Φ(y)HK = Kx, KyHK = K(x, y) Kernelization: Transform linear algorithm depending on x, yRn into nonlinear algorithms depending on K(x, y)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 64 / 103

slide-65
SLIDE 65

RKHS covariance operators

ρ = Borel probability distribution on X, with

  • X

||Φ(x)||2

HK dρ(x) =

  • X

K(x, x)dρ(x) < ∞ RKHS mean vector µΦ = Eρ[Φ(x)] =

  • X

Φ(x)dρ(x) ∈ HK

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 65 / 103

slide-66
SLIDE 66

RKHS covariance operators

RKHS covariance operator CΦ : HK → HK CΦ = Eρ[(Φ(x) − µ) ⊗ (Φ(x) − µ)] =

  • X

Φ(x) ⊗ Φ(x)dρ(x) − µ ⊗ µ

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 66 / 103

slide-67
SLIDE 67

Empirical mean and covariance

X = [x1, . . . , xm] = data matrix randomly sampled from X according to ρ, with m observations Informally, Φ gives an infinite feature matrix in the feature space HK, of size dim(HK) × m Φ(X) = [Φ(x1), . . . , Φ(xm)] Formally, Φ(X) : Rm → HK is the bounded linear operator Φ(X)w =

m

  • i=1

wiΦ(xi), w ∈ Rm

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 67 / 103

slide-68
SLIDE 68

Empirical mean and covariance

Theoretical RKHS mean µΦ =

  • X

Φ(x)dρ(x) ∈ HK Empirical RKHS mean µΦ(X) = 1 m

m

  • i=1

Φ(xi) = 1 mΦ(X)1m ∈ HK

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 68 / 103

slide-69
SLIDE 69

Empirical mean and covariance

Theoretical covariance operator CΦ : HK → HK CΦ =

  • X

Φ(x) ⊗ Φ(x)dρ(x) − µ ⊗ µ Empirical covariance operator CΦ(x) : HK → HK CΦ(X) = 1 m

m

  • i=1

Φ(xi) ⊗ Φ(xi) − µΦ(X) ⊗ µΦ(X) = 1 mΦ(X)JmΦ(X)∗ Jm = Im − 1

m1m1T m = centering matrix

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 69 / 103

slide-70
SLIDE 70

Covariance operator representation of images

Given an image F (or a patch in F), at each pixel, extract a feature vector (e.g. intensity, colors, filter responses etc) Each image corresponds to a data matrix X X = [x1, . . . , xm] = n × m matrix where m = number of pixels, n = number of features at each pixel Define a kernel K, with corresponding feature map Φ and feature matrix Φ(X) = [Φ(x1), . . . , Φ(xm)]

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 70 / 103

slide-71
SLIDE 71

Covariance operator representation of images

Each image is represented by covariance operator CΦ(X) = 1 mΦ(X)JmΦ(X)∗ This representation is implicit, since Φ is generally implicit Computations are carried out via Gram matrices

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 71 / 103

slide-72
SLIDE 72

Infinite-dimensional generalization of Sym++(n)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 72 / 103

slide-73
SLIDE 73

Outline

Covariance operators Covariance operator representation in computer vision Geometry of covariance operators Kernel methods on covariance operators

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 73 / 103

slide-74
SLIDE 74

Affine-invariant Riemannian metric

Affine-invariant Riemannian metric: Larotonda (2005), Larotonda (2007), Andruchow and Varela (2007), Lawson and Lim (2013) Larotonda, Nonpositive curvature: A geometrical approach to Hilbert-Schmidt operators, Differential Geometry and Its Applications, 2007 In the setting of RKHS covariance operators H.Q.M. Affine-invariant Riemannian distance between infinite-dimensional covariance operators, Geometric Science of Information, 2015

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 74 / 103

slide-75
SLIDE 75

Log-Determinant divergences

Zhou and Chellappa (PAMI 2006), Harandi et al (CVPR 214): finite-dimensional RKHS H.Q.M. Infinite-dimensional Log-Determinant divergences between positive definite trace class operators, Linear Algebra and its Applications, 2017 H.Q.M. Log-Determinant divergences between positive definite Hilbert-Schmidt operators, Geometric Science of Information, 2017

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 75 / 103

slide-76
SLIDE 76

Log Hilbert-Schmidt metric

H.Q.Minh, M. San Biagio, V. Murino. Log-Hilbert-Schmidt metric between positive definite operators on Hilbert spaces, NIPS 2014 H.Q.Minh, M. San Biagio, L. Bazzani, V. Murino. Approximate Log-Hilbert-Schmidt distances between covariance operators for image classification, CVPR 2016

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 76 / 103

slide-77
SLIDE 77

Distances between positive definite operators

Larotonda (2007): generalization of the manifold Sym++(n) of SPD matrices to the infinite-dimensional Hilbert manifold

Σ(H) = {A + γI > 0 : A∗ = A, A ∈ HS(H), γ ∈ R}

Hilbert-Schmidt operators on the Hilbert space H

HS(H) = {A : ||A||2

HS = tr(A∗A) = ∞

  • k=1

||Aek||2 < ∞}

for any orthonormal basis {ek}∞

k=1

Hilbert-Schmidt inner product (generalizing Frobenius inner product A, BF = tr(ATB))

A, BHS = tr(A∗B) =

∞

  • k=1

ek, A∗Bek =

∞

  • k=1

Aek, Bek

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 77 / 103

slide-78
SLIDE 78

Distances between positive definite operators

On the infinite-dimensional manifold Σ(H) Larotonda (2007): Infinite-dimensional affine-invariant Riemannian distance H.Q. Minh et al (2014): Log-Hilbert-Schmidt distance, infinite-dimensional generalization of Log-Euclidean distance H.Q. Minh (2017): Infinite-dimensional Log-Determinant divergences

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 78 / 103

slide-79
SLIDE 79

Log-Hilbert-Schmidt distance

Generalizing Log-Euclidean distance dlogE(A, B) = || log(A) − log(B)|| Log-Hilbert-Schmidt distance dlogHS[(A + γI), (B + νI)] = || log(A + γI) − log(B + νI)||eHS Extended Hilbert-Schmidt norm ||A + γI||2

eHS = ||A||2 HS + γ2

Extended Hilbert-Schmidt inner product A + γI, B + νI = A, BHS + γν

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 79 / 103

slide-80
SLIDE 80

Log-Hilbert-Schmidt distance

Why log(A + γI)? Why extended Hilbert-Schmidt norm? A ∈ Sym++(n), with eigenvalues {λk}n

k=1 and orthonormal

eigenvectors {uk}n

k=1

A =

n

  • k=1

λkukuT

k ,

log(A) =

n

  • k=1

log(λk)ukuT

k

A : H → H self-adjoint, positive, compact operator, with eigenvalues {λk}∞

k=1, λk > 0, limk→∞ λk = 0, and orthonormal

eigenvectors {uk}∞

k=1

A =

∞

  • k=1

λk(uk ⊗ uk), (uk ⊗ uk)w = uk, wuk log(A) =

∞

  • k=1

log(λk)(uk ⊗ uk), lim

k→∞ log(λk) = −∞

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 80 / 103

slide-81
SLIDE 81

Log-Hilbert-Schmidt distance

Why log(A + γI)? Why extended Hilbert-Schmidt norm? log(A) is unbounded log(A + γI) is bounded Hilbert-Schmidt norm || log(A + γI)||2

HS = ∞

  • j=1

[log(λk + γ)]2 = ∞ if γ = 1 The extended Hilbert-Schmidt norm || log(A + γI)||2

eHS = || log(A

γ + I)||2

HS + (log γ)2

=

∞

  • j=1

[log(λk γ + 1)]2 + (log γ)2 < ∞

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 81 / 103

slide-82
SLIDE 82

Log-Hilbert-Schmidt metric

Generalization from Sym++(n) to Σ(H) ⊙ : Σ(H) × Σ(H) → Σ(H) (A + γI) ⊙ (B + νI) = exp[log(A + γI) + log(B + νI)] ⊛ : R × Σ(H) → Σ(H) λ ⊛ (A + γI) = exp[λ log(A + γI)] = (A + γI)λ, λ ∈ R (Σ(H), ⊙, ⊛) is a vector space

⊙ acting as vector addition ⊛ acting as scalar multiplication

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 82 / 103

slide-83
SLIDE 83

Log-Hilbert-Schmidt metric

(Σ(H), ⊙, ⊛) is a vector space Log-Hilbert-Schmidt inner product A + γI, B + νIlogHS = log(A + γI), log(B + νI)eHS ||A + γI||logHS = || log(A + γI)||eHS (Σ(H), ⊙, ⊛, , logHS) is a Hilbert space Log-Hilbert-Schmidt distance is the Hilbert distance dlogHS(A + γI, B + νI) = || log(A + γI) − log(B + νI)||eHS = ||(A + γI) ⊙ (B + νI)−1||logHS

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 83 / 103

slide-84
SLIDE 84

Log-Hilbert-Schmidt distance between RKHS covariance operators

The distance dlogHS[(CΦ(X) + γIHK ), (CΦ(Y) + νIHK )] = dlogHS 1 mΦ(X)JmΦ(X)∗ + γIHK

  • ,

1 mΦ(Y)JmΦ(Y)∗ + νIHK

  • has a closed form in terms of m × m Gram matrices

K[X] = Φ(X)∗Φ(X), (K[X])ij = K(xi, xj), K[Y] = Φ(Y)∗Φ(Y), (K[Y])ij = K(yi, yj), K[X, Y] = Φ(X)∗Φ(Y), (K[X, Y])ij = K(xi, yj) K[Y, X] = Φ(Y)∗Φ(X), (K[Y, x])ij = K(yi, xj)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 84 / 103

slide-85
SLIDE 85

Log-Hilbert-Schmidt distance between RKHS covariance operators

1 γmJmK[X]Jm = UAΣAUT

A ,

1 µmJmK[Y]Jm = UBΣBUT

B ,

A∗B = 1 √γµmJmK[X, Y]Jm

CAB = 1T

NA log(INA + ΣA)Σ−1 A (UT A A∗BUB ◦ UT A A∗BUB)Σ−1 B log(INB + ΣB)1NB

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 85 / 103

slide-86
SLIDE 86

Log-Hilbert-Schmidt distance between RKHS covariance operators

Theorem (H.Q.M. et al - NIPS2014)

Assume that dim(HK) = ∞. Let γ > 0, ν > 0. The Log-Hilbert-Schmidt distance between (CΦ(X) + γIHK ) and (CΦ(Y) + νIHK ) is

d2

logHS[(CΦ(X) + γIHK ), (CΦ(Y) + νIHK )] = tr[log(INA + ΣA)]2 + tr[log(INB + ΣB)]2

− 2CAB + (log γ − log ν)2

The Log-Hilbert-Schmidt inner product between (CΦ(X) + γIHK ) and (CΦ(Y) + νIHK ) is (CΦ(X) + γIHK ), (CΦ(Y) + νIHK )logHS = CAB + (log γ)(log ν)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 86 / 103

slide-87
SLIDE 87

Log-Hilbert-Schmidt distance between RKHS covariance operators

Theorem (H.Q.M. et al - NIPS2014)

Assume that dim(HK) = ∞. Let γ > 0. The Log-Hilbert-Schmidt norm of the operator (CΦ(X) + γIHK ) is ||(CΦ(X) + γIHK )||2

logHS = tr[log(INA + ΣA)]2 + (log γ)2

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 87 / 103

slide-88
SLIDE 88

Log-Hilbert-Schmidt distance between RKHS covariance operators

Theorem (H.Q.M. et al - NIPS2014)

Assume that dim(HK) < ∞. Let γ > 0, ν > 0. The Log-Hilbert-Schmidt distance between (CΦ(X) + γIHK ) and (CΦ(Y) + νIHK ) is d2

logHS[(CΦ(X) + γIHK ), (CΦ(Y) + νIHK )]

= tr[log(INA + ΣA)]2 + tr[log(INB + ΣB)]2 − 2CAB + 2(log γ ν )(tr[log(INA + ΣA)] − tr[log(INB + ΣB)]) + (log γ − log ν)2dim(HK)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 88 / 103

slide-89
SLIDE 89

Log-Hilbert-Schmidt distance between RKHS covariance operators

Theorem (H.Q.M. et al - NIPS2014)

Assume that dim(HK) < ∞. Let γ > 0, ν > 0. The Log-Hilbert-Schmidt inner product between (CΦ(X) + γIHK ) and (CΦ(Y) + νIHK ) is (CΦ(X) + γIHK ), (CΦ(Y) + νIHK )logHS = CAB + (log ν)tr[log(INA + ΣA)] + (log γ)tr[log(INB + ΣB)] + (log γ log ν)dim(HK) The Log-Hilbert-Schmidt norm of (CΦ(X) + γIHK ) is ||(CΦ(X) + γIHK )||2

logHS = tr[log(INA + ΣA)]2 + 2(log γ)tr[log(INA + ΣA)]

+ (log γ)2dim(HK)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 89 / 103

slide-90
SLIDE 90

Log-Hilbert-Schmidt distance between RKHS covariance operators

Special case For linear kernel K(x, y) = x, y, x, y ∈ Rn

dlogHS[(CΦ(X) + γIHK ), (CΦ(Y) + νIHK )] = dlogE[(CX + γIn), (CY + νIn)] (CΦ(X) + γIHK ), (CΦ(Y) + νIHK )logHS = (CX + γIn), (CY + νIn)logE ||(CX + γIHK )||logHS = ||(CX + γIn)||logE

These can be used to verify the correctness of an implementation

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 90 / 103

slide-91
SLIDE 91

Log-Hilbert-Schmidt distance between RKHS covariance operators

For m ∈ N fixed, γ = ν, lim

dim(HK )→∞ dlogHS[(CΦ(X) + γIHK ), (CΦ(Y) + νIHK )] = ∞

In general, the infinite-dimensional formulation cannot be approximated by the finite-dimensional counterpart.

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 91 / 103

slide-92
SLIDE 92

Outline

Covariance operators Covariance operator representation in computer vision Geometry of covariance operators Kernel methods on covariance operators

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 92 / 103

slide-93
SLIDE 93

Kernels with Log-Hilbert-Schmidt metric

(Σ(H), ⊙, ⊛, , logHS) is a Hilbert space

Theorem (H.Q.M. et al - NIPS 2014)

The following kernels K : Σ(H) × Σ(H) → R are positive definite K[(A + γI), (B + νI)] = (c + A + γI, B + νIlogHS)d c ≥ 0, d ∈ N K[(A + γI), (B + νI)] = exp(− 1 σ2 || log(A + γI) − log(B + νI)||p

eHS)

0 < p ≤ 2, σ = 0

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 93 / 103

slide-94
SLIDE 94

Two-layer kernel machine with Log-Hilbert-Schmidt metric

1

First layer: kernel K1, inducing covariance operators

2

Second layer: kernel K2, defined using the Log-Hilbert-Schmidt distance or inner product between the covariance operators

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 94 / 103

slide-95
SLIDE 95

Two-layer kernel machine with Log-Hilbert-Schmidt metric

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 95 / 103

slide-96
SLIDE 96

Material classification

Example: KTH-TIPS2b data set (Caputo et al, ICCV, 2005) f(x, y) =

  • R(x, y), G(x, y), B(x, y),
  • G0,0(x, y)
  • , . . .
  • G3,4(x, y)
  • H.Q. Minh (IIT)

Covariance matrices & covariance operators November 13, 2017 96 / 103

slide-97
SLIDE 97

Material classification

Method KTH-TIPS2b E 55.3% (±7.6%) Stein 73.1% (±8.0%) Log-E 74.1 % (±7.4%) HS 79.3% (±8.2%) Log-HS 81.9% (±3.3%) Log-HS (CNN) 96.6% (±3.4%) CNN features = MatConvNet features

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 97 / 103

slide-98
SLIDE 98

Object recognition

Example: ETH-80 data set f(x, y) = [x, y, I(x, y), |Ix|, |Iy|]

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 98 / 103

slide-99
SLIDE 99

Approximate methods for reducing computational complexity

  • M. Faraki, M. Harandi, and F

. Porikli, Approximate infinite-dimensional region covariance descriptors for image classification, ICASSP 2015 H.Q. Minh, M. San Biagio, L. Bazzani, V. Murino. Approximate Log-Hilbert-Schmidt distances between covariance operators for image classification, CVPR 2016

  • Q. Wang, P

. Li, W. Zuo, and L. Zhang. RAID-G: Robust estimation

  • f approximate infinite-dimensional Gaussian with application to

material recognition, CVPR 2016

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 99 / 103

slide-100
SLIDE 100

Object recognition

Results obtained using approximate Log-HS distance Method ETH-80 E 64.4%(±0.9%) Stein 67.5% (±0.4%) Log-E 71.1%(±1.0%) HS 93.1 % (±0.4) Approx-LogHS 95.0% (±0.5%)

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 100 / 103

slide-101
SLIDE 101

Covariance matrices and covariance operators in computer vision

More detail in H.Q. Minh and V. Murino. Covariances in Computer Vision and Machine Learning, Morgan & Claypool Publishers, 2017 H.Q. Minh and V. Murino. From Covariance Matrices to Covariance Operators: Data Representation from Finite to Infinite-Dimensional Settings. In Algorithmic Advances in Riemannian Geometry and Applications, Springer, 2017 H.Q.Minh. International Conference on Computer Vision (ICCV 2017) Tutorial, http://www.covariance2017.eu/

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 101 / 103

slide-102
SLIDE 102

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 102 / 103

slide-103
SLIDE 103

Thank you for listening! Questions, comments, suggestions?

H.Q. Minh (IIT) Covariance matrices & covariance operators November 13, 2017 103 / 103