Spatiotemporal Pattern Extraction by Spectral Analysis of - - PowerPoint PPT Presentation

Spatiotemporal Pattern Extraction by Spectral Analysis of Vector-valued Observables

Dimitris Giannakis Center for Atmosphere Ocean Science Courant Institute of Mathematical Sciences New York University Geometry and Topology of Data ICERM, 12/11/2017 Collaborators: Abbas Ourmazd, Joanna Slawinska, Jane Zhao

10−4 m 10−1 m 102 m 105 m 107 m 1025 m

Setting

• F(xn)
• F

A X x0 xn = Φnτ x0

• F(A)

HY Y

• F(x0)
• Dynamical flow Φt : X → X on a manifold with an ergodic invariant measure

µ, supported on a compact set A ⊆ X

• Compact spatial domain Y , equipped with a finite measure ρ
• Continuous, vector-valued observation map

F : X → C(Y )

• Objective. Given time-ordered measurements

F(x0), F(x1), . . ., with xn = Φnτ(x0), decompose F into spatiotemporal patterns φj : X → C(Y ),

• F =
• j

cj φj, cj ∈ R

Separable space-time patterns A widely used approach is to recover temporal patterns through the eigenfunctions of an operator T on HA = L2(A, µ), Tϕk = λkϕk, ϕk ∈ HA Many choices for T, including:

• Covariance operators (POD, PCA, SSA, . . . )
• Heat operators (Laplacian eigenmaps, diffusion maps, . . . )
• Koopman operators (DMD, EDMD, . . . )

Spatial patterns ψk in HY = L2(Y , ν) can then be obtained by pointwise projection of the observation map onto the temporal patterns: ψk(y) = ϕk, FyHA, Fy : x ∈ A → F(x)(y) This is equivalent to treating F as a function in the tensor product space HA ⊗ HY , and performing the decomposition

• F ≈ Fl =

l

• k=0

ϕk ⊗ ψk

• In the presence of symmetries and/or spatiotemporal intermittency, pure

tensor product patterns, ϕk ⊗ ψk, may suffer from poor descriptive efficiency and physical interpretability (Aubry et al. 1993)

Hilbert spaces of observables We have the Hilbert space isomorphisms H ≃ HA ⊗ HY ≃ HM, HA = L2(A, µ), HY = L2(Y , ν), H = L2(A, µ; HY ), HM = L2(M, ρ), with M = A × Y ⊆ Ω = X × Y , ρ = µ × ν As a result, the observation map F can be equivalently thought of as:

1 A vector-valued observable

F : A → HY in H

2 An element of the tensor product space HX ⊗ HY , i.e.,

F =

jk cjkeA j ⊗ eY k

for bases {eA

j } of HA and {eY k } of HY 3 A scalar-valued observable F : M → R in HM, s.t. F(x, y) =

F(x)(y) Given x ∈ A, the function t → F(Φt(x)) corresponds to a spatiotemporal pattern

Vector-valued spectral analysis (VSA) framework We decompose F using the eigenfunctions of a compact operator PQ : H → H, PQ φj = λj φj,

• F ≈

Fl =

l

• j=0

cj φj, cj ∈ R This operator is associated with an operator-valued kernel (Micchelli & Pontil

2005, Caponnetto et al. 2008,Carmeli et al. 2010), constructed from delay-coordinate

mapped data with Q delays PQ f =

• A

LQ(·, x) f (x) dµ(x), LQ : X × X → L(HY ) Desirable properties include:

• Ability to recover patterns without a tensor product structure
• Symmetry group actions are naturally factored out
• Asymptotic commutation property with Koopman operators allows to identify

intrinsic dynamical timescales

Operator-valued kernel construction For the purposes of this work:

1 A scalar-valued kernel on Ω = X × Y will be a continuous function

k : Ω × Ω → R+, bounded above and away from zero on compact sets

2 An operator-valued kernel on X will be a continuous function

κ : X × X → L(HY ) Associated with k and κ are kernel integral operators K : HM → HM and K : H → H, respectively, where Kf =

• M

k(·, ω)f (ω) dρ(ω), K f =

• A

κ(·, x) f (x) dµ(x) We can assign k to the operator-valued kernel κ, where κ(x, x′) = Kxx′, Kxx′g(y) =

• Y

k((x, y), (x′, y ′))g(y ′) dν(y ′)

Kernels from delay-coordinate maps (G. & Majda 2012; Berry et al. 2013; G. 2017; Das & G. 2017)

1 Start from a pseudometric dQ : Ω × Ω → R0, s.t.,

d2

Q((x, y), (x′, y ′)) = 1

Q

Q−1

• q=0

|F(Φ−qτ(x), y) − F(Φ−qτ(x′), y ′)|2.

2 Choose a continuous shape function h : R0 → [0, 1], and define the kernel

kQ : Ω × Ω → R+, kQ(ω, ω′) = h(dQ(ω, ω′)); here, h(s) = e−s2/ǫ, with ǫ > 0

3 Normalize kQ to obtain a continuous Markov kernel pQ : Ω × Ω → R+ using

the procedure introduced in the diffusion maps algorithm (Coifman & Lafon 2006): pQ(ω, ω′) = kQ(ω, ω′) lQ(ω)rQ(ω′), rQ =

• M

kQ(·, ω) dρ(ω), lQ =

• M

kQ(·, ω) rQ(ω) dρ(ω) The kernel pQ induces the compact operators PQ : HM → HM and PQ : H → H, s.t. PQf =

• M

pQ(·, ω)f (ω) dρ(ω), PQ f =

• A

LQ(·, x) f (x) dµ(x) where LQ : X × X → L(HY ) is the operator-valued kernel associated with pQ

Vector-valued eigenfunctions

• Identify spatiotemporal patterns, t →

φj(Φt(x)), through the eigenfunctions of PQ: PQ φj = λj φj,

• φj ∈ H,

1 = λ0 > λ1 ≥ λ2 ≥ · · ·

• Expand the observation map

F in the { φj} eigenbasis of H, i.e.,

• F =

∞

• j=0

cj φj, cj = φ′

j,

FH, where φ′

j are eigenfunctions of P∗ Q, satisfying

φ′

j,

φkH = δjk

• Operationally, we obtain (λj,

φj) through the eigenvalue problem for PQ, PQφj = λjφj, φj ∈ H,

• φj(x)(y) = φj((x, y))
• Remark. The

φj are not restricted to a pure tensor product form, ϕj ⊗ ψj, with ϕj ∈ HA and ψj ∈ HY

Bundle structure of spatiotemporal data

• The kernel kQ can be expressed as a pullback of a kernel κQ on RQ, the space
• f delay-coordinate sequences with Q delays,

kQ(ω, ω′) = ˆ kQ(FQ(ω), FQ(ω′)), FQ(ω) = (F(ω), F(ω−1), . . . , F(ω−Q+1)), ω = (x, y), ωq = (Φqτ(x), y)

• Defining BQ = FQ(Ω) and πQ : Ω → BQ s.t. πQ(ω) = FQ(ω), the triplet

(Ω, BQ, πQ) is a topological bundle, with total space Ω, base space BQ, and projection map πQ

• This partitions Ω into equivalence classes, [·]Q, s.t. ω′ ∈ [ω]Q if

πQ(ω) = πQ(ω′)

• Every function in the closed subspace

HQ = ran PQ = span{φj : λj > 0} ⊆ HM, is a pullback of a function in L2(JQ, αQ), with JQ = πQ(M) and αQ = πQ(ρQ), i.e., it is ρ-a.e. constant on the [·]Q equivalence classes

• HQ is not necessarily expressible as a tensor product of HA and HY subspaces.

Limit of no delays If no delays are performed (Q = 1), and M is connected, then J1 = π1(M) is a closed interval

• The eigenfunctions φj are pullbacks of orthogonal functions ηj on J with respect

to the L2 inner product associated with the pushforward measure α1 = π1∗ρ, φj(ω) = ηj(π1(ω)) = ηj(F(ω))

• In particular, the φj are constant on the level sets of the obsevation map F

In a number of cases (e.g., α1 has a C 2 density wrt. Lebesgue measure, and the kernel bandwidth ǫ is small), η1 will be monotonic

• In such cases, even the one-term expansion F ≈ F1 = c1φ1 recovers the

qualitative features of the input signal

Spatial symmetries An important example with nontrivial [·]Q equivalence classes is that of PDE models with equivariant dynamics under the action of a group G on the spatial domain Y

• Suppose that X is a subset of HY (e.g., an inertial manifold of a dissipative

PDE system), and there is a group action Γ g

Y : Y → Y , g ∈ G, satisfying

Φt ◦ Γ g

X = Γ g X ◦ Φt,

Γ g

X (x) = x ◦ Γ g−1 Y

• Then, defining Γ g

Ω = Γ g X ⊗ Γ g Y , the following diagram commutes:

Ω Ω BQ

Γ g

Ω

πQ πQ

Spatial symmetries Under the previous assumptions:

1 For every ω ∈ Ω, the G-orbit ΓΩ(ω) = {Γ g Ω(ω) | g ∈ G} lies in [ω]Q 2 Moreover, the pseudometric dQ has the invariance property

dQ(Γ g

Ω(ω), Γ g′ Ω (ω′)) = dQ(ω, ω′),

for all ω, ω′ ∈ Ω and g, g ′ ∈ G If, in addition, Γ g

Ω preserves null sets with respect to ρ, then it induces a

representation of G on HM, with representatives Rg

M : HM → HM,

Rg

Mf = f ◦ Γ g Ω

• Theorem. The operators PQ and Rg

M satisfy [PQ, Rg M] = 0 and PQRg M = PQ for

all g ∈ G. As a result, every eigenspace Wj of PQ at nonzero eigenvalue is a finite-dimensional (by compactness of PQ), trivial representation space of G, i.e., Rg

Mf = f for every f ∈ Wj.

• Remark. In PCA-type decompositions, ϕj ⊗ ψj, the spatial (ψj) and temporal

(ϕj) patterns also lie in G representation spaces, but the representations are not necessarily trivial

Correspondence with Koopman operators

• Consider the unitary group of Koopman operators Ut : HA → HA, t ∈ R,

acting on scalar-valued observables in HA by composition with the flow map, Utf = f ◦ Φt

• A distinguished class of observables in HA is that of Koopman eigenfunctions,

Utzj = eiωj tzj, zj, zkHA = δjk, ωj ∈ R

• This leads to the Ut-invariant decomposition

HA = D ⊕ D⊥, D = span{zj}

• Because the system is ergodic, the eigenspaces of Ut are all one-dimensional
• Similarly, we can define a group of unitary Koopman operators ˜

Ut : HM → HM, with ˜ Utf = (Ut ⊗ IHY )f = f ◦ (Φt ⊗ IY )

• In this case, we have the ˜

Ut-invariant decomposition HM = ˜ D ⊕ ˜ D⊥, ˜ D = D ⊗ HY , but the eigenspaces of ˜ Ut, span{zj} ⊗ HY , are infinite-dimensional

Correspondence with Koopman operators In the limit of infinitely many delays, the following hold (Das & G., 2017):

1 d∞ = limQ→∞ dQ is well-defined as a function in HM ⊗ HM 2 d∞ lies in ˜

D ⊗ ˜ D

3 d∞ is invariant under ˜

Ut ⊗ ˜ Ut Therefore, we can define a compact operator P∞ : HM → HM, and ran P∞ ⊆ ˜ D

• Theorem. The operators P∞ and ˜

Ut commute for all t ∈ R. As a result, they are simultaneously diagonalizable on the finite-dimensional eigenspaces of P∞.

• Corollary. The eigenspaces Wj of P∞ corresponding to nonzero eigenvalue λj

have the form Wj = span{zj} ⊗ Vj, where zj ∈ HA is an eigenfunction of Ut and Vj a finite-dimensional subspace of HY Note also the following:

1 In the presence of symmetries, P∞, ˜

Ut, and Rg

M are mutually commuting

• perators

2 In Dynamic Mode Decomposition (DMD) (Schmidt & Henningson 2008, Rowley et al.

2009) and related techniques, one assigns a single spatial pattern

ψj(y) = zj, FyHA (Koopman mode) to a given Koopman eigenfunction; here, the number of such patterns is equal to dim Vj

Data-driven approximation

• In many cases of interest, the invariant set A is a non-smooth subset of X of

zero Lebesgue measure (e.g., a fractal attractor)

• Moreover, in realistic experimental environments, the sampled dynamical states

do not lie exacty on A Assumptions for data-driven approximation

1 The measure µ is physical; that is, there exists a set Bµ ⊆ X, of positive

Lebesgue measure, such that for every f ∈ C(X) and x ∈ Bµ, lim

NX →∞

1 NX

NX

• n=0

f (Φnτ(x)) =

• A

f dµ (1)

2 There exists a compact, forward invariant set U ⊆ X, of positive Lebesgue

measure, s.t. A ⊆ U ⊆ ¯ Bµ

3 We make measurements on Y at a sequence of points y0, y1, . . . such that an

analog of (1) holds for f ∈ C(Y )

4 Measurements F(xn, yr), xn = Φnτ(x0), are available along an (unknown) orbit

starting at x0 ∈ ¯ Bµ

Data-driven approximation

• As a data-driven analog of HM = L2(M, ρ), we employ the N-dimensional,

N = NXNY , Hilbert space HΩ,N = L2(M, ρN) associated with the sampling measure, ρN = 1 N

N−1

• j=0

δωj , ωj = (xnj , yrj ), 0 ≤ nj ≤ NX − 1, 0 ≤ rj ≤ NY − 1

• On this space, PQ is approximated by PQ,N : HΩ,N → HΩ,N, where

PQ,Nf =

• Ω

pQ,N(·, ω)f (ω) dρN(ω) = 1 N

N−1

• j=0

pQ,N(·, ωj)f (ωj), pQ,N(ω, ω′) = kQ(ω, ω′) lQ,N(ω′)rQ,N(ω), rQ,N =

• Ω

kQ(·, ω) dρN(ω), lQ,N =

• Ω

kQ(·, ω) rQ,N(ω) dρN(ω)

• The eigenvalue problem for PQ,N is equivalent to an N × N matrix eigenvalue

problem

Spectral convergence

• One issue with establishing convergence (pointwise, in operator norm, etc) of

PQ,N to PQ is that, as defined, these operators act on different spaces (HΩ,N and HM, respectively)

• We thus examine the analogous operators ˜

PQ,N and ˜ PQ on C(V), V = U × Y ⊆ Ω, defined using the same (continuous) kernels as PQ,N and PQ, respectively.

• Theorem. For every nonzero eigenvalue λj of PQ and corresponding

eigenfunction φj ∈ HM:

1 The sequence of eigenvalues λj,N of PQ,N, N ≥ j − 1, converges to λj 2 There exist eigenfunctions φj,N ∈ HΩ,N such that ˜

φj,N ∈ C(V) converges uniformly to ˜ φj ∈ C(V), where ˜ φj,N = 1 λj,N

• Ω

pQ,N(·, ω)φj,N(ω) dρN(ω), ˜ φj =

• M

pQ(·, ω)φj(ω) dρ(ω)

• Proof. Establish that (i) ˜

PQ is compact, and (ii) ˜ PQ,N converges compactly to ˜

• PQ. The claim then follows from spectral approximation results for compact
• perators (Von Luxburg et al 2008; Chatelin 2011).

Application to the Kuramoto-Sivashinksy model

• The Kuramoto-Sivashinsky (KS) model is a prototype dissipative PDE model

exhibiting complex spatiotemporal dynamics, while having a number of useful known properties such as inertial manifolds (Foias et al. 1986) and symmetries

(Kevrekidis et al. 1990, Cvitanovi´ c et al. 2009)

• The governing equation for the real-valued scalar field u(t, ·) : Y → R, t ≥ 0,

Y = [0, L], is given by ˙ u = −u∇u + ∆u − ∆2u, u ∈ HY = L2(Y , Leb), subject to periodic boundary conditions

• The domain size parameter L controls the dynamical complexity of the system;

here, we apply VSA to data generated by the KS model at the chaotic regimes L = 22 and L = 94

• For our purposes, the state space manifold X ⊆ HY will be an inertial manifold
• f the KS system

KS patterns, L = 22 At a small number of delays, Q = 15, the recovered eigenfunctions are approximately constant on the level sets of the input signal

KS patterns, L = 22

• At a larger number of delays Q = 500, the leading vector-valued eigenfunctions

capture O(2) families of unstable equilibria (wavenumber L/2 structures), and smaller-scale traveling waves embedded in those structures

• In contrast, NLSA (G. & Majda 2012), a scalar-valued kernel technique also

utilizing delays, requires several modes to capture these families

KS patterns, L = 94

Conclusions Kernel algorithms operating on spaces of vector-valued observables have a number of useful properties for spatiotemporal pattern extraction, including the ability to:

• Recover patterns with a non-separable structure in the spatial and temporal

degrees of freedom

• Quotient out symmetries
• Recover intrinsic timescales associated with the point spectrum of the

Koopman operator of the dynamical systems Physical measures allow spectral convergence of data-driven approximations of such operators for ergodic dynamical systems with non-smooth invariant sets Ongoing and future work includes applications in atmosphere ocean science and extensions of VSA to skew-product dynamical systems

References

• Giannakis, D., J. Slawinska, A. Ourmazd, Z. Zhao (2017). Vector-Valued

Spectral Analysis of Space-Time Data. Proceedings of the NIPS 2017 Time Series Workshop.

• Giannakis, D., A. Ourmazd, J. Slawinska, Z. Zhao (2017). Spatiotemporal

pattern extraction by spectral analysis of vector-valued observables. Submitted. arXiv: 1711.02798.

• Das, S., and D. Giannakis (2017). Delay-coordinate maps and the spectra of

Koopman operators. arXiv:1706.08544

• Giannakis, D. (2017). Data-driven spectral decomposition and forecasting of

ergodic dynamical systems. Applied and Computational Harmonic Analysis, doi:10.1016/j.acha.2017.09.001 Research supported by DARPA grant HR0011-16-C-0116, NSF grant DMS-1521775, ONR grant N00014-14-1-0150, and ONR YIP grant N00014-16-1-2649