Spatiotemporal Pattern Extraction by Spectral Analysis of - PowerPoint PPT Presentation

Spatiotemporal Pattern Extraction by Spectral Analysis of Vector-valued Observables Dimitris Giannakis Center for Atmosphere Ocean Science Courant Institute of Mathematical Sciences New York University Geometry and Topology of Data ICERM, 12/11/2017 Collaborators: Abbas Ourmazd, Joanna Slawinska, Jane Zhao

10 − 4 m 10 − 1 m 10 2 m 10 5 m 10 7 m 10 25 m

Setting Y � F ( x n ) x n = Φ n τ x 0 � F ( x 0 ) � F x 0 A � F ( A ) X H Y • Dynamical flow Φ t : X �→ X on a manifold with an ergodic invariant measure µ , supported on a compact set A ⊆ X • Compact spatial domain Y , equipped with a finite measure ρ • Continuous, vector-valued observation map � F : X �→ C ( Y ) Objective. Given time-ordered measurements � F ( x 0 ) , � F ( x 1 ) , . . . , with x n = Φ n τ ( x 0 ), decompose � F into spatiotemporal patterns � φ j : X �→ C ( Y ), � � c j � F = φ j , c j ∈ R j

Separable space-time patterns A widely used approach is to recover temporal patterns through the eigenfunctions of an operator T on H A = L 2 ( A , µ ), T ϕ k = λ k ϕ k , ϕ k ∈ H A Many choices for T , including: • Covariance operators (POD, PCA, SSA, . . . ) • Heat operators (Laplacian eigenmaps, diffusion maps, . . . ) • Koopman operators (DMD, EDMD, . . . ) Spatial patterns ψ k in H Y = L 2 ( Y , ν ) can then be obtained by pointwise projection of the observation map onto the temporal patterns: F y : x ∈ A �→ � ψ k ( y ) = � ϕ k , F y � H A , F ( x )( y ) This is equivalent to treating � F as a function in the tensor product space H A ⊗ H Y , and performing the decomposition l � � F ≈ F l = ϕ k ⊗ ψ k k =0 • In the presence of symmetries and/or spatiotemporal intermittency , pure tensor product patterns, ϕ k ⊗ ψ k , may suffer from poor descriptive efficiency and physical interpretability (Aubry et al. 1993)

Hilbert spaces of observables We have the Hilbert space isomorphisms H ≃ H A ⊗ H Y ≃ H M , H A = L 2 ( A , µ ) , H Y = L 2 ( Y , ν ) , H = L 2 ( A , µ ; H Y ) , H M = L 2 ( M , ρ ) , with M = A × Y ⊆ Ω = X × Y , ρ = µ × ν As a result, the observation map � F can be equivalently thought of as: 1 A vector-valued observable � F : A �→ H Y in H 2 An element of the tensor product space H X ⊗ H Y , i.e., � jk c jk e A j ⊗ e Y F = � k for bases { e A j } of H A and { e Y k } of H Y 3 A scalar-valued observable F : M �→ R in H M , s.t. F ( x , y ) = � F ( x )( y ) Given x ∈ A , the function t �→ � F ( Φ t ( x )) corresponds to a spatiotemporal pattern

Vector-valued spectral analysis (VSA) framework We decompose � F using the eigenfunctions of a compact operator P Q : H �→ H , l P Q � φ j = λ j � F ≈ � � c j � � φ j , F l = φ j , c j ∈ R j =0 This operator is associated with an operator-valued kernel (Micchelli & Pontil 2005, Caponnetto et al. 2008,Carmeli et al. 2010) , constructed from delay-coordinate mapped data with Q delays � P Q � L Q ( · , x ) � f = f ( x ) d µ ( x ) , L Q : X × X �→ L ( H Y ) A Desirable properties include: • Ability to recover patterns without a tensor product structure • Symmetry group actions are naturally factored out • Asymptotic commutation property with Koopman operators allows to identify intrinsic dynamical timescales

Operator-valued kernel construction For the purposes of this work: 1 A scalar-valued kernel on Ω = X × Y will be a continuous function k : Ω × Ω �→ R + , bounded above and away from zero on compact sets 2 An operator-valued kernel on X will be a continuous function κ : X × X �→ L ( H Y ) Associated with k and κ are kernel integral operators K : H M �→ H M and K : H �→ H , respectively, where � � K � κ ( · , x ) � Kf = k ( · , ω ) f ( ω ) d ρ ( ω ) , f = f ( x ) d µ ( x ) M A We can assign k to the operator-valued kernel κ , where � κ ( x , x ′ ) = K xx ′ , k (( x , y ) , ( x ′ , y ′ )) g ( y ′ ) d ν ( y ′ ) K xx ′ g ( y ) = Y

Kernels from delay-coordinate maps (G. & Majda 2012; Berry et al. 2013; G. 2017; Das & G. 2017) 1 Start from a pseudometric d Q : Ω × Ω �→ R 0 , s.t., Q − 1 Q (( x , y ) , ( x ′ , y ′ )) = 1 d 2 � | F ( Φ − q τ ( x ) , y ) − F ( Φ − q τ ( x ′ ) , y ′ ) | 2 . Q q =0 2 Choose a continuous shape function h : R 0 �→ [0 , 1], and define the kernel k Q ( ω, ω ′ ) = h ( d Q ( ω, ω ′ )); k Q : Ω × Ω �→ R + , here, h ( s ) = e − s 2 /ǫ , with ǫ > 0 3 Normalize k Q to obtain a continuous Markov kernel p Q : Ω × Ω �→ R + using the procedure introduced in the diffusion maps algorithm (Coifman & Lafon 2006) : k Q ( ω, ω ′ ) � � k Q ( · , ω ) p Q ( ω, ω ′ ) = l Q ( ω ) r Q ( ω ′ ) , r Q = k Q ( · , ω ) d ρ ( ω ) , l Q = r Q ( ω ) d ρ ( ω ) M M The kernel p Q induces the compact operators P Q : H M �→ H M and P Q : H �→ H , s.t. � � P Q � L Q ( · , x ) � P Q f = p Q ( · , ω ) f ( ω ) d ρ ( ω ) , f = f ( x ) d µ ( x ) M A where L Q : X × X �→ L ( H Y ) is the operator-valued kernel associated with p Q

Vector-valued eigenfunctions • Identify spatiotemporal patterns, t �→ � φ j ( Φ t ( x )), through the eigenfunctions of P Q : P Q � φ j = λ j � � φ j , φ j ∈ H , 1 = λ 0 > λ 1 ≥ λ 2 ≥ · · · • Expand the observation map � F in the { � φ j } eigenbasis of H , i.e., ∞ � � c j � c j = � � φ ′ j , � F = φ j , F � H , j =0 where � Q , satisfying � � j , � φ ′ j are eigenfunctions of P ∗ φ ′ φ k � H = δ jk • Operationally, we obtain ( λ j , � φ j ) through the eigenvalue problem for P Q , � P Q φ j = λ j φ j , φ j ∈ H , φ j ( x )( y ) = φ j (( x , y )) Remark. The � φ j are not restricted to a pure tensor product form, ϕ j ⊗ ψ j , with ϕ j ∈ H A and ψ j ∈ H Y

Bundle structure of spatiotemporal data • The kernel k Q can be expressed as a pullback of a kernel κ Q on R Q , the space of delay-coordinate sequences with Q delays, k Q ( ω, ω ′ ) = ˆ k Q ( F Q ( ω ) , F Q ( ω ′ )) , ω q = ( Φ q τ ( x ) , y ) F Q ( ω ) = ( F ( ω ) , F ( ω − 1 ) , . . . , F ( ω − Q +1 )) , ω = ( x , y ) , • Defining B Q = F Q ( Ω ) and π Q : Ω �→ B Q s.t. π Q ( ω ) = F Q ( ω ), the triplet ( Ω, B Q , π Q ) is a topological bundle , with total space Ω , base space B Q , and projection map π Q • This partitions Ω into equivalence classes , [ · ] Q , s.t. ω ′ ∈ [ ω ] Q if π Q ( ω ) = π Q ( ω ′ ) • Every function in the closed subspace H Q = ran P Q = span { φ j : λ j > 0 } ⊆ H M , is a pullback of a function in L 2 ( J Q , α Q ), with J Q = π Q ( M ) and α Q = π Q ( ρ Q ), i.e., it is ρ -a.e. constant on the [ · ] Q equivalence classes • H Q is not necessarily expressible as a tensor product of H A and H Y subspaces.

Limit of no delays If no delays are performed ( Q = 1), and M is connected, then J 1 = π 1 ( M ) is a closed interval • The eigenfunctions φ j are pullbacks of orthogonal functions η j on J with respect to the L 2 inner product associated with the pushforward measure α 1 = π 1 ∗ ρ , φ j ( ω ) = η j ( π 1 ( ω )) = η j ( F ( ω )) • In particular, the φ j are constant on the level sets of the obsevation map F In a number of cases (e.g., α 1 has a C 2 density wrt. Lebesgue measure, and the kernel bandwidth ǫ is small), η 1 will be monotonic • In such cases, even the one-term expansion F ≈ F 1 = c 1 φ 1 recovers the qualitative features of the input signal

Spatial symmetries An important example with nontrivial [ · ] Q equivalence classes is that of PDE models with equivariant dynamics under the action of a group G on the spatial domain Y • Suppose that X is a subset of H Y (e.g., an inertial manifold of a dissipative PDE system), and there is a group action Γ g Y : Y �→ Y , g ∈ G , satisfying Φ t ◦ Γ g X ( x ) = x ◦ Γ g − 1 X = Γ g X ◦ Φ t , Γ g Y • Then, defining Γ g Ω = Γ g X ⊗ Γ g Y , the following diagram commutes: Γ g Ω Ω Ω π Q π Q B Q

Spatial symmetries Under the previous assumptions: 1 For every ω ∈ Ω , the G -orbit Γ Ω ( ω ) = { Γ g Ω ( ω ) | g ∈ G } lies in [ ω ] Q 2 Moreover, the pseudometric d Q has the invariance property Ω ( ω ) , Γ g ′ d Q ( Γ g Ω ( ω ′ )) = d Q ( ω, ω ′ ) , for all ω, ω ′ ∈ Ω and g , g ′ ∈ G If, in addition, Γ g Ω preserves null sets with respect to ρ , then it induces a representation of G on H M , with representatives R g R g M f = f ◦ Γ g M : H M �→ H M , Ω Theorem. The operators P Q and R g M satisfy [ P Q , R g M ] = 0 and P Q R g M = P Q for all g ∈ G. As a result, every eigenspace W j of P Q at nonzero eigenvalue is a finite-dimensional (by compactness of P Q ), trivial representation space of G, i.e., R g M f = f for every f ∈ W j . Remark. In PCA-type decompositions, ϕ j ⊗ ψ j , the spatial ( ψ j ) and temporal ( ϕ j ) patterns also lie in G representation spaces, but the representations are not necessarily trivial