Parametrizations of manifolds with heat kernels, multiscale analysis on graphs, and applications to analysis of data sets Mauro Maggioni Mathematics and Computer Science Duke University U.S.C./I.M.I., Columbia, 3/5/08 In collaboration with R.R. Coifman, P .W. Jones, R. Schul, A.D. Szlam Funding: NSF-DMS, ONR. Mauro Maggioni Heat kernels and multiscale analysis on manifolds
Plan Review of Setting and Motivation Graphs from data sets Eigenfunction and heat kernel embeddings Multiscale construction, diffusion wavelets Examples and applications Conclusion Mauro Maggioni Heat kernels and multiscale analysis on manifolds
Low-dimensional sets in high-dimensional spaces It has been shown, at least empirically, that in such situations the geometry of the data can help construct useful priors, for tasks such as classification, regression for prediction purposes. Problems: geometric : find intrinsic properties, such as local dimensionality, and local parameterizations. approximation theory : approximate functions on such data, respecting the geometry. Mauro Maggioni Heat kernels and multiscale analysis on manifolds
Random walks and heat kernels on the data Assume the data X = { x i } ⊂ R n . Assume we can assign local similarities via a kernel function K ( x i , x j ) ≥ 0. Example: K σ ( x i , x j ) = e −|| x i − x j || 2 /σ . Model the data as a weighted graph ( G , E , W ) : vertices represent data points, edges connect x i , x j with weight W ij := K ( x i , x j ) , when positive. Let D ii = � j W ij and , T = D − 1 2 WD − 1 P = D − 1 W , H = e − t ( I − T ) 2 � �� � � �� � � �� � symm . “ random walk ′′ random walk Heat kernel Note 1: K typically depends on the type of data. Note 2: K should be “local”, i.e. close to 0 for points not sufficiently close. Mauro Maggioni Heat kernels and multiscale analysis on manifolds
Random walks and heat kernels on the data Assume the data X = { x i } ⊂ R n . Assume we can assign local similarities via a kernel function K ( x i , x j ) ≥ 0. Example: K σ ( x i , x j ) = e −|| x i − x j || 2 /σ . Model the data as a weighted graph ( G , E , W ) : vertices represent data points, edges connect x i , x j with weight W ij := K ( x i , x j ) , when positive. Let D ii = � j W ij and , T = D − 1 2 WD − 1 P = D − 1 W , H = e − t ( I − T ) 2 � �� � � �� � � �� � symm . “ random walk ′′ random walk Heat kernel Note 1: K typically depends on the type of data. Note 2: K should be “local”, i.e. close to 0 for points not sufficiently close. Mauro Maggioni Heat kernels and multiscale analysis on manifolds
Random walks and heat kernels on the data Assume the data X = { x i } ⊂ R n . Assume we can assign local similarities via a kernel function K ( x i , x j ) ≥ 0. Example: K σ ( x i , x j ) = e −|| x i − x j || 2 /σ . Model the data as a weighted graph ( G , E , W ) : vertices represent data points, edges connect x i , x j with weight W ij := K ( x i , x j ) , when positive. Let D ii = � j W ij and , T = D − 1 2 WD − 1 P = D − 1 W , H = e − t ( I − T ) 2 � �� � � �� � � �� � symm . “ random walk ′′ random walk Heat kernel Note 1: K typically depends on the type of data. Note 2: K should be “local”, i.e. close to 0 for points not sufficiently close. Mauro Maggioni Heat kernels and multiscale analysis on manifolds
Handwritten Digits Data base of about 60 , 000 28 × 28 gray-scale pictures of handwritten digits, collected by USPS. Point cloud in R 28 2 . Goal: automatic recognition. Set of 10 , 000 picture (28 by 28 pixels) of 10 handwritten digits. Color represents the label (digit) of each point. Mauro Maggioni Heat kernels and multiscale analysis on manifolds
Eigenfunction Embedding theorems, I [Joint with P .W. Jones and R. Schul] We ask whether eigenfunctions of the Laplacian can be used to parametrize Euclidean domains and manifolds, in which generality this may be true, and which conditions such an embedding may satisfy. Other recent proposed techniques include isomap, lle, Hessian eigenmaps, maximum variance embedding; we are aware of proven results only for isomap and Hessian eigenmap, and in both cases the assumptions require the manifold to the isometric image of a Euclidean domain. Also, Bérard, Besson and Gallot (’84,’94) use all the eigenfunctions to embed into ℓ 2 . Mauro Maggioni Heat kernels and multiscale analysis on manifolds
Eigenfunction Embedding theorems, I (cont’d) Independently of the boundary conditions, we will denote by ∆ the Laplacian on Ω , For the purpose of this paper (both the Dirichlet and Neumann case) we restrict our study to domains where the spectrum is discrete and the corresponding heat kernel can be written as � ϕ j ( z ) ϕ j ( w ) e − λ j t . K Ω t ( z , w ) = where the { ϕ j } form an orthonormal basis for the appropriate Hilbert space with eigenvalues 0 ≤ λ 0 ≤ · · · ≤ λ j ≤ . . . . We also require d 2 | Ω | . # { j : λ j ≤ T } ≤ C Weyl , Ω T Dirichlet case: OK, Neumann: possible problems. Mauro Maggioni Heat kernels and multiscale analysis on manifolds
Eigenfunction Embedding theorems, II Theorem (Embedding via Eigenfunctions, for Euclidean domains) Let Ω be a domain in R d , with | Ω | = 1 , and boundary as above. There are constants c 1 , . . . , c 6 > 0 that depend only on d and C Weyl , Ω , such that the following hold. For any z ∈ Ω , let R z ≤ dist ( z , ∂ Ω) . Then there exist i 1 , . . . , i d and constants d z ≤ γ 1 = γ 1 ( z ) , ..., γ d = γ d ( z ) ≤ 1 such that: c 6 R 2 (a) Φ : B c 1 R z ( z ) → R d , defined by x �→ ( γ 1 ϕ i 1 ( x ) , . . . , γ d ϕ i d ( x )) satisfies, for any x 1 , x 2 ∈ B ( z , c 1 R z ) , c 2 || x 1 − x 2 || ≤ || Φ( x 1 ) − Φ( x 2 ) || ≤ c 3 || x 1 − x 2 || . R z R z (b) c 4 R − 2 ≤ λ i 1 , . . . , λ i d ≤ c 5 R − 2 . z z Mauro Maggioni Heat kernels and multiscale analysis on manifolds
Eigenfunction Embedding theorems, III Figure: Top left: a non-simply connected domain in R 2 , and the point z with its neighborhood to be mapped. center and left: Two eigenfunctions for mapping the neighborhood to roughly a unit ball. Mauro Maggioni Heat kernels and multiscale analysis on manifolds
Eigenfunction Embedding theorems, IV Let M be a smooth, d -dimensional compact manifold, possibly with boundary. Suppose we are given a metric tensor g on M which is C α for some α > 0. For any z 0 ∈ M , let ( U , x ) be a coordinate chart such that z 0 ∈ U , g il ( x ( z 0 )) = δ il and for any w ∈ U , and any R d ≤ � d ξ, ν ∈ R d , c min ( g ) || ξ || 2 i , j = 1 g ij ( x ( w )) ξ i ξ j , � d i , j = 1 g ij ( x ( w )) ξ i ν j ≤ c max ( g ) || ξ || R d || ν || R d . We let r M ( z 0 ) = sup { r > 0 : B r ( x ( z 0 )) ⊆ x ( U ) } . 1 �� � � det g g ij ( x ) ∂ i f √ det g ∆ M f ( x ) = − ∂ j ( x ) . i , j = 1 Mauro Maggioni Heat kernels and multiscale analysis on manifolds
Eigenfunction Embedding theorems, IV Theorem (Embedding via Eigenfunctions, for Manifolds) Let ( M , g ) , z ∈ M be a d dimensional manifold and ( U , x ) be a chart as above. Also, assume |M| = 1 . There are constants c 1 , . . . , c 6 > 0 , depending on d , c min , c max , || g || α ∧ 1 , α ∧ 1 , and C Weyl , Ω , such that the following hold. Let R z = r M ( z ) . Then there exist i 1 , . . . , i d and d constants c 6 R z ≤ γ 1 = γ 1 ( z ) , ..., γ d = γ d ( z ) ≤ 1 such that: 2 (a) the map Φ : B c 1 R z ( z ) → R d , defined by x �→ ( γ 1 ϕ i 1 ( x ) , . . . , γ d ϕ i d ( x )) such that for any x 1 , x 2 ∈ B ( z , c 1 R z ) c 2 d M ( x 1 , x 2 ) ≤ || Φ( x 1 ) − Φ( x 2 ) || ≤ c 3 d M ( x 1 , x 2 ) . R z R z (b) c 4 R − 2 ≤ λ i 1 , . . . , λ i d ≤ c 5 R − 2 . z z Mauro Maggioni Heat kernels and multiscale analysis on manifolds
Robust parametrizations through heat kernels Theorem (Heat Triangulation Theorem - with P .W. Jones, R. Schul) Let ( M , g ) be a Riemannian manifold, with g at least C α , α > 0 , and z ∈ M . Let R z be the radius of the largest ball on M , centered at z, which is bi-Lipschitz equivalent to a Euclidean ball. Let p 1 , ..., p d be d linearly independent directions. There are constants c 1 , . . . , c 5 > 0 , depending on d , c min , c max , || g || α ∧ 1 , α ∧ 1 , and the smallest and largest eigenvalues of the Gramian matrix ( � p i , p j � ) i = 1 ,..., d , such that the following holds. Let y i be so that y i − z is in the direction p i , with c 4 R z ≤ d M ( y i , z ) ≤ c 5 R z for each i = 1 , . . . , d and let t z = c 6 R 2 z . The map Φ : B c 1 R z ( z ) → R d x �→ ( R d z K t z ( x , y 1 )) , . . . , R d z K t z ( x , y d )) satisfies, for any x 1 , x 2 ∈ B c 1 R z ( z ) , c 2 d M ( x 1 , x 2 ) ≤ || Φ( x 1 ) − Φ( x 2 ) || ≤ c 3 d M ( x 1 , x 2 ) . R z R z Mauro Maggioni Heat kernels and multiscale analysis on manifolds
Recommend
More recommend
