Spectral Geometry of Shapes JING HUA ( ) GIL Graphics and Imaging - - PowerPoint PPT Presentation

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Spectral Geometry of Shapes

JING HUA (华璟)

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

3D Shape Data

Polygon mesh Analytical Surface Point Cloud Volume Data

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

3D Shape Analysis

• Shape Analysis

○ Matching, indexing, searching, retrieval, registration,

etc.

• Shape Understanding

○ Pose analysis, 4D time-varying motion, etc.

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Regularization and Normalization

• Voxelization
• Geometric Mapping
• Requiring correspondence between shapes

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Challenges

• Irregular sampling
• Different triangulations
• Euclidean transforms
• Non-linear deformation

○ Near-isometric deformation ○ Non-isometric deformation

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Contributions

• Feature Computation in spectral domain

○ Invariant to Euclidean transformations, different

triangulation, and isometric deformations

• Near-isometric pose analysis

○ Registration free semantic surface segmentation

and skeletal analysis

• Non-isometric eigenvalue variation

○ Spectrum alignment for non-isometric deformations

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Outlines

• Feature computation in spectral domain
• Near-isometric pose analysis
• Non-isometric eigenvalue variation

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Laplace-Beltrami Operator

Laplace-Beltrami Operator is defined with the divergence of the gradient of a function on a manifold as Note the continuous function is arbitrary.

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Laplace Eigenvalue Problem

The Laplace-Beltrami operator introduces an eigenvalue problem On a closed manifold, the solution is a family of eigenvalues and corresponding eigenfunctions These eigenvalues only rely on the geometry.

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Discrete Laplace-Beltrami Operator

On triangle meshes, the Laplace-Beltrami

• perator can be defined on each vertex as

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Laplace Matrix

Rewrite Laplace-Beltrami operator in matrix form The Laplace eigen equation becomes a matrix equation

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Laplace Matrix cont.

The Laplace matrix can be decomposed into two symmetric matrices where

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Laplace Matrix cont.

General eigenvalue problem Inner product of vectors Eigenvectors are orthogonal to each other

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Eigenvalue and Eigenfunction of a Manifold

Numb er Eigen‐ Value 2.93 e‐17 1 2.06 e‐03 2 8.99 e‐03 10 5.08 e‐02 15 7.86 e‐02

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Shape Spectrum

The eigenvalues and the 3rd, 5th, and 10th eigenvectors of discrete Laplace matrices on different poses.

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

• Salient shape feature
• Invariant shape descriptors
• Shape matching
• Shape retrieval
• Partial matching

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Spectrum Analysis

Matching signals, coarse to fine.

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Shape Spectrum Projection

3D embedding can be projected onto eigenfunctions and reconstructed

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Shape Spectrum Reconstruction

Geometric reconstruction with first 5, 20, 100, and 400 eigenfunctions, respectively.

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Spectral Salient Features

Geometry energy between neighboring eigenfunction reconstructions Salient feature descriptors

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Spectral Salient Features

Salient feature points extracted in the spectral domain.

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Shape Matching

Shape matching between nonlinear deformations, similar shapes, and partial shapes.

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Shape Retrieval

Shape retrieval with spectral features

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Outlines

• Feature computation in spectral domain
• Near-isometric pose analysis
• Non-isometric eigenvalue variation

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

• Pose analysis
• Motion classification/recognition
• Motion retrieval

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Spectrum Space

Global position system Absolute global position system

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Near-Isometric Deformation

Transfer spatial points to spectral domain. Non- linear deformations are aligned in the spectral domain.

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Spectral Part Clustering

Classify rigid parts and articulated parts in the geometry spectral domain

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Pose Analysis

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Outlines

• Feature computation in spectral domain
• Near-isometric pose analysis
• Non-isometric eigenvalue variation

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Motivation

• Extend shape spectrum to non-isometric deformation
• Across object spectrum alignment algorithm
• 4D time varying motion analysis

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Shape Variation with Spectrum

Eigen- Value 5.29 e-17 4.31 e-03 1.62 e-02 3.03 e-02 3.76 e-02 3.88 e-2 … Eigen- Value 8.53 e-17 4.23 e-03 1.71 e-02 3.61 e-02 5.52 e-02 5.87 e-02 …

Scale Function

32

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

• On

a compact closed manifold with Riemannian metric , we define a deformation function as a time variant positive scale function

:

• and

Determining the scale function Involves derivatives

Scale Function

N M

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

• Theorem 1. λ is piecewise analytic and, at any regular

point, the t-derivative of λ is:

• Ω

is a nonnegative, continuously differentiable, and diagonal matrix.

Theorem Guarantees The Existence of Scale Function

Eigenfunctions Derivative of scale matrix Voronoi region Derivative of eigenvalues Eigenvalue index

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

• Sketch of the Proof

Eigen System

t-Derivative Multiply v

• 1

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

For mapping N M, we assume that the eigenvalues change linearly

1 , 0,1 We divide the time interval 0,1 into steps which we index them as

Eigenvalue Variation

N M

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

We focus on the global smoothness of scale factors distributed on manifold .

• Smoothness Constraints

v

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

The result Ω at each step can be used to calculate Ω 1 using: v

v

v

• W v

2 v

• ⇒ Ω 1 Ω

1 Ω

Algorithm

Theorem Linear Eigenvalue Derivative Global Smoothness 10 Number of Eigenvalues = 100

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Synthetic Results: Expansion

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Synthetic Results: Shrinkage

1 0.5

• 0.5
• 1

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Synthetic Results: Scaling

1 0.5

• 0.5
• 1

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Left hippocampus is affected by epilepsy

Epilepsy Imaging Study

Left Right Left Right 1 0.5

• 0.5
• 1

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

The 12th eigenfunction

Epilepsy Imaging Study

Left Right Left Right 1 0.5

• 0.5
• 1

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Eigenvalue Variation

Manifold Left Hippocampus 3.87, 7.76, 11.93, 14.22, 15.88, 18.49, 20.62 Right Hippocampus 4.36, 7.75, 11.20, 12.62, 16.60, 18.35, 21.73 Aligned Left one 4.36, 7.75, 11.19, 12.62 , 16.59, 18.34, 21.73

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

LV Data

Eigenfunction alignments on LV data.

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

LV Motion

Motions are represented with scale functions

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Conclusion

• Feature computation in shape spectrum. Invariant to

Euclidean transformations, isometric deformations, and different triangulations.

• Near-isometric pose analysis. Semantic segmentation,

and automatic skeletal analysis.

• Eigenvalue variation with scale function on Riemann
• metrics. Spectrum alignment across non-isometric

deformations.

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Positions available for Ph.D. Students Contact: jinghua@wayne.edu http://www.cs.wayne.edu/~jinghua

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

Thanks & Questions