[PPT] - Fully Spectral Partial Shape Matching Or Litany 1 , 2 a 3 Emanuele PowerPoint Presentation

SLIDE 1

1/30

Fully Spectral Partial Shape Matching

Or Litany1,2 Emanuele Rodol` a3 Alexander Bronstein1,2,4 Michael Bronstein1,2,3

1Tel Aviv University 2Intel 3USI Lugano 4Technion

Eurographics, 27 April 2017

SLIDE 2

2/30

3D sensing applications

LIDAR Velodyne HDL-64E (as in the Google Car); Intel RealSense R200 3D camera; FaceShift Inc. ; Me ; A cute baby

SLIDE 3

2/30

3D sensing applications

Non-rigid deformations Limited view points

LIDAR Velodyne HDL-64E (as in the Google Car); Intel RealSense R200 3D camera; FaceShift Inc. ; Me ; A cute baby

SLIDE 4

3/30

Shape correspondence problem

Isometric

SLIDE 5

3/30

Shape correspondence problem

Isometric Partial

SLIDE 6

3/30

Shape correspondence problem

Isometric Partial Topological noise

SLIDE 7

3/30

Shape correspondence problem

Isometric Partial Topological noise Different representation

SLIDE 8

3/30

Shape correspondence problem

Isometric Partial Topological noise Different representation Non-isometric

SLIDE 9

4/30

Point-wise maps

xi X yj Y t

Point-wise maps t: X → Y

SLIDE 10

4/30

Functional maps

f F(X) g F(Y ) T

Functional maps T: F(X) → F(Y )

Ovsjanikov et al. 2012

SLIDE 11

5/30

Functional correspondence

f g ↓ T ↓ Ovsjanikov et al. 2012

SLIDE 12

5/30

Functional correspondence

f g ≈ a1 + a2 + · · · + ak ≈ b1 + b2 + · · · + bk ↓ T ↓ Ovsjanikov et al. 2012

SLIDE 13

5/30

Functional correspondence

f g ≈ a1 + a2 + · · · + ak ≈ b1 + b2 + · · · + bk ↓ T ↓ ↓ C ↓ Translates Fourier coefficients from Φ to Ψ Ovsjanikov et al. 2012

SLIDE 14

5/30

Functional correspondence

f g ≈ a1 + a2 + · · · + ak ≈ b1 + b2 + · · · + bk ↓ T ↓ ↓ C ↓ Translates Fourier coefficients from Φ to Ψ ≈ Ψk Φ⊤

k

Ψ⊤

k g = CΦ⊤ k f

where Φk = (φ1, . . . , φk), Ψk = (ψ1, . . . , ψk) are Laplace-Beltrami eigenbases

Ovsjanikov et al. 2012

SLIDE 15

6/30

Fourier analysis (non-Euclidean spaces)

The Laplacian is invariant to isometries

φ1 φ2 φ3 φ4 ψ1 ψ2 ψ3 ψ4

SLIDE 16

7/30

Functional correspondence in Laplacian eigenbases

C = Ψ⊤

k TΦk ⇒ cij = ψi, Tϕj

For isometric simple spectrum shapes, C is diagonal since ψi = ±Tφi

SLIDE 17

8/30

Part-to-full correspondence

Full model Partial query

SLIDE 18

9/30

Partial Laplacian eigenvectors

φ2 φ3 φ4 φ5 φ6 φ7 φ8 φ9 ψ2 ψ3 ψ4 ψ5 ψ6 ψ7 ψ8 ψ9 ζ2 ζ3 ζ4 ζ5 ζ6 ζ7 ζ8 ζ9

Laplacian eigenvectors of a shape with missing parts (Neumann boundary conditions)

Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016

SLIDE 19

9/30

Partial Laplacian eigenvectors

φ2 φ3 φ4 φ5 φ6 φ7 φ8 φ9 ψ2 ψ3 ψ4 ψ5 ψ6 ψ7 ψ8 ψ9 ζ2 ζ3 ζ4 ζ5 ζ6 ζ7 ζ8 ζ9

Laplacian eigenvectors of a shape with missing parts (Neumann boundary conditions)

Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016

SLIDE 20

10/30

Partial Laplacian eigenvectors

Functional correspondence matrix C Slope ≈ ratio of the two surface areas

Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016

SLIDE 21

11/30

Going fully spectral

PFM has two major drawbacks: Explicit spatial indicator → runtime is O(n)

solve

⇒

SLIDE 22

11/30

Going fully spectral

PFM has two major drawbacks: Explicit spatial indicator → runtime is O(n) The partiality prior requires heavy engineering ρpart(v) = µ1

area(X) −
Y

η(v)dx 2 + µ2

Y

ξ(v)∇Y η(v)dx ξ(v) = δ

η(v) − 1

2

η(v) = 1

2(tanh(2v − 1) + 1) ρcorr(C) = µ3C ◦ W2

F + µ4

i=j

(C⊤C)2

ij + µ5

i

((C⊤C)ii − di)2

SLIDE 23

11/30

Going fully spectral

PFM has two major drawbacks: Explicit spatial indicator → runtime is O(n) The partiality prior requires heavy engineering Our idea: “reorder” and spatially localize the eigenfunctions

SLIDE 24

11/30

Going fully spectral

PFM has two major drawbacks: Explicit spatial indicator → runtime is O(n) The partiality prior requires heavy engineering Our idea: “reorder” and spatially localize the eigenfunctions No indicator → Runtime is O(k2)

SLIDE 25

11/30

Going fully spectral

PFM has two major drawbacks: Explicit spatial indicator → runtime is O(n) The partiality prior requires heavy engineering Our idea: “reorder” and spatially localize the eigenfunctions No indicator → Runtime is O(k2) One-to-one correspondence yields a simple prior

SLIDE 26

12/30

Localized basis functions

φ1 φ2 φ3 φ4 φ5 φ6 φ7 φ8 φ9 φ10 ψ1 ψ2 ψ3 ψ4 ψ5 ψ6 ψ7 ψ8 ψ9 ψ10 ˆ ψ1 ˆ ψ2 ˆ ψ3 ˆ ψ4 ˆ ψ5 ˆ ψ6 ˆ ψ7 ˆ ψ8 ˆ ψ9 ˆ ψ10

SLIDE 27

13/30

Localization

Energy minimized in PFM min

C,v CA − B(v) + ρcorr(C) + ρpart(v)

v : N → [0, 1] A = (φi, fjM) B(v) = (ψi, v · gjN )

SLIDE 28

13/30

Localization

Energy minimized in PFM min

C,v CA − B(v) + ρcorr(C) + ρpart(v)

Satisfying the data-term induces a localizing map C

SLIDE 29

14/30

Localization

SLIDE 30

14/30

Localization

SLIDE 31

14/30

Localization

SLIDE 32

14/30

Localization

SLIDE 33

15/30

Localization

SLIDE 34

15/30

Localization

Wr ×Ψ, F = Q⊤ ×Φ, G

SLIDE 35

15/30

Localization

Wr ×Ψ, F = Q⊤ ×Φ, G = Q⊤Φ, G = ˆ Φ, G

SLIDE 36

16/30

Localized basis functions

φ1 φ2 φ3 φ4 φ5 φ6 φ7 φ8 φ9 φ10 ψ1 ψ2 ψ3 ψ4 ψ5 ψ6 ψ7 ψ8 ψ9 ψ10 ˆ ψ1 ˆ ψ2 ˆ ψ3 ˆ ψ4 ˆ ψ5 ˆ ψ6 ˆ ψ7 ˆ ψ8 ˆ ψ9 ˆ ψ10

SLIDE 37

17/30