SLIDE 1
3D-CODED: 3D Correspondences by Deep Deformation
ECCV, 2018
Outline
- Abstract
- Introduction
- Related work
- Method
- Results
- Experiments
- Conclusion
ECCV, 2018 Outline Abstract Introduction Related work Method - - PowerPoint PPT Presentation
ECCV, 2018 Outline Abstract Introduction Related work Method - - PowerPoint PPT Presentation
3D-CODED: 3D Correspondences by Deep Deformation ECCV, 2018 Outline Abstract Introduction Related work Method Results Experiments Conclusion Abstract This paper proposes a new deep learning approach for matching
SLIDE 2
SLIDE 3
Abstract
- This paper proposes a new deep learning
approach for matching deformable shapes by introducing Shape Deformation Networks which jointly encode 3D shapes and correspondences.
SLIDE 4
Introduction
- This paper proposes Shape Deformation
Networks, a comprehensive, all-in-one solution to template-driven shape matching. A Shape Deformation Network learns to deform a template shape to align with an input observed
- shape. Given two input shapes, they align the
template to both inputs and obtain the final map between the inputs by reading off the correspondences from the template.
SLIDE 5
Related work
SLIDE 6
Related work
Generic shape matching
- To estimate correspondences between
articulated objects, it is common to assume that their intrinsic structure remains relatively consistent across all poses.
- HKS, WKS, conformal maps, heat kernel maps
SLIDE 7
Related work
Template-based shape matching
- While such template- based techniques provide
the best correspondence results, they require a careful parameterization of the template.
- Fitting this representation to an input 3D shape
requires also designing an objective function that is typically non-convex and involves multiple terms to guide the optimization to the right global minima.
SLIDE 8
Related work
Deep learning for 3D data
- Existing networks operate on various shape
representations
- Volumetric grids
- Point clouds
- Geometry images
- Multi-view images
SLIDE 9
Method
SLIDE 10
Key ideas
- Learn to predict a transformation between shapes
instead of directly learning the correspondences.
- To learn transformations only from one template to
any shape.
SLIDE 11
Method Overview
SLIDE 12
Encoder
g
γ
MLP MLP MLP MLP MLP
max
…
h
𝑦1 𝑦2 𝑦… 𝑦𝑜
Input point cloud Global representation
SLIDE 13
Shape Deformation using Decoder
Global representation Decoder
y z y z x x
SLIDE 14
Total Loss for case 1:
correspondences between the training data are available
Where the sums are over all P vertices of all N example shapes
SLIDE 15
Total Loss for case 2:
correspondences between the training data are not available
Reconstructed Loss: Minimize the chamfer distance between the input shape and the reconstructed one Laplacian Loss: Encourage the Laplacian operator defined on the template and the deformed template to be the same (which is the case for isometric deformations of the surface) Edge Loss: Encourage the ratio between edges length in the template and its deformed version to be close to 1
SLIDE 16
Refinement
Given parameters of the encoder and the decoder, minimize with respect to the global feature x the Chamfer distance between the reconstructed shape and the input.
SLIDE 17
Finding shape correspondences between 2 shapes
1. Get the reconstructed shape 2. Refine the reconstructed shape
SLIDE 18
Results
SLIDE 19
On testing data
Input Shape Deformed Template After refinement
SLIDE 20
On other datasets
SLIDE 21
Interesting contribution
SLIDE 22
Experiments
SLIDE 23
The importance of initial guess
SLIDE 24
The importance of regularization term
Input shape Point cloud after optimization Mesh after optimization Point cloud after optimization + Regularization Mesh after optimization + Regularization
SLIDE 25
Conclusion
- An encoder-decoder deep network architecture can
generate human shape correspondences using only simple reconstruction and correspondence losses