partial functional correspondence
play

Partial functional correspondence Michael Bronstein University of - PowerPoint PPT Presentation

Partial functional correspondence Michael Bronstein University of Lugano Intel Corporation Lyon, 7 July 2016 1/59 Microsoft Kinect 2010 (Acquired by Intel in 2012) 4/59 Different form factor computers featuring Intel RealSense 3D camera


  1. Perturbation analysis: intuition ∆ M φ 1 φ 2 φ 3 M ¯ M ∆ M φ 1 φ 2 φ 3 ¯ ¯ ¯ ∆ ¯ φ 1 φ 2 φ 3 M Ignoring boundary interaction: disjoint parts (block-diagonal matrix) Eigenvectors = Mixture of eigenvectors of the parts Rodol` a, Cosmo, B, Torsello, Cremers 2016 26/59

  2. Perturbation analysis: eigenvalues 8 . 00 · 10 − 2 M 6 . 00 4 . 00 r k N 2 . 00 0 . 00 10 20 30 40 50 eigenvalue number k ≈ |M| Slope r |N | (depends on the area of the cut) Consistent with Weil’s law Rodol` a, Cosmo, B, Torsello, Cremers 2016 27/59

  3. Perturbation analysis: details ∆ M M t E ¯ ∆ M + t D M M t E ⊤ ∆ ¯ M + t D ¯ M ∆ ¯ M Rodol` a, Cosmo, B, Torsello, Cremers 2016 28/59

  4. Perturbation analysis: details ∆ M M t E ¯ ∆ M + t D M M t E ⊤ ∆ ¯ M + t D ¯ M ∆ ¯ M “How would the Laplacian eigenvalues and eigenvectors of the red part change if we attached a blue part to it?” Rodol` a, Cosmo, B, Torsello, Cremers 2016 28/59

  5. Perturbation analysis: details P M P M n × n n × ¯ n D M E ¯ M “How would the Laplacian eigenvalues and eigenvectors of the red part change if we attached a blue part to it?” Rodol` a, Cosmo, B, Torsello, Cremers 2016 28/59

  6. Perturbation analysis: details M = ¯ Φ ¯ Λ ¯ Denote ∆ M + t P M = Φ ( t ) Λ ( t ) Φ ( t ) ⊤ , ∆ ¯ Φ ⊤ , Φ = Φ (0) , and Λ = Λ (0) . Theorem 1 (eigenvalues) The derivative of the non-trivial eigenvalues is given by � 0 � d 0 dtλ i = φ ⊤ i P M φ i P M = 0 D M Rodol` a, Cosmo, B, Torsello, Cremers 2016 29/59

  7. Perturbation analysis: details M = ¯ Φ ¯ Λ ¯ Denote ∆ M + t P M = Φ ( t ) Λ ( t ) Φ ( t ) ⊤ , ∆ ¯ Φ ⊤ , Φ = Φ (0) , and Λ = Λ (0) . Theorem 1 (eigenvalues) The derivative of the non-trivial eigenvalues is given by � 0 � d 0 dtλ i = φ ⊤ i P M φ i P M = 0 D M Theorem 2 (eigenvectors) Assuming λ i � = λ j for i � = j and λ i � = ¯ λ j for all i, j , the derivative of the non-trivial eigenvectors is given by � 0 ¯ i P ¯ n φ ⊤ n φ ⊤ � i P M φ j d φ j � � 0 ¯ dt φ i = φ j + φ j P = λ i − ¯ λ i − λ j E 0 λ j j =1 j =1 j � = i Rodol` a, Cosmo, B, Torsello, Cremers 2016 29/59

  8. Perturbation analysis: boundary interaction strength 20 10 Value of f Eigenvector perturbation depends on length and position of the boundary Perturbation strength � d dt Φ � F ≤ c � ∂ M f ( m ) dm , where n � φ i ( m ) φ j ( m ) � 2 � f ( m ) = λ i − λ j i,j =1 j � = i Rodol` a, Cosmo, B, Torsello, Cremers 2016 30/59

  9. Laplacian perturbation: typical picture Plate Punctured plate Figure: Filoche, Mayboroda 2009 31/59

  10. Partial functional maps Model shape M T Query shape N Part M ⊆ M ≈ isometric to N M N Data F , G Partial functional map TG ≈ F ( M ) M Rodol` a, Cosmo, B, Torsello, Cremers 2016 32/59

  11. Partial functional maps Model shape M C Query shape N Part M ⊆ M ≈ isometric to N M N Data F , G Partial functional map G ⊤ ΨC ≈ F ( M ) ⊤ Φ M Rodol` a, Cosmo, B, Torsello, Cremers 2016 32/59

  12. Partial functional maps Model shape M C Query shape N Part M ⊆ M ≈ isometric to N v N Data F , G Partial functional map G ⊤ ΨC ≈ F ⊤ diag( v ) Φ v ∈ F ( M ) indicator function of M M Rodol` a, Cosmo, B, Torsello, Cremers 2016 32/59

  13. Partial functional maps Model shape M C Query shape N Part M ⊆ M ≈ isometric to N v N Data F , G Partial functional map G ⊤ ΨC ≈ F ⊤ diag( η ( v )) Φ v ∈ F ( M ) indicator function of M M η ( t ) = 1 2 (tanh(2 t − 1) + 1) Optimization problem w.r.t. correspondence C and part v C , v � G ⊤ ΨC − F ⊤ diag( η ( v )) Φ � 2 , 1 + ρ corr ( C ) + ρ part ( v ) min Rodol` a, Cosmo, B, Torsello, Cremers 2016 32/59

  14. Partial functional maps C , v � G ⊤ ΨC − F ⊤ diag( η ( v )) Φ � 2 , 1 + ρ corr ( C ) + ρ part ( v ) min Rodol` a, Cosmo, B, Torsello, Cremers 2016 33/59

  15. Partial functional maps C , v � G ⊤ ΨC − F ⊤ diag( η ( v )) Φ � 2 , 1 + ρ corr ( C ) + ρ part ( v ) min Part regularization � � 2 � � ρ part ( v ) = µ 1 |N| − η ( v ) dm + µ 2 ξ ( v ) �∇ M v � dm M M � � η ( t ) − 1 where ξ ( t ) ≈ δ 2 Rodol` a, Cosmo, B, Torsello, Cremers 2016; BB 2008 33/59

  16. Partial functional maps C , v � G ⊤ ΨC − F ⊤ diag( η ( v )) Φ � 2 , 1 + ρ corr ( C ) + ρ part ( v ) min Part regularization � � 2 � � ρ part ( v ) = µ 1 |N| − η ( v ) dm + µ 2 ξ ( v ) �∇ M v � dm M M � �� � � �� � area preservation Mumford − Shah � � η ( t ) − 1 where ξ ( t ) ≈ δ 2 Rodol` a, Cosmo, B, Torsello, Cremers 2016; BB 2008 33/59

  17. Partial functional maps C , v � G ⊤ ΨC − F ⊤ diag( η ( v )) Φ � 2 , 1 + ρ corr ( C ) + ρ part ( v ) min Part regularization � � 2 � � ρ part ( v ) = µ 1 |N| − η ( v ) dm + µ 2 ξ ( v ) �∇ M v � dm M M � �� � � �� � area preservation Mumford − Shah � � η ( t ) − 1 where ξ ( t ) ≈ δ 2 Correspondence regularization � � ρ corr ( C ) = µ 3 � C ◦ W � 2 ( C ⊤ C ) 2 (( C ⊤ C ) ii − d i ) 2 F + µ 4 ij + µ 5 i � = j i Rodol` a, Cosmo, B, Torsello, Cremers 2016; BB 2008 33/59

  18. Partial functional maps C , v � G ⊤ ΨC − F ⊤ diag( η ( v )) Φ � 2 , 1 + ρ corr ( C ) + ρ part ( v ) min Part regularization � � 2 � � ρ part ( v ) = µ 1 |N| − η ( v ) dm + µ 2 ξ ( v ) �∇ M v � dm M M � �� � � �� � area preservation Mumford − Shah � � η ( t ) − 1 where ξ ( t ) ≈ δ 2 Correspondence regularization � � ρ corr ( C ) = µ 3 � C ◦ W � 2 ( C ⊤ C ) 2 (( C ⊤ C ) ii − d i ) 2 + µ 4 + µ 5 F ij � �� � i � = j i slant � �� � � �� � rank ≈ r ≈ orthogonality Rodol` a, Cosmo, B, Torsello, Cremers 2016; BB 2008 33/59

  19. Structure of partial functional correspondence 4 2 0 0 20 40 60 80 100 C ⊤ C C W singular values Rodol` a, Cosmo, B, Torsello, Cremers 2016 34/59

  20. Alternating minimization C -step: fix v ∗ , solve for correspondence C C � G ⊤ ΨC − F ⊤ diag( η ( v ∗ )) Φ � 2 , 1 + ρ corr ( C ) min v -step: fix C ∗ , solve for part v v � G ⊤ ΨC ∗ − F ⊤ diag( η ( v )) Φ � 2 , 1 + ρ part ( v ) min Rodol` a, Cosmo, B, Torsello, Cremers 2016 35/59

  21. Alternating minimization C -step: fix v ∗ , solve for correspondence C C � G ⊤ ΨC − F ⊤ diag( η ( v ∗ )) Φ � 2 , 1 + ρ corr ( C ) min v -step: fix C ∗ , solve for part v v � G ⊤ ΨC ∗ − F ⊤ diag( η ( v )) Φ � 2 , 1 + ρ part ( v ) min Iteration 1 2 3 4 Rodol` a, Cosmo, B, Torsello, Cremers 2016 35/59

  22. Example of convergence Time (sec.) 0 5 10 15 20 25 10 10 C -step 10 9 v -step 10 8 Energy 10 7 10 6 10 5 10 4 0 20 40 60 80 100 Iteration Rodol` a, Cosmo, B, Torsello, Cremers 2016 36/59

  23. Examples of partial functional maps Rodol` a, Cosmo, B, Torsello, Cremers 2016 37/59

  24. Examples of partial functional maps Rodol` a, Cosmo, B, Torsello, Cremers 2016 37/59

  25. Examples of partial functional maps Rodol` a, Cosmo, B, Torsello, Cremers 2016 37/59

  26. Examples of partial functional maps Rodol` a, Cosmo, B, Torsello, Cremers 2016 37/59

  27. Partial functional maps vs Functional maps 100 150 100 80 50 % Correspondences PFM 60 Func. maps 40 50 100 20 150 0 0 0.05 0.1 0.15 0.2 0.25 Geodesic error Correspondence performance for different rank values k Rodol` a, Cosmo, B, Torsello, Cremers 2016 38/59

  28. Partial correspondence performance Cuts Holes 100 % Correspondences 80 60 40 20 0 0 0.05 0.1 0.15 0.2 0.25 0 0.05 0.1 0.15 0.2 0.25 Geodesic Error Geodesic Error PFM RF IM EN GT SHREC’16 Partial Matching benchmark Rodol` a et al. 2016; Methods: Rodol` a, Cosmo, B, Torsello, Cremers 2016 ( PFM ); Sahillio˘ glu, Yemez 2012 (IM); Rodol` a, Bronstein, Albarelli, Bergamasco, Torsello 2012 (GT); Rodol` a et al. 2013 (EN); Rodol` a et al. 2014 (RF) 39/59

  29. Partial correspondence performance Cuts Holes 1 Mean geodesic error 0 . 8 0 . 6 0 . 4 0 . 2 0 20 40 60 80 20 40 60 80 Partiality (%) Partiality (%) PFM RF IM EN GT SHREC’16 Partial Matching benchmark Rodol` a et al. 2016; Methods: Rodol` a, Cosmo, B, Torsello, Cremers 2016 ( PFM ); Sahillio˘ glu, Yemez 2012 (IM); Rodol` a, Bronstein, Albarelli, Bergamasco, Torsello 2012 (GT); Rodol` a et al. 2013 (EN); Rodol` a et al. 2014 (RF) 40/59

  30. Deep learning + Partial functional maps Correspondence 0 . 1 0 . 0 Correspondence error Boscaini, Masci, Rodol` a, B 2016 41/59

  31. Deep learning + Partial functional maps Correspondence 0 . 1 0 . 0 Correspondence error Boscaini, Masci, Rodol` a, B 2016 42/59

  32. Outline Background: Spectral analysis on manifolds Functional correspondence Partial functional correspondence Non-rigid puzzles 43/59

  33. Litani, BB 2012

  34. Partial correspondence Rodol` a, Cosmo, B, Torsello, Cremers 2016 45/59

  35. Non-rigid puzzle Litani, Rodol` a, BB, Cremers 2016 45/59

  36. Partial Laplacian eigenvectors Functional correspondence matrix C Rodol` a, Cosmo, B, Torsello, Cremers 2016 46/59

  37. Key observation M N N M C N N C M M slant ∝ | N | slant ∝ | M | |N| |M| Litani, Rodol` a, BB, Cremers 2016 47/59

  38. Key observation M N N M C NM = C N N C N M C M M slant ∝ | N | |M| |N| | M | Litani, Rodol` a, BB, Cremers 2016 47/59

  39. Key observation M N N M C NM = C N N C N M C M M slant ∝ | N | |M| | M | = |N| |N| |M| Litani, Rodol` a, BB, Cremers 2016 47/59

  40. Non-rigid puzzles problem formulation Model Parts Input M 1 C 1 Model M N 1 N c Parts N 1 , . . . , N p 2 C 2 M 2 N c 1 N 2 Output N 1 Segmentation M i ⊆ M M 0 Located parts N i ⊆ N i Correspondences C i Clutter N c i M N 2 Missing parts M 0 Litani, Rodol` a, BB, Cremers 2016 48/59

  41. Non-rigid puzzles problem formulation Model Parts Data F i , G i M 1 C 1 Model basis Φ , Φ ( M i ) N 1 N c Part bases Ψ i , Ψ i ( N i ) 2 C 2 M 2 N c 1 N 2 Data term N 1 F ⊤ i Φ ( M i ) ≈ G ⊤ i Ψ i ( N i ) C i M 0 M N 2 Litani, Rodol` a, BB, Cremers 2016 48/59

  42. Non-rigid puzzles problem formulation p � � G ⊤ Ψ i ( N i ) C i − F ⊤ min i Ψ Ψ i Φ Φ Φ( M i ) � 2 , 1 C i i =1 M i ⊆M ,N i ⊆N i p p � � + λ M ρ part ( M i ) + λ N ρ part ( N i ) i =0 i =1 p � + λ corr ρ corr ( C i ) i =1 s.t. M i ∩ M j = ∅ ∀ i � = j M 0 ∪ M 1 ∪ · · · ∪ M p = M | M i | = | N i | ≥ α |N i | , Litani, Rodol` a, BB, Cremers 2016 49/59

  43. Non-rigid puzzles problem formulation p � � G ⊤ Ψ i C i − F ⊤ min i diag( η ( u i ))Ψ Ψ i diag( η ( v i ))Φ Φ Φ � 2 , 1 C i i =1 u i , v i p p � � + λ M ρ part ( η ( v i )) + λ N ρ part ( η ( u i )) i =0 i =1 p � + λ corr ρ corr ( C i ) i =1 p � s.t. η ( u i ) = 1 i =1 a ⊤ M u i = a ⊤ N v i ≥ α a ⊤ N i 1 Litani, Rodol` a, BB, Cremers 2016 49/59

  44. Convergence example Outer iteration 1 Litani, Rodol` a, BB, Cremers 2016 50/59

  45. Convergence example Outer iteration 2 Litani, Rodol` a, BB, Cremers 2016 50/59

  46. Convergence example Outer iteration 3 Litani, Rodol` a, BB, Cremers 2016 50/59

  47. Convergence example Time (sec) 30 32 34 36 38 40 42 44 46 48 80 90 100 110 120 130 140 150 160 Iteration number Litani, Rodol` a, BB, Cremers 2016 51/59

  48. “Perfect puzzle” example Model/Part Synthetic (TOSCA) Transformation Isometric Clutter No Missing part No Data term Dense (SHOT) Litani, Rodol` a, BB, Cremers 2016 52/59

  49. “Perfect puzzle” example Model/Part Synthetic (TOSCA) Transformation Isometric Clutter No Missing part No Data term Dense (SHOT) Segmentation Litani, Rodol` a, BB, Cremers 2016 52/59

  50. “Perfect puzzle” example Model/Part Synthetic (TOSCA) Transformation Isometric Clutter No Missing part No Data term Dense (SHOT) Correspondence Litani, Rodol` a, BB, Cremers 2016 52/59

  51. Overlapping parts example Model/Part Synthetic (FAUST) Transformation Near-isometric Clutter Yes (overlap) Missing part No Data term Dense (SHOT) Segmentation Litani, Rodol` a, BB, Cremers 2016 53/59

  52. Overlapping parts example Model/Part Synthetic (FAUST) Transformation Near-isometric Clutter Yes (overlap) Missing part No Data term Dense (SHOT) Correspondence Litani, Rodol` a, BB, Cremers 2016 53/59

  53. Overlapping parts example Model/Part Synthetic (FAUST) Transformation Near-isometric Clutter Yes (overlap) Missing part No Data term Dense (SHOT) 0.1 0.0 Correspondence error Litani, Rodol` a, BB, Cremers 2016 53/59

  54. Missing parts example Model/Part Synthetic (TOSCA) Transformation Isometric Clutter Yes (extra part) Missing part Yes Data term Dense (SHOT) Litani, Rodol` a, BB, Cremers 2016 54/59

  55. Missing parts example Model/Part Synthetic (TOSCA) Transformation Isometric Clutter Yes (extra part) Missing part Yes Data term Dense (SHOT) Segmentation Litani, Rodol` a, BB, Cremers 2016 54/59

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend


More recommend