Co-Segmentation of 3D Shapes via Subspace Clustering Ruizhen Hu Lubin Fan Ligang Liu
Co-segmentation Input Hu et al. Co-Segmentation of 3D Shapes 2
Co-segmentation Output Hu et al. Co-Segmentation of 3D Shapes 3
Related works Consistent segmentation Supervised segmentation Shuffler [Kalogerakis et al. 2010] [Kraevoy et al. 2007] [Golovinskiy and Funkhouser 2009] Joint segmentation Supervised correspondence Style separation [Huang et al. 2011] [van Kaick et al. 2011] [Xu et al. 2010] Hu et al. Co-Segmentation of 3D Shapes 4
Related works • Co-segmentation [Sidi et al. 2011] – via Descriptor-Space Spectral Clustering Result Pre-segmentation Clustering Hu et al. Co-Segmentation of 3D Shapes 5
Motivation Unsupervised Over-segmentation [Huang et al. 2011] Hu et al. Co-Segmentation of 3D Shapes 6
Key observation • Corresponding patches lie in a common subspace Co-segmentation Subspace clustering AGD Hu et al. Co-Segmentation of 3D Shapes 7
Subspace clustering [Vidal 2010] • Input: – high dimensional datasets having low intrinsic dimensions – {𝑦 𝑘 } 𝑘=1,…,𝑂 , 𝑦 𝑢 ∈ ℝ 𝐸 • Output: – multiple low-dimensional linear subspaces – 𝑀, 𝑄 1 , 𝑄 2 Hu et al. Co-Segmentation of 3D Shapes 8
Sparse subspace clustering(SSC) [Elhamifar and Vidal 2009] • Based on the observation: – each point can always be represented as a linear combination of the points belonging to the same subspace 𝑌𝑋 − min 𝑂 min 𝑋 1,1 𝑋 𝑦 𝑘 = 𝑋 𝑥 𝑗𝑘 𝑦 𝑗 , 𝑘 = 1, … , 𝑂 s. t. 𝑌 = 𝑌𝑋, diag 𝑋 = 0 𝑗=1 where 𝑌 = 𝑦 1 , … , 𝑦 𝑂 ∈ ℝ 𝐸×𝑂 , 𝑋 = (𝑥 𝑗𝑘 ) ∈ ℝ 𝑂×𝑂 Hu et al. Co-Segmentation of 3D Shapes 9
SSQP [Wang et al. 2011] 𝑌𝑋 − 𝑌𝑋 − min min 𝑋 𝑋 • 𝑋 ≥ 0 : provides better interpretations 𝑋 𝑈 𝑋 1,1 : more efficient than SSC • • Block diagonal property: 𝑋 ∗1 0 𝑋 ∗2 𝑋 ∗ = Γ −1 𝑋Γ = ⋱ 𝑋 ∗𝐿 0 𝑂×𝑂 where Γ is a permutation matrix, submatrix 𝑋 ∗𝑙 ∈ ℝ 𝑂 𝑙 ×𝑂 𝑙 Hu et al. Co-Segmentation of 3D Shapes 10
Co-segmentation 𝑌𝑋 − min • Single feature: 𝑋 𝑂 𝑌 0 56 AGD 0 39 0 88 … ⋮ ⋮ 𝐸 56 0 45 0 87 0 𝑂 = #patch of all shapes in the set 135 0 𝐸 = dim of feature vector Hu et al. Co-Segmentation of 3D Shapes 11
Co-segmentation 𝑌𝑋 − min • Single feature: 𝑋 𝑂 𝑋 𝑇 = 𝑡 𝑗𝑘 , 𝑡 𝑗𝑘 = 𝑥 𝑗𝑘 + 𝑥 𝑘𝑗 𝑥 𝑗𝑘 𝑂 The NCut method is then applied to this affinity matrix 𝑇 to segment patches into 𝐿 clusters. [Shi and Malik 2000] Hu et al. Co-Segmentation of 3D Shapes 12
Choices of features • Different sets favor different features – Single feature is not enough CF [Kalogerakis et al. 2010] [Ben-Chen and Gostman 2008] Hu et al. Co-Segmentation of 3D Shapes 13
Multiple features Hu et al. Co-Segmentation of 3D Shapes 14
How to combine different features? • Traditional way: 1. concatenate all features into one descriptor 2. use single-feature subspace clustering algorithm • Problem: – Corresponding patches may not be similar in all features – Concatenated feature vectors may not lie in a common subspace any more Hu et al. Co-Segmentation of 3D Shapes 15
How to combine different features? • Our solution: – apply subspace clustering in each feature space – add the consistent multi-feature penalty 𝐼 min ℱ 𝑋 ℎ + 𝑄 𝑑𝑝𝑜𝑡 (𝑋 1 , 𝑋 2 , … , 𝑋 𝐼 ) 𝑋 1 ,…,𝑋 𝐼 ℎ=1 s. t. 𝑋 ℎ ≥ 0, diag 𝑋 ℎ = 0, ℎ = 1,2, … , 𝐼 2 + 𝜇 𝑋 𝑈 𝑋 where ℱ 𝑋 ℎ = 𝑌 ℎ 𝑋 ℎ − 𝑌 ℎ ℎ 1,1 𝐺 ℎ Hu et al. Co-Segmentation of 3D Shapes 16
Consistent multi-feature penalty 1. To find the most similar patch pairs 2. Corresponding patches need not be similar in all features 𝑄 𝑑𝑝𝑜𝑡 𝑋 1 , 𝑋 2 , … , 𝑋 𝐼 = α 𝑋 2,1 + 𝛾 𝑋 1,1 … (𝑋 1 ) 𝑂 2 (𝑋 1 ) 11 (𝑋 1 ) 12 … (𝑋 2 ) 𝑂 2 (𝑋 2 ) 11 (𝑋 2 ) 12 𝑋 = ⋮ ⋮ ⋮ ⋱ (𝑋 𝐼 ) 11 (𝑋 𝐼 ) 12 (𝑋 𝐼 ) 𝑂 2 … Hu et al. Co-Segmentation of 3D Shapes 17
Consistent multi-feature penalty • 𝑄 𝑑𝑝𝑜𝑡 𝑋 1 , 𝑋 2 , … , 𝑋 𝐼 = α 𝑋 2,1 + 𝛾 𝑋 1,1 𝑋 2,1 : • Induces column sparsity of 𝑋 • Identify the most similar patch pairs Hu et al. Co-Segmentation of 3D Shapes 18
Consistent multi-feature penalty • 𝑄 𝑑𝑝𝑜𝑡 𝑋 1 , 𝑋 2 , … , 𝑋 𝐼 = α 𝑋 2,1 + 𝛾 𝑋 1,1 𝑋 1,1 : • Induces the sparsity within each column of 𝑋 • Enables the prominent features to pop up Hu et al. Co-Segmentation of 3D Shapes 19
Co-segmentation • Multiple features: 𝐼 1 ,…,𝑋 𝐼 min ℱ 𝑋 ℎ + 𝑄 𝑑𝑝𝑜𝑡 (𝑋 1 , 𝑋 2 , … , 𝑋 𝐼 ) 𝑋 ℎ=1 s. t. 𝑋 ℎ ≥ 0, diag 𝑋 ℎ = 0, h = 1,2, … , H Affinity matrix: 𝐼 𝐼 𝑡 𝑗𝑘 = 1 2 + 2 ( ( 𝑇 = 𝑡 𝑗𝑘 , 𝑋 ℎ ) 𝑗𝑘 𝑋 ℎ ) 𝑘𝑗 2 ℎ=1 ℎ=1 Hu et al. Co-Segmentation of 3D Shapes 20
Results • 20 categories of shapes – 16 from PSB [Chen et al. 2009, Kalogerakis et al. 2010] – 4 from [Sidi et al. 2011] Hu et al. Co-Segmentation of 3D Shapes 21
Hu et al. Co-Segmentation of 3D Shapes 22
Hu et al. Co-Segmentation of 3D Shapes 23
Hu et al. Co-Segmentation of 3D Shapes 24
Evaluation & Comparisons Category Ours CFV Category Ours CFV Human 70.4 – Plier 86.0 68.9 Cup 97.4 85.0 Fish 85.6 66.5 Glasses 98.3 97.9 Bird 71.5 71.4 Airplane 83.3 75.3 Armadillo 87.3 – Ant 92.9 69.6 Vase 80.2 66.5 Chair 89.6 83.6 Fourleg 88.7 69.2 Octopus 97.5 95.3 Candelabra 93.9 44.2 Table 99.0 99.1 Goblet 99.2 59.8 Teddy 97.1 97.0 Guitar 98.0 90.0 Hand 91.9 88.2 Lamp 90.7 59.8 Average 90.4 – CFV : the subspace clustering technique on the concatenated feature vector Hu et al. Co-Segmentation of 3D Shapes 25
Comparisons • Supervised method: [Kalogerakis et al. 2010] Hu et al. Co-Segmentation of 3D Shapes 26
Comparisons • Unsupervised method: [Sidi et al. 2011] Hu et al. Co-Segmentation of 3D Shapes 27
Comparisons • Unsupervised method: [Sidi et al. 2011] Our algorithm [Sidi et al. 2011] Hu et al. Co-Segmentation of 3D Shapes 28
Limitations Cannot always: 1. distinguish two different parts with high geometric similarity 2. recognize corresponding parts with low geometric similarity Hu et al. Co-Segmentation of 3D Shapes 29
Conclusion • Key ideas: – Formulate co-segmentation as subspace clustering – Consistent multi-feature penalty • Advantages: – More flexible and efficient – Capable of handling more kinds of models – Results are better compared to previous unsupervised methods Hu et al. Co-Segmentation of 3D Shapes 30
Future work • Look for more semantic feature descriptors • Add control on the contribution of different features Hu et al. Co-Segmentation of 3D Shapes 31
Hu et al. Co-Segmentation of 3D Shapes 32
Recommend
More recommend
