Co-Segmentation of 3D Shapes via Subspace Clustering Ruizhen Hu - - PowerPoint PPT Presentation

Co-Segmentation of 3D Shapes via Subspace Clustering

Ruizhen Hu Lubin Fan Ligang Liu

Co-segmentation

Hu et al. Co-Segmentation of 3D Shapes 2

Input

Co-segmentation

Hu et al. Co-Segmentation of 3D Shapes 3

Output

Related works

Hu et al. Co-Segmentation of 3D Shapes 4

[Kraevoy et al. 2007]

Shuffler

[Huang et al. 2011]

Joint segmentation

[Golovinskiy and Funkhouser 2009]

Consistent segmentation

[Xu et al. 2010]

Style separation

[Kalogerakis et al. 2010]

Supervised segmentation

[van Kaick et al. 2011]

Supervised correspondence

Related works

• Co-segmentation

– via Descriptor-Space Spectral Clustering

Hu et al. Co-Segmentation of 3D Shapes 5

Pre-segmentation Result Clustering [Sidi et al. 2011]

Motivation

Over-segmentation

[Huang et al. 2011]

Unsupervised

Hu et al. Co-Segmentation of 3D Shapes 6

Key observation

Hu et al. Co-Segmentation of 3D Shapes 7

• Corresponding patches lie in a common subspace

AGD

Co-segmentation Subspace clustering

Subspace clustering

• Input:

– high dimensional datasets having low intrinsic dimensions – {𝑦𝑘}𝑘=1,…,𝑂, 𝑦𝑢 ∈ ℝ𝐸

Hu et al. Co-Segmentation of 3D Shapes 8

[Vidal 2010]

• Output:

– multiple low-dimensional linear subspaces – 𝑀, 𝑄

1, 𝑄2

Sparse subspace clustering(SSC)

• Based on the observation:

– each point can always be represented as a linear combination of the points belonging to the same subspace

Hu et al. Co-Segmentation of 3D Shapes 9

min

𝑋

𝑌𝑋 − 𝑦𝑘 =

𝑗=1 𝑂

𝑥𝑗𝑘𝑦𝑗 , 𝑘 = 1, … , 𝑂

[Elhamifar and Vidal 2009]

where 𝑌 = 𝑦1, … , 𝑦𝑂 ∈ ℝ𝐸×𝑂, 𝑋 = (𝑥𝑗𝑘) ∈ ℝ𝑂×𝑂

min

𝑋

𝑋 1,1

• s. t. 𝑌 = 𝑌𝑋, diag 𝑋 = 0

SSQP

Hu et al. Co-Segmentation of 3D Shapes 10

min

𝑋

𝑌𝑋 −

• 𝑋 ≥ 0: provides better interpretations
• 𝑋𝑈𝑋 1,1: more efficient than SSC
• Block diagonal property:

𝑋∗ = Γ−1 𝑋Γ = 𝑋∗1 𝑋∗2 ⋱ 𝑋∗𝐿

𝑂×𝑂

where Γ is a permutation matrix, submatrix 𝑋∗𝑙 ∈ ℝ𝑂𝑙×𝑂𝑙 min

𝑋

𝑌𝑋 −

[Wang et al. 2011]

Co-segmentation

• Single feature:

Hu et al. Co-Segmentation of 3D Shapes 11

min

𝑋

𝑌𝑋 −

𝑌

𝑂 𝐸

56 39 88 ⋮

…

⋮ 56 45 87 135

𝑂 = #patch of all shapes in the set 𝐸 = dim of feature vector AGD

Co-segmentation

• Single feature:

Hu et al. Co-Segmentation of 3D Shapes 12

min

𝑋

𝑌𝑋 −

𝑋

𝑂 𝑂

𝑥𝑗𝑘 𝑇 = 𝑡𝑗𝑘 , 𝑡𝑗𝑘 = 𝑥𝑗𝑘 + 𝑥

𝑘𝑗

The NCut method is then applied to this affinity matrix 𝑇 to segment patches into 𝐿 clusters.

[Shi and Malik 2000]

Choices of features

• Different sets favor different features

– Single feature is not enough

Hu et al. Co-Segmentation of 3D Shapes 13

[Kalogerakis et al. 2010] [Ben-Chen and Gostman 2008]

CF

Multiple features

Hu et al. Co-Segmentation of 3D Shapes 14

How to combine different features?

• Traditional way:
• 1. concatenate all features into one descriptor
• 2. use single-feature subspace clustering algorithm

Hu et al. Co-Segmentation of 3D Shapes 15

• Problem:

– Corresponding patches may not be similar in all features – Concatenated feature vectors may not lie in a common subspace any more

How to combine different features?

• Our solution:

– apply subspace clustering in each feature space – add the consistent multi-feature penalty min

𝑋

1,…,𝑋𝐼

ℎ=1 𝐼

ℱ 𝑋

ℎ + 𝑄 𝑑𝑝𝑜𝑡(𝑋 1, 𝑋 2, … , 𝑋 𝐼)

• s. t.

𝑋

ℎ ≥ 0, diag 𝑋 ℎ = 0,

ℎ = 1,2, … , 𝐼 where ℱ 𝑋

ℎ =

𝑌ℎ𝑋

ℎ − 𝑌ℎ 𝐺 2 + 𝜇 𝑋 ℎ 𝑈𝑋 ℎ 1,1

Hu et al. Co-Segmentation of 3D Shapes 16

Consistent multi-feature penalty

• 1. To find the most similar patch pairs
• 2. Corresponding patches need not be similar in all features

𝑄

𝑑𝑝𝑜𝑡 𝑋 1, 𝑋 2, … , 𝑋 𝐼 = α 𝑋 2,1 + 𝛾 𝑋 1,1

𝑋 = (𝑋

1)11

(𝑋

1)12

… (𝑋

1)𝑂2

(𝑋

2)11

(𝑋

2)12

… (𝑋

2)𝑂2

⋮ (𝑋

𝐼)11

⋮ (𝑋

𝐼)12

⋱ … ⋮ (𝑋

𝐼)𝑂2

Hu et al. Co-Segmentation of 3D Shapes 17

Consistent multi-feature penalty

• 𝑄

𝑑𝑝𝑜𝑡 𝑋 1, 𝑋 2, … , 𝑋 𝐼 = α 𝑋 2,1 + 𝛾 𝑋 1,1

Hu et al. Co-Segmentation of 3D Shapes 18

𝑋 2,1:

• Induces column sparsity
• f 𝑋
• Identify the most similar

patch pairs

Consistent multi-feature penalty

• 𝑄

𝑑𝑝𝑜𝑡 𝑋 1, 𝑋 2, … , 𝑋 𝐼 = α 𝑋 2,1 + 𝛾 𝑋 1,1

Hu et al. Co-Segmentation of 3D Shapes 19

𝑋 1,1:

• Induces the sparsity

within each column of 𝑋

• Enables the prominent

features to pop up

Co-segmentation

• Multiple features:

Affinity matrix: 𝑇 = 𝑡𝑗𝑘 , 𝑡𝑗𝑘 = 1 2

ℎ=1 𝐼

( 𝑋

ℎ)𝑗𝑘 2 + ℎ=1 𝐼

( 𝑋

ℎ)𝑘𝑗 2

Hu et al. Co-Segmentation of 3D Shapes 20

min

𝑋

1,…,𝑋𝐼

ℎ=1 𝐼

ℱ 𝑋

ℎ + 𝑄 𝑑𝑝𝑜𝑡(𝑋 1, 𝑋 2, … , 𝑋 𝐼)

• s. t.

𝑋

ℎ ≥ 0, diag 𝑋 ℎ = 0,

h = 1,2, … , H

Results

• 20 categories of shapes

– 16 from PSB [Chen et al. 2009, Kalogerakis et al. 2010] – 4 from [Sidi et al. 2011]

Hu et al. Co-Segmentation of 3D Shapes 21

Hu et al. Co-Segmentation of 3D Shapes 22

Hu et al. Co-Segmentation of 3D Shapes 23

Hu et al. Co-Segmentation of 3D Shapes 24

Evaluation & Comparisons

Category Ours CFV Category Ours CFV Human 70.4 – Plier 86.0 68.9 Cup 97.4 85.0 Fish 85.6 66.5 Glasses 98.3 97.9 Bird 71.5 71.4 Airplane 83.3 75.3 Armadillo 87.3 – Ant 92.9 69.6 Vase 80.2 66.5 Chair 89.6 83.6 Fourleg 88.7 69.2 Octopus 97.5 95.3 Candelabra 93.9 44.2 Table 99.0 99.1 Goblet 99.2 59.8 Teddy 97.1 97.0 Guitar 98.0 90.0 Hand 91.9 88.2 Lamp 90.7 59.8 Average 90.4 –

Hu et al. Co-Segmentation of 3D Shapes 25

CFV: the subspace clustering technique on the concatenated feature vector

Comparisons

• Supervised method: [Kalogerakis et al. 2010]

Hu et al. Co-Segmentation of 3D Shapes 26

Comparisons

• Unsupervised method: [Sidi et al. 2011]

Hu et al. Co-Segmentation of 3D Shapes 27

Comparisons

• Unsupervised method: [Sidi et al. 2011]

Hu et al. Co-Segmentation of 3D Shapes 28

Our algorithm [Sidi et al. 2011]

Limitations

Cannot always:

• 1. distinguish two different parts with high geometric similarity
• 2. recognize corresponding parts with low geometric similarity

Hu et al. Co-Segmentation of 3D Shapes 29

Conclusion

• Key ideas:

– Formulate co-segmentation as subspace clustering – Consistent multi-feature penalty

• Advantages:

– More flexible and efficient – Capable of handling more kinds of models – Results are better compared to previous unsupervised methods

Hu et al. Co-Segmentation of 3D Shapes 30

Future work

• Look for more semantic feature descriptors
• Add control on the contribution of different features

Hu et al. Co-Segmentation of 3D Shapes 31

Hu et al. Co-Segmentation of 3D Shapes 32