Neu Neural C al Collab ollabor orativ ive e Subspace ce - - PowerPoint PPT Presentation

▶

Dec 11, 2023 98 likes •272 views

Neu Neural C al Collab ollabor orativ ive e Subspace ce Clustering Tong Zhang, Pan Ji , Mehrtash Harandi, Wenbing Huang, Hongdong Li International Conference of Machine Learning (ICML), Long Beach, CA, June 10-15, 2019 Subspace Clustering

SLIDE 1

International Conference of Machine Learning (ICML), Long Beach, CA, June 10-15, 2019

Neu Neural C al Collab

llabor
rativ

ive e Subspace ce Clustering

Tong Zhang, Pan Ji, Mehrtash Harandi, Wenbing Huang, Hongdong Li

SLIDE 2

International Conference of Machine Learning (ICML), Long Beach, CA, June 10-15, 2019

Subspace Clustering

Cluster data points drawn from a union of low-dimensional subspaces
Applications: image clustering, motion segmentation, etc.

SLIDE 3

International Conference of Machine Learning (ICML), Long Beach, CA, June 10-15, 2019

Subspace Clustering Methods

STOA methods consist of two steps:
1. Construct an affinity matrix for the whole dataset,
2. Apply normalized cuts or spectral clustering.

SLIDE 4

International Conference of Machine Learning (ICML), Long Beach, CA, June 10-15, 2019

Scalability Issue!!

Affinity matrix construction is expensive:
Large memory footprint;
High complexity in optimization.
Spectral clustering is expensive:
Computing SVD on large matrices is demanding.

SLIDE 5

International Conference of Machine Learning (ICML), Long Beach, CA, June 10-15, 2019

Can we avoid the construction of huge affinity matrices and bypass the spectral clustering?

SLIDE 6

International Conference of Machine Learning (ICML), Long Beach, CA, June 10-15, 2019

Our Idea

Construct affinity matrix in a batch;
Train a classifier using affinity matrices.

SLIDE 7

International Conference of Machine Learning (ICML), Long Beach, CA, June 10-15, 2019

Our Idea

Construct affinity matrix in a batch;
Train a classifier using affinity matrices.
How?

SLIDE 8

International Conference of Machine Learning (ICML), Long Beach, CA, June 10-15, 2019

Affinity from Classification

Build connection between clustering and classification via affinity

matrices

Classification Affinity Matrix from Classification

!"#$(&, () = +

SLIDE 9

International Conference of Machine Learning (ICML), Long Beach, CA, June 10-15, 2019

Affinity from Subspace

Subspace affinity

from self-expressiveness !"#$

input

Encoder

Z ZC Decoder

utput

Self-Expressive Layer

'∗ !"#$

SLIDE 10

International Conference of Machine Learning (ICML), Long Beach, CA, June 10-15, 2019

Collaborative Learning

Subspace affinity is more confident of identifying samples from the

same class. !"#$ !%&'

Positive Confidence

SLIDE 11

International Conference of Machine Learning (ICML), Long Beach, CA, June 10-15, 2019

Collaborative Learning (cont’d)

Classification affinity is more confident of identifying samples from

different classes. !"#$ !%&'

Negative Confidence

SLIDE 12

International Conference of Machine Learning (ICML), Long Beach, CA, June 10-15, 2019

Our Framework: Collaborative Learning

Data Batch

SLIDE 13

International Conference of Machine Learning (ICML), Long Beach, CA, June 10-15, 2019

Clustering via Classifier

Output the clustering simply through the classification part (bypass

the spectral clustering): !" = $%&'$() *

") , ℎ = 1, ⋯ , /,

where / is number of clusters.

SLIDE 14

International Conference of Machine Learning (ICML), Long Beach, CA, June 10-15, 2019

Experiments:

MNIST
Fashion-MNIST

SLIDE 15

International Conference of Machine Learning (ICML), Long Beach, CA, June 10-15, 2019

Conclusion

Subspace is a powerful tool to represent data in high-dimensional

space.

Introduced a collaborative learning paradigm for clustering.
Made subspace clustering scalable through batch-wise training.

SLIDE 16

International Conference of Machine Learning (ICML), Long Beach, CA, June 10-15, 2019

We’re hiring! For more details, visit http://www.nec- labs.com/research-departments/media-analytics.