SLIDE 1
Laplacian K-modes (LK) [Wang and Carreira-Perpiñán 2014]
2
Scalable Laplacian K-modes Imtiaz Masud Ziko, Eric Granger and - - PowerPoint PPT Presentation
Scalable Laplacian K-modes Imtiaz Masud Ziko, Eric Granger and - - PowerPoint PPT Presentation
Scalable Laplacian K-modes Imtiaz Masud Ziko, Eric Granger and Ismail Ben Ayed Laplacian K-modes (LK) [ Wang and Carreira-Perpin 2014 ] 2 Laplacian K-modes (LK) [ Wang and Carreira-Perpin 2014 ] K-modes Mode ( ) 2 Laplacian K-modes
SLIDE 2
SLIDE 3
Laplacian K-modes (LK) [Wang and Carreira-Perpiñán 2014]
K-modes
Mode ( )
2
SLIDE 4
Laplacian K-modes (LK) [Wang and Carreira-Perpiñán 2014]
K-modes Laplacian
Zhu ‘02, Weston ‘08, Shi ‘00, Belkin ‘03, ‘06 etc
Mode ( )
2
SLIDE 5
Laplacian K-modes (LK) [Wang and Carreira-Perpiñán 2014]
K-modes Laplacian
3
SLIDE 6
Laplacian K-modes (LK) [Wang and Carreira-Perpiñán 2014]
K-modes Laplacian
Discrete Simplex constraint
3
SLIDE 7
K-means LK ★ Handles non convex (manifold structured) clusters.
4
Why Laplacian K-modes?
SLIDE 8
K-means LK ★ Handles non convex (manifold structured) clusters. ★ Mean or Mode ?
4
Why Laplacian K-modes?
SLIDE 9
★ Handles non convex (manifold structured) clusters. ★ Mean or Mode ? ☑ Prototypes from input set
4
Why Laplacian K-modes?
SLIDE 10
Laplacian k-modes: challenges
😓 Challenging Optimization problem: 😓 simplex/integer constraint. 😓 Dependance of modes on 😓 Laplacian over discrete variable!
5
SLIDE 11
Laplacian k-modes: challenges
😓 Challenging Optimization problem: 😋 Well-known Spectral relaxation [Shi & Malik ‘00] :
5
SLIDE 12
Laplacian k-modes: challenges
😓 Challenging Optimization problem: 😓 Eigen-decomposition of Laplacian (N x N ).
5
😋 Well-known Spectral relaxation [Shi & Malik ‘00] :
SLIDE 13
Laplacian k-modes: challenges
😓 Challenging Optimization problem: 😋 Convex relaxation (relax integer constraint) [Wang and Carreira-Perpiñán ‘14] : 😓 Solve over N x L variables altogether. 😓 Projection to L-dimensional simplex. 😓 Eigen-decomposition of Laplacian (N x N ). 😋 Well-known Spectral relaxation [Shi & Malik ‘00] :
5
Not applicable in large scale clustering 😓
SLIDE 14
Laplacian k-modes: challenges
😓 Challenging Optimization problem: 😓 Solve over N x L variables altogether. 😓 Projection to L-dimensional simplex.
👊
We Tackle Concave Relaxation
5
😓 Eigen-decomposition of Laplacian (N x N ). 😋 Convex relaxation (relax integer constraint) [Wang and Carreira-Perpiñán ‘14] : 😋 Well-known Spectral relaxation [Shi & Malik ‘00] :
SLIDE 15
Laplacian k-modes: challenges
😓 Challenging Optimization problem: 😓 Solve over N x L variables altogether. 😓 Projection to L-dimensional simplex.
👊 👊
We Tackle Parallel structure
5
😋 Well-known Spectral relaxation [Shi & Malik ‘00] : 😓 Eigen-decomposition of Laplacian (N x N ). 😋 Convex relaxation (relax integer constraint) [Wang and Carreira-Perpiñán ‘14] :
SLIDE 16
Laplacian k-modes: challenges
😓 Challenging Optimization problem: 😓 Solve over N x L variables altogether. 😓 Projection to L-dimensional simplex.
👊 👊 👊
We Tackle avoid
5
😓 Eigen-decomposition of Laplacian (N x N ). 😋 Convex relaxation (relax integer constraint) [Wang and Carreira-Perpiñán ‘14] : 😋 Well-known Spectral relaxation [Shi & Malik ‘00] :
SLIDE 17
SLK Concave-Convex Relaxation
Laplacian
6
SLIDE 18
SLK Concave-Convex Relaxation
Laplacian = =
6
Concave relaxation (ours) Direct convex relaxaton When
SLIDE 19
SLK Concave-Convex Relaxation
Laplacian = = When = When
6
Concave relaxation (ours) Direct convex relaxaton
SLIDE 20
SLK Concave-Convex Relaxation
Laplacian = = =
6
Concave relaxation (ours) Direct convex relaxaton When When
SLIDE 21
SLK Concave-Convex Relaxation
7
SLIDE 22
SLK Concave-Convex Relaxation
Laplacian
7
SLIDE 23
SLK Concave-Convex Relaxation
Laplacian
7
concave
SLIDE 24
SLK Concave-Convex Relaxation
Laplacian
7
concave Linear bound
SLIDE 25
SLK Concave-Convex Relaxation
Laplacian
7
concave Linear bound
SLIDE 26
SLK Concave-Convex Relaxation
Laplacian
7
concave Linear bound
K-modes
SLIDE 27
SLK Concave-Convex Relaxation
Laplacian
7
concave
K-modes
SLIDE 28
SLK Concave-Convex Relaxation
Laplacian
7
concave
K-modes
convex
SLIDE 29
SLK Concave-Convex Relaxation
Laplacian
7
concave
K-modes
convex 👊 Avoids extra dual variables for constraints: 👊 Closed- form update duel :
SLIDE 30
SLK Proposed bound
Iterative bound: Where,
8
SLIDE 31
SLK Proposed bound
Iterative bound: Where,
Sum of independent function
8
SLIDE 32
SLK Proposed bound
Independent Iterative bound:
9
SLIDE 33
SLK Proposed bound
Independent Iterative bound:
👊 KKT conditions get closed form solution:
10
SLIDE 34
SLK-BO
Modes as byproducts of the formulated z-updates: 👊 In z - updates:
11
SLIDE 35
SLK-BO
Modes as byproducts of the formulated z-updates: 👊 In z - updates: 👊 take the form of soft approximation of hard max as:
11
SLIDE 36
SLK-BO
Modes as byproducts of the formulated z-updates: 👊 take the form of soft approximation of hard max as:
Unlike Mean-shift : ☑ No gradient ascent iterates ☑ Independent of feature dimensions ☑ Arbitrary kernels
11
Linear in N
SLIDE 37
SLK Result
NMI/Accuracy
12
Time (seconds)
SLIDE 38
SLK Result
NMI/Accuracy
12
Time (seconds)
SLIDE 39
SLK Result
LabelMe (Alexnet) MNIST (small)
Comparison of optimization quality w.r.t LK [Wang and Carreira-Perpiñán 2014]
13
SLIDE 40
Thank you
Code on: https://github.com/imtiazziko/SLK
More at poster session:
Room 210 & 230 AB #96
40