European Symposia on Algorithms16 Outline Problem Formulation - - PowerPoint PPT Presentation

A Note On Spectral Clustering

Pavel Kolev and Kurt Mehlhorn

European Symposia on Algorithms‘16

Outline

• Problem Formulation

– Algorithmic Tools

• Our Contribution

– Structural Result – Algorithmic Result

• Proof Overview
• Summary

k-way Partitioning

• Def. A cluster is a subset 𝑇 ⊆ 𝑊

with small conductance 𝜚 𝑇 =

|𝐹 𝑇, 𝑇 | 𝜈(𝑇) , where the volume 𝜈 𝑇 = 𝑤∈𝑇 deg(𝑤).

k-way Partitioning

• Def. A cluster is a subset 𝑇 ⊆ 𝑊

with small conductance

• Def. The order 𝑙 conductance constant

𝜚 𝑇 =

|𝐹 𝑇, 𝑇 | 𝜈(𝑇) , where the volume 𝜈 𝑇 = 𝑤∈𝑇 deg(𝑤).

𝜍(𝑙) = min

partition (𝑄1,…,𝑄𝑙) max 𝑗∈[1:𝑙] 𝜚 𝑄𝑗

k-way Partitioning

• Def. A cluster is a subset 𝑇 ⊆ 𝑊

with small conductance

• Def. The order 𝑙 conductance constant
• Goal: Find an approximate 𝑙-way partition w.r.t 𝜍(𝑙).

𝜚 𝑇 =

|𝐹 𝑇, 𝑇 | 𝜈(𝑇) , where the volume 𝜈 𝑇 = 𝑤∈𝑇 deg(𝑤).

𝜍(𝑙) = min

partition (𝑄1,…,𝑄𝑙) max 𝑗∈[1:𝑙] 𝜚 𝑄𝑗

k-way Partitioning

• Def. A cluster is a subset 𝑇 ⊆ 𝑊

with small conductance

• Def. The order 𝑙 conductance constant
• Goal: Find an approximate 𝑙-way partition w.r.t 𝜍(𝑙).

𝜚 𝑇 =

|𝐹 𝑇, 𝑇 | 𝜈(𝑇) , where the volume 𝜈 𝑇 = 𝑤∈𝑇 deg(𝑤).

𝜍(𝑙) = min

partition (𝑄1,…,𝑄𝑙) max 𝑗∈[1:𝑙] 𝜚 𝑄𝑗

Standard Spectral Clustering Paradigm

Input: 𝐻 = 𝑊, 𝐹 , 3 ≤ 𝑙 ≪ 𝑜 and 𝜗 ∈ (0,1). Output: An approximate 𝑙-way partition of 𝑊. Andrew Ng et al [NIPS’02]:

• 1. Computes an approximate Spectral Embedding

𝐺: 𝑊 ↦ 𝑆𝑙 using the Power Method. 2) Run a 𝑙-means clustering algorithm to compute an approximate 𝑙-way partition of 𝐺 𝑤

𝑤∈𝑊 .

Outline

• Problem Formulation

– Algorithmic Tools

• Our Contribution

– Structural Result – Algorithmic Result

• Proof Overview
• Summary

Spectral Graph Theory

• The normalized Laplacian matrix ℒ has eigenvalues
• Fact. A graph has exactly 𝑙 connected component iff

0 = 𝜇𝑙 < 𝜇𝑙+1. 0 = 𝜇1 ≤ ⋯ ≤ 𝜇𝑙 ≤ 𝜇𝑙+1 ≤ ⋯ ≤ 𝜇𝑜 ≤ 2.

Spectral Graph Theory

• The normalized Laplacian matrix ℒ has eigenvalues
• Fact. A graph has exactly 𝑙 connected component iff
• Trevisan et al. [STOC’12, SODA’14] proved a robust

version 0 = 𝜇𝑙 < 𝜇𝑙+1. 𝜇𝑙/2 ≤ 𝜍 𝑙 ≤ 𝑃 𝑙3 𝜇𝑙. 0 = 𝜇1 ≤ ⋯ ≤ 𝜇𝑙 ≤ 𝜇𝑙+1 ≤ ⋯ ≤ 𝜇𝑜 ≤ 2.

(𝜍 𝑙 is NP-hard and 𝜇𝑙 is in P) → approx. scheme!

Exact Spectral Embedding

• 𝑉𝑙 = 𝑤1, 𝑤2, … , 𝑤𝑙 ∈ 𝑆𝑊×𝑙 - the bottom 𝑙 eigenvectors of ℒ
• Normalized Spectral Embedding:

𝐺 𝑤 =

1 deg(𝑤) 𝑉𝑙 𝑤, : , for every 𝑤 ∈ 𝑊.

𝐺: 𝑊 ↦ 𝑆𝑙

Exact Spectral Embedding

• 𝑉𝑙 = 𝑤1, 𝑤2, … , 𝑤𝑙 ∈ 𝑆𝑊×𝑙 - the bottom 𝑙 eigenvectors of ℒ
• Normalized Spectral Embedding:

𝐺 𝑤 =

1 deg(𝑤) 𝑉𝑙 𝑤, : , for every 𝑤 ∈ 𝑊.

𝐺: 𝑊 ↦ 𝑆𝑙

Approximate Spectral Embedding

• 𝑉𝑙 ∈ 𝑆𝑊×𝑙 approximation of the bottom 𝑙 eigenvectors of ℒ
• Approximate Normalized Spectral Embedding:

𝐺 𝑤 =

1 deg(𝑤)

𝑉𝑙 𝑤, : , for every 𝑤 ∈ 𝑊.

Power Method 𝐺: 𝑊 ↦ 𝑆𝑙

Approximate Spectral Embedding

• 𝑉𝑙 ∈ 𝑆𝑊×𝑙 approximation of the bottom 𝑙 eigenvectors of ℒ
• Approximate Normalized Spectral Embedding:

Power Method

𝒴𝐹 = deg 𝑤 many copies of 𝐺 𝑤 𝑤 ∈ 𝑊}.

𝐺: 𝑊 ↦ 𝑆𝑙

𝒴𝑊 = 𝐺 𝑤 𝑤 ∈ 𝑊}.

Point Sets:

𝑙-means Clustering

Input: 𝒴 = 𝑞1, … , 𝑞𝑜 with 𝑞𝑗 ∈ 𝑆𝑙. Output: 𝑙-way partition of 𝒴 such that

𝐵1

⋆, … , 𝐵𝑙 ⋆

= argmin

partition 𝑌1,…,𝑌𝑙 of 𝒴 𝑗=1 𝑙 𝑞∈𝑌𝑗

𝑞 − 𝑑𝑗

2 ,

where 𝑑𝑗 is the center of 𝑌𝑗.

𝑙-means Clustering

Input: 𝒴 = 𝑞1, … , 𝑞𝑜 with 𝑞𝑗 ∈ 𝑆𝑙. Output: 𝑙-way partition of 𝒴 such that

• Def. The optimal 𝑙-means cost is

𝐵1

⋆, … , 𝐵𝑙 ⋆

= argmin

partition 𝑌1,…,𝑌𝑙 of 𝒴 𝑗=1 𝑙 𝑞∈𝑌𝑗

𝑞 − 𝑑𝑗

2 ,

where 𝑑𝑗 is the center of 𝑌𝑗. Δ𝑙 𝒴 = cost 𝐵1

⋆, … , 𝐵𝑙 ⋆ .

Outline

• Problem Formulation

– Algorithmic Tools

• Our Contribution

– Structural Result – Algorithmic Result

• Proof Overview
• Summary

Structural Result

• Peng et al. [COLT’15]

Υ ≔ 𝜇𝑙+1/𝜍 𝑙 ≥ Ω(𝑙3)

• Our Result

Ψ ≔ 𝜇𝑙+1/𝜍avr(𝑙) ≥ Ω(𝑙3) 𝜍(𝑙) = max

𝑗∈[1:𝑙] 𝜚 𝑄 𝑗

𝜍avr(𝑙) = 1 𝑙

𝑗=1 𝑙

𝜚(𝑄

𝑗)

• (𝑄

1, … , 𝑄𝑙) is an optimal k-way partition of 𝐻 w.r.t. 𝜍(𝑙).

Structural Result

• Peng et al. [COLT’15]

Υ ≔ 𝜇𝑙+1/𝜍 𝑙 ≥ Ω(𝑙3)

• Our Result

Ψ ≔ 𝜇𝑙+1/𝜍avr(𝑙) ≥ Ω(𝑙3)

• (𝑄

1, … , 𝑄𝑙) is an optimal k-way partition of 𝐻 w.r.t. 𝜍(𝑙).

• cost 𝐵1, … , 𝐵𝑙 ≤ 𝛿 ⋅ Δ𝑙

𝒴𝐹 for 𝛿 ≥ 1.

𝜍(𝑙) = max

𝑗∈[1:𝑙] 𝜚 𝑄 𝑗

𝜍avr(𝑙) = 1 𝑙

𝑗=1 𝑙

𝜚(𝑄

𝑗)

Structural Result

• Peng et al. [COLT’15]

If Υ ≔ 𝜇𝑙+1/𝜍 𝑙 ≥ Ω(𝑙3) then

• 𝜈 𝐵𝑗Δ𝑄

𝑗 ≤ (𝛿/Υ) ⋅ 𝜈 𝑄 𝑗

• Our Result

If Ψ ≔ 𝜇𝑙+1/𝜍avr(𝑙) ≥ Ω(𝑙3) then

• 𝜈 𝐵𝑗Δ𝑄

𝑗 ≤ (𝛿/Ψ𝒍) ⋅ 𝜈 𝑄 𝑗

• (𝑄

1, … , 𝑄𝑙) is an optimal k-way partition of 𝐻 w.r.t. 𝜍(𝑙).

• cost 𝐵1, … , 𝐵𝑙 ≤ 𝛿 ⋅ Δ𝑙

𝒴𝐹 for 𝛿 ≥ 1.

𝜍(𝑙) = max

𝑗∈[1:𝑙] 𝜚 𝑄 𝑗

𝜍avr(𝑙) = 1 𝑙

𝑗=1 𝑙

𝜚(𝑄

𝑗)

𝐵𝑗Δ𝑄

𝑗

Structural Result

• Peng et al. [COLT’15]

If Υ ≔ 𝜇𝑙+1/𝜍 𝑙 ≥ Ω(𝑙3) then

• 𝜈 𝐵𝑗Δ𝑄

𝑗 ≤ (𝛿/Υ) ⋅ 𝜈 𝑄 𝑗

• 𝜚 𝐵𝑗 ≤ 1 + 𝛿/Υ ⋅ 𝜚 𝑄

𝑗 + 𝛿/Υ

• Our Result

If Ψ ≔ 𝜇𝑙+1/𝜍avr(𝑙) ≥ Ω(𝑙3) then

• 𝜈 𝐵𝑗Δ𝑄

𝑗 ≤ (𝛿/Ψ𝒍) ⋅ 𝜈 𝑄 𝑗

• 𝜚 𝐵𝑗 ≤ 1 + 𝛿/Ψ𝒍 ⋅ 𝜚 𝑄

𝑗 + 𝛿/Ψ𝒍

𝜍(𝑙) = max

𝑗∈[1:𝑙] 𝜚 𝑄 𝑗

𝜍avr(𝑙) = 1 𝑙

𝑗=1 𝑙

𝜚(𝑄

𝑗)

• (𝑄

1, … , 𝑄𝑙) is an optimal k-way partition of 𝐻 w.r.t. 𝜍(𝑙).

• cost 𝐵1, … , 𝐵𝑙 ≤ 𝛿 ⋅ Δ𝑙

𝒴𝐹 for 𝛿 ≥ 1.

Structural Result

• Peng et al. [COLT’15]

If Υ ≔ 𝜇𝑙+1/𝜍 𝑙 ≥ Ω(𝑙3) then

• 𝜈 𝐵𝑗Δ𝑄

𝑗 ≤ (𝛿/Υ) ⋅ 𝜈 𝑄 𝑗

• 𝜚 𝐵𝑗 ≤ 1 + 𝛿/Υ ⋅ 𝜚 𝑄

𝑗 + 𝛿/Υ

• Our Result

If Ψ ≔ 𝜇𝑙+1/𝜍avr(𝑙) ≥ Ω(𝑙3) then

• 𝜈 𝐵𝑗Δ𝑄

𝑗 ≤ (𝛿/Ψ𝒍) ⋅ 𝜈 𝑄 𝑗

• 𝜚 𝐵𝑗 ≤ 1 + 𝛿/Ψ𝒍 ⋅ 𝜚 𝑄

𝑗 + 𝛿/Ψ𝒍

𝜍(𝑙) = max

𝑗∈[1:𝑙] 𝜚 𝑄 𝑗

𝜍avr(𝑙) = 1 𝑙

𝑗=1 𝑙

𝜚(𝑄

𝑗)

How to find such 𝑙-way partition 𝐵1, … , 𝐵𝑙 ?

Outline

• Problem Formulation

– Algorithmic Tools

• Our Contribution

– Structural Result – Algorithmic Result

• Proof Overview
• Summary

Algorithmic Result

• Peng et al. [COLT’15]

Υ ≔ 𝜇𝑙+1/𝜍 𝑙 ≥ Ω(𝒍𝟔)

Concentration Heat Kernel and Local Sensitive Hashing

more restrictive by Ω 𝑙2 -factor

Algorithmic Result

• Peng et al. [COLT’15]

Υ ≔ 𝜇𝑙+1/𝜍 𝑙 ≥ Ω(𝒍𝟔)

• Our Result

Ψ ≔ 𝜇𝑙+1/𝜍avr(𝑙) ≥ Ω(𝑙3) and ∆𝑙 𝒴𝑊 ≥ 𝑜−𝑃(1)

• Approx. Spectral Embedding

and k-means Clustering Concentration Heat Kernel and Local Sensitive Hashing

more restrictive by Ω 𝑙2 -factor

Algorithmic Result

• Peng et al. [COLT’15]

Υ ≔ 𝜇𝑙+1/𝜍 𝑙 ≥ Ω(𝒍𝟔)

• Our Result

Ψ ≔ 𝜇𝑙+1/𝜍avr(𝑙) ≥ Ω(𝑙3) and ∆𝑙 𝒴𝑊 ≥ 𝑜−𝑃(1)

• Approx. Spectral Embedding

and k-means Clustering Concentration Heat Kernel and Local Sensitive Hashing

more restrictive by Ω 𝑙2 -factor

This is the 1st rigorous algorithmic analysis of the Standard Spectral Clustering Paradigm!

Algorithmic Result

• Peng et al. [COLT’15]

Υ ≔ 𝜇𝑙+1/𝜍 𝑙 ≥ Ω(𝒍𝟔)

• Our Result

Ψ ≔ 𝜇𝑙+1/𝜍avr(𝑙) ≥ Ω(𝑙3) and ∆𝑙 𝒴𝑊 ≥ 𝑜−𝑃(1)

Concentration Heat Kernel and Local Sensitive Hashing

constant = 105 constant = 107/𝜗0 𝜗0 = 6/107 is Ostrovsky et al’s [FOCS’13]

k-means alg. constant (is not optimized!)

• Approx. Spectral Embedding

and k-means Clustering

Algorithmic Result

• Peng et al. [COLT’15]

If Υ ≔ 𝜇𝑙+1/𝜍 𝑙 ≥ Ω(𝒍𝟔) then

• 𝜈 𝐵𝑗Δ𝑄

𝑗 ≤ log2𝑙 𝑙2 /Υ ⋅ 𝜈 𝑄 𝑗

• 𝜚 𝐵𝑗 ≤ 1 + log2𝑙

𝑙2 /Υ ⋅ 𝜚 𝑄 𝑗 + log2𝑙 𝑙2 /Υ

• Our Result

If Ψ ≔ 𝜇𝑙+1/𝜍avr(𝑙) ≥ Ω(𝑙3) and ∆𝑙 𝒴𝑊 ≥ 𝑜−𝑃(1) then

• 𝜈 𝐵𝑗Δ𝑄

𝑗 ≤ (1/Ψ𝒍) ⋅ 𝜈 𝑄 𝑗

• 𝜚 𝐵𝑗 ≤ 1 + 1/Ψ𝒍 ⋅ 𝜚 𝑄

𝑗 + (1/Ψ𝒍)

• Approx. Spectral Embedding

and k-means Clustering Heat Kernel and Local Sensitive Hashing

Algorithmic Result

• Peng et al. [COLT’15]

If Υ ≔ 𝜇𝑙+1/𝜍 𝑙 ≥ Ω(𝒍𝟔) then

• 𝜈 𝐵𝑗Δ𝑄

𝑗 ≤ log2𝑙 𝑙2 /Υ ⋅ 𝜈 𝑄 𝑗

• 𝜚 𝐵𝑗 ≤ 1 + log2𝑙

𝑙2 /Υ ⋅ 𝜚 𝑄 𝑗 + log2𝑙 𝑙2 /Υ

• Our Result

If Ψ ≔ 𝜇𝑙+1/𝜍avr(𝑙) ≥ Ω(𝑙3) and ∆𝑙 𝒴𝑊 ≥ 𝑜−𝑃(1) then

• 𝜈 𝐵𝑗Δ𝑄

𝑗 ≤ (1/Ψ𝒍) ⋅ 𝜈 𝑄 𝑗

• 𝜚 𝐵𝑗 ≤ 1 + 1/Ψ𝒍 ⋅ 𝜚 𝑄

𝑗 + (1/Ψ𝒍)

• Approx. Spectral Embedding

and k-means Clustering Heat Kernel and Local Sensitive Hashing Runtime: 𝑃(𝑛log𝑑𝑜) Runtime: O 𝑛 𝑙2 + ln 𝑜

𝜇𝑙+1

Outline

• Problem Formulation

– Algorithmic Tools

• Our Contribution

– Structural Result – Algorithmic Result

• Proof Overview
• Summary

Proof Overview

𝐻 𝑉𝑙 𝒴𝑊 𝑉𝑙 𝒴𝑊 𝐻

Approximate Embedding Exact Embedding

𝑃(𝑜𝜕)

Power Method SVD

?

Proof Overview

Boutsidos et al [ICML’15] Let 𝐵1, … , 𝐵𝑙 be a partition such that then

cost 𝐵1, … , 𝐵𝑙 ≤ 1 + 4𝜗 1 + 𝛿 Δ𝑙 𝒴𝑊 + 4𝜗2. 𝐻 𝑉𝑙 𝒴𝑊 𝑉𝑙 𝒴𝑊 𝐻 𝑃(𝑜𝜕)

Power Method SVD

cost 𝐵1, … , 𝐵𝑙 ≤ (1 + 𝛿)Δ𝑙 𝒴𝑊

Δ𝑙 𝒴𝑊 ≈ Δ𝑙 𝒴𝑊

Approximate Embedding Exact Embedding

Proof Overview

Boutsidos et al [ICML’15] Let 𝐵1, … , 𝐵𝑙 be a partition such that then

cost 𝐵1, … , 𝐵𝑙 ≤ 1 + 4𝜗 1 + 𝛿 Δ𝑙 𝒴𝑊 + 4𝜗2. 𝐻 𝑉𝑙 𝒴𝑊 𝑉𝑙 𝒴𝑊 𝐻 𝑃(𝑜𝜕) 𝑃 𝑛𝑙𝑞 + 𝑜𝑙2

No fast P.M. No fast k-means Power Method SVD

𝑞 = 𝑔 𝑜, log 1 𝜗 , 𝜇𝑙, 𝜇𝑙+1

cost 𝐵1, … , 𝐵𝑙 ≤ (1 + 𝛿)Δ𝑙 𝒴𝑊

Δ𝑙 𝒴𝑊 ≈ Δ𝑙 𝒴𝑊

Approximate Embedding Exact Embedding

Proof Overview

𝐺 𝑤 = 1 𝑒𝑤 𝑉𝑙 𝑤, : 𝐻 𝑉𝑙 𝒴𝑊 𝑉𝑙 𝒴𝑊 𝐻 𝒴𝐹 𝒴𝐹 𝑃(𝑜𝜕) 𝑃 𝑛𝑙𝑞 + 𝑜𝑙2

?

𝐺 𝑤 = 1 𝑒𝑤 𝑉𝑙 𝑤, :

Δ𝑙 𝒴𝑊 ≈ Δ𝑙 𝒴𝑊

Approximate Embedding Exact Embedding No fast P.M. No fast k-means

Proof Overview

𝐺 𝑤 = 1 𝑒𝑤 𝑉𝑙 𝑤, : 𝐻 𝑉𝑙 𝒴𝑊 𝑉𝑙 𝒴𝑊 𝐻 𝒴𝐹 𝒴𝐹 𝑃(𝑜𝜕) 𝑃 𝑛𝑙𝑞 + 𝑜𝑙2

?

𝐺 𝑤 = 1 𝑒𝑤 𝑉𝑙 𝑤, :

Δ𝑙 𝒴𝑊 ≈ Δ𝑙 𝒴𝑊

Approximate Embedding Exact Embedding No fast P.M. No fast k-means Questions:

• 1. Find an efficient 𝑙-means clustering algorithm for

𝒴𝐹?

• 2. Extend Boutsidos et al’s [ICML’15] analysis?

Proof Overview

𝐺 𝑤 = 1 𝑒𝑤 𝑉𝑙 𝑤, : 𝐻 𝑉𝑙 𝒴𝑊 𝑉𝑙 𝒴𝑊 𝐻 𝑃(𝑜𝜕) 𝑃 𝑛𝑙𝑞 + 𝑜𝑙2

Ostrovsky et al’s [FOCS’13] gave an approximate k-means algorithm with fast runtime 𝑃 𝑛𝑙2 , but requires Δ𝑙 𝒴 ≤ 𝜗0

2 ⋅ Δ𝑙−1 𝒴

𝐺 𝑤 = 1 𝑒𝑤 𝑉𝑙 𝑤, :

where 𝜗0 = 6/107. Δ𝑙 𝒴𝑊 ≈ Δ𝑙 𝒴𝑊

Approximate Embedding Exact Embedding No fast P.M. No fast k-means

𝒴𝐹 𝒴𝐹

?

Proof Overview

𝐺 𝑤 = 1 𝑒𝑤 𝑉𝑙 𝑤, : 𝐻 𝑉𝑙 𝒴𝑊 𝑉𝑙 𝒴𝑊 𝐻 𝑃(𝑜𝜕) 𝑃 𝑛𝑙𝑞 + 𝑜𝑙2

Ostrovsky et al’s [FOCS’13] gave an approximate k-means algorithm with fast runtime 𝑃 𝑛𝑙2 , but requires Δ𝑙 𝒴 ≤ 𝜗0

2 ⋅ Δ𝑙−1 𝒴

𝐺 𝑤 = 1 𝑒𝑤 𝑉𝑙 𝑤, :

Δ𝑙 𝒴𝑊 ≈ Δ𝑙 𝒴𝑊

Approximate Embedding Exact Embedding No fast P.M. No fast k-means

𝒴𝐹 𝒴𝐹

?

where 𝜗0 = 6/107.

Proof Overview

𝐺 𝑤 = 1 𝑒𝑤 𝑉𝑙 𝑤, : 𝐻 𝑉𝑙 𝒴𝑊 𝐺 𝑤 = 1 𝑒𝑤 𝑉𝑙 𝑤, : 𝒴𝑊 𝒴𝐹 𝑃(𝑜𝜕) 𝜇𝑙+1 𝜍avr(𝑙) = Ω(𝑙3) 𝑃 𝑛𝑙𝑞 + 𝑜𝑙2

Ostrovsky et al’s [FOCS’13]

Δ𝑙 𝒴𝐹 ≤ 𝜗0

2 ⋅ Δ𝑙−1

𝒴𝐹

?

𝒴𝐹

Δ𝑙 𝒴𝑊 ≈ Δ𝑙 𝒴𝑊

𝐻

Approximate Embedding Exact Embedding

𝑉𝑙

Fast k-means Alg. runtime: 𝑃 𝑛𝑙2

Proof Overview

𝐺 𝑤 = 1 𝑒𝑤 𝑉𝑙 𝑤, : 𝐻 𝑉𝑙 𝒴𝑊 𝐺 𝑤 = 1 𝑒𝑤 𝑉𝑙 𝑤, : 𝑉𝑙 𝒴𝑊 𝒴𝐹 𝑃(𝑜𝜕) 𝑃 𝑛𝑙 ln 𝑜 𝜇𝑙+1

?

𝒴𝐹 𝐻 𝜇𝑙+1 𝜍avr(𝑙) = Ω(𝑙3)

Δ𝑙 𝒴𝑊 ≈ Δ𝑙 𝒴𝑊

Approximate Embedding Exact Embedding Fast k-means Alg. runtime: 𝑃 𝑛𝑙2 Ostrovsky et al’s [FOCS’13]

Δ𝑙 𝒴𝐹 ≤ 𝜗0

2 ⋅ Δ𝑙−1

𝒴𝐹

Proof Sketch (Overview)

𝐺 𝑤 = 1 𝑒𝑤 𝑉𝑙 𝑤, : 𝐻 𝑉𝑙 𝒴𝑊 𝐺 𝑤 = 1 𝑒𝑤 𝑉𝑙 𝑤, : 𝑉𝑙 𝒴𝑊 𝒴𝐹 𝒴𝐹 𝑃(𝑜𝜕) 𝑃 𝑛𝑙 ln 𝑜 𝜇𝑙+1 Δ𝑙 𝒴𝐹 ≈ Δ𝑙 𝒴𝐹 𝐻 𝜇𝑙+1 𝜍avr(𝑙) = Ω(𝑙3)

Δ𝑙 𝒴𝑊 ≈ Δ𝑙 𝒴𝑊

Approximate Embedding Exact Embedding Extend Boutsidos et al [ICML’15] Fast k-means Alg. runtime: 𝑃 𝑛𝑙2 Ostrovsky et al’s [FOCS’13]

Δ𝑙 𝒴𝐹 ≤ 𝜗0

2 ⋅ Δ𝑙−1

𝒴𝐹

Outline

• Problem Formulation

– Algorithmic Tools

• Our Contribution

– Structural Result – Algorithmic Result

• Proof Overview
• Summary

Summary

• We proved rigorously that

the Standard Spectral Clustering Paradigm efficiently computes a 𝑙-way partition under asymptotically less restrictive gap assumption.

Open Problems

• Show that the SSCP has a good behavior on small graphs.

Our approach fails due to large constants in Ψ ≥ Ω(𝑙3):

– 107/𝜗0 - Ostrovsky et al. (is not optimized)

Δ𝑙 𝒴 ≤ 𝜗0

2 ⋅ Δ𝑙−1 𝒴 , where 𝜗0 = 6/107.

Open Problems

• Show that the SSCP has a good behavior on small graphs.

Our approach fails due to large constants in Ψ ≥ Ω(𝑙3):

– 107/𝜗0 - Ostrovsky et al. (is not optimized)

• Can we obtain a multiplicative conductance guarantee:

𝜚 𝐵𝑗 ≤ 1 + 𝛿/Ψ𝒍 ⋅ 𝜚 𝑄

𝑗

+ 𝛿/Ψ𝒍. Δ𝑙 𝒴 ≤ 𝜗0

2 ⋅ Δ𝑙−1 𝒴 , where 𝜗0 = 6/107.

Thank you!