spectral graph theory and clustering linear algebra reminder Real - PowerPoint PPT Presentation

spectral graph theory and clustering

linear algebra reminder Real symmetric matrices have real eigenvalues and eigenvectors. 𝐵 𝑗𝑘 = 𝐵 𝑘𝑗 2 −1 3 1 −1 −1 𝐵 = −1 −2 −1 = 0 0 = (−1) 0 3 −1 2 1 −1 1 eigenvalues: −2.2749 −1 5.2749 1 −1 1 eigenvectors: 7.2749 0 −0.2749 1 1 1

heat flow

heat flow

a version in discrete time and space An undirected graph 𝐻 = (𝑊, 𝐹) For now, assume that 𝐻 is 𝒆 -regular for some number 𝑒 .

a version in discrete time and space Random walk matrix: An undirected graph 𝐻 = (𝑊, 𝐹) 𝑋 is an 𝑜 × 𝑜 real symmetric matrix. 𝑗𝑗 = 1 𝑋 2 𝑗𝑘 = 1 𝑋 {𝑗, 𝑘 } an edge 2𝑒 𝑋 𝑗𝑘 = 0 {𝑗, 𝑘 } not an edge 𝑊 = 1, 2, … , 𝑜 𝑋𝑣 𝑗 = 1 2 𝑣 𝑗 + 1 1 𝑣 𝑘 2 𝑒 𝑣 = 𝑣 1 , 𝑣 2 , … , 𝑣 𝑜 ∈ ℝ 𝑜 𝑘∶ 𝑗,𝑘 ∈𝐹

heat dispersion on a graph

evolution of the random walk / heat flow 𝑣 = 𝑣 1 , 𝑣 2 , … , 𝑣 𝑜 eigenvalues/ eigenvectors of 𝑋 𝑜 𝑜 𝑜 𝜈 1 𝑤 1 𝑋𝑣 = 𝑋 1,𝑗 𝑣 𝑗 , 𝑋 2,𝑗 𝑣 𝑗 , … , 𝑋 𝑜,𝑗 𝑣 𝑗 𝑗=1 𝑗=1 𝑗=1 𝜈 2 𝑤 2 𝑜 𝑜 ⋯ 𝑋 2 𝑣 = 𝑋 1,𝑘 𝑋 𝑘,𝑗 𝑣 𝑗 , … , 𝑋 𝑜,𝑘 𝑋 𝑘,𝑗 𝑣 𝑗 𝑗,𝑘=1 𝑗,𝑘=1 𝜈 𝑜 𝑤 𝑜 𝑣 = 𝛽 1 𝑤 1 + 𝛽 2 𝑤 2 + ⋯ + 𝛽 𝑜 𝑤 𝑜 𝜈 1 = 1 𝑋𝑣 = 𝜈 1 𝛽 1 𝑤 1 + 𝜈 2 𝛽 2 𝑤 2 + ⋯ + 𝜈 𝑜 𝛽 𝑜 𝑤 𝑜 𝑋 2 𝑣 = 𝜈 1 2 𝛽 1 𝑤 1 + 𝜈 2 2 𝛽 2 𝑤 2 + ⋯ + 𝜈 𝑜 2 𝛽 𝑜 𝑤 𝑜 1 𝑜 , … , 1 𝑤 1 = 𝑜 𝑙 𝛽 1 𝑤 1 + 𝜈 2 𝑙 𝛽 2 𝑤 2 + ⋯ + 𝜈 𝑜 𝑋 𝑙 𝑣 = 𝜈 1 𝑙 𝛽 𝑜 𝑤 𝑜

evolution of the random walk / heat flow 𝑣 = 𝑣 1 , 𝑣 2 , … , 𝑣 𝑜 eigenvalues/ eigenvectors of 𝑋 𝑜 𝑜 𝑜 𝜈 1 𝑤 1 𝑋𝑣 = 𝑋 1,𝑗 𝑣 𝑗 , 𝑋 2,𝑗 𝑣 𝑗 , … , 𝑋 𝑜,𝑗 𝑣 𝑗 𝑗=1 𝑗=1 𝑗=1 𝜈 2 𝑤 2 𝑜 𝑜 ⋯ 𝑋 2 𝑣 = 𝑋 1,𝑘 𝑋 𝑘,𝑗 𝑣 𝑗 , … , 𝑋 𝑜,𝑘 𝑋 𝑘,𝑗 𝑣 𝑗 𝑗,𝑘=1 𝑗,𝑘=1 𝜈 𝑜 𝑤 𝑜 𝑣 = 𝛽 1 𝑤 1 + 𝛽 2 𝑤 2 + ⋯ + 𝛽 𝑜 𝑤 𝑜 𝜈 1 = 1 𝑋𝑣 = 𝜈 1 𝛽 1 𝑤 1 + 𝜈 2 𝛽 2 𝑤 2 + ⋯ + 𝜈 𝑜 𝛽 𝑜 𝑤 𝑜 𝑋 2 𝑣 = 𝜈 1 2 𝛽 1 𝑤 1 + 𝜈 2 2 𝛽 2 𝑤 2 + ⋯ + 𝜈 𝑜 2 𝛽 𝑜 𝑤 𝑜 1 𝑜 , … , 1 𝑤 1 = 𝑜 𝑙 𝛽 2 𝑤 2 + ⋯ + 𝜈 𝑜 𝑋 𝑙 𝑣 = 𝑙 𝛽 𝑜 𝑤 𝑜 𝛽 1 𝑤 1 + 𝜈 2

evolution of the random walk / heat flow 𝑣 = 𝑣 1 , 𝑣 2 , … , 𝑣 𝑜 eigenvalues/ eigenvectors of 𝑋 𝑜 𝑜 𝑜 𝜈 1 𝑤 1 𝑋𝑣 = 𝑋 1,𝑗 𝑣 𝑗 , 𝑋 2,𝑗 𝑣 𝑗 , … , 𝑋 𝑜,𝑗 𝑣 𝑗 𝑗=1 𝑗=1 𝑗=1 𝜈 2 𝑤 2 𝑜 𝑜 ⋯ 𝑋 2 𝑣 = 𝑋 1,𝑘 𝑋 𝑘,𝑗 𝑣 𝑗 , … , 𝑋 𝑜,𝑘 𝑋 𝑘,𝑗 𝑣 𝑗 𝑗,𝑘=1 𝑗,𝑘=1 𝜈 𝑜 𝑤 𝑜 𝑣 = 𝛽 1 𝑤 1 + 𝛽 2 𝑤 2 + ⋯ + 𝛽 𝑜 𝑤 𝑜 𝜈 1 = 1 𝑋𝑣 = 𝜈 1 𝛽 1 𝑤 1 + 𝜈 2 𝛽 2 𝑤 2 + ⋯ + 𝜈 𝑜 𝛽 𝑜 𝑤 𝑜 𝑋 2 𝑣 = 𝜈 1 2 𝛽 1 𝑤 1 + 𝜈 2 2 𝛽 2 𝑤 2 + ⋯ + 𝜈 𝑜 2 𝛽 𝑜 𝑤 𝑜 1 𝑜 , … , 1 𝑤 1 = 𝑜 𝑙 𝛽 2 𝑤 2 + ⋯ + 𝜈 𝑜 𝑋 𝑙 𝑣 = 𝑙 𝛽 𝑜 𝑤 𝑜 𝛽 1 𝑤 1 + 𝜈 2

spectral embedding 𝑤 2

bottlenecks 𝐻 = (𝑊, 𝐹) Φ ∗ 𝐻 = min Φ 𝑇 𝑇 ≤𝑜 2 𝑇 Φ 𝑇 = 𝐹 𝑇 𝑇

PCA cannot find non-linear structure

spectral partitioning can... [photo credit: Ma-Wu-Luo-Feng 2011]

spectral partitioning can... [photo credit: Sidi, et. al. 2011]