An eigenvalue optimization problem for graph partitioning Chris - PowerPoint PPT Presentation

An eigenvalue optimization problem for graph partitioning Chris White UT-Austin February 5, 2014 Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 1 / 28

Outline Outline Quick introduction to clustering & graph partitioning Previous related work The Dirichlet Energy Definitions A Relaxation A Rearrangement Algorithm Connections Nonnegative Matrix Factorization Reaction-Diffusion Equations Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 2 / 28

Intro to Clustering Cluster Analysis Example Clustering Cluster Analysis seeks to find meaningful groups within data, by optimizing some measure of similarity or dissimilarity. Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 3 / 28

Intro to Clustering Graph Partitioning In the graph partitioning framework, one forms a graph where the nodes represent the observed data points and the edge weights represent some measure of similarity, with the goal of utilizing geometric tools and insights to analyze the data. Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 4 / 28

Intro to Clustering Challenges Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 5 / 28

Intro to Clustering Challenges Large, high dimensional data sets Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 5 / 28

Intro to Clustering Challenges Large, high dimensional data sets Typical formulations of graph problems lead to NP-hard problems Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 5 / 28

Intro to Clustering Challenges Large, high dimensional data sets Typical formulations of graph problems lead to NP-hard problems Measure of optimality for clustering can be application dependent Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 5 / 28

Intro to Clustering Some Notation The graph Laplacian(s) : ∆ G , r := D 1 − r − D − r / 2 WD − r / 2 The gradient of a function f : V → R : ∇ f ( v , w ) := f ( v ) − f ( w ) The inner product: � d r � f , g � V , r := i f ( i ) g ( i ) i ∈ V Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 6 / 28

Some Previous Work Previous Work The Cheeger cut (or balanced cut) of a graph ( V , E ) is the following quantity: | ∂ S | h ( G ) := min min { vol( S ) , vol( S c ) } S ⊂ V � where | ∂ S | := w ij is the perimeter of the vertex set S and i ∈ S , j / ∈ S � vol( S ) = d i . i ∈ S Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 7 / 28

Some Previous Work Previous Work The Cheeger cut (or balanced cut) of a graph ( V , E ) is the following quantity: | ∂ S | h ( G ) := min min { vol( S ) , vol( S c ) } S ⊂ V � where | ∂ S | := w ij is the perimeter of the vertex set S and i ∈ S , j / ∈ S � vol( S ) = d i . i ∈ S Provides a geometrically meaningful bi-partition of the graph NP-hard to compute Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 7 / 28

Some Previous Work Some Previous Work [Bresson et al. (2013)] attempt to solve R | ∂ A r | � min r |} . min { λ | A r | , | A c r =1 Relaxation: R � f r � TV � min � f r − med λ ( f r ) � 1 ,λ r =1 R � subject to f i : V → [0 , 1] , f r = 1. r =1 Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 8 / 28

Some Previous Work Some Previous Work Using ideas from materials science, [Bertozzi and Flenner (2012)] introduce the following graph-based Ginzburg-Landau functional Ginzburg-Landau functional E ( u ) := � u , ∆ u � + 1 ( u 2 ( v ) − 1) 2 + F ( u , u 0 ) . � 2 ǫ v ∈ V Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 9 / 28

Some Previous Work Some Previous Work Using ideas from materials science, [Bertozzi and Flenner (2012)] introduce the following graph-based Ginzburg-Landau functional Ginzburg-Landau functional E ( u ) := � u , ∆ u � + 1 ( u 2 ( v ) − 1) 2 + F ( u , u 0 ) . � 2 ǫ v ∈ V Using numerical methods for mean curvature flow [MBO], in [Merkurjev et al. (2012)] a fast spectral-based algorithm was developed for finding minima. Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 9 / 28

Some Previous Work Some Previous Work Using ideas from materials science, [Bertozzi and Flenner (2012)] introduce the following graph-based Ginzburg-Landau functional Ginzburg-Landau functional E ( u ) := � u , ∆ u � + 1 ( u 2 ( v ) − 1) 2 + F ( u , u 0 ) . � 2 ǫ v ∈ V Using numerical methods for mean curvature flow [MBO], in [Merkurjev et al. (2012)] a fast spectral-based algorithm was developed for finding minima. This has inspired interesting work on analogues of mean curvature on graphs [van Gennip et al. (2013)]. Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 9 / 28

A New Approach We will now describe a new approach to graph partitioning based on Dirichlet eigenvalues, inspired by the analogous continuous problem. Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 10 / 28

A New Approach We will now describe a new approach to graph partitioning based on Dirichlet eigenvalues, inspired by the analogous continuous problem. Advantages: Easy to implement algorithm, with convergence and local optimality guarantees Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 10 / 28

A New Approach We will now describe a new approach to graph partitioning based on Dirichlet eigenvalues, inspired by the analogous continuous problem. Advantages: Easy to implement algorithm, with convergence and local optimality guarantees Representatives for each cluster Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 10 / 28

A New Approach We will now describe a new approach to graph partitioning based on Dirichlet eigenvalues, inspired by the analogous continuous problem. Advantages: Easy to implement algorithm, with convergence and local optimality guarantees Representatives for each cluster Interesting PageRank interpretation Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 10 / 28

A New Approach We will now describe a new approach to graph partitioning based on Dirichlet eigenvalues, inspired by the analogous continuous problem. Advantages: Easy to implement algorithm, with convergence and local optimality guarantees Representatives for each cluster Interesting PageRank interpretation Relationship to other areas: geometric domain decomposition, reaction-diffusion equations This is joint work with Braxton Osting and ´ Edouard Oudet [Osting et al. (2013)]. Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 10 / 28

A New Approach Dirichlet Eigenvalues Recall that for a domain Ω ⊂ R n , the Dirichlet Eigenvalue λ 1 (Ω) is defined to be the smallest number λ for which there exists a solution to the following Dirichlet problem: � ∆ ψ = λψ in Ω ψ = 0 on ∂ Ω �∇ ψ � 2 2 Equivalently, λ 1 (Ω) = inf . � ψ � 2 ψ � =0 2 ψ | ∂ Ω =0 Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 11 / 28

A New Approach Dirichlet Energy Analogously, for a vertex subset S ⊂ V we define �∇ ψ � 2 2 λ 1 ( S ) = min (1) . � ψ � 2 ψ � =0 2 ψ | Sc =0 λ 1 ( S ) is a Dirichlet eigenvalue for ∆ G , and Perron-Frobenius theory tells us that the associated eigenvector can be taken to be strictly positive inside S . Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 12 / 28

A New Approach Dirichlet Energy On a graph, we have the following inequalities: Gershgorin Circle Theorem | ∂ S | � min i ∈ S [ d i − w ij ] ≤ λ 1 ( S ) ≤ vol( S ) j ∈ S Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 13 / 28

A New Approach Dirichlet Energy On a graph, we have the following inequalities: Gershgorin Circle Theorem | ∂ S | � min i ∈ S [ d i − w ij ] ≤ λ 1 ( S ) ≤ vol( S ) j ∈ S Local Cheeger Inequality [Chung (2007)] h S ≤ λ 1 ( S ) ≤ h 2 S 2 Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 13 / 28

A New Approach Dirichlet Energy On a graph, we have the following inequalities: Gershgorin Circle Theorem | ∂ S | � min i ∈ S [ d i − w ij ] ≤ λ 1 ( S ) ≤ vol( S ) j ∈ S Local Cheeger Inequality [Chung (2007)] h S ≤ λ 1 ( S ) ≤ h 2 S 2 Thus we seek to minimize the Dirichlet energy of a k -partition { V i } k i =1 k � min λ 1 ( V i ) . (2) V = ∐ k i =1 V i i =1 Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 13 / 28

A New Approach Dirichlet Energy : Relaxation For a vertex function φ : V → [0 , 1] and α > 0, consider the quantity � ψ � 2 =1 �∇ ψ � 2 + α � ψ � 2 λ α ( φ ) := min (1 − φ ) and the relaxed Dirichlet energy k Λ α, ∗ � λ α ( φ i ) := min (3) k { φ i } k i =1 ∈A k i =1 i =1 : φ i : V → [0 , 1] and � k { φ i } k � � where A k := i =1 φ i = 1 . Chris White (UT-Austin) An eigenvalue optimization problem February 5, 2014 14 / 28