Facial reduction for symmetry reduced semidefinite programs Hao Hu a , Renata Sotirov a and Henry Wolkowicz b Updated: 2019/08/07 a Tilburg University b University of Waterloo
The graph partitioning problem
The graph partitioning problem (GP) • GP: partition the vertices of a graph into k subsets of given sizes so that the number of edges between different subsets is minimized • Partition the graph below into 2 sets of equal size 4 3 4 3 4 3 1 2 1 2 1 2 Input graph objective value 4 objective value 2 2
Symmetry in the graph partitioning problem • The input graph is ”invariant” under certain permutation of its vertices 4 3 1 4 3 4 1 2 2 3 2 1 Input graph rotate clock-wise flip horizontally • How can we exploit the symmetry to attack the problem? 3
Matrix ∗ -algebra
Matrix ∗ -algebra • A set M ⊆ C n × n is a matrix ∗ -algebra over C if it is closed under addition, scalar and matrix multiplication, and taking conjugate transpose, i.e., α X + β Y ∈ M ∀ α, β ∈ C X ∗ ∈ M XY ∈ M , for all X , Y ∈ M 5
Block diagonalization of Matrix ∗ -algebra • Theorem ( Wedderburn 1907 ) Matrix ∗ -algebras containing the identity matrix have a canonical block-diagonal structure after some unitary transformation, i.e., there exists a unitary matrix Q and some integer t such that M 1 0 · · · 0 . . 0 M 2 . Q ∗ M Q = , . ... . . 0 0 · · · 0 M t where each M i ⊆ C n i × n i is basic 6
An example of block-diagonalization • Consider the matrix ∗ -algebra spanned by 1 0 0 0 0 1 0 1 0 0 1 0 0 0 0 0 0 0 0 0 1 1 1 1 B 0 = , B 1 = , B 2 = , 0 0 1 0 0 1 0 1 1 0 0 0 0 0 0 1 1 0 1 0 0 1 0 0 where B 1 is the adjacency matrix in the graph partitioning example • The unitary matrix Q below diagonalizes B 0 , B 1 , B 2 1 1 1 1 Q = 1 1 − i − 1 i 2 1 − 1 1 − 1 1 i − 1 − i 7
An example of block-diagonalization • The matrix ∗ -algebra spanned by 0 0 0 0 0 0 0 0 1 1 1 1 0 1 0 0 1 0 1 0 0 0 0 1 B 0 = , B 1 = , B 2 = 0 0 1 0 0 1 0 1 1 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 • The (block)-diagonalized matrices ˜ B i = Q T B i Q are 0 0 0 0 0 0 0 0 0 1 8 4 0 1 0 0 0 0 0 0 0 − 4 0 0 ˜ , ˜ , ˜ B 0 = B 1 = B 2 = 0 0 0 0 0 − 8 0 0 0 0 1 4 0 0 0 1 0 0 0 0 0 0 0 − 4 8
Symmetry reduction
Semidefinite program (SDP) • Consider an SDP in standard form X {� A 0 , X � | � A i , X � = b i for i = 1 , . . . , m , X ∈ S n inf + } , (1) where S n + is the cone of positive semidefinite matrices Assume the data matrices A 0 , . . . , A m and the identity matrix are contained in a matrix ∗ -algebra M . If SDP (1) has an optimal solution, then it has an optimal solution in M . • References: a) Kanno et al., 2001, de Klerk 2009, etc b) Gatermann, Parrilo 2004, Vallentin 2009, etc c) Schrijver 2005, Laurent 2007, etc 10
Symmetry reduction for SDP • Assume B 1 , . . . , B d is a basis of M . There exists an optimal solution d � X = x k B k ∈ M k =1 • The p.s.d. constraint X ∈ S n + can be simplifies as ˜ B ∗ 1 ( x ) 0 · · · 0 block-diagonal . . ˜ B ∗ � �� � 0 2 ( x ) . � d ( Q T B k Q ) ∈ S n ∈ S n k =1 x k ⇐ ⇒ . + ... + . . 0 ˜ B ∗ 0 · · · 0 t ( x ) where ˜ B ∗ j ( x ) is the j -th block j ( x ) ∈ S n j + if and only if ˜ • We have X ∈ S n B ∗ + for every j = 1 . . . , t 11
An example of symmetry reduction • An SDP relaxation for the cut minimization problem (Pong et al. ’14) min X � C , X � s.t. A ( X ) = b , X ≥ 0 X ∈ S nk + , where n is the number of vertices and k is the number of subsets in the partition • Instance can161 with n = 161 vertices and k = 3 partitions • The size of X ∈ S nk + is nk = 483, and very difficult to solve � 12
An example of symmetry reduction • The feasible solutions X under certain unitary transformation, i.e., Q T XQ , has the following block-diagonal structure 0 50 100 150 200 250 300 350 400 450 0 100 200 300 400 nz = 27189 • The sizes of these 9 blocks are 60 , 60 , 60 , 60 , 60 , 60 , 60 , 33 , 30 13
An example of symmetry reduction • An SDP relaxation for cut minimization problems (Pong et al. ’14) min X � C , X � s.t. A ( X ) = b , X ≥ 0 X ∈ S 483 /////////// + ˜ 1 ( x ) ∈ S 60 B ∗ + . . . ˜ B ∗ 9 ( x ) ∈ S 30 + • After symmetry reduction, the sizes of p.s.d. constraints Original SDP 483 Symmetry reduced 60 , 60 , 60 , 60 , 60 , 60 , 60 , 33 , 30 Instance can161 14
Facial reduction
Facial reduction • Slater’s condition ( strict feasibility ) is a constraint qualification in convex optimization problems • Without strict feasibility: - the KKT conditions may not be necessary for the optimality - strong duality may not hold - small perturbations may render the problem infeasible - many solvers might run into numerical errors • Facial reduction is a regularization technique that can be used for semidefinite programs that fail strict feasibility ( Borwein, Wolkowicz, ’81 ) 16
Facial reduction • Given the SDP in standard form X {� C , X � | A ( X ) = b , X ∈ S n inf + } (2) Then exactly one of the following alternatives holds 1. The SDP (2) is strictly feasible: A ( X ) = b , X ∈ S n ++ 2. The auxiliary system is consistent: 0 � = A ∗ ( y ) ∈ S n + and � b , y � = 0 • We call A ∗ ( y ) an exposing vector • The feasible region of (2) is contained in A ∗ ( y ) ⊥ ∩ S n + 17
Facial reduction for the cut minimization problem • An SDP relaxation for the cut minimization problem (Pong et al. ’14) min X � C , X � s.t. A ( X ) = b , X ≥ 0 X ∈ S nk + where n is the number of vertices and k is the number of subsets in the partition 18
Facial reduction for the cut minimization problem • An SDP relaxation for the cut minimization problem (Pong et al. ’14) min X � C , X � s.t. A ( X ) = b , X ≥ 0 ⇒ X = VRV T , R ∈ S ( n − 1)( k − 1) X ∈ S nk ////////// = + + where the columns of V span A ∗ ( y ) ⊥ the sizes of p.s.d. constraints Original SDP 483 Facially reduced 321 Symmetry reduced 60 , 60 , 60 , 60 , 60 , 60 , 60 , 33 , 30 Instance can161 19
Facial reduction for symmetry re- duced SDP
Facial reduction for symmetry reduced SDP Theorem (H., Sotirov, Wolkowicz) Let W be an exposing vector of the minimal face of a given SDP instance. Then 1. There exists an exposing vector W G ∈ M of the minimal face of the input SDP instance 2. Q T W G Q is an exposing vector of the minimal face of the symmetry reduced SDP • In plain words, we know how to do facial reduction for the symmetry reduced SDP now 21
Facial reduction for the symmetry reduced program • An SDP relaxation for the cut minimization problem (Pong et al. ’14) min X � C , X � s.t. A ( X ) = b , X ≥ 0 X ∈ S nk + where n is the number of vertices and k is the number of subsets in the partition 22
Facial reduction for the symmetry reduced program • A symmetry reduced SDP relaxation for cut minimization problems min X � C , X � s.t. A ( X ) = b , X ≥ 0 X ∈ S 483 /////////// + ˜ B ∗ 1 ( x ) ∈ S 60 + . . . ˜ 9 ( x ) ∈ S 30 B ∗ + 23
Facial reduction for the symmetry reduced program • The facially + symmetry reduced SDP relaxation for cut minimization problems min X � C , X � s.t. A ( X ) = b , X ≥ 0 X ∈ S 483 /////////// + ˜ 1 ( x ) = ˜ V 1 ˜ R 1 ˜ 1 and ˜ B ∗ V T R 1 ∈ S 40 + , . . . ˜ 9 ( x ) = ˜ V 9 ˜ R 9 ˜ 9 and ˜ R 9 ∈ S 20 B ∗ V T + , 24
Facial reduction for symmetry reduced SDP • In the cut minimization problem, we obtain the sizes of p.s.d. constraints Original SDP 483 Facially reduced 321 Symmetry reduced 60 , 60 , 60 , 60 , 60 , 60 , 60 , 33 , 30 Facially + Symmetry 40 , 40 , 40 , 40 , 38 , 40 , 40 , 21 , 20 Instance can161 • Now lets check if our theory works? 25
Numerical results on the cut minimization problem • We solve the SDP relaxation from Pong et al. ’14 using interior point method, and the number of partition k = 3 • Instance Symmetry Facial+Symmetry bound 0.3838 0.6233 can144 iteration 35 18 time 32.27s 5.8s bound 0.4828 0.5485 can161 iteration 24 20 time 375.63s 108.05s • Without facial reduction, it takes longer time and iteration to get a weaker bound. 26
Recommend
More recommend
