Facial reduction for symmetry reduced semidefinite programs Hao Hu a - - PowerPoint PPT Presentation

Facial reduction for symmetry reduced semidefinite programs

Hao Hua , Renata Sotirova and Henry Wolkowiczb

Updated: 2019/08/07

a Tilburg University b University of Waterloo

The graph partitioning problem

The graph partitioning problem (GP)

• GP: partition the vertices of a graph into k subsets of given sizes

so that the number of edges between different subsets is minimized

• Partition the graph below into 2 sets of equal size

1 2 3 4 1 2 3 4 1 2 3 4 Input graph

• bjective value 4
• bjective value 2

2

Symmetry in the graph partitioning problem

• The input graph is ”invariant” under certain permutation of its

vertices 1 2 3 4 2 3 4 1 2 1 4 3 Input graph rotate clock-wise flip horizontally

• How can we exploit the symmetry to attack the problem?

3

Matrix ∗-algebra

Matrix ∗-algebra

• A set M ⊆ Cn×n is a matrix ∗-algebra over C if it is closed

under addition, scalar and matrix multiplication, and taking conjugate transpose, i.e., αX + βY ∈ M ∀α, β ∈ C X ∗ ∈ M XY ∈ M, for all X, Y ∈ M

5

Block diagonalization of Matrix ∗-algebra

• Theorem (Wedderburn 1907) Matrix ∗-algebras containing the

identity matrix have a canonical block-diagonal structure after some unitary transformation, i.e., there exists a unitary matrix Q and some integer t such that Q∗MQ =       M1 · · · M2 . . . . . . ... · · · Mt       , where each Mi ⊆ Cni×ni is basic

6

An example of block-diagonalization

• Consider the matrix ∗-algebra spanned by

B0 =      1 1 1 1      , B1 =      1 1 1 1 1 1 1 1      , B2 =      1 1 1 1      , where B1 is the adjacency matrix in the graph partitioning example

• The unitary matrix Q below diagonalizes B0, B1, B2

Q = 1 2      1 1 1 1 1 −i −1 i 1 −1 1 −1 1 i −1 −i     

7

An example of block-diagonalization

• The matrix ∗-algebra spanned by

B0 =      1 1 1 1      , B1 =      1 1 1 1 1 1 1 1      , B2 =      1 1 1 1     

• The (block)-diagonalized matrices ˜

Bi = QTBiQ are ˜ B0 =      1 1 1 1      , ˜ B1 =      8 −8      , ˜ B2 =      4 −4 4 −4     

8

Symmetry reduction

Semidefinite program (SDP)

• Consider an SDP in standard form

inf

X {A0, X | Ai, X = bi for i = 1, . . . , m, X ∈ Sn +},

(1) where Sn

+ is the cone of positive semidefinite matrices

Assume the data matrices A0, . . . , Am and the identity matrix are contained in a matrix ∗-algebra M. If SDP (1) has an optimal solution, then it has an optimal solution in M.

• References:

a) Kanno et al., 2001, de Klerk 2009, etc b) Gatermann, Parrilo 2004, Vallentin 2009, etc c) Schrijver 2005, Laurent 2007, etc

10

Symmetry reduction for SDP

• Assume B1, . . . , Bd is a basis of M. There exists an optimal

solution X =

d

• k=1

xkBk ∈ M

• The p.s.d. constraint X ∈ Sn

+ can be simplifies as

d

k=1 xk block-diagonal

• (QTBkQ) ∈ Sn

+

⇐ ⇒       ˜ B∗

1(x)

· · · ˜ B∗

2(x)

. . . . . . ... · · · ˜ B∗

t (x)

      ∈ Sn

+

where ˜ B∗

j (x) is the j-th block

• We have X ∈ Sn

+ if and only if ˜

B∗

j (x) ∈ Snj + for every j = 1 . . . , t

11

An example of symmetry reduction

• An SDP relaxation for the cut minimization problem (Pong et al.

’14) minX C, X s.t. A(X) = b, X ≥ 0 X ∈ Snk

+ ,

where n is the number of vertices and k is the number of subsets in the partition

• Instance can161 with n = 161 vertices and k = 3 partitions
• The size of X ∈ Snk

+ is nk = 483, and very difficult to solve

12

An example of symmetry reduction

• The feasible solutions X under certain unitary transformation,

i.e., QTXQ, has the following block-diagonal structure

100 200 300 400 nz = 27189 50 100 150 200 250 300 350 400 450

• The sizes of these 9 blocks are 60, 60, 60, 60, 60, 60, 60, 33, 30

13

An example of symmetry reduction

• An SDP relaxation for cut minimization problems (Pong et al.

’14) minX C, X s.t. A(X) = b, X ≥ 0 X ∈ S483

+

///////////          ˜ B∗

1(x) ∈ S60 +

. . . ˜ B∗

9(x) ∈ S30 +

• After symmetry reduction,

the sizes of p.s.d. constraints Original SDP 483 Symmetry reduced 60, 60, 60, 60, 60, 60, 60, 33, 30 Instance can161

14

Facial reduction

Facial reduction

• Slater’s condition (strict feasibility) is a constraint qualification

in convex optimization problems

• Without strict feasibility:
• the KKT conditions may not be necessary for the optimality
• strong duality may not hold
• small perturbations may render the problem infeasible
• many solvers might run into numerical errors
• Facial reduction is a regularization technique that can be used

for semidefinite programs that fail strict feasibility (Borwein, Wolkowicz, ’81)

16

Facial reduction

• Given the SDP in standard form

inf

X {C, X | A(X) = b, X ∈ Sn +}

(2) Then exactly one of the following alternatives holds

• 1. The SDP (2) is strictly feasible:

A(X) = b, X ∈ Sn

++

• 2. The auxiliary system is consistent:

0 = A∗(y) ∈ Sn

+ and b, y = 0

• We call A∗(y) an exposing vector
• The feasible region of (2) is contained in A∗(y)⊥ ∩ Sn

+

17

Facial reduction for the cut minimization problem

• An SDP relaxation for the cut minimization problem (Pong et al.

’14) minX C, X s.t. A(X) = b, X ≥ 0 X ∈ Snk

+

where n is the number of vertices and k is the number of subsets in the partition

18

Facial reduction for the cut minimization problem

• An SDP relaxation for the cut minimization problem (Pong et al.

’14) minX C, X s.t. A(X) = b, X ≥ 0 X ∈ Snk

+

////////// = ⇒ X = VRV T, R ∈ S(n−1)(k−1)

+

where the columns of V span A∗(y)⊥ the sizes of p.s.d. constraints Original SDP 483 Facially reduced 321 Symmetry reduced 60, 60, 60, 60, 60, 60, 60, 33, 30 Instance can161

19

Facial reduction for symmetry re- duced SDP

Facial reduction for symmetry reduced SDP

Theorem (H., Sotirov, Wolkowicz) Let W be an exposing vector of the minimal face of a given SDP instance. Then

• 1. There exists an exposing vector WG ∈ M of the

minimal face of the input SDP instance

• 2. QTWGQ is an exposing vector of the minimal face of

the symmetry reduced SDP

• In plain words, we know how to do facial reduction for the

symmetry reduced SDP now

21

Facial reduction for the symmetry reduced program

• An SDP relaxation for the cut minimization problem (Pong et al.

’14) minX C, X s.t. A(X) = b, X ≥ 0 X ∈ Snk

+

where n is the number of vertices and k is the number of subsets in the partition

22

Facial reduction for the symmetry reduced program

• A symmetry reduced SDP relaxation for cut minimization

problems minX C, X s.t. A(X) = b, X ≥ 0 X ∈ S483

+

///////////          ˜ B∗

1(x) ∈ S60 +

. . . ˜ B∗

9(x) ∈ S30 +

23

Facial reduction for the symmetry reduced program

• The facially + symmetry reduced SDP relaxation for cut

minimization problems minX C, X s.t. A(X) = b, X ≥ 0 X ∈ S483

+

///////////          ˜ B∗

1(x) = ˜

V1 ˜ R1 ˜ V T

1 and ˜

R1 ∈ S40

+ ,

. . . ˜ B∗

9(x) = ˜

V9 ˜ R9 ˜ V T

9 and ˜

R9 ∈ S20

+ ,

24

Facial reduction for symmetry reduced SDP

• In the cut minimization problem, we obtain

the sizes of p.s.d. constraints Original SDP 483 Facially reduced 321 Symmetry reduced 60, 60, 60, 60, 60, 60, 60, 33, 30 Facially + Symmetry 40, 40, 40, 40, 38, 40, 40, 21, 20 Instance can161

• Now lets check if our theory works?

25

Numerical results on the cut minimization problem

• We solve the SDP relaxation from Pong et al. ’14 using interior

point method, and the number of partition k = 3

• Instance

Symmetry Facial+Symmetry can144 bound 0.3838 0.6233 iteration 35 18 time 32.27s 5.8s can161 bound 0.4828 0.5485 iteration 24 20 time 375.63s 108.05s

• Without facial reduction, it takes longer time and iteration to get

a weaker bound.

26

Summary

Input 1 Input 2 Output SDP matrix ∗-algebra symmetry reduced SDP + reduced problem size – numerical issues SDP exposing vector facially reduced SDP + numerically stable – symmetry not exploited symmetry reduced SDP exposing vector in the algebra facially & symmetry reduced SDP + numerically stable + reduced problem size

27