BiqCrunch: a semidefinite-based solver for binary quadratic problems - - PowerPoint PPT Presentation

BiqCrunch: a semidefinite-based solver for binary quadratic problems

Nathan Krislock

PIMS Postdoctoral Fellow, Computer Science University of British Columbia, Canada

Canadian Discrete and Algorithmic Mathematics Conference Memorial University of Newfoundland June 10–13, 2013 Joint work with J´ erˆ

• me Malick (CNRS)

and Fr´ ed´ eric Roupin (Universit´ e Paris 13)

Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 1 / 22

Pacific Institute for the Mathematical Sciences

BiqCrunch

Binary Quadratic Problems

BiqCrunch is a branch & bound solver for binary quadratic problems: maximize zTA0z + bT

0 z + c0

subject to zTAiz + bT

i z = ci,

i ∈ {1, . . . , mE} zTAjz + bT

j z ≤ cj,

j ∈ {1, . . . , mI} z ∈ {0, 1}n examples of binary quadratic problems:

◮ Max-Cut / Unconstrained Binary Quadratic ◮ Maximum k-Cluster ◮ Maximum Independent Set / Quadratic Stable Set ◮ Quadratic Knapsack ◮ Quadratic Assignment ◮ . . . Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 2 / 22

BiqCrunch

BiqCrunch

uses our improved semidefinite bounding procedure written in C and Fortran and uses:

◮ L-BFGS-B, a library for quasi-Newton bound-constrained optimization ◮ BOB, a branch-and-bound framework

uses our BC (BiqCrunch) format – is very similar to the SDPA format available as an online solver

Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 3 / 22

BiqCrunch

Vector form

maximize zTA0z + bT

0 z + c0

subject to zTAiz + bT

i z = ci,

i ∈ {1, . . . , mE} zTAjz + bT

j z ≤ cj,

j ∈ {1, . . . , mI} z ∈ {0, 1}n

Matrix form

maximize Q0, Z subject to Qi, Z = ci, i ∈ {1, . . . , mE} Qj, Z ≤ cj, j ∈ {1, . . . , mI} Z = zzT z zT 1

• ,

z ∈ {0, 1}n Q0 = A0

1 2b0 1 2bT

c0

• , Qi =

Ai

1 2bi 1 2bT i

• , Qj =

Aj

1 2bj 1 2bT j

• Nathan Krislock (PIMS, UBC)

BiqCrunch: SDP-based solver for BQP CanaDAM 2013 4 / 22

Max-Cut

Max-Cut

maximize

• ij∈E

wij 1 − xixj 2

• subject to

x ∈ {−1, 1}n G = (V , E) (n = |V |)

Equivalent formulation

(MC) maximize xTQx subject to x ∈ {−1, 1}n Q = 1

4L ∈ Sn

(L is the Laplacian matrix of G)

Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 6 / 22

Max-Cut

Max-Cut

maximize

• ij∈E

wij 1 − xixj 2

• subject to

x ∈ {−1, 1}n G = (V , E) (n = |V |)

Equivalent formulation

(MC) maximize xTQx subject to x ∈ {−1, 1}n Q = 1

4L ∈ Sn

(L is the Laplacian matrix of G)

Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 6 / 22

Max-Cut

Max-Cut

(MC) maximize xTQx subject to x ∈ {−1, 1}n

X = xxT, x ∈ {−1, 1}n ⇐ ⇒ diag(X) = e, X 0, rank(X) = 1

(MC) maximize Q, X subject to diag(X) = e, X 0 rank(X) = 1 Q, X =

ij QijXij = trace(QX)

diag(X) = (X11, . . . , Xnn)T, e = (1, . . . , 1)T X 0 : X is symmetric and positive semidefinite

Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 7 / 22

Semidefinite Relaxation of Max-Cut

Semidefinite relaxation

(SDP) maximize Q, X subject to diag(X) = e, X 0 gives an upper bound on Max-Cut: (MC) ≤ (SDP) tight bounds needed for B&B methods to solve Max-Cut efficiently Goemans and Williamson (1995): (SDP) < 1.14(MC) can be solved very efficiently by a primal-dual interior-point method bound is too weak to be used to solve Max-Cut efficiently, so we add some triangle inequalities

Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 8 / 22

Semidefinite Relaxation of Max-Cut

Enhancing the SDP bound

There are 4 n

3

• triangle inequalities (for 1 ≤ i < j < k ≤ n):

Xij + Xik + Xjk ≥ −1, −Xij + Xik − Xjk ≥ −1 Xij − Xik − Xjk ≥ −1, −Xij − Xik + Xjk ≥ −1 Choosing a subset I of the inequalities, we have (SDPI) maximize Q, X subject to diag(X) = e, X 0 AI(X) ≥ −e can greatly enhance the bound: (MC) ≤ (SDPI) ≤ (SDP)

Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 9 / 22

Our Semidefinite Bounds

A simple fact

diag(X) = e, X 0 = ⇒ X2

F ≤ n2

Our semidefinite bounds

(SDPα

I)

maximize Q, X + α

2

• n2 − X2

F

• subject to

diag(X) = e, X 0 AI(X) ≥ −e for α ≥ 0: (SDPI) ≤ (SDPα

I)

smaller α gives tighter upper bounds: α < α′ = ⇒ (SDPα

I) < (SDPα′ I )

limα→0(SDPα

I) = (SDPI)

Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 10 / 22

Our Semidefinite Bounds

Dual function F α

I (y, z)

The Lagrangian is (y ∈ Rn, z ∈ R|I|

+ ):

L(X; y, z) := Q, X+α 2

• n2 − X2

F

• + y, e − diag(X) + z, e + AI(X)

The dual function is: F α

I (y, z) := max X0 L(X; y, z)

= 1 2α

• XI(y, z)
• 2

F + eTy + eTz + α

2 n2 where XI(y, z) :=

• Q − Diag(y) + A∗

I(z)

• +

Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 11 / 22

Our Semidefinite Bounds

Dual function F α

I (y, z)

The Lagrangian is (y ∈ Rn, z ∈ R|I|

+ ):

L(X; y, z) := Q − Diag(y) + A∗

I(z), X − α

2 X2

F

+ eTy + eTz + α 2 n2 The dual function is: F α

I (y, z) := max X0 L(X; y, z)

= 1 2α

• XI(y, z)
• 2

F + eTy + eTz + α

2 n2 where XI(y, z) :=

• Q − Diag(y) + A∗

I(z)

• +

Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 11 / 22

Our Semidefinite Bounds

Generic semidefinite relaxation

(SDP) maximize Q, X subject to AX ≤ a BX = b X 0 diag(X) = e constraints included in the equality constraints BX = b

Generic semidefinite bounds

F α(y, z) = 1 2α

• X(y, z)
• 2

F + bTy + aTz + α

2 n2 X(y, z) =

• Q − B∗(y) + A∗(z)
• +

Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 12 / 22

Our Semidefinite Bounds

Computing the best bound F α

I (y, z)

(DSDPα

I)

minimize F α

I (y, z)

subject to z ≥ 0

(DSDPα

I) is a smooth convex optimization problem

∇yF α

I (y, z) = e − diag

1 αXI(y, z)

• ∇zF α

I (y, z) = e + AI

1 αXI(y, z)

• can be solved with a quasi-Newton method like L-BFGS-B

ill-conditioned for very small α

Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 13 / 22

Efficiently Computing the Bounds

Algorithm: Improved semidefinite bounding procedure

Given: Initial vectors and inequalities: y0 = 0 ∈ Rn, I0 = ∅, z0 = 0 ∈ R0 Scalars: α1, ε1 > 0 Parameters: 0 < θ ≤ 1 and 0 < ρ ≤ 1 For k = 1, 2, . . . do:

1 Starting from (yk−1, zk−1), compute (yk, ˆ

zk) such that max

• ∇yF αk

Ik−1(yk, ˆ

zk)

• ∞,
• ∇zF αk

Ik−1(yk, ˆ

zk)

• −
• ∞
• < εk

2 Update I, α, ε:

◮ Add / remove triangle inequalities to get Ik and zk ◮ Reduce α: αk+1 ← θαk ◮ Reduce ε: εk+1 ← ρεk

Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 14 / 22

Efficiently Computing the Bounds

Convergence Theorem

If: (X k, yk, zk, Ik) generated by Algorithm, where X k :=

1 αk XIk(yk, zk)

αk → 0 and εk → 0 ( ¯ X, ¯ y, ¯ z, ¯ I) is an accumulation point of (X k, yk, zk, Ik) Then: ( ¯ X, ¯ y, ¯ z) is a primal-dual optimal solution for (SDP¯

I)

the bounds converge to (SDP¯

I):

lim

k→∞ F αk Ik (yk, zk) = (SDP¯ I)

Furthermore, if all violated inequalities are added (in the limit), we have lim

k→∞ F αk Ik (yk, zk) = (SDPIall)

Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 15 / 22

Comparing to Biq Mac

Our bounds: Fα

I (y, z)

F α

I (y, z) = 1 2α

• Q − Diag(y) + A∗

I(z)

• +
• 2

F + eTy + eTz + α 2 n2

F α

I is convex and smooth

evaluate F α

I (y, z) and ∇F α I (y, z) by computing a partial eigenvalue

decomposition minimize F α

I (y, z) by a quasi-Newton method

Biq Mac bounds: θI(z)

θI(z) = eTz + minimize Q + A∗

I(z), X

subject to diag(X) = e, X 0 θI is convex and nonsmooth evaluate θI(z) and φ ∈ ∂θI(z) by solving an SDP minimize θI(z) by a bundle method

Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 16 / 22

Comparing to Biq Mac

Our bounds: Fα

I (y, z)

F α

I (y, z) = 1 2α

• Q − Diag(y) + A∗

I(z)

• +
• 2

F + eTy + eTz + α 2 n2

F α

I is convex and smooth

evaluate F α

I (y, z) and ∇F α I (y, z) by computing a partial eigenvalue

decomposition minimize F α

I (y, z) by a quasi-Newton method

Biq Mac bounds: θI(z)

θI(z) = eTz + minimize Q + A∗

I(z), X

subject to diag(X) = e, X 0 θI is convex and nonsmooth evaluate θI(z) and φ ∈ ∂θI(z) by solving an SDP minimize θI(z) by a bundle method

Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 16 / 22

Comparing to Biq Mac

Our bounds: Fα

I (y, z)

F α

I (y, z) = 1 2α

• Q − Diag(y) + A∗

I(z)

• +
• 2

F + eTy + eTz + α 2 n2

F α

I is convex and smooth

evaluate F α

I (y, z) and ∇F α I (y, z) by computing a partial eigenvalue

decomposition minimize F α

I (y, z) by a quasi-Newton method

Biq Mac bounds: θI(z)

θI(z) = eTz + minimize Q + A∗

I(z), X

subject to diag(X) = e, X 0 θI is convex and nonsmooth evaluate θI(z) and φ ∈ ∂θI(z) by solving an SDP minimize θI(z) by a bundle method

Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 16 / 22

Comparing to Biq Mac

50 100 150 200 41,200 41,300 41,400 41,500 41,600 41,700 41,800 41,900 42,000 42,100 42,200 Bound CPU Time (s) 10 20 30 40 50 41,200 41,400 41,600 41,800 42,000 42,200 Our method Biq Mac

Nathan Krislock, J´ erˆ

• me Malick, and Fr´

ed´ eric Roupin. (2012) Improved semidefinite bounding procedure for solving Max-Cut problems to optimality. Mathematical Programming. Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 17 / 22

Comparing to Biq Mac

2 4 6 8 10 12 14 0.2 0.4 0.6 0.8 1 τ θ (τ ) Our method (R3) Our method (R2) Biq Mac (R2) Biq Mac (R3) Nathan Krislock, J´ erˆ

• me Malick, and Fr´

ed´ eric Roupin. (2012) Improved semidefinite bounding procedure for solving Max-Cut problems to optimality. Mathematical Programming. Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 18 / 22

Maximum k-cluster

Maximum k-cluster

maximize

• ij∈E wijzizj

subject to n

i=1 zi = k

z ∈ {0, 1}n G = (V , E) (n = |V |)

Reinforcing equality constraints

(KC) maximize

1 2zTWz

subject to n

i=1 zi = k

n

i=1 zizj = kzj,

j ∈ {1, . . . , n} z ∈ {0, 1}n

Alain Faye and Fr´ ed´ eric Roupin. (2007) Partial Lagrangian relaxation for general quadratic programming. 4OR: A Quarterly Journal of Operations Research. Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 19 / 22

Results for maximum k-cluster

Medium problems

n k d(%) nodes time (s) 100 25 25 20.6 31 50 35.0 42 75 30.6 32 50 25 3.4 5 50 25.4 46 75 1.4 2 75 25 1.0 2 50 1.8 4 75 1.0 2 n k d(%) nodes time (s) 120 30 25 119.4 316 50 194.2 425 75 422.2 890 60 25 59.8 199 50 85.8 263 75 43.0 143 90 25 1.8 7 50 22.2 97 75 1.0 3

• n average 8 times faster than previous best methods

up to more than 60 times faster than previous best methods

Nathan Krislock, Jerome Malick, and Fr´ ed´ eric Roupin. (2012) Improved semidefinite branch-and-bound algorithm for k-cluster. Technical Report, INRIA Grenoble Rhˆ

• ne-Alpes.

Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 20 / 22

Results for maximum k-cluster

Large probems

n k d(%) nodes time (s) 140 35 25 366.2 1166 50 1063.4 2889 75 1558.6 4080 70 25 134.2 543 50 780.6 3035 75 52.2 203 105 25 2.6 14 50 11.0 62 75 6.6 35 n k d(%) nodes time (s) 160 40 25 744.6 2856 50 11325.4 37565 75 8050.6 26303 80 25 395.4 1835 50 993.4 4655 75 3829.0 18654 120 25 31.4 220 50 17.4 143 75 9.8 82

first time problems of size n = 140 and n = 160 solved

Nathan Krislock, Jerome Malick, and Fr´ ed´ eric Roupin. (2012) Improved semidefinite branch-and-bound algorithm for k-cluster. Technical Report, INRIA Grenoble Rhˆ

• ne-Alpes.

Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 21 / 22

Summary & Future Work

Summary

BiqCrunch available as an online solver able to solve 75% of the test problems faster than the leading Biq Mac method able to solve k-cluster problems much faster than previously possible more numerical results on BiqCrunch website

Future Work

make a public release of the BiqCrunch code make a version of BiqCrunch to solve Quadratic Assignment Problems investigate facial reduction to handle problems with a semidefinite relaxation that is not strictly feasible

Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 22 / 22