Impact of different levels of quadratic convex reformulation for - - PowerPoint PPT Presentation

Impact of different levels of quadratic convex reformulation for E-kQKPs

Monique GUIGNARD OPIM, University of Pennsylvania Lucas LETOCART University of Paris-13 Michael BUSSIECK GAMS Dev. Corp. MINLP workshop, Sevilla, 3/30 to 4/1, 2015

Xiaoling Sun passed away April 24, 2014. He was here for the Seville MINLP workshop in 2010. This talk owes a lot to his work.

Quadratic Convex Reformulations

Replace (P) Max{ ( )

. . , {0,1 } }

n

f x s t x X x  

by (P’) Max{ ( )

. . , {0,1 } }

n

g x s t x X x  

where X is a polyhedron, f(x) and g(x) are quadratic functions of x binary and g is concave.

Quadratic Convex Reformulations (QCR)

Replace (P) Max{ ( )

. . , {0,1 } }

n

f x s t x X x  

by (P’) Max{ ( )

. . , {0,1 } }

n

g x s t x X x  

where X is a polyhedron, f(x) and g(x) are quadratic functions of x binary and g is concave (convex if you minimize). Idea 1: replace f(x) by a concave function g(x) which coincides with f(x) at every x feasible for (P).

FOR A MIN PROBLEM: This shows an example with two convex functions of one variable that coincide for x=0 and for x=1, but with clearly different minima for x∈[0, 1].

y=g2(x) y=g1(x) 1

Quadratic Convex Reformulations (QCR)

Replace (P) Max{ ( )

. . , {0,1 } }

n

f x s t x X x  

by (P’) Max{ ( )

. . , {0,1 } }

n

g x s t x X x  

where X is a polyhedron, f(x) and g(x) are quadratic functions of x binary and g is concave. Idea 1: replace f(x) by a concave function g(x) which coincides with f(x) at every x feasible for (P). idea 2: choose g(x) such that the continuous relaxation bound from (P’) is as tight as possible among all possible g of a certain type.

APPLIED TO E-kQKP

QKP: quadratic knapsack problem E-kQKP: quadratic knapsack problem with a cardinality constraint

Problem (E − kQKP) Formulation

Formulation

Mathematical formulation (E-kQKP)                            max f(x) =

n

• i=1

n

• j=1

cijxixj s.t. n

j=1 ajxj ≤ b

n

j=1 xj = k

xj ∈ {0, 1} j = 1, . . . , n without constraint 2 : the 0-1 quadratic knapsack problem (QKP) without constraint 1 : the k-cluster problem

(1) (2)

Problem (E − kQKP) Formulation

Formulation

Notations n : number of items aj : weight of item j (j = 1, . . . , n) b : capacity of the knapsack cij : profit associated with the selection of items i and j (i, j = 1, . . . , n) k : number of items to be filled in the knapsack Assumptions cij ∈ N i, j = 1, . . . , n, aj ∈ N j = 1, . . . , n, b ∈ N maxj=1,...,n aj ≤ b < n

j=1 aj

k ∈ {2, . . . , kmax}

−

Complexity and experimental environment

Complexity NP-hard problem : generalization of (QKP) and k-cluster problems Experimental environment Carried out on SMP partition of MAGI using 30 cores Xeon 2 GHz with 500 GB of RAM CSDP integrated into COIN-OR for solving SDP programs IBM-CPLEX 12.4 : standard solver (with default settings) Average values over 10 instances n ∈ {10, 20, . . . , 200} k ∈ [1, n/4], b ∈ [50, 30k], aj, cij ∈ [1, 100]

Lucas L´ etocart (LIPN) Multithread et OC JCS P13 6 / 57

Quadratic Convex Reformulations

Replace (P) Max { ( ) . . , {0,1 } }

n

f x st x X x  

by (P') Max { ( )

. . , {0,1 } }

n

g x s t x X x  

   

where and are quadratic functions of binary f x g x x  

and is concave of the form g x

( ) ( ) ( ) g x f x w x  

where ( ) 0 whenever is feasible for (P). w x x 

PRIOR PROGRESSIVE IMPROVEMENTS:

Hammer and Rubin: eigenvalue method to get a uniform diagonal modification. Alain Billionnet, Sourour Elloumi: solve an SDP problem to get the best diagonal modifications. Alain Billionnet, Sourour Elloumi, MC Plateau: solve an SDP problem to get the best entire matrix modification using equality constraints if any. Shuhui Ji, Xiaojin Zheng and Xiaoling Sun: extension of the whole matrix modification.

−

Quadratic Convex Reformulation (QCR) : a two-phase method

[Billionnet, Elloumi and M-C. Plateau, Discrete Applied Mathematics, 2009] Phase 1 : Reformulate the objective function f(x) into an equivalent 0-1 program with a concave quadratic objective function fu,α(x) depending

• n two parameters u and α both in Rn.

→ an equivalent convex 0-1 program Phase 2 : Apply a standard 0-1 convex quadratic solver to this new problem. QCR - Phase I : Addition of two functions null on the feasible set qu(x) =

n

• i=1

ui

• x2

i − xi

• qα(x) =

n

• i=1

αixi

• n
• j=1

xj − k

• r

qv(x) = v

• n
• j=1

xj − k 2

−

QCR : A two-phase method

QCR - Phase I : Reformulation 1 New objective function (concave iff quadratic terms matrix is SDP) : fu,α(x) = f(x) −

n

• i=1

ui

• x2

i − xi

• −

n

• i=1

αixi  

n

• j=1

xj − k   Reformulation 2 [Faye, Roupin, 4OR, 2007 ; Billionnet, Elloumi and Lambert, Math. Prog., 2009] fu,v(x) = f(x) −

n

• i=1

ui

• x2

i − xi

• − v

 

n

• j=1

xj − k  

2

identical bounds consumes the least amount of computation times

introduce new variables

ij

X to replace the products

i j

x x , do not write

ij i j

X x x  because it is nonlinear, but add constraints that come from the definition of these new variables (X is viewed as a matrix) defined by

1 1 1 1 n n n n

x x x x X x x x x            = xxT this implies that  X is of rank 1 (ignore)  X - xxT = 0 (still nonlinear !) therefore relax as X - xxT 0 (nonlinear, or equivalently (*))

T

1 x x X       

2

( )

ii i i

X x x  

________________________________________________________ (*) Schur’s lemma says that

1

A y C xA y x C



        ________________________________________________________

−

QCR : A two-phase method

Best values for the parameters u ∈ Rn and v ∈ R Solve the semi-definite relaxation of the problem (E − kQKP) :

(SDP)                                                      max n

i=1

n

j=1 cijXij

s.t. Xii = xi i = 1, . . . , n (u∗)

n

• i=1

n

• j=1

Xij − 2k

n

• j=1

xj = −k2 (v∗) n

j=1 ajxj ≤ b

n

j=1 xj = k

1 xt x X

• x ∈ Rn, X ∈ Rn×n

Lucas L´ etocart (LIPN) Multithread et OC JCS P13 9 / 57

−

An improved convex 0-1 quadratic reformulation for (E − kQKP) (1/4) [S. Ji, X. Zheng and X. Sun, Pacific Journal of Optimization, 2012] From f(x) = xtCx Consider the decomposition of f(x) f(x) = xt(C − M)x + xtMx, where C − M 0 and M = Diag(u) + P + N with u ∈ Rn, P ∈ Rn×n

+

and N ∈ Rn×n

−

−

An improved convex 0-1 quadratic reformulation for (E − kQKP) (2/4) f(x) = xt(C − Diag(u) − P − N)x + xt(Diag(u) + P + N)x = xt(C − Diag(u) − P − N)x + utx + n

i,j=1[Pijxixj + Nijxixj]

= xt(C − Diag(u) − P − N)x + utx + n

i,j=1[Pijsij − Nijtij]

where sij = min(xi, xj), tij = −max(0, xi + xj − 1) (xixj = min(xi, xj) = max(0, xi + xj − 1) for any xi, xj ∈ {0, 1})

−

An improved convex 0-1 quadratic reformulation for (E − kQKP) (3/4) Relax sij = min(xi, xj) to two linear inequalities sij ≤ xi and sij ≤ xj and tij = −max(0, xi + xj − 1) to tij ≤ 0 and tij ≤ 1 − xi − xj . without affecting the optimal solution of the problem. The reformulation is equivalent to the convex 0-1 quadratic program :                  max f(x) = xt(C − Diag(u) − P − N)x + utx + n

i,j=1[Pijsij + Nijtij]

s.t. n

j=1 ajxj ≤ b

n

j=1 xj = k

sij ≤ xi, sij ≤ xj i, j = 1, . . . , n tij ≤ 0, tij ≤ 1 − xi − xj i, j = 1, . . . , n xj ∈ {0, 1} j = 1, . . . , n

−

An improved convex 0-1 quadratic reformulation for (E − kQKP) (4/4) The optimal parameters (u∗, v∗, P∗, N∗) can be found by solving a SDP :

                                                                 max n

i=1

n

j=1 cijXij

s.t. Xii = xi i = 1, . . . , n (u∗)

n

• i=1

n

• j=1

Xij − 2k

n

• j=1

xj = −k2 i = 1, . . . , n (v∗) n

j=1 ajxj ≤ b

n

j=1 xj = k

Xij ≤ xi, Xij ≤ xj i, j = 1, . . . , n (P∗) Xij ≥ xi + xj − 1, Xij ≥ 0 i, j = 1, . . . , n (N∗) 1 xt x X

• x ∈ Rn, X ∈ Sn

(1 and 2) (3 and 4)

−

How to manage the 4n2 bound constraints (1) Xij ≤ xi i, j = 1, . . . , n ; (2) Xij ≤ xj i, j = 1, . . . , n (3) Xij ≥ xi + xj − 1 i, j = 1, . . . , n ; (4) Xij ≥ 0 i, j = 1, . . . , n IQCR : Ji, Zheng and Sun propose to keep only the constraints such that | i − j | ≤ 5 MQCR (our iterative method) : V ← φ Repeat Solve the SDP problem with constraints of V /*QCR at 1st iteration*/ Detect the violated constraints of (1) such that Xij > xi + 10−4 Keep the set V of the most violated constraints (at most 1000 + 15n) Until the number of violated constraints is less than 30

• constraints (2) : transformed into constraints (1) thanks to the

symmetry of X

• constraints (3) and (4) : small number of violated constraints ; weak

improvement of the bound

QUADRATIC CONVEX REFORMULATION (QCR) P.L. Hammer, A. Rubin: Some remarks on quadratic programming with 0-1 variables, RIRO, (3), 67-79, 1970.

• A. Billionnet, S. Elloumi: Using a Mixed Integer Quadratic

Programming Solver for the Unconstrained Quadratic 0-1 Problem, Math.

• Program. 109(1), 55-68, 2007.
• A. Billionnet, S. Elloumi, M.-Ch. Plateau: Improving the performance of

standard solvers for quadratic 0-1 programs by a tight convex reformulation: The QCR method, Discrete Applied Mathematics 157(6), 1185-1197, 2009. __________________________________________________________ GAMS Dev. and M. Guignard, GQAPSDP: SDP Convexifications of the Generalized Quadratic Assignment Problem (GQAPSDP,SEQ=339), GAMS Model Library, 2008.

GAMS Dev. and M. Guignard, KQKPSDP: SDP Convexifications of the Cardinality Constraint Quadratic Knapsack Problem (KQKPSDP,SEQ=355), GAMS Model Library, 2011. ___________________________________________________________ MQCR

• S. H. Ji, X. J. Zheng, X. L. Sun: An improved convex 0-1 quadratic

program reformulation for quadratic knapsack problems,Pacific Journal

• f Optimization, 8(1), 75-87, 2012.
• M. Guignard, L. Letocart, G. Plateau: MQCR: improved QCR for the 0-1

exact k-item quadratic knapsack problem, Working Paper, LIPN, Univ. Paris Nord, Feb. 2012, revised April 2015.

\delta = 100 n CPLEX 12.6 HR UQCR QCR MQCR Gap root % Gap Heur % Gap root % Gap Heur % Gap root % Gap Heur % Gap root % Gap Heur v % Gap Heur alpha % Gap root % Gap Heur v % Gap Heur alpha % 200 2343.5 3.1 5184.5 6.0 2547.7 6.0 151.6 4.6 4.6 146.0 4.6 4.6 412.0 1.0 850.8 0.7 412.7 0.7 9.8 0.1 0.1 9.1 0.1 0.1 1277.8 0.8 2605.1 3.6 1280.1 3.6 62.6 1.7 1.7 60.4 0.0 1.0 306.4 1.4 722.2 0.0 347.5 0.1 7.7 0.1 0.0 7.0 0.0 0.0 539.5 3.7 1724.7 2.4 773.8 2.2 30.9 0.0 0.0 27.6 2.3 0.0 101.2 0.3 503.9 0.1 199.4 0.2 3.9 0.1 0.1 2.3 0.0 0.1 91.1 0.3 1033.2 0.0 237.5 0.0 7.8 0.0 0.0 4.2 0.1 0.0 258.0 0.5 548.9 0.5 264.6 0.4 4.8 0.0 0.0 4.1 0.1 0.8 177.7 1.2 969.1 0.0 343.6 0.0 10.2 0.0 0.0 6.8 0.0 0.0 26.4 0.2 245.7 0.4 76.2 0.4 1.3 0.0 0.0 0.3 0.3 0.0 \delta = 25 n CPLEX 12.6 HR QCR MQCR Gap root % Gap Heur % Gap root % Gap Heur % Gap root % Gap Heur % Gap root % Gap Heur v % Gap Heur alpha % Gap root % Gap Heur v % Gap Heur alpha % 200 428.0 2.3 969.2 14.4 446.3 0.0 69.1 0.5 0.5 64.1 2.3 2.3 No_Sol No_Sol No_Sol No_Sol No_Sol No_Sol No_Sol No_Sol No_Sol No_Sol No_Sol No_Sol 62.5 0.0 818.6 1.0 189.0 0.0 28.4 0.0 0.0 13.1 0.0 0.0 285.7 7.1 755.1 0.0 336.4 1.7 45.9 0.0 0.0 41.8 1.6 0.0 275.6 0.8 629.4 0.2 282.3 1.7 32.6 0.0 0.2 29.5 0.0 0.1 37.1 0.9 320.0 0.9 103.8 0.3 7.1 0.3 0.3 2.6 0.3 0.3 124.6 0.0 305.8 0.2 129.1 0.4 8.0 0.2 0.2 5.4 0.2 0.8 157.6 1.7 395.5 2.3 172.2 2.5 14.8 0.0 0.0 12.7 1.6 0.0 173.4 0.0 479.4 1.6 207.7 2.0 20.8 0.8 1.5 17.4 0.9 0.9 205.1 1.8 557.4 1.8 244.9 1.0 23.2 0.5 0.5 20.7 1.2 0.5

BiqCrunch User’s Guide

Nathan Krislock, Jérôme Malick, Frédéric Roupin

October 29, 2014

n CPLEX 12.6 MQCR BiqCrunch BestHybrid Gap root % Gap Heur % Gap root % Gap Heur v % Gap Heur alpha % Gap root % Gap Heur % Gap Heur % 200 2343.5 3.1 146.0 4.6 4.6 116.9 2.1 0.0 412.0 1.0 9.1 0.1 0.1 3.8 0.1 0.0 1277.8 0.8 60.4 0.0 1.0 37.0 0.2 1.6 306.4 1.4 7.0 0.0 0.0 2.9 0.0 0.4 539.5 3.7 27.6 2.3 0.0 15.0 1.1 3.4 101.2 0.3 2.3 0.0 0.1 1.0 0.0 2.5 91.1 0.3 4.2 0.1 0.0 3.6 0.0 0.1 258.0 0.5 4.1 0.1 0.8 1.7 0.0 0.0 177.7 1.2 6.8 0.0 0.0 2.9 0.0 0.0 26.4 0.2 0.3 0.3 0.0 0.2 1.6 0.1 \delta = 25 n CPLEX 12.6 MQCR BiqCrunch BestHybrid Gap root % Gap Heur % Gap root % Gap Heur v % Gap Heur alpha % Gap root % Gap Heur % Gap Heur % 200 428.0 2.3 64.1 2.3 2.3 35.5 3.7 2.3 No_Sol No_Sol No_Sol No_Sol No_Sol No_Sol No_Sol No_Sol 62.5 0.0 13.1 0.0 0.0 8.4 0.0 0.0 285.7 7.1 41.8 1.6 0.0 21.2 0.8 6.1 275.6 0.8 29.5 0.0 0.1 13.0 0.0 0.0 37.1 0.9 2.6 0.3 0.3 1.0 0.0 0.3 124.6 0.0 5.4 0.2 0.8 2.0 0.0 3.5 157.6 1.7 12.7 1.6 0.0 5.4 0.0 7.7 173.4 0.0 17.4 0.9 0.9 6.9 0.0 3.9 205.1 1.8 20.7 1.2 0.5 8.6 0.0 2.0

\delta = 100 n Instance CPLEX 12.6 HR UQCR QCR MQCR BiqCrunch Time (s) Time (s) Time (s) Time (s) (u,v) Time (s) (u, alpha) Time (s) (u,v) Time (s) (u, alpha) Time (s) 50 1 0.13 0.03 0.04 0.14 0.99 1.87 0.85 2 0.41 0.03 0.04 0.14 0.99 1.87 1.56 3 0.1 0.03 0.04 0.13 0.99 1.87 1.5 4 0.22 0.03 0.04 0.16 0.99 1.87 0.63 5 0.29 0.03 0.04 0.14 0.99 1.87 1.54 6 0.12 0.03 0.05 0.17 0.99 1.87 0.47 7 0.17 0.03 0.04 0.14 0.99 1.87 1.17 8 0.25 0.02 0.04 0.14 0.99 1.87 0.78 9 0.41 0.02 0.04 0.14 0.99 1.87 1.39 10 0.26 0.03 0.04 0.14 0.99 1.87 0.56 200 1 23.96 0.03 0.67 0.95 15.8 970.02 1603.05 24.59 2 23.07 0.03 0.76 1.08 14.8 970.02 1603.05 22.07 3 12.83 0.02 0.69 1.07 15.71 970.02 1603.05 13.28 4 40.34 0.03 0.7 1.04 15.76 970.02 1603.05 19.97 5 33.16 0.03 0.7 1.12 15.65 970.02 1603.05 16.8 6 32.13 0.03 0.94 1.27 18.47 970.02 1603.05 34.15 7 9.69 0.03 1.05 2.63 21.15 970.02 1603.05 18.99 8 43.64 0.03 0.75 0.99 15.72 970.02 1603.05 26.02 9 11.67 0.03 0.91 1.23 23.03 970.02 1603.05 28.53 10 8.03 0.03 1.06 1.33 22.1 970.02 1603.05 26.35

BiqCrunch BiqCrunch is a semidefinite-based solver for binary quadratic problems. It uses a branch-and-bound method featuring an improved semidefinite bounding procedure [3], mixed with a polyhedral approach (see [1,2] for details). BiqCrunch uses a particular BC input file format that is similar to the SDPA format to describe the combinatorial

• problems. We provide also a LP to BC conversion tool .

Documentation is available here. The first release of BiqCrunch is now available as a free and open-source software. Specific versions are provided for solving Max-Cut, k-cluster and Max-independent set problems, as well as conversion tools for these

• problems. You can also use BiqCrunch online to solve any

0-1 quadratic problem to optimality, or simply to get an SDP- quality bound [3].

 [1] N. Krislock, J. Malick, F. Roupin. Improved semidefinite

branch-and-bound algorithm for k-cluster.Submitted.

 [2] N. Krislock, J. Malick, F. Roupin: Improved

semidefinite bounding procedure for solving Max-Cut problems to optimality. Mathematical Programming A 143(1/2): 61-86, 2014.

 [3] J. Malick, F. Roupin: On the bridge between

combinatorial optimization and nonlinear optimization: a family of semidefinite bounds for 0-1 quadratic problems leading to quasi-Newton methods.Mathematical Programming B (140)1 pp 99-124, 2013.

Next?  Run BiqCrunch’s BB phase to see how it behaves.  Do similar experiments with the max-GQAP, this is an extremely difficult problem, with applications in logistics. Until now, there is no way of getting optimal solutions for problems with more than about 180 0-1 variables.  Are there instances where the knowledge of an equivalent concave (for max) objective function would be useful?

Thank You!