A computational study of Gomory cut generators ejols 1 , Fran cois - - PowerPoint PPT Presentation

A computational study of Gomory cut generators

Gerard Cornu´ ejols1, Fran¸ cois Margot1, Giacomo Nannicini2

• 1. CMU Tepper School of Business, Pittsburgh, PA.
• 2. Singapore University of Technology and Design, Singapore, and

MIT Sloan School of Management, Cambridge, MA.

January 12, 2012

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 1 / 36

Summary of Talk

1

Motivation

2

Cut generation parameters

3

Failures and Feasibility

4

Dive-and-Cut

5

Computational framework

6

Optimizing GMI cut generators

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 2 / 36

1

Motivation

2

Cut generation parameters

3

Failures and Feasibility

4

Dive-and-Cut

5

Computational framework

6

Optimizing GMI cut generators

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 3 / 36

The problem

We consider the following problem: min c⊤x Ax ≥ b x ∈ Rn

+

∀j ∈ NI xj ∈ Z,        (MILP) where c ∈ Qn, b ∈ Qm, A ∈ Qm×n and NI ⊂ {1, . . . , n} We denote by (LP) the Linear Programming relaxation of (MILP)

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 4 / 36

Introduction

General-purpose solvers for (MILP) rely on Branch-and-Cut Gomory Mixed-Integer (GMI) [Gomory, 1960] cuts are one of the most important cutting plane families [Balas et al., 1996, Bixby and Rothberg, 2007] used by state-of-the-art software Surprisingly, there is no rigorous study of GMI cut generation in the literature

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 5 / 36

1

Motivation

2

Cut generation parameters

3

Failures and Feasibility

4

Dive-and-Cut

5

Computational framework

6

Optimizing GMI cut generators

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 6 / 36

The GMI formula

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 7 / 36

GMI cuts for dummies

Cut generation loop:

1

Solve LP relaxation

2

For each basic integer variable that has a fractional value, generate a GMI cut

3

Add round of cuts to the LP relaxation

4

Repeat

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 8 / 36

GMI cuts for dummies

Cut generation loop:

1

Solve LP relaxation

2

For each basic integer variable that has a fractional value, generate a GMI cut

3

Add round of cuts to the LP relaxation

4

Repeat

That’s it!

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 8 / 36

GMI cuts for dummies

Cut generation loop:

1

Solve LP relaxation

2

For each basic integer variable that has a fractional value, generate a GMI cut

3

Add round of cuts to the LP relaxation

4

Repeat

That’s it! . . . is it?

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 8 / 36

How I wish things were that simple

This scheme is not a good idea Because of floating point (finite precision) arithmetic, not all rational numbers can be represented exactly: round-off error

◮ For q ∈ Q, let R(q) be its representation as a 64-bit floating point

number

◮ We can have q1 + q2 = q3 but R(q1) + R(q2) = R(q3)!

Operations on numbers can inflate errors In rational arithmetic, computations can be checked exactly [Espinoza, 2006, Cook et al., 2011] Typically, we use floating point because of its rapidity

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 9 / 36

GMI cuts in practice

Several operations are performed on generated cutting planes to reduce occurrence of numerical problems A number of empirical rules are applied in open-source codes (COIN-OR Cgl, SCIP)

◮ Cut modification ◮ Cut rejection

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 10 / 36

GMI cuts in practice

Several operations are performed on generated cutting planes to reduce occurrence of numerical problems A number of empirical rules are applied in open-source codes (COIN-OR Cgl, SCIP)

◮ Cut modification ◮ Cut rejection

Analysis applies to all tableau cut generators

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 10 / 36

Cut modification and rejection routines

Cut modification (and its parameters)

◮ Coefficient Removal (EPS ELIM for surplus variables, EPS COEFF

for original variables)

◮ Rhs Relaxation (EPS RELAX ABS, EPS RELAX REL)

Cut rejection (and its parameters)

◮ Fractionality Check (AWAY) ◮ Dynamism Check (MAX DYN) ◮ Support Check (MAX SUPP ABS, MAX SUPP REL) ◮ Violation Check (MIN VIOL)

Scaling – ignored here Some cut generators distinguish variables with large bounds (≥ LUB) and treat them separately (e.g. EPS COEFF LUB, MAX DYN LUB)

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 11 / 36

Cut generation parameters

12 parameters involved in the generation of each cutting plane Their value is chosen after tests on a small number of instances Are all the parameters necessary? What value should they take? To answer these questions, we need a framework for testing cut generators

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 12 / 36

Strength vs Safety

Properties of a good cut generator:

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 13 / 36

Strength vs Safety

Properties of a good cut generator:

◮ Decreases solution time with Branch-and-Cut

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 13 / 36

Strength vs Safety

Properties of a good cut generator:

◮ Decreases solution time with Branch-and-Cut ← Strength

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 13 / 36

Strength vs Safety

Properties of a good cut generator:

◮ Decreases solution time with Branch-and-Cut ← Strength ◮ Does not cut off feasible solutions or lead to numerical failures of LP

solver

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 13 / 36

Strength vs Safety

Properties of a good cut generator:

◮ Decreases solution time with Branch-and-Cut ← Strength ◮ Does not cut off feasible solutions or lead to numerical failures of LP

solver ← Safety

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 13 / 36

Strength vs Safety

Properties of a good cut generator:

◮ Decreases solution time with Branch-and-Cut ← Strength ◮ Does not cut off feasible solutions or lead to numerical failures of LP

solver ← Safety

Safety must be tested before strength

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 13 / 36

Strength vs Safety

Properties of a good cut generator:

◮ Decreases solution time with Branch-and-Cut ← Strength ◮ Does not cut off feasible solutions or lead to numerical failures of LP

solver ← Safety

Safety must be tested before strength Goal: develop a framework for testing safety of cut generators in a Branch-and-Cut setting, so that strength of generators with similar safety can be compared Related work: [Margot, 2009]

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 13 / 36

Strength vs Safety

Properties of a good cut generator:

◮ Decreases solution time with Branch-and-Cut ← Strength ◮ Does not cut off feasible solutions or lead to numerical failures of LP

solver ← Safety

Safety must be tested before strength Goal: develop a framework for testing safety of cut generators in a Branch-and-Cut setting, so that strength of generators with similar safety can be compared Related work: [Margot, 2009] Define “safe” and “unsafe”

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 13 / 36

1

Motivation

2

Cut generation parameters

3

Failures and Feasibility

4

Dive-and-Cut

5

Computational framework

6

Optimizing GMI cut generators

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 14 / 36

Failures

Assume that we employ a cut generator in the typical cut generation loop A failure of a given cut generator on (MILP) is the occurrence of one

• f the following events:

1

A cutting plane that cuts off a known integral feasible solution to (MILP) is generated

2

(LP) becomes infeasible after the addition of cutting planes, but an integral feasible solution for the original problem is known

3

A time limit for cut generation and (LP) resolve is hit

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 15 / 36

Feasibility

In floating point arithmetic, we have to accept slight violations of the constraints Notation:

◮ Rows of A: ai, i = 1, . . . , m ◮ For k > 0, [k] is the set {1, . . . , k}

A point x∗ is (ǫabs, ǫrel, ǫint)-feasible for (MILP) if:

1

∀i ∈ NI, x∗

i − ⌊x∗ i ⌉ ≤ ǫint

2

mini∈[m]{aix∗ − bi} ≥ −ǫabs

3

mini∈[m]{(aix∗ − bi)/ai2} ≥ −ǫrel

4

∃x′ : Ax′ ≥ b, x′ ≥ 0, x∗ − x′ ≤ ǫrel

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 16 / 36

Why relative tolerance?

1 Invariant to rescaling (condition 3) 2 Sanity check (condition 4): ◮ ˜

x does not satisfy condition 4 and would mark the cut as invalid

◮ x∗ satisfies condition 4 and the cut is valid (up to tolerances)

ǫrel ǫrel x′ a2x ≥ b2 a1x ≥ b1 ˜ x ǫrel x∗ (λ1a1 + λ2a2)x ≥ (λ1b1 + λ2b2) Valid cut:

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 17 / 36

Generation of feasible solutions

We picked ǫabs = 10−9, ǫrel = 10−9, ǫint = 0 We generated solution for all instances in the union of miplib3, miplib2003, and the Benchmark Set of miplib2010: 170 instances For each problem instance, we form a set S of (ǫabs, ǫrel, ǫint)-feasible solutions We used Cplex 12.2 to guide the Branch-and-Cut search and QSopt ex [Applegate et al., 2007] as rational LP solver to check feasibility and distances For 160 instances out of 170, we found at least a feasible solution: Data Set

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 18 / 36

Dataset

Histogram of num_solutions

num_solutions Frequency 50 100 150 200 5 10 15 20 25 30 35

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 19 / 36

1

Motivation

2

Cut generation parameters

3

Failures and Feasibility

4

Dive-and-Cut

5

Computational framework

6

Optimizing GMI cut generators

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 20 / 36

The Dive-and-Cut algorithm

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 21 / 36

The Dive-and-Cut algorithm

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 21 / 36

The Dive-and-Cut algorithm

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 21 / 36

The Dive-and-Cut algorithm

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 21 / 36

The Dive-and-Cut algorithm

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 21 / 36

The Dive-and-Cut algorithm

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 21 / 36

The Dive-and-Cut algorithm

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 21 / 36

The Dive-and-Cut algorithm

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 21 / 36

The Dive-and-Cut algorithm

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 21 / 36

The Dive-and-Cut algorithm

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 21 / 36

1

Motivation

2

Cut generation parameters

3

Failures and Feasibility

4

Dive-and-Cut

5

Computational framework

6

Optimizing GMI cut generators

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 22 / 36

Comparing cut generators

We identified a set of 51 “difficult” instances (out of the 160 in Data Set): Failure Set We can compare safety of two cut generators by applying statistical tests to verify whether one yields more failures than the other We assume that the result of an experiment (e.g. failure/not failure) is a random variable but we do not assume an a priori distribution The comparison are based on ranking and are performed at a given significance level Examples: Friedman, Quade, Cochran Q test Detection power and number of failures

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 23 / 36

Failures and number of dives

50 100 150 200 250 300 350 400 450 50 100 150 200 250 300 # of failures # of dives Type1 Type2 Type3

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 24 / 36

1

Motivation

2

Cut generation parameters

3

Failures and Feasibility

4

Dive-and-Cut

5

Computational framework

6

Optimizing GMI cut generators

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 25 / 36

Our goal

Now that we have a framework for testing cut generators, we can try to optimize over the set of all GMI cut generators Each point in the parameter space corresponds to a cut generator Objective? Constraints?

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 26 / 36

Our goal

Now that we have a framework for testing cut generators, we can try to optimize over the set of all GMI cut generators Each point in the parameter space corresponds to a cut generator Objective? Constraints?

clean GMI cuts GMI cuts GMI cuts dangerous

and rejection Cut modification

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 26 / 36

The problem

Constraints: maximum failure rate (i.e. (# failures)/(# dives)), at least as safe as a reference generator (= Cplex) Vector-valued objective function: average cut rejection rate per instance, Friedman test to perform comparisons at the 95% significance level Several cut generators are indistinguishable: multiple local minima

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 27 / 36

Don’t try this at home!

Black-box optimization over a discretized parameter space Costly evaluation of objective function and constraints Not well suited for traditional response surface methods How many function evaluations do we need?

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 28 / 36

Don’t try this at home!

Black-box optimization over a discretized parameter space Costly evaluation of objective function and constraints Not well suited for traditional response surface methods How many function evaluations do we need? Many thanks to Jeff Linderoth and the Condor team at UW-Madison!

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 28 / 36

Shrinking the parameter space

Even with a huge computational effort, optimizing over 12-dimensional parameter space is too difficult We use standard regression techniques to find the most important parameters 6 parameters suffice to build a “good” model of the cut rejection rate and the failure rate:

◮ AWAY ◮ EPS COEFF ◮ EPS RELAX ABS ◮ EPS RELAX REL ◮ MAX DYN ◮ MIN VIOL

We perform:

◮ RhsRelaxation ◮ CoefficientRemoval ◮ Fractionality Check ◮ Dynamism Check ◮ Violation Check

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 29 / 36

The optimization algorithm

Good news: rejection rate is “convex”!

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 30 / 36

The optimization algorithm

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 30 / 36

The optimization algorithm

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 30 / 36

The optimization algorithm

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 30 / 36

The optimization algorithm

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 30 / 36

The optimization algorithm

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 30 / 36

The optimization algorithm

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 30 / 36

The optimization algorithm

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 30 / 36

The optimization algorithm

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 30 / 36

The optimization algorithm

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 30 / 36

Results

6 iterations Final parameter ranges: AWAY 102 EPS RELAX REL [0, 10−11] EPS COEFF [10−13, 10−10] MAX DYN [107, 109] EPS RELAX ABS [0, 10−11] MIN VIOL [0, 10−11] One optimal generator BestGen: AWAY 102 EPS RELAX REL 0 EPS COEFF 10−11 MAX DYN 107 EPS RELAX ABS 0 MIN VIOL 10−16

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 31 / 36

Does this work?

We compare BestGen over 300 dives on Failure Set against:

◮ Cplex ◮ CglGomory ◮ CglLandP (used to generate GMI cuts) ◮ CglGomoryClone (“clone” of CglGomory using our own code) ◮ CglLandPClone (“clone” of CglLandP using our own code)

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 32 / 36

Either it works or we got lucky

We use a Cochran Q test to compare the number of dives that end with a failure at the 95% significance level, and obtain the following ranking (failure rate in round brackets):

1

BestGen (0.27%) = CglGomoryClone (0.22%) = Cplex (0.19%)

2

CglLandP (1.41%)

3

CglLandPClone (1.73%)

4

CglGomory (3.35%)

We use a Friedman test to compare the gap closed per dive at the 95% significance level by the 3 “equally safe” generators:

1

BestGen

2

CglGomoryClone

3

Cplex

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 33 / 36

Conclusions

Framework for testing safety cut generators

◮ Dataset of instances and feasible solutions ◮ Algorithms and their implementation

Attempt to find an “optimal” parameterization of a GMI cut generator – some parameters are not needed Future research:

◮ Fine tuning of the parameters ◮ Testing strength ◮ . . .

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 34 / 36

. . . and that’s all

Thank you!

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 35 / 36

Bibliography

Applegate, D. L., Cook, W., Dash, S., and Espinoza, D. G. (2007). Exact solutions to linear programming problems. Operations Research Letters, 35(6):693–699. Balas, E., Ceria, S., Cornu´ ejols, G., and Natraj, N. (1996). Gomory cuts revisited. Operations Research Letters, 19(1):1–9. Bixby, R. and Rothberg, E. (2007). Progress in computational mixed integer programming – a look back from the other side of the tipping point. Annals of Operations Research, 149(1):37–41. Cook, W., Koch, T., Steffy, D. E., and Wolter, K. (2011). An exact rational mixed-integer programming solver. In G¨ unl¨ uk, O. and Woeginger, G. J., editors, Proceedings of IPCO 2011, volume 6655 of Lecture Notes in Computer Science, pages 104–116, Berlin Heidelberg. Springer-Verlag. Espinoza, D. G. (2006). On Linear Programming, Integer Programming and Cutting Planes. PhD thesis, School of Industrial and Systems Engineering, Georgia Institute of Technology. Gomory, R. E. (1960). An algorithm for the mixed-integer problem. Technical Report RM-2597, RAND Corporation. Margot, F. (2009). Testing cut generators for mixed-integer linear programming. Mathematical Programming Computation, 1(1):69–95.

• G. Nannicini (SUTD/MIT)

Study of Gomory cut generators January 12, 2012 36 / 36