Recent Developments in Disjunctive Programming Egon Balas Carnegie - - PowerPoint PPT Presentation

Recent Developments in Disjunctive Programming

Egon Balas Carnegie Mellon University

Recent Developments in Disjunctive Programming 01.09.17 1 / 56

Recent Developments in Disjunctive Programming

• 1. Background and basic results
• 2. Convexification and extended formulations
• 3. L&P cuts from split disjunctions
• 4. General (non-split, multiple-term) disjunctions
• 5. The convex hull in Rn

(Parts 4-5 based on joint work with, respectively, Tamas Kis and Aleksandr Kazachkov)

Recent Developments in Disjunctive Programming 01.09.17 2 / 56

• 1. Background and basic results
• Convexity – the dividing line between “tame” and “wild” problems
• Nonconvex sets can often be modeled via Integer Programming — which is

NP-complete

• Disjunctive programming: optimization over disjunctive sets, defined by

inequalities joined by connectives ∧, ∨, ⇒, ¬ (nonconvex because of ∨)

• The largest known class of nonconvex sets convexifiable in polynomial time
• Many equivalent forms, two extreme:
• CNF: intersection of elementary disjunctive sets

F = ∧

j∈TSj,

Sj = {x : ∨

i∈Qj(aix ≥ ai0)},

j ∈ T

• DNF: union of polyhedra

F = ∪

i∈QPi,

Pi = {x : Aix ≥ bi}, i ∈ Q Any disjunctive set can be brought to either form by a sequence of simple steps.

Recent Developments in Disjunctive Programming 01.09.17 3 / 56

Disjunctive sets in DNF: Unions of polyhedra are a large class of nonconvex sets with a compact convex representation. The union of q polyhedra in Rn (= the disjunction between q systems of linear inequalities in n variables) has a convex hull representation as a polyhedron in R(qn) (= a linear system in O(qn) variables). Disjunctive sets in CNF: If facial, any such set can be convexified sequentially, i.e. by imposing the elementary disjunctions one at a time, each time generating the convex hull

• f the current set:

0-1 MIP’s are facial, (general) MIP’s are not. This is the basic property distinguishing 0-1 programs from general integer programs.

Recent Developments in Disjunctive Programming 01.09.17 4 / 56

The convex hull of a disjunctive set

(E.B., Disjunctive Programming, CMU MSRR 348, 1974, Discrete Applied Math, 1998) Basic observation An inequality αx ≥ α0 is valid for the disjunction ∨

i∈Q(Aix ≥ bi)

if and only if it is valid for each system Aix ≥ bi, i ∈ Q. With this in mind, we have Theorem 1.1 Farkas’ Lemma for disjunctive sets Let F = ∨

i∈Q(Aix ≥ bi), and Q∗ := {i ∈ Q : {x : Aix ≥ bi} = ∅}. The inequality

αx ≥ β is satisfied by all x ∈ F if and only if there exist vectors ui ∈ Rm, ui ≥ 0, such that α = uiAi, β ≤ uibi, i ∈ Q∗.

Recent Developments in Disjunctive Programming 01.09.17 5 / 56

Thus, an elementary disjunction of the form ∨

i∈Q(aix ≥ bi), where ai ∈ Rn,

bi > 0, i ∈ Q, and each term is feasible, implies the valid inequality

n

• j=1

max

i∈Q

ai bi

• xj ≥ 1.

which is a weakening of each aix ≥ bi.

Recent Developments in Disjunctive Programming 01.09.17 6 / 56

Theorem 1.2 Let F = ∪

i∈QPi, Pi = {x ∈ Rn : Aix ≥ bi}, i ∈ Q, and let

Q∗ = {i ∈ Q : Pi = ∅}. Then conv F = {x ∈ Rn : x −

i∈Q∗y i

= Aiy i − biy i ≥ y i ≥ i ∈ Q∗

• i∈Q∗y i

= 1 for some vectors (y i, y i

0), i ∈ Q∗}.

(1.1) Denoting C i := {x : Aix ≥ 0}, i ∈ Q, Q∗ can be replaced with Q if

• ∪

i∈Q\Q∗C i

• ⊆
• ∪

i∈Q∗C i

• Recent Developments in Disjunctive Programming 01.09.17

7 / 56

To obtain the convex hull in Rn, project the set S(Q) defined by (1.1) onto the x-space. Projection cone: W =    (α, β, {ui}i∈Q) : −α + uiAi = β − uibi ≥ i ∈ Q ui ≥    Theorem 1.3 Proj xS(Q) = {x ∈ Rn : αx ≥ β for all (α, β, {ui}i∈Q) ∈ extrW } The inequalities αx ≥ β can be generated as the (α, β)-components of extreme rays of W , i.e. by solving a system of O(qn) variables.

Recent Developments in Disjunctive Programming 01.09.17 8 / 56

The above representation of conv F was derived by using the H-polyhedral representation of F. What about the V -polyhedral representation? F = ∪

i∈QPi,

Pi = conv V i + cone W i, i ∈ Q Theorem 1.4 (M. Perregaard and E.B., IPCO 2001) The inequality γx ≥ δ is valid for F if and only if γp ≥ δ for all p ∈ V i γr ≥ 0 for all r ∈ W i

• i ∈ Q

(1.2)

• Proof. γx ≥ δ is valid for F if and only if it is valid for all Pi, i ∈ Q.

Thus conv F is defined by the inequalities γx ≥ δ corresponding to basic solutions (γ, δ) of the system (1.2). The system (1.2) defines conv F in Rn, but it has ∪

i∈Q(|V i| + |W i) inequalities.

Recent Developments in Disjunctive Programming 01.09.17 9 / 56

Two consequences of the compact convex hull representation

(1) The fact that a nonconvex set in Rn can be described by a convex polyhedron in a higher dimensional space gave rise to extended formulations. Two types of benefits: (a) tighter LP relaxations (b) integrality of higher dimensional representation proves integrality of original polyhedron (2) The description of the convex hull of a disjunctive set by the inequalities corresponding to basic solutions of a CGLP has led to the study of lift-and-project cuts for MIP’s. First to be studied were cuts from split disjunctions.

Recent Developments in Disjunctive Programming 01.09.17 10 / 56

• 2. Convexification and extended formulations

(E.B., SIAM J on Algebraic Discrete Methods, 1985)

• A general technique for tightening formulations of MIP’s:

replace the “ big M” representation of some disjunctive subset by its convex hull representation

• CNF and DNF are both intersections of unions of polyhedra

F = ∩

j∈TSj,

Sj = ∪

i∈QPi,

j ∈ T (2.1)

• Call (2.1) a regular form (RF)

Theorem 2.1 Any disjunctive set F in RF(2.1) can be brought to DNF by |T| − 1 applications of the following basic step, which preserves regularity: For some k, ℓ ∈ T, bring Sk ∩ Sℓ to DNF, i.e. replace it with Skℓ = ∪i∈Qk

j∈Qℓ

(Pi ∩ Pj)

Recent Developments in Disjunctive Programming 01.09.17 11 / 56

Given a disjunctive set in regular form (2.1), define the hull-relaxation of F = ∩

j∈TSj as h-rel F = ∩ j∈Tconv Sj

Theorem 2.2 For i = 0, 1, . . ., t, let Fi = ∩

j∈TiSj

be a sequence of regular forms, such that (i) F0 is in CNF (ii) Ft is in DNF (iii) for i = 1, . . . , t, Fi is obtained from Fi−1 by a basic step. Then h-rel F0 ⊇ h-rel F1 ⊇ · · · ⊇ h-rel Ft = conv Ft

Recent Developments in Disjunctive Programming 01.09.17 12 / 56

Clearly, conv Skℓ ⊆ conv Sk ∩ conv Sℓ. When to replace a disjunctive set in RF by its convex hull? –When it makes conv Skℓ tighter than (conv Sk) ∩ (conv Sℓ). Theorem 2.3 Let Sj = ∪

i∈QjPi, j = 1, 2. Then

conv (S1 ∩ S2) = (conv S1) ∩ (conv S2) if and only if every extreme point (direction) of (conv S1) ∩ (conv S2) is an extreme point (direction) of Pi ∩ Pk for some (i, k) ∈ Q1 × Q2.

Recent Developments in Disjunctive Programming 01.09.17 13 / 56

• Example. Let Pi, i = 1, . . . , 4 be as in the figure, and

F = S1 ∩ S2, S1 = P1 ∪ P2, S2 = P3 ∪ P4

(0,0) (1,0) (0,1) (1,1)

P2 P1

(½,1) (1,½)

P3

(0,½) (½,0)

P4

(½,1) (1,½)

(conv S1) I (conv S2)

(0,½) (½,0) (1,½)

conv (S1 I S2)

(0,½) (1,0) (0,1)

Then conv (S1 ∩ S2)(conv S1) ∩ (conv S2) since vertices ( 1

2, 0) and ( 1 2, 1) of

conv S1 ∩ conv S2 are not vertices of either P1 ∩ P3, P1 ∩ P4, P2 ∩ P3 or P2 ∩ P4.

Recent Developments in Disjunctive Programming 01.09.17 14 / 56

If a conjunct Sj = ∪

i∈QjPi of a RF Fk = ∩ j∈TSj, is replaced with conv Sj, we get

x −

i∈Qj

y i = Aiy i − biy i ≥ i ∈ Qj y i ≥

• i∈Qj

y i = 1                j ∈ T (2.2) In any solution nonbasic for the j-th subsystem, y i

0 ∈ {0, 1}

Imposing y i

0 ∈ {0, 1} for i ∈ Qj, j ∈ T

? No: if the CNF is F0 = ∧

r∈T0Sr,

Sr = ∨

s∈Qr(asx ≥ bs),

then replace the last equation of (2.2) by

• i∈Qj|s∈Mi

y r

0 − δr

= 0, s ∈ Qr r ∈ T0 (Mi = row index set of Ai)

• s∈Qr

δr

s

= 1 r ∈ T0 δr

s ∈ {0, 1}

s ∈ Qr, r ∈ T0 i.e. we need the same number of 0-1 variables as in CNF.

Recent Developments in Disjunctive Programming 01.09.17 15 / 56

Application: job shop scheduling with sequence-dependent setup times min tj tj − ti ≥ dij (i, j) ∈ A tj − ti ≥ dij ∨ ti − tj ≥ dji (i, j) ∈ Ek, k ∈ M ti = start time of job i dij = duration of i + setup time for j A = set of precedence arcs M = set of machines Ek = set of disjunctive pairs of arcs for machine k

Recent Developments in Disjunctive Programming 01.09.17 16 / 56

Machine-clique

• Selection: one of each pair
• Feasible: non-conflicting and acyclic

(=tournament)

• Transitive closure of a directed Hamilton

path Finding an

• ptimal

jobshop schedule amounts to finding in each machine-clique an optimal directed Hamilton path subject to time window constraints Tight formulation: path variables

Recent Developments in Disjunctive Programming 01.09.17 17 / 56

Union of polytopes in different spaces

(E.B., A.Bockmayr, N. Pisaruk, L. Wolsey) Convex hull description without additional variables. Theorem 2.2. Let Z = P ∪ Q Z := {(x, y) : x ∈ P or y ∈ Q} where P and Q are upper mononone polytopes, i.e. P := {x ∈ {0, 1}m : Ax ≥ 1}, Q := {y ∈ {0, 1}n : By ≥ 1}, with A ≥ 0, B ≥ 0, and let M = {1, . . . , m}, N = {1, . . . , n}. Then conv Z = {(x, y) ∈ {0, 1}m+n :

j∈S aij 1−

h∈M\S

aih xj + ℓ∈T bkℓ 1−

h∈N\T

bkh yℓ ≥ 1

for all S ⊆ M s.t.

• h∈M\S

aih < 1 and T ⊆ N s.t.

• h∈N\T

aih < 1}.

Recent Developments in Disjunctive Programming 01.09.17 18 / 56

Application: logical inference

• i∈M

xi ≥ k ⇒

• j∈N

yj ≥ ℓ M = {1, . . ., m}, N = {1, . . . , n}

• i∈M

xi ≤ k − 1 ∨

• j∈N

yj ≥ ℓ

• i∈M

¯ xi ≥ p ∨

• j∈N

yj ≥ ℓ, ¯ xi = 1 − xi, p = m − k + 1 (2.3) Denote ¯ x(S) =

i∈S

¯ xi, y(T) =

j∈T

yj Theorem 2.3 The convex hull of points (¯ x, y) ∈ {0, 1}m+n satisfying (2.3) is defined by 0 ≤ ¯ xi ≤ 1, i ∈ M, 0 ≤ yj ≤ 1, j ∈ N, and (|T| + ℓ − n)¯ x(S) + (|S| + p − m)y(T) ≥ (|S| + p − m)(|T| + ℓ − n) (2.4) for all S ⊆ M, m − p + 1 ≤ |S| ≤ m and all T ⊆ N, n − ℓ + 1 ≤ |T| ≤ n.

• For 1 ≤ p ≤ m − 1, 1 ≤ ℓ ≤ n − 1, every inequality of (2.4) is facet defining
• Given some (x∗, y ∗) ∈ {0, 1}m+n, there is an algorithm for identifying the

inequality of (2.4) most violated by (x∗, y ∗) in O(m + n) time

Recent Developments in Disjunctive Programming 01.09.17 19 / 56

• 3. Disjunctive (lift-and-project) cuts from split disjunctions

Cut from a split disjunction xk ≤ ⌊¯ xk⌋ ∨ xk ≥ ⌈¯ xk⌉, where xk = ak0 −

• j∈J

akjxj, (ak0 = ¯ xk) is αx ≥ 1, with αj = max akj ak0 , −akj 1 − ak0

• .

But αx ≥ 1 can also be viewed as an intersection cut from the convex set S := {x : ⌊¯ xk⌋ ≤ xk ≤ ⌈¯ xk⌉}

xk ≤ ⌊¯ xk⌋ xk ≤ ⌈¯ xk⌉ ¯ x Recent Developments in Disjunctive Programming 01.09.17 20 / 56

More generally: If

• ¯

x is a basic solution to P := {x : Ax ≥ b, x ≥ 0}, and

• S is some convex set s.t.

– ¯ x ∈ int S – PI ∩ int S = ∅, where PI := P ∩ Zn (S is a PI-free convex set) then

• Intersect the n extreme rays ¯

x + rjλj, λj ≥ 0, j ∈ J, of the LP cone C(¯ x) with bd S

• Result: the hyperplane through the n intersection points r ∩ bd S defines a

valid cut—the intersection cut

Recent Developments in Disjunctive Programming 01.09.17 21 / 56

Two definitions of intersection cuts:

• original (standard) def. (E.B., Oper Res 1971)

S is a PI-free convex set (PI ∩ int S = ∅)

• more recent (restricted) def.

S is a lattice-free convex set (Zn ∩ int S = ∅) (Difference: a PI-free set is not necessarily lattice-free.) Theorem (M. Conforti, G. Cornu´ ejols and G. Zambelli, OR Letters 2010) Any intersection cut from an LP cone C(¯ x) and a lattice-free polyhedron S is valid for the corner polyhedron conv (C(¯ x) ∩ Zn). All the facets of the corner polyhedron are defined by intersection cuts from lattice-free polyhedra.

Recent Developments in Disjunctive Programming 01.09.17 22 / 56

Lift-and-project cuts for 0-1 MIP’s

(E.B., S. Ceria and G. Cornu´ ejols, Math Prog 1993)

• min{cx : Ax ≥ b, x ≥ 0, xj ∈ {0, 1}, j ∈ N1 ⊂ N}

(MIP) min{cx : ˜ Ax ≥ ˜ b} Optimal LP solution: ¯ x (LP)

• L&P cut αx ≥ β from
• ˜

Ax ≥ ˜ b −xk ≥ 0

• ∨

˜ Ax ≥ ˜ b xk ≥ 1

• min

α¯ x −β α −u ˜ A +u0ek = α −v ˜ A −v0ek = −β +u˜ b = −β +v ˜ b +v0 = ue + ve + u0 + v0 = 1 u, v, u0, v0 ≥ (CGLP)k The last equation is a normalization constraint.

Recent Developments in Disjunctive Programming 01.09.17 23 / 56

Theorems (3.1-3.6)

• 1. Any basic feasible solution (α, β, u, u0, v, v0) to (CGLP)k with u0, v0 > 0

yields a valid cut αx ≥ β; and together these cuts define conv {x ∈ Rn : ˜ Ax ≥ ˜ b, xk ∈ {0, 1}}

• 2. Solutions with u0 = 0 or v0 = 0 yield inequalities of P
• 3. The cut αx ≥ β is facet defining for

conv {x : ˜ Ax ≥ ˜ b, xk ∈ {0, 1}} iff (α, β) is an extreme ray of W0

• 4. If the normalization constraint of (CGLP)k is replaced with u0 + v0 = 1,

then the cut αx ≥ β defined by the optimal solution to (CGLP)k is the simple disjunctive cut (MIG cut) from xk ≤ ⌊¯ xk⌋ ∨ xk ≥ ⌈¯ xk⌉, where xk = ¯ ak0 −

j∈J

¯ akjxj

• 5. If some of the variables xj, j ∈ J, are integer-constrained, the cut αx ≥ β

can be strengthened by modularization of its coefficients

• 6. If the cuts generated under 1 are added to Ax ≥ b and the procedure is

iterated for k = 1, . . . , n, the cuts generated define conv {x ∈ Rn : ˜ Ax ≥ ˜ b, x ∈ {0, 1}n}

Recent Developments in Disjunctive Programming 01.09.17 24 / 56

The correspondence between lift-and-project cuts and mixed integer Gomory cuts

(E.B. and M. Perregaard, Mathematical Programming 2003)

Theorem A. Let αx ≥ β be the lift-and-project cut associated with the basic feasible solution (α, β, u, v, u0, v0) to (CGLP)k , where u0, v0 > 0 and M1 := {i : ui is basic }, M2 := {i : vi is basic }. Then αx ≥ β is equivalent to πsJ ≥ π0, the cut from the disjunction xk ≤ 0 ∨ xk ≥ 1 applied to xk = ak0 −

j∈J akjsj

where J = M1 ∪ M2. Theorem B. Let πsJ ≥ π0 be the cut from the disjunction xk ≤ 0 ∨ xk ≥ 1 applied to xk = ak0 −

j∈J akjsj, and let (M1, M2) be any partition of J such that

j ∈ M1 if akj < 0, j ∈ M2 if akj > 0. Then πsJ ≥ π0 is equivalent to αx ≥ β, the lift-and-project cut associated with any basic solution (α, β, u, v, u0, v0) to (CGLP)k such that u0, v0 > 0 and ui is basic if i ∈ M1, vi is basic in M2.

Recent Developments in Disjunctive Programming 01.09.17 25 / 56

The above correspondence implies that every lift-and-project cut from a basic feasible solution to (CGLP)k corresponds to a Gomory mixed integer cut from xk ≤ 0 ∨ xk ≥ 1, with xk expressed in terms of some feasible or infeasible basis

• f the LP.

Upshot I:

• There is a many-to-one correspondence between basic feasible solutions to

(CGLP)k such that ui and vi are basic for i ∈ M1 and i ∈ M2, respectively, and the basic (feasible or infeasible) solution to (LP) corresponding to the nonbasic set indexed by J = M1 ∪ M2

• Hence: (CGLP)k can be solved implicitly, by mimicking its pivots on the

(LP) tableau – the reduced costs of (CGLP)k can be computed from the LP tableau – a pivot in the LP tableau corresponds to several pivots in the CGLP tableau

Recent Developments in Disjunctive Programming 01.09.17 26 / 56

Upshot II: The procedure for solving the CGLP by pivoting in the LP simplex tableau can be viewed as an algorithm for finding the best mixed integer Gomory cut from the disjunction xk ≤ 0 ∨ xk ≥ 1; more precisely, for finding the best J to yield such a

• cut. Namely:
• Start with mixed the MIG cut from the xk row of the optimal LP simplex

tableau.

• Evaluate it as a lift-and-project cut (by computing the reduced costs).
• If it is not optimal (as a lift-and-project cut) then identify a pivot (in the LP

tableau) which can improve it.

• After the pivot (which creates an LP tableau that is in general neither
• ptimal nor feasible), the MIG cut from the xk row is guaranteed to be

stronger. From this perspective, lift-and-project theory provides the means by which, given a variable xk, we can find among all possible bases containing xk, the one in which the MIG cut from xk is strongest.

Recent Developments in Disjunctive Programming 01.09.17 27 / 56

Implementations

Commercial codes: XPRESS – Perregaard MOPS – Wesselmann CPLEX – Tramontani Publicly available implementation (P. Bonami): COIN-OR’s Cut Generation Library (Cgl): http://projects.coin-or.org/Cgl/Wiki/CglL&P

Recent Developments in Disjunctive Programming 01.09.17 28 / 56

Computational Testing

E.B. and P. Bonami, Math Prog Computation 2009)

• 3 different normalizations
• 2 ways of selecting the pivot column
• strengthening through disjunctive modularization

Test-bed: all 65 instances of MIPLIB3 library

• MIG cuts vs. 3 variants of L&P cuts
• 2 comparisons:

(a) integrality gap after 10 rounds of 50 cuts at root node (b) CPU time and tree size (# nodes) required for solving by branch and bound after 10 rounds at root node

Recent Developments in Disjunctive Programming 01.09.17 29 / 56

(a) For the 33 harder instances

L&P Integrality gap closed at the root node (%) MIG Variant 1 Variant 2 Variant 3 21 30 33 30

(b) For the 24 hardest instances

L&P MIG Variant 1 Variant 2 Variant 3

Complete branch-and- bound run

CPU(s) Nodes CPU(s) Nodes CPU(s) Nodes CPU(s) Nodes

Geo Mean

73 23,449 42 10,444 61 16,590 42 10,021

Recent Developments in Disjunctive Programming 01.09.17 30 / 56

Generalized intersection cuts

E.B. and F. Margot, Math Programming 2013

• Polyhedral relaxation C of P
• PI-free convex set S (¯

x ∈ int S, int S ∩ PI = ∅)

• Intersection points pj = rj ∩ bd S, j ∈ P, of rays (extended edges) of C with

bd S Theorem 3.7. Let pj, j ∈ P, be a proper collection of intersection points. Let ¯ α be any solution to either of the systems αpj ≥ β, j ∈ P for β ∈ {1, −1, 0} such that α¯ x < β. Then ¯ αx ≥ β is a valid generalized intersection cut (GIC) for PI.

• Motivation: to get rid of recursive cut generation

that leads to numerical difficulties

• New paradigm: (a) create a proper collection Q,

(b) use it to generate deep cuts without recursion

Recent Developments in Disjunctive Programming 01.09.17 31 / 56

Generalized intersection cuts

p1=q1 p2 q2 p3 q3

α1x = β1

• ✠

¯ x

q1, q2, q3 – intersection points of extreme rays of C1 with bd S p1, p2, p3 – intersection points of extreme rays of C with bd S

Recent Developments in Disjunctive Programming 01.09.17 32 / 56

• 4. General disjunctive cuts

(E.B. and T. Kis, Math Programming 2016) General disjunction:

• t∈T
• ˜

Ax ≥ ˜ b Dtx ≥ dt

• (4.1)

Convex hull representation still compact (grows linearly with |Q|)

min α¯ x − β α − ut ˜ A − v tDt = −β + ut ˜ b + v tdt = t ∈ T

• t∈T

utem +

• t∈T

v tep = 1 ut, v t ≥ 0, t ∈ T (CGLP)T

If the set Dtx ≥ dt

0 is replaced with a single inequality then (4.1) is called simple.

Recent Developments in Disjunctive Programming 01.09.17 33 / 56

L&P cuts and GIC’s

Let

S := {x ∈ RN : dtx ≤ dt0, t ∈ T}

be a maximal PI-free polyhedron. Let

F :=   x ∈ RN :

• t∈T
• ˜

Ax ≥ ˜ b dtx ≥ dt0   

and

conv F := {x ∈ RN : αx ≥ β for all (α, β) satisfying (∗)}

where

α − ut ˜ A − ut

0dt

= −β + ut ˜ b + ut

0dt0

= t ∈ T

• t∈T(ute

+ ut

0)

= 1 ut, ut ≥ (∗)

Theorem 4.1. The family of GIC’s from S is equivalent to the family of L&P cuts αx ≥ β from basic feasible solutions to (∗) such that ut

0 > 0 for all t ∈ T.

Recent Developments in Disjunctive Programming 01.09.17 34 / 56

Let F :=

• x ∈ RN : ∨t∈T
• ˜

Ax ≥ ˜ b Dtx ≥ dt0

• and

conv F := {x ∈ RN : αx ≥ β for all α, β satisfying (∗∗)}, where α − ut ˜ A − v tDt = −β + ut ˜ b + v tdt0 = t ∈ T

• t∈T(ute

+ v te) = 1 ut, v t ≥ 0, t ∈ T (∗∗) Theorem 4.2. The L&P cut ¯ αx ≥ ¯ β from a basic feasible solution (¯ α, ¯ β, {¯ ut, ¯ v t}t∈T) to (**) such that ¯ v te > 0 for all t ∈ T is equivalent to a GIC from the maximal PI-free polyhedron S := {x ∈ RN : (¯ v tDt)x ≤ ¯ v tdt0, t ∈ T}.

Recent Developments in Disjunctive Programming 01.09.17 35 / 56

L&P cut from GIC from ∨

t∈T

• ˜

Ax ≥ ˜ b dtx ≥ dt0

• ⇐

⇒ S := {x ∈ Rn : dtx ≤ dt0, t ∈ T} (PI- free polyhedron) ∨

t∈T

• ˜

Ax ≥ ˜ b Dtx ≥ dt

• ⇐

⇒ S := {x ∈ Rn : v tDtx ≤ v tdt0, t ∈ T} (family of PI-free polyhedra)

Recent Developments in Disjunctive Programming 01.09.17 36 / 56

L&P cuts and SIC’s

Theorem 4.3. Let ¯ x be an optimal LP solution with nonbasic set J, and let S := {x ∈ Rn : dtx ≤ dt0, t ∈ T} be a maximal PI-free polyhedron with ¯ x ∈ int S. The SIC πx ≥ 1 from ¯ x and S is equivalent to the L&P cut from a basic feasible solution to

α − ut ˜ A − ut

0dt

= −β + ut ˜ b + ut

0dt0

= t ∈ T

• t∈T(ute

+ ut

0)

= 1 ut, ut ≥ (∗)

in which, for each t ∈ T, all but one of the variables ut

j with j ∈ J are basic, and

all the variables ut

j with j ∈ J are nonbasic, except for the ut 0, which are all basic

and positive. Theorem 4. Let w := (α, β, {ut, ut

0}t∈T) be a basic feasible solution to (*) such

that ut

0 > 0, t ∈ T. If there exists a n × n nonsingular submatrix ˜

AJ of ˜ A such that ut

j = 0 for all t ∈ T and j ∈ J, then the L&P cut αx ≥ β is equivalent to

the SIC πx ≥ 1 from S.

Recent Developments in Disjunctive Programming 01.09.17 37 / 56

Let F = ∨

t∈T

• ˜

Ax ≥ ˜ b Dtx ≥ dt

• , and consider the (CGLP)T with the constraint set

α − ut ˜ A − v tDt = t ∈ T −β + ut ˜ b + v tdt =

• t∈T

(ute + v te) = 1 ut, v t ≥ (∗∗) Theorem 4.5. Let ¯ w = (¯ α, ¯ β, {¯ ut, ¯ v t}t∈T) be a basic feasible solution to (**) such that ¯ v te > 0 for all t ∈ T. If there exists a n × n nonsingular submatrix ˜ AJ

• f ˜

A such that ¯ ut

j = 0 for all t ∈ T and j ∈ J, then the L&P cut ¯

αx ≥ ¯ β is equivalent to the SIC πxJ ≥ 1 from the LP solution with nonbasic set indexed by J, and the PI-free polyhedron S(¯ v) := {x ∈ Rn : (¯ v tDt)x ≤ ¯ v tdt

0, t ∈ T}.

Recent Developments in Disjunctive Programming 01.09.17 38 / 56

Theorem 4.6. Let ¯ w := (¯ α, ¯ β, {¯ ut, ¯ v t}t∈T) be a basic feasible solution to α − ut ˜ A − v tDt = −β + ut ˜ b + v tdt0 = t ∈ T

• t∈T(ute

+ v te) = 1 ut, v t ≥ (∗∗) such that ¯ v te > 0 for all t ∈ T. If there is no n × n nonsingular ˜ AJ such that ut

j = 0 for all t ∈ T and j ∈ J, and there is no basic feasible solution ˜

w to (∗∗) with (˜ α, ˜ β) = θ(¯ α, ¯ β) for some θ > 0 that satisfies this condition, then there exists no SIC from any member of the family of polyhedra S(v) = {x ∈ Rn : (v tDt)x ≤ v tdt0, t ∈ T}, where v ≥ 0, v = 0, equivalent to ¯ αx ≥ ¯ β. Furthermore, if ¯ α¯ x − ¯ β uniquely minimizes α¯ x − β over (∗∗), then ¯ α¯ x − ¯ β < ¯ α¯ x − ¯ β for any L&P cut ˜ αx ≥ ˜ β equivalent to an intersection cut from S(v).

Recent Developments in Disjunctive Programming 01.09.17 39 / 56

A feasible basis for (CGLP)T and the associated solution is called regular if the cut that it defines is equivalent to an intersection cut, i.e. if it satisfies the condition of Theorem 4.5, irregular otherwise. A cut defined by an irregular solution w is called irregular, unless there exists a regular solution w ′ with the same (α, β)-component as that of w; in which case it is called regular. There are two types of irregular basic solutions. Let B be a feasible basis matrix

• f (CGLP)T , and let ˜

AK be the submatrix of ˜ A whose rows are contained in the columns Bj of B such that ut

j is basic for some t ∈ T. Then the two types are:

Type 1. ˜ AK contains a n × n nonsingular submatrix ˜ AJ, but K \ J is nonempty. Type 2. ˜ AK contains no n × n nonsingular submatrix

Recent Developments in Disjunctive Programming 01.09.17 40 / 56

L&P cuts and corner polyhedra

• Intersection cuts from ¯

x and a lattice-free polyhedron are valid for the corner polyhedron associated with ¯ x; but

• Irregular L&P cuts obtained by minimizing α¯

x − β may cut off parts

• f the corner polyhedron conv (C(¯

x) ∩ Zh).

Recent Developments in Disjunctive Programming 01.09.17 41 / 56

Frequency of irregular L&P cuts

Let ˜ A be m × n, n < m ≤ 3n, and let the disjunction have k = |T| terms.

• An arbitrary basis contains km candidate variables ut

j

for (k − 1)n places in a basis

• A regular basis contains variables ut

j with

exactly n different subscripts j

• All bases containing variables ut

j with

fewer or more than n different subscripts j are irregular The ratio irregular regular increases with k = |T|.

Recent Developments in Disjunctive Programming 01.09.17 42 / 56

Early computational results indicate the irregular cuts tend to vastly outnumber regular ones. In a recent computational experiment (E.B. and T. Serra) with L&P cuts from disjunctions of the form x1 ≤ 0 ∨ x2 ≤ 0 ∨ · · · ∨ xk ≤ 0 ∨ (xj ≥ 1, j = 1, . . . , k) (orthant cuts), all the cuts for k = 2, 3, 4 were generated for each of the instances of Steiner triples with n = 9, 15, 27 and 45. For n = 9, 15, all cuts were irregular of type 1. For n = 27, 45, ≈ 90% of the cuts were irregular of type 1, while ≈ 10% were irregular of type 2.

Recent Developments in Disjunctive Programming 01.09.17 43 / 56

Numerical Example

Consider the MIP min y such that y − 1.1x1 + x2 ≥ −0.15 y + x1 − 1.1x2 ≥ −0.2 y + x1 + x2 ≥ 0.6 x1, x2 ∈ {0, 1}, y ≥ 0 The optimal solution of the LP relaxation is x∗

1 = 23/105, x∗ 2 = 8/21, y ∗ = 0, the

• ptimal simplex tableau is

s1 x1 x2 y s2 s3 RHS 1 21/10 1 1/10 29/100 1 1 −10/21 −11/21 23/105 1 10/21 −10/21 8/21

Recent Developments in Disjunctive Programming 01.09.17 44 / 56

The LP feasible set

Recent Developments in Disjunctive Programming 01.09.17 45 / 56

Formulate a CGLP with respect to the 3-term disjunction −x1 ≥ 0 ∨ −x2 ≥ 0 ∨ x1 + x2 ≥ 2 After eliminating α, β, the CGLP is

min 29 100 u1

1 +

23 105 u1

5 +

82 105 u1

6 +

8 21 u1

7 +

13 21 u1

8 −

23 105 u1        ˜ AT ˜ bT ˜ AT ˜ bT eT        u1 +       − ˜ AT −˜ bT eT       u2 +       − ˜ AT −˜ bT eT       u3 +      −ex1 −ex1 1      u1

0+

     ex2 1      u2

0 +

     −ex1,x2 −2 1      u3

0 =

     1     

with

˜ A =            1 −1.1 1 1 1 −1.1 1 1 1 1 1 −1 1 −1            and ˜ b =            −0.15 −0.20 0.60 −1 −1            Recent Developments in Disjunctive Programming 01.09.17 46 / 56

The optimal CGLP solution has basic variables {u1

0, u2 0, u3 0; u1 2, u1 3, u2 1, u2 3, u3 2, u3 4}

The solution is irregular of type 1, as |{1, 2, 3, 4}| = 4 > 3 = n The resulting cut 3.5222y + 0.4049x1 + 0.5997x2 ≥ 1 (1) cuts off the LP optimum by more than any SIC. It also cuts off integer points of every corner polyhedron associated with any basis of the LP relaxation.

Recent Developments in Disjunctive Programming 01.09.17 47 / 56

The lift-and-project cut (irregular)

cut hyperplane

Recent Developments in Disjunctive Programming 01.09.17 48 / 56

• 5. Cuts from the V -polyhedral representation of

conv ∪

i∈Q Pi (E.B. and A. Kazachkov, in preparation) F = ∪

h∈QPh,

Ph = conv V h + cone W h, h ∈ Q where V h and W h are the vertices and extreme rays of Ph. From Theorem 1.4, the set of inequalities γx ≥ δ defining conv F is given by the solution set to the system γp ≥ δ, ∀p ∈ ∪

h∈QV h

γr ≥ 0, ∀r ∈ ∪

h∈QW h

(5.1) To generate valid cuts for conv F that chop off ¯ x we need only a small subset of the inequalities (5.1).

Recent Developments in Disjunctive Programming 01.09.17 49 / 56

V -polyhedral Disjunctive Cuts

F = ∪

h∈QPh,

Ph = conv V h + cone W h, h ∈ Q For h ∈ Q, let xh = arg min{cx : x ∈ Ph}, let C(xh) = {xh + rjλj, λj ≥ 0, j ∈ J} be the LP cone associated with xh, and let ˜ Ph be any relaxation of Ph contained in C(xh), i.e. such that Ph ⊆ ˜ Ph ⊆ C(xh). Finally, let ˜ V h and ˜ W h be the set of vertices and extreme rays, respectively, of ˜ Ph. Theorem 5.1. Any solution to the system γp ≥ δ p ∈ ∪

h∈Q

˜ V h γh ≥ 0 r ∈ ∪

h∈Q

˜ W h (5.2) defines a valid cut γx ≥ δ for conv F. Together, these cuts define conv ∪

h∈Q

˜ Ph. The system (5.2) is much smaller than (5.1). In case we choose ˜ Ph = C(xh), h ∈ Q, it has only q = |Q| nonhomogeneous and q(n − q) homogeneous inequalities.

Recent Developments in Disjunctive Programming 01.09.17 50 / 56

V -polyhedral cuts versus L&P cuts

Let ˜ Ph = {x ∈ Rn : ˜ Dhx ≥ ˜ dh

0 }, h ∈ Q

be the H-polyhedral representation of ˜ Ph = conv ˜ V h + cone ˜ W h, h ∈ Q Consider the L&P cut generating system α − uh ˜ Dh = h ∈ Q β + uh ˜ dh ≥

• h∈Q

uhe = 1 uh ≥ 0, h ∈ Q (5.3) Theorem 5.2 The L&P cuts αx ≥ β corresponding to basic solutions to (5.3) are equivalent to the V -polyhedral cuts γx ≥ δ corresponding to basic solutions to (5.2).

• Proof. Both (5.2) and (5.3) describe conv

∪

h∈Q

˜ Ph.

Recent Developments in Disjunctive Programming 01.09.17 51 / 56

Harnessing branch-and-bound information for cut generation

• The set of active nodes of a partial branch-and-bound tree offer a

convenient disjunction to derive V -polyhedral cuts from.

• At each active node h, xh and C(xh) are readily available.
• Cuts γx ≥ δ can be derived from

min ¯ xTγ − δ γxh ≥ δ h ∈ Q γrj ≥ 0 rj ∈ ext C(xh), h ∈ Q

• Cuts are valid for the whole tree

Recent Developments in Disjunctive Programming 01.09.17 52 / 56

Cuts from a partial branch and bound tree

x1 x4 x3 x2

(0,0,1) (0,1,0) (0,1,1,1) (1,1,0,0) (1,1,0,1)

5-term disjunction, |Q| = 5

D1x ≤ d1 D2x ≤ d2 D3x ≤ d3 D4x ≤ d4 D5x ≤ d5 x1 ≤ 0 x1 ≤ 0 x1 ≤ 0 x1 ≥ 1 x1 ≥ 1 x2 ≤ 0 x2 ≥ 1 x2 ≥ 1 x2 ≥ 1 x2 ≥ 1 x3 ≥ 1 x3 ≤ 0 x3 ≥ 1 x3 ≤ 0 x3 ≤ 0 x4 ≥ 1 x4 ≤ 0 x4 ≥ 1 Resulting cut can be added to the LP relax- ation for each active node. Branching on: Recent Developments in Disjunctive Programming 01.09.17 53 / 56

References to Aussois 2017 Lecture

1 K. Andersen, G. Cornuejols, Y. Li, Split closure and intersection cuts. Mathematical Programming 102, 2005, 457–493. 2 E. Balas, Disjunctive Programming. In M. Juenger et al (editors), 50 Years of Integer Programming 1958-2008: From the Early Years to the State-of-the-Art. Springer-Verlag (2010) 289–340. 3 E. Balas, Disjunctive programming. Properties of the convex hull of feasible points, invited paper with a Foreword by G. Cornu´ ejols and G. Pulleyblank, Discrete Applied Mathematics 89 (1998) 1–44. 4 E. Balas, Disjunctive programming and a hierarchy of relaxations for discrete optimization

• problems. SIAM Journal on Algebraic and Discrete Methods 6 (1985) 466–486.

5 E. Balas, A. Bockmayr, N. Pisaruk and L. Wolsey, On unions and dominants of polytopes, Mathematical Programming A (2004) 223–239. DOI: 10.1007/s10107-003-0432-4 6 E. Balas and P. Bonami, Generating lift-and-project cuts from the LP simplex tableau:

• pen source implementation and testing of new variants. Mathematical Programming

Computation 1 (2009) 165–199. 7 E. Balas, S. Ceria and G. Cornu´ ejols, A lift-and-project cutting plane algorithm for mixed 0-1 programs, Mathematical Programming 58 (1993) 295–324. 8 E. Balas, S. Ceria, G. Cornu´ ejols and N. Natraj, Gomory cuts revisited, Operations Research Letters 19 (1996) 1–10. 9 E. Balas and R. Jeroslow, Strengthening cuts for mixed integer programs. MSRR No. 359, Carnegie-Mellon University (February 1975).

Recent Developments in Disjunctive Programming 01.09.17 54 / 56

References, continued

10 E. Balas and T. Kis, Intersection cuts–standard versus restricted, Discrete Optimization 18 (2015) 189-192. 11 E. Balas and T. Kis, On the relationship between standard intersection cuts, lift-and-project cuts, and generalized intersection cuts Mathematical Programming A (2016) DOI :10.1007/s10107-015-0975-1 12 E. Balas, F. Margot, Generalized intersection cuts and a new cut generating paradigm. Mathematical Programming A, 135, 2013, 19–35 13 E. Balas and M. Perregaard, A precise correspondence between lift-and-project cuts, simple disjunctive cuts, and mixed integer Gomory cuts for 0-1 programming, Mathematical Programming 94 (2003) 221–245. 14 E. Balas, A. Qualizza, Intersection cuts from multiple rows: A disjunctive programming

• approach. EURO Journal on Computational Optimization, 1, 2013, 3–49.

15 P. Bonami, On Optimizing over lift-and-project closures. Mathematical Programming Computation, 4, 2012, 151–179. 16 M. Conforti, G. Cornuejols and G. Zambelli, Equivalence between intersection cuts and the corner polyhedron. Operations Research Letters, 33, 210, 153–155. 17 M. Fischetti, A. Lodi, A. Tramontani, On the separation of disjunctive cuts. Mathematical Programming 128, 2011, 205–230. 18 R. Gomory, Some polyhedra related to combinatorial problems. Linear Algebra and Its Applications 2, 1969, 451–558.

Recent Developments in Disjunctive Programming 01.09.17 55 / 56

References, continued

19 R.G. Jeroslow, Cutting plane theory: Disjunctive methods, Ann Discrete Math, vol. 1: Studies in Integer Programming (1977) 293–330. 20 T. Kis, Lift-and-Project for general two-term disjunctions. Discrete Optimization 12, 2014, 98–114. 21 L. Lov´ asz and A. Schrijver, Cones of matrices and set functions and 0-1 optimization, SIAM Journal of Optimization (1991) 166–190. 22 M. Perregaard and E. Balas, Generating Cuts from multiple term disjunctions, IPCO 2001, LNCS 2081, 348–360.

Recent Developments in Disjunctive Programming 01.09.17 56 / 56