[PPT] - Exact General-Purpose Solvers for Mixed-Integer Bilevel Linear PowerPoint Presentation

SLIDE 1

Exact General-Purpose Solvers for Mixed-Integer Bilevel Linear Programs

Tutorial

I. Ljubi´

c

ESSEC Business School of Paris, France

JFRO 2018, March 26, Paris

SLIDE 2

Based on the papers:

Part I: M. Fischetti, I. Ljubi´

c, M. Monaci, M. Sinnl: On the Use of Intersection Cuts for Bilevel Optimization, Mathematical Programming, to appear, 2018

Part II: M. Fischetti, I. Ljubi´

c, M. Monaci, M. Sinnl: A new general-purpose algorithm for mixed-integer bilevel linear programs, Operations Research 65(6): 1615-1637, 2017

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 2

SLIDE 3

Bilevel Optimization

General bilevel optimization problem min

x∈Rn1,y∈Rn2F(x, y)

(1) G(x, y) ≤ 0 (2) y ∈ arg min

y ′∈Rn2{f (x, y ′) : g(x, y ′) ≤ 0 }

(3)

Stackelberg game: two-person sequential game
Leader takes follower’s optimal reaction into account
Nx = {1, . . . , n1}, Ny = {1, . . . , n2}
n = n1 + n2: total number of decision variables

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 3

SLIDE 4

Bilevel Optimization

General bilevel optimization problem min

x∈Rn1,y∈Rn2F(x, y)

(1) G(x, y) ≤ 0 (2) y ∈ arg min

y ′∈Rn2{f (x, y ′) : g(x, y ′) ≤ 0 }

(3)

Leader

Stackelberg game: two-person sequential game
Leader takes follower’s optimal reaction into account
Nx = {1, . . . , n1}, Ny = {1, . . . , n2}
n = n1 + n2: total number of decision variables

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 3

SLIDE 5

Bilevel Optimization

General bilevel optimization problem min

x∈Rn1,y∈Rn2F(x, y)

(1) G(x, y) ≤ 0 (2) y ∈ arg min

y ′∈Rn2{f (x, y ′) : g(x, y ′) ≤ 0 }

(3)

Leader Follower

Stackelberg game: two-person sequential game
Leader takes follower’s optimal reaction into account
Nx = {1, . . . , n1}, Ny = {1, . . . , n2}
n = n1 + n2: total number of decision variables

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 3

SLIDE 6

Optimistic vs Pessimistic Solution

The Stackelberg game under:

Perfect information: both agents have perfect knowledge of each others

strategy

Rationality: agents act optimally, according to their respective goals
What if there are multiple optimal solutions for the follower?

◮ Optimistic Solution: among the follower’s solution, the one leading to the

best outcome for the leader is assumed

◮ Pessimistic Solution: among the follower’s solution, the one leading to the

worst outcome for the leader is assumed

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 4

SLIDE 7

Optimistic vs Pessimistic Solution

The Stackelberg game under:

Perfect information: both agents have perfect knowledge of each others

strategy

Rationality: agents act optimally, according to their respective goals
What if there are multiple optimal solutions for the follower?

◮ Optimistic Solution: among the follower’s solution, the one leading to the

best outcome for the leader is assumed

◮ Pessimistic Solution: among the follower’s solution, the one leading to the

worst outcome for the leader is assumed

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 4

SLIDE 8

Our Focus: Mixed-Integer Bilevel Linear Programs (MIBLP)

(MIBLP) min cT

x x + cT y y

(4) Gxx + Gyy ≤ 0 (5) y ∈ arg min{dTy : Ax + By ≤ 0, (6) yj integer, ∀j ∈ Jy} (7) xj integer, ∀j ∈ Jx (8) (x, y) ∈ Rn (9) where cx, cy, Gx, Gy, A, B are given rational matrices/vectors of appropriate size.

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 5

SLIDE 9

Complexity Bilevel Linear Programs

Bilevel LPs are strongly NP-hard (Audet et al. [1997], Hansen et al. [1992]). min cTx Ax = b x ∈ {0, 1} ⇔ min cTx Ax = b v = 0 v ∈ arg max{w : w ≤ x, w ≤ 1 − x, w ≥ 0}

x w Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 6

SLIDE 10

Complexity Bilevel Mixed-Integer Linear Programs

MIBLP is ΣP

2 -hard (Lodi et al. [2014]): there is no way of formulating MIBLP as a

MILP of polynomial size unless the polynomial hierarchy collapses.

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 7

SLIDE 11

Overview Part I

Develop a finitely convergent branch-and-bound approach (under certain

conditions)

Capable of dealing with unboundedness and infeasibility
Introduce intersection cuts to speed-up convergence

Part II

Introduce a fully-fledged branch-and-cut for MIBLPs

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 8

SLIDE 12

STEP 1: VALUE FUNCTION REFORMULATION

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 9

SLIDE 13

Our Focus: Mixed-Integer Bilevel Linear Programs (MIBLP)

Value Function Reformulation: (MIBLP) min cT

x x + cT y y

(10) Gxx + Gyy ≤ 0 (11) Ax + By ≤ 0 (12) (x, y) ∈ Rn (13) dTy ≤ Φ(x) (14) xj integer, ∀j ∈ Jx (15) yj integer, ∀j ∈ Jy (16) where Φ(x) is non-convex, non-continuous: Φ(x) = min{dTy : Ax + By ≤ 0, yj integer, ∀j ∈ Jy}

dropping dTy ≤ Φ(x) → High Point Relaxation (HPR) which is a MILP →

we can use MILP solvers with all their tricks

let HPR be LP-relaxation of HPR

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 10

SLIDE 14

Our Focus: Mixed-Integer Bilevel Linear Programs (MIBLP)

Value Function Reformulation: (HPR) min cT

x x + cT y y

(10) Gxx + Gyy ≤ 0 (11) Ax + By ≤ 0 (12) (x, y) ∈ Rn (13) (14) xj integer, ∀j ∈ Jx (15) yj integer, ∀j ∈ Jy (16)

I am a Mixed-Integer Linear Program (MILP)

where Φ(x) is non-convex, non-continuous: Φ(x) = min{dTy : Ax + By ≤ 0, yj integer, ∀j ∈ Jy}

dropping dTy ≤ Φ(x) → High Point Relaxation (HPR) which is a MILP →

we can use MILP solvers with all their tricks

let HPR be LP-relaxation of HPR

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 10

SLIDE 15

Our Focus: Mixed-Integer Bilevel Linear Programs (MIBLP)

Value Function Reformulation: (HPR) min cT

x x + cT y y

(10) Gxx + Gyy ≤ 0 (11) Ax + By ≤ 0 (12) (x, y) ∈ Rn (13) (14) (15) (16) where Φ(x) is non-convex, non-continuous: Φ(x) = min{dTy : Ax + By ≤ 0, yj integer, ∀j ∈ Jy}

dropping dTy ≤ Φ(x) → High Point Relaxation (HPR) which is a MILP →

we can use MILP solvers with all their tricks

let HPR be LP-relaxation of HPR

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 10

SLIDE 16

Example

notorious example from Moore and Bard [1990]
HPR
value-function reformulation

min

x∈Z −x − 10y

y ∈ arg min

y ′∈Z{y ′ :

−25x + 20y ′ ≤ 30 x + 2y ′ ≤ 10 2x − y ′ ≤ 15 2x + 10y ′ ≥ 15} x y 1 2 3 4 5 6 7 8 1 2 3 4

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 11

SLIDE 17

Example

notorious example from Moore and Bard [1990]
HPR
value-function reformulation

min

x,y∈Z −x − 10y

−25x + 20y ≥ 30 x + 2y ≤ 10 2x − y ≤ 15 2x + 10y ≥ 15 x y 1 2 3 4 5 6 7 8 1 2 3 4

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 11

SLIDE 18

Example

notorious example from Moore and Bard [1990]
HPR
value-function reformulation

min

x,y∈Z −x − 10y

−25x + 20y ≥ 30 x + 2y ≤ 10 2x − y ≤ 15 2x + 10y ≥ 15 y ≤ Φ(x) x y 1 2 3 4 5 6 7 8 1 2 3 4 Φ(x)

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 11

SLIDE 19

General Idea General Procedure

Start with the HPR- (or HPR-)relaxation
Get rid of bilevel infeasible solutions on the fly
Apply branch-and-bound or branch-and-cut algorithm

There are some unexpected difficulties along the way...

Optimal solution can be unattainable
HPR can be unbounded

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 12

SLIDE 20

(Un)expected Difficulties: Unattainable Solutions Example from K¨

ppe et al. [2010]

Continuous variables in the leader, integer variables in the follower ⇒ optimal solution may be unattainable inf

x,y

x − y 0 ≤ x ≤ 1 y ∈ arg min

y ′ {y ′ : y ′ ≥ x, 0 ≤ y ′ ≤ 1, y ′ ∈ Z}.

Equivalent to inf

x {x − ⌈x⌉ : 0 ≤ x ≤ 1}

x y 1 1

Bilevel feasible set is neither convex nor closed. Crucial assumption for us: follower subproblem depends only on integer leader variables JF ⊆ Jx.

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 13

SLIDE 21

(Un)expected Difficulties: Unattainable Solutions Example from K¨

ppe et al. [2010]

Continuous variables in the leader, integer variables in the follower ⇒ optimal solution may be unattainable inf

x,y

x − y 0 ≤ x ≤ 1 y ∈ arg min

y ′ {y ′ : y ′ ≥ x, 0 ≤ y ′ ≤ 1, y ′ ∈ Z}.

Equivalent to inf

x {x − ⌈x⌉ : 0 ≤ x ≤ 1}

x y 1 1

Bilevel feasible set is neither convex nor closed. Crucial assumption for us: follower subproblem depends only on integer leader variables JF ⊆ Jx.

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 13

SLIDE 22

(Un)expected Difficulties: Unbounded HPR-Relaxation Example from Xu and Wang [2014]

Unboundness of HPR-relaxation does not allow to draw conclusions on the

ptimal solution of MIBLP
unbounded
infeasible
admit an optimal solution

max

x,y

x + y 0 ≤ x ≤ 2 x ∈ Z y ∈ arg max

y ′ {d · y ′ : y ′ ≥ x, y ′ ∈ Z}.

max

x,y

x + y 0 ≤ x ≤ 2 y ≥ x x, y ∈ Z d = 1 ⇒ Φ(x) = ∞ (MIBLP infeasible) d = 0 ⇒ Φ(x) feasible for all y ∈ Z (MIBLP unbounded) d = −1 ⇒ x∗ = 2, y ∗ = 2 (optimal MIBLP solution)

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 14

SLIDE 23

(Un)expected Difficulties: Unbounded HPR-Relaxation Example from Xu and Wang [2014]

Unboundness of HPR-relaxation does not allow to draw conclusions on the

ptimal solution of MIBLP
unbounded
infeasible
admit an optimal solution

max

x,y

x + y 0 ≤ x ≤ 2 x ∈ Z y ∈ arg max

y ′ {d · y ′ : y ′ ≥ x, y ′ ∈ Z}.

max

x,y

x + y 0 ≤ x ≤ 2 y ≥ x x, y ∈ Z d = 1 ⇒ Φ(x) = ∞ (MIBLP infeasible) d = 0 ⇒ Φ(x) feasible for all y ∈ Z (MIBLP unbounded) d = −1 ⇒ x∗ = 2, y ∗ = 2 (optimal MIBLP solution)

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 14

SLIDE 24

STEP 2: BRANCH-AND-CUT ALGORITHM

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 15

SLIDE 25

Assumption

All the integer-constrained variables x and y have finite lower and upper bounds both in HPR and in the follower MILP.

Assumption

Continuous leader variables xj (if any) do not appear in the follower problem. If for all HPR solutions, the follower MILP is unbounded ⇒ MIBLP is infeasible. Preprocessing (solving a single LP) allows to check this. Hence:

Assumption

For an arbitrary HPR solution, the follower MILP is well defined.

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 16

SLIDE 26

Algorithm 1: A basic branch-and-bound scheme for MIBLP

Apply a standard LP-based B&B to HPR, inhibit incumbent update, and node-fathoming due to unboundedness of HPR for each unfathomed B&B node where standard branching cannot be performed do if HPR is not unbounded then Let (x∗, y ∗) be the HPR solution at the current node; Compute Φ(x∗) by solving the follower MILP for x = x∗; if dTy ∗ ≤ Φ(x∗) then The current solution (x∗, y ∗) is bilevel feasible: update the incumbent, fathom the current node, and continue with another node end end if all variables xj with j ∈ JF are fixed by branching (x∗

F ) then

Refinement: Solve HPR with x = x∗

F , dTy ≤ Φ(x∗ F ). If unbounded return

UNBOUNDED; Possibly update the incumbent with the resulting solution (ˆ x, ˆ y), if any; Fathom the current node else Branch on any xj (j ∈ JF) not fixed by branching yet (even if x∗

j is integer

in the LP-solution at the node) end end

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 16

SLIDE 27

Our Goal: Design MILP-based solver for MIBLP

For the rest of presentation: Assume HPR value is bounded.

Our Goal

solve MIBLP by using a standard simplex-based branch-and-cut algorithm; enforce dTy ≤ Φ(x) on the fly, by adding cutting planes

given optimal vertex (x∗, y ∗) of HPR

◮ (x∗, y ∗) infeasible for HPR (i.e., fractional) → branch as usual ◮ (x∗, y ∗) feasible for HPR and f (x∗, y ∗) ≤ Φ(x∗) → update the incumbent as

usual

◮ (x∗, y ∗) feasible for HPR and f (x∗, y ∗) > Φ(x∗), i.e., bilevel-infeasible → we

need to do something!

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 17

SLIDE 28

Our Goal: Design MILP-based solver for MIBLP

For the rest of presentation: Assume HPR value is bounded.

Our Goal

solve MIBLP by using a standard simplex-based branch-and-cut algorithm; enforce dTy ≤ Φ(x) on the fly, by adding cutting planes

given optimal vertex (x∗, y ∗) of HPR

◮ (x∗, y ∗) infeasible for HPR (i.e., fractional) → branch as usual ◮ (x∗, y ∗) feasible for HPR and f (x∗, y ∗) ≤ Φ(x∗) → update the incumbent as

usual

◮ (x∗, y ∗) feasible for HPR and f (x∗, y ∗) > Φ(x∗), i.e., bilevel-infeasible → we

need to do something!

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 17

SLIDE 29

Our Goal: Design MILP-based solver for MIBLP

For the rest of presentation: Assume HPR value is bounded.

Our Goal

solve MIBLP by using a standard simplex-based branch-and-cut algorithm; enforce dTy ≤ Φ(x) on the fly, by adding cutting planes

given optimal vertex (x∗, y ∗) of HPR

◮ (x∗, y ∗) infeasible for HPR (i.e., fractional) → branch as usual ◮ (x∗, y ∗) feasible for HPR and f (x∗, y ∗) ≤ Φ(x∗) → update the incumbent as

usual

◮ (x∗, y ∗) feasible for HPR and f (x∗, y ∗) > Φ(x∗), i.e., bilevel-infeasible → we

need to do something!

Moore and Bard [1990] (Branch-and-Bound)

◮ branching to cut-off bilevel infeasible solutions ◮ no y-variables in leader-constraints ◮ either all x-variables integer or all y-variables continuous Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 17

SLIDE 30

Our Goal: Design MILP-based solver for MIBLP

For the rest of presentation: Assume HPR value is bounded.

Our Goal

solve MIBLP by using a standard simplex-based branch-and-cut algorithm; enforce dTy ≤ Φ(x) on the fly, by adding cutting planes

given optimal vertex (x∗, y ∗) of HPR

◮ (x∗, y ∗) infeasible for HPR (i.e., fractional) → branch as usual ◮ (x∗, y ∗) feasible for HPR and f (x∗, y ∗) ≤ Φ(x∗) → update the incumbent as

usual

◮ (x∗, y ∗) feasible for HPR and f (x∗, y ∗) > Φ(x∗), i.e., bilevel-infeasible → we

need to do something!

DeNegre [2011], DeNegre & Ralphs (Branch-and-Cut)

◮ cuts based on slack ◮ needs all variables and coefficients to be integer ◮ open-source solver MibS Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 17

SLIDE 31

Our Goal: Design MILP-based solver for MIBLP

For the rest of presentation: Assume HPR value is bounded.

Our Goal

solve MIBLP by using a standard simplex-based branch-and-cut algorithm; enforce dTy ≤ Φ(x) on the fly, by adding cutting planes

given optimal vertex (x∗, y ∗) of HPR

◮ (x∗, y ∗) infeasible for HPR (i.e., fractional) → branch as usual ◮ (x∗, y ∗) feasible for HPR and f (x∗, y ∗) ≤ Φ(x∗) → update the incumbent as

usual

◮ (x∗, y ∗) feasible for HPR and f (x∗, y ∗) > Φ(x∗), i.e., bilevel-infeasible → we

need to do something!

Xu and Wang [2014], Wang and Xu [2017] (Branch-and-Bound)

◮ multiway branching to cut-off bilevel infeasible solutions ◮ all x-variables integer and bounded, follower coefficients of x-variables must be

integer

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 17

SLIDE 32

Our Goal: Design MILP-based solver for MIBLP

For the rest of presentation: Assume HPR value is bounded.

Our Goal

solve MIBLP by using a standard simplex-based branch-and-cut algorithm; enforce dTy ≤ Φ(x) on the fly, by adding cutting planes

given optimal vertex (x∗, y ∗) of HPR

◮ (x∗, y ∗) infeasible for HPR (i.e., fractional) → branch as usual ◮ (x∗, y ∗) feasible for HPR and f (x∗, y ∗) ≤ Φ(x∗) → update the incumbent as

usual

◮ (x∗, y ∗) feasible for HPR and f (x∗, y ∗) > Φ(x∗), i.e., bilevel-infeasible → we

need to do something!

Our Approach (Branch-and-Cut)

◮ Use Intersection Cuts (Balas [1971]) to cut off bilevel infeasible solutions Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 17

SLIDE 33

STEP 3: INTERSECTION CUTS

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 18

SLIDE 34

Intersection Cuts (ICs)

powerful tool to separate a bilevel infeasible point (x∗, y ∗) from a set of

bilevel feasible points (X, Y ) by a linear cut

what we need to derive ICs

◮ a cone pointed at (x∗, y ∗) containing all (X, Y ) (if (x∗, y ∗) is a vertex of

HPR-relaxation, a possible cone comes from LP-basis)

◮ a convex set S with (x∗, y ∗) but no bilevel feasible points ((x, y) ∈ (X, Y )) in

its interior

◮ important: (x∗, y ∗) should not be on the frontier of S. Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 19

SLIDE 35

Intersection Cuts (ICs)

powerful tool to separate a bilevel infeasible point (x∗, y ∗) from a set of

bilevel feasible points (X, Y ) by a linear cut

what we need to derive ICs

◮ a cone pointed at (x∗, y ∗) containing all (X, Y ) (if (x∗, y ∗) is a vertex of

HPR-relaxation, a possible cone comes from LP-basis)

◮ a convex set S with (x∗, y ∗) but no bilevel feasible points ((x, y) ∈ (X, Y )) in

its interior

◮ important: (x∗, y ∗) should not be on the frontier of S. Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 19

SLIDE 36

Intersection Cuts (ICs)

powerful tool to separate a bilevel infeasible point (x∗, y ∗) from a set of

bilevel feasible points (X, Y ) by a linear cut

what we need to derive ICs

◮ a cone pointed at (x∗, y ∗) containing all (X, Y ) (if (x∗, y ∗) is a vertex of

HPR-relaxation, a possible cone comes from LP-basis)

◮ a convex set S with (x∗, y ∗) but no bilevel feasible points ((x, y) ∈ (X, Y )) in

its interior

◮ important: (x∗, y ∗) should not be on the frontier of S. Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 19

SLIDE 37

Intersection Cuts (ICs)

powerful tool to separate a bilevel infeasible point (x∗, y ∗) from a set of

bilevel feasible points (X, Y ) by a linear cut

what we need to derive ICs

◮ a cone pointed at (x∗, y ∗) containing all (X, Y ) (if (x∗, y ∗) is a vertex of

HPR-relaxation, a possible cone comes from LP-basis)

◮ a convex set S with (x∗, y ∗) but no bilevel feasible points ((x, y) ∈ (X, Y )) in

its interior

◮ important: (x∗, y ∗) should not be on the frontier of S. Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 19

SLIDE 38

Intersection Cuts (ICs)

powerful tool to separate a bilevel infeasible point (x∗, y ∗) from a set of

bilevel feasible points (X, Y ) by a linear cut IC

what we need to derive ICs

◮ a cone pointed at (x∗, y ∗) containing all (X, Y ) (if (x∗, y ∗) is a vertex of

HPR-relaxation, a possible cone comes from LP-basis)

◮ a convex set S with (x∗, y ∗) but no bilevel feasible points ((x, y) ∈ (X, Y )) in

its interior

◮ important: (x∗, y ∗) should not be on the frontier of S. Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 19

SLIDE 39

Intersection Cuts for Bilevel Optimization

we need a bilevel-free set S

Theorem

For any feasible solution of the follower ˆ y ∈ Rn2, the set S(ˆ y) = {(x, y) ∈ Rn : dTy > dT ˆ y, Ax + B ˆ y ≤ b} does not contain any bilevel-feasible point (not even on its frontier).

note: S(ˆ

y) is a polyhedron

problem: bilevel-infeasible (x∗, y ∗) can be on the frontier of bilevel-free set

S → IC based on S(ˆ y) may not be able to cut off (x∗, y ∗)

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 20

SLIDE 40

Intersection Cuts for Bilevel Optimization

we need a bilevel-free set S

Theorem

For any feasible solution of the follower ˆ y ∈ Rn2, the set S(ˆ y) = {(x, y) ∈ Rn : dTy > dT ˆ y, Ax + B ˆ y ≤ b} does not contain any bilevel-feasible point (not even on its frontier).

note: S(ˆ

y) is a polyhedron

problem: bilevel-infeasible (x∗, y ∗) can be on the frontier of bilevel-free set

S → IC based on S(ˆ y) may not be able to cut off (x∗, y ∗)

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 20

SLIDE 41

Intersection Cuts for Bilevel Optimization

we need a bilevel-free set S

Theorem

For any feasible solution of the follower ˆ y ∈ Rn2, the set S(ˆ y) = {(x, y) ∈ Rn : dTy > dT ˆ y, Ax + B ˆ y ≤ b} does not contain any bilevel-feasible point (not even on its frontier).

note: S(ˆ

y) is a polyhedron

problem: bilevel-infeasible (x∗, y ∗) can be on the frontier of bilevel-free set

S → IC based on S(ˆ y) may not be able to cut off (x∗, y ∗)

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 20

SLIDE 42

Intersection Cuts for Bilevel Optimization Assumption

Ax + By − b is integer for all HPR solutions (x, y).

Theorem

Under the previous assumption, for any feasible solution of the follower ˆ y ∈ Rn2, the extended polyhedron S+(ˆ y) = {(x, y) ∈ Rn : dTy ≥ dT ˆ y, Ax + B ˆ y ≤ b + 1}, (17) where 1 = (1, · · · , 1) denote a vector of all ones of suitable size, does not contain any bilevel feasible point in its interior.

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 21

SLIDE 43

Intersection Cuts for Bilevel Optimization Assumption

Ax + By − b is integer for all HPR solutions (x, y).

Theorem

Under the previous assumption, for any feasible solution of the follower ˆ y ∈ Rn2, the extended polyhedron S+(ˆ y) = {(x, y) ∈ Rn : dTy ≥ dT ˆ y, Ax + B ˆ y ≤ b + 1}, (17) where 1 = (1, · · · , 1) denote a vector of all ones of suitable size, does not contain any bilevel feasible point in its interior.

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 21

SLIDE 44

Intersection Cuts for Bilevel Optimization

application sketch on the example from Moore and Bard [1990]
solve HPR→ obtain (x∗, y ∗) = (2, 4) and LP-cone, take ˆ

y = 2

solve HPR again → obtain (x∗, y ∗) = (6, 2) and LP-cone, take ˆ

y = 1 min

x∈Z −x − 10y

y ∈ arg min

y ′∈Z{y ′ :

−25x + 20y ′ ≤ 30 x + 2y ′ ≤ 10 2x − y ′ ≤ 15 2x + 10y ′ ≥ 15} x y 1 2 3 4 5 6 7 8 1 2 3 4

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 22

SLIDE 45

Intersection Cuts for Bilevel Optimization

application sketch on the example from Moore and Bard [1990]
solve HPR→ obtain (x∗, y ∗) = (2, 4) and LP-cone, take ˆ

y = 2

solve HPR again → obtain (x∗, y ∗) = (6, 2) and LP-cone, take ˆ

y = 1 min

x∈Z −x − 10y

y ∈ arg min

y ′∈Z{y ′ :

−25x + 20y ′ ≤ 30 x + 2y ′ ≤ 10 2x − y ′ ≤ 15 2x + 10y ′ ≥ 15} x y 1 2 3 4 5 6 7 8 1 2 3 4

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 22

SLIDE 46

Intersection Cuts for Bilevel Optimization

application sketch on the example from Moore and Bard [1990]
solve HPR→ obtain (x∗, y ∗) = (2, 4) and LP-cone, take ˆ

y = 2

solve HPR again → obtain (x∗, y ∗) = (6, 2) and LP-cone, take ˆ

y = 1 min

x∈Z −x − 10y

y ∈ arg min

y ′∈Z{y ′ :

−25x + 20y ′ ≤ 30 x + 2y ′ ≤ 10 2x − y ′ ≤ 15 2x + 10y ′ ≥ 15} x y 1 2 3 4 5 6 7 8 1 2 3 4

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 22

SLIDE 47

Intersection Cuts for Bilevel Optimization

application sketch on the example from Moore and Bard [1990]
solve HPR→ obtain (x∗, y ∗) = (2, 4) and LP-cone, take ˆ

y = 2

solve HPR again → obtain (x∗, y ∗) = (6, 2) and LP-cone, take ˆ

y = 1 min

x∈Z −x − 10y

y ∈ arg min

y ′∈Z{y ′ :

−25x + 20y ′ ≤ 30 x + 2y ′ ≤ 10 2x − y ′ ≤ 15 2x + 10y ′ ≥ 15} x y 1 2 3 4 5 6 7 8 1 2 3 4

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 22

SLIDE 48

Intersection Cuts for Bilevel Optimization

application sketch on the example from Moore and Bard [1990]
solve HPR→ obtain (x∗, y ∗) = (2, 4) and LP-cone, take ˆ

y = 2

solve HPR again → obtain (x∗, y ∗) = (6, 2) and LP-cone, take ˆ

y = 1 min

x∈Z −x − 10y

y ∈ arg min

y ′∈Z{y ′ :

−25x + 20y ′ ≤ 30 x + 2y ′ ≤ 10 2x − y ′ ≤ 15 2x + 10y ′ ≥ 15} x y 1 2 3 4 5 6 7 8 1 2 3 4

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 22

SLIDE 49

Intersection Cuts for Bilevel Optimization

application sketch on the example from Moore and Bard [1990]
solve HPR→ obtain (x∗, y ∗) = (2, 4) and LP-cone, take ˆ

y = 2

solve HPR again → obtain (x∗, y ∗) = (6, 2) and LP-cone, take ˆ

y = 1 min

x∈Z −x − 10y

y ∈ arg min

y ′∈Z{y ′ :

−25x + 20y ′ ≤ 30 x + 2y ′ ≤ 10 2x − y ′ ≤ 15 2x + 10y ′ ≥ 15} x y 1 2 3 4 5 6 7 8 1 2 3 4

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 22

SLIDE 50

Intersection Cuts for Bilevel Optimization

application sketch on the example from Moore and Bard [1990]
solve HPR→ obtain (x∗, y ∗) = (2, 4) and LP-cone, take ˆ

y = 2

solve HPR again → obtain (x∗, y ∗) = (6, 2) and LP-cone, take ˆ

y = 1 min

x∈Z −x − 10y

y ∈ arg min

y ′∈Z{y ′ :

−25x + 20y ′ ≤ 30 x + 2y ′ ≤ 10 2x − y ′ ≤ 15 2x + 10y ′ ≥ 15} x y 1 2 3 4 5 6 7 8 1 2 3 4

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 22

SLIDE 51

Intersection Cuts for Bilevel Optimization

application sketch on the example from Moore and Bard [1990]
solve HPR→ obtain (x∗, y ∗) = (2, 4) and LP-cone, take ˆ

y = 2

solve HPR again → obtain (x∗, y ∗) = (6, 2) and LP-cone, take ˆ

y = 1 min

x∈Z −x − 10y

y ∈ arg min

y ′∈Z{y ′ :

−25x + 20y ′ ≤ 30 x + 2y ′ ≤ 10 2x − y ′ ≤ 15 2x + 10y ′ ≥ 15} x y 1 2 3 4 5 6 7 8 1 2 3 4

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 22

SLIDE 52

Separating Intersection Cuts

given bilevel infeasible (x∗, y ∗), how do we determine convex bilevel-free set

S+(ˆ y)?

a natural option: use the optimal solution ˆ

y of the follower subproblem for x = x∗

◮ needs to be solved in any case to check bilevel-feasibility of (x∗, y ∗)

separation procedure is a MILP:

SEP − 1 : ˆ y ∈ arg min{dTy Ax∗ + By ≤ b yj integer ∀j ∈ Jy} SEP-1 maximizes distance of (x∗, y ∗) to dTy ≥ dT ˆ y.

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 23

SLIDE 53

Separating Intersection Cuts

given bilevel infeasible (x∗, y ∗), how do we determine convex bilevel-free set

S+(ˆ y)?

a natural option: use the optimal solution ˆ

y of the follower subproblem for x = x∗

◮ needs to be solved in any case to check bilevel-feasibility of (x∗, y ∗)

separation procedure is a MILP:

SEP − 1 : ˆ y ∈ arg min{dTy Ax∗ + By ≤ b yj integer ∀j ∈ Jy} SEP-1 maximizes distance of (x∗, y ∗) to dTy ≥ dT ˆ y.

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 23

SLIDE 54

Separating Intersection Cuts

given bilevel infeasible (x∗, y ∗), how do we determine convex bilevel-free set

S+(ˆ y)?

a natural option: use the optimal solution ˆ

y of the follower subproblem for x = x∗

◮ needs to be solved in any case to check bilevel-feasibility of (x∗, y ∗)

separation procedure is a MILP:

SEP − 1 : ˆ y ∈ arg min{dTy Ax∗ + By ≤ b yj integer ∀j ∈ Jy} SEP-1 maximizes distance of (x∗, y ∗) to dTy ≥ dT ˆ y.

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 23

SLIDE 55

COMPUTATIONAL RESULTS (First insights about usefulness of intersection cuts)

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 24

SLIDE 56

Computational Results

C, CPLEX 12.6.3, Intel Xeon E3-1220V2 3.1 GHz, four threads

Table: Our testbed. Column #inst reports the total number of instances in the class, while column type indicates whether the instances are binary (B) or integer (I).

Class source # inst type Notes DENEGRE DeNegre [2011] 50 I randomly generated INTERDICTION DeNegre [2011] 125 B interdiction inst.s MIPLIB Fischetti et al. [2016] 57 B from MIPLIB 3.0

Table: Our tested settings. #cutsr/#cutso: maximum number of cuts added at root node/all other nodes

Name Sep. #cutsr #cutso SEP-1a SEP-1 20 20 SEP-1b SEP-1 20 BENCHMARK

ur benchmark code implementing cuts in DeNegre [2011]

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 25

SLIDE 57

Computational Results

C, CPLEX 12.6.3, Intel Xeon E3-1220V2 3.1 GHz, four threads

Table: Our testbed. Column #inst reports the total number of instances in the class, while column type indicates whether the instances are binary (B) or integer (I).

Class source # inst type Notes DENEGRE DeNegre [2011] 50 I randomly generated INTERDICTION DeNegre [2011] 125 B interdiction inst.s MIPLIB Fischetti et al. [2016] 57 B from MIPLIB 3.0

Table: Our tested settings. #cutsr/#cutso: maximum number of cuts added at root node/all other nodes

Name Sep. #cutsr #cutso SEP-1a SEP-1 20 20 SEP-1b SEP-1 20 BENCHMARK

ur benchmark code implementing cuts in DeNegre [2011]

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 25

SLIDE 58

Computational Results

C, CPLEX 12.6.3, Intel Xeon E3-1220V2 3.1 GHz, four threads

Table: Our testbed. Column #inst reports the total number of instances in the class, while column type indicates whether the instances are binary (B) or integer (I).

Class source # inst type Notes DENEGRE DeNegre [2011] 50 I randomly generated INTERDICTION DeNegre [2011] 125 B interdiction inst.s MIPLIB Fischetti et al. [2016] 57 B from MIPLIB 3.0

Table: Our tested settings. #cutsr/#cutso: maximum number of cuts added at root node/all other nodes

Name Sep. #cutsr #cutso SEP-1a SEP-1 20 20 SEP-1b SEP-1 20 BENCHMARK

ur benchmark code implementing cuts in DeNegre [2011]

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 25

SLIDE 59

Computational Results

Table: Summary of obtained results. We report the number of solved instances (#), the shifted geometric mean for computing time (t[s]) and for number of nodes (nodes), and the average gaps (g[%]).

MIPLIB (57 inst.s) INTERDICTION (125 inst.s) DENEGRE (50 inst.s) setting # t[s] nodes g[%] # t[s] nodes g[%] # t[s] nodes g[%] SEP-1a 20 599 9655.9 27.65 83 148 36769.3 33.06 42 40 574.0 4.61 SEP-1b 18 660 100475.8 27.85 64 245 240859.4 48.39 45 35 12452.1 3.89 BENCHMARK 15 954 234670.7 31.78 44 496 1310639.5 63.45 38 58 27918.5 9.20

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 26

SLIDE 60

Computational Results

Figure: Performance profile plot over all instances (classes DENEGRE, INTERDICTION and MIPLIB).

●
●
●
●●
● ●

20 40 60 1 10 100 1000

No more than x times worse than best setting Percentage of instances

Setting

SEP−1a

SEP−1b BENCHMARK

The leftmost point of the graph for a setting s shows the percentage of instances for which s is the fastest setting. The rightmost point shows the percentage of instances solved to optimality by s.

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 27

SLIDE 61

PART II: MILP-BASED SOLVER for MIBLP

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 28

SLIDE 62

MILP-based solver for MIBLP

Basic Solution Scheme

standard simplex-based branch-and-cut algorithm . . .
. . . that enforces dTy ≤ Φ(x), on the fly, by adding cutting planes.

New features:

Follower preprocessing.
Follower Upper-Bound cuts.
Intersection Cuts (ICs):

◮ New families of ICs; ◮ Separation of ICs. Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 29

SLIDE 63

Follower Preprocessing

ˆ y ∈ arg min{dTy Ax + By ≤ b l ≤ y ≤ u yj integer ∀j ∈ Jy}

Theorem

Let yj be a follower variable and let lj be its lower bound in the follower. If dj > 0 and Bj ≥ 0 then yj = lj in any optimal solution.

Idea: for any x∗ ∈ Rn1, fixing variable yj to the lower bound decreases the

follower cost and does not reduce the associated feasible set.

Fix yj = lj in the HPR as well.
Large impact in the performance of the algorithm.
Observation: to preserve equivalent optimal solutions for the follower, we

require dj be strictly positive.

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 30

SLIDE 64

Follower Preprocessing

ˆ y ∈ arg min{dTy Ax + By ≤ b l ≤ y ≤ u yj integer ∀j ∈ Jy}

Theorem

Let yj be a follower variable and let uj be its upper bound in the follower. If dj < 0 and Bj ≤ 0 then yj = uj in any optimal solution.

Idea: for any x∗ ∈ Rn1, fixing variable yj to the upper bound decreases the

follower cost and does not reduce the associated feasible set.

Fix yj = uj in the HPR as well.
Large impact in the performance of the algorithm.
Observation: to preserve equivalent optimal solutions for the follower, we

require dj be strictly negative.

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 30

SLIDE 65

Follower Upper-Bound (FUB) cuts Observation:

Let FUB be an upper bound for the value of the follower’s solution, independently

n the choice of x. Then:

dTy ≤ FUB is a valid cut for HPR.

Tighter Bounds

Tighter FUB values could be obtained inside the B&B tree, but these cuts are

nly locally valid.

Overrestricting the Follower

By replacing original constraints Ax + By ≤ b by more restricting ones (independent on the choice of x), a FUB can be obtained.

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 31

SLIDE 66

Follower Upper-Bound cuts Theorem

Let (x−, x+) denote the bounds for the x variables at the current B&B node. The following inequality dTy ≤ FUB(x−, x+) is locally valid for the current node, where FUB(x−, x+) := min{dTy

j∈Nx

max{Aijx−

j , Aijx+ j } +

j∈Ny

Bijyj ≤ bi, i = 1, . . . , m yj integer, ∀j ∈ Jy}.

FUB(x−, x+) is an overestimator of the follower objective at the current

node (all x’s are set to their worst value).

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 32

SLIDE 67

Follower Upper-Bound cuts Theorem

Let (x−, x+) denote the bounds for the x variables at the current B&B node. The following inequality dTy ≤ FUB(x−, x+) is locally valid for the current node, where FUB(x−, x+) := min{dTy

j∈Nx

max{Aijx−

j , Aijx+ j } +

j∈Ny

Bijyj ≤ bi, i = 1, . . . , m yj integer, ∀j ∈ Jy}.

FUB(x−, x+) is an overestimator of the follower objective at the current

node (all x’s are set to their worst value).

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 32

SLIDE 68

MORE ON INTERSECTION CUTS

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 33

SLIDE 69

Intersection Cuts (ICs)

Main ingredient of our basic branch-and-cut algorithm.
Given an infeasible x∗ and the associated simplex cone, the definition of an

IC asks for the definition of a convex set S with x∗ but no bilevel-feasible x ∈ X in its interior.

The choice of bilevel-free polyhedra is not unique.
The larger the bilevel-free set, the better the IC.

Theorem (Fischetti et al. [2018])

Given ˆ y ∈ Rn

2 such that ˆ

yj integer ∀j ∈ Jy, the following set S+(ˆ y) = {(x, y) ∈ Rn : dTy ≥ dT ˆ y, Ax + B ˆ y ≤ b + 1} is bilevel-feasible free.

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 34

SLIDE 70

Other Bilevel-Free Sets can be defined

Motivated by the results Xu [2012], Wang and Xu [2017]: Assumption: Ax + By − b is integer for all HPR solutions (x, y).

Theorem (Fischetti et al. [2017])

Given ∆ˆ y ∈ Rn

2 such that dT∆ˆ

y < 0 and ∆ˆ yj integer ∀j ∈ Jy, the following set X +(∆ˆ y) = {(x, y) ∈ Rn : Ax + By + B∆ˆ y ≤ b + 1} has no bilevel-feasible points in its interior. Proof: by contradiction. Assume (˜ x, ˜ y) ∈ X +(∆ˆ y) is bilevel-feasible. But then, dT ˜ y > dT(˜ y + ∆ˆ y) and (˜ x, ˜ y + ∆ˆ y) is feasible for the follower, hence contradiction.

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 35

SLIDE 71

SEPARATION of INTERSECTION CUTS

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 36

SLIDE 72

Separation of ICs associated to S+(ˆ y)

Given ˆ y ∈ Rn

2 such that ˆ

yj integer ∀j ∈ Jy, the following set S+(ˆ y) = {(x, y) ∈ Rn : dTy ≥ dT ˆ y, Ax + B ˆ y ≤ b + 1} is bilevel-feasible free. How to compute ˆ y?

SEP1

ˆ y ∈ arg min

y∈Rn2{dTy : By ≤ b − Ax∗,

yj integer ∀j ∈ Jy}.

◮ ˆ

y is the optimal solution of the follower when x = x∗.

◮ Maximize the distance of (x∗, y ∗) from the facet dTy ≥ dT ˆ

y of S(ˆ y).

SEP2 Alternatively, try to find ˆ

y such that some of the facets in Ax + bˆ y ≤ b can be removed (making thus S(ˆ y) larger!)

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 37

SLIDE 73

Separation of ICs associated to S+(ˆ y)

Given ˆ y ∈ Rn

2 such that ˆ

yj integer ∀j ∈ Jy, the following set S+(ˆ y) = {(x, y) ∈ Rn : dTy ≥ dT ˆ y, Ax + B ˆ y ≤ b + 1} is bilevel-feasible free. How to compute ˆ y?

SEP2 (Fischetti et al. [2018])

ˆ y ∈ arg min

m

i=1

wi dTy ≤ dTy ∗ − 1 By + s = b si + (Lmax

i

− L∗

i ) wi ≥ Lmax i

, ∀i = 1, . . . , m yj integer, ∀j ∈ Jy s free , w ∈ {0, 1}m where L∗

i :=

j∈Nx

Aijx∗

j

≤ Lmax

i

:=

j∈Nx

max{Aijx−

j , Aijx+ j }.

◮ wi = 0 if i-th facet of Ax + B ˆ

y ≤ b can be removed

◮ the number of “removable facets” is maximized → larger S+(ˆ

y).

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 37

SLIDE 74

Separation of ICs associated to X +(∆ˆ y)

Given ∆ˆ y ∈ Rn

2 such that dT∆ˆ

y < 0 and ∆ˆ yj integer ∀j ∈ Jy, the following set X +(∆ˆ y) = {(x, y) ∈ Rn : Ax + By + B∆ˆ y ≤ b + 1} has no bilevel-feasible points in its interior. How to compute ∆ˆ y?

XU (Xu [2012])

∆ˆ y ∈ arg min

˜ m

i=1

ti dT∆y ≤ −1 B∆y ≤ b − Ax∗ − By ∗ ∆yj integer, ∀j ∈ Jy B∆y ≤ t and t ≥ 0.

◮ variable ti has value 0 in case ( ˜

B∆y)i ≤ 0 (“removable facet”);

◮ “maximize the size” of the bilevel-feasible set associated with ∆ˆ

y.

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 38

SLIDE 75

COMPUTATIONAL STUDY

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 39

SLIDE 76

Settings

C, CPLEX 12.6.3, Intel Xeon E3-1220V2 3.1 GHz, four threads.

Class Source Type #Inst #OptB #Opt DENEGRE DeNegre [2011],Ralphs and Adams [2016] I 50 45 50 MIPLIB Fischetti et al. [2016] B 57 20 27 XUWANG Xu and Wang [2014] I,C 140 140 140 INTER-KP DeNegre [2011],Ralphs and Adams [2016] B 160 79 138 INTER-KP2 Tang et al. [2016] B 150 53 150 INTER-ASSIG DeNegre [2011],Ralphs and Adams [2016] B 25 25 25 INTER-RANDOM DeNegre [2011],Ralphs and Adams [2016] B 80

80

INTER-CLIQUE Tang et al. [2016] B 80 10 80 INTER-FIRE Baggio et al. [2016] B 72

72

total 814 372 762

#OptB = number of optimal solutions known before our work.
#Opt = number of optimal solutions known after our work.

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 40

SLIDE 77

Effects of FUB cuts

Speed-ups achieved by FUB cuts for the instance set DENEGRE.
25

50 75 100 100 101 102 103

At least x times faster using FUB cuts #instances [%]

Setting

SEP1/SEP1+

SEP2/SEP2+

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 41

SLIDE 78

Effects of follower preprocessing

Speed-ups achieved using follower preprocessing.
25

50 75 100 100 101 102 103

At least x times faster when using preprocessing #instances [%]

Setting

SEP1+/SEP1++

SEP2+/SEP2++

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 42

SLIDE 79

Combining FUB cuts and follower preprocessing

Final gaps for settings SEP2 and SEP2++ for instance set MIPLIB, obtained

when the time-limit of one hour is reached.

40

60 80 100 25 50 75 100

%gap #instances [%]

Setting

SEP2++

SEP2

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 43

SLIDE 80

Effects of different ICs

MIX++: combination of settings SEP2++ and XU++ (both ICs being separated

at each separation call).

Performance profile on the subsets of (bilevel and interdiction) instances that

could be solved to optimality by all three settings within the given time-limit

f one hour.
● ●
25

50 75 100 1 2 5 10

No more than x times worse than best configuration #instances [%]

Setting

SEP2++

XU++ MIX++

●
25

50 75 100 1 10 100 1000

No more than x times worse than best configuration #instances [%]

Setting

SEP2++

XU++ MIX++

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 44

SLIDE 81

Comparison with the literature (1)

Results for the instance set XUWANG

MIX++ Xu and Wang [2014] n1 i = 1 i = 2 i = 3 i = 4 i = 5 i = 6 i = 7 i = 8 i = 9 i = 10 avg avg 10 3 3 3 3 2 3 2 3 2 3 2.6 1.4 60 2 1 1 1 1 1 2 2 0.9 45.6 110 2 1 2 2 1 2 1 2 2 12 2.8 111.9 160 2 2 3 2 3 1 4 1 1 3 2.1 177.9 210 2 3 1 1 3 3 3 2 5 3 2.6 1224.5 260 3 4 3 6 3 5 6 2 7 11 5.0 1006.7 310 5 10 11 14 7 16 15 8 5 3 9.4 4379.3 360 17 28 11 13 11 15 7 19 9 14 14.4 2972.4 410 19 10 29 8 21 10 9 15 23 42 18.7 4314.2 460 22 10 22 35 21 21 32 22 23 23 23.1 6581.4 B1-110 1 1 9 1.3 132.3 B1-160 1 1 3 1 2 1 3 2 1.3 184.4 B2-110 16 2 2 8 1 25 15 5 1 122 19.7 4379.8 B2-160 8 38 21 91 34 4 40 3 12 123 37.4 22999.7

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 45

SLIDE 82

Comparison with the literature (2)

Results for the instance sets INTER-KP2 (left) and INTER-CLIQUE (right)

MIX++ Tang et al. [2016] n1 k t[s] t[s] #unsol 20 5 5.4 721.4 20 10 1.7 2992.6 3 20 15 0.2 129.5 22 6 10.3 1281.2 6 22 11 2.3 3601.8 10 22 17 0.2 248.2 25 7 33.6 3601.4 10 25 13 8.0 3602.3 10 25 19 0.4 1174.6 28 7 97.9 3601.0 10 28 14 22.6 3602.5 10 28 21 0.5 3496.9 8 30 8 303.0 3601.0 10 30 15 31.8 3602.3 10 30 23 0.6 3604.5 10 MIX++ Tang et al. [2016] ν d t[s] t[s] #unsol 8 0.7 0.1 373.0 8 0.9 0.2 3600.0 10 10 0.7 0.3 3600.1 10 10 0.9 0.7 3600.2 10 12 0.7 0.8 3600.3 10 12 0.9 1.9 3600.4 10 15 0.7 2.2 3600.3 10 15 0.9 12.6 3600.2 10

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 46

SLIDE 83

Conclusions

We presented an enhanced branch-and-cut algorithm, based on

◮ follower preprocessing; ◮ new locally-valid cuts; ◮ new separation procedures for ICs.

Detailed computational analysis (available on the paper) shows that

◮ both preprocessing and FUB cuts can have a large impact on branch-and-cut

performance;

◮ the new algorithm outperforms previous methods from the literature (including

ur original branch-and-cut) by a large margin.
Byproduct: the optimal solution for more than 300 instances previously

unsolved instances from literature is now available.

Code is publicly available:

https://msinnl.github.io/pages/bilevel.html

Thanks for Your Attention! Questions?

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 47

SLIDE 84

Conclusions

We presented an enhanced branch-and-cut algorithm, based on

◮ follower preprocessing; ◮ new locally-valid cuts; ◮ new separation procedures for ICs.

Detailed computational analysis (available on the paper) shows that

◮ both preprocessing and FUB cuts can have a large impact on branch-and-cut

performance;

◮ the new algorithm outperforms previous methods from the literature (including

ur original branch-and-cut) by a large margin.
Byproduct: the optimal solution for more than 300 instances previously

unsolved instances from literature is now available.

Code is publicly available:

https://msinnl.github.io/pages/bilevel.html

Thanks for Your Attention! Questions?

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 47

SLIDE 85

Conclusions

We presented an enhanced branch-and-cut algorithm, based on

◮ follower preprocessing; ◮ new locally-valid cuts; ◮ new separation procedures for ICs.

Detailed computational analysis (available on the paper) shows that

◮ both preprocessing and FUB cuts can have a large impact on branch-and-cut

performance;

◮ the new algorithm outperforms previous methods from the literature (including

ur original branch-and-cut) by a large margin.
Byproduct: the optimal solution for more than 300 instances previously

unsolved instances from literature is now available.

Code is publicly available:

https://msinnl.github.io/pages/bilevel.html

Thanks for Your Attention! Questions?

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 47

SLIDE 86

Conclusions

We presented an enhanced branch-and-cut algorithm, based on

◮ follower preprocessing; ◮ new locally-valid cuts; ◮ new separation procedures for ICs.

Detailed computational analysis (available on the paper) shows that

◮ both preprocessing and FUB cuts can have a large impact on branch-and-cut

performance;

◮ the new algorithm outperforms previous methods from the literature (including

ur original branch-and-cut) by a large margin.
Byproduct: the optimal solution for more than 300 instances previously

unsolved instances from literature is now available.

Code is publicly available:

https://msinnl.github.io/pages/bilevel.html

Thanks for Your Attention! Questions?

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 47

SLIDE 87

Literature I

C. Audet, P. Hansen, B. Jaumard, and G. Savard. Links between linear bilevel and

mixed 0–1 programming problems. Journal of Optimization Theory and Applications, 93(2):273–300, 1997.

A. Baggio, M. Carvalho, A. Lodi, and A. Tramontani. Multilevel approaches for

the critical node problem. Working paper. ´ Ecole Polytechnique de Montreal, 2016.

E. Balas. Intersection cuts–a new type of cutting planes for integer programming.

Operations Research, 19(1):19–39, 1971.

S. DeNegre. Interdiction and Discrete Bilevel Linear Programming. PhD

thesis, Lehigh University, 2011.

M. Fischetti, I. Ljubic, M. Monaci, and M. Sinnl. Intersection cuts for bilevel
ptimization. In IPCO Proceedings, LNCS. Springer, 2016.
M. Fischetti, I. Ljubi´

c, M. Monaci, and M. Sinnl. A new general-purpose algorithm for mixed-integer bilevel linear programs. Operations Research, 65 (60):1615–1637, 2017.

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 48

SLIDE 88

Literature II

M. Fischetti, I. Ljubi´

c, M. Monaci, and M. Sinnl. On the use of intersection cuts for bilevel optimization. Mathematical Programming, 2018. doi: 10.1007/s10107-017-1189-5.

P. Hansen, B. Jaumard, and G. Savard. New branch-and-bound rules for linear

bilevel programming. SIAM Journal on Scientific and Statistical Computing, 13(5):1194–1217, 1992.

M. K¨
ppe, M. Queyranne, and C. T. Ryan. Parametric integer programming

algorithm for bilevel mixed integer programs. Journal of Optimization Theory and Applications, 146(1):137–150, 2010.

A. Lodi, T. K. Ralphs, and G. J. Woeginger. Bilevel programming and the

separation problem. Math. Program., 146(1-2):437–458, 2014.

J. Moore and J. Bard. The mixed integer linear bilevel programming problem.

Operations Research, 38(5):911–921, 1990.

T. K. Ralphs and E. Adams. Bilevel instance library, 2016.

http://coral.ise.lehigh.edu/data-sets/bilevel-instances/.

Y. Tang, J.-P. P. Richard, and J. C. Smith. A class of algorithms for mixed-integer

bilevel min–max optimization. Journal of Global Optimization, 66(2): 225–262, 2016.

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 49

SLIDE 89

Literature III

L. Wang and P. Xu. The watermelon algorithm for the bilevel integer linear

programming problem. SIAM Journal on Optimization, 27(3):1403–1430, 2017.

P. Xu. Three essays on bilevel optimization algorithmsand applications.

PhD thesis, Iowa State University, 2012.

P. Xu and L. Wang. An exact algorithm for the bilevel mixed integer linear

programming problem under three simplifying assumptions. Computers & Operations Research, 41:309–318, 2014.

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 50

SLIDE 90

Hypercube Intersection Cuts

Simple polyhedron that can be used to generate IC even when Ax + By − b is

NOT integer.

Theorem

Assume JF := {j ∈ Nx : Aj = 0} ⊆ Jx and let (ˆ x, ˆ y) an optimal bilevel-feasible solution with ˆ xj = x∗

j ∀j ∈ JF (if any). Then the following hypercube

HC +(x∗) = {(x, y) ∈ Rn : x∗

j − 1 ≤ xj ≤ x∗ j + 1, ∀j ∈ JF}

does not contain any bilevel-feasible solution (or any bilevel-feasible solution strictly better than (ˆ x, ˆ y), if the latter is defined) in its interior.

Idea: the interior of HC +(x∗) only contains bilevel-feasible solutions (x, y)

with xj = ˆ xj = x∗

j

∀j ∈ JF

Ivana Ljubi´ c (ESSEC) Exact General-Purpose Solvers for MIBLPs JFRO 2018, March 26, Paris 51