Interdiction Branching Andrea Lodi 1 , Ted Ralphs 2 , Fabrizio Rossi - - PowerPoint PPT Presentation

Interdiction Branching

Andrea Lodi1, Ted Ralphs2, Fabrizio Rossi3, Stefano Smriglio3

1DEIS, University of Bologna 2COR@L Lab, Department of Industrial and Systems Engineering, Lehigh University 3Dipartimento di Informatica, University of L’Aquila Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 1 / 17

Outline

1

Branching Methods in MILP

2

The 0-1 IP Case and Multi-variable Branching

3

Interdiction Branching by Bilevel Programming

4

Computational Experiments

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 2 / 17

Branching Methods

Definition 1 Branching is a method of partitioning of the feasible region of a mathematical program by means of a logical disjunction. Definition 2 A (linear) disjunction is a logical operator consisting of a finite set of systems of inequalities that evaluates TRUE with respect to a given ˜ x ∈ Rn if and only if at least one of the systems is satisfied by ˜ x. Specifically, a disjunction is a logical operator of the form

• h∈Q

Ahx ≥ bh, x ∈ S (1) where Ah ∈ Qmh×n, bh ∈ Qmh, n ∈ N, mh ∈ N, h ∈ Q. The disjunction evaluates TRUE for ˜ x if and only if there exists h ∈ Q such that Ah˜ x ≥ bh.

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 3 / 17

The Branching Decision

In branch and bound, branching creates one new subproblem for each term in the branching disjunction. Each resulting subproblem is solved recursively. Key Question: How should we select a disjunction?

Typically, the set of disjunctions to be considered is limited a priori in some fashion. From this limited set, one must choose the “best” disjunction by a given measure.

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 4 / 17

The Branching Decision

In branch and bound, branching creates one new subproblem for each term in the branching disjunction. Each resulting subproblem is solved recursively. Key Question: How should we select a disjunction?

Typically, the set of disjunctions to be considered is limited a priori in some fashion. From this limited set, one must choose the “best” disjunction by a given measure.

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 4 / 17

Selecting a Disjunction and Bilevel Programming

The overall goal of any branching scheme is to reduce running time. As a proxy, most branching schemes try to minimize the (estimated) bound decrease resulting from imposing the disjunction. The problem of selecting the disjunction whose imposition results in the largest bound improvement has a natural bilevel structure. This comes from the fact that the bound is computed by solving another

• ptimization problem: roughly speaking, one is solving an optimization problem
• ver the set of optimal solutions of another optimization problem.

The disjunction selection problem can sometimes be formulated as a bilevel program:

The upper-level variables can be used to model the choice of disjunction (we’ll see an example shortly). The lower-level problem models the bound computation after the disjunction has been imposed.

In strong branching, we are essentially solving the bilevel program by enumeration while interdiction branching is to select the branching disjunction directly by solving a bilevel program.

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 5 / 17

Selecting a Disjunction and Bilevel Programming

The overall goal of any branching scheme is to reduce running time. As a proxy, most branching schemes try to minimize the (estimated) bound decrease resulting from imposing the disjunction. The problem of selecting the disjunction whose imposition results in the largest bound improvement has a natural bilevel structure. This comes from the fact that the bound is computed by solving another

• ptimization problem: roughly speaking, one is solving an optimization problem
• ver the set of optimal solutions of another optimization problem.

The disjunction selection problem can sometimes be formulated as a bilevel program:

The upper-level variables can be used to model the choice of disjunction (we’ll see an example shortly). The lower-level problem models the bound computation after the disjunction has been imposed.

In strong branching, we are essentially solving the bilevel program by enumeration while interdiction branching is to select the branching disjunction directly by solving a bilevel program.

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 5 / 17

Selecting a Disjunction and Bilevel Programming

The overall goal of any branching scheme is to reduce running time. As a proxy, most branching schemes try to minimize the (estimated) bound decrease resulting from imposing the disjunction. The problem of selecting the disjunction whose imposition results in the largest bound improvement has a natural bilevel structure. This comes from the fact that the bound is computed by solving another

• ptimization problem: roughly speaking, one is solving an optimization problem
• ver the set of optimal solutions of another optimization problem.

The disjunction selection problem can sometimes be formulated as a bilevel program:

The upper-level variables can be used to model the choice of disjunction (we’ll see an example shortly). The lower-level problem models the bound computation after the disjunction has been imposed.

In strong branching, we are essentially solving the bilevel program by enumeration while interdiction branching is to select the branching disjunction directly by solving a bilevel program.

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 5 / 17

Selecting a Disjunction and Bilevel Programming

The overall goal of any branching scheme is to reduce running time. As a proxy, most branching schemes try to minimize the (estimated) bound decrease resulting from imposing the disjunction. The problem of selecting the disjunction whose imposition results in the largest bound improvement has a natural bilevel structure. This comes from the fact that the bound is computed by solving another

• ptimization problem: roughly speaking, one is solving an optimization problem
• ver the set of optimal solutions of another optimization problem.

The disjunction selection problem can sometimes be formulated as a bilevel program:

The upper-level variables can be used to model the choice of disjunction (we’ll see an example shortly). The lower-level problem models the bound computation after the disjunction has been imposed.

In strong branching, we are essentially solving the bilevel program by enumeration while interdiction branching is to select the branching disjunction directly by solving a bilevel program.

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 5 / 17

Issues on Variable Branching

For certain combinatorial problems, branching on single variables can result in very unbalanced trees. Consider the knapsack or set-partitioning problems, for instance.

Fixing a variable to 1 is typically very strong. Fixing a variable to zero can have little or not effect for difficult instances.

Often, this phenomenon is caused by symmetry or near-symmetry of the variables. Fixing a single variable to zero has no effect because there will be another (symmetric) variable to take its place. A number of authors have proposed methods specific to certain combinatorial problems, see, e.g., Ryan & Foster (1981), Balas & Yu (1986). There have also been attempts to derive general methods of multi-variable branching, e.g., SOS branching. Of course, one can try to use more general two-term disjunctions (split disjunctions) but in this talk we are concerned with multi-variable branching.

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 6 / 17

Issues on Variable Branching

For certain combinatorial problems, branching on single variables can result in very unbalanced trees. Consider the knapsack or set-partitioning problems, for instance.

Fixing a variable to 1 is typically very strong. Fixing a variable to zero can have little or not effect for difficult instances.

Often, this phenomenon is caused by symmetry or near-symmetry of the variables. Fixing a single variable to zero has no effect because there will be another (symmetric) variable to take its place. A number of authors have proposed methods specific to certain combinatorial problems, see, e.g., Ryan & Foster (1981), Balas & Yu (1986). There have also been attempts to derive general methods of multi-variable branching, e.g., SOS branching. Of course, one can try to use more general two-term disjunctions (split disjunctions) but in this talk we are concerned with multi-variable branching.

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 6 / 17

Branching Sets

Consider a binary integer program max{cTx | x ∈ P ∩ Bn}, where c ∈ Qn is the objective function and P is a polyhedron. For any set S = {i1, . . . , i|S|} ⊆ In = {1, . . . , n}, the disjunction xi1 = 1 ∨ xi2 = 1 ∨ . . . ∨ xi|S| = 1 ∨

• i∈S

xi = 0 is trivially valid. Such a disjunction can be easily strengthened as xi1 = 1∨(xi2 = 1∧xi1 = 0)∨. . .∨(xi|S| = 1∧xi1 = 0∧. . . ∧xi|S|−1 = 0)∨

• i∈S

xi = 0 (2)

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 7 / 17

Branching Sets

Consider a binary integer program max{cTx | x ∈ P ∩ Bn}, where c ∈ Qn is the objective function and P is a polyhedron. For any set S = {i1, . . . , i|S|} ⊆ In = {1, . . . , n}, the disjunction xi1 = 1 ∨ xi2 = 1 ∨ . . . ∨ xi|S| = 1 ∨

• i∈S

xi = 0 is trivially valid. Such a disjunction can be easily strengthened as xi1 = 1∨(xi2 = 1∧xi1 = 0)∨. . .∨(xi|S| = 1∧xi1 = 0∧. . . ∧xi|S|−1 = 0)∨

• i∈S

xi = 0 (2)

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 7 / 17

Branching Sets

Consider a binary integer program max{cTx | x ∈ P ∩ Bn}, where c ∈ Qn is the objective function and P is a polyhedron. For any set S = {i1, . . . , i|S|} ⊆ In = {1, . . . , n}, the disjunction xi1 = 1 ∨ xi2 = 1 ∨ . . . ∨ xi|S| = 1 ∨

• i∈S

xi = 0 is trivially valid. Such a disjunction can be easily strengthened as xi1 = 1∨(xi2 = 1∧xi1 = 0)∨. . .∨(xi|S| = 1∧xi1 = 0∧. . . ∧xi|S|−1 = 0)∨

• i∈S

xi = 0 (2)

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 7 / 17

Improving Solution Covers

Of course, we are not interested in any set S, but sets of variables playing a fundamental role for improving the current incumbent solution. More precisely,

we say that a variable index i covers a feasible solution ˆ x if ˆ xi = 1, and we denote as F(a,¯ z) the set of improving solutions at a any node a with respect to the incumbent value ¯ z.

Definition 3 An improving solution cover at node a with respect to ¯ z is an index set S(a,¯ z) = {i1, . . . , i|S(a,¯

z)|} ⊆ Na covering F(a,¯

z), where Na is the set of indices of undecided variables at node a. Note that an improving solution cover may not exist: either F(a,¯ z) = ∅ or the

• nly improving solution has xk = 0 for all k ∈ Na.

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 8 / 17

Improving Solution Covers (cont.d)

Because the second condition above is easy to check, the right-most child node in disjunction (2) does not correspond to any improving solution (and can be disregarded) while Theorem 1 If S(a,¯ z) is a minimal improving solution cover, then each (other) term of the disjunction (2) is satisfied by at least one solution in F(a,¯ z) where an improving solution cover S(a,¯ z) is said to be minimal if S(a,¯ z) \ {i} is not an improving solution cover for any i ∈ S(a,¯ z). Such a theorem implies that we are only enumerating nodes which include improving solutions, i.e., a very strong result, whose corollary is a bound on the number of branch-and-bound nodes polynomial in |F(a0,¯ z)|. Of course, there is no “free lunch" and computing improving solution covers (and especially minimal ones) MUST be very difficult.

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 9 / 17

Improving Solution Covers and Bilevel Programming

Indeed, it is not difficult to characterize improving solution covers: Theorem 2 A nonempty index set S(a,¯ z) ⊆ Na is an improving solution cover for a with respect to ¯ z if and only if max

x∈{0,1}n{c⊤x | x ∈ F(a), xi = 0 for all i ∈ S(a,¯

z)} ≤ ¯ z. (3) and, especially for minimal ones, this requires solving the following Interdiction Branching Problem (IBP) (at node a): min

• i∈In

yi (4) s.t.

• i∈In

cixi ≤ ¯ z (5) y ∈ {0, 1}n (6) x ∈ arg max

x

cTx (7) s.t. xi + yi ≤ 1, i ∈ In (8) x ∈ F(a). (9)

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 10 / 17

Interdiction Branching on a 0-1 Knapsack Instance

max 19620x1 + 13407x2 + 10464x3 + 7521x4 + 16848x5 + 15876x6 + 4860x7 + 2592x8 s.t. 1680x1 + 1148x2 + 896x3 + 644x4 + 1456x5 + 1372x6 + 420x7 + 224x8 ≤ 2613 x ∈ {0, 1}8

C x2 = 1 B I x1 = 1 H x1 = 0 x5 = 1 x2 = 0 x3 = 1 A E x1 = 1 D G x5 = 1 F x5 = 0 x6 = 1 x1 = 0 x7 = 1 x2 = 0 x3 = 0 x4 = 1 S(0, 27141) = {2, 3, 4} S(A, 27141) = {1, 7} S(B, 29733) = {1, 5} S(D, 27141) = {5, 6}

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 11 / 17

Practically Solving IBP

As it is IBP (4)–(9) is too difficult to solve but two observations help in making the problem more tractable:

1

On the one side, if one gives up the requirement to compute a minimal improving solution cover, any set S satisfying (3), i.e., any feasible solution of (4)–(9), is a valid branching set.

2

On the other hand, one can relax the integrality requirement x ∈ F(a) in (9) with the weaker requirement that x satisfies just the linear constraints of the original problem. Such a problem is denoted as Interdiction Branching Linear Problem (IBLP).

The price to pay in both cases is minimality. This in turn implies the property that, after branching, the feasible region of each of the child nodes contains at least one improving solution is no longer guaranteed (recall Theorem 1). However, in the computational experiments we used a heuristic algorithm for IBLP with a strong improvement on the efficiency and without losing too much in terms of accuracy. As an example, on the previous 0-1 KP the heuristic for IBLP results in a branch-and-bound tree with 17 nodes, wrt the 10 nodes of the exact IBP case and the 71 nodes of a pure algorithm branching on variables.

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 12 / 17

Practically Solving IBP

As it is IBP (4)–(9) is too difficult to solve but two observations help in making the problem more tractable:

1

On the one side, if one gives up the requirement to compute a minimal improving solution cover, any set S satisfying (3), i.e., any feasible solution of (4)–(9), is a valid branching set.

2

On the other hand, one can relax the integrality requirement x ∈ F(a) in (9) with the weaker requirement that x satisfies just the linear constraints of the original problem. Such a problem is denoted as Interdiction Branching Linear Problem (IBLP).

The price to pay in both cases is minimality. This in turn implies the property that, after branching, the feasible region of each of the child nodes contains at least one improving solution is no longer guaranteed (recall Theorem 1). However, in the computational experiments we used a heuristic algorithm for IBLP with a strong improvement on the efficiency and without losing too much in terms of accuracy. As an example, on the previous 0-1 KP the heuristic for IBLP results in a branch-and-bound tree with 17 nodes, wrt the 10 nodes of the exact IBP case and the 71 nodes of a pure algorithm branching on variables.

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 12 / 17

Practically Solving IBP

As it is IBP (4)–(9) is too difficult to solve but two observations help in making the problem more tractable:

1

On the one side, if one gives up the requirement to compute a minimal improving solution cover, any set S satisfying (3), i.e., any feasible solution of (4)–(9), is a valid branching set.

2

On the other hand, one can relax the integrality requirement x ∈ F(a) in (9) with the weaker requirement that x satisfies just the linear constraints of the original problem. Such a problem is denoted as Interdiction Branching Linear Problem (IBLP).

The price to pay in both cases is minimality. This in turn implies the property that, after branching, the feasible region of each of the child nodes contains at least one improving solution is no longer guaranteed (recall Theorem 1). However, in the computational experiments we used a heuristic algorithm for IBLP with a strong improvement on the efficiency and without losing too much in terms of accuracy. As an example, on the previous 0-1 KP the heuristic for IBLP results in a branch-and-bound tree with 17 nodes, wrt the 10 nodes of the exact IBP case and the 71 nodes of a pure algorithm branching on variables.

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 12 / 17

Computational Experiments: Implementation

A knapsack solver, referred to here as KNAP, has been implemented in the tree search framework ALPS. The solver uses a pure branch-and-bound approach and can utilize either standard 0-1 or interdiction branching. Bounding is done using the Dantzig algorithm. Note that the branching is the most computationally intensive procedure. Therefore, we put the node back in the queue after bounding and only branch it when it is chosen again. This is only possible due to the generality of ALPS.

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 13 / 17

Computational Experiments: Setup

We generated difficult 0-1 KP instances of the classes:

Strongly correlated instances (SC) Strongly correlated spanner instances (SP)

according to Pisinger (2005). 20 instance each of size n = {50, 60, 70, 80, 90, 100, 110} Run on a 2.8 GHz Intel Quad Core with 4 GB of RAM under Linux. Time limit of 3,600 CPU seconds. Comparison setting:

We compare the interdiction branching (best version DF_w) with the standard 0-1 branching in KNAP (KNAP_01) and IBM-CPLEX 11.2 (Cplex). Because we are interested in comparing branching rules, we disabled the algorithmic features which could interfere with such an analysis. Specifically, we ran the algorithms without primal heuristic, cut generation and preprocessing. To visualize the results we use the performance profiles.

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 14 / 17

Computational Experiments: Comparisons

0.2 0.4 0.6 0.8 1 1 2 4 8 DF_w KNAP_01 Cplex 0.2 0.4 0.6 0.8 1 1 2 4 8 DF_w KNAP_01 Cplex

Figure: Performance profiles for SC instances: comparison with 0-1 branching

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 15 / 17

Computational Experiments: Comparisons (cont.d)

0.2 0.4 0.6 0.8 1 1 2 4 8 16 32 64 128 256 DF_w KNAP_01 Cplex 0.2 0.4 0.6 0.8 1 1 2 4 8 16 32 64 128 256 DF_w KNAP_01 Cplex

Figure: Performance profiles for SP instances: comparison with 0-1 branching

The comparison between interdiction branching and 0-1 branching in KNAP_01 is fair, since both algorithms are implemented in the same framework. This is not quite the case for Cplex in terms of computing time because the continuous relaxations are solved by KNAP combinatorially while Cplex uses the simplex algorithm.

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 16 / 17

Conclusions

We presented a simple branching rule whose main characteristics are:

Interdiction branching takes into account both the current incumbent value and the bound provided by the LP relaxation and can thus be seen as targeting improvements in both the upper and lower bounds. As in the traditional branching on variables, child subproblems are generated by imposing variable bounds, without introducing additional constraints. In all but one of the k children, at least two variables are fixed, often yielding a remarkable improvement in the bound provided by the LP relaxation.

Although the method has proven effective on 0-1 KP instances, it has to be confirmed that these performance gains can be realized in state-of-the-art solvers. However, it is easy to see how interdiction branching can be incorporated within any branch-and-cut framework, even in a hybrid manner, i.e., interleaved with standard variable branching. If you want to play with it, you can download the solver at www.coin-or.org

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 17 / 17

Conclusions

We presented a simple branching rule whose main characteristics are:

Interdiction branching takes into account both the current incumbent value and the bound provided by the LP relaxation and can thus be seen as targeting improvements in both the upper and lower bounds. As in the traditional branching on variables, child subproblems are generated by imposing variable bounds, without introducing additional constraints. In all but one of the k children, at least two variables are fixed, often yielding a remarkable improvement in the bound provided by the LP relaxation.

Although the method has proven effective on 0-1 KP instances, it has to be confirmed that these performance gains can be realized in state-of-the-art solvers. However, it is easy to see how interdiction branching can be incorporated within any branch-and-cut framework, even in a hybrid manner, i.e., interleaved with standard variable branching. If you want to play with it, you can download the solver at www.coin-or.org

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 17 / 17

Conclusions

We presented a simple branching rule whose main characteristics are:

Interdiction branching takes into account both the current incumbent value and the bound provided by the LP relaxation and can thus be seen as targeting improvements in both the upper and lower bounds. As in the traditional branching on variables, child subproblems are generated by imposing variable bounds, without introducing additional constraints. In all but one of the k children, at least two variables are fixed, often yielding a remarkable improvement in the bound provided by the LP relaxation.

Although the method has proven effective on 0-1 KP instances, it has to be confirmed that these performance gains can be realized in state-of-the-art solvers. However, it is easy to see how interdiction branching can be incorporated within any branch-and-cut framework, even in a hybrid manner, i.e., interleaved with standard variable branching. If you want to play with it, you can download the solver at www.coin-or.org

Lodi, Ralphs, Rossi & Smriglio () Interdiction Branching Aussois XIV, January 5, 2010 17 / 17