Bilevel Programming and the Separation Problem Andrea Lodi - PowerPoint PPT Presentation

Bilevel Programming and the Separation Problem Andrea Lodi University of Bologna, Italy andrea.lodi@unibo.it joint work with Ted K. Ralphs and Gerhard J. Woeginger January 9, 2012 @ Aussois A. Lodi, Bilevel Programming and the Separation Problem

Context • We consider a general Mixed Integer Linear Program (MIP) in the form min { c T x : Ax ≥ b, x ≥ 0 , x j integer , j ∈ I} (1) and we do not assume matrix A having any special structure. 1 A. Lodi, Bilevel Programming and the Separation Problem

Context • We consider a general Mixed Integer Linear Program (MIP) in the form min { c T x : Ax ≥ b, x ≥ 0 , x j integer , j ∈ I} (1) and we do not assume matrix A having any special structure. • We are considering a solution method based on branch and bound and bounds computed by iteratively solving the LP relaxations through a general-purpose LP solver. 1 A. Lodi, Bilevel Programming and the Separation Problem

Context • We consider a general Mixed Integer Linear Program (MIP) in the form min { c T x : Ax ≥ b, x ≥ 0 , x j integer , j ∈ I} (1) and we do not assume matrix A having any special structure. • We are considering a solution method based on branch and bound and bounds computed by iteratively solving the LP relaxations through a general-purpose LP solver. • We are interested in one of the major components of MIP technology, namely cutting plane generation, where the LP relaxation at hand is iteratively strengthened through the addition of valid (linear) inequalities. 1 A. Lodi, Bilevel Programming and the Separation Problem

Context • We consider a general Mixed Integer Linear Program (MIP) in the form min { c T x : Ax ≥ b, x ≥ 0 , x j integer , j ∈ I} (1) and we do not assume matrix A having any special structure. • We are considering a solution method based on branch and bound and bounds computed by iteratively solving the LP relaxations through a general-purpose LP solver. • We are interested in one of the major components of MIP technology, namely cutting plane generation, where the LP relaxation at hand is iteratively strengthened through the addition of valid (linear) inequalities. • In this talk we discuss the relationship between bilevel programming and cutting plane generation. 1 A. Lodi, Bilevel Programming and the Separation Problem

Bilevel Programming • Multi-level programs are a framework devised by Candler, Norton and Townsley (1977, 1982) and derive from a problem of agricultural development which was analyzed by the World Bank. 2 A. Lodi, Bilevel Programming and the Separation Problem

Bilevel Programming • Multi-level programs are a framework devised by Candler, Norton and Townsley (1977, 1982) and derive from a problem of agricultural development which was analyzed by the World Bank. • In that setting, once economic policy makers set certain parameters of agricultural policy, farmers were viewed as then optimizing their criteria, which differed from that of the policy makers. • The problem, then, was to set the policy parameters to achieve an optimal effect from the policy perspective, after understanding how the farmers reacted to these parameters. 2 A. Lodi, Bilevel Programming and the Separation Problem

Bilevel Programming • Multi-level programs are a framework devised by Candler, Norton and Townsley (1977, 1982) and derive from a problem of agricultural development which was analyzed by the World Bank. • In that setting, once economic policy makers set certain parameters of agricultural policy, farmers were viewed as then optimizing their criteria, which differed from that of the policy makers. • The problem, then, was to set the policy parameters to achieve an optimal effect from the policy perspective, after understanding how the farmers reacted to these parameters. • Roughly speaking, in bilevel programming one is solving an optimization problem over the set of optimal solutions of another optimization problem. 2 A. Lodi, Bilevel Programming and the Separation Problem

Bilevel Programming • Multi-level programs are a framework devised by Candler, Norton and Townsley (1977, 1982) and derive from a problem of agricultural development which was analyzed by the World Bank. • In that setting, once economic policy makers set certain parameters of agricultural policy, farmers were viewed as then optimizing their criteria, which differed from that of the policy makers. • The problem, then, was to set the policy parameters to achieve an optimal effect from the policy perspective, after understanding how the farmers reacted to these parameters. • Roughly speaking, in bilevel programming one is solving an optimization problem over the set of optimal solutions of another optimization problem. • This was a bilevel game , but the more general problem of multiple levels was also defined. 2 A. Lodi, Bilevel Programming and the Separation Problem

The P -hierarchy • Informally, the P -hierarchy is a scheme for classifying multi-level and multi-stage decision problems that extends the classes P and NP to problems with multiple decision-makers (and multiple objectives, in the case of optimization models). • The set of problems on level zero, denoted as Σ p 0 , are those that can be solved in polynomial time, the class usually denoted as P . 3 A. Lodi, Bilevel Programming and the Separation Problem

The P -hierarchy • Informally, the P -hierarchy is a scheme for classifying multi-level and multi-stage decision problems that extends the classes P and NP to problems with multiple decision-makers (and multiple objectives, in the case of optimization models). • The set of problems on level zero, denoted as Σ p 0 , are those that can be solved in polynomial time, the class usually denoted as P . • Roughly speaking, the class of problems on level k ∈ N + , denoted as Σ p k , are those that can be solved in nondeterministic polynomial time, given an oracle for problems in the class Σ p k − 1 . This means that, for example, Σ p 1 = NP . • It is clear that Σ p j ⊆ Σ p k for all j, k ∈ N , j ≤ k , but it is unknown whether any of the inclusions are strict. 3 A. Lodi, Bilevel Programming and the Separation Problem

The P -hierarchy • Informally, the P -hierarchy is a scheme for classifying multi-level and multi-stage decision problems that extends the classes P and NP to problems with multiple decision-makers (and multiple objectives, in the case of optimization models). • The set of problems on level zero, denoted as Σ p 0 , are those that can be solved in polynomial time, the class usually denoted as P . • Roughly speaking, the class of problems on level k ∈ N + , denoted as Σ p k , are those that can be solved in nondeterministic polynomial time, given an oracle for problems in the class Σ p k − 1 . This means that, for example, Σ p 1 = NP . • It is clear that Σ p j ⊆ Σ p k for all j, k ∈ N , j ≤ k , but it is unknown whether any of the inclusions are strict. • The hierarchy was first introduced by Stockmeyer (1977), who showed how to generalize the well-known satisfiability problem to obtain, for every k ∈ N , a class of problems involving the satisfiability of Boolean formulas in a multi-round game that is complete for Σ p k . • Jeroslow (1985) noted the relationship between decision games and optimization and showed that k -level discrete optimization problems are Σ p k -hard. 3 A. Lodi, Bilevel Programming and the Separation Problem

Cutting Plane Generation • Most of the time in branch-and-cut algorithms we are interested in solving the so-called separation problem: x ∈ R n Definition 1. The separation problem for a polyhedron Q is to determine for a given ˆ β ) ∈ R n +1 valid for Q and for α, ¯ whether or not ˆ x ∈ Q and if not, to produce an inequality (¯ x < ¯ α ⊤ ˆ which ¯ β , where, most of the times, ˆ x is a feasible solution of the continuous relaxation of the MIP, i.e., the relaxation obtained by dropping the integrality requirement on the x j variables, j ∈ I . 4 A. Lodi, Bilevel Programming and the Separation Problem

Cutting Plane Generation • Most of the time in branch-and-cut algorithms we are interested in solving the so-called separation problem: x ∈ R n Definition 1. The separation problem for a polyhedron Q is to determine for a given ˆ β ) ∈ R n +1 valid for Q and for α, ¯ whether or not ˆ x ∈ Q and if not, to produce an inequality (¯ x < ¯ α ⊤ ˆ which ¯ β , where, most of the times, ˆ x is a feasible solution of the continuous relaxation of the MIP, i.e., the relaxation obtained by dropping the integrality requirement on the x j variables, j ∈ I . • Specifically, we will consider the case in which, among all possible ( α, β ) inequalities that cut off ˆ x , we want the one maximizing the violation, i.e.: α, ¯ β ) ∈ argmin ( α,β ) ∈ R n +1 { α ⊤ ˆ x − β | α ⊤ x ≥ β ∀ x ∈ Q} . (¯ (2) 4 A. Lodi, Bilevel Programming and the Separation Problem

Cutting Plane Generation (cont.d) • Because it might be too hard to find a “completely general” inequality in the form α ⊤ x ≥ β (cutting off ˆ x ), we often restrict ourselves to specific classes (i.e., subsets) of valid inequalities that share some structure. 5 A. Lodi, Bilevel Programming and the Separation Problem