First-order representations for integer programming James Cussens, - PowerPoint PPT Presentation

First-order representations for integer programming James Cussens, University of York Vienna, 2014-07-17 James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 1 / 28

Outline A running example Introduction to (mixed) integer programming Solving MIPs Logical methods in integer programming Clausal cuts and resolution Prospects for first-order techniques James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 2 / 28

A running example The MAP problem A | B | A | D | B | C | C | D | - | - | ---- - | - | ---- - | - | ---- - | - | ---- 0 | 0 | 0.10 0 | 0 | 0.90 0 | 0 | 0.40 0 | 0 | 0.50 0 | 1 | 0.20 * 0 | 1 | 0.20 * 0 | 1 | 0.70 * 0 | 1 | 0.20 1 | 0 | 0.30 1 | 0 | 0.70 1 | 0 | 0.30 1 | 0 | 0.40 1 | 1 | 0.20 1 | 1 | 0.10 1 | 1 | 0.10 1 | 1 | 0.10 ◮ This is a four clique Markov network ◮ Which joint instantiation of A, B, C and D has maximal probability? James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 3 / 28

A running example Encoding the problem A | B | A | D | B | C | C | D | - | - | ---- - | - | ---- - | - | ---- - | - | ---- 0 | 0 | 0.10 0 | 0 | 0.90 0 | 0 | 0.40 0 | 0 | 0.50 0 | 1 | 0.20 * 0 | 1 | 0.20 * 0 | 1 | 0.70 * 0 | 1 | 0.20 1 | 0 | 0.30 1 | 0 | 0.70 1 | 0 | 0.30 1 | 0 | 0.40 1 | 1 | 0.20 1 | 1 | 0.10 1 | 1 | 0.10 1 | 1 | 0.10 ◮ Have 16 (‘weighted’) binary variables, one for each choice of clique variables instantiation. ◮ Have 4 binary variables, for the 4 instantiations of A, B, C and D. ◮ Variable and clique instantiations must match up. ◮ (A suboptimal encoding btw!) James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 4 / 28

Introduction to (mixed) integer programming A ZIMPL representation of the MAP problem for graphical models var x[CLIQUE_INSTS] binary; var y[VAR_INSTS] binary; maximize prob: sum <c,ci> in CLIQUE_INSTS: d[c,ci]*x[c,ci]; subto convex_clique: forall <c> in CLIQUES do sum <c,ci> in CLIQUE_INSTS: x[c,ci] == 1; subto convex_vars: forall <v> in VARS do sum <v,vi> in VAR_INSTS: y[v,vi] == 1; subto incidence: forall <c> in CLIQUES do forall <v> in VARS_IN[c] do forall <v,vi> in VAR_INSTS do sum <c,ci> in CLIQUE_INSTS with <c,ci> in CONSIS[v,vi] : x[c,ci] == y[v,vi]; James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 5 / 28

Introduction to (mixed) integer programming Defining relations extensionally This is for a bigger MAP problem . . . set VARS := { 0..439 }; set CLIQUES := { 0..859 }; set CLIQUE_INSTS := { <0,0>, <0,1>, <1,0>, <1,1>, <2,0>, ... set VARS_IN[CLIQUES] := <0> {420}, <1> {421}, <2> {422}, ... ... <578> {365,418,405}, <579> {385,419,405} James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 6 / 28

Introduction to (mixed) integer programming Standard ‘ground’ MIP representation Maximize prob: +2.2999999544 x#0#1 +2.2999999544 x#1#1 +2.2999999544 x#2#1 .. +1.40000000818 x#23#1 ... ... Subject to convex_clique_1: + x#0#1 + x#0#0 = 1 ... incidence_1071: - y#400#0 + x#478#6 + x#478#4 + x#478#2 + x#478#0 = 0 .... James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 7 / 28

Solving MIPs Solving the linear relaxation ◮ x = 4 , y = 2 is the optimal integer solution. ◮ x = 2 . 5 , y = 2 . 8 is the solution to the linear relaxation. ◮ Linear relaxation can be solved quickly, and provides an upper bound. James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 8 / 28

Solving MIPs Separating the LP solution with a cutting plane James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 9 / 28

Solving MIPs Separating the LP solution with a cutting plane James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 9 / 28

Solving MIPs Facets: Not all cuts are equal James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 10 / 28

Solving MIPs Facets: Not all cuts are equal James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 10 / 28

Solving MIPs Facets: Not all cuts are equal James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 10 / 28

Solving MIPs Branch-and-bound James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 11 / 28

Solving MIPs Branch-and-bound James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 11 / 28

Solving MIPs Branch and cut 1. Let x* be the LP solution. 2. If x* worse than incumbent then exit. 3. If there are valid inequalities not satisfied by x* add them and go to 1. Else if x* is integer-valued then the current problem is solved Else branch on a variable with non-integer value in x* to create two new sub-problems (propagate if possible) James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 12 / 28

Solving MIPs Column generation ◮ In the simplex algorithm for solving the linear relaxation one repeatedly moves from the current vertex to a neighbouring one with a better objective value. ◮ Algebraically this corresponds to choosing a variable with ‘reduced cost’ whose value is currently zero and making it positive (it ‘enters the basis’). ◮ In the column (i.e. variable) generation approach, a variable is not explicitly represented until it enters the basis. ◮ An algorithm (a ‘pricer’) is used to choose which variable(s) to create and put into the basis. ◮ Such an approach allows one to formulate problems with very many (implicitly defined) variables. James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 13 / 28

Logical methods in integer programming Recall our ZIMPL representation . . . var x[CLIQUE_INSTS] binary; var y[VAR_INSTS] binary; maximize prob: sum <c,ci> in CLIQUE_INSTS: d[c,ci]*x[c,ci]; subto convex_clique: forall <c> in CLIQUES do sum <c,ci> in CLIQUE_INSTS: x[c,ci] == 1; subto convex_vars: forall <v> in VARS do sum <v,vi> in VAR_INSTS: y[v,vi] == 1; subto incidence: forall <c> in CLIQUES do forall <v> in VARS_IN[c] do forall <v,vi> in VAR_INSTS do sum <c,ci> in CLIQUE_INSTS with <c,ci> in CONSIS[v,vi] : x[c,ci] == y[v,vi]; James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 14 / 28

Logical methods in integer programming A na¨ ıve first-order approach ◮ Suppose the sets CLIQUE INSTS , CLIQUES , VARS IN , etc were too big to ‘ground out’ before solving. ◮ Suppose we had a way of enumerating these sets, i.e. an algorithm for generating every ground instance of the the corresponding predicates and relations. ◮ A na¨ ıve column generation algorithm generates variables until it comes up with one with reduced cost. ◮ A na¨ ıve cutting plane algorithm generates (ground) linear inequalities until it comes up with one which separates the current LP relaxation. ◮ Can we exploit techniques from first-order logic to do better? James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 15 / 28

Logical methods in integer programming Some general points ◮ Often one wants to generate cutting planes (‘valid inequalities’) not in the original problem definition. ◮ The goal is to get a tight linear relaxation, so facets of the convex hull are ideal. ◮ So just spitting out ground instances of inequalities in the problem definition is typically not good enough. James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 16 / 28

Logical methods in integer programming Some general points ◮ Often one wants to generate cutting planes (‘valid inequalities’) not in the original problem definition. ◮ The goal is to get a tight linear relaxation, so facets of the convex hull are ideal. ◮ So just spitting out ground instances of inequalities in the problem definition is typically not good enough. ◮ Column generation and separation are dual problems. ◮ Note also that if the original problem is NP-hard then the separation problem will be also. James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 16 / 28

Clausal cuts and resolution Clausal constraints ◮ Continuing the the 4-variable, 4-clique MAP running example, let a 0 d 0 be the binary variable indicating that in the clique for { A , D } the instantiation A = 0 , D = 0 was chosen. ◮ The following 4 clauses are given in the problem definition. ¬ a 0 d 0 ∨ a 0 b 0 ∨ a 0 b 1 ¬ a 0 b 0 ∨ b 0 c 0 ∨ b 0 c 1 ¬ b 0 c 1 ∨ c 1 d 0 ∨ c 1 d 1 ¬ a 0 d 0 ∨ ¬ c 1 d 1 James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 17 / 28

Clausal cuts and resolution Clauses are linear inequalities These two representations are equivalent: ¬ a 0 d 0 ∨ a 0 b 0 ∨ a 0 b 1 ¬ a 0 b 0 ∨ b 0 c 0 ∨ b 0 c 1 ¬ b 0 c 1 ∨ c 1 d 0 ∨ c 1 d 1 ¬ a 0 d 0 ∨ ¬ c 1 d 1 (1 − a 0 d 0 ) + a 0 b 0 + a 0 b 1 ≥ 1 (1 − a 0 b 0 ) + b 0 c 0 + b 0 c 1 ≥ 1 (1 − b 0 c 1 ) + c 1 d 0 + c 1 d 1 ≥ 1 (1 − a 0 d 0 ) + (1 − c 1 d 1 ) ≥ 1 James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 18 / 28

Clausal cuts and resolution Logic versus arithmetic ◮ It is interesting to compare numerical inequalities to logic formulae . . . James Cussens, University of York 1st-order logic for IP Vienna, 2014-07-17 19 / 28