Some Observations on Boolean Logic and Optimization John Hooker - - PowerPoint PPT Presentation

Slide 1

Some Observations on Boolean Logic and Optimization

John Hooker Carnegie Mellon University January 2009

Slide 2

Outline

• Logic and cutting planes
• Logic of 0-1 inequalities
• Logic and linear programming
• Inference duality
• Constraint programming
• Good logic models

Slide 3

Logic and Cutting Planes

Slide 4

Theorem (Quine). The resolution method generates all prime implicates of a set of logical clauses.

Logic and cutting planes

Prime implicates = undominated implications.

Slide 5

Theorem (Quine). The resolution method generates all prime implicates of a set of logical clauses.

Logic and cutting planes

Prime implicates = undominated implications. This means that resolution is a complete inference method for clauses.

Slide 6

Theorem (Quine). The resolution method generates all prime implicates of a set of logical clauses.

Logic and cutting planes

Example

1 2 1 3 2 3

x x x x x x ∨ ∨ ∨

Slide 7

Theorem (Quine). The resolution method generates all prime implicates of a set of logical clauses.

Logic and cutting planes

Resolve on x1

1 2 1 3 2 3

x x x x x x ∨ ∨ ∨

Slide 8

Theorem (Quine). The resolution method generates all prime implicates of a set of logical clauses.

Logic and cutting planes

Resolve on x1

1 2 1 3 2 3

x x x x x x ∨ ∨ ∨

1 2 1 2 3 3 2 3

x x x x x x x x ∨ ∨ ∨ ∨

Slide 9

Theorem (Quine). The resolution method generates all prime implicates of a set of logical clauses.

Logic and cutting planes

1 2 1 3 2 3

x x x x x x ∨ ∨ ∨

1 2 1 2 3 3 2 3

x x x x x x x x ∨ ∨ ∨ ∨

Resolve on x2

Slide 10

Theorem (Quine). The resolution method generates all prime implicates of a set of logical clauses.

Logic and cutting planes

1 2 1 3 2 3

x x x x x x ∨ ∨ ∨

1 2 1 2 3 3 2 3

x x x x x x x x ∨ ∨ ∨ ∨

1 2 1 3 2 2 3 3 3

x x x x x x x x x ∨ ∨ ∨ ∨

Resolve on x2

Slide 11

Theorem (Quine). The resolution method generates all prime implicates of a set of logical clauses.

Logic and cutting planes

1 2 1 3 2 3

x x x x x x ∨ ∨ ∨

1 2 1 2 3 3 2 3

x x x x x x x x ∨ ∨ ∨ ∨

1 2 1 3 2 2 3 3 3

x x x x x x x x x ∨ ∨ ∨ ∨

Drop redundant clauses

Slide 12

Theorem (Quine). The resolution method generates all prime implicates of a set of logical clauses.

Logic and cutting planes

1 2 1 3 2 3

x x x x x x ∨ ∨ ∨

1 2 1 2 3 3 2 3

x x x x x x x x ∨ ∨ ∨ ∨

1 2 1 3 2 2 3 3 3

x x x x x x x x x ∨ ∨ ∨ ∨

1 2 3

x x x ∨

Drop redundant clauses

Slide 13

Theorem (Quine). The resolution method generates all prime implicates of a set of logical clauses.

Logic and cutting planes

1 2 3

x x x ∨

Prime implicates remain

1 2 1 3 2 3

x x x x x x ∨ ∨ ∨

1 2 1 2 3 3 2 3

x x x x x x x x ∨ ∨ ∨ ∨

1 2 1 3 2 2 3 3 3

x x x x x x x x x ∨ ∨ ∨ ∨

Slide 14

Logic and cutting planes

This might be regarded as the fundamental theorem of cutting plane theory. Theorem (Chvátal). Every cutting plane for a 0-1 system Ax ≥ b can be generated by repeatedly taking nonnegative linear combinations and rounding up.

Slide 15

Logic and cutting planes

This might be regarded as the fundamental theorem of cutting plane theory. A key step of the proof uses the resolution method. Theorem (Chvátal). Every cutting plane for a 0-1 system Ax ≥ b can be generated by repeatedly taking nonnegative linear combinations and rounding up.

Slide 16

Theorem (Chvátal). Every cutting plane for a 0-1 system Ax ≥ b can be generated by repeatedly taking nonnegative linear combinations and rounding up.

Logic and cutting planes

This might be regarded as the fundamental theorem of cutting plane theory. A key step of the proof uses the resolution method. This suggests there are deep connections between resolution and cutting planes.

Slide 17

A resolution step generates a rank 1 cut (i.e., a cut generated by

• ne step of Chvátal’s method).

Logic and cutting planes

1 2 1 3

x x x x ∨ ∨

Convert to 0-1 inequalities

1 2 1 3

(1 ) 1 (1 ) 1 x x x x + − ≥ − + ≥

Slide 18

A resolution step generates a rank 1 cut (i.e., a cut generated by

• ne step of Chvátal’s method).

Logic and cutting planes

1 2 1 3

x x x x ∨ ∨

Take linear combination

1 2 1 3 2 3

(1/ 2) ( (1 ) 1 (1 ) 1 (1 ) 1/ 2) (1/ 2) (1/ 2 ) x x x x x x + − ≥ − + ≥ − ≥ ≥

2 3

(1 ) 1/ 2 x x − + ≥

Slide 19

A resolution step generates a rank 1 cut (i.e., a cut generated by

• ne step of Chvátal’s method).

Logic and cutting planes

1 2 1 3

x x x x ∨ ∨

1 2 1 3 2 3

(1/ 2) ( (1 ) 1 (1 ) 1 (1 ) 1/ 2) (1/ 2) (1/ 2 ) x x x x x x + − ≥ − + ≥ − ≥ ≥

2 3 2 3

(1 ) 1/ 2 (1 ) 1 x x x x − + ≥ − + ≥

Round up

2 3

x x ∨

Resolvent

Slide 20

Theorem (JNH). Input resolution generates precisely those clauses that are rank 1 cuts.

Logic and cutting planes

1 2 3 1 3 2 3

x x x x x x x ∨ ∨ ∨

Result of input resolution Input resolution = use at least one of the original clauses to obtain each resolvent

Slide 21

Theorem (JNH). Input resolution generates precisely those clauses that are rank 1 cuts.

Logic and cutting planes

1 2 3 1 3 2 3

x x x x x x x ∨ ∨ ∨

1 2 1 3 2 3 2 3 3

(1/ 4) (1/ 4) ( (1 ) 1 (1 ) 1 1 (1 ) 1/ 4 1/ 2) (1/ 4) (1/ 4) x x x x x x x x x + − ≥ − + ≥ + ≥ − ≥ ≥ ≥ Result of input resolution Take linear combination

Slide 22

Theorem (JNH). Input resolution generates precisely those clauses that are rank 1 cuts.

Logic and cutting planes

1 2 3 1 3 2 3

x x x x x x x ∨ ∨ ∨

1 2 1 3 2 3 2 3 3 3

(1 ) 1 (1 ) 1 1 (1/ 4) (1/ 4) ( (1 ) 1/ 1/ 2) (1/ 4) (1/ 4 4 1 ) x x x x x x x x x x + − ≥ − + ≥ + ≥ − ≥ ≥ ≥ ≥ Result of input resolution Round up

Slide 23

By generating enough Chvátal cuts, we obtain the convex hull of the 0-1 solutions.

Logic and cutting planes

Slide 24

By generating enough Chvatal cuts, we obtain the convex hull of the 0-1 solutions.

Logic and cutting planes

Can we obtain the convex hull by generating resolvents?

Slide 25

By generating enough Chvatal cuts, we obtain the convex hull of the 0-1 solutions.

Logic and cutting planes

Can we obtain the convex hull by generating resolvents? That is, do the prime implicates define the convex hull?

Slide 26

By generating enough Chvatal cuts, we obtain the convex hull of the 0-1 solutions.

Logic and cutting planes

Can we obtain the convex hull by generating resolvents? That is, do the prime implicates define the convex hull? Not in general. They do, if and only if the underlying set covering problems define convex hulls.

Slide 27

Theorem (JNH). The prime implicates of a clause set define an integral polytope if and only if all maximal monotone subsets of the prime implicates define an integral polytope.

Logic and cutting planes

monotone = every variable has the same sign in all occurrences. A monotone subset of clauses is a set covering problem (after complementing negated variables).

Slide 28

Theorem (JNH). The prime implicates of a clause set define an integral polytope if and only if all maximal monotone subsets of the prime implicates define an integral polytope.

Logic and cutting planes

1 2 1 3 2 3

x x x x x x ∨ ∨ ∨

Prime implicates

Slide 29

Theorem (JNH). The prime implicates of a clause set define an integral polytope if and only if all maximal monotone subsets of the prime implicates define an integral polytope.

Logic and cutting planes

1 2 1 3 2 3

x x x x x x ∨ ∨ ∨

1 2 1 3

x x x x ∨ ∨

1 3 2 3

x x x x ∨ ∨

Prime implicates Maximal monotone subsets

Slide 30

Theorem (JNH). The prime implicates of a clause set define an integral polytope if and only if all maximal monotone subsets of the prime implicates define an integral polytope.

Logic and cutting planes

1 2 1 3 2 3

x x x x x x ∨ ∨ ∨

1 2 1 3

x x x x ∨ ∨

1 3 2 3

x x x x ∨ ∨

Prime implicates Maximal monotone subsets

1 2 1 3

1 1 x x x x + ≥ + ≥

1 3 2 3

1 (1 ) 1 x x x x + ≥ − + ≥ These systems define integral polytopes

Slide 31

Theorem (JNH). The prime implicates of a clause set define an integral polytope if and only if all maximal monotone subsets of the prime implicates define an integral polytope.

Logic and cutting planes

1 2 1 3 2 3

x x x x x x ∨ ∨ ∨

1 2 1 3

x x x x ∨ ∨

1 3 2 3

x x x x ∨ ∨

Prime implicates Maximal monotone subsets

1 2 1 3

1 1 x x x x + ≥ + ≥

1 3 2 3

1 (1 ) 1 x x x x + ≥ − + ≥ These systems define integral polytopes

1 2 1 3 2 3

1 1 (1 ) 1 x x x x x x + ≥ + ≥ − + ≥ Therefore this system defines an integral polytope

Slide 32

Theorem (JNH). The prime implicates of a clause set define an integral polytope if and only if all maximal monotone subsets of the prime implicates define an integral polytope. Generalized by Guenin, and by Nobili & Sassano.

Logic and cutting planes

Slide 33

Logic of 0-1 Inequalities

Slide 34

Logic of 0-1 inequalities

0-1 inequalities can be viewed as logical propositions.

Slide 35

Logic of 0-1 inequalities

0-1 inequalities can be viewed as logical propositions. Can the resolution algorithm be generalized to 0-1 inequalities?

Slide 36

Logic of 0-1 inequalities

0-1 inequalities can be viewed as logical propositions. Can the resolution algorithm be generalized to 0-1 inequalities?

• Yes. This results in a logical analog of Chvátal’s theorem.

Slide 37

Logic of 0-1 inequalities

0-1 inequalities can be viewed as logical propositions. Can the resolution algorithm be generalized to 0-1 inequalities?

• Yes. This results in a logical analog of Chvátal’s theorem.

Theorem (JNH). Classical resolution + diagonal summation generates all 0-1 prime implicates (up to logical equivalence).

Slide 38

Logic of 0-1 inequalities

Diagonal summation:

1 2 3 4 1 2 3 4 1 2 3 4 1 2 1 2 3 4 3

5 3 4 2 4 3 4 2 5 2 4 2 5 3 3 4 2 5 5 x x x x x x x x x x x x x x x x x x x + + + ≥ + + + ≥ + + + + + ≥ + + ≥ ≥ +

Each inequality is implied by an inequality in the set to which 0-1 resolution is implied. Diagonal sum

Slide 39

Logic of 0-1 inequalities

Diagonal summation:

1 2 3 4 1 2 3 4 1 3 4 1 2 4 1 2 3 2 3 4

5 3 4 2 3 4 2 5 4 2 5 3 4 4 2 2 5 3 5 x x x x x x x x x x x x x x x x x x x x + + + ≥ + + + ≥ + + + ≥ + ≥ + + + + + ≥

Each inequality is implied by an inequality in the set to which 0-1 resolution is implied. Diagonal sum

Slide 40

Logic and Linear Programming

Slide 41

Logic and linear programming

Theorem: A renamable Horn set of clauses is satisfiable if and

• nly if it has a unit refutation.

Horn = at most one positive literal per clause Renamable Horn = Horn after complementing some variables. Unit refutation = resolution proof of unsatisfiability in which at least

• ne parent of each resolvent is a unit clause.

Slide 42

Logic and linear programming

Theorem: A renamable Horn set of clauses is satisfiable if and

• nly if it has a unit refutation.

Horn = at most one positive literal per clause Renamable Horn = Horn after complementing some variables. Unit refutation = resolution proof of unsatisfiability in which at least

• ne parent of each resolvent is a unit clause.

1 1 2 1 3 1 2 3

x x x x x x x x ∨ ∨ ∨ ∨

Horn set

Slide 43

Logic and linear programming

Theorem: A renamable Horn set of clauses is satisfiable if and

• nly if it has a unit refutation.

Horn = at most one positive literal per clause Renamable Horn = Horn after complementing some variables. Unit refutation = resolution proof of unsatisfiability in which at least

• ne parent of each resolvent is a unit clause.

1 1 2 1 3 1 2 3

x x x x x x x x ∨ ∨ ∨ ∨

2 3 1 2 1 1 3 1

x x x x x x x x ∨ ∨ ∨ ∨

Horn set Unit resolution

Slide 44

Logic and linear programming

Theorem: A renamable Horn set of clauses is satisfiable if and

• nly if it has a unit refutation.

Horn = at most one positive literal per clause Renamable Horn = Horn after complementing some variables. Unit refutation = resolution proof of unsatisfiability in which at least

• ne parent of each resolvent is a unit clause.

1 1 2 1 3 1 2 3

x x x x x x x x ∨ ∨ ∨ ∨

2 3 1 2 1 1 3 1

x x x x x x x x ∨ ∨ ∨ ∨

1 1 3 1 2 1 2 3

x x x x x x x x ∨ ∨ ∨ ∨

Horn set Unit resolution Unit resolution

Slide 45

Logic and linear programming

We don’t know a necessary and sufficient condition for solubility by unit refutation. But we can identify sufficient conditions by generalizing Horn sets. For example, to extended Horn sets, which rely on a rounding property of linear programming.

Slide 46

Logic and linear programming

Theorem: A satisfiable Horn set can be solved by rounding down a solution of the linear programming relaxation.

Slide 47

Logic and linear programming

Theorem: A satisfiable Horn set can be solved by rounding down a solution of the linear programming relaxation.

1 1 2 3 2 3 1 2 3

x x x x x x x x x ∨ ∨ ∨ ∨ ∨

Horn set

Slide 48

Logic and linear programming

Theorem: A satisfiable Horn set can be solved by rounding down a solution of the linear programming relaxation.

1 1 2 3 2 3 1 2 3

x x x x x x x x x ∨ ∨ ∨ ∨ ∨

Horn set

1 1 2 3 2 3 1 2 3

1 (1 ) (1 ) 1 (1 ) (1 ) 1 (1 ) (1 ) 1 1

j

x x x x x x x x x x ≥ − + − + ≥ − + − ≥ − + + − ≥ ≤ ≤

LP relaxation

Slide 49

Logic and linear programming

Theorem: A satisfiable Horn set can be solved by rounding down a solution of the linear programming relaxation.

1 1 2 3 2 3 1 2 3

x x x x x x x x x ∨ ∨ ∨ ∨ ∨

Horn set

1 1 2 3 2 3 1 2 3

1 (1 ) (1 ) 1 (1 ) (1 ) 1 (1 ) (1 ) 1 1

j

x x x x x x x x x x ≥ − + − + ≥ − + − ≥ − + + − ≥ ≤ ≤

LP relaxation Solution: (x1,x2,x3) = (1,1/2,1/2)

Slide 50

Logic and linear programming

Theorem: A satisfiable Horn set can be solved by rounding down a solution of the linear programming relaxation.

1 1 2 3 2 3 1 2 3

x x x x x x x x x ∨ ∨ ∨ ∨ ∨

Horn set

1 1 2 3 2 3 1 2 3

1 (1 ) (1 ) 1 (1 ) (1 ) 1 (1 ) (1 ) 1 1

j

x x x x x x x x x x ≥ − + − + ≥ − + − ≥ − + + − ≥ ≤ ≤

LP relaxation Solution: (x1,x2,x3) = (1,1/2,1/2) Round down: (x1,x2,x3) = (1,0,0)

Slide 51

Logic and linear programming

To generalize this, we use the following: Theorem (Chandrasekaran): If Ax ≥ b has integral components and T is nonsingular such that:

• T and T−1 are integral
• Each row of T−1 contains at most one negative entry, namely −1
• Each row of AT−1 contains at most one negative entry, namely −1

Then if x solves Ax ≥ b, so does

1

T Tx

− 

  

Slide 52

Logic and linear programming

A clause has the extended star-chain property if it corresponds to a set of edge-disjoint flows into the root of an arborescence and a flow on one additional chain.

1 3 4 5 6 7

x x x x x x ∨ ∨ ∨ ∨ ∨

x1 x4 x2 x3 x5 x6 x7

Slide 53

Logic and linear programming

A clause set is extended Horn if there is an arborescence for which every clause in the set has the extended star-chain property.

1 3 4 5 6 7

x x x x x x ∨ ∨ ∨ ∨ ∨

x1 x4 x2 x3 x5 x6 x7

Slide 54

Logic and linear programming

Theorem (Chandru and JNH). A satisfiable extended Horn clause set can be solved by rounding a solution of the LP relaxation as shown:

1 3 4 5 6 7

x x x x x x ∨ ∨ ∨ ∨ ∨

x1 x4 x2 x3 x5 x6 x7 down down up

Slide 55

Logic and linear programming

• Corollary. A satisfiable extended Horn clause set can be solved by

assigning values as shown:

1 3 4 5 6 7

x x x x x x ∨ ∨ ∨ ∨ ∨

x1 x4 x2 x3 x5 x6 x7 1

Slide 56

Logic and linear programming

Theorem (Chandru and JNH). A renamable extended Horn clause is satisfiable if and only if it has no unit refutation.

Slide 57

Logic and linear programming

Theorem (Chandru and JNH). A renamable extended Horn clause is satisfiable if and only if it has no unit refutation. Theorem (Schlipf, Annexstein, Franco & Swaminathan). These results hold when then incoming chains are not edge disjoint. .

Slide 58

Logic and linear programming

Theorem (Chandru and JNH). A renamable extended Horn clause is satisfiable if and only if it has no unit refutation. Theorem (Schlipf, Annexstein, Franco & Swaminathan). These results hold when then incoming chains are not edge disjoint. Corollary (Schlipf, Annexstein, Franco & Swaminathan). A one-step lookahead algorithm solves a satisfiable extended Horn problem without knowledge of the arborescence.

Slide 59

Inference duality

Slide 60

Inference duality

Consider an optimization problem:

min ( ) f x x D ∈

• Constraint set

Variable domain

Slide 61

Inference duality

Consider an optimization problem:

min ( ) f x x D ∈

• Constraint set

Variable domain An inference dual is:

( )

max ( )

P

v f x v v P , ⇒ ≥ ∈ ∈ ℝ

• There is a proof P of f(x) ≥ v

from premises in Family of admissible proofs

Slide 62

Inference duality

Linear programming:

min cx Ax b x ≥ ≥

Inference dual is:

( ) ( )

max

P

v Ax b cx v v P , ≥ ⇒ ≥ ∈ ∈ ℝ

• Let Ax ≥ b ⇒ cx ≥ v when uAx ≥ ub

dominates cx ≥ v for some u ≥ 0. dominates = uA ≤ c and ub ≥ v

Slide 63

Inference duality

Linear programming:

min cx Ax b x ≥ ≥

Inference dual is:

( ) ( )

max

P

v Ax b cx v v P , ≥ ⇒ ≥ ∈ ∈ ℝ

• This becomes the classical

LP dual. Let Ax ≥ b ⇒ cx ≥ v when uAx ≥ ub dominates cx ≥ v for some u ≥ 0. dominates = uA ≤ c and ub ≥ v

Slide 64

Inference duality

Linear programming:

min cx Ax b x ≥ ≥

Inference dual is:

( ) ( )

max

P

v Ax b cx v v P , ≥ ⇒ ≥ ∈ ∈ ℝ

• This becomes the classical

LP dual. This is a strong dual because the inference method is complete (Farkas Lemma). Let Ax ≥ b ⇒ cx ≥ v when uAx ≥ ub dominates cx ≥ v for some u ≥ 0. dominates = uA ≤ c and ub ≥ v

Slide 65

Inference duality

General inequality constraints:

min ( ) ( ) f x g x x S ≥ ∈

Inference dual is:

( ) ( )

max ( ) ( )

P

v g x f x v v P , ≥ ⇒ ≥ ∈ ∈ ℝ

• Let g(x) ≥ 0 ⇒ f(x) ≥ v when ug(x) ≥ 0

implies f(x) ≥ v for some u ≥ 0. implies = all x ∈ S satisfying ug(x) ≥ 0 satisfy f(x) ≥ v.

Slide 66

Inference duality

General inequality constraints:

min ( ) ( ) f x g x x S ≥ ∈

Inference dual is:

( ) ( )

max ( ) ( )

P

v g x f x v v P , ≥ ⇒ ≥ ∈ ∈ ℝ

• This becomes the surrogate dual.

Let g(x) ≥ 0 ⇒ f(x) ≥ v when ug(x) ≥ 0 implies f(x) ≥ v for some u ≥ 0. implies = all x ∈ S satisfying ug(x) ≥ 0 satisfy f(x) ≥ v.

Slide 67

Inference duality

General inequality constraints:

min ( ) ( ) f x g x x S ≥ ∈

Inference dual is:

( ) ( )

max ( ) ( )

P

v g x f x v v P , ≥ ⇒ ≥ ∈ ∈ ℝ

• Let g(x) ≥ 0 ⇒ f(x) ≥ v when ug(x) ≥ 0

dominates f(x) ≥ v for some u ≥ 0.

Slide 68

Inference duality

This becomes the Lagrangean dual Let g(x) ≥ 0 ⇒ f(x) ≥ v when ug(x) ≥ 0 dominates f(x) ≥ v for some u ≥ 0. General inequality constraints:

min ( ) ( ) f x g x x S ≥ ∈

Inference dual is:

( ) ( )

max ( ) ( )

P

v g x f x v v P , ≥ ⇒ ≥ ∈ ∈ ℝ

Slide 69

Inference duality

Integer linear programming:

min cx Ax b x S ≥ ∈

Inference dual is: Let Ax ≥ b ⇒ cx ≥ v when h(Ax) ≥ h(b) dominates cx ≥ v for some subadditive and homogeneous function h.

( ) ( )

max

P

v Ax b cx v v P , ≥ ⇒ ≥ ∈ ∈ ℝ

Slide 70

Inference duality

Integer linear programming:

min cx Ax b x S ≥ ∈

Inference dual is:

( ) ( )

max

P

v Ax b cx v v P , ≥ ⇒ ≥ ∈ ∈ ℝ

• This becomes the subadditive dual.

Let Ax ≥ b ⇒ cx ≥ v when h(Ax) ≥ h(b) dominates cx ≥ v for some subadditive and homogeneous function h.

Slide 71

Inference duality

Integer linear programming:

min cx Ax b x S ≥ ∈

Inference dual is: This becomes the subadditive dual. This is a strong dual because the inference method is complete, due to Chvátal’s theorem. Appropriate Chvátal function is subadditive and can found by Gomory’s cutting plane method. Let Ax ≥ b ⇒ cx ≥ v when h(Ax) ≥ h(b) dominates cx ≥ v for some subadditive and homogeneous function h.

( ) ( )

max

P

v Ax b cx v v P , ≥ ⇒ ≥ ∈ ∈ ℝ

Slide 72

Inference duality

Inference duality permits a generalization of Benders decomposition.

Slide 73

Inference duality

Inference duality permits a generalization of Benders decomposition. In classical Benders, a Benders cut is a linear combination

• f the subproblem constraints using dual multipliers.

Slide 74

Inference duality

Inference duality permits a generalization of Benders decomposition. In classical Benders, a Benders cut is a linear combination

• f the subproblem constraints using dual multipliers.

The Benders cut rules out solutions of the master problem for which the proof of optimality in the subproblem is still valid.

Slide 75

Inference duality

Inference duality permits a generalization of Benders decomposition. In classical Benders, a Benders cut is a linear combination

• f the subproblem constraints using dual multipliers.

The Benders cut rules out solutions of the master problem for which the proof of optimality in the subproblem is still valid. For general optimization, a Benders cut does the same, but the proof of optimality is a solution of the general inference dual.

Slide 76

Inference duality

Inference duality permits a generalization of Benders decomposition. In classical Benders, a Benders cut is a linear combination

• f the subproblem constraints using dual multipliers.

The Benders cut rules out solutions of the master problem for which the proof of optimality in the subproblem is still valid. For general optimization, a Benders cut does the same, but the proof of optimality is a solution of the general inference dual. This has led to orders-of-magnitude speedups in solution of scheduling and other problems by logic-based Benders decomposition.

Slide 77

Constraint Programming

Slide 78

Constraint programming uses logical inference to reduce backtracking.

Constraint programming

Slide 79

Constraint programming uses logical inference to reduce backtracking. Inference takes the form of consistency maintenance.

Constraint programming

Slide 80

Constraint programming uses logical inference to reduce backtracking. Inference takes the form of consistency maintenance. A constraint set S containing variables x1, …, xn is k-consistent if

• for any subject of variables x1, …, xj, xj+1
• and any partial assignment (x1, …, xj) = (v1, …, vj) that violates no

constraint in S, there is a vj+1 such that (x1, …, xj+1) = (v1, …, vj+1) violates no constraint in S.

Constraint programming

Slide 81

Constraint programming uses logical inference to reduce backtracking. Inference takes the form of consistency maintenance. A constraint set S containing variables x1, …, xn is k-consistent if

• for any subject of variables x1, …, xj, xj+1
• and any partial assignment (x1, …, xj) = (v1, …, vj) that violates no

constraint in S, there is a vj+1 such that (x1, …, xj+1) = (v1, …, vj+1) violates no constraint in S. S is strongly k-consistent if it is j-consistent for j = 1, …, k.

Constraint programming

Slide 82

Theorem (Freuder). If constraint set S is strongly k-consistent, and its dependency graph has width less than k (with respect to the branching order), then S can be solved without backtracking.

Constraint programming

Slide 83

Theorem (Freuder). If constraint set S is strongly k-consistent, and its dependency graph has width less than k (with respect to the branching order), then S can be solved without backtracking.

Constraint programming

1 2 3 1 2 4 3 5 4 5 6

x x x x x x x x x x x ∨ ∨ ∨ ∨ ∨ ∨ ∨

x1 x2 x4 x3 x6 x5

Dependency graph

Slide 84

Theorem (Freuder). If constraint set S is strongly k-consistent, and its dependency graph has width less than k (with respect to the branching order), then S can be solved without backtracking.

Constraint programming

1 2 3 1 2 4 3 5 4 5 6

x x x x x x x x x x x ∨ ∨ ∨ ∨ ∨ ∨ ∨

x1 x2 x4 x3 x6 x5

Width = max in-degree = 2 Dependency graph

Slide 85

Theorem (Freuder). If constraint set S is strongly k-consistent, and its dependency graph has width less than k (with respect to the branching order), then S can be solved without backtracking.

Constraint programming

1 2 3 1 2 4 3 5 4 5 6

x x x x x x x x x x x ∨ ∨ ∨ ∨ ∨ ∨ ∨

x1 x2 x4 x3 x6 x5

Width = max in-degree = 2 Dependency graph We will show that this is strongly 3-consistent. We can therefore solve it without backtracking

Slide 86

Theorem (Freuder). If constraint set S is strongly k-consistent, and its dependency graph has width less than k (with respect to the branching order), then S can be solved without backtracking.

Constraint programming

1 2 3 1 2 4 3 5 4 5 6

x x x x x x x x x x x ∨ ∨ ∨ ∨ ∨ ∨ ∨

x1 x2 x4 x3 x6 x5

Width = max in-degree = 2 Dependency graph

1 2 3 4 5 6

x x x x x x

Slide 87

Theorem (Freuder). If constraint set S is strongly k-consistent, and its dependency graph has width less than k (with respect to the branching order), then S can be solved without backtracking.

Constraint programming

2 4 3 5 4 5 6

x x x x x x x ∨ ∨ ∨ ∨

x1 x2 x4 x3 x6 x5

Width = max in-degree = 2 Dependency graph

1 2 3 4 5 6

x x x x x x

Slide 88

Theorem (Freuder). If constraint set S is strongly k-consistent, and its dependency graph has width less than k (with respect to the branching order), then S can be solved without backtracking.

Constraint programming

2 4 3 5 4 5 6

x x x x x x x ∨ ∨ ∨ ∨

x1 x2 x4 x3 x6 x5

Width = max in-degree = 2 Dependency graph

1 2 3 4 5 6

x x x x x x

Slide 89

Theorem (Freuder). If constraint set S is strongly k-consistent, and its dependency graph has width less than k (with respect to the branching order), then S can be solved without backtracking.

Constraint programming

3 5 4 5 6

x x x x x ∨ ∨ ∨

x1 x2 x4 x3 x6 x5

Width = max in-degree = 2 Dependency graph

1 2 3 4 5 6

x x x x x x

Slide 90

Theorem (Freuder). If constraint set S is strongly k-consistent, and its dependency graph has width less than k (with respect to the branching order), then S can be solved without backtracking.

Constraint programming

3 5 4 5 6

x x x x x ∨ ∨ ∨

x1 x2 x4 x3 x6 x5

Width = max in-degree = 2 Dependency graph

1 2 3 4 5 6

x x x x x x

Slide 91

Theorem (Freuder). If constraint set S is strongly k-consistent, and its dependency graph has width less than k (with respect to the branching order), then S can be solved without backtracking.

Constraint programming

5 4 5 6

x x x x ∨ ∨

x1 x2 x4 x3 x6 x5

Width = max in-degree = 2 Dependency graph

1 2 3 4 5 6

x x x x x x

Slide 92

Theorem (Freuder). If constraint set S is strongly k-consistent, and its dependency graph has width less than k (with respect to the branching order), then S can be solved without backtracking.

Constraint programming

5 4 5 6

x x x x ∨ ∨

x1 x2 x4 x3 x6 x5

Width = max in-degree = 2 Dependency graph

1 2 3 4 5 6

x x x x x x

Slide 93

Theorem (Freuder). If constraint set S is strongly k-consistent, and its dependency graph has width less than k (with respect to the branching order), then S can be solved without backtracking.

Constraint programming

5 5 6

x x x ∨

x1 x2 x4 x3 x6 x5

Width = max in-degree = 2 Dependency graph

1 2 3 4 5 6

x x x x x x

Slide 94

Theorem (Freuder). If constraint set S is strongly k-consistent, and its dependency graph has width less than k (with respect to the branching order), then S can be solved without backtracking.

Constraint programming

5 5 6

x x x ∨

x1 x2 x4 x3 x6 x5

Width = max in-degree = 2 Dependency graph

1 2 3 4 5 6

1 x x x x x x

Slide 95

Theorem (Freuder). If constraint set S is strongly k-consistent, and its dependency graph has width less than k (with respect to the branching order), then S can be solved without backtracking.

Constraint programming

6

x

x1 x2 x4 x3 x6 x5

Width = max in-degree = 2 Dependency graph

1 2 3 4 5 6

1 x x x x x x

Slide 96

Theorem (Freuder). If constraint set S is strongly k-consistent, and its dependency graph has width less than k (with respect to the branching order), then S can be solved without backtracking.

Constraint programming

6

x

x1 x2 x4 x3 x6 x5

Width = max in-degree = 2 Dependency graph

1 2 3 4 5 6

1 1 x x x x x x

Slide 97

• Theorem. Application of k-resolution makes a clause set

strongly k-consistent. k-resolution = generate only resolvents with fewer than k literals.

Constraint programming

1 2 3 1 2 4 3 5 4 5 6

x x x x x x x x x x x ∨ ∨ ∨ ∨ ∨ ∨ ∨

All resolvents have 3 or more literals. Clause set is therefore strongly 3-consistent, as claimed.

Slide 98

Constraint programmers are primarily concerned with domain consistency.

Constraint programming

Slide 99

Constraint programmers are primarily concerned with domain consistency. A constraint set S is domain consistent if for any given variable xj and any value vj in its domain, xj = vj in some solution of S.

Constraint programming

Slide 100

Constraint programmers are primarily concerned with domain consistency. A constraint set S is domain consistent if for any given variable xj and any value vj in its domain, xj = vj in some solution of S. Domain consistency = generalized arc consistency = hyperarc consistency.

Constraint programming

Slide 101

Constraint programmers are primarily concerned with domain consistency. A constraint set S is domain consistent if for any given variable xj and any value vj in its domain, xj = vj in some solution of S. Domain consistency = generalized arc consistency = hyperarc consistency. Filtering algorithms that achieve or approximate domain consistency have been devised for a wide variety of constraints.

Constraint programming

Slide 102

Constraint programmers are primarily concerned with domain consistency. A constraint set S is domain consistent if for any given variable xj and any value vj in its domain, xj = vj in some solution of S. Domain consistency = generalized arc consistency = hyperarc consistency. Filtering algorithms that achieve or approximate domain consistency have been devised for a wide variety of constraints. The resolution algorithm achieves domain consistency for clause sets.

Constraint programming

Slide 103

Constraint programmers are primarily concerned with domain consistency. A constraint set S is domain consistent if for any given variable xj and any value vj in its domain, xj = vj in some solution of S. Domain consistency = generalized arc consistency = hyperarc consistency. Filtering algorithms that achieve or approximate domain consistency have been devised for a wide variety of constraints. The resolution algorithm achieves domain consistency for clause sets. Filtering (= logical inference) is the workhorse of constraint programming, as solving relaxations is the workhorse of integer programming.

Constraint programming

Slide 104

Good Logic Models

Slide 105

Boolean models should be reformulated before solution to achieve two goals:

Good logic models

Slide 106

Boolean models should be reformulated before solution to achieve two goals:

• A high degree of consistency (in the constraint programming

sense)

• We talked about resolution as a means of achieving

consistency.

Good logic models

Slide 107

Boolean models should be reformulated before solution to achieve two goals:

• A high degree of consistency (in the constraint programming

sense)

• We talked about resolution as a means of achieving

consistency for boolean models.

• A tight linear relaxation.
• We talked about logic and cutting planes.
• Logic constraints can also be given convex hull

formulations…

Good logic models

Slide 108

Example: cardinality rules We have 3 possible sites for factories and 3 possible products. Rule 1: If at least 2 plants are built, then at least 2 products should be made. Rule 2. Only 1 product should be made, unless plants are built at both sites 1 and 2.

Good logic models

1 2 3 1 2 3

( 2) ( 2) x x x y y y + + ≥ ⇒ + + ≥

1 2 3 1 2

( 2) ( 2) y y y x x + + ≥ ⇒ + ≥

Slide 109

Good logic models

1 2 3 1 2 3

( 2) ( 2) x x x y y y + + ≥ ⇒ + + ≥

Inequality form:

1 2 3 1 2 3 1 2 1 2 3 1 3 1 2 3 2 3 1 2 3 1 2 3 1 2 1 2 3 1 3 1 2 3 2 3

2( ) 2( ) 2 2( ) 2 2( ) 2 2( ) 2 2( ) 1 2( ) 1 2( ) 1 x x x y y y x x y y y x x y y y x x y y y x x x y y x x x y y x x x y y − + + + + + ≥ − − + + + + ≥ − − + + + + ≥ − − + + + + ≥ − − − − + + ≥ − − − − + + ≥ − − − − + + ≥ −

1 2 1 2 1 2 1 3 1 2 2 3 1 3 1 2 1 3 1 3 1 3 2 3 2 3 1 2 2 3 1 3 2 3 2 3

1 1 1 1 1 1 1 1 1 x x y y x x y y x x y y x x y y x x y y x x y y x x y y x x y y x x y y − − + + ≥ − − − + + ≥ − − − + + ≥ − − − + + ≥ − − − + + ≥ − − − + + ≥ − − − + + ≥ − − − + + ≥ − − − + + ≥ −

Slide 110

Good logic models

Inequality form:

1 2 3 1 1 2 1 1 3 1 2 3 1 1 2 3 2 1 2 1 1 3 1 2 3 1

2( ) 3 2( ) 1 2( ) 1 2( ) 1 2( ) 3 2( ) 1 2( ) 1 2( ) 1 y y y x y y x y y x y y x y y y x y y x y y x y y x − + + + ≥ − − + + ≥ − − + + ≥ − − + + ≥ − − + + + ≥ − − + + ≥ − − + + ≥ − − + + ≥ −

1 2 3 1 2

( 2) ( 2) y y y x x + + ≥ ⇒ + ≥

Slide 111

Good logic models

Theorem (Yan and JNH): These describe the convex hull of the feasible set. Generalized by Balas, Bockmayr, Pisaruk & Wolsey.