Integer Programming and Intelligent Exhaustive Search : basics - - PowerPoint PPT Presentation

Integer Programming and Intelligent Exhaustive Search : basics

Rumen Andonov

Université de Rennes 1 et INRIA Rennes Bretagne-Atlantique

Contenu du cours

• 1. Integer linear programming (IP) : problem formulation
• 2. Divide and Conquer (D&C) approach for IP
• 3. Branch and Bound (B&B) approach for IP
• 4. Applications
• 5. Lagrangian relaxation

2/29 2/29

Divide and Conquer (D&C) approach

3/29 3/29

Problem formulation

An integer linear programming problem :

(IP) ZIP = max{cx | x ∈ S}, S = {x ∈ Z n

+ | Ax ≤ b},

(min can be found by maximizing −cx). where c,Am×n,b are with integral coefficients.

S is a finite set, but too large ⇒ it is difficult to

• ptimize directly on S

First idea : Splitting S into smaller subsets ! This leads to Divide and Conquer (D&C) approach.

4/29 4/29

Divide and Conquer (D&C) approach (suite)

Proposition 1

(IPi) Z i

IP = max{cx | x ∈ Si},

where {Si | i = 1,...,k} is a division of S. ZIP = maxk

i=1 Z i IP.

Optimizing IP can be done by :

• 1. splitting S into {Si | i = 1,...,k}.
• 2. Optimizing each corresponding IPi.
• 3. And putting the results together.

This process is done recursively if IPi cannot be solved directly.

5/29 5/29

D&C : Fathoming

In extreme case, splitting leads to totally enumerate S. To be efficient, D&C should avoid splitting S too much. When can we stop splitting ?

Proposition 2

Node IPi of the problem tree can be pruned (or fathomed, or killed) if any of these conditions holds

• 1. Infeasibility : Si does not contain feasible solution.
• 2. Optimality : An optimal solution Z i

IP is known.

6/29 6/29

Problem tree

D&C can be viewed as an exploration of a problem tree, where : The root is the original IP. Sons of a node IPi, are its subproblems IPi,j. Leafs are subproblems that

can be solved directly are infeasible.

For each node IPi which is not a leaf, Z i

IP is the max of Z i,j IP

7/29 7/29

Problem tree

D&C can be viewed as an exploration of a problem tree, where : The root is the original IP. Sons of a node IPi, are its subproblems IPi,j. Leafs are subproblems that

can be solved directly are infeasible.

For each node IPi which is not a leaf, Z i

IP is the max of Z i,j IP

8/29 8/29

Problem tree

D&C can be viewed as an exploration of a problem tree, where : The root is the original IP. Sons of a node IPi, are its subproblems IPi,j. Leafs are subproblems that

can be solved directly are infeasible.

For each node IPi which is not a leaf, Z i

IP is the max of Z i,j IP

9/29 9/29

Problem tree

D&C can be viewed as an exploration of a problem tree, where : The root is the original IP. Sons of a node IPi, are its subproblems IPi,j. Leafs are subproblems that

can be solved directly are infeasible.

For each node IPi which is not a leaf, Z i

IP is the max of Z i,j IP

10/29 10/29

Problem tree

D&C can be viewed as an exploration of a problem tree, where : The root is the original IP. Sons of a node IPi, are its subproblems IPi,j. Leafs are subproblems that

can be solved directly are infeasible.

For each node IPi which is not a leaf, Z i

IP is the max of Z i,j IP

11/29 11/29

Problem tree

D&C can be viewed as an exploration of a problem tree, where : The root is the original IP. Sons of a node IPi, are its subproblems IPi,j. Leafs are subproblems that

can be solved directly are infeasible.

For each node IPi which is not a leaf, Z i

IP is the max of Z i,j IP

12/29 12/29

Problem tree

D&C can be viewed as an exploration of a problem tree, where : The root is the original IP. Sons of a node IPi, are its subproblems IPi,j. Leafs are subproblems that

can be solved directly are infeasible.

For each node IPi which is not a leaf, Z i

IP is the max of Z i,j IP

13/29 13/29

Problem tree

D&C can be viewed as an exploration of a problem tree, where : The root is the original IP. Sons of a node IPi, are its subproblems IPi,j. Leafs are subproblems that

can be solved directly are infeasible.

For each node IPi which is not a leaf, Z i

IP is the max of Z i,j IP

14/29 14/29

Problem tree

D&C can be viewed as an exploration of a problem tree, where : The root is the original IP. Sons of a node IPi, are its subproblems IPi,j. Leafs are subproblems that

can be solved directly are infeasible.

For each node IPi which is not a leaf, Z i

IP is the max of Z i,j IP

15/29 15/29

Problem tree

D&C can be viewed as an exploration of a problem tree, where : The root is the original IP. Sons of a node IPi, are its subproblems IPi,j. Leafs are subproblems that

can be solved directly are infeasible.

For each node IPi which is not a leaf, Z i

IP is the max of Z i,j IP

Z IP=maxZ IP

1 , Z IP 2 

Z IP

2 =maxZ IP 21, Z IP 22

Z IP

21

Z IP

22

Z IP

1 =maxZ IP 11 , Z IP 12

∅ Z IP

12=maxZ IP 121 , Z IP 122

Z IP

121

Z IP

122

IP IP

1

IP

2

IP

11

IP

12

IP

121

IP

122

IP

21

IP

22

S S

2

S

1

S

11

S

12

S

21

S

22

S

121

S

122

16/29 16/29

D&C efficiency depends on :

The splitting strategy

Should avoid redundancy -> partitioning.

The ability to solve "big" subproblem.

Avoid enumerating S.

The ability to prove infeasibility

As soon as possible.

It does not depend on the exploration strategy.

Question : Can we make D&C more intelligent ?.

Yes : If we do fathoming more frequently. This leads to implicit enumeration also called Branch and Bound (B&B).

17/29 17/29

Branch and bound (B&B)

18/29 18/29

Branch and Bound (B&B)

An integer programming :

(IP) ZIP = max{cx | x ∈ S}, S = {x ∈ Z n

+ | Ax ≤ b},

(min can be found by maximizing −cx).

Idea behind the Branch and bound

Split S into smaller subsets but avoid visiting subproblems which cannot contain ZIP by using lower and upper bounds of the

• bjective function.

Notation

In the following, we will denote : ZIP an upper bound (UB) of ZIP.

I.e : ZIP ≥ ZIP.

ZIP a lower bound (LB) of ZIP.

I.e : ZIP ≤ ZIP.

19/29 19/29

Branch and bound

Branch and bound implies

• 1. A method to divide the search space.
• 2. A strategy for exploration of the problem tree
• 3. An algorithm to find an UB.
• 4. An algorithm to find a LB and the corresponding feasible

solution.

Branch and bound efficiency depends on :

All these elements !

20/29 20/29

How do we explore the problem tree ?

The exploration depends on the way the problems are stored in memory :

• 1. In a FIFO –> Breadth-First Search (BFS).
• 2. In a LIFO –> Deep-First Search (DFS).

Frequently used heuristic is –> Best First Search ! Problems are stored in the list

• rdered by their upper bound and the most promising

subproblem is selected. This requires priority queue implementation.

21/29 21/29

Proposition 3

A node IPi of the problem tree can be pruned (fathomed, killed) if any of the following conditions holds

• 1. Infeasibility : Its domain (Si) is empty.
• 2. Optimality : Z i

IP = Z i IP

• 3. Value dominance : Z i

IP ≤ ZIP = LB

22/29 22/29

Possible upper bounds generator : relaxation

A relaxation of IP is any maximization problem (RP) ZR = max{ZR(x) | x ∈ SR}, s.t. S ⊆ SR and ZR(x) ≥ cx,∀x ∈ S.

Proposition 4

The node IPi can be pruned if any of the following conditions holds :

• 1. Infeasibility : RPi is infeasible.
• 2. Optimality : Optimal solution xi

R to RPi satisfies xi R ∈ Si

(hence Z i

R = cxi).

• 3. Value dominance : Z i

R ≤ ZIP = LB, where ZIP is the value

• f some feasible solution of IP.

The relaxed problem should be easier to solve than the original !

23/29 23/29

General Branch and Bound Algorithm I

begin List of problems L = {IP},LB = −∞ while L = {/

0} do

Select and delete a problem IPi from L . Find Z i

IP and Z i IP

if (Z i

IP > LB) then LB ← Z i IP end if

if(Z i

IP ≤ LB) then Fathom IPi

else Divide Si into {Sij}k

j=1

Add {IPij}k

j=1 to L

end if end while x0 that yielded ZIP = cx0 is optimal. end

24/29 24/29

Branch and Bounds (B&B) : alternative

Subproblems are generated by partial solutions.

25/29 25/29

B&B for Traveling Salesman Problem (TSP)

1 A B C D E F G H 2 1 1 2 1 2 1 1 1 1 5 1 A B C D E F G H 1 1 1 1 1 1 1

FIGURE 1: A graph and its optimal traveling salesman tour.

Consider a TSP on G = (V,E). A partial solution is a path from a to b (a b). It will be denoted by a tuple [a,S,b] where a ∈ S,b ∈ S and S ⊆ V. The corresponding subproblem is to find a completion of the tour, i.e. the shortest complementary path b a with intermediate nodes V − S. Its cost is at least the sum of the following : the lightest edge from a to V − S. the lightest edge from b to V − S. the minimum spanning tree of V − S. Efficient solvers exist !.

26/29 26/29

B&B for Traveling Salesman Problem (TSP) II

27/29 27/29

Lagrangian relaxation

28/29 28/29

Lagrangian relaxation principle

IP problem P : ZP

=

max cx s.t. x ∈ X — “easy” constraints Ax ≤ b — “complicating” constraints

29/29 29/29

Lagrangian relaxation principle

IP problem P : ZP

=

max cx s.t. x ∈ X — “easy” constraints Ax ≤ b — “complicating” constraints Lagrangian relaxation LR(λ) : ZLR(λ) = max {cx +λ(b − Ax)|x ∈ X}

29/29 29/29

Lagrangian relaxation principle

IP problem P : ZP

=

max cx s.t. x ∈ X — “easy” constraints Ax ≤ b — “complicating” constraints Lagrangian relaxation LR(λ) : ZLR(λ) = max {cx +λ(b − Ax)|x ∈ X} LR(λ) is also an IP problem, but easier to solve than P LR(λ) is relaxation (upperbound) of P for any λ (i.e. ZP ≤ ZLR(λ))

29/29 29/29

Lagrangian relaxation principle

IP problem P : ZP

=

max cx s.t. x ∈ X — “easy” constraints Ax ≤ b — “complicating” constraints Lagrangian relaxation LR(λ) : ZLR(λ) = max {cx +λ(b − Ax)|x ∈ X} LR(λ) is also an IP problem, but easier to solve than P LR(λ) is relaxation (upperbound) of P for any λ (i.e. ZP ≤ ZLR(λ)) The best bound is given by solving the Lagrangian dual : Lagrangian dual ZLD : ZLD = min

λ

ZLR(λ)

29/29 29/29