Tabu Search Search Tabu Page 1 Part I Part I Tabu Search - - PDF document

Page 1

Tabu Tabu Search Search

Page 2

Part I Part I Tabu Tabu Search Principles Search Principles

Page 3

Tabu Tabu Search Search

Principles

Memory types Tabu classifications Choice rules

Additional TS elements

Strategic oscillation Tabu tenure strategies

Page 4

( (Very Very) Simple ) Simple Tabu Search Design Tabu Search Design

Construct an Initial Solution Repeat until some termination criterion

is satisfied:

Change something in the current solution

(but do not change anything that is tabu- active).

Declare what you just changed tabu-active.

Tabu items are only remembered for a

short period of time.

Page 5

Memory Types Memory Types

• Explicit Memory

Explicit Memory

A record of complete solutions Avoiding visiting solutions more than once

(limited use).

Elite solutions in tabu search may be used to

expand local search.

• Attributive Memory

Attributive Memory

A record of attributes that change when

moving from one solution to another.

Page 6

Illustrative Preview: Illustrative Preview: Attributive Memory Attributive Memory

Add Delete

Optimal Tree Problem with Nonlinear Objective Optimal Tree Problem with Nonlinear Objective

3 6 4 5 2 1 7

Iteration k

3 6 4 5 2 1 7

Iteration k+1

3 6 4 5 2 1 7

Iteration k+2

(1,3)in (4,6)out (1,4)in (2,4)out

Page 7

Recency Recency-

• Based Memory

Based Memory

Short term memory most commonly used

in tabu search implementations.

It keeps track of solution attributes that

have changed during the recent past.

Selected attributes that occur in solutions recently

visited are labeled tabu-active.

Solutions that contain tabu-active elements, or

particular combinations of these attributes, are those that become tabu.

Page 8

Minimum Minimum k k-

• Tree Problem

Tree Problem

1 4 6 9 11 7 8 10 2 3 5 12

1 26 25 15 16 24 17 20 18 16 6 23 16 9 20 9 8 6 16 7 9

Page 9

Greedy Construction Greedy Construction

Step 1: Step 1: Pick the arc with the minimum cost Pick the arc with the minimum cost 1 4 6 9 11 7 8 10 2 3 5 12

1 26 25 15 16 24 17 20 18 16 6 23 16 9 20 9 8 6 16 7 9

Page 10

Step 2: Step 2: Pick a min Pick a min-

• cost arc that

cost arc that connects node 1 or 2 connects node 1 or 2

1 4 6 9 11 7 8 10 2 3 5 12

1 26 25 15 16 24 17 20 18 16 6 23 16 9 20 9 8 6 16 7 9

Page 11

Step 3: Step 3: Pick a min Pick a min-

• cost arc that

cost arc that connects node 2 or 4 connects node 2 or 4

1 4 6 9 11 7 8 10 2 3 5 12

1 26 25 15 16 24 17 20 18 16 6 23 16 9 20 9 8 6 16 7 9

Page 12

Step 4: Step 4: Pick a min Pick a min-

• cost arc that

cost arc that connects node 2, 4 or 7 connects node 2, 4 or 7

1 4 6 9 11 7 8 10 2 3 5 12

1 26 25 15 16 24 17 20 18 16 6 23 16 9 20 9 8 6 16 7 9

Total Cost = 40 Total Cost = 40

Page 13

Swap Move Types Swap Move Types

2 1 4 6 7 1 25 8 6 Greedy solution Total weight: 40 2 1 4 6 7 1 25 15 6 Best static swap Total weight: 47 8 3 17 2 1 4 7 1 25 8 Best dynamic swap Total weight: 51 6 6

Page 14

Illustrative Tabu Classifications Illustrative Tabu Classifications

Table 2.3 TS iterations. Iteration Tabu-active net tenure Add Drop Weight 1 2 1 (4,6) (4,7) 47 2 (4,6) (4,7) (6,8) (6,7) 57 3 (6,8), (4,7) (6,7) (8,9) (1,2) 63

Page 15

Iteration 1: Iteration 1: Add (4,6) and Drop (4,7) Add (4,6) and Drop (4,7)

7 8 10 1 4 6 9 11 2 3 5 12

26 25 15 16 24 20 18 16 6 23 16 9 20 9 8 6 16 7 9 1 17

Total Cost = 40 + Total Cost = 40 + (15 (15-

• 8)

8) = 47 = 47

Page 16

Iteration 2: Iteration 2: Add (6,8) and Drop (6,7) Add (6,8) and Drop (6,7)

7 8 10 1 4 6 9 11 2 3 5 12

26 25 15 16 24 20 18 16 6 23 16 9 20 9 8 6 16 7 9 1 17

Total Cost = 47 + Total Cost = 47 + (16 (16-

• 6)

6) = 57 = 57

Page 17

Iteration 3: Iteration 3: Add (8,9) and Drop (1,2) Add (8,9) and Drop (1,2)

7 8 10 1 4 6 9 11 2 3 5 12

26 25 15 16 24 20 18 16 6 23 16 9 20 9 8 6 16 7 9 1 17

Total Cost = 57 + Total Cost = 57 + (7 (7-

• 1)

1) = 63 = 63

Page 18

Iteration 4: Iteration 4: Add (4,7) and Drop (1,4) Add (4,7) and Drop (1,4)

7 8 10 1 4 6 9 11 2 3 5 12

26 25 15 16 24 20 18 16 6 23 16 9 20 9 8 6 16 7 9 1 17

Total Cost = 63 + Total Cost = 63 + (8 (8-

• 25)

25) = 46 = 46

Page 19

Iteration 5: Iteration 5: Add (6,7) and Drop (4,6) Add (6,7) and Drop (4,6)

7 8 10 1 4 6 9 11 2 3 5 12

26 25 15 16 24 20 18 16 6 23 16 9 20 9 8 6 16 7 9 1 17

Total Cost = 46 + Total Cost = 46 + (6 (6-

• 15)

15) = 37 = 37

Page 20

Iteration 6: Iteration 6: Add (6,9) and Drop (6,8) Add (6,9) and Drop (6,8)

7 8 10 1 4 6 9 11 2 3 5 12

26 25 15 16 24 20 18 16 6 23 16 9 20 9 8 6 16 7 9 1 17

Total Cost = 37 + Total Cost = 37 + (16 (16-

• 16)

16) = 37 = 37

Page 21

Iteration 7: Iteration 7: Add (8,10) and Drop (4,7) Add (8,10) and Drop (4,7)

7 8 10 1 4 6 9 11 2 3 5 12

26 25 15 16 24 20 18 16 6 23 16 9 20 9 8 6 16 7 9 1 17

Total Cost = 37 + Total Cost = 37 + (9 (9-

• 9)

9) = 38 = 38

Page 22

Iteration 8: Iteration 8: Add (9,12) and Drop (6,7) Add (9,12) and Drop (6,7)

7 8 10 1 4 6 9 11 2 3 5 12

26 25 15 16 24 20 18 16 6 23 16 9 20 9 8 6 16 7 9 1 17

Total Cost = 38 + Total Cost = 38 + (9 (9-

• 6)

6) = 41 = 41

Page 23

Iteration 9: Iteration 9: Add (10,11) and Drop (6,9) Add (10,11) and Drop (6,9)

7 8 10 1 4 6 9 11 2 3 5 12

26 25 15 16 24 20 18 16 6 23 16 9 20 9 8 6 16 7 9 1 17

Total Cost = 41 + Total Cost = 41 + (9 (9-

• 16)

16) = 34 = 34

Page 24

Iteration 10: Iteration 10: Add (5,9) and Drop (9,12) Add (5,9) and Drop (9,12)

7 8 10 1 4 6 9 11 2 3 5 12

26 25 15 16 24 20 18 16 6 23 16 9 20 9 8 6 16 7 9 1 17

Total Cost = 34 + Total Cost = 34 + (16 (16-

• 9)

9) = 41 = 41

Page 25

Search Trajectory Search Trajectory

30 35 40 45 50 55 60 65 1 2 3 4 5 6 7 8 9 10

Iterations Weight Current Weight Best Weight

Page 26

Recency Recency-

• Based Memory for

Based Memory for Add/Drop Moves Add/Drop Moves

After a move (Added, Dropped)

TabuDropStart(Added) = Iter TabuAddStart(Dropped) = Iter.

TestAdd is tabu-active when:

Iter ≤ TabuAddStart(TestAdd) +

TabuAddTenure

TestDrop is tabu-active when:

Iter ≤ TabuDropStart(TestDrop) +

TabuDropTenure

Page 27

Streamlining Streamlining

After a move (Added, Dropped)

TabuEnd(Added) = Iter + TabuDropTenure TabuEnd(Dropped) = Iter + TabuAddTenure.

TestAdd is tabu-active when:

Iter ≤ TabuEnd(TestAdd)

TestDrop is tabu-active when:

Iter ≤ TabuEnd(TestDrop).

Page 28

TS Exercise: 0 TS Exercise: 0-

• 1 Knapsack

1 Knapsack

Maximize 10x1 + 14x2 + 9x3 + 8x4 + 7x5 + 5x6 + 9x7 + 3x8 Subject to 7x1 + 12x2 + 8x3 + 9x4 + 8x5 + 6x6 + 11x7 + 5x8 ≤ 38 xi ≥ 0 for i = 1, …, 8

Note: Variables are ordered in decreasing bang-for-buck value

Page 29

Move Mechanism Move Mechanism

Set xi = 1 whenever a feasible

assignment of this type exists and

• therwise set some xi = 0

Page 30

Choice Rule Choice Rule

• 1. Choose the largest bang-for-buck value

when setting xi = 1 and the smallest such value when setting xi = 0

• 2. Choose the largest profit value when

setting xi = 1 and the smallest such value when setting xi = 0

Page 31

Tabu Tenure Tabu Tenure

Effective tabu tenures have been empirically

shown to depend on the size of the problem instance.

Effective tabu tenures can usually be determined

for a class of problems by some experimentation.

Varying the tabu tenure during the search provides

• ne way to induce a balance between closely

examining one region and moving to different parts of the solution space.

Page 32

Dynamic Tabu Tenure Dynamic Tabu Tenure

Simple Strategies

Probabilistic Systematic

Specialized Strategies

The Reverse Elimination Method Moving Gaps The Tabu Cycle Method Conditional Probability Method

Page 33

Probabilistic Tenure Values Probabilistic Tenure Values

tmin tmax tmin tmax

Page 34

Systematic Changes to the Systematic Changes to the Tabu Tabu Tenure Tenure

Start with a fairly small tabu tenure and

increase it when there is evidence of cycling or when no new solutions have appeared for some time.

When an improvement occurs, let the

tabu tenure drop back to allow a more flexible search of the region where the improvement occurred.

Page 35

Part II Part II Tabu Tabu Search Advanced Search Advanced Strategies and Applications Strategies and Applications

Page 36

Tabu Tabu Search Part II Search Part II

Aspiration Criteria Candidate List Strategies

Aspiration Plus Elite Candidate List

Strategic Oscillation Longer Term Memory

Frequency-Based Approach

Applications

Page 37

Aspiration Criteria Aspiration Criteria

Aspiration criteria are

introduced in tabu search to determine when tabu activation rules can be

• verridden.

Aspiration by …

Default Objective

» Global » Regional

Search Direction Influence

Page 38

Neighborhood Definition and Neighborhood Definition and Search Search

Full neighborhood Computational Effort Single sample Candidate list strategy Neighborhood Size

Page 39

Candidate List Strategies Candidate List Strategies

Aspiration Plus Elite Candidate List Successive Filter Strategy Sequential Fan Candidate List Bounded Change Candidate List

Page 40

Aspiration Plus Aspiration Plus

1 2 3 4 5 6 7 8 9 10 11 12 Number of moves examined Move quality Aspiration Plus First Min Max

Page 41

Elite Candidate List Elite Candidate List

Iterations 1 2 3 Solution quality Threshold Master List Rebuild

Page 42

Strategic Oscillation Strategic Oscillation

Strategic oscillation is closely linked to the origins

• f tabu search.

It provides a means to achieve an effective

interplay between intensification and diversification.

It operates by orienting moves in relation to a

critical level.

Page 43

Graphical Representation Graphical Representation

Iterations 1 2 3 Oscillation Boundary Depth Level or Functional Value

Page 44

Simple Oscillation: Simple Oscillation: 0-

• 1 Knapsack

1 Knapsack

Set xi = 1 whenever the current solution

is feasible (even if xi = 1 makes the solution infeasible) and set xi = 0 whenever the current solution is infeasible

Page 45

Frequency Frequency-

• Based Memory

Based Memory

Frequencies consist of ratios. Numerator

Transition Measure Resident Measure

Denominator

Total number of occurrences of all events represented

by the numerators.

The sum (or average) of the numerators. The maximum numerator value.

Page 46

Example of Frequency Example of Frequency Measures Measures

Table 4.1. Example of frequency measures. Problem Residence Measure Transition Measure Sequencing Number of times job j has

• ccupied position π(j).

Number of times job i has exchanged positions with job j. Sum of tardiness of job j when this job occupies position π(j). Number of times job j has been moved to an earlier position in the sequence. Min k-Tree Problem Number of times edge (i, j) has been part of the current solution. Number of times edge (i, j) has been deleted from the current solution when edge (k, l) has been added. Sum of total solution weight when edge (i, j) is part of the solution. Number of times edge (i, j) has been added during improving moves.

Page 47

Applications Applications

Bandwidth Packing Problem

Laguna, M. and F. Glover (1993) “Bandwidth Packing: A Tabu Search Approach,” Management Science, vol. 39, no. 4, pp. 492-500.

Sonet Ring Design

Laguna, M. (1994) “Clustering for the Design of SONET Rings in Interoffice Telecommunications,” Management Science, vol. 40, no. 11, pp. 1533-1541.

Reducing the Bandwidth of a Sparse

Matrix

Martí, R., M. Laguna, F. Glover and V. Campos (2000) “Reducing the Bandwidth

• f a Sparse Matrix with Tabu Search,” University of Valencia.

Page 48

Bandwidth Packing Problem Bandwidth Packing Problem

Assign calls to paths in a capacitated

graph

Maximize total profit Subject to

Capacities are not violated

Note: It may not be possible to assign all

calls

Page 49

Telecommunications Network Telecommunications Network for Example Problem for Example Problem

4 2 10 1 3 8 7 6 5 9 Link number (capacity, cost)

6 (10, $8) 1 (25, $5) 5 (15, $4) 2 (35, $7) 7 (20, $40) 3 (40, $9) 4 (20, $5) 8 (15, $5) 12 (20, $7) 10 (15, $6) 11 (10, $3) 9 (10, $14)

Page 50

Profit Calculation Profit Calculation

4 2 10 1 3 8 7 6 5 9

Call from 4 to 6 with a bandwidth requirement of 6 and a revenue of $850

1 (25, $5) 5 (15, $4) 2 (35, $7) 7 (20, $40) 3 (40, $9) 4 (20, $5) 8 (15, $5) 12 (20, $7) 9 (10, $14) 11 (10, $3) 10 (15, $6) 6 (10, $8)

Profit = 850 - 6 * (8 + 5 + 5 + 6) = 850 - 144 = $706

Page 51

k k Shortest Paths Shortest Paths

Path Profit No.

Call from 4 to 6 Call from 4 to 6

4 → 2 → 1 → 9 → 6 706 1 4 → 2 → 1 → 9 → 8 → 6 616 3 4 → 2 → 1 → 8 → 6 634 2 4 2 10 1 3 8 7 6 5 9

6 (10, $8) 1 (25, $5) 5 (15, $4) 2 (35, $7) 7 (20, $40) 3 (40, $9) 4 (20, $5) 8 (15, $5) 12 (20, $7) 9 (10, $14) 11 (10, $3) 10 (15, $6)

4 → 2 → 1 → 3 → 8 → 6 406 4 4 → 2 → 1 → 3 → 8 → 9 → 6 412 5

Page 52

Greedy Construction Procedure Greedy Construction Procedure

Find k paths for each call Order calls by decreasing value of

revenue/bandwidth requirement ratio

Consider one call at a time and try to

assign it to its most profitable path

Discard call if no path is found with

enough capacity

Finish when all calls have been

considered

Page 53

Move Mechanism Move Mechanism

Calls 1 2 3 4 1 2 3 4 5 5 Paths 1 1 1 1 1 1

Page 54

Tabu Activation Tabu Activation

Calls 1 2 3 4 1 2 3 4 5 5 Paths 1 1 1 1 1

Page 55

Search Performance Graph Search Performance Graph

TS for Bandwidth Packing

3500 3700 3900 4100 4300 4500 4700 4900 1 20 39 58 77 96 115 134 153 172 191 210 229 248 267 286 Iteration Profit Current Best

Page 56

Other TS Features Other TS Features

Dynamic tabu tenure values

Controlled with a circular S, M and L list

Long term memory

Frequency counts to discourage non-

improving moves to assign calls to frequent paths

Page 57

Sonet Ring Design Sonet Ring Design

Given a telecommunications network and

point-to-point demand between central

• ffices:

Find office clusters that can be used as the

basis for designing Sonet rings

Minimize the total cost associated with

equipment and interconnection

Page 58

Hub Architecture Hub Architecture

Central Offices Hub Building

Page 59

Uni Uni-

• directional SONET Rings

directional SONET Rings

Page 60

SONET Planning Tool SONET Planning Tool

User Interface Clustering Logical Rings Physical Rings Routing

Page 61

Demand Network Demand Network

C (3) H (25) I (24) D (106) A (54) E (85) J (7) B (34) F (29) G (5)

2 1 4 12 4 3 13 8 3 8 5 5 23 15 14 19 36

Page 62

Optimal Solution Optimal Solution with OC with OC-

• 48 Rings

48 Rings

C (3) H (25) I (24) D (106) A (54) E (85) J (7) B (34) F (29) G (5)

Page 63

Traffic Distribution and Cost Traffic Distribution and Cost

Cluster Central Offices Internal External Total 1 B, D, E and F 48 48 2 D, E and I 45 3 48 3 A, D, F and G 45 45 4 A, C, D, E, H and J 45 3 48 Number of ADMs = 17 Fixed Cost = 1,938,578 Interconnection = 45,576 Total = $1,984,154

Page 64

Tabu Search Structure Tabu Search Structure

Delete Change or Duplicate Aspiration Plus (Circular Candidate List) Random Dynamic Tabu Tenures LP Routing Subproblem (Move Evaluation)

Page 65

Medium Medium-

• size Metropolitan Area

size Metropolitan Area

20 Central offices and 170 point-to-point demands

Original Solution ADMs Fixed Cost Interconnection TS Solution Total Cost 41 29 4,674 3,306 233 682 4,907 3,988

18.7% of total cost savings 18.7% of total cost savings

Page 66

Matrix Bandwidth Reduction Matrix Bandwidth Reduction

Savings on the computational effort

associated with solving the system of equations

Other applications include finite element

methods for approximating solutions of partial differential equations, large-scale power transmissions systems, circuit design, hypertext layout, chemical kinetics and numerical geophysics

Page 67

Bandwidth Reduction Example Bandwidth Reduction Example

1 2 3 4 5 1 2 3 4 5 1 1 1 1 1 1 1 1 1 1 1 6 7 6 7 1 1 1 1 1 1 1 Bandwidth = 3 3 7 4 1 6 5 2 3 7 4 1 6 5 2 1 2 3 4 5 1 2 3 4 5 1 1 1 1 1 1 1 1 6 1 7 1 1 6 7 1 1 1 1 1 1 1 1 Bandwidth = 6

Page 68

Graph Formulation Graph Formulation

1 4 6 5 7 3 2

Bandwidth = 6

4 3 5 6 2 1 7

Bandwidth = 3

Page 69

TS Features TS Features

Short-term memory

Unconventional move value calculation

(degree of change w.r.t. bandwidth of current solution)

Elite candidate list strategy Longer-term memory

Frequency-based approach Restarting procedure

Page 70

Summary of the Procedure Summary of the Procedure

• 1. Build initial labeling f and update best
• 2. iterwi = 0 and iter = 0

while ( iterwi < maxiter ) {

• 3. iter = iter + 1
• 4. Build

( ) ( ) ( ) { }

iter v tabu v f C f C > − = :

*

while ( ( )

∅ ≠ f C *

) {

• 5. Select the first vertex v from

( )

f C*

• 6. Select the best vertex u from

( ) ( ) { }

iter u tabu u v N > − ′ :

• 7. Execute

( )

u v move ,

8.

( ) ( )

tenure iter u tabu v tabu + = =

9. ( ) ( ) { }

v f C f C − =

* *

}

• 10. Evaluate

( )

G B f

if (

( )

G B f

improves the best ) { Update best iterwi = 0 } else iterwi = iterwi + 1 }