[PPT] - IE538 : Genetic Algorithms and Tabu search Term Project : Traveling PowerPoint Presentation

SLIDE 1

IE538 : Genetic Algorithms and Tabu search

Term Project : Traveling Salesman Problem (TSP) – Final presentation

GA & TSP - KJ.Kim, M.Buil

SLIDE 2

Description of the Problem

Problem simple to state
Shortest route to visit a collection of cities and return to the starting point (minimization problem)
Well-known NP-hard combinatorial optimization problem
GAs are global search algorithms for problems with huge search spaces such as the TSP
Parameters generally consist of given number of cities and their relative distances to each other

3 GA & TSP - KJ.Kim, M.Buil

SLIDE 4

Description of the Project

Search Area of 540,000 km2 (like France)
100 cities, spread out on a circle (to check our results)
Initial information given : matrix of distances between

every pair cities

We want to compare two crossover operators :

Edge Recombination Crossover Sub-tour Exchange Crossover

(SXX works best with high-fitness sub-tours. ERX can find good local solutions.)

In two cases:

Cities are spread out in a circle (so that we can check the results) Cities are spread out randomly (so that we can check the robustness of the operators)

4 GA & TSP - KJ.Kim, M.Buil

Shortest length: 735xπ = 2309

SLIDE 5

Fitness Function

Our problem is a minimization problem.
We have to maximize the following fitness function :

= 0 Otherwise Where n: total number of cities Xi: city number in position I D(Xi , Xj ): distance from Xi to Xj Cmax: Sum of all values of our distance matrix (naïve upper bound)

Also, we assume that our problem is symmetric in distance, so that :

D(Xi , Xj ) = D(Xj , Xi ) If g(x) < Cmax ,

5 GA & TSP - KJ.Kim, M.Buil

SLIDE 6

Operators

Selection operator

Binary tournament

Changing Crossover operator (CXO)

ERX to SXX SXX to ERX

Crossover operator

ERX SXX

Mutation operator

Swap each city in the string with another with the mutation probability

6 GA & TSP - KJ.Kim, M.Buil

SLIDE 7

Operators - Edge Recombination Crossover

Requires that the number of edges inherited from both parents be as large as possible

edge (7 6) or (6 7) Ex: Available edges Algorithm Parent 1 : (1 2 3 4 5 6 7 8 9) City 1: (1 9), (1 2), (1 4)

Choose the start city

with the smallest Parent 2 : (4 1 2 8 7 6 9 3 5) City 2: (2 1), (2 3), (2 8) number of edges …

Select next city which

City 8: (8 7), (8 9), (8 2) has smallest number of City 9: (9 8), (9 1), (9 6), (9 3) available edges.

Continue

Parents d(1 9) = d(9 1) In this algorithm, the resulting path is (1 4 5 6 7 8 2 3 9)

Smallest number of edges criterion

Selecting the city with the smallest number of edges maximizes the probability that you will finish the tour using the parental set of edges From previous studies experiments, failure occurs in less than 1.5% of the cases with this criterion

7 GA & TSP - KJ.Kim, M.Buil

SLIDE 8

Operators - Sub-tour Exchange Crossover

8 GA & TSP - KJ.Kim, M.Buil

Sub-tours S1 and S2 are sub-paths of parent tour 1 and 2 respectively
Algorithm

Select two strings randomly For one string, randomly select a starting site and a length For all genes (cities) in sub-path 1, determine sub-path 2, consisting of the same cities From these two first strings, the two sub-paths and the two reverse-order-sub-paths, 4 children can result of crossover Fitness of all 4 children is calculated, and the two with higher fitness are selected Ex: Parent 1 : (1 2 3 4 5) sub-path 1: 3 4 5 reverse-order-sub-path 1: 5 4 3 Parent 2 : (2 4 1 5 3) sub-path 2: 4 5 3 reverse order sub-path 2: 3 5 4 4 children: 1 2 4 5 3 (string 1, s2) 1 2 3 5 4 (string 1, rs2) 2 3 1 4 5 (string 2, s1) 2 5 1 4 3 (string 2, rs1)

SLIDE 9

Operators - Mutation

Mutation is actually done at the same time as crossover, on each new string created in the new pool.
Our mutation operator consists in swapping the gene in successive positions (1st position, 2nd position,

3rd, etc), according to the probability of mutation, with another gene, selected randomly.

Algorithm: Consider a chromosome of n genes (n cities)

Take gene in 1st position: test the probability of swapping with Pm If swap=true, then swap the gene with another random gene (2nd, 3rd, …, nth) Move to next position, and redo the above process Continue, until the nth gene is reached

Example: string (1 2 3 4 5)

Take gene in 1st position: assume swap=true, and the random gene is the 4th one. The new string is (4 2 3 1 5) Move to next position, and redo the above process. Assume swapping is done with the gene in 1st position, than the new string is (2 4 3 1 5) Continue, until the 5th gene is reached

9 GA & TSP - KJ.Kim, M.Buil

SLIDE 10

Various Criteria

Chromosome Representation : Each gene represents a city number the length of the string is equal

to the total number of cities

Initial Population : 100
Cmax: 4,679,300
Crossover probability : 0.8
Mutation probability : 0.0005
In case of changing operators: change is set as when the last 10 generations have similar average

distances - calculated by or if the best so far is the same for the last 50 generations. We consider there is similarity in average distances when in the last 10 generations 0.999 ≤ r ≤ 1.001

Termination Criteria : Termination is set as when the population reaches 1,000 generations

10 GA & TSP - KJ.Kim, M.Buil

ㅡ ㅡ

SLIDE 11

Algorithm diagram - 1 (Without changing operators)

11 GA & TSP - KJ.Kim, M.Buil

SLIDE 12

Algorithm diagram - 2 (With changing operators)

12 GA & TSP - KJ.Kim, M.Buil

SLIDE 13

Algorithm diagram - 3 (With changing operators)

13 GA & TSP - KJ.Kim, M.Buil

SLIDE 14

Results - ERX - 1

Evolution of average fitness after 1,000 generations & 25 runs:
Average fitness after 1,000 generations: 4,641,801
Best fitness after 1,000 generations: 4,651,952

14 GA & TSP - KJ.Kim, M.Buil

SLIDE 15

Results - ERX - 2

Evolution of path distance after 1,000 generations & 25 runs:
Overall fastest generation at which the overall best string is found: T = 585
Overall shortest path at generation 1,000: 27,348

15 GA & TSP - KJ.Kim, M.Buil

SLIDE 16

Results - ERX - 3

Representation of the best string found in 1,000 generations and 25 runs:

16 GA & TSP - KJ.Kim, M.Buil

SLIDE 17

Results - SXX - 1

Evolution of average fitness after 1,000 generations & 25 runs:
Average fitness after 1,000 generations: 4,675,462
Best fitness after 1,000 generations: 4,676,862

17 GA & TSP - KJ.Kim, M.Buil

SLIDE 18

Results - SXX - 2

Evolution of average path distance after 1,000 generations & 25 runs:
Overall fastest generation at which the overall best string is found: T = 832
Overall shortest path at generation 1,000: 2,438 (which is not the best that can be found - 2,309 - but

close)

18 GA & TSP - KJ.Kim, M.Buil

SLIDE 19

Results - SXX - 3

Representation of the best string found in 1,000 generations and 25 runs:

81~96 - 98 - 97 - 99~39 - 41 - 40 - 42~53 - 55 - 54 - 56~80

19 GA & TSP - KJ.Kim, M.Buil

SLIDE 20

Results - ERX changing with SXX - 1

Evolution of average fitness after 1,000 generations & 25 runs:
Average fitness after 1,000 generations: 4,675,482
Best fitness after 1,000 generations: 4,676,770

20 GA & TSP - KJ.Kim, M.Buil

Average first convergence - change of operators from ERX to SXX

SLIDE 21

Results - ERX changing with SXX - 2

Evolution of average path distance after 1,000 generations & 25 runs:
Average generation of operators’ change: T = 107
Average length of path at the generation of operators’ change: about 29,000
Overall fastest generation at which the overall best string is found: T = 908
Overall shortest path at generation 1,000: 2530

21 GA & TSP - KJ.Kim, M.Buil

SLIDE 22

Results - ERX changing with SXX - 3

Representation of the best string found in 1,000 generations and 25 runs:

70~96-35-98-0-99-1~10-12-11-13~20-22-21-23-25-24-26-27-29-28-30~34-97-36~69

22 GA & TSP - KJ.Kim, M.Buil

SLIDE 23

Results - SXX changing with ERX - 1

Evolution of average fitness after 1,000 generations & 25 runs:
Average fitness after 1,000 generations: 4,673,417
Best fitness after 1,000 generations: 4,676,402

23 GA & TSP - KJ.Kim, M.Buil

SLIDE 24

Results - SXX changing with ERX - 2

Evolution of average path distance after 1,000 generations & 25 runs:
Average generation of operators’ change: T = 543
Average length of path at the generation of operators’ change: about 4,440
Overall fastest generation at which the overall best string is found: T = 946
Overall shortest path at generation 1,000: 2,898

24 GA & TSP - KJ.Kim, M.Buil

SLIDE 25

Results - SXX changing with ERX - 3

Representation of the best string found in 1,000 generations and 25 runs:

76~81-83-82-84-86-85-87~96-98-97-99-0-2-1-3~42-44-43-45~54-56-55-58-57-59-61-60-62~65-69-68- 67-66-70-72-71-74-73-75

25 GA & TSP - KJ.Kim, M.Buil

SLIDE 26

Results - Comparison of ERX, SXX, ERX-SXX and SXX-ERX - 1

26 GA & TSP - KJ.Kim, M.Buil

Average fitness values of each crossover operator (simple circle problem)

SLIDE 27

Results - Comparison of ERX, SXX, ERX-SXX and SXX-ERX - 2

SXX shows better results than all other combinations of crossover operators

It finds optimal solutions the fastest However, it takes a lot of time to find the overall best

ERX cannot find the best path

It finds its optimal solution quite fast

ERX-SXX eventually reaches results similar to SXX alone, but a little worse

For our simple first problem (cities located on a circle), this combination proves useless compared to SXX alone

SXX-ERX is deteriorated by the use of ERX towards the end

Fast finding of optimal solutions due to SXX somewhat late finding of overall best

27 GA & TSP - KJ.Kim, M.Buil

SLIDE 28

Results - Comparison of ERX, SXX, ERX-SXX and SXX-ERX - 3

28 GA & TSP - KJ.Kim, M.Buil

Average fitness values of each crossover operator (random location problem)

SLIDE 29

Results - Comparison of ERX, SXX, ERX-SXX and SXX-ERX - 4

Results with random location of cities are similar to the results with cities located on a circle

SXX proves to be better than all other crossover operators when comparing average fitness ERX is the worst crossover operator when comparing average fitness

Because of random location of the cities, the average fitness of all operators is significantly lower

(almost 900,000)

Results could be improved by implementing a higher Pm with a larger population, but we could not do

it due to computer memory problems

This second experiment tends to show that the combination of ERX and SXX is less powerful than

SXX alone in every cases of TSP.

However, previous independent studies with harder problems showed that SXX used alone may have

difficulties with finding the optimal solution.

29 GA & TSP - KJ.Kim, M.Buil

SLIDE 30

Discussion / Further improvements

The results agree with what was expected from previous studies:

ERX converges fast, but can get stuck at local optimal solutions SXX can approximate best solutions well, but then struggles to find an optimal solution ERX-SXX in more difficult problems can combine the ERX fast convergence to local

ptima with the searching power of SXX
EAX proved difficult to implement in this project, but would be interesting to compare with previous
perators, together with the combination of SXX
The various criteria used (Pm, Pc, termination criteria, initial population, …), although optimal for our

computation power, were perhaps not the best, and further work could try to study the effects of changing these criteria

The implementation of the algorithm and especially of all operators was not optimal, and can be

perfected

There was some limitations due to computer memory problems or excel function parameters, such as:

the population at each generation could only consist of 100 individuals

nly 25 runs could be kept into memory, thus reducing the range of study
With more computation power, with a more difficult (i.e. random location, large size, …) problem,

more accurate results could be obtained

30 GA & TSP - KJ.Kim, M.Buil

SLIDE 31

Conclusion

Overall, SXX proved to be better than each of the other crossover operators
In more complex problems, the combination ERX-SXX might find better solutions than the other

crossover operators studied, but that could not be proven here

These results are consistent with what we expected from previous independent studies of each

crossover operators

A good way to put into practice what has been taught during the two thirds of the semester

Which operators to use ? How to implement these operators ? How to understand the results ? What to modify to make the results better ?

TSP is a well-known problem and it proved very interesting to work on it and try to get some results

with our algorithm

31 GA & TSP - KJ.Kim, M.Buil

SLIDE 32

References

“Solving the Traveling Salesman Problem through Genetic Algorithms with Changing Crossover

Operators” – Ryouei Takahashi – IMCLA‘05 (2005)

“An analysis of Edge Assembly Crossover for the Traveling Salesman Problem” – Yuichi Nagata,

Shigenobu Kobayashi – IEEE (1999)

“An improved Genetic Algorithm for TSP” – Li-Ying Wang, Jie Zhang, Hua Li – The 54th Institute of

CETC (2007)

“The Traveling Salesrep Problem, Edge Assembly Crossover, and 2-opt” – J.Watson & Al. – IEEE

(1998)

“Performance Analysis for Crossover Operators of Genetic Algorithm” – Kengo Katayama, Hisayuki

Hirabayashi , Hiroyuki Narihisa – Systems and Computers in Japan (1999)

http://www.cs.uml.edu/~giam/91.510/Lectures/Lecture6.ppt

32 GA & TSP - KJ.Kim, M.Buil

IE538 : Genetic Algorithms and Tabu search Term Project : Traveling - - PowerPoint PPT Presentation