3 - - PowerPoint PPT Presentation
3 4
Similar to Datastructures
IntroductiontoAlgorithms (3rd edition).
homework set A.2 homework set B.1 homework set B.2 homework set C.1 homework set C.2 Organization of the course Part I: Techniques for optimization backtracking exercises greedy algorithms exercises dynamic programming I exercises dynamic programming II exercises Part II: Graph algorithms shortest paths I exercises shortest paths II exercises max flow exercises matching exercises Part III: Selected topics NP>hardness I exercises NP>hardness II exercises approximation algorithms exercises linear programming exercises
and so on …: see the course webpage
Grading Homework (no copying, solutions submitted in pdf by email to instructor)
(but: handed in per part, so only three deadlines)
Exam
Final grade
register this week for one
Optimization problems
associate cost to every solution, find min>cost solution
associate profit to every solution, find max>profit solution
17 15 33 Optimization problems: examples Traveling Salesman Problem
A B C D E F A B C D E F A > 17 B > C > 15 33 D > E > F >
Optimization problems: examples Traveling Salesman Problem
Knapsack
5 14 10 4.5 7 6 profit 6 11 8 5 7 5 weight 6 5 4 3 2 1 item W = 18 solutions: 1,2,6: weight 18, profit 18
Optimization problems: examples Traveling Salesman Problem
Knapsack
5 14 10 4.5 7 6 profit 6 11 8 5 7 5 weight 6 5 4 3 2 1 item W = 18 solutions: 1,2,6: weight 18, profit 18 2,5: weight 18, profit 21 etcetera
Optimization problems: examples Traveling Salesman Problem
Knapsack
Linear Programming minimize: c1 x1 + HHH + cn xn subject to: a1,1 x1 + HHH + a1,n xn ≤ b1 am,1 x1 + HHH + am,n xn ≤ bm even hard to find any solution!
for each option, recursively make other choices
seems best
recursively make other choices
Example 1: Traveling Salesman Problem (TSP) given: n cities and the (non>negative) distances between them Input: matrix Dist [1..n,1..n ], where Dist[i,j ] = distance from i to j goal: find shortest tour visiting all cities and returning to starting city Output: permutation of {1,…,n} such that visiting cities in that order gives min>length tour choices: what is first city to visit ? what is second city to visit ? … what is last city to visit ? 1 2 4 5 6 3 start start w.l.o.g. in city 1 start
for each option for next city, recursively try all ways to finish the tour for each recursive call:
( = part of the tour we fixed in earlier calls)
( = remaining cities, for which we need to decide visiting order)
compute length of tour, compare to length of shortest tour found so far parameters of algorithm
S = set of remaining cities (initially: S = { 2, …, n } ) TSP_BruteForce1(R,S)
2. minCost ← length of the tour represented by R 3. minCost ← 4. each city iin S 5. !Remove ifrom S, and append ito R. 6. minCost ← min(minCost,TSP_BruteForce1(R,S) ) 7. Reinsert iin S, and remove ifrom R.
We want to compute a shortest tour visiting all cities in RU S, under the condition that the tour starts by visiting the cities from Rin the given order. 8 iis next city undo choice all choices have been made recursively compute best way to make remaining choices try all remaining cities as next city
Parameters: R = sequence of already visited cities (initially: R = city 1) S = set of remaining cities (initially: S = { 2, …, n } ) TSP_BruteForce1(R,S)
2. minCost ← length of the tour represented by R 3. minCost ← 4. each city iin S 5. !Remove ifrom S, and append ito R. 6. minCost ← min(minCost,TSP_BruteForce1(R,S) ) 7. Reinsert iin S, and remove ifrom R.
We want to compute a shortest tour visiting all cities in RU S, under the condition that the tour starts by visiting the cities from Rin the given order. 8 iis next city undo choice all choices have been made recursively compute best way to make remaining choices try all remaining cities as next city Note: this algorithm computes length of optimal tour, not tour itself
Possible implementation: R and S are linked lists Analysis: nR = size of R, nS = size of S
O(1) O(nR) O(1) T(nR,nS) =nS " ( O(1) +T(nR+1,nS$1) ) with T(nR,0) =O(nR)
Possible implementation: R and S are linked lists Analysis: nR = size of R, nS = size of S
O(1) O(nR) O(1) T(nR,nS) =nS " ( O(1) +T(nR+1,nS$1) ) with T(nR,0) =O(nR) T(nR,nS) = O( (nR+nS) H nS! ) total running time = O( n! )
R = sequence of already visited cities (initially: R = city 1) S = set of remaining cities (initially: S = { 2, …, n } ) TSP_BruteForce1(A,k ) 1. n ← length[A] 2. k=n 3. minCost ← length of the tour represented by A[1..n] 4. minCost ← 5. i←k+1 n 6. !Swap A[i ] and A[k+1] 7.
9. Swap A[k+1] and A[i ]
Alternative implementation: ▪ A[1..n] = array of cities, parameter 1 ≤ k≤ n
8
R = sequence of already visited cities (initially: R = city 1) S = set of remaining cities (initially: S = { 2, …, n } ) TSP_BruteForce1(A,k ) 1. n ← length[A] 2. k=n 3. minCost ← length of the tour represented by A[1..n] 4. minCost ← 5. i←k+1 n 6. !Swap A[i ] and A[k+1] 7.
9. Swap A[k+1] and A[i ]
Alternative implementation: ▪ A[1..n] = array of cities, parameter 1 ≤ k≤ n
8 improvement: maintain lengthSoFar = length of initial part given by A[1] .. A[k] lengthSoFar +Dist[ A[n],A[1] ] lengthSoFar newLength
R = sequence of already visited cities (initially: R = city 1) S = set of remaining cities (initially: S = { 2, …, n } ) TSP_BruteForce1(A,k ) 1. n ← length[A] 2. k=n 3. minCost ← length of the tour represented by A[1..n] 4. minCost ← 5. i←k+1 n 6. !Swap A[i ] and A[k+1] 7.
9. Swap A[k+1] and A[i ]
Alternative implementation: ▪ A[1..n] = array of cities, parameter 1 ≤ k≤ n
8 improvement: maintain lengthSoFar = length of initial part given by A[1] .. A[k] lengthSoFar +Dist[ A[n],A[1] ] lengthSoFar newLength Old running time: O(n) for all permutations of 2,…n, so O(n!) in total New running time: O((n>1)!) still very very slow
TSP_BruteForce1(A,k,lengthSoFar,) 1. n ← length[A] 2. k=n 3. minCost ← lengthSoFar +Dist[ A[n],A[1] ] 4. minCost ← 5. i←k+1 n "# !Swap A[i ] and A[k+1]
7. minCost ←min(minCost,TSP_BruteForce1(A,k+1) ) 8. Swap A[k+1] and A[i ]
A[1..n] = array of cities, parameter 1 ≤ k≤ n A[1..k] contains initial part of tour, A [ k+1 .. n] contains remaining cities 8 pruning: don’t recurse if initial part cannot lead to optimal tour newLength minCost newLength ≥ minCost $
minCost
backtracking: not only for optimization problems parameters of algorithm? running time?
cluster 1 cluster 2 cluster 3 cost of single cluster: sum of distance to cluster average cost of total clustering: sum of costs of its clusters 1,3,4,6: average = 3.5, cost = 6
cluster 1 cluster 2 cluster 3 choices to be made
for each option, recursively try all ways to finish the clustering for each recursive call:
( = starting points we fixed in earlier calls)
( = starting points for remaining clusters)
compute cost of clustering, compare to best clustering found so far parameters of algorithm
k = desired number of clusters, A[1..k] = starting points m = number of starting points that have already been fixed Clustering(X,k,A,m)
2. minCost ← total cost of clustering represented by A[1..k] 3. minCost ← 4. i ← A[m]+1 n– (k$m$1) 5. !A [m+1] ← i 6. minCost ← min(minCost,Clustering(X,k,A,m+1) ) 7. A [m+1] ←
We want to compute an optimal partitioning of X into k clusters, under the condition that the starting points of clusters 1,..,m are as given by A[1..m]. 8 number of clusterings: ( ) = O( nk) running time: O( nk+1)
n k
k = desired number of clusters, A[1..k] = starting points m = number of starting points that have already been fixed Clustering(X,k,A,m,minCost,costSoFar)
2. minCost ← min (minCost,costSoFar +cost last cluster ) 3. minCost ← 4. i ← A[m]+1 n– (k$m$1) # !A [m+1] ← i newCost ← costSoFar +cost m$th cluster newCost ≥ minCost $ 6. minCost ← min(minCost,Clustering(X,k,A,m+1) ) 7. A [m+1] ←
Pruning: do not recurse if solution cannot get better than current best 8 minCost,newCost
Parameters: X[1..n] = sorted set of numbers (points in 1>dimensional space) k = desired number of clusters, A[1..k] = starting points m = number of starting points that have already been fixed Clustering(X,k,A,m,minCost,costSoFar)
2. minCost ← min (minCost,costSoFar +cost last cluster ) 3. minCost ← 4. i ← A[m]+1 n– (k$m$1) # !A [m+1] ← i newCost ← costSoFar +cost m$th cluster newCost ≥ minCost $ 6. minCost ← min(minCost,Clustering(X,k,A,m+1) ) 7. A [m+1] ←
Pruning: do not recurse if solution cannot get better than current best 8 minCost,newCost NOTE: This problem can actually be solved much more efficiently, for example using dynamic programming (as we will see later).
adjacent regions should have different colors parameters of algorithm? running time? what if we want to check if 2 colors are enough?
Summary
anything about how much the running time improves (and algorithm usually remains slow)
recursive calls, but pruning itself becomes more expensive