Merge Sort 5/6/2003 1:27 PM Outline and Reading The General - PDF document

Merge Sort 5/6/2003 1:27 PM Outline and Reading The General Technique (§5.3.2) Dynamic Programming 0-1 Knapsack Problem (§5.3.3) Matrix Chain-Product (§5.3.1) Dynamic Programming version 1.4 1 Dynamic Programming version 1.4 2 Dynamic Programming revealed Computing Fibonacci Break problem into subproblems that are shared Dynamic Programming Recursive solution: � have subproblem optimality (optimal subproblem is a general algorithm � � int fib(int x) solution helps solve overall problem) design paradigm: if (x=0) return 0; subproblem optimality means can write recursive � else if (x=1) return 1; Iteratively solves small realtionship between subproblems! � else return fib(x-1) + subproblems which are Defining subproblems is hardest part! � fib(x-2); combined to solve overall problem. Compute solutions to small subproblems Dynamic Programming Fibonacci numbers Store solutions in array A. Solution: defined Combine already computed solutions into solutions for larger subproblems � f[0]=0; f[1]=1; � F 0 = 0 for i ← 2 to x do Solutions Array A is iteratively filled � F 1 = 1 f[i] ← f[i-1] + f[i-2]; � F n = F n-1 + F n-2 , for n > 1 return f[x]; (Optional: reduce space needed by reusing array) Dynamic Programming version 1.4 3 Dynamic Programming version 1.4 4 Reducing Space for The General Dynamic Computing Fibonacci Programming Technique Applies to a problem that at first seems to store only previous 2 values to compute next require a lot of time (possibly exponential), value provided we have: � int fib(x) � Simple subproblems: the subproblems can be if (x=0) return 0; defined in terms of a few variables, such as j, k, l, else if (x=1) return 1; m, and so on. else int last ← 1; nextlast ← 0; � Subproblem optimality: the global optimum value for i ← 2 to x do can be defined in terms of optimal subproblems temp ← last + nextlast; � Subproblem overlap: the subproblems are not nextlast ← last; independent, but instead they overlap (hence, last ← temp; should be constructed bottom-up). return temp; Dynamic Programming version 1.4 5 Dynamic Programming version 1.4 6 1

Merge Sort 5/6/2003 1:27 PM The 0/1 Knapsack Problem Example Given: A set S of n items, with each item i having Given: A set S of n items, with each item i having � b i - a positive benefit � b i - a positive benefit � w i - a positive weight � w i - a positive weight Goal: Choose items with maximum total benefit but with Goal: Choose items with maximum total benefit but with weight at most W. weight at most W. “knapsack” If we are not allowed to take fractional amounts, then this is the 0/1 knapsack problem . Solution: � In this case, we let T denote the set of items we take Items: • 5 (2 in) ∑ • 3 (2 in) b 1 2 3 4 5 � Objective: maximize i • 1 (4 in) Weight: ∈ T i 4 in 2 in 2 in 6 in 2 in 9 in Benefit: ∑ $20 $3 $6 $25 $80 ≤ w W � Constraint: i ∈ i T Dynamic Programming version 1.4 7 Dynamic Programming version 1.4 8 A 0/1 Knapsack Algorithm, A 0/1 Knapsack Algorithm, First Attempt Second Attempt S k : Set of items numbered 1 to k. S k : Set of items numbered 1 to k. Define B[k] = best selection from S k . Define B[k,w] = best selection from S k Problem: does not have subproblem optimality: with weight exactly equal to w � Consider S={(3,2),(5,4),(8,5),(4,3),(10,9)} benefit-weight pairs Good news: this does have subproblem optimality: Best for S 4 :  − > B [ k 1 , w ] if w w = k B [ k , w ]  − − − + max{ B [ k 1 , w ], B [ k 1 , w w ] b } else  k k Best for S 5 : I.e., best subset of S k with weight limit exactly w is either the best subset of S k-1 w/ weight w or the best subset of S k-1 w/ weight w-w k plus benefit of item k. Dynamic Programming version 1.4 9 Dynamic Programming version 1.4 10 Towards the 0/1 Knapsack Towards the 0/1 Knapsack Algorithm Algorithm  = 0 if k 0 S k : Set of items numbered 1 to k = {(b 1 ,w 1 ), (b 2 ,w 2 ),  = B [ k , j ]  − > B [ k 1 , j ] if w j …, (b k ,w k )} k  − − − +  max{ B [ k 1 , j ], B [ k 1 , j w ] b } otherwise k k Define B[k,j] = maximum benefit of optimal subset Algorithm rec01Knap ( S, W ): from S k with total weight at most j B[k,j] = maximum benefit Input: set S of k items w/ benefit b 1 , b 2 , … b k , ; weights w 1 , w 2 , … w kj and max. Recursive definition of B[k,j]: of optimal subset from S k weight W with total weight at most j Output: benefit of best subset with  0 if k = 0 weight at most W Recursive version of  = B [ k , j ]  − if w > j B [ k 1 , j ] if k =0 then {S = emptyset} algorithm based on k  − − − + max{ B [ k 1 , j ], B [ k 1 , j w ] b } otherwise  return 0 k k recursive subproblem remove item k (benefit-weight (b k ,w k )) relationship. from S Not a dynamic if w k > W then {item k does not fit} return rec01Knap(S,W) programming version. return max (rec01Knap(S,W), rec01Knap(S,W-w k ) + b k ) Dynamic Programming version 1.4 11 Dynamic Programming version 1.4 12 2