CS3000: Algorithms & Data Jonathan Ullman Lecture 5: Dynamic - PowerPoint PPT Presentation

CS3000: Algorithms & Data Jonathan Ullman Lecture 5: Dynamic Programming: • Fibonacci Numbers, Interval Scheduling Jan 22, 2020

Dynamic Programming • Don’t think too hard about the name • I thought dynamic programming was a good name. It was something not even a congressman could object to. So I used it as an umbrella for my activities. -Bellman • Dynamic programming is careful recursion • Break the problem up into small pieces • Recursively solve the smaller pieces • Key Challenge: identifying the pieces E

Warmup: Fibonacci Numbers

� Fibonacci Numbers f Nfl D • 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, … • * + = * + − 1 + * + − 2 • * + → 0 1 ≈ 1.62 1 56 7 • 0 = is the golden ratio 9

Fibonacci Numbers: Take I FibI(n): If (n = 0): return 0 ElseIf (n = 1): return 1 Else: return FibI(n-1) + FibI(n-2) • How many recursive calls does FibI(n) make? of calls made by FibI n n y n ich 4 Clm tf Un n tf Chi Foth 1.62 f r r

Fibonacci Numbers: Take II Memoication Top Down Dynamic Programming M ← empty array, M[0] ← 0, M[1] ← 1 bICo FibII(n): 12 If (M[n] is not empty): return M[n] ElseIf (M[n] is empty): M[n] ← FibII(n-1) + FibII(n-2) return M[n] • How many recursive calls does FibII(n) make? new elements of M We fill n l recursive calls fills one elf Each pair of 21h D calls At most Oln

Fibonacci Numbers: Take III FEED DODD BB M FibIII(n): M[0] ← 0, M[1] ← 1 For i = 2,…,n: M[i] ← M[i-1] + M[i-2] return M[n] • What is the running time of FibIII(n) ? Bottom Up Dynamic Programming

Fibonacci Numbers • 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, … • * + = * + − 1 + * + − 2 • Solving the recurrence recursively takes ≈ 1.62 1 time • Problem: Recompute the same values * < many times • Two ways to improve the running time • Remember values you’ve already computed (“top down”) • Iterate over all values * < (“bottom up”) • Fact: Can solve even faster using Karatsuba’s algorithm!

Dynamic Programming: Interval Scheduling

� Interval Scheduling • How can we optimally schedule a resource? • This classroom, a computing cluster, … • Input: + intervals = > , ? > each with value @ > O • Assume intervals are sorted so ? 5 < ? 9 < ⋯ < ? 1 • Output: a compatible schedule C maximizing the total value of all intervals • A schedule is a subset of intervals C ⊆ {1, … , +} • A schedule C is c ompatible if no <, G ∈ C overlap • The total value of C is ∑ @ > >∈J

Interval Scheduling value El 3,53 8 4 2 2 O i i nde corea o i i 0 i

Possible Algorithms • Choose intervals in decreasing order of @ >

Possible Algorithms • Choose intervals in increasing order of = >

Possible Algorithms • Choose intervals in increasing order of ? > − = >

A Recursive Formulation • Let K be the optimal schedule • Bold Statement: K either contains the last interval or it does not.

A Recursive Formulation • Let K be the optimal schedule • Case 1: Final interval is not in K (i.e. 6 ∉ K ) The 0 must be the optimal schedule for El 2,3 4,53 111111111

A Recursive Formulation • Let K be the optimal schedule • Case 2: Final interval is in K (i.e. 6 ∈ K ) the optimal schedule for 9112,33 63 O must be 1111111111 11 11141 1411111 O

A Recursive Formulation • Let K > be the optimal schedule using only the intervals 1, … , < • Case 1: Final interval is not in K ( < ∉ K ) Oi Oi • Then K must be the optimal solution for 1, … , < − 1 • Case 2: Final interval is in K ( < ∈ K ) • Assume intervals are sorted so that ? 5 < ? 9 < ⋯ < ? 1 • Let M < be the largest G such that ? N < = > EstOpe Oi • Then K must be < + the optimal solution for 1, … , M < i TeiD are compatible with i pl il of 9,2 Any t.io value Oi 1 If vitvalve Open then Oi i3 Open

A Recursive Formulation • Let KOP(<) be the value of the optimal schedule using only the intervals 1, … , < • Case 1: Final interval is not in K ( < ∉ K ) • Then K must be the optimal solution for 1, … , < − 1 • Case 2: Final interval is in K ( < ∈ K ) • Assume intervals are sorted so that ? 5 < ? 9 < ⋯ < ? 1 • Let M < be the largest G such that ? N < = > • Then K must be < + the optimal solution for 1, … , M < • KOP < = max KOP < − 1 , @ 1 + KOP M < a • KOP 0 = 0, KOP 1 = @ 5

Interval Scheduling: Take I // All inputs are global vars FindOPT(n): if (n = 0): return 0 elseif (n = 1): return v 1 else: return max{FindOPT(n-1), v n + FindOPT(p(n))} • What is the running time of FindOPT(n) ? Can be exponential in n

Interval Scheduling: Take II Top Down ME i OPT i stores // All inputs are global vars M ← empty array, M[0] ← 0, M[1] ← v 1 FindOPT(n): if (M[n] is not empty): return M[n] else: M[n] ← max{FindOPT(n-1), v n + FindOPT(p(n))} return M[n] • What is the running time of FindOPT(n) ? 2 0 array eH calls recursive n n 1 array elts x

Interval Scheduling: Take III // All inputs are global vars FindOPT(n): M[0] ← 0, M[1] ← v 1 for (i = 2,…,n): Find OPT ploy mi M[i] ← max{FindOPT(n-1), v n + FindOPT(p(n))} qui humorous return M[n] • What is the running time of FindOPT(n) ? 01N time

Interval Scheduling: Take II o Pcl 2 O p L 3 l p I p 4 O i t 5 3 p I I 3 p G MELT MEN v Mfs max MEI ist Meo ME23 max max 4,4 233 6 maxE2 4 03 4 no yes yes yes yes yes M[0] M[1] M[2] M[3] M[4] M[5] M[6] 8 6 4 7 0 2 value of the max's MED rotMC 333 MEG optimal schedule I 163 max 980

Now You Try 1 @ 5 = 4 M 1 = 0 @ 9 = 2 2 M 2 = 1 3 @ V = 12 M 3 = 0 4 M 4 = 2 @ W = 5 5 @ 7 = 8 M 5 = 2 6 @ X = 1 M 6 = 3 7 @ Y = 10 M 7 = 1 M[0] M[1] M[2] M[3] M[4] M[5] M[6] M[7] 0 4

Finding the Optimal Schedule Doe hsgo bth • Let KOP(<) be the value of the optimal schedule using only the intervals 1, … , < I • Case 1: Final interval is not in K ( < ∉ K ) OPT i OPT o l • Case 2: Final interval is in K ( < ∈ K ) vi t OPT pci OPT i if there are multiple urls Be careful Both could be true • KOP < = max KOP < − 1 , @ 1 + KOP M < do the If this is If this is the max max then Oi S 3 10pct the Oi Oi

Interval Scheduling: Take II M[0] M[1] M[2] M[3] M[4] M[5] M[6]

Interval Scheduling: Take III // All inputs are global vars FindSched(M,n): if (n = 0): return ∅ elseif (n = 1): return {1} elseif (v n + M[p(n)] > M[n-1]): return {n} + FindSched(M,p(n)) else: return FindSched(M,n-1) • What is the running time of FindSched(n) ?

Dynamic Programming Recap • Express the optimal solution as a recurrence • Identify a small number of subproblems • Relate the optimal solution on subproblems • Efficiently solve for the value of the optimum • Simple implementation is exponential time • Top-Down: store solution to subproblems • Bottom-Up: iterate through subproblems in order • Find the solution using the table of values