CS3000: Algorithms & Data Jonathan Ullman Lecture 5: Dynamic - - PowerPoint PPT Presentation

CS3000: Algorithms & Data Jonathan Ullman

Lecture 5:

• Dynamic Programming:

Fibonacci Numbers, Interval Scheduling Jan 22, 2020

Dynamic Programming

• Don’t think too hard about the name
• I thought dynamic programming was a good name. It

was something not even a congressman could object to. So I used it as an umbrella for my activities. -Bellman

• Dynamic programming is careful recursion
• Break the problem up into small pieces
• Recursively solve the smaller pieces
• Key Challenge: identifying the pieces

E

Warmup: Fibonacci Numbers

Fibonacci Numbers

• 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, …
• * + = * + − 1 + * + − 2
• * + → 01 ≈ 1.621
• 0 =

56 7

• 9

is the golden ratio

f NflD

Fibonacci Numbers: Take I

• How many recursive calls does FibI(n) make?

FibI(n): If (n = 0): return 0 ElseIf (n = 1): return 1 Else: return FibI(n-1) + FibI(n-2)

n

• f calls made by FibI

n

y

n

Clm Un n

ich 4

tf

tf

Chi

Foth

1.62

r r

f

Fibonacci Numbers: Take II

• How many recursive calls does FibII(n) make?

M ← empty array, M[0] ← 0, M[1] ← 1 FibII(n): If (M[n] is not empty): return M[n] ElseIf (M[n] is empty): M[n] ← FibII(n-1) + FibII(n-2) return M[n]

Memoication

Top Down DynamicProgramming

bICo

12

We fill

n l

new elements of M

Each pair of

recursive calls fills

• ne elf

At

most

21h D calls

Oln

Fibonacci Numbers: Take III

• What is the running time of FibIII(n)?

FibIII(n): M[0] ← 0, M[1] ← 1 For i = 2,…,n: M[i] ← M[i-1] + M[i-2] return M[n]

M

DODD

BB

FEED

Bottom Up Dynamic Programming

Fibonacci Numbers

• 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, …
• * + = * + − 1 + * + − 2
• Solving the recurrence recursively takes ≈ 1.621 time
• Problem: Recompute the same values * < many times
• Two ways to improve the running time
• Remember values you’ve already computed (“top down”)
• Iterate over all values * < (“bottom up”)
• Fact: Can solve even faster using Karatsuba’s algorithm!

Dynamic Programming: Interval Scheduling

Interval Scheduling

• How can we optimally schedule a resource?
• This classroom, a computing cluster, …
• Input: + intervals =>, ?

> each with value @>

• Assume intervals are sorted so ?

5 < ? 9 < ⋯ < ? 1

• Output: a compatible schedule C maximizing the

total value of all intervals

• A schedule is a subset of intervals C ⊆ {1, … , +}
• A schedule C is compatible if no <, G ∈ C overlap
• The total value of C is ∑

@>

• >∈J

O

Interval Scheduling

value El 3,53

2

4 2 8

O

i

nde

i

corea

• i

i

i

Possible Algorithms

• Choose intervals in decreasing order of @>

Possible Algorithms

• Choose intervals in increasing order of =>

Possible Algorithms

• Choose intervals in increasing order of ?

> − =>

A Recursive Formulation

• Let K be the optimal schedule
• Bold Statement: K either contains the last

interval or it does not.

A Recursive Formulation

• Let K be the optimal schedule
• Case 1: Final interval is not in K (i.e. 6 ∉ K)

The

0 mustbe the optimal schedule for El 2,3 4,53

111111111

A Recursive Formulation

• Let K be the optimal schedule
• Case 2: Final interval is in K (i.e. 6 ∈ K)

O must be

63 theoptimal schedule for 9112,33

1111111111 11 11141 1411111

O

A Recursive Formulation

• Let K> be the optimal schedule using only the

intervals 1, … , <

• Case 1: Final interval is not in K (< ∉ K)
• Then K must be the optimal solution for 1, … , < − 1
• Case 2: Final interval is in K (< ∈ K)
• Assume intervals are sorted so that ?

5 < ? 9 < ⋯ < ? 1

• Let M < be the largest G such that ?

N < =>

• Then K must be < + the optimal solution for 1, … , M <

Oi

Oi

Oi

EstOpe

Any

• f 9,2

plil

are compatiblewith i

t.io

i TeiD

If vitvalve Open

value Oi 1

then Oi

i3 Open

A Recursive Formulation

• Let KOP(<) be the value of the optimal schedule

using only the intervals 1, … , <

• Case 1: Final interval is not in K (< ∉ K)
• Then K must be the optimal solution for 1, … , < − 1
• Case 2: Final interval is in K (< ∈ K)
• Assume intervals are sorted so that ?

5 < ? 9 < ⋯ < ? 1

• Let M < be the largest G such that ?

N < =>

• Then K must be < + the optimal solution for 1, … , M <
• KOP < = max KOP < − 1 , @1 + KOP M <
• KOP 0 = 0, KOP 1 = @5

a

Interval Scheduling: Take I

• What is the running time of FindOPT(n)?

// All inputs are global vars FindOPT(n): if (n = 0): return 0 elseif (n = 1): return v1 else: return max{FindOPT(n-1), vn + FindOPT(p(n))}

Can beexponential in n

Interval Scheduling: Take II

• What is the running time of FindOPT(n)?

// All inputs are global vars M ← empty array, M[0] ← 0, M[1] ← v1 FindOPT(n): if (M[n] is not empty): return M[n] else: M[n] ← max{FindOPT(n-1), vn + FindOPT(p(n))} return M[n]

Top Down

MEi

stores

OPT i

n

2

recursive

calls

array eH

x

n 1 array elts

Interval Scheduling: Take III

• What is the running time of FindOPT(n)?

// All inputs are global vars FindOPT(n): M[0] ← 0, M[1] ← v1 for (i = 2,…,n): M[i] ← max{FindOPT(n-1), vn + FindOPT(p(n))} return M[n]

mi

qui

Find

OPT ploy

humorous

01N time

Interval Scheduling: Take II

M[0] M[1] M[2] M[3] M[4] M[5] M[6]

Pcl

• p

2 O

l

p

3

L

I

i

p 4

O

t

p

5 3

I

I

p G 3

ME23

max

MEI

ist Meo

Mfs

max

MEN

v

MELT

maxE2

4 03 4

max 4,4 233

6

yes

yes yes

yes

yes

no

2

4

6 7

8

MEG max'sMED rotMC

333

valueof the

max 980

I 163

• ptimal schedule

Now You Try

@5 = 4 @V = 12 @9 = 2 @7 = 8 @W = 5 @X = 1 1 2 3 4 5 6 M 1 = 0 M 2 = 1 M 3 = 0 M 4 = 2 M 5 = 2 M 6 = 3

M[0] M[1] M[2] M[3] M[4] M[5] M[6] M[7] 4

@Y = 10 7 M 7 = 1

Finding the Optimal Schedule

• Let KOP(<) be the value of the optimal schedule

using only the intervals 1, … , <

• Case 1: Final interval is not in K (< ∉ K)
• Case 2: Final interval is in K (< ∈ K)
• KOP < = max KOP < − 1 , @1 + KOP M <

Doe

hsgo bth

I

OPT i

OPT o l

OPT i

vi t OPT pci

Be careful

Both could be true

ifthere

are multiple urls

do

Ifthis is the max

If this is

the

max

the Oi Oi

then Oi S 3 10pct

Interval Scheduling: Take II

M[0] M[1] M[2] M[3] M[4] M[5] M[6]

Interval Scheduling: Take III

• What is the running time of FindSched(n)?

// All inputs are global vars FindSched(M,n): if (n = 0): return ∅ elseif (n = 1): return {1} elseif (vn + M[p(n)] > M[n-1]): return {n} + FindSched(M,p(n)) else: return FindSched(M,n-1)

Dynamic Programming Recap

• Express the optimal solution as a recurrence
• Identify a small number of subproblems
• Relate the optimal solution on subproblems
• Efficiently solve for the value of the optimum
• Simple implementation is exponential time
• Top-Down: store solution to subproblems
• Bottom-Up: iterate through subproblems in order
• Find the solution using the table of values