Memoization & Greedy: Wrapping up programmatical approaches - - PowerPoint PPT Presentation

Memoization Greedy Algorithms Example

Memoization & Greedy: Wrapping up programmatical approaches

with thanks to Kosie Van Der Merwe

Tslil Clingman 2nd August

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

Outline

1

Memoization

2

Greedy Algorithms

3

Example

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

The problem

Consider the naive implementation of fib(n) = fib(n − 1) + fib(n − 2) We take fib(0) = 0, fib(1) = 1 Look at the case of fib(5) - about half the work is computed twice! It can actually be shown that running such a recursive call takes O(ϕn) time! Surely once we have computed the value of fib(n) we needn’t compute it again?

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

The idea

What if, instead of just blindly computing the value at every step we ... Keep track of all previously computed results in some table Look up the arguments, if it has been computed already, we needn’t do any work Otherwise, compute the result but store it in the table It’s fun to see that this doesn’t actually rely on global state at all We can also have n many base cases now, without polluting the function definition

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

Pseudo-code

def fib_memo(n,dct): if n in dct: return dct[n] else res = fib_memo(n-1,dct)+fib_memo(n-2,dct) dct[res] = n return res def fib(n): return fib_memo(n,{0:0, 1:1})

We now have a fib(n) implementation which takes only O(n) time!

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

That’s all folks!

In principle, that’s all there is to it Just about any recursive function can be transformed into a more efficient version through memoization The general approach remains the same as we have seen above If you’re worried about duplicated computing effort, consider memoization Space-time trade-off

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

Outline

1

Memoization

2

Greedy Algorithms

3

Example

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

Gimme gimme gimme

The idea here is, at every step where some decision is required, consume as much (or as little) as the case may be Hope that this leads to the optimal solution ... I know. I said hope. ... So then why?

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

The why

By and large, greedy algorithms are very very simple to code The are comparatively easy to understand, the logic is intuitive to us Constitute the ‘obvious’ solution Far more often than not, they run in O(n) time.

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

Enough talk!

Consider the classic change problem There is a certain total which you must account for given some set of denominations You seek the solution with the least overall coins (for convenience and customer satisfaction) In specific, say denominations = {1, 2, 5, 10}, need change for an amount of 14 By hand, we could see the solution is 1 × 10 + 2 × 2 for a total of three coins No ... this is compsci! Particulars nothing, we want to solve the class of problems, not this instance

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

The proposed solution

# denoms is a sorted array of the denominations # change is the amount given in for change def make_change(change, denoms): issued = [] while change > 0: k = len(denoms)-1 while denoms[k]>change: k-=1 largest = denoms[k] change -= largest issued.append(largest) return issued change(14,[1,2,5,10]) >> [10, 2, 2] change(16,[1,2,5,10]) >> [10, 5, 1]

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

See what we have wrought!

On first examination, this seems to work well enough - no

• bvious flaw like issuing the wrong coin

However, there are two somewhat serious errors in this logic:

1

It can fail to converge on a solution

2

It will converge on sub-optimal solutions

Can you see such cases?

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

Oopsy

change(14,[7, 10]) >> IndexError: list index out of range change(14,[2, 7, 10]) >> [10, 2, 2] but optimal is [7, 7]

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

Hark, a pitfall!

Greedy may be somewhat more formally described as an algorithm which attempts to find a globally optimum solution by traversing the problem space and making locally optimum choices at each step Clearly, this will not always succeed Thus bear the following in mind when opting for greedy

Sub-optimal yet correct answers Failure to converge altogether Depend on proof of correctness

I have a wonderful proof but the margin is not wide enough to contain it... These proofs are not simple.

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

So ...

When can you use them? It’s complicated. I’m sorry. Some examples include: Kruskal’s algorithm (MST), Huffman Encoding, Dominating set ...

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

So ...

When can you use them? It’s complicated. I’m sorry. Some examples include: Kruskal’s algorithm (MST), Huffman Encoding, Dominating set ...

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

Outline

1

Memoization

2

Greedy Algorithms

3

Example

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

Activity Selection

Given a set S of n activities with start time si and finish time fi You have to find the maximal size subset of S such that no two activities are done at the same time. Any suggestions for a greedy algorithm to solve this problem?

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

Activity Selection (Continued)

We can find a counter examples for almost all greedy algorithms except where you select activities by the least finishing time. So let’s prove this works:

First, we need to show there exists an optimal solution that makes the greedy choice. Secondly, we need to show that if we combine this greedy choice with the optimal solution for the remaining sub-problem we get an optimal solution to the original problem.

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

Activity Selection (Proof)

So let’s prove that there exists an optimal solution that makes the greedy choice: Let A be an optimal solution. Let activity 1 be the greedy choice. If 1 ∈ A, then the proof is done. If 1 / ∈ A, then we need to show that A′ = A − {a} + {1} is another optimal solution that includes activity 1, where a is the activity with the least finish time in A.

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

Activity Selection (Moar)

Since our activities are sorted by finishing time in our algorithm, we have f1 ≤ fa. But if f1 ≤ sa we could add 1 to A and A would thus not be

• ptimal. So f1 > sa and thus 1 and a overlap.

Since f1 ≤ fa we could remove a and add 1 to get another compatible solution A′ = A − {a} + {1} and |A′| = |A|.

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

Activity Selection (Break it down!)

Now we have to prove that combining the greedy choice and the optimal solution to the sub-problem gives us the global optimal solution: Let 1 be the greedy choice. Let S′ be the subset of activities that do not overlap with 1. S′ = {x|sx ≥ f1} Let B be an optimal solution for S′. From the definition of S′, A′ = {1} + B is compatible and a solution to the original problem, but not necessarily an

• ptimal one.

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

Activity Selection (Getting there...)

Using Proof by Contradiction: Assume A′ is not an optimal solution. Let A be an optimal solution that contains 1. So |A′| < |A|, and |A − {1}| > |A′ − {1}| = |B| But A − {1} is also a solution to the problem of S′, contradicting the assumption that B is an optimal solution to S′. Thus A′ is an optimal solution to the original problem.

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

Activity Selection, a Solution

Eventually ... code:

#sort the N activities by the finish time. A = [0] lastadded = 0 for i in range(1,N-1): if s[i] >= f[lastadded]: A.append(i) lastadded = i

Wasn’t that easy? But merely looking at this we have no way of seeing that it is always optimal

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

In summary

Proofs are long, tricky and difficult to create In general, we don’t bother. It pays to simply try and suss things out Attempt to find degenerate cases of the problem and try to break your greedy The more subtle tests it passes, the more likely it is to be correct

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

Problems from a friend

Don’t forget to thank Kosie for any suffering incurred here ;) http://www.topcoder.com/stat?c=problem_ statement&pm=10316

Give N3 copies of a dice with the numbers specified, construct a cube where the sum of the numbers on the

• utside is minimised.

http://icpcres.ecs.baylor.edu/onlinejudge/ external/115/11520.html

Given a grid with some letters filled in, find the lexicographically smallest grid where no two adjacent letters are the same.

http://www.topcoder.com/stat?c=problem_ statement&pm=7534

Given a pipe with leaks at certain locations, cover all the leaks using the least number of L-inch long pieces of adhesive tape.

Tslil Clingman Memoization and Greedy

Memoization Greedy Algorithms Example

Licensing

This lecture was created by Tslil Clingman for UCT Algorithm Circle and is licensed under the Creative Commons Attribution-ShareAlike 3.0 Unported License.

Tslil Clingman Memoization and Greedy