Divide-and-conquer, part 2: Master Theorem Russell Impagliazzo and - - PowerPoint PPT Presentation

Divide-and-conquer, part 2: Master Theorem

http://cseweb.ucsd.edu/classes/sp16/cse21-bd/ April 15, 2016 Russell Impagliazzo and Miles Jones Thanks to Janine Tiefenbruck

Summation Lemma

Consider the summation

• It behaves differently for different values of .

What happens when r < 1 ?

• A. The sum tends to 0
• B. The sum tends to infinity
• C. The sum converges
• D. None of the above

Summation Lemma

Consider the summation

• It behaves differently for different values of .

If then this sum converges. This means that the sum is bounded above by some constant . Therefore

Summation Lemma

Consider the summation

• It behaves differently for different values of .

What happens when r = 1 ?

• A. The sum tends to 0
• B. The sum tends to infinity
• C. The sum converges
• D. None of the above

Summation Lemma

Consider the summation

• It behaves differently for different values of .

If then this sum is just summing 1 over and over n times. Therefore

Summation Lemma

Consider the summation

• It behaves differently for different values of .

What happens when r > 1 ?

• A. The sum tends to 0
• B. The sum tends to infinity
• C. The sum converges
• D. None of the above

Summation Lemma

Consider the summation

• It behaves differently for different values of .

If then this sum is exponential with base .

Summation Lemma

Consider the summation

• It behaves differently for different values of .

Recall: Merge Sort: WHEN

θ(1) TMS(n/2) TMS(n/2) TMerge(n/2 + n/2) TMerge(n) is in O(n) ? ? If TMS(n) is runtime of MergeSort on list of size n, TMS(0) = c0 TMS(1) = c' TMS(n) = 2TMS(n/2) + cn where c0, c, c' are some constants

Master Theorem

The Master Theorem is a rule of thumb used to solve recurrence relations like TMS(n) = 2TMS(n/2) + cn In fact, it will solve any recurrence relation of the form for any values of

Master Theorem

Theorem: If for some constants , Then

Master Theorem: Solving the recurrence

Size Size Size

• Size 1

Depth

• subproblem

subproblems

subproblems subproblems

Master Theorem: Solving the recurrence

After levels, there are

subproblems, each

• f size

.

So, during the th level of recursion, the time complexity is

Master Theorem: Solving the recurrence

After levels, there are

subproblems, each of size .

So, during the th level, the time complexity is

• After
• levels, the subproblem size is reduced to 1, which usually is the size
• f the base case.

So the entire algorithm is a sum of each level.

Master Theorem: Proof

• Case 1:
• Then we have that
• and the series converges to a constant so

Master Theorem: Proof

• Case 2:
• Then we have that
• and so each term is and the series converges to a

constant so

Master Theorem: Proof

• Case 2:
• Then the summation is exponential and grows proportional to its last term
• so

Master Theorem Applied to Mergesort

The recursion for the runtime of mergesort is TMS(n) = 2TMS(n/2) + cn So we have that a=2, b=2, and d=1. In this case, so

Merge Sort

In terms of worst-case performance, Merge Sort outperforms all other sorting algorithms we've seen. n n2 n log n 1 000 1 000 000 ~10 000 1 000 000 1 000 000 000 000 ~20 000 000 Divide and conquer wins big!

Divide & Conquer

What we saw: Dividing into subproblems each with a fraction of the size was a big win Will this work in other contexts?

Multiplication: WHAT

Given two n-digit (or bit) integers a = an-1…a1a0 and b = bn-1…b1b0 return the decimal (or binary) representation of their product.

Rosen p. 252 25 x 17 175 + 250 425

Multiplication: HOW

Given two n-digit (or bit) integers a = an-1…a1a0 and b = bn-1…b1b0 return the decimal (or binary) representation of their product.

Rosen p. 252 25 x 17 175 + 250 425 Compute partial products (using single digit multiplications), shift, then add. How many operations? O(n2)

Multiplication: HOW

Divide and conquer? Divide n-digit numbers into two n/2-digit numbers. If a = 12345678 and b = 24681357, we can write a = (1234) * 104 + (5678) b = (2468) * 104 + (1357) To multiply:

((1234) * 104 + (5678))((2468) * 104 + (1357))=

(1234)(2468) * 108 + (1234)(1357) * 104 + (2468)(5678) * 104 + (1357)(5678)

Multiplication: WHEN

(12345678)(24681357)=((1234) * 104 + (5678))((2468) * 104 + (1357))=

(1234)(2468) * 108 + (1234)(1357) * 104 + (2468)(5678) * 104 + (1357)(5678)

One 8-digit multiplication Four 4-digit multiplications (plus some shifts, sums)

Multiplication: WHEN

(12345678)(24681357)=((1234) * 104 + (5678))((2468) * 104 + (1357))=

(1234)(2468) * 108 + (1234)(1357) * 104 + (2468)(5678) * 104 + (1357)(5678)

One 8-digit multiplication Four 4-digit multiplications (plus some shifts, sums) T(n) = 4 T(n/2) + cn with T(1) = c' and c, c' constants

Multiplication: WHEN (Master Theorem)

T(n) = 4 T(n/2) + cn with T(1) = c' and c, c' constants Using the Master Theorem in this case, a=4, b=2 and d=1. In this case we have

and so

• Note that this not an improvement on gradeschool multiplication.

Multiplication: HOW

Rosen p. 528 Insight: replace

• ne (of the 4)

multiplications by (linear time) subtraction

Multiplication: HOW

Rosen p. 528 Insight: replace one (of the 4) multiplications by (linear time) subtraction

(12345678)(24681357)=((1234) * 104 + (5678))((2468) * 104 + (1357))=

(1234)(2468) * 108 + (1234)(1357) * 104 + (2468)(5678) * 104 + (1357)(5678)

(1234)(2468) * (108+104) + [(1234) - (5678)][ (1357)-(2468) ] * 104 + (1357)(5678) * (104+1)

Karatsuba Multiplication: WHEN

Rosen p. 528 Instead of T(n) = 4 T(n/2) + cn with T(1) = c' and c, c' constants get TK(n) = 3 TK(n/2) + p n with TK(1) = p' and p, p' constants Using the Master Theorem of this improved algorithm, we have a=3, b=2, and d=1 so In this case we have

and so . !!!!!!

Karatsuba Multiplication: WHEN

Rosen p. 528

n1.58…

better than n2 Progress since then … 1963: Toom and Cook develop series of algorithms that are time O(n1+…). 2007: Furer uses number theory to achieve the best known time for multiplication. 2016: Still open whether there is a linear time algorithm for multiplication.