Divide-and-conquer, part 2: Master Theorem Russell Impagliazzo and - PowerPoint PPT Presentation

Divide-and-conquer, part 2: Master Theorem Russell Impagliazzo and Miles Jones Thanks to Janine Tiefenbruck http://cseweb.ucsd.edu/classes/sp16/cse21-bd/ April 15, 2016

Summation Lemma Consider the summation � � ��� It behaves differently for different values of . What happens when r < 1 ? A. The sum tends to 0 B. The sum tends to infinity C. The sum converges D. None of the above

Summation Lemma Consider the summation � � ��� It behaves differently for different values of . If then this sum converges. This means that the sum is bounded above by some constant . Therefore � � � � ��� ���

Summation Lemma Consider the summation � � ��� It behaves differently for different values of . What happens when r = 1 ? A. The sum tends to 0 B. The sum tends to infinity C. The sum converges D. None of the above

Summation Lemma Consider the summation � � ��� It behaves differently for different values of . If then this sum is just summing 1 over and over n times. Therefore � � � ��� ���

Summation Lemma Consider the summation � � ��� It behaves differently for different values of . What happens when r > 1 ? A. The sum tends to 0 B. The sum tends to infinity C. The sum converges D. None of the above

Summation Lemma Consider the summation � � ��� It behaves differently for different values of . If then this sum is exponential with base . � � � � � � ��� ���

Summation Lemma Consider the summation � � ��� It behaves differently for different values of . � � � ���

Recall: Merge Sort: WHEN θ(1) ? T Merge (n) is in O(n) ? T Merge (n/2 + n/2) T MS (n/2) T MS (n/2) If T MS (n) is runtime of MergeSort on list of size n, T MS (0) = c 0 T MS (1) = c' T MS (n) = 2T MS (n/2) + cn where c 0 , c, c' are some constants

Master Theorem The Master Theorem is a rule of thumb used to solve recurrence relations like T MS (n) = 2T MS (n/2) + cn In fact, it will solve any recurrence relation of the form for any values of

Master Theorem Theorem: If for some constants , Then �

Master Theorem: Solving the recurrence Size subproblem Size subproblems � Size � subproblems Depth � Size 1 ��� � � subproblems

Master Theorem: Solving the recurrence � subproblems, each � . After levels, there are of size � � � So, during the th level of recursion, the time complexity is � � � � � � � � � �

Master Theorem: Solving the recurrence � subproblems, each of size � . After levels, there are � � � � � � So, during the th level, the time complexity is � � � � � � � After levels, the subproblem size is reduced to 1, which usually is the size � of the base case. So the entire algorithm is a sum of each level. ��� � � � � � ���

Master Theorem: Proof ��� � � � � � ��� � Case 1: � Then we have that and the series converges to a constant so � � �

Master Theorem: Proof ��� � � � � � ��� � Case 2: � Then we have that and so each term is and the series converges to a � � constant so � �

Master Theorem: Proof ��� � � � � � ��� � Case 2: Then the summation is exponential and grows proportional to its last term ��� � � � so � � ��� � � � ��� � � �

Master Theorem Applied to Mergesort � � � � ��� � � � The recursion for the runtime of mergesort is T MS (n) = 2T MS (n/2) + cn So we have that a=2, b=2, and d=1. In this case, so

Merge Sort In terms of worst-case performance, Merge Sort outperforms all other sorting algorithms we've seen. n n 2 n log n 1 000 1 000 000 ~10 000 1 000 000 1 000 000 000 000 ~20 000 000 Divide and conquer wins big!

Divide & Conquer What we saw: Dividing into subproblems each with a fraction of the size was a big win Will this work in other contexts?

Multiplication: WHAT Rosen p. 252 Given two n -digit (or bit) integers a = a n-1 …a 1 a 0 and b = b n-1 …b 1 b 0 return the decimal (or binary) representation of their product. 25 x 17 175 + 250 425

Multiplication: HOW Rosen p. 252 Given two n -digit (or bit) integers a = a n-1 …a 1 a 0 and b = b n-1 …b 1 b 0 return the decimal (or binary) representation of their product. 25 x 17 Compute partial products (using single digit multiplications), 175 shift, then add. + 250 425 How many operations? O(n 2 )

Multiplication: HOW Divide and conquer? Divide n-digit numbers into two n/2-digit numbers. If a = 12345678 and b = 24681357, we can write a = (1234) * 10 4 + (5678) b = (2468) * 10 4 + (1357) To multiply: ( (1234) * 10 4 + (5678) )( (2468) * 10 4 + (1357) )= (1234)(2468) * 10 8 + (1234)(1357) * 10 4 + (2468)(5678) * 10 4 + (1357)(5678)

Multiplication: WHEN One 8-digit multiplication ( 12345678 )( 24681357 ) = ( (1234) * 10 4 + (5678) )( (2468) * 10 4 + (1357) ) = (1234)(2468) * 10 8 + (1234)(1357) * 10 4 + (2468)(5678) * 10 4 + (1357)(5678) Four 4-digit multiplications (plus some shifts, sums)

Multiplication: WHEN One 8-digit multiplication ( 12345678 )( 24681357 ) = ( (1234) * 10 4 + (5678) )( (2468) * 10 4 + (1357) ) = (1234)(2468) * 10 8 + (1234)(1357) * 10 4 + (2468)(5678) * 10 4 + (1357)(5678) Four 4-digit multiplications (plus some shifts, sums) T(n) = 4 T(n/2) + cn with T(1) = c' and c, c' constants

Multiplication: WHEN (Master Theorem) T(n) = 4 T(n/2) + cn with T(1) = c' and c, c' constants Using the Master Theorem in this case, a=4, b=2 and d=1. � and so ��� � � ��� � � � In this case we have Note that this not an improvement on gradeschool multiplication.

Multiplication: HOW Rosen p. 528 Insight: replace one (of the 4) multiplications by (linear time) subtraction

Multiplication: HOW Rosen p. 528 ( 12345678 )( 24681357 ) = ( (1234) * 10 4 + (5678) )( (2468) * 10 4 + (1357) ) = (1234)(2468) * 10 8 + (1234)(1357) * 10 4 + (2468)(5678) * 10 4 + (1357)(5678) (1234)(2468) * (10 8 +10 4 ) + [(1234) - (5678)][ (1357)-(2468) ] * 10 4 + (1357)(5678) * (10 4 +1) Insight: replace one (of the 4) multiplications by (linear time) subtraction

Karatsuba Multiplication: WHEN Rosen p. 528 Instead of T(n) = 4 T(n/2) + cn with T(1) = c' and c, c' constants get T K (n) = 3 T K (n/2) + p n with T K (1) = p' and p, p' constants Using the Master Theorem of this improved algorithm, we have a=3, b=2, and d=1 so � and so �.�� !!!!!! ��� � � ��� � � In this case we have

Karatsuba Multiplication: WHEN Rosen p. 528 n 1.58… better than n 2 Progress since then … 1963: Toom and Cook develop series of algorithms that are time O(n 1+… ). 2007: Furer uses number theory to achieve the best known time for multiplication. 2016: Still open whether there is a linear time algorithm for multiplication.