The Divide and Conquer Paradigm &6(�������$OJRULWKPV�DQG� &RPSXWDWLRQDO�&RPSOH[LW\ ❚ Outline: ❙ General Idea Winter 2002 ❙ Review of Merge Sort ❙ Why does it work? Instructor: W. L. Ruzzo ❘ Importance of balance Lectures 9-12 ❘ Importance of super-linear growth Divide and Conquer Algorithms ❙ Two interesting applications ❘ Polynomial Multiplication ❘ Matrix Multiplication 2 1 Algorithm Design Techniques Mergesort (review) ❚ Divide & Conquer Mergesort: (recursively) sort 2 half-lists, ❙ Reduce problem to one or more sub-problems of the then merge results. same type ❙ Typically, each sub-problem is at most a constant fraction of the size of the original problem ❚ T(n)=2T(n/2)+cn, n ≥ 2 ❘ e.g. Mergesort, Binary Search, Strassen’s Algorithm, ❚ T(1)=0 Quicksort (kind of) Log n levels O(n) ❚ Solution: Θ (n log n) work per level 3 4 Why Balanced Subdivision? Another D&C Approach ❚ Alternative "divide & conquer" algorithm: ❚ Suppose we've already invented DumbSort, taking time n 2 ❙ Sort n-1 ❚ Try Just One Level of divide & conquer: ❙ Sort last 1 ❙ Merge them ❙ DumbSort(first n/2 elements) ❙ DumbSort(last n/2 elements) ❚ T(n)=T(n-1)+T(1)+3n for n ≥ 2 ❙ Merge results ❚ Time: (n/2) 2 + (n/2) 2 + n = n 2 /2 + n ❚ T(1)=0 ❚ Solution: 3n + 3(n-1) + 3(n-2) … = Θ (n 2 ) ❙ Almost twice as fast! 5 6 1
Another D&C Approach, cont. Another D&C Approach, cont. ❚ Moral 1: ❚ Moral 3: unbalanced division less good: Two problems of half size are better than one ❙ (.1n) 2 + (.9n) 2 + n = .82n 2 /2 + n full-size problem, even given the O(n) overhead ❘ The 18% savings compounds significantly if you carry of recombining, since the base algorithm has recursion to more levels, actually giving O(nlogn), but with a super-linear complexity. bigger constant. So worth doing if you can’t get 50-50 split, ❚ Moral 2: but balanced is better if you can. If a littl e’s good, then more’s better—two levels ❘ This is intuitively why Quicksort with random splitter is good – badly unbalanced splits are rare, and not instantly fatal. of D&C would be almost 4 times faster, 3 levels ❙ (1) 2 + (n-1) 2 + n = n 2 - 2n + 2 + n almost 8, etc., even though overhead is growing. Best is usually full recursion down to ❘ Little improvement here. some small constant size (balancing "work" vs "overhead"). 7 8 Another D&C Example: Multiplying Faster Notes on Polynomials ❚ On the first HW you analyzed our usual ❙ These are just formal sequences of algorithm for multiplying numbers coefficients so when we show something multiplied by x k it just means shifted k places ❙ Θ (n 2 ) time to the left – basically no work ❚ We can do better! ❙ Usual 3x 2 + 2x + 2 Polynomial x 2 - 3x + 1 ❙ We’ll describe the basic ideas by multiplying Multiplication: polynomials rather than integers 3x 2 + 2x + 2 -9x 3 - 6x 2 - 6x ❙ Advantage is we don’t get confused by worrying about carries at first 3x 4 + 2x 3 + 2x 2 3x 4 - 7x 3 - x 2 - 4x + 2 9 10 Polynomial Multiplication Naive Divide and Conquer ❚ Given: ❚ Assume m=2k x + a 2 x 2 + ... + a k-2 x k-2 + a k-1 x k-1 ) + Degree m-1 polynomials P and Q ❙ P = (a 0 + a 1 ❙ (a k + a k+1 x + … + a m-2 x k-2 + a m-1 x k-1 ) x k ❘ P = a 0 + a 1 x + a 2 x 2 + … + a m-2 x m-2 + a m-1 x m-1 = P 0 + P 1 x k ❘ Q = b 0 + b 1 x+ b 2 x 2 + … + b m-2 x m-2 + b m-1 x m-1 ❙ Q = Q 0 + Q 1 x k ❚ Compute: ❙ Degree 2m-2 Polynomial P Q ❚ P Q = (P 0 +P 1 x k )(Q 0 +Q 1 x k ) = P 0 Q 0 + (P 1 Q 0 +P 0 Q 1 )x k + P 1 Q 1 x 2k ❙ P Q = a 0 b 0 + (a 0 b 1 +a 1 b 0 ) x + (a 0 b 2 +a 1 b 1 +a 2 b 0 ) x 2 +...+ (a m-2 b m-1 +a m-1 b m-2 ) x 2m-3 + a m-1 b m-1 x 2m-2 ❚ 4 sub-problems of size k=m/2 plus linear combining ❚ Obvious Algorithm: ❙ T(m)=4T(m/2)+cm ❙ Compute all a i b j and collect terms ❙ Solution T(m) = O(m 2 ) ❙ Θ (n 2 ) time 11 12 2
Karatsuba: Prod1 Mid Karatsuba’s Algorithm Details Prod2 R ❚ A better way to compute the terms PolyMul(P, Q): 2m-1 m m/2 0 // P, Q are length m =2k vectors, with P[i], Q[i] being ❙ Compute // the coefficient of x i in polynomials P, Q respectively. ❘ P 0 Q 0 Let Pzero be elements 0..k-1 of P; Pone be elements k..m-1 ❘ P 1 Q 1 Qzero, Qone : similar Prod1 = PolyMul(Pzero, Qzero); // result is a (2k-1)-vector ❘ (P 0 +P 1 )(Q 0 +Q 1 ) which is P 0 Q 0 +P 1 Q 0 +P 0 Q 1 +P 1 Q 1 Prod2 = PolyMul(Pone, Qone); // ditto ❙ Then Pzo = Pzero + Pone; // add corresponding elements ❘ P 0 Q 1 +P 1 Q 0 = (P 0 +P 1 )(Q 0 +Q 1 ) - P 0 Q 0 - P 1 Q 1 Qzo = Qzero + Qone; // ditto ❙ 3 sub-problems of size m/2 plus O(m) work Prod3 = polyMul(Pzo, Qzo); // another (2k-1)-vector ❘ T(m) = 3 T(m/2) + cm Mid = Prod3 – Prod1 – Prod2; // subtract corr. elements ❘ T(m) = O(m α ) where α = log 2 3 = 1.59... R = Prod1 + Shift(Mid, m/2) +Shift(Prod2,m) // a (2m-1)-vector Return( R); 13 14 Solve: T(n) = 2 T(n/2) + cn Solve: T(n) = 4 T(n/2) + cn Level Num Size Work Level Num Size Work 0 1=2 0 n cn 0 1=4 0 n cn 1 2=2 1 n/2 2 c n/2 1 4=4 1 n/2 4 c n/2 . 16=4 2 n/4 2 4=2 2 n/4 4 c n/4 2 16 c n/4 . . … … … … … … … … . 2 i c n/2 i 4 i c n/2 i i 2 i n/2 i i 4 i n/2 i . ... . … … … … … … … … 2 k-1 c n/2 k-1 4 k-1 c n/2 k-1 k-1 2 k-1 n/2 k-1 k-1 4 k-1 n/2 k-1 n/2 k =1 2 k T(1) n/2 k =1 4 k T(1) k 2 k k 4 k 15 16 Solve: T(1) = c Solve: T(1) = c T(n) = 3 T(n/2) + cn T(n) = 3 T(n/2) + cn (cont.) Level Num Size Work i i k 0 1=3 0 n cn T ( n ) 3 cn / 2 = ∑ i 0 = 1 3=3 1 n/2 3 c n/2 i i k cn 3 / 2 = ∑ 2 9=3 2 n/4 9 c n/4 i k i 0 = x = ∑ i = 0 … … … … ( ) i k 3 3 i c n/2 i = cn ∑ i 3 i n/2 i i = 0 2 . . ... k + 1 x − 1 . . … … … … ( ) . . k 1 + 3 3 k-1 c n/2 k-1 − 1 x 1 k-1 3 k-1 n/2 k-1 − 2 = cn ( ) 1 n = 2 k ; k = log 2 n n/2 k =1 3 k T(1) k 3 k 3 − ( ) x 1 ≠ 2 i i k 3 cn / 2 ∑ = Total Work: T(n) = i 0 17 18 3
Solve: T(1) = c Solve: T(1) = c T(n) = 3 T(n/2) + cn (cont.) T(n) = 3 T(n/2) + cn (cont.) log n ( ) 3 k 1 2 + 3 2 cn 1 3 cn = − = 2 log n 2 2 ( ) k + 1 3 log n log n 2 cn < 3 2 a b 2 = 3 cn n ( ) ( ) k log n 3 log a b 3 cn = b = b log n 2 3 c 3 = 2 ( ) ( ) log a k log n b 3 log 3 = b b 3 c n = 2 3 cn = k ( ) 2 log a 1 . 59 ... n = b = O n 19 20 Master Divide and Conquer Recurrence Multiplication – The Bottom Line ❚ If T(n)=aT(n/b)+cn k for n>b then ❚ Polynomials ❙ if a>b k then T(n) is ❙ Naïve: Θ (n 2 ) ( n log a ) Θ b ❙ Karatsuba: Θ (n 1.59… ) ❙ if a<b k then T(n) is Θ (n k ) ❙ Best known: Θ (n log n) ❘ "Fast Fourier Transform" ❙ if a=b k then T(n) is Θ (n k log n) ❚ Integers ❙ Similar, but some ugly details re: carries, etc. gives Θ (n log n loglog n), ❚ Works even if it is n/b instead of n/b. ❘ but mostly unused in practice 21 22 Hints towards FFT: Hints towards FFT: I. Interpolation I. Interpolation Given set of values at 5 points Given set of values at 5 points Find unique degree 4 polynomial going through these points 23 24 4
Recommend
More recommend
