Week 3 Oliver Kullmann Divide-and- Conquer Solving Recurrences - PowerPoint PPT Presentation

CS 270 Algorithms Week 3 Oliver Kullmann Divide-and- Conquer Solving Recurrences Merge Sort Solving Recurrences Recursion Divide-and-Conquer 1 Trees Master Theorem Merge Sort Divide-and- Conquer Matrix Solving Recurrences multiplication 2 Tutorial Recursion Trees Master Theorem Divide-and-Conquer 3 Matrix multiplication Tutorial 4

CS 270 General remarks Algorithms Oliver Kullmann Divide-and- Conquer Merge Sort First we continue with an important example for Solving Recurrences Divide-and-Conquer, namely Merge Sort . Recursion Trees Then we present a basic tool for analysing algorithms by Master Theorem Solving Recurrences . Divide-and- Conquer We conclude by considering an example, namely Matrix Matrix multiplication Multiplication . Tutorial Reading from CLRS for week 3 Chapter 4

CS 270 Another example: Merge-Sort Algorithms A sorting algorithm based on divide and conquer. The worst-case Oliver Kullmann running time has a lower order of growth than insertion sort. Divide-and- Conquer Again we are dealing with subproblems of sorting subarrays Merge Sort Solving A [ p . . q ] Initially, p = 1 and q = A . length , but these values Recurrences change again as we recurse through subproblems. Recursion Trees Master Theorem To sort A [ p . . q ]: Divide-and- Conquer Matrix multiplication Divide by splitting into two subarrays A [ p . . . r ] and Tutorial A [ r +1 . . . q ], where r is the halfway point of A [ p . . . q ]. Conquer by recursively sorting the two subarrays A [ p . . . r ] and A [ r +1 . . . q ]. Combine by merging the two sorted subarrays A [ p . . . r ] and A [ r +1 . . . q ] to produce a single sorted subarray A [ p . . . q ]. The recursion bottoms out when the subarray has just 1 element, so that it is trivially sorted.

CS 270 Another example: Merge-Sort Algorithms A sorting algorithm based on divide and conquer. The worst-case Oliver Kullmann running time has a lower order of growth than insertion sort. Divide-and- Conquer Again we are dealing with subproblems of sorting subarrays Merge Sort Solving A [ p . . q ] Initially, p = 1 and q = A . length , but these values Recurrences change again as we recurse through subproblems. Recursion Trees Master Theorem To sort A [ p . . q ]: Divide-and- Conquer Matrix multiplication Divide by splitting into two subarrays A [ p . . . r ] and Tutorial A [ r +1 . . . q ], where r is the halfway point of A [ p . . . q ]. Conquer by recursively sorting the two subarrays A [ p . . . r ] and A [ r +1 . . . q ]. Combine by merging the two sorted subarrays A [ p . . . r ] and A [ r +1 . . . q ] to produce a single sorted subarray A [ p . . . q ]. The recursion bottoms out when the subarray has just 1 element, so that it is trivially sorted.

CS 270 Another example: Merge-Sort Algorithms A sorting algorithm based on divide and conquer. The worst-case Oliver Kullmann running time has a lower order of growth than insertion sort. Divide-and- Conquer Again we are dealing with subproblems of sorting subarrays Merge Sort Solving A [ p . . q ] Initially, p = 1 and q = A . length , but these values Recurrences change again as we recurse through subproblems. Recursion Trees Master Theorem To sort A [ p . . q ]: Divide-and- Conquer Matrix multiplication Divide by splitting into two subarrays A [ p . . . r ] and Tutorial A [ r +1 . . . q ], where r is the halfway point of A [ p . . . q ]. Conquer by recursively sorting the two subarrays A [ p . . . r ] and A [ r +1 . . . q ]. Combine by merging the two sorted subarrays A [ p . . . r ] and A [ r +1 . . . q ] to produce a single sorted subarray A [ p . . . q ]. The recursion bottoms out when the subarray has just 1 element, so that it is trivially sorted.

CS 270 Another example: Merge-Sort Algorithms Oliver Kullmann Divide-and- Conquer Merge Sort Solving Merge-Sort ( A , p , q ) Recurrences Recursion Trees 1 if p < q / / check for base case Master Theorem 2 r = ⌊ ( p + q ) / 2 ⌋ / divide / Divide-and- 3 Merge-Sort ( A , p , r ) / / conquer Conquer Matrix 4 Merge-Sort ( A , r +1 , q ) / conquer multiplication / Tutorial 5 Merge ( A , p , r , q ) / / combine Initial call: Merge-Sort ( A , 1 , A . length )

CS 270 Merge Algorithms Oliver Kullmann Divide-and- Conquer Input: Array A and indices p , r , q such that Merge Sort Solving Recurrences p ≤ r < q Recursion Trees Subarrays A [ p . . r ] and subarray A [ r +1 . . q ] are sorted. By Master Theorem the restriction on p , r , q neither subarray is empty. Divide-and- Conquer Matrix Output: The two subarrays are merged into a single sorted multiplication Tutorial subarray in A [ p . . q ]. We implement is so that it takes Θ( n ) time, with n = q − p + 1 = the number of elements being merged.

CS 270 Algorithms Merge ( A , p , r , q ) Oliver Kullmann 1 n 1 = r − p + 1 Divide-and- 2 n 2 = q − r Conquer 3 let L [1 . . n 1 +1] and R [1 . . n 2 +1] be new arrays Merge Sort Solving 4 for i = 1 to n 1 Recurrences 5 L [ i ] = A [ p + i − 1] Recursion Trees Master 6 for j = 1 to n 2 Theorem 7 R [ j ] = A [ r + j ] Divide-and- Conquer 8 L [ n 1 +1] = R [ n 2 +1] = ∞ Matrix multiplication 9 i = j = 1 Tutorial 10 for k = p to q 11 if L [ i ] ≤ R [ j ] 12 A [ k ] = L [ i ] 13 i = i +1 14 else A [ k ] = R [ j ] 15 j = j +1

CS 270 Analysis of Merge-Sort Algorithms Oliver Kullmann Divide-and- Conquer The runtime T ( n ), where n = q − p +1 > 1, satisfies: Merge Sort Solving Recurrences T ( n ) = 2 T ( n / 2) + Θ( n ) . Recursion Trees Master Theorem We will show that T ( n ) = Θ( n lg n ). Divide-and- Conquer It can be shown (see tutorial-section) that Ω( n lg n ) Matrix multiplication comparisons are necessary in the worst case to sort n Tutorial numbers for any comparison-based algorithm: this is thus an (asymptotic) lower bound on the problem. Hence Merge-Sort is provably (asymptotically) optimal.

CS 270 Analysing divide-and-conquer algorithms Algorithms Oliver Recall the divide-and-conquer paradigm: Kullmann Divide the problem into a number of subproblems that are Divide-and- Conquer smaller instances of the same problem. Merge Sort Solving Conquer the subproblems by solving them recursively. Recurrences Recursion Base case: If the subproblem are small enough, just solve Trees Master them by brute force. Theorem Divide-and- Combine the subproblem solutions to give a solution to the Conquer Matrix original problem. multiplication Tutorial We use recurrences to characterise the running time of a divide-and-conquer algorithm. Solving the recurrence gives us the asymptotic running time. A recurrence is a function defined in terms of one or more base cases, and itself, with smaller arguments

CS 270 Analysing divide-and-conquer algorithms Algorithms Oliver Recall the divide-and-conquer paradigm: Kullmann Divide the problem into a number of subproblems that are Divide-and- Conquer smaller instances of the same problem. Merge Sort Solving Conquer the subproblems by solving them recursively. Recurrences Recursion Base case: If the subproblem are small enough, just solve Trees Master them by brute force. Theorem Divide-and- Combine the subproblem solutions to give a solution to the Conquer Matrix original problem. multiplication Tutorial We use recurrences to characterise the running time of a divide-and-conquer algorithm. Solving the recurrence gives us the asymptotic running time. A recurrence is a function defined in terms of one or more base cases, and itself, with smaller arguments

CS 270 Analysing divide-and-conquer algorithms Algorithms Oliver Recall the divide-and-conquer paradigm: Kullmann Divide the problem into a number of subproblems that are Divide-and- Conquer smaller instances of the same problem. Merge Sort Solving Conquer the subproblems by solving them recursively. Recurrences Recursion Base case: If the subproblem are small enough, just solve Trees Master them by brute force. Theorem Divide-and- Combine the subproblem solutions to give a solution to the Conquer Matrix original problem. multiplication Tutorial We use recurrences to characterise the running time of a divide-and-conquer algorithm. Solving the recurrence gives us the asymptotic running time. A recurrence is a function defined in terms of one or more base cases, and itself, with smaller arguments

CS 270 Examples for recurrences Algorithms Oliver � Kullmann 1 if n = 1 T ( n ) = Divide-and- T ( n − 1) + 1 if n > 1 Conquer Merge Sort Solution: T ( n ) = n . Solving Recurrences � 1 if n = 1 Recursion Trees T ( n ) = Master 2 T ( n / 2) + n if n > 1 Theorem Divide-and- Conquer Solution: T ( n ) = n lg n + n . Matrix multiplication � 0 if n = 2 Tutorial T ( n ) = T ( √ n ) + 1 if n > 2 Solution: T ( n ) = lg lg n . � 1 if n = 1 T ( n ) = T ( n / 3) + T (2 n / 3) + n if n > 1 Solution: T ( n ) = Θ( n lg n ).

CS 270 Examples for recurrences Algorithms Oliver � Kullmann 1 if n = 1 T ( n ) = Divide-and- T ( n − 1) + 1 if n > 1 Conquer Merge Sort Solution: T ( n ) = n . Solving Recurrences � 1 if n = 1 Recursion Trees T ( n ) = Master 2 T ( n / 2) + n if n > 1 Theorem Divide-and- Conquer Solution: T ( n ) = n lg n + n . Matrix multiplication � 0 if n = 2 Tutorial T ( n ) = T ( √ n ) + 1 if n > 2 Solution: T ( n ) = lg lg n . � 1 if n = 1 T ( n ) = T ( n / 3) + T (2 n / 3) + n if n > 1 Solution: T ( n ) = Θ( n lg n ).