SLIDE 1
Divide-and-Conquer (D&C)
2
CS141: Intermediate Data Structures and Algorithms Divide and - - PowerPoint PPT Presentation
CS141: Intermediate Data Structures and Algorithms Divide and - - PowerPoint PPT Presentation
CS141: Intermediate Data Structures and Algorithms Divide and Conquer: Design and Analysis Amr Magdy Divide-and-Conquer (D&C) (Problem X) (Problem X) (Problem X) 2 Merge Sort 3 Merge Sort 4 Merge Sort 5 Merge Sort 6 Merge Sort
SLIDE 2
SLIDE 3
Merge Sort
3
SLIDE 4
Merge Sort
4
SLIDE 5
Merge Sort
5
SLIDE 6
Merge Sort
6
SLIDE 7
Merge Sort
7
SLIDE 8
Merge Sort
8
SLIDE 9
Merge Sort
9
SLIDE 10
Merge Sort
10
SLIDE 11
Matrix Multiplication
11
SLIDE 12
Matrix Multiplication
12
SLIDE 13
Simple Matrix Multiplication
13
SLIDE 14
Simple Matrix Multiplication
14
SLIDE 15
Simple Matrix Multiplication
15
SLIDE 16
D&C Matrix Multiplication
16
SLIDE 17
D&C Matrix Multiplication
Implementation details
Partitioning matrices: index calculation not copying
Is this faster than Θ(n3)? How to analyze a recursive algorithm?
17
SLIDE 18
Analyzing Recursive Algorithms
1.
Determine the recursive relation
2.
Analyze the complexity of the recursive relation
Two methods: Recursive tree expansion (or substitution method) Master theorem
18
SLIDE 19
Analyzing Recursive Algorithms: Merge Sort
19
SLIDE 20
Analyzing Recursive Algorithms: Merge Sort
20
……………………………… T(n) …………………………………………… Θ(1) ……………………………… Θ(1) ……………………… T(n/2) …………………. T(n/2) …………………………… Θ(n)
SLIDE 21
Analyzing Recursive Algorithms: Merge Sort
21
……………………………… T(n) …………………………………………… Θ(1) ……………………………… Θ(1) ……………………… T(n/2) …………………. T(n/2) …………………………… Θ(n) T(n) = Θ(1) + Θ(1) + T(n/2) + T(n/2) + Θ(n) T(n) = 2 T(n/2) + Θ(n)
SLIDE 22
Analyzing Recursive Algorithms: Merge Sort
Recursion tree expansion for T(n) = 2 T(n/2) + Θ(n)
22
SLIDE 23
Analyzing Recursive Algorithms: Merge Sort
Recursion tree expansion for T(n) = 2 T(n/2) + Θ(n)
23
SLIDE 24
Analyzing Recursive Algorithms: Merge Sort
Recursion tree expansion for T(n) = 2 T(n/2) + Θ(n)
24
SLIDE 25
Analyzing Recursive Algorithms
General recursion tree
25
SLIDE 26
Analyzing Recursive Algorithms
Recursive Fibonacci algorithm?
26
SLIDE 27
Analyzing Recursive Algorithms: Master Theorem
Master Theorem: Used to solve recurrences in the form T(n) = aT(n/b) + f(n), a >= 1, b > 1, f(n) is a function, T(n) defined on nonnegative integers T(n) bound depends on polynomial comparison between f(n) and 𝑜log𝑐 𝑏 if f(n) polynomial less 𝑜log𝑐 𝑏 T(n) = Θ(𝑜log𝑐 𝑏) if f(n) polynomial equal 𝑜log𝑐 𝑏 T(n) = Θ(𝑜log𝑐 𝑏 log 𝑜) if f(n) polynomial greater 𝑜log𝑐 𝑏 T(n) = Θ(𝑔(𝑜))
27
SLIDE 28
Analyzing Recursive Algorithms: Master Theorem
Master Theorem: Used to solve recurrences in the form T(n) = aT(n/b) + f(n), a >= 1, b > 1, f(n) is a function, T(n) defined on nonnegative integers T(n) bound depends on polynomial comparison between f(n) and 𝑜log𝑐 𝑏 Formally:
28
SLIDE 29
Analyzing Recursive Algorithms: Merge Sort
T(n) = 2 T(n/2) + Θ(n) In the general form of Master theorem, a=2, b=2, f(n)=cn 𝑜log𝑐 𝑏 = 𝑜log2 2 = 𝑜 then f(n) = Θ(𝑜log𝑐 𝑏), case 2 T(n) = Θ(𝑜log𝑐 𝑏 log 𝑜) = Θ(𝑜 log 𝑜)
29
SLIDE 30
Analyzing Recursive Algorithms
Recursive Fibonacci algorithm?
30
SLIDE 31
Analyzing Recursive Algorithms: Matrix Multiplication
T(n) = 8 T(n/2) + Θ(n2)
31
……………..……………………….…… T(n) ……………..…………………………………….….. Θ(1) ……………..………………………………... Θ(1) (2T(n/2) + 𝑜
2 ∗ 𝑜 2)*4
SLIDE 32
Analyzing Recursive Algorithms: Matrix Multiplication
T(n) = 8 T(n/2) + Θ(n2) In the general form of Master theorem, a=8, b=2, f(n)= Θ(n2) 𝑜log𝑐 𝑏 = 𝑜log2 8 = 𝑜3 then f(n) = O(𝑜log𝑐 𝑏−𝜁) = O(𝑜3−1) , ϵ = 1 , case 1 T(n) = Θ(𝑜log𝑐 𝑏) = Θ(𝑜3) D&C matrix multiplication is as fast as the simple matrix multiplication
32
SLIDE 33
Strassen’s Matrix Multiplication
33
SLIDE 34
Strassen’s Matrix Multiplication
Each P has T(n/2) and Θ(n2) (adding two
𝑜 2 𝑦 𝑜 2 matrices)
T(n) = 7T(n/2) + Θ(n2)
34
SLIDE 35
Analyzing Recursive Algorithms: Strassen’s Matrix Multiplication
T(n) = 7 T(n/2) + Θ(n2) In the general form of Master theorem, a=7, b=2, f(n)= Θ(n2) 𝑜log𝑐 𝑏 = 𝑜log2 7 = 𝑜2.807 then f(n) = O(𝑜log𝑐 𝑏−𝜁) = O(𝑜2.807−0.8), ϵ = 0.8, case 1 T(n) = Θ(𝑜log𝑐 𝑏) = Θ(𝑜2.807)
35
SLIDE 36
Analyzing Recursive Algorithms: Strassen’s Matrix Multiplication
T(n) = 7 T(n/2) + Θ(n2) In the general form of Master theorem, a=7, b=2, f(n)= Θ(n2) 𝑜log𝑐 𝑏 = 𝑜log2 7 = 𝑜2.807 then f(n) = O(𝑜log𝑐 𝑏−𝜁) = O(𝑜2.807−0.8), ϵ = 0.8, case 1 T(n) = Θ(𝑜log𝑐 𝑏) = Θ(𝑜2.807)
36
n n2.807 n3 10 641.2096 1000 100 411149.7 1000000 1000 2.64E+08 1E+09 10000 1.69E+11 1E+12 100000 1.08E+14 1E+15 1000000 6.95E+16 1E+18 10000000 4.46E+19 1E+21
SLIDE 37
When Master Theorem Fails?
When 𝑜log𝑐 𝑏 and f(n) are not polynomially comparable Example 1: T(n) = 3 T(n/4) + n log n a=3, b=4, f(n) = n log n 𝑜log𝑐 𝑏 = 𝑜log4 3 = 𝑜0.79 𝑔(𝑜)/𝑜log𝑐 𝑏 = 𝑜0.21 log 𝑜 f(n) is polynomially larger than 𝑜log𝑐 𝑏 and case 3 applies What values of constant c satisfies the condition a f(n/b) <= cf(n)?
37
SLIDE 38
When Master Theorem Fails?
When 𝑜log𝑐 𝑏 and f(n) are not polynomially comparable Example 2: T(n) = 2 T(n/2) + n log n a=2, b=2, f(n) = n log n 𝑜log𝑐 𝑏 = 𝑜log2 2 = 𝑜 𝑔(𝑜)/𝑜log𝑐 𝑏 = log 𝑜 f(n) is not polynomially larger than 𝑜log𝑐 𝑏 (i.e., there is not polynomial factor nx in the ratio 𝑔(𝑜)/𝑜log𝑐 𝑏, 𝑜𝑝 𝜁 > 0 exists) and Master theorem does not apply
38
SLIDE 39
Credits & Book Readings
Book Readings
2.3, Ch. 4 Intro, 4.2, 4.3, 4.4, 4.5
Credits
- Prof. Ahmed Eldawy notes
Online Sources https://upload.wikimedia.org/wikipedia/commons/e/e6/Merge_s
- rt_algorithm_diagram.svg
http://www.geeksforgeeks.org/wp- content/uploads/stressen_formula_new_new.png
39