CS4102 Algorithms Fall 2020 Warm up 2 What is ? =0 Finite - PowerPoint PPT Presentation

CS4102 Algorithms Fall 2020 Warm up 𝑙 2 𝑗 What is ? 𝑗=0

𝑀 𝑏 𝑗 Finite Geometric Series 𝑗=0 If 𝑏 > 1 − − = The series The next term The series The first term multiplied by 𝑏 in the series 1 + 𝑏 + 𝑏 2 + ⋯ + 𝑏 𝑀 𝑏 1 + 𝑏 + 𝑏 2 + ⋯ + 𝑏 𝑀 1 𝑏 𝑀+1 1 2

𝑀 𝑏 𝑗 Finite Geometric Series 𝑗=0 If 𝑏 < 1 − − = The series The next term The series The first term multiplied by 𝑏 in the series 1 + 𝑏 + 𝑏 2 + ⋯ + 𝑏 𝑀 𝑏 1 + 𝑏 + 𝑏 2 + ⋯ + 𝑏 𝑀 1 𝑏 𝑀+1 1 3 Solve for the series

CS4102 Algorithms Fall 2020 Another Warm up 1) Rewrite log 10 𝑜 so the log on 𝑜 has base 2 2) Rewrite 𝑏 log 𝑐 𝑜 so that 𝑜 is not in the exponent

1) Rewrite log 10 𝑜 so the log on 𝑜 has base 2 Log Rules Rewriting 1. log 𝑐 𝑦 + log 𝑐 𝑧 = log 𝑐 𝑦 ⋅ 𝑧 1. log 𝑐 𝑜 2. y ⋅ log 𝑐 𝑦 = log 𝑐 𝑦 𝑧 2. log 𝑐 a log 𝑏 𝑜 (rule 3) 3. 𝑐 log 𝑐 𝑦 = 𝑦 3. log 𝑏 𝑜 ⋅ log 𝑐 𝑏 (rule 2) 4. log 𝑐 𝑐 = 1 5. log 𝑐 1 = 0 log 𝑐 𝑏 is a constant! (doesn’t depend on 𝑜 ) log 𝑐 𝑜 = Θ(log 𝑏 𝑜)

2) Rewrite 𝑏 log 𝑐 𝑜 so that 𝑜 is not in the exponent Log Rules Rewriting 1. 𝑏 log 𝑐 𝑜 1. log 𝑐 𝑦 + log 𝑐 𝑧 = log 𝑐 𝑦 ⋅ 𝑧 2. y ⋅ log 𝑐 𝑦 = log 𝑐 𝑦 𝑧 2. 𝑏 log 𝑐 𝑏 log𝑏 𝑜 (rule 3) 3. 𝑐 log 𝑐 𝑦 = 𝑦 3. 𝑏 (log 𝑏 n)⋅(log 𝑐 𝑏) (rule 2) 4. log 𝑐 𝑐 = 1 𝑏 log 𝑏 𝑜 (log 𝑐 𝑏) 4. (exponentiation rule) 5. log 𝑐 1 = 0 5. 𝑜 log 𝑐 𝑏 (rule 3)

Divide and Conquer • Divide: – Break the problem into multiple subproblems, each smaller instances of the original • Conquer: – If the suproblems are “large”: • Solve each subproblem recursively – If the subproblems are “small”: • Solve them directly (base case) • Combine: – Merge together solutions to subproblems

Analyzing Divide and Conquer 1. Break into smaller subproblems 2. Use recurrence relation to express recursive running time 3. Use asymptotic notation to simplify • Divide: 𝐸(𝑜) time, • Conquer: recurse on small problems, size 𝑡 • Combine: C(𝑜) time • Recurrence: – 𝑈 𝑜 = 𝐸 𝑜 + 𝑈(𝑡) + 𝐷(𝑜)

Recurrence Solving Techniques Tree ? Guess/Check “Cookbook” 9

Analyzing Merge Sort 1. Break into smaller subproblems 2. Use recurrence relation to express recursive running time 3. Use asymptotic notation to simplify • Divide: 0 comparisons • Conquer: recursively solve 2 small subproblems, size 𝑜 2 • Combine: 𝑜 comparisons • Recurrence: 𝑜 – 𝑈 𝑜 = 2 𝑈 2 + 𝑜 10

Recurrence Solving Techniques 1. Draw the recursive structure 2. Label work done within each recursive call Tree 3. Sum work done per recursive depth 4. Find pattern ? 5. Sum work done total Guess/Check “ Cookbook ” 11

Tree method 𝑈 𝑜 = 2𝑈 𝑜 Size of the list 2 + 𝑜 “non - recursive” work done 𝑜 A call to 𝑜 mergesort  𝑜 total / level Recursive Calls 𝑜 𝑜 𝑜 2 𝑜 2 2 2 log 2 𝑜 levels 𝑜 𝑜 𝑜 𝑜 𝑜 4 𝑜 4 𝑜 4 𝑜 4 4 4 4 4 of recursion … … … … … 1 1 1 1 1 1 1 1 1 1 1 1 log 2 𝑜 𝑈 𝑜 = 𝑜 = 𝑜 log 2 𝑜 Base Cases! 𝑗=1 12

Multiplication • Want to multiply large numbers together 4 1 0 2 𝑜 -digit numbers × 1 8 1 9 • What makes a “good” algorithm? • How do we measure input size? • What do we “count” for run time? 13

“Schoolbook” Method How many total 4 1 0 2 multiplications? 𝑜 -digit numbers × 1 8 1 9 3 6 9 1 8 𝑜 mults 4 1 0 2 𝑜 mults 𝑜 levels 3 2 8 1 6 𝑜 mults ⇒ Θ(𝑜 2 ) + 4 1 0 2 𝑜 mults 7 4 6 1 5 3 8 14

Divide and Conquer method 1. Break into smaller subproblems 𝑜 = 10 4 1 0 2 2 + a b a b 𝑜 = 10 × 1 8 1 9 2 c d + c d 10 𝑜 × ( ) + a c 𝑜 10 × × 2 + ( ) + a d b c × ( ) b d 15

D&C Multiplication Pseudocode def dc_mult(x, y): What’s missing?! n = length of larger of x,y a = first n/2 digits of x b = last n/2 digits of x c = first n/2 digits of y d = last n/2 digits of y ac = dc_mult(a, c) ad = dc_mult(a, d) bc = dc_mult(b, c) bd = dc_mult(b, d) return ac*10^n + (ad + bc)*10^(n/2) + bd

D&C Multiplication Pseudocode def dc_mult(x, y): n = length of larger of x,y if n == 1: return x*y a = first n/2 digits of x b = last n/2 digits of x Divide c = first n/2 digits of y d = last n/2 digits of y ac = dc_mult(a, c) ad = dc_mult(a, d) Conquer bc = dc_mult(b, c) bd = dc_mult(b, d) return ac*10^n + (ad + bc)*10^(n/2) + bd Combine

Divide and Conquer Multiplication • Divide: – Break 𝑜 -digit numbers into four numbers of 𝑜 2 digits each (call them 𝑏 , 𝑐 , 𝑑 , 𝑒 ) • Conquer: – If 𝑜 > 1 : • Recursively compute 𝑏𝑑 , 𝑏𝑒 , 𝑐𝑑 , 𝑐𝑒 – If 𝑜 = 1 : (i.e. one digit each) • Compute 𝑏𝑑 , 𝑏𝑒 , 𝑐𝑑 , 𝑐𝑒 directly (base case) • Combine: 𝑜 10 𝑜 𝑏𝑑 + 10 2 𝑏𝑒 + 𝑐𝑑 + 𝑐𝑒 18

Divide and Conquer method How much total arithmetic? 2. Use recurrence relation to express recursive running time 𝑜 10 𝑜 𝑏𝑑 + 10 2 𝑏𝑒 + 𝑐𝑑 + 𝑐𝑒 Recursively solve 𝑈 𝑜 = 4𝑈 𝑜 2 + 5𝑜 19

Divide and Conquer method log 2 𝑜 𝑈 𝑜 = 5𝑜 2 𝑗 3. Use asymptotic notation to simplify 𝑈 𝑜 = 4𝑈 𝑜 2 + 5𝑜 𝑗=0 5𝑜 5𝑜 𝑜 𝑜 𝑜 𝑜 𝑜 4 5𝑜 5𝑜 5𝑜 5𝑜 2 ⋅ 5𝑜 2 2 2 2 2 2 2 2 𝑜 𝑜 𝑜 𝑜 𝑜 𝑜 𝑜 𝑜 16 … 5𝑜 5𝑜 5𝑜 5𝑜 5𝑜 5𝑜 5𝑜 5𝑜 4 ⋅ 5𝑜 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 … … … … … … … … … 2 log 2 𝑜 ⋅ 5𝑜 … 5 5 5 5 5 5 5 5 5 5 5 5 5 5 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20

Divide and Conquer method 3. Use asymptotic notation to simplify 𝑈 𝑜 = 4𝑈 𝑜 2 + 5𝑜 log 2 𝑜 𝑈 𝑜 = 5𝑜 2 𝑗 𝑗=0 𝑈 𝑜 = 5𝑜 2 log 2 𝑜+1 − 1 2 − 1 𝑈 𝑜 = 5𝑜(2𝑜 − 1) = Θ(𝑜 2 ) 21

Karatsuba 1. Break into smaller subproblems 𝑜 = 10 4 1 0 2 2 + a b a b 𝑜 = 10 × 1 8 1 9 2 c d + c d 10 𝑜 × ( ) + a c 𝑜 10 × × 2 + ( ) + a d b c × ( ) b d 22

a b Karatsuba × c d 𝑜 10 𝑜 𝑏𝑑 + 10 2 𝑏𝑒 + 𝑐𝑑 + 𝑐𝑒 This can be Can’t avoid these simplified 𝑏 + 𝑐 𝑑 + 𝑒 = 𝑏𝑑 + 𝑏𝑒 + 𝑐𝑑 + 𝑐𝑒 𝑏𝑒 + 𝑐𝑑 = 𝑏 + 𝑐 𝑑 + 𝑒 − 𝑏𝑑 − 𝑐𝑒 Two One multiplication multiplications 23

Karatsuba Pseudocode def dc_mult(x, y): n = length of larger of x,y if n == 1: return x*y a = first n/2 digits of x b = last n/2 digits of x Divide c = first n/2 digits of y d = last n/2 digits of y ac = dc_mult(a, c) bd = dc_mult(b, d) Conquer adbc = dc_mult(a+b, c+d) – ac – bd return ac*10^n + (adbc)*10^(n/2) + bd Combine

a b How much total Karatsuba × c d arithmetic? 2. Use recurrence relation to express recursive running time 𝑜 10 𝑜 𝑏𝑑 + 10 𝑏 + 𝑐 𝑑 + 𝑒 − 𝑏𝑑 − 𝑐𝑒 + 𝑐𝑒 2 Recursively solve 𝑈 𝑜 = 3𝑈 𝑜 2 + 8𝑜 25

Karatsuba • Divide: – Break 𝑜 -digit numbers into four numbers of 𝑜 2 digits each (call them 𝑏 , 𝑐 , 𝑑 , 𝑒 ) • Conquer: – If 𝑜 > 1 : • Recursively compute 𝑏𝑑 , 𝑐𝑒 , 𝑏 + 𝑐 𝑑 + 𝑒 – If 𝑜 = 1 : • Compute 𝑏𝑑 , 𝑐𝑒 , 𝑏 + 𝑐 𝑑 + 𝑒 directly (base case) • Combine: 𝑜 – 10 𝑜 𝑏𝑑 + 10 𝑏 + 𝑐 𝑑 + 𝑒 − 𝑏𝑑 − 𝑐𝑒 + 𝑐𝑒 2 26

a b Karatsuba Algorithm × c d 1.Recursively compute: 𝑏𝑑 , 𝑐𝑒 , (𝑏 + 𝑐)(𝑑 + 𝑒) 2. 𝑏𝑒 + 𝑐𝑑 = 𝑏 + 𝑐 𝑑 + 𝑒 − 𝑏𝑑 − 𝑐𝑒 𝑜 3. Return 10 𝑜 𝑏𝑑 + 10 2 𝑏𝑒 + 𝑐𝑑 + 𝑐𝑒 𝑈 𝑜 = 3𝑈 𝑜 2 + 8𝑜 Pseudocode 1. 𝑦 ← Karatsuba(𝑏, 𝑑) 2. 𝑧 ← Karatsuba(𝑐, 𝑒) 3. 𝑨 ← Karatsuba(𝑏 + 𝑐, 𝑑 + 𝑒) − 𝑦 − 𝑧 4. Return 10 𝑜 𝑦 + 10 𝑜 2 𝑨 + 𝑧 27

Karatsuba log 2 𝑜 𝑈 𝑜 = 8𝑜 (3 2 ) 𝑗 3. Use asymptotic notation to simplify 𝑈 𝑜 = 3𝑈 𝑜 2 + 8𝑜 𝑗=0 8𝑜 𝑜 8𝑜 ⋅ 1 𝑜 𝑜 𝑜 8𝑜 ⋅ 3 8𝑜 8𝑜 8𝑜 2 2 2 2 2 2 2 𝑜 𝑜 𝑜 𝑜 𝑜 𝑜 8𝑜 ⋅ 9 … 8𝑜 8𝑜 8𝑜 8𝑜 8𝑜 8𝑜 4 4 4 4 4 4 4 4 4 4 4 4 4 … … … … … … … 8𝑜 ⋅ 3 log 2 𝑜 … 8 8 8 8 8 8 8 8 8 8 1 1 1 1 1 1 2 log 2 𝑜 1 1 1 1 28

Karatsuba 3. Use asymptotic notation to simplify 𝑈 𝑜 = 3𝑈 𝑜 2 + 8𝑜 log 2 𝑜 𝑈 𝑜 = 8𝑜 (3 2 ) 𝑗 𝑗=0 (3 2 ) log 2 𝑜+1 −1 𝑈 𝑜 = 8𝑜 3 2 − 1 Lots of arithmetic (see pdf) 𝑈 𝑜 = 24 𝑜 log 2 3 − 16𝑜 = Θ(𝑜 log 2 3 ) ≈ Θ(𝑜 1.585 ) 29

𝑜 2 𝑜 1.585 30

Recurrence Solving Techniques Tree ? Guess/Check (induction) 1. Use Tree method to make a guess at the asymptotic running time “ Cookbook ” 2. Select a specific function with that asymptotic running time 3. Use induction to show the specific function is an upper/lower bound on the actual running time 31