CS4102 Algorithms Fall 2020 Warm up 2 What is ? =0 Finite - - PowerPoint PPT Presentation

β–Ά
cs4102 algorithms
SMART_READER_LITE
LIVE PREVIEW

CS4102 Algorithms Fall 2020 Warm up 2 What is ? =0 Finite - - PowerPoint PPT Presentation

Dec 12, 2023 379 likes β€’776 views

CS4102 Algorithms Fall 2020 Warm up 2 What is ? =0 Finite Geometric Series =0 If > 1 = The series The next term The series The first term multiplied by in the series 1 + + 2

slide-1
SLIDE 1

Warm up What is 2𝑗

𝑙 𝑗=0

?

CS4102 Algorithms

Fall 2020

slide-2
SLIDE 2

Finite Geometric Series

2

= βˆ’ βˆ’

If 𝑏 > 1

The series multiplied by 𝑏 The series The first term 1 + 𝑏 + 𝑏2 + β‹― + 𝑏𝑀 𝑏 1 + 𝑏 + 𝑏2 + β‹― + 𝑏𝑀 1 𝑏𝑀+1 1 The next term in the series

𝑏𝑗

𝑀 𝑗=0

slide-3
SLIDE 3

Finite Geometric Series

3

The series multiplied by 𝑏 The series The first term

= βˆ’ βˆ’

1 + 𝑏 + 𝑏2 + β‹― + 𝑏𝑀 𝑏 1 + 𝑏 + 𝑏2 + β‹― + 𝑏𝑀 1 𝑏𝑀+1 1 The next term in the series

If 𝑏 < 1

𝑏𝑗

𝑀 𝑗=0

Solve for the series

slide-4
SLIDE 4

Another Warm up 1) Rewrite log10 π‘œ so the log on π‘œ has base 2 2) Rewrite 𝑏log𝑐 π‘œ so that π‘œ is not in the exponent

CS4102 Algorithms

Fall 2020

slide-5
SLIDE 5

1) Rewrite log10 π‘œ so the log on π‘œ has base 2

Log Rules

  • 1. log𝑐 𝑦 + log𝑐 𝑧 = log𝑐 𝑦 β‹… 𝑧
  • 2. y β‹… log𝑐 𝑦 = log𝑐 𝑦𝑧
  • 3. 𝑐log𝑐 𝑦 = 𝑦
  • 4. log𝑐 𝑐 = 1
  • 5. log𝑐 1 = 0

Rewriting

  • 1. log𝑐 π‘œ
  • 2. log𝑐 alog𝑏 π‘œ

(rule 3)

  • 3. log𝑏 π‘œ β‹… log𝑐 𝑏

(rule 2) log𝑐 𝑏 is a constant! (doesn’t depend on π‘œ) log𝑐 π‘œ = Θ(log𝑏 π‘œ)

slide-6
SLIDE 6

2) Rewrite 𝑏log𝑐 π‘œ so that π‘œ is not in the exponent

Log Rules

  • 1. log𝑐 𝑦 + log𝑐 𝑧 = log𝑐 𝑦 β‹… 𝑧
  • 2. y β‹… log𝑐 𝑦 = log𝑐 𝑦𝑧
  • 3. 𝑐log𝑐 𝑦 = 𝑦
  • 4. log𝑐 𝑐 = 1
  • 5. log𝑐 1 = 0

Rewriting

  • 1. 𝑏log𝑐 π‘œ
  • 2. 𝑏log𝑐 𝑏log𝑏 π‘œ

(rule 3)

  • 3. 𝑏(log𝑏n)β‹…(log𝑐 𝑏)

(rule 2) 4. 𝑏log𝑏 π‘œ (log𝑐 𝑏) (exponentiation rule)

  • 5. π‘œlog𝑐 𝑏

(rule 3)

slide-7
SLIDE 7

Divide and Conquer

  • Divide:

– Break the problem into multiple subproblems, each smaller instances of the original

  • Conquer:

– If the suproblems are β€œlarge”:

  • Solve each subproblem recursively

– If the subproblems are β€œsmall”:

  • Solve them directly (base case)
  • Combine:

– Merge together solutions to subproblems

slide-8
SLIDE 8

Analyzing Divide and Conquer

  • 1. Break into smaller subproblems
  • 2. Use recurrence relation to express recursive running time
  • 3. Use asymptotic notation to simplify
  • Divide: 𝐸(π‘œ) time,
  • Conquer: recurse on small problems, size 𝑑
  • Combine: C(π‘œ) time
  • Recurrence:

– π‘ˆ π‘œ = 𝐸 π‘œ + π‘ˆ(𝑑) + 𝐷(π‘œ)

slide-9
SLIDE 9

Recurrence Solving Techniques

Tree Guess/Check β€œCookbook”

9

?

slide-10
SLIDE 10

Analyzing Merge Sort

  • 1. Break into smaller subproblems
  • 2. Use recurrence relation to express recursive running time
  • 3. Use asymptotic notation to simplify
  • Divide: 0 comparisons
  • Conquer: recursively solve 2 small subproblems, size π‘œ

2

  • Combine: π‘œ comparisons
  • Recurrence:

– π‘ˆ π‘œ = 2 π‘ˆ

π‘œ 2 + π‘œ

10

slide-11
SLIDE 11

Recurrence Solving Techniques

Tree Guess/Check β€œCookbook”

11

?

  • 1. Draw the recursive structure
  • 2. Label work done within each recursive call
  • 3. Sum work done per recursive depth
  • 4. Find pattern
  • 5. Sum work done total
slide-12
SLIDE 12

Tree method

12

οƒžο€ π‘œ total / level log2 π‘œ levels

  • f recursion

π‘œ

π‘ˆ π‘œ = 2π‘ˆ π‘œ 2 + π‘œ

π‘ˆ π‘œ = π‘œ

log2 π‘œ 𝑗=1

= π‘œ log2 π‘œ

π‘œ 2 π‘œ 2 π‘œ 4 π‘œ 4 π‘œ 4 π‘œ 4

… … … …

1 1 1

…

1 1 1

π‘œ π‘œ 2 π‘œ 2 π‘œ 4 π‘œ 4 π‘œ 4 π‘œ 4 1 1 1 1 1 1 A call to mergesort Size of the list β€œnon-recursive” work done Recursive Calls Base Cases!

slide-13
SLIDE 13

Multiplication

  • Want to multiply large numbers together
  • What makes a β€œgood” algorithm?
  • How do we measure input size?
  • What do we β€œcount” for run time?

13

4 1 0 2 Γ— 1 8 1 9

π‘œ-digit numbers

slide-14
SLIDE 14

β€œSchoolbook” Method

14

4 1 0 2 Γ— 1 8 1 9 3 6 9 1 8 4 1 0 2 3 2 8 1 6 4 1 0 2 7 4 6 1 5 3 8 +

How many total multiplications? π‘œ-digit numbers π‘œ mults π‘œ mults π‘œ mults π‘œ mults π‘œ levels β‡’ Θ(π‘œ2)

slide-15
SLIDE 15

Divide and Conquer method

15

  • 1. Break into smaller subproblems

4 1 0 2 Γ— 1 8 1 9

a b c d a b

+ = 10

π‘œ 2

c d

+ = 10

π‘œ 2

( )

a c

Γ— 10π‘œ

a d

10

π‘œ 2

Γ— ( ) ( )

b c

Γ— +

b d

Γ— + +

slide-16
SLIDE 16

D&C Multiplication Pseudocode

def dc_mult(x, y): n = length of larger of x,y a = first n/2 digits of x b = last n/2 digits of x c = first n/2 digits of y d = last n/2 digits of y ac = dc_mult(a, c) ad = dc_mult(a, d) bc = dc_mult(b, c) bd = dc_mult(b, d) return ac*10^n + (ad + bc)*10^(n/2) + bd

What’s missing?!

slide-17
SLIDE 17

D&C Multiplication Pseudocode

def dc_mult(x, y): n = length of larger of x,y if n == 1: return x*y a = first n/2 digits of x b = last n/2 digits of x c = first n/2 digits of y d = last n/2 digits of y ac = dc_mult(a, c) ad = dc_mult(a, d) bc = dc_mult(b, c) bd = dc_mult(b, d) return ac*10^n + (ad + bc)*10^(n/2) + bd

Divide Conquer Combine

slide-18
SLIDE 18

Divide and Conquer Multiplication

  • Divide:

– Break π‘œ-digit numbers into four numbers of π‘œ 2 digits each (call them 𝑏, 𝑐, 𝑑, 𝑒)

  • Conquer:

– If π‘œ > 1:

  • Recursively compute 𝑏𝑑, 𝑏𝑒, 𝑐𝑑, 𝑐𝑒

– If π‘œ = 1: (i.e. one digit each)

  • Compute 𝑏𝑑, 𝑏𝑒, 𝑐𝑑, 𝑐𝑒 directly (base case)
  • Combine:

10π‘œ 𝑏𝑑 + 10

π‘œ 2 𝑏𝑒 + 𝑐𝑑 + 𝑐𝑒

18

slide-19
SLIDE 19

Divide and Conquer method

19

10π‘œ 𝑏𝑑 + 10

π‘œ 2 𝑏𝑒 + 𝑐𝑑 + 𝑐𝑒 Recursively solve

π‘ˆ π‘œ = 4π‘ˆ π‘œ 2 + 5π‘œ

  • 2. Use recurrence relation to express recursive running time

How much total arithmetic?

slide-20
SLIDE 20

Divide and Conquer method

20

π‘ˆ π‘œ = 4π‘ˆ π‘œ 2 + 5π‘œ

  • 3. Use asymptotic notation to simplify

π‘œ

5π‘œ 5π‘œ 2 5 5π‘œ 2 5π‘œ 2

π‘œ 2 π‘œ 2 π‘œ 2 π‘œ 2

5π‘œ 2 5π‘œ 4 5π‘œ 4 5π‘œ 4 5π‘œ 4 5π‘œ 4 5π‘œ 4 5π‘œ 4

…

π‘œ 4 π‘œ 4 π‘œ 4 π‘œ 4 π‘œ 4 π‘œ 4 π‘œ 4 π‘œ 4

5π‘œ 4 5 5 5 5 5 5 5 5 5 5 5 5 5

… … … … … … … …

1 1 1 1 1 1 1 1 1 1 1 1 1 1

…

5π‘œ 4 2 β‹… 5π‘œ 16 4 β‹… 5π‘œ 2log2 π‘œ β‹… 5π‘œ

…

π‘ˆ π‘œ = 5π‘œ 2𝑗

log2 π‘œ 𝑗=0

slide-21
SLIDE 21

Divide and Conquer method

21

π‘ˆ π‘œ = 4π‘ˆ π‘œ 2 + 5π‘œ

  • 3. Use asymptotic notation to simplify

π‘ˆ π‘œ = 5π‘œ 2𝑗

log2 π‘œ 𝑗=0

π‘ˆ π‘œ = 5π‘œ 2log2 π‘œ+1 βˆ’ 1 2 βˆ’ 1 π‘ˆ π‘œ = 5π‘œ(2π‘œ βˆ’ 1) = Θ(π‘œ2)

slide-22
SLIDE 22

Karatsuba

22

4 1 0 2 Γ— 1 8 1 9

a b c d a b

+ = 10

π‘œ 2

c d

+ = 10

π‘œ 2

( )

a c

Γ— 10π‘œ

a d

10

π‘œ 2

Γ— ( ) ( )

b c

Γ— +

b d

Γ—

  • 1. Break into smaller subproblems

+ +

slide-23
SLIDE 23

Karatsuba

23

10π‘œ 𝑏𝑑 + 10

π‘œ 2 𝑏𝑒 + 𝑐𝑑 + 𝑐𝑒 Can’t avoid these This can be simplified

𝑏 + 𝑐 𝑑 + 𝑒 = 𝑏𝑑 + 𝑏𝑒 + 𝑐𝑑 + 𝑐𝑒 𝑏𝑒 + 𝑐𝑑 = 𝑏 + 𝑐 𝑑 + 𝑒 βˆ’ 𝑏𝑑 βˆ’ 𝑐𝑒

One multiplication Two multiplications

a

Γ—

b c d

slide-24
SLIDE 24

Karatsuba Pseudocode

def dc_mult(x, y): n = length of larger of x,y if n == 1: return x*y a = first n/2 digits of x b = last n/2 digits of x c = first n/2 digits of y d = last n/2 digits of y ac = dc_mult(a, c) bd = dc_mult(b, d) adbc = dc_mult(a+b, c+d) – ac – bd return ac*10^n + (adbc)*10^(n/2) + bd

Divide Conquer Combine

slide-25
SLIDE 25

Karatsuba

25

10π‘œ 𝑏𝑑 + 10

π‘œ 2

𝑏 + 𝑐 𝑑 + 𝑒 βˆ’ 𝑏𝑑 βˆ’ 𝑐𝑒 + 𝑐𝑒

Recursively solve

π‘ˆ π‘œ = 3π‘ˆ π‘œ 2 + 8π‘œ

  • 2. Use recurrence relation to express recursive running time

a

Γ—

b c d

How much total arithmetic?

slide-26
SLIDE 26

Karatsuba

  • Divide:

– Break π‘œ-digit numbers into four numbers of π‘œ 2 digits each (call them 𝑏, 𝑐, 𝑑, 𝑒)

  • Conquer:

– If π‘œ > 1:

  • Recursively compute 𝑏𝑑, 𝑐𝑒, 𝑏 + 𝑐

𝑑 + 𝑒

– If π‘œ = 1:

  • Compute 𝑏𝑑, 𝑐𝑒, 𝑏 + 𝑐

𝑑 + 𝑒 directly (base case)

  • Combine:

– 10π‘œ 𝑏𝑑 + 10

π‘œ 2

𝑏 + 𝑐 𝑑 + 𝑒 βˆ’ 𝑏𝑑 βˆ’ 𝑐𝑒 + 𝑐𝑒

26

slide-27
SLIDE 27

Karatsuba Algorithm

27

a

Γ—

b c d

1.Recursively compute: 𝑏𝑑, 𝑐𝑒, (𝑏 + 𝑐)(𝑑 + 𝑒)

  • 2. 𝑏𝑒 + 𝑐𝑑 = 𝑏 + 𝑐

𝑑 + 𝑒 βˆ’ 𝑏𝑑 βˆ’ 𝑐𝑒

  • 3. Return 10π‘œ 𝑏𝑑 + 10

π‘œ 2 𝑏𝑒 + 𝑐𝑑 + 𝑐𝑒

π‘ˆ π‘œ = 3π‘ˆ π‘œ 2 + 8π‘œ Pseudocode

  • 1. 𝑦 ← Karatsuba(𝑏, 𝑑)
  • 2. 𝑧 ← Karatsuba(𝑐, 𝑒)
  • 3. 𝑨 ← Karatsuba(𝑏 + 𝑐, 𝑑 + 𝑒) βˆ’ 𝑦 βˆ’ 𝑧
  • 4. Return 10π‘œπ‘¦ + 10π‘œ 2

𝑨 + 𝑧

slide-28
SLIDE 28

Karatsuba

28

π‘ˆ π‘œ = 3π‘ˆ π‘œ 2 + 8π‘œ

  • 3. Use asymptotic notation to simplify

π‘œ

8π‘œ 8π‘œ 2 8π‘œ 2 8π‘œ 2

π‘œ 2 π‘œ 2 π‘œ 2

8π‘œ 4 8π‘œ 4 8π‘œ 4 8π‘œ 4 8π‘œ 4 8π‘œ 4

…

π‘œ 4 π‘œ 4 π‘œ 4 π‘œ 4 π‘œ 4 π‘œ 4

8 8 8 8 8 8 8 8 8 8

… … … … … …

1 1 1 1 1 1 1 1 1 1

…

8π‘œ β‹… 1 8π‘œ β‹… 3 2 8π‘œ β‹… 9 4 8π‘œ β‹… 3log2 π‘œ 2log2 π‘œ

…

π‘ˆ π‘œ = 8π‘œ (3 2 )𝑗

log2 π‘œ 𝑗=0

slide-29
SLIDE 29

Karatsuba

29

π‘ˆ π‘œ = 3π‘ˆ π‘œ 2 + 8π‘œ

  • 3. Use asymptotic notation to simplify

π‘ˆ π‘œ = 8π‘œ (3 2 )𝑗

log2 π‘œ 𝑗=0

π‘ˆ π‘œ = 8π‘œ (3 2 )log2 π‘œ+1βˆ’1 3 2 βˆ’ 1

Lots of arithmetic (see pdf)

π‘ˆ π‘œ = 24 π‘œlog2 3 βˆ’ 16π‘œ = Θ(π‘œlog2 3) β‰ˆ Θ(π‘œ1.585)

slide-30
SLIDE 30

30

π‘œ2 π‘œ1.585

slide-31
SLIDE 31

Recurrence Solving Techniques

Tree Guess/Check β€œCookbook”

31

?

(induction)

  • 1. Use Tree method to make a guess at

the asymptotic running time

  • 2. Select a specific function with that

asymptotic running time

  • 3. Use induction to show the specific

function is an upper/lower bound on the actual running time

slide-32
SLIDE 32

Induction (review)

32

Goal: βˆ€π‘™ ∈ β„•, 𝑄(𝑙) holds Base case(s): 𝑄 1 holds Hypothesis: βˆ€π‘¦ ≀ 𝑦0, 𝑄 𝑦 holds Inductive step: show 𝑄 𝑦0 β‡’ 𝑄 𝑦0 + 1

slide-33
SLIDE 33

Guess and Check Intuition

  • Show: π‘ˆ π‘œ ∈ 𝑃(𝑕 π‘œ )
  • Consider: π‘•βˆ— π‘œ = 𝑑 β‹… 𝑕(π‘œ) for some constant 𝑑, i.e. pick π‘•βˆ— π‘œ ∈ 𝑃(𝑕 π‘œ )
  • Goal: show βˆƒπ‘œ0 such that βˆ€π‘œ > π‘œ0, π‘ˆ π‘œ ≀ π‘•βˆ—(π‘œ)

– (definition of big-O)

  • Technique: Induction

– Base cases:

  • show π‘ˆ 1 ≀ π‘•βˆ— 1 , π‘ˆ 2 ≀ π‘•βˆ— 2 , … for a small number of cases (may need additional base cases)

– Hypothesis:

  • βˆ€π‘œ ≀ 𝑦0, π‘ˆ π‘œ ≀ π‘•βˆ—(π‘œ)

– Inductive step:

  • Show π‘ˆ 𝑦0 + 1 ≀ π‘•βˆ— 𝑦0 + 1

33

Inductive step must appeal to a base case or to the inductive hypothesis

slide-34
SLIDE 34

String Finding Guess and Check

  • π‘ˆ π‘œ = 5 + π‘ˆ

π‘œ 2

  • To show: π‘ˆ π‘œ ∈ 𝑃 log π‘œ
  • Idea:

– pick a function π‘•βˆ— π‘œ ∈ 𝑃 log π‘œ , show that π‘ˆ π‘œ < π‘•βˆ— n for sufficiently large π‘œ

  • Let’s try π‘•βˆ— π‘œ = 6 log2 π‘œ
slide-35
SLIDE 35

String Finding Guess and Check

35

π‘ˆ π‘œ = π‘ˆ π‘œ 2 + 5

Goal: π‘ˆ π‘œ ≀ 6 log2 π‘œ = 𝑃(log π‘œ) Base case(s): π‘ˆ 2 = 5 + 1 ≀ 6 π‘ˆ 4 = 5 + 6 ≀ 6 log2 4 Hypothesis: βˆ€π‘œ ≀ 𝑦0, π‘ˆ π‘œ ≀ 6 log2 π‘œ Inductive step: Show that π‘ˆ 𝑦0 + 1 ≀ 6 log2 𝑦0 + 1

slide-36
SLIDE 36

Inductive Step: π‘ˆ 𝑦0 + 1 ≀ 6 log2 𝑦0 + 1

  • 1. π‘ˆ 𝑦0 + 1 = 5 + π‘ˆ

𝑦0+1 2

  • 2. ≀ 5 + 6 log2

𝑦0+1 2

  • 3. < 6 + 6 log2

𝑦0+1 2

  • 4. = 6 1 + log2

𝑦0+1 2

  • 5. = 6 log2 2 + log2

𝑦0+1 2

  • 6. = 6 log2 2 + log2

𝑦0+1 2

  • 7. = 6 log2 𝑦0 + 1
slide-37
SLIDE 37

Mergesort Guess and Check

37

π‘ˆ π‘œ = 2π‘ˆ π‘œ 2 + π‘œ

Goal: π‘ˆ π‘œ ≀ π‘œ log2 π‘œ = 𝑃(π‘œ log π‘œ) Base cases: π‘ˆ 1 = 0 ≀ 0 π‘ˆ 2 = 2 0 + 2 = 2 ≀ 2 log2 2 Hypothesis: βˆ€π‘œ ≀ 𝑦0, π‘ˆ π‘œ ≀ π‘œ log2 π‘œ Inductive step: Show that π‘ˆ 𝑦0 + 1 ≀ 𝑦0 + 1 log2 𝑦0 + 1

slide-38
SLIDE 38

Inductive Step: π‘ˆ 𝑦0 + 1 ≀ 𝑦0 + 1 log2 𝑦0 + 1

  • 1. π‘ˆ 𝑦0 + 1 = 2T

x0+1 2

+ x0 + 1

  • 2. ≀ 2 𝑦0+1

2

log2

𝑦0+1 2

+ 𝑦0 + 1

  • 3. = (𝑦0 + 1) log2

𝑦0+1 2

+ 𝑦0 + 1

  • 4. = 𝑦0 + 1

log2

𝑦0+1 2

+ 1

  • 5. = 𝑦0 + 1

log2 𝑦0 + 1 βˆ’ log2 2 + 1

  • 6. = 𝑦0 + 1

log2 𝑦0 + 1 βˆ’ 1 + 1

  • 7. = 𝑦0 + 1 log2 𝑦0 + 1