Objectives Divide and conquer algorithms Recurrence relations - - PDF document

▶
objectives
SMART_READER_LITE
LIVE PREVIEW

Objectives Divide and conquer algorithms Recurrence relations - - PDF document

Mar 15, 2024 225 likes •387 views

3/8/19 Objectives Divide and conquer algorithms Recurrence relations Counting inversions March 8, 2019 CSCI211 - Sprenkle 1 Review What approach are we learning to solve problems (as of Wednesday)? What is the recurrence

slide-1
SLIDE 1

3/8/19 1

Objectives

  • Divide and conquer algorithms

Ø Recurrence relations Ø Counting inversions

March 8, 2019 1 CSCI211 - Sprenkle

Review

  • What approach are we learning to solve

problems (as of Wednesday)?

  • What is the recurrence relation for merge sort?

Ø What is a recurrence relation in general?

March 8, 2019 CSCI211 - Sprenkle 2

slide-2
SLIDE 2

3/8/19 2

Merge Sort’s Recurrence Relation

  • T(n) = number of comparisons to mergesort an

input of size n

  • Goal: put an upperbound on T(n):

March 8, 2019 CSCI211 - Sprenkle 3

For some constant c, T(n) ≤ 2 T(n/2) + cn when n > 2, T(2) ≤ c O(n) Solve T(n) to come up with explicit bound

basecase

Approaches to Solving Recurrences

  • Unroll recursion

Ø Look for patterns in runtime at each level Ø Sum up running times over all levels

  • Substitute guess solution into recurrence

Ø Check that it works Ø Induction on n

March 8, 2019 CSCI211 - Sprenkle 4

slide-3
SLIDE 3

3/8/19 3

Unrolling Recurrence: T(n)

March 8, 2019 CSCI211 - Sprenkle 5

T(n) = 2 T(n/2) + cn

Unrolling Recurrence: 2 T(n/2) + cn

  • First level: 2 T(n/2) + cn

March 8, 2019 CSCI211 - Sprenkle 6

cn T(n/2) T(n/2)

How does the next level break down?

slide-4
SLIDE 4

3/8/19 4

Unrolling Recurrence: 2 T(n/2) + cn

  • Next level:

Each one is 2 T(n/4) + c(n/2)

March 8, 2019 CSCI211 - Sprenkle 7

cn c n/2 c n/2 T(n/4) T(n/4) T(n/4) T(n/4)

Next level?

Unrolling Recurrence

  • Next level:

Each one is 2 T(n/8) + c(n/4)

March 8, 2019 CSCI211 - Sprenkle 8

cn c n/2 c n/2 c n/4 c n/4 c n/4 c n/4

And so on…

T(n/8) T(n/8)

…

T(n/8) T(n/8)

What does the final level look like?

slide-5
SLIDE 5

3/8/19 5

Unrolling Recurrence

  • How much does each level cost, in terms of the level?
  • How many levels are there (assuming n is a power of 2)?
  • What is the total run time?

March 8, 2019 CSCI211 - Sprenkle 9

cn c n/2 c n/2 c n/4 c n/4 c n/4 c n/4 c c c c c c c c T(n / 2k) T(n) T(2) 1 2

Unrolling Recurrence

  • How many levels are there (assuming n is a power of 2)?
  • How much does each level cost, in terms of the level?
  • What is the total run time?

March 8, 2019 CSCI211 - Sprenkle 10

cn c n/2 c n/2 c n/4 c n/4 c n/4 c n/4 c c c c c c c c T(n / 2k) T(n) T(2) 1 2

2k problems Size: n/2k Each level takes 2k * c * (n/2k) = cn Number of levels: log2n O(n log n)

slide-6
SLIDE 6

3/8/19 6

Alternative: Proof by Induction

  • Claim. If T(n) satisfies the recurrence

T(n) ≤ 2 T(n/2) + cn, then T(n) ≤ cn log2 n.

  • Pf. (by induction on n)

Ø Base case: n = 2 Ø Inductive hypothesis: T(n) ≤ cn log2 n Ø Goal: show that T(2n) ≤ 2cn log2 (2n)

March 8, 2019 CSCI211 - Sprenkle 11

Why doubling n?

Proof by Induction

  • Claim. If T(n) satisfies the recurrence

T(n) ≤ 2 T(n/2) + cn, then T(n) ≤ cn log2 n.

  • Pf. (by induction on n)

Ø Inductive hypothesis: T(n) ≤ cn log2 n Ø Goal: show that T(2n) ≤ 2cn log2 (2n)

March 8, 2019 CSCI211 - Sprenkle 12

T(2n) ≤ 2T(n) + c2n ≤ 2cn log2n + 2cn ≤ 2cn (log2(2n)-1) + 2cn ≤ 2cn log2(2n) - 2cn + 2cn ≤ 2cn log2(2n) ✔

  • Replace T(n) w/ induction hypothesis
  • Recurrence relation
  • Log rules: what is the

difference between log2(2n) and log2(n)?

slide-7
SLIDE 7

3/8/19 7

Another Recurrence Relation: Binary Search

  • How does binary search work?
  • What is its recurrence relation?

March 8, 2019 CSCI211 - Sprenkle 13

Analyzing Binary Search

March 8, 2019 CSCI211 - Sprenkle 14

BinarySearch( L[1…n], key ): if len(L) == 1 and L[1] == key: return 1 #return the index else: return NOT_FOUND mid = n/2 if L[mid] == key: return mid #return the index if L[mid] < key: return BinarySearch(L[mid+1:], key) else: return BinarySearch(L[:mid], key)

What is the recurrence relation?

slide-8
SLIDE 8

3/8/19 8

Analyzing Binary Search

March 8, 2019 CSCI211 - Sprenkle 15

BinarySearch( L[1…n], key ): if len(L) == 1 and L[1] == key: return 1 #return the index else: return NOT_FOUND mid = n/2 if L[mid] == key: return mid #return the index if L[mid] < key: return BinarySearch(L[mid+1:], key) else: return BinarySearch(L[:mid], key) What is the recurrence relation? T(n) = T(n/2) + c

Unroll the Recurrence

  • T(n) = T(n/2) + c
  • Which makes the runtime?

March 8, 2019 CSCI211 - Sprenkle 16

slide-9
SLIDE 9

3/8/19 9

Unroll the Recurrence

  • T(n) = T(n/2) + c

Ø Constant work at each level Ø Number of levels: log n

  • Which makes the runtime? O(log n)

March 8, 2019 CSCI211 - Sprenkle 17

Another Recurrence Relation

  • Instead of recursively solving 2 problems, solve q

problems

Ø Size of problems is still n/2

  • Combining solutions is still O(n)

March 8, 2019 CSCI211 - Sprenkle 18

What is the recurrence relation? n n/2 n/2 n/2

Example: q=3:

slide-10
SLIDE 10

3/8/19 10

Another Recurrence Relation

  • Instead of recursively solving 2 problems, solve q

problems

Ø Size of problems is still n/2

  • Combining solutions is still O(n)
  • Recurrence relation:

Ø For some constant c,

T(n) ≤ q T(n/2) + cn when n > 2 T(2) ≤ c

March 8, 2019 CSCI211 - Sprenkle 19

Intuition about running time?

Unrolling Recurrence, q > 2

March 8, 2019 CSCI211 - Sprenkle 20

T(n) ≤ q T(n/2) + cn

slide-11
SLIDE 11

3/8/19 11

Unrolling Recurrence, q > 2

  • First level:

q T(n/2) + cn

March 8, 2019 CSCI211 - Sprenkle 21

cn T(n/2) T(n/2)

…

q

Unrolling Recurrence, q > 2

  • Next level:

q T(n/4) + c(n/2)

March 8, 2019 CSCI211 - Sprenkle 22

cn c n/2 c n/2

…

q T(n/4) T(n/4) T(n/4) T(n/4)

…

q

…

q

…

slide-12
SLIDE 12

3/8/19 12 How much does each level cost, in terms of the level? Number of levels? What is the total run time?

Unrolling Recurrence, q > 2

March 8, 2019 CSCI211 - Sprenkle 23

cn c n/2 c n/2

…

q T(n/4) T(n/4) T(n/4) T(n/4)

…

q

…

q

qk problems at level k Size: n/2k Each level takes qk * c * (n/2k) = (q/2)j cn à Total work per level is increasing as level increases Number of levels: log2n

1

How much does each level cost, in terms of the level? Number of levels? What is the total run time?

Unrolling Recurrence, q > 2

March 8, 2019 CSCI211 - Sprenkle 24

cn c n/2 c n/2

…

q T(n/4) T(n/4) T(n/4) T(n/4)

…

q

…

q 1

T(n) ≤ Σj=0,logn (q/2)j cn Geometric series:

(constant ratio between successive terms) Multiplying previous term by (q/2)

O(n log2 q)

slide-13
SLIDE 13

3/8/19 13

Unrolling the Recurrence

  • Generalize: What are the steps?

March 8, 2019 CSCI211 - Sprenkle 25

Summary

  • Use recurrences to analyze the runtime of divide

and conquer algorithms

  • Need to figure out

Ø Number of sub problems Ø Size of sub problems Ø Number of times divided (number of levels) Ø Cost of merging problems

March 8, 2019 CSCI211 - Sprenkle 26

slide-14
SLIDE 14

3/8/19 14

Know Your Recurrence Relations

Recurrence Algorithm Running Time T(n) = T(n/2) + O(1) T(n) = T(n-1) + O(1) T(n) = 2 T(n/2) + O(1) T(n) = T(n-1) + O(n) T(n) = 2 T(n/2) + O(n)

March 8, 2019 CSCI211 - Sprenkle 27

What algorithm has this recurrence relation? What is that algorithm’s running time?

Looking Ahead

  • Problem Set 7 – due next Friday
  • Wiki – 4.8, 5-5.3

March 8, 2019 CSCI211 - Sprenkle 28