Recurrence Relations Sorting overview After today, you should be - - PowerPoint PPT Presentation

Recurrence Relations Sorting overview

After today, you should be able to… …write recurrences for code snippets …solve recurrences using telescoping, recurrence trees, and the master method

T(N) =      θ(N logba) if a > bk θ(N klogN) if a = bk θ(N k) if a < bk

A technique for analyzing recursive algorithms

} An equation (or inequality) that relates the

Nth element of a sequence to certain of its predecessors (recursive case)

} Includes an initial condition (base case) } So

Solu lutio ion: A function of N.

• Example. Solve using backward substitution.

T(N) = 2T(N/2) + N T(1) = 1

For Forwar ard su subst stitution

• n

Ba Backward substitution Re Recu currence ce trees Te Telesco coping Ma Master The Theorem

Simple Often can’t solve difficult relations Visual Great intuition for div-and-conquer Widely applicable Difficult to formulate Not intuitive Immediate Only for div-and-conquer Only gives Big-Theta

What’s N?

What’s N?

1

void sort(a) { sort(a, a.length-1); } void sort(a, last) { if (last == 0) return; find max value in a from 0 to last swap max to last sort(a, last-1) }

} Basic idea: tweak the relation somehow so

successive terms cancel

} Example: T(1) = 1, T(N) = 2T(N/2) + N

where N = 2k for some k

} Divide by N to get a “piece of the telescope”: 2–3

4

N N/2 N/4 N/4 N/2 N/4 N/4

⋮ ⋮ ⋮ ⋮ ⋮

1 1 1 1 1 1 1 1 1 … 1 1 1

Level 1 2 ⋮ ?

• How many nodes at level i?
• How much work at level i?
• Index of last level?

2i 2i (N/2i) = N log2 N

Total:

Recurrence: T(N) = 2T(N/2) + N T(1) = 1

} For Divide-and-conquer algorithms

• Divide data into two or more parts of the same size
• Solve problem on one or more of those parts
• Combine "parts" solutions to solve whole problem

} Examples

• Binary search
• Merge Sort
• MCSS recursive algorithm we studied last time

Th Theorem m 7. 7.5 5 in Weiss

5

Theorem 7.5 in Weiss

5–7 } For any recurrence in the form: } The solution is

Example: 2T(N/4) + N

T(N) =      θ(N logba) if a > bk θ(N klogN) if a = bk θ(N k) if a < bk

T(N) = aT(N/b) + θ(N k) with a ≥ 1, b > 1

cNk c(N/b)k c(N/b2)k c(N/b2)k c(N/b2)k c(N/b)k c(N/b2)k c(N/b2)k c(N/b2)k c(N/b)k c(N/b2)k c(N/b2)k c(N/b2)k

⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮

c c c c c c c c c c c c … c c c

• How many nodes at level i?
• How much work at level i?
• Index of last level?

ai ai c(N/bi)k = cNk(a/bk)i logb N

Recurrence: T(N) = aT(N/b) + cNk T(1) ≤ c … … … … …

Level 1 2 ⋮ ?

Summation:

} Upper bound on work at level i: } a = “Rate of subproblem proliferation”

😠

} bk = “Rate of work shrinkage”

😄

Ca Case 😠 a < bk 😄 😠 a = bk 😄 😠 a > bk 😄 As As level i in increases… 😄 work goes down! 😑 work stays same 😠 work goes up! T( T(N) dominated by by work do done at at… Root of tree Every level similar Leaves of tree Ma Master Th Theorem sa says s T(N) N) in in… Θ(Nk) Θ(Nk lo log N) Θ(Nlo

logba)

} Case 1. a < bk } Case 2. a = bk } Case 3. a > bk

cN k

logb N

X

i=0

1 = cN k(logb N + 1)

cN k

logb N

X

i=0

⇣ a bk ⌘i

cN k ✓(a/bk)logb N+1 − 1 (a/bk) − 1 ◆ ≈ cN k(a/bk)logb N = calogb N = cN logb a

cN k ✓1 − (a/bk)logb N+1 1 − (a/bk) ◆ ≈ cN k ✓ 1 1 − (a/bk) ◆

} Analyze code to determine relation

• Base case in code gives base case for relation
• Number and “size” of recursive calls determine

recursive part of recursive case

• Non-recursive code determines rest of recursive

case

} Apply a strategy

• Guess and check (substitution)
• Telescoping
• Recurrence tree
• Master theorem

Quick look at several sorting methods Focus on quicksort Quicksort average case analysis

} Name as many as you can } How does each work? } Running time for each (sorting N items)?

• best
• worst
• average
• extra space requirements

} Spend 10 minutes with a group of three, answering

these questions. Then we will summarize

Put list on board 8–10 10

} Some possible answers (C

(Collect them on the boar ard)

• Bubble sort

(Do Don't 't say ay th the b-wo word rd!)

• Insertion sort Li

Like so sorting files s in manila fo fold lders

• Selection sort

Se Sele lect the lar largest, t , then th the seco cond lar largest, …

• Merge sort Sp

Split lit, r , recursively s sort, , me merge

• Binary tree sort In

Insert all all in into to BST ST, th then in inOrder trav aversal al

• (Quicksort)

Not Not so so elementary. We’ll do it in det detail – http://students.ceid.upatras.gr/~pirot/java/Quicksort/

• (Heapsort) We

We’ll ll als also do th this is one in in det detail

• (Shellsort) In

Interesting va variation on

• n ins

nsertion

• n so

sort

• (Radix sort)

An Anoth ther one

• ne tha

hat we we’ll consider r in some det detail

Best, worst, average time? Extra space requirements?

http://www.xkcd.com/1185/

Stacksort connects to StackOverflow, searches for “sort a list”, and downloads and runs code snippets until the list is sorted.