Recurrence relations February 03, 2020 Cinda Heeren / Andy Roth / - - PowerPoint PPT Presentation

Recurrence relations

February 03, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 1

Analyzing recursive functions

• On the previous lesson: Geoff's recursion tree analysis for

running time of recursive functions

– e.g. Merge sort running time = number of recursion levels × work at each level – OK, but let's learn about a more precise method

• Recall: recursive functions are defined in terms of themselves

– The running time of some particular recursive call can also be defined in terms of the running time of its own recursive call(s)

• e.g 𝑈 𝑜 = something + 𝑈 a smaller 𝑜 + …

February 03, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 2

Recurrence relations

𝑈 1 ≤ 𝑐 𝑈 𝑜 ≤ 𝑑 + 𝑈 𝑜 − 1

• Analysis

𝑈 𝑜 ≤ 𝑑 + 𝑑 + 𝑈 𝑜 − 2 (by substitution) 𝑈 𝑜 ≤ 𝑑 + 𝑑 + 𝑑 + 𝑈 𝑜 − 3 (by substitution, again) 𝑈 𝑜 ≤ 𝑙 ∙ 𝑑 + 𝑈 𝑜 − 𝑙 (extrapolating, 0 < 𝑙 ≤ 𝑜) 𝑈 𝑜 ≤ 𝑜 − 1 ∙ 𝑑 + 𝑈 1 = 𝑜 − 1 ∙ 𝑑 + 𝑐 for 𝑙 = 𝑜 − 1

• 𝑈 𝑜 ∈ 𝑃 𝑜

February 03, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 3

Example: Recursive max in an array

double arrMax(double arr[], int size, int start) { if (start == sz – 1) return arr[start]; else return max( arr[start], arrMax(arr, size, start + 1) ); }

Merge sort analysis

• Merge sort algorithm

– Split list in half, sort first half, sort second half, merge together 𝑈 1 ≤ 𝑐 𝑈 𝑜 ≤ 2 ∙ 𝑈 𝑜/2 + 𝑑 ∙ 𝑜

• Analysis

𝑈 𝑜 ≤ 2 ∙ 𝑈 𝑜/2 + 𝑑 ∙ 𝑜 ≤ 2 2 ∙ 𝑈 𝑜/4 + 𝑑 𝑜/2 + 𝑑𝑜 = 4 ∙ 𝑈 𝑜/4 + 𝑑 ∙ 𝑜 + 𝑑 ∙ 𝑜 ≤ 4 2 ∙ 𝑈 𝑜/8 + 𝑑 𝑜/4 + 𝑑 ∙ 𝑜 + 𝑑 ∙ 𝑜 = 8 ∙ 𝑈 𝑜/8 + 𝑑 ∙ 𝑜 + 𝑑 ∙ 𝑜 + 𝑑 ∙ 𝑜 ≤ 2𝑙 ∙ 𝑈 𝑜/2𝑙 + 𝑙 ∙ 𝑑 ∙ 𝑜 extrapolating, 1 < 𝑙 ≤ 𝑜 ≤ 𝑜 ∙ 𝑈 1 + 𝑑 ∙ 𝑜 log 𝑜 for 2𝑙 = 𝑜, or 𝑙 = log2 𝑜

• 𝑈 𝑜 ∈ 𝑃 𝑜 log 𝑜

February 03, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 4

Now with more math!

In the numeric domain

• 𝑈 𝑜 = 2 ∙ 𝑈 𝑜 − 1 + 1, 𝑈 0 = 0

February 03, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 5

Solving for exact closed forms

Recursion tree analysis

• Recall the recursion tree analysis for Merge sort

– 𝑈 𝑜 ≤ 𝑑 ∙ 𝑜 log2 𝑜 + 𝑐 ∙ 𝑜

• Prove this result inductively, with the recurrence relation:

𝑈 1 ≤ 𝑐 𝑈 𝑜 ≤ 2 ∙ 𝑈 𝑜/2 + 𝑑 ∙ 𝑜 Correctness of recursion tree

February 03, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 6

Detour – back to sorting

• Selection sort, Insertion sort, Merge sort

– Worst case complexities Θ 𝑜2 , Θ 𝑜2 , Θ 𝑜 log 𝑜 respectively – All sort based on comparison of two values, 𝑏 and 𝑐 – The algorithm performs some action based on the result

• 𝑏 < 𝑐, 𝑏 > 𝑐, 𝑏 ≤ 𝑐, 𝑏 ≥ 𝑐, 𝑏 = 𝑐

– Each comparison is a decision point in the algorithm

• Do one thing if comparison is true, do another if false

– The algorithm can be expressed as a (binary) decision tree

February 03, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 7

Lower bounds on worst case

The big question: Is it possible for a comparison-based sorting algorithm to have better asymptotic worst-case performance than Merge sort?

Lower bounds on sorting

• Given some permutation of items to be sorted, in positions

𝑏, 𝑐, 𝑑, 𝑒, 𝑓 …

February 03, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 8

a b c d …

a < b a < c a < c Y N b < c b < c Y N b < c b < c Y N Y N Y N Y N Y N 𝑏 < 𝑐 < 𝑑 < ⋯ sorted … < 𝑑 < 𝑐 < 𝑏 sorted sorted sorted sorted sorted

… … … … … … The leaves represent the sorted output for some particular input permutation Each path from root to a leaf is the sequence of decisions made to sort some input permutation Longest path: maximum decisions in worst case

Lower bounds on sorting

• How many leaves are there in this decision tree?

– How many starting permutations are there?

• The algorithm must be able to correctly sort every permutation

– 𝑜! input permutations, therefore 𝑜! different paths to a leaf

• If there were fewer than 𝑜! paths, it means the algorithm is unable to

correctly sort some input permutations

• What is the minimum height of a tree with 𝑜! leaves?

– A perfect tree with 𝑜! leaves

February 03, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 9

Decision tree model

Lower bounds on sorting

• A tree with ℎ levels has 2ℎ+1 − 1 nodes

– Bottom level has 2h nodes (i.e. leaves), 2ℎ = 𝑜! – log 2ℎ = log 𝑜!

February 03, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 10

Height of a perfect decision tree

 

 

     

n n n n n i n h

n i n i

log 2 / log 2 2 / log log ! log

2 / 1 1

      

 

 

The worst case of any comparison-based sorting algorithm can be no better than Ω 𝑜 log 𝑜 The longest decision path can be no shorter than this But – there are non-comparison-based sorting algorithms that can perform better (with assumptions). Stay tuned in CPSC 320 (maybe)!

Readings for this lesson

• Epp:

– Chapter 5.6 – 5.7 (Recurrence relations)

• Next class:

– Carrano & Henry: Chapter 15.1 – 15.2 (Tree terminology, binary trees & traversals)

February 03, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 11