Recurrence relations February 03, 2020 Cinda Heeren / Andy Roth / - PowerPoint PPT Presentation

Recurrence relations February 03, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 1

Analyzing recursive functions • On the previous lesson: Geoff's recursion tree analysis for running time of recursive functions – e.g. Merge sort running time = number of recursion levels × work at each level – OK, but let's learn about a more precise method • Recall: recursive functions are defined in terms of themselves – The running time of some particular recursive call can also be defined in terms of the running time of its own recursive call(s) • e.g 𝑈 𝑜 = something + 𝑈 a smaller 𝑜 + … February 03, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 2

Recurrence relations Example: Recursive max in an array double arrMax(double arr[], int size, int start) { if (start == sz – 1) return arr[start]; else return max( arr[start], arrMax(arr, size, start + 1) ); } 𝑈 1 ≤ 𝑐 𝑈 𝑜 ≤ 𝑑 + 𝑈 𝑜 − 1 • Analysis 𝑈 𝑜 ≤ 𝑑 + 𝑑 + 𝑈 𝑜 − 2 (by substitution) 𝑈 𝑜 ≤ 𝑑 + 𝑑 + 𝑑 + 𝑈 𝑜 − 3 (by substitution, again) 𝑈 𝑜 ≤ 𝑙 ∙ 𝑑 + 𝑈 𝑜 − 𝑙 (extrapolating, 0 < 𝑙 ≤ 𝑜 ) 𝑈 𝑜 ≤ 𝑜 − 1 ∙ 𝑑 + 𝑈 1 = 𝑜 − 1 ∙ 𝑑 + 𝑐 for 𝑙 = 𝑜 − 1 • 𝑈 𝑜 ∈ 𝑃 𝑜 February 03, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 3

Merge sort analysis Now with more math! • Merge sort algorithm – Split list in half, sort first half, sort second half, merge together 𝑈 1 ≤ 𝑐 𝑈 𝑜 ≤ 2 ∙ 𝑈 𝑜/2 + 𝑑 ∙ 𝑜 • Analysis 𝑈 𝑜 ≤ 2 ∙ 𝑈 𝑜/2 + 𝑑 ∙ 𝑜 ≤ 2 2 ∙ 𝑈 𝑜/4 + 𝑑 𝑜/2 + 𝑑𝑜 = 4 ∙ 𝑈 𝑜/4 + 𝑑 ∙ 𝑜 + 𝑑 ∙ 𝑜 ≤ 4 2 ∙ 𝑈 𝑜/8 + 𝑑 𝑜/4 + 𝑑 ∙ 𝑜 + 𝑑 ∙ 𝑜 = 8 ∙ 𝑈 𝑜/8 + 𝑑 ∙ 𝑜 + 𝑑 ∙ 𝑜 + 𝑑 ∙ 𝑜 ≤ 2 𝑙 ∙ 𝑈 𝑜/2 𝑙 + 𝑙 ∙ 𝑑 ∙ 𝑜 extrapolating, 1 < 𝑙 ≤ 𝑜 for 2 𝑙 = 𝑜 , or 𝑙 = log 2 𝑜 ≤ 𝑜 ∙ 𝑈 1 + 𝑑 ∙ 𝑜 log 𝑜 • 𝑈 𝑜 ∈ 𝑃 𝑜 log 𝑜 February 03, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 4

In the numeric domain Solving for exact closed forms • 𝑈 𝑜 = 2 ∙ 𝑈 𝑜 − 1 + 1 , 𝑈 0 = 0 February 03, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 5

Recursion tree analysis Correctness of recursion tree • Recall the recursion tree analysis for Merge sort – 𝑈 𝑜 ≤ 𝑑 ∙ 𝑜 log 2 𝑜 + 𝑐 ∙ 𝑜 • Prove this result inductively, with the recurrence relation: 𝑈 1 ≤ 𝑐 𝑈 𝑜 ≤ 2 ∙ 𝑈 𝑜/2 + 𝑑 ∙ 𝑜 February 03, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 6

Detour – back to sorting Lower bounds on worst case • Selection sort, Insertion sort, Merge sort – Worst case complexities Θ 𝑜 2 , Θ 𝑜 2 , Θ 𝑜 log 𝑜 respectively – All sort based on comparison of two values, 𝑏 and 𝑐 – The algorithm performs some action based on the result • 𝑏 < 𝑐 , 𝑏 > 𝑐 , 𝑏 ≤ 𝑐 , 𝑏 ≥ 𝑐 , 𝑏 = 𝑐 – Each comparison is a decision point in the algorithm • Do one thing if comparison is true, do another if false – The algorithm can be expressed as a (binary) decision tree The big question: Is it possible for a comparison-based sorting algorithm to have better asymptotic worst-case performance than Merge sort? February 03, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 7

Lower bounds on sorting • Given some permutation of items to be sorted, in positions 𝑏, 𝑐, 𝑑, 𝑒, 𝑓 … … a b c d Each path from root to a The leaves represent the leaf is the sequence of sorted output for some decisions made to sort particular input permutation a < b N Y some input permutation a < c a < c N N Y Y b < c b < c b < c b < c N N N N Y Y Y Y … … … … … sorted … sorted sorted … < 𝑑 < 𝑐 < 𝑏 𝑏 < 𝑐 < 𝑑 < ⋯ sorted Longest path: maximum sorted decisions in worst case sorted February 03, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 8

Lower bounds on sorting Decision tree model • How many leaves are there in this decision tree? – How many starting permutations are there? • The algorithm must be able to correctly sort every permutation – 𝑜! input permutations, therefore 𝑜! different paths to a leaf • If there were fewer than 𝑜! paths, it means the algorithm is unable to correctly sort some input permutations • What is the minimum height of a tree with 𝑜! leaves? – A perfect tree with 𝑜! leaves February 03, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 9

Lower bounds on sorting Height of a perfect decision tree • A tree with ℎ levels has 2 ℎ+1 − 1 nodes – Bottom level has 2 h nodes (i.e. leaves), 2 ℎ = 𝑜! – log 2 ℎ = log 𝑜!      h log n ! n   log i The longest decision path  i 1 can be no shorter than this n / 2     log n / 2  i 1 The worst case of any comparison-based n     log n / 2 sorting algorithm can be no better than 2 Ω 𝑜 log 𝑜     n log n But – there are non-comparison-based sorting algorithms that can perform better (with assumptions). Stay tuned in CPSC 320 (maybe)! February 03, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 10

Readings for this lesson • Epp: – Chapter 5.6 – 5.7 (Recurrence relations) • Next class: – Carrano & Henry: Chapter 15.1 – 15.2 (Tree terminology, binary trees & traversals) February 03, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 11