Basic sorting Insertion sort January 15, 2020 Cinda Heeren / Andy - PowerPoint PPT Presentation

Basic sorting Insertion sort January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 1

Announcements • HW1 out, due next week • LaTeX workshops – slides available January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 2

Life is full of mysteries void ___________ (vector<string>& arr, int p) { // assumes arr[0..p-1] are in sorted order string temp = arr[p]; int j = p; while (j > 0 && arr[j-1] > temp) { arr[j] = arr[j-1]; j--; } arr[j] = temp; } • What does this function do? What would be a good name for it? • Running time? January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 3

Life is full of mysteries Correctness void ___________ (vector<string>& arr, int p) { Slide // assumes arr[0..p-1] are in sorted order string temp = arr[p]; int j = p; while (j > 0 && arr[j-1] > temp) { arr[j] = arr[j-1]; j--; } arr[j] = temp; } • Loop invariant? 𝑏𝑠𝑠 0. . 𝑘 − 1 is sorted, and 𝑏𝑠𝑠[𝑘 + 1. . 𝑞] is sorted A. 𝑏𝑠𝑠 0. . 𝑘 − 1 ∪ 𝑏𝑠𝑠[𝑘 + 1. . 𝑞] ∪ 𝑢𝑓𝑛𝑞 form the original elements of B. 𝑏𝑠𝑠 0. . 𝑞 Try the proof as an exercise! January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 4

Insertion sort • Another simple sorting algorithm – Divides array into sorted and unsorted parts • The sorted part of the array is expanded one element at a time – Find the correct place in the sorted part to place the 1 st element of the unsorted part • By searching through all of the sorted elements – Move the elements after the insertion point up one position to make space – we insert the element into its correct place (among the items processed so far) January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 5

Insertion sort First element is already "sorted" Locate position for 41 – 1 comparison 23 41 33 81 7 19 11 45 Locate position for 33 – 2 comparisons 23 41 33 81 7 19 11 45 Locate position for 81 – 1 comparison 23 33 41 81 7 19 11 45 Locate position for 7 – 4 comparisons 23 33 41 81 7 19 11 45 Locate position for 19 – 5 comparisons 7 23 33 41 81 19 11 45 Locate position for 11 – 6 comparisons 7 19 23 33 41 81 11 45 Locate position for 45 – 2 comparisons 7 11 19 23 33 41 81 45 7 11 19 23 33 41 45 81 Sorted January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 6

Insertion sort algorithm vector<string> InsertionSort(vector<string>& arr) { Main loop: for (int i = 1; i < arr.size(); i++) 𝑜 − 1 times Slide(arr, i); return arr; } Cost of Slide : 𝑃 𝑗 ∈ 𝑃 𝑜 Is the cost of InsertionSort 𝑃 𝑜 2 ? Θ 𝑜 2 ? January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 7

Insertion sort cost Sorted Worst-case Worst-case Search Shuffle Elements 0 0 0 1 1 1 2 2 2 … … … 𝑜 − 1 𝑜 − 1 𝑜 − 1 𝑜(𝑜 − 1)/2 𝑜(𝑜 − 1)/2 January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 8

Insertion sort best case • The efficiency of insertion sort is affected by the state of the array to be sorted • In the best case the array is already completely sorted! – No movement of array elements is required – Requires 𝑜 comparisons January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 9

Insertion sort worst case • In the worst case the array is in reverse order • Every item has to be moved all the way to the front of the array – The outer loop runs 𝑜 − 1 times • In the first iteration, one comparison and move • In the last iteration, 𝑜 − 1 comparisons and moves • On average, 𝑜/2 comparisons and moves – For a total of 𝑜 ∗ (𝑜 − 1) / 2 comparisons and moves January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 10

Insertion sort average case • What is the average case cost? – Is it closer to the best case? – Or the worst case? • If random data is sorted, insertion sort is usually closer to the worst case – Around 𝑜 ∗ (𝑜 − 1) / 4 comparisons January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 11

Insertion sort summary • When do we use Insertion sort? – Insertion sort is a good choice when the data are nearly sorted (only a few elements out of place), or when the problem size is small (because it has low overhead) Name Best Average Worst Memory 𝑃 𝑜 2 𝑃 𝑜 2 𝑃 𝑜 2 𝑃 1 Selection sort 𝑃 𝑜 2 𝑃 𝑜 2 𝑃 𝑜 𝑃 1 Insertion sort January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 12

Insertion sort correctness vector<string> InsertionSort(vector<string>& arr) Loop invariant: { Before processing index 𝑗 , for (int i = 1; i < arr.size(); i++) arr [0…i -1] is sorted Slide(arr, i); return arr; } • Base case, i == 1 – arr [0…0] has only one element, so it is always in sorted order – Note: this is only for arrays with at least 1 element January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 13

Insertion sort correctness vector<string> InsertionSort(vector<string>& arr) Loop invariant: { Before processing index 𝑗 , for (int i = 1; i < arr.size(); i++) arr [0…i -1] is sorted Slide(arr, i); return arr; } • Induction hypothesis – Just before we test k < size , arr [0…k -1] are in sorted order – When the loop starts, i == k • Induction step: Assuming Slide works correctly! – Slide shuffles up elements larger than arr[i] and puts arr[i] into place, so arr [0… i] is in sorted order January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 14

Insertion sort correctness vector<string> InsertionSort(vector<string>& arr) Loop invariant: { Before processing index 𝑗 , for (int i = 1; i < arr.size(); i++) arr [0…i -1] is sorted Slide(arr, i); return arr; } • Termination – Loop ends when i == size – This will happen since size is non-negative and i increases – By the loop invariant, arr [0…i -1] is sorted, and since i == size , arr [0…i -1] is arr [0…size -1] which is the whole array This is still entirely dependent on Slide being correct! January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 15

Iterative sorting summary • Comparing Selection sort and Insertion sort, which is better: – Asymptotically? – Empirically? – What if the list is already sorted? – What if the list is almost sorted? – What if the list is in reverse order? January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 16

Readings for this lesson • Carrano & Henry – Chapter 11.1.3 (Insertion sort) • Epp 4e/5e: – Chapter 5.5 (Loop invariants) • Next class: – Carrano & Henry, Chapter 4 (Linked lists) – Highly recommended to read Carrano & Henry Chapter C2.1-C2.3, C2.5 (Pointers, memory allocation) January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 17