Basic sorting Insertion sort January 15, 2020 Cinda Heeren / Andy - - PowerPoint PPT Presentation

Basic sorting

Insertion sort

January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 1

Announcements

• HW1 out, due next week
• LaTeX workshops – slides available

January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 2

Life is full of mysteries

• What does this function do? What would be a good name for

it?

• Running time?

January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 3

void ___________ (vector<string>& arr, int p) { // assumes arr[0..p-1] are in sorted order string temp = arr[p]; int j = p; while (j > 0 && arr[j-1] > temp) { arr[j] = arr[j-1]; j--; } arr[j] = temp; }

Life is full of mysteries

• Loop invariant?

A. 𝑏𝑠𝑠 0. . 𝑘 − 1 is sorted, and 𝑏𝑠𝑠[𝑘 + 1. . 𝑞] is sorted B. 𝑏𝑠𝑠 0. . 𝑘 − 1 ∪ 𝑏𝑠𝑠[𝑘 + 1. . 𝑞] ∪ 𝑢𝑓𝑛𝑞 form the original elements of 𝑏𝑠𝑠 0. . 𝑞

January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 4

Correctness

void ___________ (vector<string>& arr, int p) { // assumes arr[0..p-1] are in sorted order string temp = arr[p]; int j = p; while (j > 0 && arr[j-1] > temp) { arr[j] = arr[j-1]; j--; } arr[j] = temp; } Try the proof as an exercise! Slide

Insertion sort

• Another simple sorting algorithm

– Divides array into sorted and unsorted parts

• The sorted part of the array is expanded one element at a time

– Find the correct place in the sorted part to place the 1st element of the unsorted part

• By searching through all of the sorted elements

– Move the elements after the insertion point up one position to make space – we insert the element into its correct place (among the items processed so far)

5 January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien

Insertion sort

January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 6

23 41 33 81 7 19 11 45 Locate position for 41 – 1 comparison First element is already "sorted" 23 41 33 81 7 19 11 45 Locate position for 33 – 2 comparisons 23 33 41 81 7 19 11 45 Locate position for 81 – 1 comparison 23 33 41 81 7 19 11 45 Locate position for 7 – 4 comparisons 7 23 33 41 81 19 11 45 Locate position for 19 – 5 comparisons 7 19 23 33 41 81 11 45 Locate position for 11 – 6 comparisons 7 11 19 23 33 41 81 45 Locate position for 45 – 2 comparisons 7 11 19 23 33 41 45 81 Sorted

Insertion sort algorithm

7

vector<string> InsertionSort(vector<string>& arr) { for (int i = 1; i < arr.size(); i++) Slide(arr, i); return arr; }

Main loop: 𝑜 − 1 times

January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien

Cost of Slide: 𝑃 𝑗 ∈ 𝑃 𝑜 Is the cost of InsertionSort 𝑃 𝑜2 ? Θ 𝑜2 ?

Insertion sort cost

8

Sorted Elements Worst-case Search Worst-case Shuffle 1 1 1 2 2 2 … … … 𝑜 − 1 𝑜 − 1 𝑜 − 1 𝑜(𝑜 − 1)/2 𝑜(𝑜 − 1)/2

January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien

Insertion sort best case

• The efficiency of insertion sort is affected by the state of the

array to be sorted

• In the best case the array is already completely sorted!

– No movement of array elements is required – Requires 𝑜 comparisons

9 January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien

Insertion sort worst case

• In the worst case the array is in reverse order
• Every item has to be moved all the way to the front of the

array

– The outer loop runs 𝑜 − 1 times

• In the first iteration, one comparison and move
• In the last iteration, 𝑜 − 1 comparisons and moves
• On average, 𝑜/2 comparisons and moves

– For a total of 𝑜 ∗ (𝑜 − 1) / 2 comparisons and moves

10 January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien

Insertion sort average case

• What is the average case cost?

– Is it closer to the best case? – Or the worst case?

• If random data is sorted, insertion sort is usually closer to the

worst case

– Around 𝑜 ∗ (𝑜 − 1) / 4 comparisons

11 January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien

Insertion sort summary

• When do we use Insertion sort?

– Insertion sort is a good choice when the data are nearly sorted (only a few elements out of place), or when the problem size is small (because it has low overhead)

January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 12

Name Best Average Worst Memory Selection sort 𝑃 𝑜2 𝑃 𝑜2 𝑃 𝑜2 𝑃 1 Insertion sort 𝑃 𝑜 𝑃 𝑜2 𝑃 𝑜2 𝑃 1

Insertion sort correctness

• Base case, i == 1

– arr[0…0] has only one element, so it is always in sorted order – Note: this is only for arrays with at least 1 element

January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 13

vector<string> InsertionSort(vector<string>& arr) { for (int i = 1; i < arr.size(); i++) Slide(arr, i); return arr; }

Loop invariant: Before processing index 𝑗, arr[0…i-1] is sorted

Insertion sort correctness

• Induction hypothesis

– Just before we test k < size, arr[0…k-1] are in sorted order – When the loop starts, i == k

• Induction step:

– Slide shuffles up elements larger than arr[i] and puts arr[i] into place, so arr[0…i] is in sorted order

January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 14

vector<string> InsertionSort(vector<string>& arr) { for (int i = 1; i < arr.size(); i++) Slide(arr, i); return arr; }

Assuming Slide works correctly! Loop invariant: Before processing index 𝑗, arr[0…i-1] is sorted

Insertion sort correctness

• Termination

– Loop ends when i == size – This will happen since size is non-negative and i increases – By the loop invariant, arr[0…i-1] is sorted, and since i == size, arr[0…i-1] is arr[0…size-1] which is the whole array

January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 15

vector<string> InsertionSort(vector<string>& arr) { for (int i = 1; i < arr.size(); i++) Slide(arr, i); return arr; }

This is still entirely dependent on Slide being correct! Loop invariant: Before processing index 𝑗, arr[0…i-1] is sorted

Iterative sorting summary

• Comparing Selection sort and Insertion sort, which is better:

– Asymptotically? – Empirically? – What if the list is already sorted? – What if the list is almost sorted? – What if the list is in reverse order?

January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 16

Readings for this lesson

• Carrano & Henry

– Chapter 11.1.3 (Insertion sort)

• Epp 4e/5e:

– Chapter 5.5 (Loop invariants)

• Next class:

– Carrano & Henry, Chapter 4 (Linked lists) – Highly recommended to read Carrano & Henry Chapter C2.1-C2.3, C2.5 (Pointers, memory allocation)

January 15, 2020 Cinda Heeren / Andy Roth / Geoffrey Tien 17