Basic sorting Selection sort Correctness January 18, 2019 Cinda - - PowerPoint PPT Presentation

Basic sorting

Selection sort Correctness

January 18, 2019 Cinda Heeren / Will Evans / Geoffrey Tien 1

Queue operations

• A queue should implement at least the first two of these
• perations:

– enqueue – insert item at the back of the queue – dequeue – remove an item from the front – peek – return the item at the front of the queue without removing it – isEmpty – check if the queue does not contain any items

• Like stacks, it is assumed that these operations will be

implemented efficiently

– That is, in constant time

Cinda Heeren / Will Evans / Geoffrey Tien 2 January 18, 2019

Circular arrays

• Trick: use a circular array to insert and remove items from a

queue in constant time

• The idea of a circular array is that the end of the array “wraps

around” to the start of the array

Cinda Heeren / Will Evans / Geoffrey Tien 3

1 3 2 4 5 6 7 1 2 3 4 5 6 7

January 18, 2019

The modulo operator

• The mod operator (%) calculates remainders:

– 1%5 = 1, 2%5 = 2, 5%5 = 0, 8%5 = 3

• The mod operator can be used to calculate the front and back

positions in a circular array

– Thereby avoiding comparisons to the array size – The back of the queue is:

• (front + num) % arrlength
• where num is the number of items in the queue

– After removing an item, the front of the queue is:

• (front + 1) % arrlength

Cinda Heeren / Will Evans / Geoffrey Tien 4 January 18, 2019

Member attributes: int front; int arrlength; int* arr; int num;

Array queue example

Cinda Heeren / Will Evans / Geoffrey Tien 5

Queue q;

front num

q.enqueue(6);

Insert item at (front + num) % queue.length, then increment num 1 1 2 3 4 5

January 18, 2019

6

Array queue example

Cinda Heeren / Will Evans / Geoffrey Tien 6

Queue q;

front num

q.enqueue(6);

Insert item at (front + num) % queue.length, then increment num 6 5

q.enqueue(4); q.enqueue(7); q.enqueue(3); q.enqueue(8);

4 7 3 8

q.dequeue();

Remove item at front, then decrement num and make front = (front + 1) % queue.length 4 1

q.dequeue();

3 2 1 2 3 4 5

January 18, 2019

Array queue example

Cinda Heeren / Will Evans / Geoffrey Tien 7

front num 2 Insert item at (front + num) % queue.length, then increment num 5 3 7 3 8 Remove item at front, then decrement num and make front = (front + 1) % queue.length 1 2 3 4 5

q.enqueue(9);

9 4

q.enqueue(5);

5 Need to check that the back of the queue does not overtake the front

January 18, 2019

Queue q; q.enqueue(6); q.enqueue(4); q.enqueue(7); q.enqueue(3); q.enqueue(8); q.dequeue(); q.dequeue();

Array queue resizing

• Suppose we have an array-based queue and we have

performed some enqueue and dequeue operations

– Then we perform more enqueues to fill the array – How should we resize the array to allow for more enqueue operations?

January 18, 2019 Cinda Heeren / Will Evans / Geoffrey Tien 8

12 5 76 33 2 41 ? ? ? ? ? ? ? ? ? ? ? ? front

Queue implementation

• Removing items from the front of the queue is straightforward
• Items should be inserted at the back of the queue in constant

time

– So we must avoid traversing through the list – Use a second node pointer to keep track of the node at the back of the queue

• Requires a little extra administration

Cinda Heeren / Will Evans / Geoffrey Tien 9

Using a Linked List

January 18, 2019

Member attributes: Node* front; Node* back; int num;

List queue example

Cinda Heeren / Will Evans / Geoffrey Tien 10

Queue q;

front back

q.enqueue(6);

6

q.enqueue(4);

4

q.enqueue(7);

7

q.enqueue(3);

3

q.dequeue();

January 18, 2019

Selection sort

January 18, 2019 Cinda Heeren / Will Evans / Geoffrey Tien 11

Algorithm analysis – simple sorting

• Suppose we are playing a card game (alcoholic beverages
• ptional) and I have dealt you a random hand of cards

– How might you organize your hand (algorithmically)?

• Selection sort is a simple sorting algorithm that repeatedly

finds the smallest item – we select the smallest item to put into place

– The array is divided into a sorted part and an unsorted part

• Repeatedly swap the first unsorted item with the smallest

unsorted item

– Starting with the element with index 0, and – Ending with last but one element (index 𝑜 − 1) Selection sort

12 January 18, 2019 Cinda Heeren / Will Evans / Geoffrey Tien

Selection sort

January 18, 2019 Cinda Heeren / Will Evans / Geoffrey Tien 13

23 41 33 81 7 19 11 45 Find smallest unsorted item: 7 comparisons 7 41 33 81 23 19 11 45 Find smallest unsorted item: 6 comparisons 7 11 33 81 23 19 41 45 Find smallest unsorted item: 5 comparisons 7 11 19 81 23 33 41 45 Find smallest unsorted item: 4 comparisons 7 11 19 23 81 33 41 45 Find smallest unsorted item: 3 comparisons 7 11 19 23 33 81 41 45 Find smallest unsorted item: 2 comparisons 7 11 19 23 33 41 81 45 Find smallest unsorted item: 1 comparison 7 11 19 23 33 41 45 81 Sorted

Selection sort comparison operations

14

Unsorted elements Comparisons 𝑜 𝑜 − 1 𝑜 − 1 𝑜 − 2 … … 3 2 2 1 1 𝑜 𝑜 − 1 2

January 18, 2019 Cinda Heeren / Will Evans / Geoffrey Tien

Selection sort algorithm

15

void SelectionSort(vector<int>& arr) { for (int i = 0; i < arr.size() -1; ++i) { int smallest = i; // Find the index of the smallest element for (int j = i + 1; j < arr.size(); ++j) { if (arr[j] < arr[smallest]) { smallest = j; } } // Swap the smallest with the current item temp = arr[i];{ arr[i] = arr[smallest]; arr[smallest] = temp; } }

Outer loop: 𝑜 times Inner loop: 𝑜 − 𝑗 times each, 𝑜 𝑜 − 1 2 times total

January 18, 2019 Cinda Heeren / Will Evans / Geoffrey Tien

Selection sort summary

• In broad terms and ignoring the actual number of executable

statements, selection sort

– Makes 𝑜 ∗ 𝑜 − 1 2 comparisons, regardless of the original order of the input – Performs 𝑜 − 1 swaps

• Neither of these operations are substantially affected by the
• rganization of the input
• When might we use Selection sort?

– In asymptotic terms, how many swaps are done? – This might work in extremely limited systems where reads are cheap, but writes are expensive

16 January 18, 2019 Cinda Heeren / Will Evans / Geoffrey Tien

Analyzing loops

• Loop invariants

– Properties of the algorithm or structure that are always true (i.e. do not vary) at particular points in the program – Within a loop body, these properties may become violated briefly, but the rest of the loop instructions should fix the properties for the next iteration

• E.g. we have an algorithm for Selection sort. How can we

prove that it really works for all possible input configurations?

January 18, 2019 Cinda Heeren / Will Evans / Geoffrey Tien 17

Proving algorithm correctness using loop invariants

Selection sort correctness

• The inner loop's purpose is to identify the index of the smallest

value in the unordered portion of the array

– use loop invariant to rigourously prove that this happens

• A partial inner loop progress:

January 18, 2019 Cinda Heeren / Will Evans / Geoffrey Tien 18

Inner loop

int smallest = i; // Find the index of the smallest element for (int j = i + 1; j < arr.size(); ++j) { if (arr[j] < arr[smallest]) smallest = j; }

7 ... 2 12 4 9 6 5 8 3 16 smallest:

14 15 16 17 18 19 20 21 22

𝑗 𝑘

Selection sort

• Invariant property:

– Before iteration 𝑘 of the inner loop, smallest contains the index of the smallest element of 𝑏𝑠𝑠 𝑗. . 𝑘 − 1 – Plain English version: smallest contains the index of the smallest element seen so far – Use induction to build up towards loop termination, where "so far" is the entire unordered subarray

• Base case: 𝑘 = 𝑗 + 1

– Before the first iteration of the inner loop, the invariant states that smallest contains the index of the smallest element of 𝑏𝑠𝑠 𝑗. . 𝑗 – This is set explicitly by the line int smallest = i; – Therefore the loop invariant holds in the base case

January 18, 2019 Cinda Heeren / Will Evans / Geoffrey Tien 19

Inner loop correctness

Selection sort

• Invariant property:

– Before iteration 𝑘 of the inner loop, smallest contains the index of the smallest element of 𝑏𝑠𝑠 𝑗. . 𝑘 − 1

• Maintenance: 𝑘 = 𝑙, 𝑗 ≤ 𝑙 < arr.size()

– Assume loop invariant holds prior to processing index 𝑙, i.e. smallest contains the index of the smallest element of 𝑏𝑠𝑠 𝑗. . 𝑙 − 1 – Case 1: arr[k] < arr[smallest]

• By assumption, 𝑏𝑠𝑠 𝑙 is smaller than any element of 𝑏𝑠𝑠 𝑗. . 𝑙 − 1
• smallest is set to 𝑙, and contains the index of the smallest element of

𝑏𝑠𝑠 𝑗. . 𝑙 , thus loop invariant holds after processing index 𝑙

– Case 2: arr[k] ≮ arr[smallest]

• By assumption, 𝑏𝑠𝑠 𝑙 is not smaller than any element of 𝑏𝑠𝑠 𝑗. . 𝑙 − 1
• No action taken, and smallest contains the index of the smallest element
• f 𝑏𝑠𝑠 𝑗. . 𝑙 , thus loop invariant holds after processing index 𝑙

January 18, 2019 Cinda Heeren / Will Evans / Geoffrey Tien 20

Inner loop correctness

Selection sort

• Invariant property:

– Before iteration 𝑘 of the inner loop, smallest contains the index of the smallest element of 𝑏𝑠𝑠 𝑗. . 𝑘 − 1

• Termination: 𝑘 = arr.size()

– Invariant property states that smallest contains the index of the smallest element of 𝑏𝑠𝑠 𝑗. . 𝑏𝑠𝑠. 𝑡𝑗𝑨𝑓() − 1 – This range describes the entire unordered subarray – Therefore at termination, smallest contains the index of the smallest element in the entire unordered subarray

January 18, 2019 Cinda Heeren / Will Evans / Geoffrey Tien 21

Inner loop correctness

Now that we know the inner loop works properly, we can prove that the outer loop works properly

Selection sort

• The outer loop's purpose is to place the smallest item from the

unordered subarray at the end of the ordered subarray

• A partial outer loop progress:

January 18, 2019 Cinda Heeren / Will Evans / Geoffrey Tien 22

Outer loop correctness 2 3 5 6 8 14 25 18 23 10 20 16

1 2 3 4 5 6 7 8 9 10 11

𝑗

• Invariant property:

– Before iteration 𝑗 of the outer loop, 𝑏𝑠𝑠 0. . 𝑗 − 1 contains the 𝑗 smallest elements of arr in ascending order

Also important to note 𝑏𝑠𝑠 𝑗. . 𝑏𝑠𝑠. 𝑡𝑗𝑨𝑓() − 1 contains the arr. size() − 𝑗 largest elements of arr, unordered, but we will focus on the ordered subarray

Selection sort

• Invariant property:

– Before iteration 𝑗 of the outer loop, 𝑏𝑠𝑠 0. . 𝑗 − 1 contains the 𝑗 smallest elements of arr in ascending order

• Base case: 𝑗 = 0

– Before executing the iteration for 𝑗 = 0, the invariant states that 𝑏𝑠𝑠 0. . −1 contains the 0 smallest elements of arr in ascending order – Since the interval 0. . −1 contains no elements, the invariant holds in the base case

January 18, 2019 Cinda Heeren / Will Evans / Geoffrey Tien 23

Outer loop correctness

Selection sort

• Invariant property:

– Before iteration 𝑗 of the outer loop, 𝑏𝑠𝑠 0. . 𝑗 − 1 contains the 𝑗 smallest elements of arr in ascending order

• Maintenance: 𝑗 = 𝑙, 0 ≤ 𝑙 < arr.size()

– Assume the invariant holds before processing index k, i.e. 𝑏𝑠𝑠 0. . 𝑙 − 1 contains the 𝑙 smallest elements of arr in ascending order – The inner loop correctly identifies the index of the smallest item from 𝑏𝑠𝑠 𝑙. . 𝑏𝑠𝑠. 𝑡𝑗𝑨𝑓() − 1

• This item is at least as large as every item in 𝑏𝑠𝑠 0. . 𝑙 − 1

– This item swaps with arr[k] – Thus after processing index k, 𝑏𝑠𝑠 0. . 𝑙 contains the 𝑙 + 1 smallest elements of arr in ascending order and the loop invariant holds

January 18, 2019 Cinda Heeren / Will Evans / Geoffrey Tien 24

Outer loop correctness

Why?

Selection sort

• Invariant property:

– Before iteration 𝑗 of the outer loop, 𝑏𝑠𝑠 0. . 𝑗 − 1 contains the 𝑗 smallest elements of arr in ascending order

• Termination: 𝑗 = 𝑏𝑠𝑠. 𝑡𝑗𝑨𝑓() − 1

– The invariant states that 𝑏𝑠𝑠 0. . 𝑏𝑠𝑠. 𝑡𝑗𝑨𝑓() − 2 contains the 𝑏𝑠𝑠. 𝑡𝑗𝑨𝑓() − 1 smallest elements of arr in ascending order – 𝑏𝑠𝑠[𝑏𝑠𝑠. 𝑡𝑗𝑨𝑓() − 1] contains a single element at least as large as all elements in 𝑏𝑠𝑠 0. . 𝑏𝑠𝑠. 𝑡𝑗𝑨𝑓() − 2

• Thus 𝑏𝑠𝑠 0. . 𝑏𝑠𝑠. 𝑡𝑗𝑨𝑓() − 1 is in ascending order, and this interval

covers the complete array

January 18, 2019 Cinda Heeren / Will Evans / Geoffrey Tien 25

Outer loop correctness

Readings for this lesson

• Carrano & Henry

– Chapter 11.1.1 (Selection sort)

• Epp

– Chapter 5.5 (Loop invariants, correctness)

• Next class:

– Carrano & Henry: Chapter 11.1.3 (Insertion sort)

January 18, 2019 Cinda Heeren / Will Evans / Geoffrey Tien 26