Quicksort Quicksort In-place Merge Sort g T(n) = (n lg(n)) Not - PowerPoint PPT Presentation

Sorting Review g Introduction to Algorithms Introduction to Algorithms � Insertion Sort � T(n) = Θ (n 2 ) Quicksort Quicksort � In-place � Merge Sort g � T(n) = Θ (n lg(n)) � Not in-place CSE 680 � Selection Sort (from homework) � Selection Sort (from homework) � T(n) = Θ (n 2 ) Prof. Roger Crawfis � In-place Seems pretty good. Can we do better? Can we do better? � Heap Sort � Heap Sort � T(n) = Θ (n lg(n)) � In-place Sorting g Comparison Sorting p g � Assumptions � Assumptions � Given a set of n values there can be n! � Given a set of n values, there can be n! permutations of these values. No knowledge of the keys or numbers we 1. are sorting on. are sorting on � So if we look at the behavior of the � So if we look at the behavior of the Each key supports a comparison interface sorting algorithm over all possible n! 2. or operator. or operator. inputs we can determine the worst-case inputs we can determine the worst case Sorting entire records, as opposed to 3. complexity of the algorithm. numbers, is an implementation detail. , p Each key is unique (just for convenience). 4. Comparison Sorting Comparison Sorting

Decision Tree Decision Tree Model � Decision tree model 1:2 ≤ > � Full binary tree 2:3 1:3 > � A full binary tree (sometimes proper binary tree or 2- ≤ ≤ > tree ) is a tree in which every node other than the leaves h has two children t hild <1,2,3> 1:3 <2,1,3> 2:3 ≤ ≤ > > � Internal node represents a comparison. � Ignore control, movement, and all other operations, just <1,3,2> <3,1,2> <2,3,1> <3,2,1> see comparison see comparison � Each leaf represents one possible result (a permutation of the elements in sorted order). Internal node i : j indicates comparison between a i and a j . � The height of the tree (i e � The height of the tree (i.e., longest path) is the longest path) is the suppose three elements < a 1, a 2, a 3> with instance <6,8,5> suppose three elements < a 1 a 2 a 3> with instance <6 8 5> Leaf node < π (1), π (2), π (3)> indicates ordering a π (1) ≤ a π (2) ≤ a π (3) . lower bound. Path of bold lines indicates sorting path for <6,8,5>. There are total 3!=6 possible permutations (paths). Decision Tree Model QuickSort Design g � The longest path is the worst case number of � Follows the divide-and-conquer paradigm. comparisons. The length of the longest path is the � Divide : Partition (separate) the array A [ p .. r ] into two height of the decision tree. (possibly empty) subarrays A [ p .. q– 1] and A [ q+ 1 ..r ]. � Each element in A [ p .. q– 1] < A [ q ]. � Theorem 8.1 : Any comparison sort algorithm y p g requires Ω ( n lg n ) comparisons in the worst case. � A [ q ] < each element in A [ q+ 1 ..r ]. � Index q is computed as part of the partitioning procedure. � Proof: � Conquer : Sort the two subarrays by recursive calls to � Suppose height of a decision tree is h , and number of � Suppose height of a decision tree is h and number of quicksort. i k t paths (i,e,, permutations) is n !. � Since a binary tree of height h has at most 2 h leaves, � Combine : The subarrays are sorted in place – no � n ! ≤ 2 h , so h ≥ lg ( n !) ≥ Ω ( n lg n ) (By equation 3.18). work is needed to combine them. work is needed to combine them. , g ( ) ( g ) ( y q ) � That is to say: any comparison sort in the worst � How do the divide and combine steps of quicksort case needs at least n lg n comparisons . compare with those of merge sort?

Pseudocode Example p p r Quicksort(A, p, r) initially: 2 5 8 3 9 4 1 7 10 6 note: pivot (x) = 6 if p < r then if p < r then Partition(A, p, r) P titi (A ) i j q := Partition(A, p, r); x, i := A[r], p – 1; Quicksort(A, p, q – 1); for j := p to r – 1 do next iteration: 2 5 8 3 9 4 1 7 10 6 Quicksort(A, q + 1, r) Quicksort(A, q 1, r) if A[j] ≤ x then if A[j] ≤ x then i i j j Partition(A, p, r) Partition(A p r) i := i + 1; x, i := A[r], p – 1; A[i] ↔ A[j] next iteration: 2 5 8 3 9 4 1 7 10 6 A[p..r] for j := p to r – 1 do A[i + 1] ↔ A[r]; A[i + 1] ↔ A[r]; if A[j] ≤ x then if A[j] ≤ x then i j j 5 return i + 1 i := i + 1; next iteration: 2 5 8 3 9 4 1 7 10 6 A[i] ↔ A[j] i j A[p..q – 1] A[q+1..r] A[i + 1] ↔ A[r]; [ ] [ ]; return i + 1 Partition next iteration: 2 5 3 8 9 4 1 7 10 6 5 i j ≤ 5 ≤ 5 ≥ 5 ≥ 5 Example (Continued) p ( ) Partitioning next iteration: 2 5 3 8 9 4 1 7 10 6 Select the last element A[ r ] in the subarray � i i j j A [ p .. r ] as the pivot – the element around which next iteration: 2 5 3 8 9 4 1 7 10 6 to partition. i j � As the procedure executes, the array is p , y next iteration: next iteration: 2 5 3 4 9 8 1 7 10 6 2 5 3 4 9 8 1 7 10 6 Partition(A, p, r) Partition(A p r) partitioned into four (possibly empty) regions. x, i := A[r], p – 1; i j A [ p .. i ] — All entries in this region are < pivot . next iteration: 2 5 3 4 1 8 9 7 10 6 for j := p to r – 1 do 1. if A[j] ≤ x then if A[j] ≤ x then A [ i 1.. j A [ i +1.. j – 1] — All entries in this region are > pivot . 1] All entries in this region are pivot . i i j j 2. 2. i := i + 1; A [ r ] = pivot . next iteration: 2 5 3 4 1 8 9 7 10 6 3. A[i] ↔ A[j] i j A [ j .. r – 1] — Not known how they compare to pivot . 4. A[i + 1] ↔ A[r]; [ ] [ ] next iteration: 2 5 3 4 1 8 9 7 10 6 � � The above hold before each iteration of the for The above hold before each iteration of the for i j return i + 1 loop, and constitute a loop invariant. (4 is not part after final swap: 2 5 3 4 1 6 9 7 10 8 of the loopi.) i j

Correctness of Partition Correctness of Partition � Use loop invariant. Case 1: � Initialization: I iti li ti i j r p � Before first iteration > x x � A [ p .. i ] and A [ i +1.. j – 1] are empty – Conds. 1 and 2 are satisfied � A [ p i ] and A [ i +1 j 1] are empty Conds 1 and 2 are satisfied (trivially). Partition(A, p, r) ≤ x � r is the index of the pivot > x x i := A[r] p – 1; x, i : A[r], p 1; j j � Cond. 3 is satisfied. � Cond 3 is satisfied i i r r p p for j := p to r – 1 do � Maintenance: x if A[j] ≤ x then i := i + 1; ; � Case 1: A [ j ] x � Case 1: A [ j ] > x A[i] ↔ A[j] ≤ x > x � Increment j only. A[i + 1] ↔ A[r]; � Loop Invariant is maintained. return i + 1 Correctness of Partition Correctness of Partition � Case 2: A [ j ] ≤ x � Termination: [ j ] � Increment j � Condition 2 is � When the loop terminates, j = r , so all elements � Increment i maintained. in A are partitioned into one of the three cases: � Swap A [ i ] and A [ j ] A [ r ] is unaltered. � � A [ p .. i ] ≤ pivot � A [ p i ] ≤ pivot � Condition 1 is � Condition 1 is Condition 3 is � maintained. � A [ i +1.. j – 1] > pivot maintained. i j r � A [ r ] = pivot p ≤ x ≤ x x x � The last two lines swap A [ i +1] and A [ r ] � The last two lines swap A [ i +1] and A [ r ]. � Pivot moves from the end of the array to ≤ x between the two subarrays. > x i j r � Thus, procedure partition correctly performs p the divide step. x ≤ x > x

Complexity of Partition p y Quicksort Overview � PartitionTime( n ) is given by the number � PartitionTime( n ) is given by the number � To sort a[ left...right] : T t [ l ft i ht] of iterations in the for loop. 1. if left < right: � Θ ( n ) : n = r � Θ ( n ) : n = r – p + 1. p + 1 1.1. Partition a[ left...right] such that: 1 1 P titi [ l ft i ht] h th t Partition(A, p, r) all a[ left...p-1] are less than a[ p] , and x, i := A[r], p – 1; for j : p to r for j := p to r – 1 do 1 do all a[ p+ 1...right] are > = a[ p] if A[j] ≤ x then i := i + 1; 1.2. Quicksort a[ left...p-1] A[i] ↔ A[j] [ ] [j] A[i + 1] ↔ A[r]; 1.3. Quicksort a[ p+ 1...right] return i + 1 2. Terminate Partitioning in Quicksort g Alternative Partitioning � Choose an array value (say, the first) to use y ( y, ) � A key step in the Quicksort algorithm is � A key step in the Quicksort algorithm is as the pivot partitioning the array � Starting from the left end, find the first � We choose some (any) number p in the ( y) p element that is greater than or equal to the element that is greater than or equal to the array to use as a pivot pivot � We partition the array into three parts: � Searching backward from the right end, find Searching backward from the right end, find the first element that is less than the pivot � Interchange (swap) these two elements p � Repeat, searching from where we left off, p until done numbers greater than or numbers less equal to p than p