Heapsort Chapter 6 1 CPTR 430 Algorithms Heapsort - PowerPoint PPT Presentation

Heapsort Chapter 6 1 CPTR 430 Algorithms Heapsort

✁ ☎ ✁ ☎ ✆ ✝ ✆ ✝ ✞ ✞ ✞ ✝ ✆ ✄ ✁ ✁ ✁ ✁ � ✁ ✁ ✁ ✁ � ✁ ☎ ✆ ✄ ✂ ✂ ✁ ☎ ✄ ✁ ✆ ✂ ✂ ✆ ✁ ✁ ✁ ✂ � ✂ � Sorting ■ Input : a sequence of n numbers a 0 a 1 a n 1 ■ Output : a permutation (reordering) a a a 0 1 n 1 such that a a a 0 1 n 1 ■ Instance of problem : 31 41 59 26 41 58 Input: Output: 26 31 41 41 58 59 2 CPTR 430 Algorithms Heapsort

Sorting ■ Numbers all by themselves are rarely sorted ■ Usually, records are sorted by numeric keys ❚ The rest of the record is satellite data and goes along with the keys when sorting ❚ If records are large (lots of satellite data), then an array of pointers to the records can be sorted to minimize the data movement ■ An algorithm can concentrate on sorting plain numbers; sorting records would be a relatively minor implementation detail 3 CPTR 430 Algorithms Heapsort

What’s the Big Deal about Sorting? ■ Many applications need to sort data ■ Many algorithms sort data via a subroutine in the course of their actions ■ There are many different sorting algorithms, each with its own interesting characteristics ■ Bounds can be easily determined for sorting algorithms; this experience transfers to other algorithms ■ The implementations of sorting algorithms demonsrate many software engineering issues: data shaping, the memory hierarchy, etc. 4 CPTR 430 Algorithms Heapsort

Sorts We Will Examine ■ Insertion sort (we saw this in Chapter 2) ■ Merge sort (we saw this in Chapter 2) ■ Heapsort (this chapter) ■ Quicksort (Chapter 7) 5 CPTR 430 Algorithms Heapsort

� ✁ � ✁ Heapsort ■ Like merge sort, heapsort runs in O n lg n time (insertion sort takes n 2 O time) ■ Like insertion sort, heapsort sorts in place (merge sort requires extra space) ■ Heapsort uses a specialized data structure to manage the sort ❚ The heap ❚ Not to be confused with the “heap” used for dynamic memory allocation in Java and C++ ❚ Also makes a nice priority queue 6 CPTR 430 Algorithms Heapsort

Heap ■ The heap is an array that can be viewed as a nearly complete binary tree ❚ The tree is completely filled at all levels except the lowest ❚ The lowest level is filled from the left 0 16 1 2 14 10 3 4 5 6 8 7 9 3 7 8 9 4 1 2 4 16 14 10 8 7 9 3 2 1 1 2 3 4 5 6 7 8 9 0 7 CPTR 430 Algorithms Heapsort

✁ ✄ � ✁ ✆ ✁ � ✂ ✄ � ✂ ☎ ✂ ✁ � Array as Binary Tree? 0 16 i 2 i 1 left 1 2 14 10 3 4 5 6 i 2 i 2 right 8 3 7 9 7 8 9 4 1 2 i 1 i parent 2 14 10 8 7 9 3 2 4 1 16 1 2 3 4 5 6 7 8 9 0 For every node i except the root (root is index 0 ): A[parent i A[ i ] ] 8 CPTR 430 Algorithms Heapsort

� � � ✝ ✁ ✁ Max-heap vs. Min-heap i A[ i ] ■ In a max-heap, A[parent ] ❚ The root holds the largest value ❚ Max-heaps are used for sorting ■ In a min-heap, A[parent i A[ i ] ] ❚ The root holds the smallest value ❚ Min-heaps are often used for priority queues 9 CPTR 430 Algorithms Heapsort

✁ � ✁ � � Heap Height ■ The height of a node in a heap is the number of edges on the longest path from the node to a leaf ■ The height of a heap is the height of its root height of heap = Θ lg n ■ Heap is based on a complete binary tree ■ Basic operations on a heap run in time proportional to the height of the heap— O lg n 10 CPTR 430 Algorithms Heapsort

Heap Operations ■ max-heapify: maintains the max-heap property ■ build-max-heap: produces a max-heap from an unsorted array ■ heapsort: sorts an array in place (using a heap) ■ max-heap-insert, heap-extract-max, heap-increase-key, heap-maximum: used for priority queues 11 CPTR 430 Algorithms Heapsort

Maintaining the Heap Property public static void maxHeapify(int[] a, int heapSize, int i) { (Code removed due to Assignment #4) } ■ a is an array ■ heapSize is the size of the heap ■ i is an index into the array 12 CPTR 430 Algorithms Heapsort

Maintaining the Heap Property public static void maxHeapify(int[] a, int heapSize, int i) { (Code removed due to Assignment #4) } ■ left(i) and right(i) must be roots of max-heaps ■ a[i] may be smaller than either of its children (thus violating the max- heap property) ■ maxHeapify() moves a[i] down into the heap so that the subtree rooted at i becomes a max-heap 13 CPTR 430 Algorithms Heapsort

maxHeapify() maxHeapify(a, 1); 0 0 16 16 1 2 1 2 i 4 14 10 10 3 4 5 6 3 4 5 6 i 14 7 9 3 4 7 9 3 7 8 9 7 8 9 8 8 2 1 2 1 0 16 1 2 14 10 3 4 5 6 8 3 7 9 7 8 9 2 4 1 i 14 CPTR 430 Algorithms Heapsort

� � � ✁ ✁ � � � ✁ ✁ maxHeapify() Running Time For a subtree of size n rooted at node i , T n is: T (fix up relationships among a[i] , a[left(i)] , and a[right(i)] ) + T (run maxHeapify() on one of the children of i ) but ■ The time to fix up the relationships is constant ■ Each subtree of i has size at most 2 n 3 Worst case is when last row is half full and so Θ T n T 2 n 3 1 ✁ ✄✂ 15 CPTR 430 Algorithms Heapsort

� � ✁ � ✁ ✁ ✁ � � � ✁ ✁ ✁ � � ✁ ✁ maxHeapify() Running Time Based on our study of Chapter 4, the recurrence relation Θ T n T 2 n 3 1 ✁ ✄✂ has the solution lg n T n O If the dimension of interest is the height of the tree, h , then T n O h 16 CPTR 430 Algorithms Heapsort

✂ ✂ ✂ ✁ � ✂ � ✂ ✁ Building a Heap ■ To build a max-heap from an arbitrary array, apply maxHeapify() in a bottom-up manner ■ All the elements in the range n 2 n 1 are leaves of the tree ❚ Each of the leaves is a subtree of size 1 1 ❚ n a.length public static void buildMaxHeap(int[] a) { (Code removed due to Assignment #4) } 17 CPTR 430 Algorithms Heapsort

buildMaxHeap() in Action 0 0 4 4 1 2 1 2 1 3 1 3 3 4 5 6 3 4 5 6 i i 2 16 10 2 16 10 9 9 7 8 9 7 8 9 14 8 7 14 8 7 0 0 4 4 1 2 1 2 i i 1 3 1 10 3 4 5 6 3 4 5 6 14 16 9 10 14 16 9 3 7 8 9 7 8 9 2 8 7 2 8 7 0 0 i 4 16 1 2 1 2 i 16 10 14 10 3 4 5 6 3 4 5 6 14 7 3 8 7 3 9 9 7 8 9 7 8 9 2 8 1 2 4 1 18 CPTR 430 Algorithms Heapsort

� ✂ ✁ � � ✂ ✁ ✁ � ✂ � ✁ ✁ � � ✁ ✂ � ✁ ✂ ✂ ✂ ✁ ✁ ✂ ✂ ✁ ✂ ✂ ✂ ✁ ✂ ✁ Correctness of buildMaxHeap() Loop invariant : At the start of each for iteration, each node i i 1 i 2 n 1 is the root of a max-heap Proof: Initialization : Before the first iteration, i n 2 1 . Each node n 2 n 2 1 n 2 2 n 1 is a leaf, and therefore each is a root of a max-heap of size 1 . 19 CPTR 430 Algorithms Heapsort

✁ ✁ ✁ ✂ ✂ ✂ ✁ ✂ ✁ ✂ ✂ ✁ ✂ ✂ ✂ ✁ ✂ ✁ ✂ ✁ Correctness of buildMaxHeap() Mainentance : Observe that the children of node i have values greater than i . By the loop invariant, they are all roots of max- maxHeapify() can thus be applied to make i a max-heap. heaps. maxHeapify() ensures that i i 1 i 2 n 1 are all roots of max- heaps. The for loop decrements i to reestablish the loop invariant for the next iteration. Termination : When the for loop finishes, i 1 . According to the loop invariant, 0 1 2 n 1 are all roots of max-heaps. Since 0 is the root, the array is a heap. 20 CPTR 430 Algorithms Heapsort

✁ � ✁ � ✁ ✁ � ✁ � Running Time of buildMaxHeap() Simple upper bound: lg n ■ Each call to maxHeapify() takes O time ■ maxHeapify() is called O n times ■ T n O n lg n Can we get a tighter bound? ■ maxHeapify() ’s run time depends on the height of the node within the tree ■ Most nodes are not very high (more nodes are on lower levels than higher levels) 21 CPTR 430 Algorithms Heapsort

✁ � ✁ � � ✁ ✁ � ✆ � ☎ ✁ ✄ ✂ Tighter Running Time Bound for buildMaxHeap() ■ An n -element heap has height lg n 2 h 1 ■ An n -element heap has at most n nodes of some height h ■ The time for maxHeapify() to run on a heap of height h is O h ■ The running time of buildMaxHeap() is thus lg n n ∑ 1 O h 2 h h 0 22 CPTR 430 Algorithms Heapsort

✁ ✁ ✆ ✄ ☎ ☎ ✄ ✁ ✆ � Tighter Running Time Bound for buildMaxHeap() We can simplify this a bit: lg n lg n n h ∑ ∑ 1 O h O n 2 h 2 h h 0 h 0 23 CPTR 430 Algorithms Heapsort

✁ ✆ ✁ � � ✂ ✆ � ✁ ✁ ✁ ✁ ✁ ✂ ✆ ✁ � ✄ ☎ ✆ Tighter Running Time Bound for buildMaxHeap() lg n h ∑ O n 2 h h 0 We can apply the mathematical identity ∞ x ∑ kx k (Page 1061) 2 1 x k 0 1 where x 2 and k h : h ∞ ∞ h 1 1 2 ∑ ∑ h 2 2 2 h 2 1 1 2 h 0 h 0 24 CPTR 430 Algorithms Heapsort

✁ ✁ � ✄ ✁ ☎ ✆ ✆ Tighter Running Time Bound for buildMaxHeap() ∞ lg n h h ∑ ∑ O n O n O n 2 h 2 h h 0 h 0 Thus we can build a max-heap in linear time 25 CPTR 430 Algorithms Heapsort