heaps and heapsort on n elements height of a heap is in (log n ) - - PowerPoint PPT Presentation

data structures and algorithms 2020 09 17 lecture 6

heaps and heapsort on n elements

height of a heap is in Θ(log n) building a heap bottum-up in O(n) analysis via picture (learn) or using summations (know result) building a heap one-by-one in O(n log n) maxheapify in O(log n) (also Θ(log n)) heapsort in O(n log n); also in Θ(n log n)? yes, see worst-case input or see lower-bound more: see smooth sort

use of priority queue: Dijkstra’s algorithm

input: directed graph G = (V , E) with positive weights w and start vertex s in G Algorithm Dijkstra(G, w, s): initialize(G, s) S := ∅ Q := G.V while Q = ∅ do u := extractMin(Q) S := S ∪ {u} for each v ∈ G.Adj[u] do relax(u, v, w) the set S contains vertices for which the weight of a shortest path has been found if priority queue implemented with heap then in O(|E| · log |V |)

quicksort on n elements

worst-case in Θ(n2) best-case in Θ(n log n) balanced-case in O(n log n) average case (assuming all n! possible inputs are equally likely) in O(n log n) expected case (using randomization) in O(n log n) more: replace small recursive calls by insertion sort and sorting networks

• verview

lower bound on sorting counting sort radix sort bucket sort

• verview

lower bound on sorting counting sort radix sort bucket sort

recall: bounds

asymptotic upper bound f ∈ O(g) f is eventually bound from above by g up to a constant asymptotic tight bound f ∈ Θ(g) f is eventually sandwiched by g up to two constants asymptotic lower bound f ∈ Ω(g) f is eventually bound from below by g up to a constant (there are more bounds!)

upper and lower bound

f , g : N → R+ asymptotic upper bound: f (n) ∈ O(g(n)) if ∃c > 0 ∈ R+ ∃n0 > 0 ∈ N ∀n ∈ N : n ≥ n0 ⇒ f (n) ≤ c · g(n) asymptotic lower bound: f (n) ∈ Ω(g(n)) if ∃c > 0 ∈ R+ ∃n0 > 0 ∈ N ∀n ∈ N : n ≥ n0 ⇒ f (n) ≥ c · g(n)

question

comparison-based sorting: the crucial tests are a < b, a ≤ b, a = b, a ≥ b, a > b which are all decidable in elementary time so far we have seen only comparison-based sorting algorithms is O(n log n) the best worst-case that we have?

lower bound on running time

we consider sorting algorithms based on comparisons k < k′ examples: merge sort, heapsort, quicksort non-examples: counting sort, bucket sort Ω(n log n) comparisons are needed in the worst case so merge sort, heapsort are asymptotically optimal

lowerbound: proof

we consider decision trees node contains a comparison a < b? leaf corresponds to a permutation of {1, . . . .n} every comparison-based sorting algorithm has for every number of inputs a decision tree execution of that sorting algorithm corresponds to a path in the tree every possible permutation (of total n!) must occur every permutation must be reachable

decision tree: example

a c b c a b b a c b c a c b a a < b? b < c? b < c? a < c? a < c? a b c

we analyze the height of a decision tree with n nodes

the number of comparisons is height h of the tree we have at least n! (because decision tree), and at most 2h (because binary tree) leaves, hence: n! ≤ #leaves ≤ 2h n! ≤ 2h log n! ≤ h h ≥ log n!

we continue to find a bound on the height

we have h ≥ ⌈log n!⌉

• mitting floors and ceilings we find:

h ≥ log(n!) = log(1 · 2 · . . . · ( n

2 − 1) · n 2 . . . · n)

≥ log(1 · 1 · . . . · 1 · n

2 . . . · n 2)

= log(( n

2)

n 2 )

=

n 2 log( n 2)

hence h ∈ Ω(n log n) (see also 3.19 in the book that uses Stirling’s approximation)

results

Theorem: any comparison-based sorting algorithm uses Ω(n log n) comparisons in the worst case. Consequence: heapsort and merge sort are asymptotically optimal However: for specific inputs there are linear sorting algorithms, not based on comparisons

back to puzzle from Knuth

can we sort 5 elements in ⌈log 5!⌉ = 7 comparisons?

• verview

lower bound on sorting counting sort radix sort bucket sort

counting sort

assumption: numbers in input come from fixed range {0, . . . , k}. algorithm idea: count the number of occurrences of each i from 0 to k time complexity: in Θ(n + k) for a input-array of length n drawback: fixed range, and requires additional counting array C andoutput array B counting sort is a stable sorting algorithm what happens if the last loop is done for j := 1 up to j := A.length?

counting sort: pseudo-code

input array A, output array B, range from 0 up to k Algorithm countingSort(A, B, k): new array C[0 . . . k] for i := 0 to k do C[i] := 0 for j := 1 to A.length do C[A[j]] := C[A[j]] + 1 for i := 1 to k do C[i] := C[i] + C[i − 1] for j := A.length downto 1 do B[C[A[j]]] := A[j] C[A[j]] := C[A[j]] − 1

• verview

lower bound on sorting counting sort radix sort bucket sort

radix sort radix sort: intuition

the old days: used for sorting punched cards 80 columns with one hole in one of the 12 possible places

• ur use: sorting numbers considered as tuples

number of columns: (fixed) amout of digits used number of places: 10 for decimal numbers 329 457 329 329 457 657 839 457 657 329 457 657 839 839 657 839 recall the lexicographic ordering for tuples: (x1, . . . , xd) < (y1, . . . , yd) if (x1 < y1) or ((x1 = y1) and (x2, . . . , xn) < (y2, . . . , yn))

radix sort: ‘pseudo-pseudocode’

intuition: sort per dimension using a stable sorting algorithm Algorithm radixSort(A, d): for i := 1 to d do use some stable sort on digit d 1 is the lowest-order digit and d is the highest-order digit what happens if we take the other order?

radix sort: time complexity

if counting sort (stable!) per dimension is in Θ(n + k) then radix sort is in Θ(d(n + k)) if d is constant and k is in O(n) then radix sort is in linear time is radix sort preferable to comparison-based sorting? application: order {0, . . . , 8} in representation with basis 3: 00, 01, 02, 10, 11, 12, 20, 21, 22

• verview

lower bound on sorting counting sort radix sort bucket sort

bucket sort

similar to counting sort assumption for correctness: keys in [0, 1) assumption for time complexity: key uniformly distributed over [0, 1) an elementary operation on the key gives the index several keys can belong to the same index

bucket sort: pseudo-pseudocode

array A with 0 ≤ A[i] < 1 for i = 1, . . . , A.length Algorithm bucketSort(A): n := A.length new array B[0 . . . n − 1] for i := 0 to n − 1 do make B[i] an empty list init B for i := 1 to n do insert A[i] into list B[⌊n · A[i]⌋] for i := 0 to n − 1 do insertionSort(B[i]) concatenate B[0], B[1], . . . , B[n − 1]

bucket sort: average-case time complexity

bucket sort on an input-array A of length n elements of A in [0, 1) elements of A uniformly distributed over [0, 1) average-case time complexity (without proof): in Θ(n) worst-case time complexity in Θ(n2); why ? can we improve the worst-case time complexity ?