Comparison complexity of priority-queue operations Jyrki Katajainen - - PowerPoint PPT Presentation

c

Performance Engineering Laboratory

Visit at the Max-Planck-Institut f¨ ur Informatik (1)

Comparison complexity of priority-queue operations

Jyrki Katajainen (University of Copenhagen) Joint work with Amr Elmasry (Max-Planck-Institut f¨ ur Informatik) and Claus Jensen (University of Copenhagen)

16 January 2009 Updated 31 January 2009 These slides are available at http://cphstl.dk

c

Performance Engineering Laboratory

Visit at the Max-Planck-Institut f¨ ur Informatik (2)

Priority-queue kernel

insert(Q, p). Add

an element with locator p to Q.

extract(Q). Extract an unspeci-

fied element from Q and re- turn a locator to that elem-

• ent. Precondition: Q = ∅.

delete(Q, p). Remove

the elem- ent with locator p from Q (without destroying it).

meld(Q, R). Move

all elements from Q and R to a new pri-

• rity queue S, destroy Q and

R, and return S.

decrease(Q, p, x). . . . not relevant

for this talk. . .

create(Q). Create Q. Postcondi-

tion: Q = ∅.

destroy(Q). Destroy Q. Precondi-

tion: Q = ∅.

find-min(Q). Return a locator to

an element that, of all elem- ents in Q, has a minimum

• value. Precondition: Q = ∅.

size(Q). Return the number of

elements stored in Q.

swap(Q, R). Make Q refer to the

data structure referred to by R, and vice versa.

c

Performance Engineering Laboratory

Visit at the Max-Planck-Institut f¨ ur Informatik (3)

Focus

• Comparison complexity
• Worst-case efficiency
• Constant factors

God’s rule 1: Do care about the size of O

! !

Read [Sch¨

• nhage et al. 1994]

I make no claims about the practical utility of the data structures discussed, even though the development of a library component that provides good practical performance is a challenging task.

c

Performance Engineering Laboratory

Visit at the Max-Planck-Institut f¨ ur Informatik (4)

Bug report

The binary-heap implementation in LEDA 6.1 is broken.

• 1. The time bounds are guaranteed in the amortized or average-case

sense, not in the worst-case sense, contrary to what is claimed in the documentation.

• 2. insert is extremely slow; it does not take O

!

expected time as I would expect.

c

Performance Engineering Laboratory

Visit at the Max-Planck-Institut f¨ ur Informatik (5)

Research question

Assumptions: Trivial operations have O

! worst-case running time and

perform no element comparisons. The worst-case running time of non-trivial operations is proportional to the number of element comparisons performed. (If not, specify explicitly.) Question: What are the best worst-case bounds for the number

• f element comparisons performed by the non-trivial operations

insert, extract, delete, and meld?

Answer: It is still open whether optimal bounds O

! for insert, extract,

and meld; and lg n + O

! for delete can be achieved or not.

n, m: # of elements stored just prior to an operation

c

Performance Engineering Laboratory

Visit at the Max-Planck-Institut f¨ ur Informatik (6)

Biased history

Year Inventor

insert extract delete meld

1964 Williams lg n + O ! O ! 2 lg n + O ! O ! lg n lg m 1978 Vuillemin lg n + O ! O ! lg n time 2 lg n + O ! lg n + O ! 1988 Driscoll & al. O ! lg n + O ! 3 lg n + O ! lg m + O ! 1995 Brodal O ! O ! lg n time 6 lg n + O ! O ! 1996 Brodal & Okasaki O ! O ! lg n time 4 lg n + O ! O ! 2004 Elmasry O ! – 1.44 lg n + O ! lg lg n – Jensen & me O ! O ! lg n + O ! lg lg n – Elmasry O ! O ! lg n + O ! – Now New results O ! O ! lg n + O ! 2 lg n + 2 lg m + O ! O ! lg n + O ! 3 lg n + O ! O !

• O

! ≤ 10; n ≥ m.

• Bounds displayed in red are not proved in the original papers.
• See our joint paper Multipartite priority queues in ACM Transac-

tions on Algorithms 5,1 (2008), Article 14.

c

Performance Engineering Laboratory

Visit at the Max-Planck-Institut f¨ ur Informatik (7)

Binomial queues

n = 1010two B3 B1 min

1 x y 3

Bk−1 Bk−1 B0 ≡ Bk ≡

• heap-ordered x ≤ y
• at most ⌊lg n⌋ + 1 binomial trees

Read [Cormen et al. 2001]

c

Performance Engineering Laboratory

Visit at the Max-Planck-Institut f¨ ur Informatik (8)

Redundant binary numbers

Binary representation: n =

⌊lg n⌋

• i=0

di2i, where di ∈ {0, 1} for all i ∈ {0, 1, . . . , ⌊lg n⌋}. Redundant representation: n =

⌊lg n⌋

• i=0

di2i, where di ∈ {0, 1, 2} for all i ∈ {0, 1, . . . , ⌊lg n⌋}. Read [Okasaki 1998] 1 1 1 + 1 1 1 1 1 + 1 1 1 2 Assuming a carry stack is avail- able, ++ is performed as follows:

• 1. Fix the topmost carry if the

stack is not empty.

• 2. Add one as desired.
• 3. If the least significant digit

becomes 2, push this carry

• nto the stack.

c

Performance Engineering Laboratory

Visit at the Max-Planck-Institut f¨ ur Informatik (9)

Bug report

The binomial-queue implementation in [Cormen et al. 2001] can be improved:

• 1. By maintaining a pointer to

the minimum, the running time of find-min can be im- proved from O

! lg n to O ! .

• 2. With

the redundant binary representation, the running time of insert can be impro- ved from O

! lg n to O ! .

• 3. It would be easier to maintain

referential integrity if parent pointers were only maintain- ed for the largest children.

B3

1

min B1 n = 1010redundant two

3

c

Performance Engineering Laboratory

Visit at the Max-Planck-Institut f¨ ur Informatik (10)

Prefix-minimum pointers

i < j < k

7 2 5 4

Bk Bj Bi

The key observation, what I call the ↑→ property, is that when a value in tree Bj is updated, the number of element comparisons made to fix Bj is j + O

!

and the number of element comparisons made to fix the prefix-minimum pointers at higher trees is at most lg n − j + O

! .

Read [Elmasry et al. 2008]

c

Performance Engineering Laboratory

Visit at the Max-Planck-Institut f¨ ur Informatik (11)

Delivery buffer

5 9 3 7 Maintain a buffer whose size is between 1 and lg n + O

! . The first

node stores the minimum element held in the buffer. The buffer can be used for borrowing and inserting elements. Special handling is necessary when the buffer becomes too small or too large.

c

Performance Engineering Laboratory

Visit at the Max-Planck-Institut f¨ ur Informatik (12)

Incremental processing

Perform a task taking O

! t time incrementally during the forthcoming

t modifying operations (insert, extract, delete, and meld) so that each subtask is about equal in size. It is important that there is at most

• ne incremental process running at a time.

The simplest way of implementing this idea is to divide the task into primitive steps and execute a fixed number of primitive steps in connection with each subtask. An alternative way is to use coroutines in languages that support them.

c

Performance Engineering Laboratory

Visit at the Max-Planck-Institut f¨ ur Informatik (13)

Multipartite priority queues

binomial queue (redundant) 0..1

2 lg n + O

! elements

doubly-linked list 1..lg n + O

! elements

doubly-linked list at most lg n + O

! prefix-minimum pointers

binomial queue (binary) at most n elements Floating tree binomial tree at most n elements Buffer Upper store Main store Recall where the overall minimum is!

c

Performance Engineering Laboratory

Visit at the Max-Planck-Institut f¨ ur Informatik (14)

Priority-queue operations (Part I)

insert:

– Insert the given node into the buffer. – If the buffer is larger than

7 8 lg n + O ! , construct Bk, where 1 4 lg n + O ! ≤ 2k < 1 2 lg n + O ! , and

unite it with the main store. [Done incrementally during the next 1

8 lg n + O ! operations!]

extract:

– Remove a node from the buf- fer. – If the buffer is smaller than

1 8 lg n + O ! , take the smallest tree

from the main store and split it (if possible). If the smallest tree is of size 1, move it to the buffer. After such a move, update the upper store accordingly. Do this splitting repeatedly until the buf- fer contains more than 1

4 lg n + O !

elements. [Done incrementally during the next 1

8 lg n + O ! operations!]

c

Performance Engineering Laboratory

Visit at the Max-Planck-Institut f¨ ur Informatik (15)

Priority-queue operations (Part II)

delete:

– If the removed node is in the buffer, remove it and update the buffer minimum. Stop. – Swap the removed node with its parent until it reaches the root

• f its tree Bk.

– Borrow a node from the buffer. – Remove the root of Bk. – Rebuild Bk using the root’s subtrees and the borrowed node. – If the removed node was in the main store, update the prefix- minimum pointers for trees larger than Bk, if necessary.

meld(Q, R): (if size(Q) ≤ size(R))

– Throw away both upper stores. – Concatenate the two buffers. – If Q had a floating tree under construction, complete this. – If R had a floating tree under construction, continue this. – If the buffer is still too large, start the creation of a new float- ing tree. – Unite the two main stores, the two new trees (if any), and the two floating trees (if any). – Create an upper store for the new main store.

c

Performance Engineering Laboratory

Visit at the Max-Planck-Institut f¨ ur Informatik (16)

Bug report

I claimed that O

! ≤ 10!

• 1. Construction of a binomial queue of size 1

2 lg n+O ! requires 1 2 lg n+

O

! element comparisons.

• 2. Union of the floating tree and a binomial queue of size n can

require up to lg n + O

! element comparisons.

• 3. Finding the minimum of 4 elements requires 3 element compari-

sons.

• 4. This work is shared by 1

8 lg n + O !

modifying operations. Because

• f the incremental process itself, O

! > 15.

How to recover from this error?

c

Performance Engineering Laboratory

Visit at the Max-Planck-Institut f¨ ur Informatik (17)

Bootstrapping

The nodes contain (element, priority queue) pairs. 5 . . . The element at a node is a representative for the whole group.

c

Performance Engineering Laboratory

Visit at the Max-Planck-Institut f¨ ur Informatik (18)

Bootstrapped priority queues

The whole priority queue is represented as a pair locator. From the

• utside, this priority queue is accessed with element locators. The

priority queues inside are accessed with pair locators. Implement the priority queues inside using binomial queues (redun- dant). # of element comparisons without a minimum pointer:

find-min insert extract delete meld

lg n + O

!

O

!

O

!

lg n + O

!

lg n + O

!

c

Performance Engineering Laboratory

Visit at the Max-Planck-Institut f¨ ur Informatik (19)

Priority-queue operations (Part I)

insert(Q, x):

(y, R) ← pair[Q] if element[x] < element[y] p ← (y, –)

insert(R, p) pair[Q] ← (x, R)

else p ← (x, –)

insert(R, p) pair[Q] ← (y, R) extract(Q):

(x, R) ← pair[Q] (y, S) ← extract(R) T ← meld(R, S)

pair[Q] ← (x, T)

return y

c

Performance Engineering Laboratory

Visit at the Max-Planck-Institut f¨ ur Informatik (20)

Priority-queue operations (Part II)

delete(Q, x):

(y, R) ← pair[Q] if x = y and size(R) = 0

pair[Q] ← (–, –)

else if x = y and size(R) > 0 (z, S) ← p ← find-min(R)

delete(R, p)

T ← meld(R, S)

pair[Q] ← (z, T)

else (x, S) ← p ← pair[x]

delete(R, p)

T ← meld(R, S)

pair[Q] ← (y, T) meld(Q, R):

(x, S) ← pair[Q] (y, T) ← pair[R] if element[x] < element[y] p ← (y, T)

insert(S, p) pair[Q] ← (x, S)

else p ← (x, S)

insert(T, p) pair[Q] ← (y, T)

return Q

c

Performance Engineering Laboratory

Visit at the Max-Planck-Institut f¨ ur Informatik (21)

Open problems

• What is the best bound for delete if insert, extract, and meld run in

O

! worst-case time?

• Another story if decrease is to be supported in O

! worst-case time.

Read [Brodal 1996].

• How to implement a worst-case efficient priority queue in an

industry-strength program library?