Data Structures Fibonacci Heaps, Amortized Analysis Algorithm Theory - - PowerPoint PPT Presentation

Chapter 4

Data Structures

Fibonacci Heaps, Amortized Analysis

Algorithm Theory WS 2012/13 Fabian Kuhn

Algorithm Theory, WS 2012/13 Fabian Kuhn 2

Fibonacci Heaps

Lacy‐merge variant of binomial heaps:

• Do not merge trees as long as possible…

Structure: A Fibonacci heap consists of a collection of trees satisfying the min‐heap property. Variables:

• . : root of the tree containing the (a) minimum key
• . : circular, doubly linked, unordered list containing

the roots of all trees

• . : number of nodes currently in

Algorithm Theory, WS 2012/13 Fabian Kuhn 3

Trees in Fibonacci Heaps

Structure of a single node :

• . : points to circular, doubly linked and unordered list of

the children of

• . , . : pointers to siblings (in doubly linked list)
• . : will be used later…

Advantages of circular, doubly linked lists:

• Deleting an element takes constant time
• Concatenating two lists takes constant time

left parent right key degree child mark

Algorithm Theory, WS 2012/13 Fabian Kuhn 4

Example

Figure: Cormen et al., Introduction to Algorithms

Algorithm Theory, WS 2012/13 Fabian Kuhn 5

Simple (Lazy) Operations

Initialize‐Heap :

• . ≔ . ≔

Merge heaps and ′:

• concatenate root lists
• update .

Insert element into :

• create new one‐node tree containing  H′
• merge heaps and ′

Get minimum element of :

• return .

Algorithm Theory, WS 2012/13 Fabian Kuhn 6

Operation Delete‐Min

Delete the node with minimum key from and return its element: 1. ≔ . ; 2. if . 0 then 3. remove . from . ; 4. add . . (list) to . 5. . ; // Repeatedly merge nodes with equal degree in the root list // until degrees of nodes in the root list are distinct. // Determine the element with minimum key 6. return

Algorithm Theory, WS 2012/13 Fabian Kuhn 7

Rank and Maximum Degree

Ranks of nodes, trees, heap: Node :

• : degree of

Tree :

• : rank (degree) of root node of

Heap :

• : maximum degree of any node in

Assumption (: number of nodes in ):

– for a known function

Algorithm Theory, WS 2012/13 Fabian Kuhn 8

Merging Two Trees

Given: Heap‐ordered trees , ′ with

• Assume: min‐key of min‐key of ′

Operation , :

• Removes tree ′ from root list

and adds to child list of

• ≔ 1
• . ≔
• ′
• ′

Algorithm Theory, WS 2012/13 Fabian Kuhn 9

Consolidation of Root List

Array pointing to find roots with the same rank: Consolidate: 1. for ≔ 0 to do ≔ null; 2. while . null do 3. ≔ “delete and return first element of . ” 4. while null do 5. ≔ ; 6. ≔ ; 7. ≔ , 8. ≔ 9. Create new . and .

1 2

• |.

Time: |.

Algorithm Theory, WS 2012/13 Fabian Kuhn 10

Operation Decrease‐Key

Decrease‐Key, : (decrease key of node to new value ) 1. if . then return; 2. . ≔ ; update . ; 3. if ∈ . ∨ . . then return 4. repeat 5. ≔ . ; 6. . ; 7. ≔ ; 8. until . ∨ ∈ . ; 9. if ∉ . then . ≔ ;

Algorithm Theory, WS 2012/13 Fabian Kuhn 11

Operation Cut( )

Operation . :

• Cuts ’s sub‐tree from its parent and adds to rootlist

1. if ∉ . then 2. // cut the link between and its parent 3. . ≔ . 1; 4. remove from . . (list) 5. . ≔ null; 6. add to .

25 2 8 1 13 15 3 7 19 31

• 25

2 8 1 13 15 3 7 19 31

Algorithm Theory, WS 2012/13 Fabian Kuhn 12

Decrease‐Key Example

• Green nodes are marked

14 25 5 20 12 22 9 18 2 8 17 1 13 15 3 7 19 31

Decrease‐Key,

• 14

25 5 20 12 22 9 18 2 8 17 1 13 15 3 7 19 31

Algorithm Theory, WS 2012/13 Fabian Kuhn 13

Fibonacci Heap Marks

History of a node : is being linked to a node . ≔ a child of is cut . ≔ a second child of is cut .

• Hence, the boolean value . indicates whether node

has lost a child since the last time was made the child of another node.

Algorithm Theory, WS 2012/13 Fabian Kuhn 14

Cost of Delete‐Min & Decrease‐Key

Delete‐Min:

1. Delete min. root and add . to . time: 1 2. Consolidate . time: length of .

• Step 2 can potentially be linear in (size of )

Decrease‐Key (at node ):

1. If new key parent key, cut sub‐tree of node time: 1 2. Cascading cuts up the tree as long as nodes are marked time: number of consecutive marked nodes

• Step 2 can potentially be linear in
• Exercises: Both operations can take time in the worst case!

Algorithm Theory, WS 2012/13 Fabian Kuhn 15

Cost of Delete‐Min & Decrease‐Key

• Cost of delete‐min and decrease‐key can be Θ…

– Seems a large price to pay to get insert and merge in 1 time

• Maybe, the operations are efficient most of the time?

– It seems to require a lot of operations to get a long rootlist and thus, an expensive consolidate operation – In each decrease‐key operation, at most one node gets marked: We need a lot of decrease‐key operations to get an expensive decrease‐key operation

• Can we show that the average cost per operation is small?
• We can  requires amortized analysis

Algorithm Theory, WS 2012/13 Fabian Kuhn 16

Amortization

• Consider sequence , , … , of operations

(typically performed on some data structure )

• : execution time of operation
• ≔ ⋯ : total execution time
• The execution time of a single operation might

vary within a large range (e.g., ∈ 1, )

• The worst case overall execution time might still be small

 average execution time per operation might be small in the worst case, even if single operations can be expensive

Algorithm Theory, WS 2012/13 Fabian Kuhn 17

Analysis of Algorithms

• Best case
• Worst case
• Average case
• Amortized worst case

What it the average cost of an operation in a worst case sequence of operations?

Algorithm Theory, WS 2012/13 Fabian Kuhn 18

Example: Binary Counter

Incrementing a binary counter: determine the bit flip cost:

Operation Counter Value Cost 00000 1 00001 1 2 00010 2 3 00011 1 4 00100 3 5 00101 1 6 00110 2 7 00111 1 8 01000 4 9 01001 1 10 01010 2 11 01011 1 12 01100 3 13 01101 1

Algorithm Theory, WS 2012/13 Fabian Kuhn 19

Accounting Method

Observation:

• Each increment flips exactly one 0 into a 1

001001111 ⟹ 001000000 Idea:

• Have a bank account (with initial amount 0)
• Paying to the bank account costs
• Take “money” from account to pay for expensive operations

Applied to binary counter:

• Flip from 0 to 1: pay 1 to bank account (cost: 2)
• Flip from 1 to 0: take 1 from bank account (cost: 0)
• Amount on bank account = number of ones

 We always have enough “money” to pay!

Algorithm Theory, WS 2012/13 Fabian Kuhn 20

Accounting Method

Op. Counter Cost To Bank From Bank Net Cost Credit 0 0 0 0 0 1 0 0 0 0 1 1 2 0 0 0 1 0 2 3 0 0 0 1 1 1 4 0 0 1 0 0 3 5 0 0 1 0 1 1 6 0 0 1 1 0 2 7 0 0 1 1 1 1 8 0 1 0 0 0 4 9 0 1 0 0 1 1 10 0 1 0 1 0 2

Algorithm Theory, WS 2012/13 Fabian Kuhn 21

Potential Function Method

• Most generic and elegant way to do amortized analysis!

– But, also more abstract than the others…

• State of data structure / system: ∈ (state space)

Potential function : →

• Operation :

– : actual cost of operation – : state after execution of operation (: initial state) – ≔ Φ: potential after exec. of operation – : amortized cost of operation :

≔

Algorithm Theory, WS 2012/13 Fabian Kuhn 22

Potential Function Method

Operation : actual cost: amortized cost: Φ Φ Overall cost: ≔

• Φ Φ

Algorithm Theory, WS 2012/13 Fabian Kuhn 23

Binary Counter: Potential Method

• Potential function:

:

• Clearly, Φ 0 and Φ 0 for all 0
• Actual cost :
• 1 flip from 0 to 1
• 1 flips from 1 to 0
• Potential difference: Φ Φ 1 1 2
• Amortized cost: Φ Φ 2

Algorithm Theory, WS 2012/13 Fabian Kuhn 24

Back to Fibonacci Heaps

• Worst‐case cost of a single delete‐min or decrease‐key
• peration is Ω
• Can we prove a small worst‐case amortized cost for

delete‐min and decrease‐key operations? Remark:

• Data structure that allows operations , … ,
• We say that operation has amortized cost if for every

execution the total time is ⋅

• ,

where is the number of operations of type

Algorithm Theory, WS 2012/13 Fabian Kuhn 25

Amortized Cost of Fibonacci Heaps

• Initialize‐heap, is‐empty, get‐min, insert, and merge

have worst‐case cost

• Delete‐min has amortized cost
• Decrease‐key has amortized cost
• Starting with an empty heap, any sequence of operations

with at most delete‐min operations has total cost (time) .

• We will now need the marks…
• Cost for Dijkstra: || || log ||

Algorithm Theory, WS 2012/13 Fabian Kuhn 26

26 Winter term 11/12

Fibonacci Heaps: Marks

Cycle of a node:

• 1. Node is removed from root list and linked to a node

.

• 2. Child node of is cut and added to root list

.

• 3. Second child of is cut

node is cut as well and moved to root list .

• The boolean value . indicates whether node has lost a

child since the last time was made the child of another node.

Algorithm Theory, WS 2012/13 Fabian Kuhn 27

Potential Function

System state characterized by two parameters:

• : number of trees (length of . )
• : number of marked nodes that are not in the root list

Potential function: ≔ Example:

• 7, 2  Φ 11

14 25 5 20 12 22 9 18 2 8 17 1 13 15 3 7 19 31

Algorithm Theory, WS 2012/13 Fabian Kuhn 28

Actual Time of Operations

• Operations: initialize‐heap, is‐empty, insert, get‐min, merge

actual time: 1

– Normalize unit time such that

, , , , 1

• Operation delete‐min:

– Actual time: length of . – Normalize unit time such that

length of .

• Operation descrease‐key:

– Actual time: length of path to next unmarked ancestor – Normalize unit time such that

length of path to next unmarked ancestor

Algorithm Theory, WS 2012/13 Fabian Kuhn 29

Amortized Times

Assume operation is of type:

• initialize‐heap:

– actual time: 1, potential: Φ Φ 0 – amortized time: Φ Φ 1

• is‐empty, get‐min:

– actual time: 1, potential: Φ Φ (heap doesn’t change) – amortized time: Φ Φ 1

• merge:

– Actual time: 1 – combined potential of both heaps: Φ Φ – amortized time: Φ Φ 1

Algorithm Theory, WS 2012/13 Fabian Kuhn 30

Amortized Time of Insert

Assume that operation is an insert operation:

• Actual time: 1
• Potential function:

– remains unchanged (no nodes are marked or unmarked, no marked nodes are moved to the root list) – grows by 1 (one element is added to the root list) , 1 Φ Φ 1

• Amortized time:

Algorithm Theory, WS 2012/13 Fabian Kuhn 31

Amortized Time of Delete‐Min

Assume that operation is a delete‐min operation: Actual time: . Potential function :

• : changes from . to at most
• : (# of marked nodes that are not in the root list)

– no new marks – if node is moved away from root list, . is set to false  value of does not change!

, . Φ Φ |. | Amortized Time:

Algorithm Theory, WS 2012/13 Fabian Kuhn 32

Amortized Time of Decrease‐Key

Assume that operation is a decrease‐key operation at node : Actual time: length of path to next unmarked ancestor Potential function :

• Assume, node and nodes , … , are moved to root list

– , … , are marked and moved to root list, . mark is set to true

• marked nodes go to root list, 1 node gets newly marked
• grows by 1, grows by 1 and is decreased by

1, 1 Φ Φ 1 2 1 Φ 3 Amortized time:

Algorithm Theory, WS 2012/13 Fabian Kuhn 33

Complexities Fibonacci Heap

• Initialize‐Heap:
• Is‐Empty:
• Insert:
• Get‐Min:
• Delete‐Min:
• Decrease‐Key:
• Merge (heaps of size and , ):
• How large can get?
• amortized

Algorithm Theory, WS 2012/13 Fabian Kuhn 34

Rank of Children

Lemma: Consider a node of rank and let , … , be the children of in the order in which they were linked to . Then, . Proof:

Algorithm Theory, WS 2012/13 Fabian Kuhn 35

Size of Trees

Fibonacci Numbers:

• 0,
• 1,

∀ 2:

• Lemma:

In a Fibonacci heap, the size of the sub‐tree of a node with rank is at least

.

Proof:

• : minimum size of the sub‐tree of a node of rank

Algorithm Theory, WS 2012/13 Fabian Kuhn 36

Size of Trees

1, 2, ∀ 2: 2

• Claim about Fibonacci numbers:

∀ 0:

1

Algorithm Theory, WS 2012/13 Fabian Kuhn 37

Size of Trees

1, 2, ∀ 2: 2 ,

• 1
• Claim of lemma:

Algorithm Theory, WS 2012/13 Fabian Kuhn 38

Size of Trees

Lemma: In a Fibonacci heap, the size of the sub‐tree of a node with rank is at least

.

Theorem: The maximum rank of a node in a Fibonacci heap of size is at most . Proof:

• The Fibonacci numbers grow exponentially:
• 1

5 ⋅ 1 5 2

• 1

5 2

• For , we need

nodes.

Algorithm Theory, WS 2012/13 Fabian Kuhn 39

Summary: Binomial and Fibonacci Heaps

Binomial Heap Fibonacci Heap initialize

• insert
• get‐min
• delete‐min

* decrease‐key * merge

• is‐empty
• ∗ amortized time