Advanced Data Structures Lecturer: Shi Li Department of Computer - PowerPoint PPT Presentation

CSE 431/531: Algorithm Analysis and Design (Spring 2020) Advanced Data Structures Lecturer: Shi Li Department of Computer Science and Engineering University at Buffalo

Outline Heap: Concrete Data Structure for Priority Queue 1 Self-Balancing Binary-Search Tree 2 Counting inversions using Self-Balancing Binary-Search Tree Binary Search Tree Longest Increasing Subsequence using Self-Balancing BST 2/39

Let V be a ground set of size n . Def. A priority queue is an abstract data structure that maintains a set U ⊆ V of elements, each with an associated key value, and supports the following operations: insert ( v, key value ) : insert an element v ∈ V \ U , with associated key value key value . decrease key ( v, new key value ) : decrease the key value of an element v ∈ U to new key value extract min () : return and remove the element in U with the smallest key value · · · 3/39

Simple Implementations for Priority Queue n = size of ground set V data structures insert extract min decrease key array O (1) O ( n ) O (1) sorted array O ( n ) O (1) O ( n ) heap O (lg n ) O (lg n ) O (lg n ) 4/39

Heap The elements in a heap is organized using a complete binary tree: 1 Nodes are indexed as { 1 , 2 , 3 , · · · , s } Parent of node i : ⌊ i/ 2 ⌋ 2 3 Left child of node i : 2 i 4 5 6 7 Right child of node i : 2 i + 1 8 10 9 5/39

Heap A heap H contains the following fields s : size of U (number of elements in the heap) A [ i ] , 1 ≤ i ≤ s : the element at node i of the tree p [ v ] , v ∈ U : the index of node containing v key [ v ] , v ∈ U : the key value of element v 1 f s = 5 A = (‘ f ’ , ‘ g ’ , ‘ c ’ , ‘ e ’ , ‘ b ’) 2 3 g c p [‘ f ’] = 1 , p [‘ g ’] = 2 , p [‘ c ’] = 3 , p [‘ e ’] = 4 , p [‘ b ’] = 5 4 5 e b 6/39

Heap The following heap property is satisfied: for any two nodes i , j such that i is the parent of j , we have key [ A [ i ]] ≤ key [ A [ j ]] . 2 4 5 10 9 7 11 15 17 20 17 15 8 16 23 21 16 19 A heap. Numbers in the circles denote key values of elements. 7/39

insert ( v, key value ) 2 3 5 4 9 7 11 15 10 20 17 15 8 16 23 21 16 19 17 8/39

heapify-up( i ) insert ( v, key value ) while i > 1 1 s ← s + 1 j ← ⌊ i/ 2 ⌋ 1 2 A [ s ] ← v if key [ A [ i ]] < key [ A [ j ]] then 2 3 p [ v ] ← s swap A [ i ] and A [ j ] 3 4 key [ v ] ← key value p [ A [ i ]] ← i , p [ A [ j ]] ← j 4 5 heapify up ( s ) i ← j 5 6 else break 7 9/39

extract min() 17 3 17 4 3 5 17 10 4 9 7 11 15 17 10 20 17 15 8 16 23 21 16 19 10/39

extract min() heapify-down ( i ) ret ← A [1] while 2 i ≤ s 1 1 if 2 i = s or A [1] ← A [ s ] 2 2 key [ A [2 i ]] ≤ key [ A [2 i + 1]] then p [ A [1]] ← 1 3 j ← 2 i 3 s ← s − 1 4 else if s ≥ 1 then 4 5 j ← 2 i + 1 heapify down (1) 5 6 if key [ A [ j ]] < key [ A [ i ]] then return ret 6 7 swap A [ i ] and A [ j ] 7 decrease key ( v, key value ) p [ A [ i ]] ← i , p [ A [ j ]] ← j 8 key [ v ] ← key value 1 i ← j 9 heapify-up ( p [ v ]) 2 else break 10 11/39

Running time of heapify up and heapify down: O (lg n ) Running time of insert, exact min and decrease key: O (lg n ) data structures insert extract min decrease key array O (1) O ( n ) O (1) sorted array O ( n ) O (1) O ( n ) heap O (lg n ) O (lg n ) O (lg n ) 12/39

Two Definitions Needed to Prove that the Procedures Maintain Heap Property Def. We say that H is almost a heap except that key [ A [ i ]] is too small if we can increase key [ A [ i ]] to make H a heap. Def. We say that H is almost a heap except that key [ A [ i ]] is too big if we can decrease key [ A [ i ]] to make H a heap. 13/39

Outline Heap: Concrete Data Structure for Priority Queue 1 Self-Balancing Binary-Search Tree 2 Counting inversions using Self-Balancing Binary-Search Tree Binary Search Tree Longest Increasing Subsequence using Self-Balancing BST 14/39

Outline Heap: Concrete Data Structure for Priority Queue 1 Self-Balancing Binary-Search Tree 2 Counting inversions using Self-Balancing Binary-Search Tree Binary Search Tree Longest Increasing Subsequence using Self-Balancing BST 15/39

Counting Inversions 15 3 16 12 32 7 inversions ( A, n ) T ← empty Binary Search 1 i = 1: rank(15) = 1 Tree i = 2: rank( 3) = 1 c ← 0 i = 3: rank(16) = 3 2 i = 4: rank(12) = 2 for i ← 1 to n 3 i = 5: rank(32) = 5 c ← c + i − T. rank ( A [ i ]) 4 i = 6: rank( 7) = 2 T .insert ( A [ i ]) 5 c = (1 − 1) + (2 − 1) + (3 − 3) return c +(4 − 2) + (5 − 5) + (6 − 2) = 7 6 16/39

Outline Heap: Concrete Data Structure for Priority Queue 1 Self-Balancing Binary-Search Tree 2 Counting inversions using Self-Balancing Binary-Search Tree Binary Search Tree Longest Increasing Subsequence using Self-Balancing BST 17/39

A self-balancing binary search tree T maintains a set of comparable elements and supports: Insertion of an element to T Deletion of an element from T Whether an element exists in T Return the rank of an element in T (i.e, 1 plus number of elements in T smaller than the element) Return the i -th smallest element in T ... Each operation takes time O (lg n ) 18/39

Binary Search Trees For any node v in tree: key in v must be greater than 8 all keys on the left-sub-tree of 3 10 v key in v must be smaller than all keys on the right-sub-tree 1 6 14 of v 4 7 13 in-order traversal of tree gives a sorted list of keys 19/39

Binary Search Trees: Insertition 8 3 10 1 6 14 4 7 13 5 20/39

Binary Search Trees: Insertion insert ( v, key ) if key < v.key 1 if v.left = nil then 2 create a new node u 3 u.key ← key, u.left ← nil , u.right ← nil 4 v.left ← u 5 else insert ( v.left, key ) 6 else 7 if v.right = nil then 8 create a new node u 9 u.key ← key, u.left ← nil , u.right ← nil 10 v.right ← u 11 else insert ( v.right, key ) 12 21/39

Binary Search Trees: Deletion 20 8 7 3 10 1 5 14 2 4 7 6 13 22/39

Binary Search Trees: Rank Need to maintain a field “size” 8 9 3 10 3 5 1 6 14 1 3 2 4 7 13 1 1 1 23/39

Binary Search Trees: Rank rank ( v, key ) if key ≤ v.key 1 if v.left = nil then return 1 2 else return rank ( v.left, key ) 3 else 4 if v.right = nil then return v.size + 1 5 else return v.size − v.right.size + rank ( v.right, key ) 6 24/39

Running Time for Operations each operation takes time O ( d ) . d = depth of tree best case: d = Θ(lg n ) worst case: d = Θ( n ) 1 2 3 4 4 5 2 6 6 1 3 5 7 7 25/39

Self-Balancing BST: automatically keep the height of tree small AVL tree red-black tree Splay tree Treap ... 26/39

AVL Tree Property of an AVL tree For every node v in the tree, the depths of the left-sub-tree of v and right-sub-tree of v differ by at most 1. 8 8 0 vs 2 3 10 3 10 1 6 14 1 6 9 14 4 7 13 4 7 13 not balanced balanced 27/39

AVL Tree Property of an AVL tree For every node v in the tree, the depths of the left-sub-tree of v and right-sub-tree of v differ by at most 1. Why does the property guarantee that the height of a tree is O (log n ) ? f ( d ) : minimum number of nodes in an AVL tree of depth d f (0) = 0 , f (1) = 1 , f (2) = 2 , f (3) = 4 , f (4) = 7 · · · 28/39

f ( d ) : minimum number of nodes in an AVL tree of depth d Recursion: f (0) = 0 f (1) = 1 f ( d ) = f ( d − 1) + f ( d − 2) + 1 d ≥ 2 f ( d ) = 2 Θ( d ) 29/39

Depth of AVL tree f ( d ) : minimum number of nodes in an AVL tree of depth d f ( d ) = 2 Θ( d ) If a AVL tree has size n and depth d , then n ≥ f ( d ) Thus, d = O (log n ) 30/39

Property of an AVL tree For every node v in the tree, the depths of the left-sub-tree of v and right-sub-tree of v differ by at most 1. 8 8 0 vs 2 3 10 3 10 1 6 14 1 6 9 14 4 7 13 4 7 13 not balanced balanced How can we maintain the property? Assume we only do insertions; there are no deletions. 31/39

Maintain Balance Property A : the deepest node such that the balance property is not satisfied after insertion Wlog, we inserted an element to the left-sub-tree of A B : the root of left-sub-tree of A case 1: we inserted an element to the left-sub-tree of B d + 2 A B d + 1 A d + 2 d + 1 d B B L A R d d B R A R d d + 1 B R B L 32/39

Maintain Balance Property A : the deepest node such that the balance property is not satisfied after insertion Wlog, we inserted an element to the left-sub-tree of A B : the root of left-sub-tree of A case 2: we inserted an element to the right-sub-tree of B C : the root of right-sub-tree of B d + 2 A C d + 1 d + 1 d + 2 d B A B A R d + 1 d C d − 1 d B L d d C R C L A R B L d − 1 d C R C L 33/39

Outline Heap: Concrete Data Structure for Priority Queue 1 Self-Balancing Binary-Search Tree 2 Counting inversions using Self-Balancing Binary-Search Tree Binary Search Tree Longest Increasing Subsequence using Self-Balancing BST 34/39