B INARY S EARCH T REES Acknowledgement: The course slides are - - PowerPoint PPT Presentation

BBM 202 - ALGORITHMS

BINARY SEARCH TREES


• DEPT. OF COMPUTER ENGINEERING

Acknowledgement: The course slides are adapted from the slides prepared by 


• R. Sedgewick and K. Wayne of Princeton University.

TODAY

• BSTs
• Ordered operations
• Deletion

Binary Search Tree (BST)

• Last lecture, we talked about binary search & linear search
• One had high cost for reorganisation,
• The other had high cost for searching
• In this lecture we will use Binary Trees, for searching
• Plan in a nutshell:
• Assert a more strict property compared to the Heap-Property (in

priority-queues), Remember what that was?

• Know exactly which subtree to look for at each node

3

• Definition. A BST is a binary tree in symmetric order.


 
 A binary tree is either:

• Empty.
• Two disjoint binary trees (left and right).


 
 
 Symmetric order. Each node has a key,
 and every node’s key is:

• Larger than all keys in its left subtree.
• Smaller than all keys in its right subtree.

4

Binary search trees

right child

• f root

a left link a subtree root

null links

Anatomy of a binary tree value associated with R parent of A and R left link

• f E

keys smaller than E keys larger than E key A C E H R S X

9

Anatomy of a binary search tree

Java definition. A BST is a reference to a root Node. A Node is comprised of four fields:

• A Key and a Value.
• A reference to the left and right subtree.

5

BST representation in Java

smaller keys larger keys

private class Node { private Key key; private Value val; private Node left, right; public Node(Key key, Value val) { this.key = key; this.val = val; } }

Key and Value are generic types; Key is Comparable

Binary search tree

BST with smaller keys BST with larger keys

key left right val

BST

Node

public class BST<Key extends Comparable<Key>, Value> { private Node root; private class Node { /* see previous slide */ } public void put(Key key, Value val) { /* see next slides */ } public Value get(Key key) { /* see next slides */ } public void delete(Key key) { /* see next slides */ } public Iterable<Key> iterator() { /* see next slides */ } }

6

BST implementation (skeleton)

root of BST

• Search. If less, go left; if greater, go right; if equal, search hit.

7

Binary search tree operations

successful search for H

X R A C H E S M

• Search. If less, go left; if greater, go right; if equal, search hit.

8

Binary search tree operations

black nodes could match the search key compare H and S (go left)

X R A C H E S M H

successful search for H

• Search. If less, go left; if greater, go right; if equal, search hit.

9

Binary search tree operations

X R A C H E S M H

successful search for H

• Search. If less, go left; if greater, go right; if equal, search hit.

10

Binary search tree operations

X R A C H E S M H

compare H and E (go right)

successful search for H

• Search. If less, go left; if greater, go right; if equal, search hit.

11

Binary search tree operations

X R A C H E S M H

successful search for H

• Search. If less, go left; if greater, go right; if equal, search hit.

12

Binary search tree operations

X R A C H E S M H

compare H and R (go left)

successful search for H

• Search. If less, go left; if greater, go right; if equal, search hit.

13

Binary search tree operations

X R A C H E S M H

successful search for H

• Search. If less, go left; if greater, go right; if equal, search hit.

14

Binary search tree operations

X R A C H E S M H

compare H and H (search hit)

successful search for H

• Search. If less, go left; if greater, go right; if equal, search hit.

15

Binary search tree operations

unsuccessful search for G

X R A C H E S M

• Search. If less, go left; if greater, go right; if equal, search hit.

16

Binary search tree operations

compare G and S (go left)

X R A C H E S M G

unsuccessful search for G

• Search. If less, go left; if greater, go right; if equal, search hit.

17

Binary search tree operations

X R A C H E S M G

unsuccessful search for G

• Search. If less, go left; if greater, go right; if equal, search hit.

18

Binary search tree operations

X R A C H E S M G

compare G and E (go right)

unsuccessful search for G

• Search. If less, go left; if greater, go right; if equal, search hit.

19

Binary search tree operations

X R A C H E S M G

unsuccessful search for G

• Search. If less, go left; if greater, go right; if equal, search hit.

20

Binary search tree operations

X R A C H E S M G

compare G and R (go left)

unsuccessful search for G

• Search. If less, go left; if greater, go right; if equal, search hit.

21

Binary search tree operations

X R A C H E S M G

unsuccessful search for G

• Search. If less, go left; if greater, go right; if equal, search hit.

22

Binary search tree operations

X R A C H E S M G

compare G and H (go left)

unsuccessful search for G

• Search. If less, go left; if greater, go right; if equal, search hit.

23

Binary search tree operations

X R A C H E S M G

unsuccessful search for G

• Search. If less, go left; if greater, go right; if equal, search hit.

24

Binary search tree operations

X R A C H E S M

unsuccessful search for G

G

no more tree (search miss)

• Insert. If less, go left; if greater, go right; if null, insert.

25

Binary search tree operations

insert G

X R A C H E S M

• Insert. If less, go left; if greater, go right; if null, insert.

26

Binary search tree operations

compare G and S (go left)

X R A C H E S M G

insert G

• Insert. If less, go left; if greater, go right; if null, insert.

27

Binary search tree operations

X R A C H E S M G

insert G

• Insert. If less, go left; if greater, go right; if null, insert.

28

Binary search tree operations

X R A C H E S M G

compare G and E (go right)

insert G

• Insert. If less, go left; if greater, go right; if null, insert.

29

Binary search tree operations

X R A C H E S M G

insert G

• Insert. If less, go left; if greater, go right; if null, insert.

30

Binary search tree operations

X R A C H E S M G

compare G and R (go left)

insert G

• Insert. If less, go left; if greater, go right; if null, insert.

31

Binary search tree operations

X R A C H E S M G

insert G

• Insert. If less, go left; if greater, go right; if null, insert.

32

Binary search tree operations

X R A C H E S M G

compare G and H (go left)

insert G

• Insert. If less, go left; if greater, go right; if null, insert.

33

Binary search tree operations

X R A C H E S M G

insert G

• Insert. If less, go left; if greater, go right; if null, insert.

34

Binary search tree operations

X R A C H E S M G

no more tree (insert here)

insert G

• Insert. If less, go left; if greater, go right; if null, insert.

35

Binary search tree operations

X R A C H E S M

insert G

G

• Insert. If less, go left; if greater, go right; if null, insert.

36

Binary search tree operations

X R A C H E S M

insert G

G

• Get. Return value corresponding to given key, or null if no such key.

37

BST search

R is less than S

so look to the left black nodes could match the search key gray nodes cannot match the search key found R (search hit) so return value

R is greater than E

so look to the right A C E H M R S X A C E H M R S X A C E H M R S X

T is less than X

so look to the left link is null so T is not in tree (search miss)

T is greater than S

so look to the right A C E H M R S X A C E H M R S X

successful search for R unsuccessful search for T

• Get. Return value corresponding to given key, or null if no such key.

• Cost. Number of compares is equal to 1 + depth of node.

38

BST search: Java implementation

public Value get(Key key) { Node x = root; while (x != null) { int cmp = key.compareTo(x.key); if (cmp < 0) x = x.left; else if (cmp > 0) x = x.right; else if (cmp == 0) return x.val; } return null; }

• Put. Associate value with key.

Search for key, then two cases:

• Key in tree ⇒ reset value.
• Key not in tree ⇒ add new node.

39

BST insert

search for L ends at this null link reset links

• n the way up

create new node A C E H M P R S X A C E H L M P R S X A C E H L M P R S X Insertion into a BST

inserting L

• Put. Associate value with key.

• Cost. Number of compares is equal to 1 + depth of node.

40

BST insert: Java implementation

public void put(Key key, Value val) { root = put(root, key, val); } private Node put(Node x, Key key, Value val) { if (x == null) return new Node(key, val); int cmp = key.compareTo(x.key); if (cmp < 0)
 x.left = put(x.left, key, val);
 else if (cmp > 0)
 x.right = put(x.right, key, val);
 else if (cmp == 0)
 x.val = val; return x; }

concise, but tricky, recursive code;
 read carefully!

Always assign the subtree returned from recursive call to a child, but does it actually change in each call ?

41

BST trace: standard indexing client

S A C E H R S X A C E H R S A C E H R S A C E R S A E R A E S S E S S

6

S 0 E 1 A 2 R 3 C 4 H 5 E 6 X 7 red nodes are new black nodes are accessed in search changed value changed value changed value gray nodes are untouched A C E H M P R S X A C E H M R S X A C E H R S X A C E H L M P R S X A C E H L M P R S X

12 8

A 8 M 9 P 10 L 11 E 12

key value key value

• Many BSTs correspond to same set of keys.
• Number of compares for search/insert is equal to 1 + depth of node.
• Remark. Tree shape depends on order of insertion.

42

Tree shape

X S R C E H A

best case

A C E H R S X

typical case

A H S R X C E

worst case

43

BST insertion: random order visualization

• Ex. Insert keys in random order.

• Remark. Correspondence is 1-1 if array has no duplicate keys.

44

Correspondence between BSTs and quicksort partitioning

A D H O P T U C Y E I M L

• Proposition. If N distinct keys are inserted into a BST in random order,


the expected number of compares for a search/insert is O(log N).

• Pf. 1-1 correspondence with quicksort partitioning.


 
 But… Worst-case height is N.
 (exponentially small chance when keys are inserted in random order)

45

BSTs: mathematical analysis

46

ST implementations: frequency counter

Costs for java FrequencyCounter 8 < tale.txt using BinarySearchST 5737 14350

• perations

cost

484 Costs for java FrequencyCounter 8 < tale.txt using BST 20 14350

• perations

cost

13.9

47

ST implementations: summary

implementation guarantee average case

• rdered
• ps?
• perations
• n keys

search insert search hit insert sequential search (unordered list) N N N/2 N no equals() binary search (ordered array) lg N N lg N N/2 yes compareTo() BST N N lg N lg N stay tuned compareTo()

BINARY SEARCH TREES

• BSTs
• Ordered operations
• Deletion

• Minimum. Smallest key in table.
• Maximum. Largest key in table.
• Q. How to find the min / max?

Minimum and maximum

49

A C E H M R S X

min() max()

max min

• Floor. Largest key ≤ to a given key.
• Ceiling. Smallest key ≥ to a given key.
• Q. How to find the floor /ceiling?

Floor and ceiling

50

A C E H M R S X

min() max()

floor(D) ceiling(Q) floor(G)

Case 1. [k equals the key at root]
 The floor of k is k. 
 Case 2. [k is less than the key at root]
 The floor of k is in the left subtree. 
 Case 3. [k is greater than the key at root]
 The floor of k is in the right subtree
 (if there is any key ≤ k in right subtree);


• therwise it is the key in the root.

Computing the floor

51

floor(G)in left

subtree is null result

fjnding floor(G) G is greater than E so floor(G) could be

• n the right

G is less than S so floor(G) must be

• n the left

A C E H M R S X A C E H M R S X A C E H M R S X A C E H M R S X

Computing the floor

52

public Key floor(Key key)
 { 
 Node x = floor(root, key);
 if (x == null) return null;
 return x.key;
 } private Node floor(Node x, Key key)
 { 
 if (x == null) return null;
 int cmp = key.compareTo(x.key); if (cmp == 0) return x; if (cmp < 0) return floor(x.left, key); Node t = floor(x.right, key);
 if (t != null) return t;
 else return x; }

floor(G)in left

subtree is null result

fjnding floor(G) G is greater than E so floor(G) could be

• n the right

G is less than S so floor(G) must be

• n the left

A C E H M R S X A C E H M R S X A C E H M R S X A C E H M R S X

In each node, we store the number of nodes in the subtree rooted at that node; to implement size(), return the count at the root.

• Remark. This facilitates efficient implementation of rank() and select().

53

Subtree counts

A A C E H M R S X C E H M R S X

2 6 8 1 1 1 3 2

node count N A A C E H M R S X C E H M R S X

5 8 1 1 1 2 2 2

public int size() { return size(root); } private int size(Node x) { if (x == null) return 0; return x.N; }

54

BST implementation: subtree counts

private class Node { private Key key; private Value val; private Node left; private Node right; private int N; } private Node put(Node x, Key key, Value val) { if (x == null) return new Node(key, val); int cmp = key.compareTo(x.key); if (cmp < 0) x.left = put(x.left, key, val); else if (cmp > 0) x.right = put(x.right, key, val); else if (cmp == 0) x.val = val; x.N = 1 + size(x.left) + size(x.right); return x; }

number of nodes
 in subtree

• k to call when

x is null

55

Rank

• Rank. How many keys < k ?

Easy recursive algorithm (3 cases!)

public int rank(Key key) { return rank(key, root); } private int rank(Key key, Node x) { if (x == null) return 0; int cmp = key.compareTo(x.key); if (cmp < 0) return rank(key, x.left); else if (cmp > 0) return 1 + size(x.left) + rank(key, x.right); else if (cmp == 0) return size(x.left); }

A A C E H M R S X C E H M R S X

2 6 8 1 1 1 3 2

node count N

• Select. Key of given rank.

Selection

56

public Key select(int k) { if (k < 0) return null; if (k >= size()) return null; Node x = select(root, k); return x.key; } private Node select(Node x, int k) { if (x == null) return null; int t = size(x.left); if (t > k) return select(x.left, k); else if (t < k) return select(x.right, k-t-1); else if (t == k) return x; }

8 keys in left subtree

so search for key of rank 3 on the left count N

8 2 keys in left subtree so

search for key of rank

3-2-1 = 0 on the right 2 0 keys in left subtree

and searching for key of rank 0 so return H

2 keys in left subtree

so search for key of rank 0 on the left

2

A C E H M R S X A C E H M R S X A C E H M R S X A C E H M R S X

fjnding select(3) the key of rank 3

• Traverse left subtree.
• Enqueue key.
• Traverse right subtree.

• Property. Inorder traversal of a BST yields keys in ascending order.

key key val

BST with smaller keys smaller keys, in order larger keys, in order all keys, in order BST with larger keys

left right

BST

Inorder traversal

57

public Iterable<Key> keys() { Queue<Key> q = new Queue<Key>(); inorder(root, q); return q; } private void inorder(Node x, Queue<Key> q) { if (x == null) return; inorder(x.left, q); q.enqueue(x.key); inorder(x.right, q); }

• Traverse left subtree.
• Enqueue key.
• Traverse right subtree.

Inorder traversal

58

function call stack

inorder(S) inorder(E) inorder(A) enqueue A inorder(C) enqueue C enqueue E inorder(R) inorder(H) enqueue H inorder(M) enqueue M enqueue R enqueue S inorder(X) enqueue X A C E H M R S X S S E S E A S E A C S E R S E R H S E R H M S X

queue recursive calls

A A C E H M R S X C E H M R S X

59

BST: ordered symbol table operations summary

sequential
 search binary
 search BST search N lg N h insert 1 N h min / max N 1 h floor / ceiling N lg N h rank N lg N h select N 1 h

• rdered iteration

N log N N N

h = height of BST
 (proportional to log N
 if keys inserted in random order)

• rder of growth of running time of ordered symbol table operations

BINARY SEARCH TREES

• BSTs
• Ordered operations
• Deletion

61

ST implementations: summary

• Next. Deletion in BSTs.

implementation guarantee average case

• rdered

iteration?

• perations
• n keys

search insert delete search hit insert delete sequential search
 (linked list) N N N N/2 N N/2 no equals() binary search
 (ordered array) lg N N N lg N N/2 N/2 yes compareTo() BST N N N lg N lg N ? ? ? yes compareTo()

To remove a node with a given key:

• Set its value to null.
• Leave key in tree to guide searches (but don't consider it equal to search key).

• Cost. O(log N’) per insert, search, and delete (if keys in random order),


where N' is the number of key-value pairs ever inserted in the BST.

Unsatisfactory solution. Tombstone (memory) overload.

62

BST deletion: lazy approach

delete I

S E C A N R H I S E C A N R H

☠

tombstone

To delete the minimum key:

• Go left until finding a node with a null left link.
• Replace that node by its right link.
• Update subtree counts.

63

Deleting the minimum

public void deleteMin() { root = deleteMin(root); } private Node deleteMin(Node x) { if (x.left == null) return x.right; x.left = deleteMin(x.left); x.N = 1 + size(x.left) + size(x.right); return x; }

go left until reaching null left link return that node’s right link available for garbage collection

5 7

update links and node counts after recursive calls A C E H M R S X A C E H M R S X C E H M R S X

node to delete replace with null link available for garbage collection update counts after recursive calls

5 1 7

A C E H M C R S X A E H M R S X A E H M R S X

deleting C

To delete a node with key k: search for node t containing key k. Case 0. [0 children] Delete t by setting parent link to null.

64

Hibbard deletion

To delete a node with key k: search for node t containing key k. Case 1. [1 child] Delete t by replacing parent link.

65

Hibbard deletion

node to delete replace with child link available for garbage collection A C C C E H M R R S X A E H M S X A E H M S X

deleting R

update counts after recursive calls

5 7

To delete a node with key k: search for node t containing key k. Case 2. [2 children]

• Find successor x of t.
• Delete the minimum in t's right subtree.
• Put x in t's spot.

66

Hibbard deletion

x has no left child but don't garbage collect x still a BST

deleteMin(t.right)

5 7

t.left

x

update links and node counts after recursive calls A C H A C H M R M R S X E S X reaching null left link search for key E node to delete

t x

successor min(t.right) A C E H M R S X A C E H M R S X S go right, then go left until reaching null left link

67

Hibbard deletion: Java implementation

public void delete(Key key) { root = delete(root, key); } private Node delete(Node x, Key key) { if (x == null) return null; int cmp = key.compareTo(x.key); if (cmp < 0) x.left = delete(x.left, key); else if (cmp > 0) x.right = delete(x.right, key); else { if (x.right == null) return x.left; Node t = x; x = min(t.right); x.right = deleteMin(t.right); x.left = t.left; } x.N = size(x.left) + size(x.right) + 1; return x; }

no right child replace with successor search for key update subtree counts

68

Hibbard deletion: analysis

Unsatisfactory solution. Not symmetric. 
 
 
 
 
 
 
 
 
 
 
 
 Surprising consequence. Trees not random (!) ⇒ sqrt (N) per op. Longstanding open problem. Simple and efficient delete for BSTs. If we always delete from the same side, the shape of tree will be not random, the right subtrees are trimmed!

Red-black BST. Guarantee logarithmic performance for all operations.

69

ST implementations: summary

implementation guarantee average case

• rdered

iteration?

• perations
• n keys

search insert delete search hit insert delete sequential search
 (linked list) N N N N/2 N N/2 no equals() binary search
 (ordered array) lg N N N lg N N/2 N/2 yes compareTo() BST N N N lg N lg N √N yes compareTo()

• ther operations also become √N

if deletions allowed