Balanced Search Trees 2-3-4 trees red-black trees References: - - PowerPoint PPT Presentation

Balanced Search Trees

2-3-4 trees red-black trees

References: Algorithms in Java (handout)

Balanced search trees

Dynamic sets

• Search
• Insert
• Delete
• Maximum
• Minimum
• Successor(x) (find minimum element ≥ x)
• Predecessor(x) (find maximum element ≤ x)

This lecture: 2-3-4 trees, red-black trees Next time: Tiered vektor (not a binary search tree, but maintains a dynamic set). In two weeks time: Splay trees

2

Dynamic set implementations

Worst case running times In worst case h=n. In best case h= log n (fully balanced binary tree) Today: How to keep the trees balanced.

3

Implementation search insert delete minimum maximum successor predecessor linked lists O(n) O(1) O(1) O(n) O(n) O(n) O(n)

• rdered array

O(log n) O(n) O(n) O(1) O(1) O(log n) O(log n) BST O(h) O(h) O(h) O(h) O(h) O(h) O(h)

2-3-4 trees

2-3-4 trees

2-3-4 trees. Allow nodes to have multiple keys. Perfect balance. Every path from root to leaf has same length. Allow 1, 2, or 3 keys per node

• 2-node: one key, 2 children
• 3-node: 2 keys, 3 children
• 4-node: 3 keys, 4 children

5

S V K R M O X A C L N Q Y Z

smaller than K larger than R between K and R

B D F G J E

Search.

• Compare search key against keys in node.
• Find interval containing search key
• Follow associated link (recursively)

Searching in a 2-3-4 tree

6

S V F G J K R C E M O X A D L N Q Y Z

Search.

• Compare search key against keys in node.
• Find interval containing search key
• Follow associated link (recursively)
• Ex. Search for L

Searching in a 2-3-4 tree

7

between K and R smaller than M

S V F G J K R C E M O X A D L N Q Y Z

found L

Where is the predecessor of L? And the successor of L?

Predecessor and successor in a 2-3-4 tree

8

between K and R smaller than M

S V F G J K R C E M O X A D L N Q Y Z

found L

Insertion in a 2-3-4 tree

9

S V F G J K R C E M O X A D L N Q Y Z

Insertion in a 2-3-4 tree

Insert.

• Search to bottom for key.
• Ex. Insert B

10

S V F G J K R C E M O X A D L N Q Y Z

smaller than K B not found smaller than C

Insertion in a 2-3-4 tree

Insert.

• Search to bottom for key.
• 2-node at bottom: convert to 3-node
• Ex. Insert B

11

S V F G J K R C E M O X A B D L N Q Y Z

smaller than K smaller than C B fits here

Insert.

• Search to bottom for key.
• 2-node at bottom: convert to 3-node
• Ex. Insert X

Insertion in a 2-3-4 tree

12

S T F G J K R C E M O U A D L N Q Y Z

X not found larger than R larger than U

Insert.

• Search to bottom for key.
• 2-node at bottom: convert to 3-node
• 3-node at bottom: convert to 4-node
• Ex. Insert X

Insertion in a 2-3-4 tree

13

S T F G J K R C E M O U A D L N Q X Y Z

X fits here larger than R larger than W

larger than E H not found

Insertion in a 2-3-4 tree

14

S V F G J K R C E M O X A D L N Q Y Z

smaller than K

Insert.

• Search to bottom for key.
• 2-node at bottom: convert to 3-node
• 3-node at bottom: convert to 4-node
• Ex. Insert H

larger than E H does not fit here!

Insertion in a 2-3-4 tree

15

S V F G J K R C E M O X A D L N Q Y Z

smaller than K

Insert.

• Search to bottom for key.
• 2-node at bottom: convert to 3-node
• 3-node at bottom: convert to 4-node
• 4-node at bottom: ??
• Ex. Insert H

Splitting a 4-node in a 2-3-4 tree

Idea: split the 4-node to make room Problem: Doesn’t work if parent is a 4-node Solution 1: Split the parent (and continue splitting while necessary). Solution 2: Split 4-nodes on the way down.

16

H does not fit here

F G J C E D A B D

H does fit here!

A B C E G F J D A B C E G F H J

Idea: split 4-nodes on the way down the tree.

• Ensures last node is not a 4-node.
• Transformations to split 4-nodes:
• Invariant. Current node is not a 4-node.
• Consequence. Insertion at bottom is easy

since it's not a 4-node.

Splitting 4-nodes in a 2-3-4 tree

17

F G J B G B F J

x y z v x y z v

F G J

x y z v

F

x y

J

z v

B D B D G

x y z v

B D G D B G

x y z v

root

Insert.

• Search to bottom for key.
• 2-node at bottom: convert to 3-node
• 3-node at bottom: convert to 4-node
• 4-node at bottom: ??
• Ex. Insert H

Insertion in a 2-3-4 tree

18

S V F G J K R C E M O X A D L N Q Y Z

not a 4-node not a 4-node 4-node

Insert.

• Search to bottom for key.
• 2-node at bottom: convert to 3-node
• 3-node at bottom: convert to 4-node
• 4-node at bottom: ??
• Ex. Insert H

Insertion in a 2-3-4 tree

19

S V K R C E G M O X A D L N Q Y Z F J

Insert.

• Search to bottom for key.
• 2-node at bottom: convert to 3-node
• 3-node at bottom: convert to 4-node
• 4-node at bottom: ??
• Ex. Insert H

Insertion in a 2-3-4 tree

20

S V K R C E G M O X A D L N Q Y Z F H J

Local transformations that work anywhere in the tree.

• Ex. Splitting a 4-node attached to a 2-node

Splitting 4-nodes in a 2-3-4 tree

A-C E-J L-P R-V X-Z A-C E-J L-P R-V X-Z

K Q W D Q D K W

could be huge unchanged

21

Local transformations that work anywhere in the tree

• Ex. Splitting a 4-node attached to a 3-node

Splitting 4-nodes in a 2-3-4 tree

A-C I-J L-P R-V X-Z I-J L-P R-V X-Z

K Q W K W

could be huge unchanged E-G

D H

A-C E-G

D H Q

22

Splitting 4-nodes in a 2-3-4 tree

Local transformations that work anywhere in the tree. Splitting a 4-node attached to a 4-node never happens when we split nodes on the way down the tree.

• Invariant. Current node is not a 4-node.

23

Insertion 2-3-4 trees

24

S A B C E R H I N A B C E R H I N S U

Insert G Insert U Split

A B C E I R S U H N A B C E I R S U N G H A B C S U N G H E R I A B C S T U N G H E R I

Insert G Split Insert T

Insert U Insert G Insert T

Deletions in 2-3-4 trees

Delete minimum:

• minimum always in leftmost leaf
• If 3- or 4-node: delete key
• Ex. Delete minimum

25

S V F G J K R C E M O X A B D L N Q Y Z

A is minimum

Deletions in 2-3-4 trees

Delete minimum:

• minimum always in leftmost leaf
• If 3- or 4-node: delete key
• Ex. Delete minimum

26

S V F G J K R C E M O X B D L N Q Y Z

Delete A

Deletions in 2-3-4 trees

Delete minimum:

• minimum always in leftmost leaf
• If 3- or 4-node: delete key
• 2-node??
• Ex. Delete minimum

27

S V F G J K R C E M O X B D L N Q Y Z

Delete B?

Idea: On the way down maintain the invariant that current node is not a 2-node.

• Child of root and root is a 2-node:
• on the way down:

Deletions in 2-3-4 trees

28

z w y x z w y x

A B C A C B

r s w z y x r s w z y x

• r

A B C D E A B C D E

r s w z y x r s w z y x

A C D E B F G C F G A B D E

z w y x z w y x

• r

A B D E C D E A B C

Deletions in 2-3-4 trees

Delete minimum:

• minimum always in leftmost leaf
• If 3- or 4-node: delete key
• 2-node: split/merge on way down.
• Ex. Delete minimum

29

S V F G J K R C E M O X B D L N Q Y Z

not a 2-node 2-node

Deletions in 2-3-4 trees

Delete minimum:

• minimum always in leftmost leaf
• If 3- or 4-node: delete key
• 2-node: split/merge on way down.
• Ex. Delete minimum

30

S V F G J K R E M O X B C D L N Q Y Z

Deletions in 2-3-4 trees

Delete minimum:

• minimum always in leftmost leaf
• If 3- or 4-node: delete key
• 2-node: split/merge on way down.
• Ex. Delete minimum

31

S V F G J K R E M O X C D L N Q Y Z

Deletions in 2-3-4 trees

Delete:

32

S V F G J K R C E M O X B D L N Q Y Z

Deletions in 2-3-4 trees

Delete:

• During search maintain invariant that current node is not a 2-node

33

S V F G J K R C E M O X B D L N Q Y Z

Deletions in 2-3-4 trees

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key

34

S V F G J K R C E M O X B D L N Q Y Z

Deletions in 2-3-4 trees

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key
• Key not in leaf: replace with successor (always leaf in subtree) and delete

successor from leaf.

35

S V F G J K R C E M O X B D L N Q Y Z

Deletions in 2-3-4 trees

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key
• Key not in leaf: replace with successor (always leaf in subtree) and delete

successor from leaf.

• Ex. Delete K

36

S V F G J K R C E M O X B D L N Q Y Z

Deletions in 2-3-4 trees

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key
• Key not in leaf: replace with successor (always leaf in subtree) and delete

successor from leaf.

• Ex. Delete K
• Find successor

37

S V F G J K R C E M O X B D L N Q Y Z

Deletions in 2-3-4 trees

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key
• Key not in leaf: replace with successor (always leaf in subtree) and delete

successor from leaf.

• Ex. Delete K
• Find successor

38

S V F G J K R C E M O X B D L N Q Y Z

Deletions in 2-3-4 trees

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key
• Key not in leaf: replace with successor (always leaf in subtree) and delete

successor from leaf.

• Ex. Delete K
• Find successor

39

S V F G J K R C E M O X B D L N Q Y Z

not a 2-node

Deletions in 2-3-4 trees

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key
• Key not in leaf: replace with successor (always leaf in subtree) and delete

successor from leaf.

• Ex. Delete K
• Find successor

40

S V F G J K R C E M O X B D L N Q Y Z

not a 2-node 2-node

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key
• Key not in leaf: replace with successor (always leaf in subtree) and delete

successor from leaf.

• Ex. Delete K
• Find successor

Deletions in 2-3-4 trees

41

S V F G J K R C E O X B D Q Y Z L M N

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key
• Key not in leaf: replace with successor (always leaf in subtree) and delete

successor from leaf.

• Ex. Delete K
• Find successor
• Delete L from leaf

Deletions in 2-3-4 trees

42

S V F G J K R C E O X B D Q Y Z L M N

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key
• Key not in leaf: replace with successor (always leaf in subtree) and delete

successor from leaf.

• Ex. Delete K
• Find successor
• Delete L from leaf

Deletions in 2-3-4 trees

43

S V F G J K R C E O X B D Q Y Z M N

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key
• Key not in leaf: replace with successor (always leaf in subtree) and delete

successor from leaf.

• Ex. Delete K
• Find successor
• Delete L from leaf
• Replace K with L

Deletions in 2-3-4 trees

44

S V F G J K R C E O X B D Q Y Z M N

Deletions in 2-3-4 trees

45

S V F G J L R C E O X B D Q Y Z M N

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key
• Key not in leaf: replace with successor (always leaf in subtree) and delete

successor from leaf.

• Ex. Delete K
• Find successor
• Delete L from leaf
• Replace K with L

2-3-4 Tree: Balance

• Property. All paths from root to leaf have same length.

Tree height. Worst case: lg N [all 2-nodes] Best case: log4 N = 1/2 lg N [all 4-nodes] Between 10 and 20 for a million nodes. Between 15 and 30 for a billion nodes.

46

Dynamic set implementations

Worst case running times

47

Implementation search insert delete minimum maximum successor predecessor linked lists O(n) O(1) O(1) O(n) O(n) O(n) O(n)

• rdered array

O(log n) O(n) O(n) O(1) O(1) O(log n) O(log n) BST O(h) O(h) O(h) O(h) O(h) O(h) O(h) 2-3-4 tree O(log n) O(log n) O(log n) O(log n) O(log n) O(log n) O(log n)

Red-black trees

Represent 2-3-4 tree as a binary search tree

• Use colors on nodes to represent 3- and 4-nodes.

Red-black tree (Guibas-Sedgewick, 1979)

G K O K G O H L

• r

H L H L

B B

49

≈ ≈ ≈

Represent 2-3-4 tree as a binary search tree

• Use colors on nodes to represent 3- and 4-nodes.
• Connection between 2-3-4 trees and red-black trees:

Red-black tree (Guibas-Sedgewick, 1979)

A A C H I N E R S

R H N C A

E I A S

50

≈

• r

≈

Represent 2-3-4 tree as a binary search tree

• Use colors on nodes to represent 3- and 4-nodes.
• Connection between 2-3-4 trees and red-black trees:

Red-black tree (Guibas-Sedgewick, 1979)

A A C H I N E R S

R H N C A

E I A S

51

≈

• r

≈

Red-black tree

Properties of red-black trees:

• The root is always black
• All root-to-leaf paths have the same number of black nodes.
• Red nodes do not have red children

52

Red-black tree

Connection between 2-3-4 trees and red-black trees:

A A C H I N E R S

53

Red-black tree

Connection between 2-3-4 trees and red-black trees:

A A C H I N E R S

R H N C A

E I A S

54

Red-black tree

Connection between 2-3-4 trees and red-black trees:

A A C H I N E R S

R H N C A

E I A S

E

R

H N

I

C A

A S

• r

55

Insertion in red-black trees

Insertion: Insert a new red leaf.

56

F C H F C F C F

Insert C Insert H

Red-black tree: Parent is red

What if the parent is also red? Easy case:

I D M B B D I M

C C

Red-black tree: Parent is red

What if both the parent and the grandparent are red?

I D M B B D I M

C

??

Red-black tree: Parent is red

What if both the parent and the grandparent are red?

I D M B B D I M

C C recurse

Red-black tree: Parent is red

What if the parent is also red?

I D M B B D I M

C

??

Rotations in red-black trees

Two types of rotations

61

A B C

a b c d

B A C

b c d a

A B C

a b c d

A B C

a b c d

Rotations in red-black trees

Two types of rotations:

62

A B C a b c d B A C b c d a A B C a b c d A B C a b c d B A C b c d a C B A a b c d C A B a b c d C B A a b c d

Insert x: Search to bottom after key (x) Insert red leaf Balance: 3 cases (+ symmetric)

Insertion in red-black tree

63

a a b Keep balancing with z b d z c d b d b c z a c b d z d a b c z z a b a z c c d a b c z d d

64

Example

U C

E

R H N

B I S

A C

E

R H N

B I S

A

Insert U Insert V

S C

E

R H N

B I U

A V V U C

E

R H N

B I S

A

Rotate U

65

Example

S C

E

R H N

B I U

A V

Insert G

S C

E

R H N

B I

U A V G S C

E

R H N

B

I

U

A V G

C E R H N B I A S U V G

I C

B

A R N S

U

V H G

E

Rotate I Rotate I

Running times in red-black trees

• Time for insertion:
• Search to bottom after key:
• Insert red leaf:
• Perform recoloring and rotations on way up:
• Can recolor many times (but at most h)
• At most 2 rotations.
• Total O(h).
• Time for search
• Same as BST: O(h)
• Height: O(log n)

66

O(h) O(1) O(h)

Dynamic set implementations

Worst case running times

67

Implementation search insert delete minimum maximum successor predecessor linked lists O(n) O(1) O(1) O(n) O(n) O(n) O(n)

• rdered array

O(log n) O(n) O(n) O(1) O(1) O(log n) O(log n) BST O(h) O(h) O(h) O(h) O(h) O(h) O(h) 2-3-4 tree O(log n) O(log n) O(log n) O(log n) O(log n) O(log n) O(log n) red-black tree O(log n) O(log n) O(log n) O(log n) O(log n) O(log n) O(log n)

Balanced trees: implementations

Redblack trees: Java: java.util.TreeMap, java.util.TreeSet. C++ STL: map, multimap, multiset. Linux kernel: linux/rbtree.h.

68