Welcome Inge Li Grtz. Reverse teaching and discussion of exercises: - - PowerPoint PPT Presentation

02110

Inge Li Gørtz

Thank you to Kevin Wayne for inspiration to slides

Welcome

• Inge Li Gørtz.
• Reverse teaching and discussion of exercises:
• 3 teaching assistants
• 8.00-9.15 Group work
• 9.15-9.45 Discussions of your solutions in class
• 10.00-11.15 Lecture
• 11.15-11.45 Work on exercises in the new material
• 11.45-12.00 Round up
• Weekly assignments (You have to get 40 points + pass 2 implementation exercises in order to be

able to attend the written exam).

• Prerequisites: 02105/02326 Algorithms and Data Structures I

2

Balanced Search Trees

2-3-4 trees red-black trees

References: CLRS Chapter 13, Algorithms in Java (handout)

Balanced search trees

Dynamic sets

• Search
• Insert
• Delete
• Maximum
• Minimum
• Successor(x) (find minimum element ≥ x)
• Predecessor(x) (find maximum element ≤ x)

This lecture: 2-3-4 trees, red-black trees Next time: Splay trees

4

Dynamic set implementations

Worst case running times In worst case h=n. In best case h= log n (fully balanced binary tree) Today: How to keep the trees balanced.

5

Implementation search insert delete minimum maximum successor predecessor linked lists O(n) O(1) O(1) O(n) O(n) O(n) O(n)

• rdered array

O(log n) O(n) O(n) O(1) O(1) O(log n) O(log n) BST O(h) O(h) O(h) O(h) O(h) O(h) O(h)

2-3-4 trees 2-3-4 trees

2-3-4 trees. Allow nodes to have multiple keys. Perfect balance. Every path from root to leaf has same length. Allow 1, 2, or 3 keys per node

• 2-node: one key, 2 children
• 3-node: 2 keys, 3 children
• 4-node: 3 keys, 4 children

7

S V K R M O X A C L N Q Y Z

smaller than K larger than R between K and R

B D F G J E

Search.

• Compare search key against keys in node.
• Find interval containing search key
• Follow associated link (recursively)
• Ex. Search for L

Searching in a 2-3-4 tree

8

between K and R smaller than M

S V F G J K R C E M O X A D L N Q Y Z

found L

Insertion in a 2-3-4 tree

9

S V F G J K R C E M O X A D L N Q Y Z

Insertion in a 2-3-4 tree

Insert.

• Search to bottom for key.
• Ex. Insert B

10

S V F G J K R C E M O X A D L N Q Y Z

smaller than K B not found smaller than C

Insertion in a 2-3-4 tree

Insert.

• Search to bottom for key.
• 2-node at bottom: convert to 3-node
• Ex. Insert B

11

S V F G J K R C E M O X A B D L N Q Y Z

smaller than K smaller than C B fits here

Insert.

• Search to bottom for key.
• 2-node at bottom: convert to 3-node
• Ex. Insert X

Insertion in a 2-3-4 tree

12

S T F G J K R C E M O U A D L N Q Y Z

X not found larger than R larger than U

Insert.

• Search to bottom for key.
• 2-node at bottom: convert to 3-node
• 3-node at bottom: convert to 4-node
• Ex. Insert X

Insertion in a 2-3-4 tree

13

S T F G J K R C E M O U A D L N Q X Y Z

X fits here larger than R larger than W larger than E H not found

Insertion in a 2-3-4 tree

14

S V F G J K R C E M O X A D L N Q Y Z

smaller than K

Insert.

• Search to bottom for key.
• 2-node at bottom: convert to 3-node
• 3-node at bottom: convert to 4-node
• Ex. Insert H

larger than E H does not fit here!

Insertion in a 2-3-4 tree

15

S V F G J K R C E M O X A D L N Q Y Z

smaller than K

Insert.

• Search to bottom for key.
• 2-node at bottom: convert to 3-node
• 3-node at bottom: convert to 4-node
• 4-node at bottom: ??
• Ex. Insert H

Splitting a 4-node in a 2-3-4 tree

Idea: split the 4-node to make room Problem: Doesn’t work if parent is a 4-node Solution 1: Split the parent (and continue splitting while necessary). Solution 2: Split 4-nodes on the way down.

16

H does not fit here

F G J C E D A B D

H does fit here!

A B C E G F J D A B C E G F H J

Idea: split 4-nodes on the way down the tree.

• Ensures last node is not a 4-node.
• Transformations to split 4-nodes:
• Invariant. Current node is not a 4-node.
• Consequence. Insertion at bottom is easy

since it's not a 4-node.

Splitting 4-nodes in a 2-3-4 tree

17

F G J B G B F J

x y z v x y z v

F G J

x y z v

F

x y

J

z v

B D B D G

x y z v

B D G D B G

x y z v

root

Insert.

• Search to bottom for key.
• 2-node at bottom: convert to 3-node
• 3-node at bottom: convert to 4-node
• 4-node at bottom: ??
• Ex. Insert H

Insertion in a 2-3-4 tree

18

S V F G J K R C E M O X A D L N Q Y Z

not a 4-node not a 4-node 4-node

Insert.

• Search to bottom for key.
• 2-node at bottom: convert to 3-node
• 3-node at bottom: convert to 4-node
• 4-node at bottom: ??
• Ex. Insert H

Insertion in a 2-3-4 tree

19

S V K R C E G M O X A D L N Q Y Z F J

Insert.

• Search to bottom for key.
• 2-node at bottom: convert to 3-node
• 3-node at bottom: convert to 4-node
• 4-node at bottom: ??
• Ex. Insert H

Insertion in a 2-3-4 tree

20

S V K R C E G M O X A D L N Q Y Z F H J

Local transformations that work anywhere in the tree.

• Ex. Splitting a 4-node attached to a 2-node

Splitting 4-nodes in a 2-3-4 tree


 A-C 
 E-J 
 L-P 
 R-V 
 X-Z 
 A-C 
 E-J 
 L-P 
 R-V 
 X-Z

K Q W D Q D K W

could be huge unchanged

21

Local transformations that work anywhere in the tree

• Ex. Splitting a 4-node attached to a 3-node

Splitting 4-nodes in a 2-3-4 tree


 A-C 
 I-J 
 L-P 
 R-V 
 X-Z 
 I-J 
 L-P 
 R-V 
 X-Z

K Q W K W

could be huge unchanged 
 E-G

D H


 A-C 
 E-G

D H Q

22

Splitting 4-nodes in a 2-3-4 tree

Local transformations that work anywhere in the tree. Splitting a 4-node attached to a 4-node never happens when we split nodes on the way down the tree.

• Invariant. Current node is not a 4-node.

23

Insertion 2-3-4 trees

24

S A B C E R H I N A B C E R H I N S U

Insert G Insert U Split

A B C E I R S U H N A B C E I R S U N G H A B C S U N G H E R I A B C S T U N G H E R I

Insert G Split Insert T

Insert U Insert G Insert T

Deletions in 2-3-4 trees

Delete minimum:

• minimum always in leftmost leaf
• If 3- or 4-node: delete key
• Ex. Delete minimum

25

S V F G J K R C E M O X A B D L N Q Y Z

A is minimum

Deletions in 2-3-4 trees

Delete minimum:

• minimum always in leftmost leaf
• If 3- or 4-node: delete key
• Ex. Delete minimum

26

S V F G J K R C E M O X B D L N Q Y Z

Delete A

Deletions in 2-3-4 trees

Delete minimum:

• minimum always in leftmost leaf
• If 3- or 4-node: delete key
• 2-node??
• Ex. Delete minimum

27

S V F G J K R C E M O X B D L N Q Y Z

Delete B?

Idea: On the way down maintain the invariant that current node is not a 2-node.

• Child of root and root is a 2-node:
• on the way down:

Deletions in 2-3-4 trees

28 z w y x z w y x

A B C A C B

r s w z y x r s w z y x

• r

A B C D E A B C D E

r s w z y x r s w z y x

A C D E B F G C F G A B D E

z w y x z w y x

• r

A B D E C D E A B C

Deletions in 2-3-4 trees

Delete minimum:

• minimum always in leftmost leaf
• If 3- or 4-node: delete key
• 2-node: split/merge on way down.
• Ex. Delete minimum

29

S V F G J K R C E M O X B D L N Q Y Z

not a 2-node 2-node

Deletions in 2-3-4 trees

Delete minimum:

• minimum always in leftmost leaf
• If 3- or 4-node: delete key
• 2-node: split/merge on way down.
• Ex. Delete minimum

30

S V F G J K R E M O X B C D L N Q Y Z

Deletions in 2-3-4 trees

Delete minimum:

• minimum always in leftmost leaf
• If 3- or 4-node: delete key
• 2-node: split/merge on way down.
• Ex. Delete minimum

31

S V F G J K R E M O X C D L N Q Y Z

Deletions in 2-3-4 trees

Delete:

32

S V F G J K R C E M O X B D L N Q Y Z

Deletions in 2-3-4 trees

Delete:

• During search maintain invariant that current node is not a 2-node

33

S V F G J K R C E M O X B D L N Q Y Z

Deletions in 2-3-4 trees

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key

34

S V F G J K R C E M O X B D L N Q Y Z

Deletions in 2-3-4 trees

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key
• Key not in leaf: replace with successor (always leaf in subtree) and delete

successor from leaf.

35

S V F G J K R C E M O X B D L N Q Y Z

Deletions in 2-3-4 trees

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key
• Key not in leaf: replace with successor (always leaf in subtree) and delete

successor from leaf.

• Ex. Delete K

36

S V F G J K R C E M O X B D L N Q Y Z

Deletions in 2-3-4 trees

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key
• Key not in leaf: replace with successor (always leaf in subtree) and delete

successor from leaf.

• Ex. Delete K
• Find successor

37

S V F G J K R C E M O X B D L N Q Y Z

Deletions in 2-3-4 trees

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key
• Key not in leaf: replace with successor (always leaf in subtree) and delete

successor from leaf.

• Ex. Delete K
• Find successor

38

S V F G J K R C E M O X B D L N Q Y Z

Deletions in 2-3-4 trees

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key
• Key not in leaf: replace with successor (always leaf in subtree) and delete

successor from leaf.

• Ex. Delete K
• Find successor

39

S V F G J K R C E M O X B D L N Q Y Z

not a 2-node

Deletions in 2-3-4 trees

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key
• Key not in leaf: replace with successor (always leaf in subtree) and delete

successor from leaf.

• Ex. Delete K
• Find successor

40

S V F G J K R C E M O X B D L N Q Y Z

not a 2-node 2-node

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key
• Key not in leaf: replace with successor (always leaf in subtree) and delete

successor from leaf.

• Ex. Delete K
• Find successor

Deletions in 2-3-4 trees

41

S V F G J K R C E O X B D Q Y Z L M N

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key
• Key not in leaf: replace with successor (always leaf in subtree) and delete

successor from leaf.

• Ex. Delete K
• Find successor
• Delete L from leaf

Deletions in 2-3-4 trees

42

S V F G J K R C E O X B D Q Y Z L M N

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key
• Key not in leaf: replace with successor (always leaf in subtree) and delete

successor from leaf.

• Ex. Delete K
• Find successor
• Delete L from leaf

Deletions in 2-3-4 trees

43

S V F G J K R C E O X B D Q Y Z M N

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key
• Key not in leaf: replace with successor (always leaf in subtree) and delete

successor from leaf.

• Ex. Delete K
• Find successor
• Delete L from leaf
• Replace K with L

Deletions in 2-3-4 trees

44

S V F G J K R C E O X B D Q Y Z M N

Deletions in 2-3-4 trees

45

S V F G J L R C E O X B D Q Y Z M N

Delete:

• During search maintain invariant that current node is not a 2-node
• If key is in a leaf: delete key
• Key not in leaf: replace with successor (always leaf in subtree) and delete

successor from leaf.

• Ex. Delete K
• Find successor
• Delete L from leaf
• Replace K with L

2-3-4 Tree: Balance

• Property. All paths from root to leaf have same length.

Tree height. Worst case: lg N [all 2-nodes] Best case: log4 N = 1/2 lg N [all 4-nodes] Between 10 and 20 for a million nodes. Between 15 and 30 for a billion nodes.

46

Dynamic set implementations

Worst case running times

47

Implementation search insert delete minimum maximum successor predecessor linked lists O(n) O(1) O(1) O(n) O(n) O(n) O(n)

• rdered array

O(log n) O(n) O(n) O(1) O(1) O(log n) O(log n) BST O(h) O(h) O(h) O(h) O(h) O(h) O(h) 2-3-4 tree O(log n) O(log n) O(log n) O(log n) O(log n) O(log n) O(log n)

Red-black trees

Represent 2-3-4 tree as a binary search tree

• Use colors on nodes to represent 3- and 4-nodes.

Red-black tree (Guibas-Sedgewick, 1979)

G K O K G O H L

• r

H L H L

B B

49

≈ ≈ ≈

Represent 2-3-4 tree as a binary search tree

• Use colors on nodes to represent 3- and 4-nodes.
• Connection between 2-3-4 trees and red-black trees:

Red-black tree (Guibas-Sedgewick, 1979)

A A C H I N E R S

R H N C A

E I A S

50

≈

• r

≈

Represent 2-3-4 tree as a binary search tree

• Use colors on nodes to represent 3- and 4-nodes.
• Connection between 2-3-4 trees and red-black trees:

Red-black tree (Guibas-Sedgewick, 1979)

A A C H I N E R S

R H N C A

E I A S

51

≈

• r

≈

Red-black tree

Properties of red-black trees:

• The root is always black
• All root-to-leaf paths have the same number of black nodes.
• Red nodes do not have red children
• CLRS: All leaves (NIL) are black

52

Red-black tree

Connection between 2-3-4 trees and red-black trees:

A A C H I N E R S

53

Red-black tree

Connection between 2-3-4 trees and red-black trees:

A A C H I N E R S

R H N C A

E I A S

54

Red-black tree

Connection between 2-3-4 trees and red-black trees:

A A C H I N E R S

R H N C A

E I A S

E

R

H N

I

C A

A S

• r

55

Insertion in red-black trees

Insertion in 2-node: Insertion in 3-node:

56

F C H C F C F H F C F C F C F F

Insert C Insert H

≈ ≈ ≈ ≈

Red-black tree: Splitting 4-nodes

Two easy cases:

57

E I B D M B B M D D I M B I D I I D B B M B D E I M D E M M E I B D I M ≈ ≈ ≈ ≈

Insertion in red-black trees

Insertion in 3-node (continued):

58

Insert H

F C H C F C F H F C

≈ ≈

F H C C F

??

C F

Insert H

≈ ≈

C F C F E C F

??

Insert E

≈ ≈

Red-black trees: Splitting of 4-nodes

Example of hard case:

??

B L D H K

K H

B

B D L D H K

Solution: Rotations! ≈ ≈

Rotations in red-black trees

Two types of rotations

60

A B C

a b c d

B A C

b c d a

A B C

a b c d

A B C

a b c d

Rotations in red-black trees

Two types of rotations:

61

A B C a b c d B A C b c d a A B C a b c d A B C a b c d B A C b c d a C B A a b c d C A B a b c d C B A a b c d

Insertion in red-black trees

Insertion in 3-node:

62

F H C C F C F C F E C F C F

Insert H Insert E

F C H

≈ ≈ ≈

??

≈

??

Insertion in red-black trees

Insertion in 3-node:

63

F H C C F C F C F E C F C F

Insert H Insert E

C F H F C H

≈ ≈ ≈ ≈

??

Insertion in red-black trees

Insertion in 3-node:

64

F H C C F C F C F E C F C F

Insert H Insert E

C F H E F C E C F F C H

≈ ≈

C E F

≈ ≈

Insert x: Search to bottom after key (x) Insert red leaf Balance: 3 cases (+ symmetric)

Insertion in red-black tree

65

a a b Keep balancing with z b d z c d b d b c z a c b d z d a b c z z a b a z c c d a b c z d d

66

Eksempel

U C

E

R H N

B I S

A C

E

R H N

B I S

A

Insert U Insert V

S C

E

R H N

B I U

A V V U C

E

R H N

B I S

A

Rotate U

67

Example

S C E R H N B I U A V Insert G S C E R H N B I U A V G S C E R H N B I U A V G C E R H N B I A S U V G

I C B A R N S U V H G E

Rotate I Rotate I

Running times in red-black trees

• Time for insertion:
• Search to bottom after key:
• Insert red leaf:
• Perform recoloring and rotations on way up:
• Can recolor many times (but at most h)
• At most 2 rotations.
• Total O(h).
• Time for search
• Same as BST: O(h)
• Height: O(log n)

68

O(h) O(1) O(h)

Dynamic set implementations

Worst case running times

69

Implementation search insert delete minimum maximum successor predecessor linked lists O(n) O(1) O(1) O(n) O(n) O(n) O(n)

• rdered array

O(log n) O(n) O(n) O(1) O(1) O(log n) O(log n) BST O(h) O(h) O(h) O(h) O(h) O(h) O(h) 2-3-4 tree O(log n) O(log n) O(log n) O(log n) O(log n) O(log n) O(log n) red-black tree O(log n) O(log n) O(log n) O(log n) O(log n) O(log n) O(log n)

Balanced trees: implementations

Redblack trees: Java: java.util.TreeMap, java.util.TreeSet. C++ STL: map, multimap, multiset. Linux kernel: linux/rbtree.h.

70

Exercise time

Start working on the exercises on weekplan BST. 11.45 talk about solutions to the first 2-3 exercises + info about the mandatory exercises.

71

Mandatory exercise

• To be handed in before 8PM the following Sunday on DTU Learn.
• This week: Binary tries from weekplan warmup.

72

Grading of mandatory assignments

You can get up to 20 points for an exercise:

• 5 points for the example (draw tree/run algorithm etc)
• 15 points for design an algorithm exercise:
• 4 points for the description of your algorithm
• 4 points for the quality of your algorithm
• 4 points for the complexity analysis
• 3 points for the correctness arguments

73