Lecture 6: Binary Search Trees (BST) and Red-Black Trees (RBTs) - - PowerPoint PPT Presentation

▶
lecture 6 binary search trees bst and red black trees rbts
SMART_READER_LITE
LIVE PREVIEW

Lecture 6: Binary Search Trees (BST) and Red-Black Trees (RBTs) - - PowerPoint PPT Presentation

Dec 26, 2022 256 likes •1.15k views

Lecture 6: Binary Search Trees (BST) and Red-Black Trees (RBTs) Instructor: Saravanan Thirumuruganathan CSE 5311 Saravanan Thirumuruganathan Outline 1 Data Structures for representing Dynamic Sets Binary Search Trees (BSTs) Balanced Search

slide-1
SLIDE 1

Lecture 6: Binary Search Trees (BST) and Red-Black Trees (RBTs)

Instructor: Saravanan Thirumuruganathan

CSE 5311 Saravanan Thirumuruganathan

slide-2
SLIDE 2

Outline

1 Data Structures for representing Dynamic Sets

Binary Search Trees (BSTs) Balanced Search Trees Balanced Binary Trees - Red Black Trees (RBTs)

CSE 5311 Saravanan Thirumuruganathan

slide-3
SLIDE 3

In-Class Quizzes URL: http://m.socrative.com/ Room Name: 4f2bb99e

CSE 5311 Saravanan Thirumuruganathan

slide-4
SLIDE 4

Data Structures Key Things to Know for Data Structures

Motivation Distinctive Property Major operations Key Helper Routines Representation Algorithms for major operations Applications

CSE 5311 Saravanan Thirumuruganathan

slide-5
SLIDE 5

Trees Non-Linear Data Structures:

Very common and useful category of data structures Most popular one is hierarchical

CSE 5311 Saravanan Thirumuruganathan

slide-6
SLIDE 6

Trees - Applications1 Family Tree:

1http://interactivepython.org/runestone/static/pythonds/Trees/

trees.html

CSE 5311 Saravanan Thirumuruganathan

slide-7
SLIDE 7

Trees - Applications2 Taxonomy Tree:

2http://interactivepython.org/runestone/static/pythonds/Trees/

trees.html

CSE 5311 Saravanan Thirumuruganathan

slide-8
SLIDE 8

Trees - Applications3 Directory Tree:

3http://interactivepython.org/runestone/static/pythonds/Trees/

trees.html

CSE 5311 Saravanan Thirumuruganathan

slide-9
SLIDE 9

Trees - Applications4 HTML DOM (Parse) Tree:

4http://interactivepython.org/runestone/static/pythonds/Trees/

trees.html

CSE 5311 Saravanan Thirumuruganathan

slide-10
SLIDE 10

Tree - Terminology

Node Edge Root Children Parent Sibling Subtree Leaf/External node Internal node Level (node) Height (tree) Arity

CSE 5311 Saravanan Thirumuruganathan

slide-11
SLIDE 11

Tree - Abstract Representation5

5http://interactivepython.org/runestone/static/pythonds/Trees/

trees.html

CSE 5311 Saravanan Thirumuruganathan

slide-12
SLIDE 12

Motivation

Store dynamic set efficiently Use good ideas from ordered list (OL) and ordered doubly linked list (ODLL) Use hierarchical storage to avoid pitfalls of OL and ODLL First attempt at hierarchical data structure that tries to implement all 7 operations efficiently

CSE 5311 Saravanan Thirumuruganathan

slide-13
SLIDE 13

Binary Trees

Each node has at most 2 children Commonly referred to as left and right child The descendants of left child constitute left subtree The descendants of right child constitute right subtree

CSE 5311 Saravanan Thirumuruganathan

slide-14
SLIDE 14

BST Property

For every node x, the keys of left subtree ≤ key(x) For every node x, the keys of right subtree ≥ key(x)

6

6http:

//www.cs.princeton.edu/~rs/AlgsDS07/08BinarySearchTrees.pdf

CSE 5311 Saravanan Thirumuruganathan

slide-15
SLIDE 15

BST Examples7

7http://en.wikipedia.org/wiki/Binary_search_tree CSE 5311 Saravanan Thirumuruganathan

slide-16
SLIDE 16

BST Height8

There exists multiple possible BSTs to store same set of elements Minimum and Maximum Height:

8https://engineering.purdue.edu/~ee608/handouts/lec10.pdf CSE 5311 Saravanan Thirumuruganathan

slide-17
SLIDE 17

BST Height8

There exists multiple possible BSTs to store same set of elements Minimum and Maximum Height: lg n and n Best and worst case analysis (or specify analysis wrt height)

8https://engineering.purdue.edu/~ee608/handouts/lec10.pdf CSE 5311 Saravanan Thirumuruganathan

slide-18
SLIDE 18

Representation - I

key: Stores key information that is used to compare two nodes value: Stores satellite/auxillary information parent: Pointer to parent node. parent(root) = NULL left: Pointer to left child if it exists. Else NULL right: Pointer to right child if it exists. Else NULL

CSE 5311 Saravanan Thirumuruganathan

slide-19
SLIDE 19

Representation - I9

9CLRSFig10.9 CSE 5311 Saravanan Thirumuruganathan

slide-20
SLIDE 20

Representation - II

CSE 5311 Saravanan Thirumuruganathan

slide-21
SLIDE 21

Major Operations

Search Insert Minimum/Maximum Successor/Predecessor Deletion Traversals

CSE 5311 Saravanan Thirumuruganathan

slide-22
SLIDE 22

Key Helper Routines

Successor/Predecessor Traversals Case by Case Analysis:

Analysis by number of children : 0, 1, 2 Analysis by type of children: left, right

CSE 5311 Saravanan Thirumuruganathan

slide-23
SLIDE 23

BST: Search Search: 4

CSE 5311 Saravanan Thirumuruganathan

slide-24
SLIDE 24

BST: Search

CSE 5311 Saravanan Thirumuruganathan

slide-25
SLIDE 25

BST: Search

Tree-Search(x, k): if x == NULL or k == x.key return x if k < x.key return Tree-Search(x.left, k) else return Tree-Search(x.right, k) Analysis:

CSE 5311 Saravanan Thirumuruganathan

slide-26
SLIDE 26

BST: Search

Tree-Search(x, k): if x == NULL or k == x.key return x if k < x.key return Tree-Search(x.left, k) else return Tree-Search(x.right, k) Analysis: O(h) Best Case: lg n and Worst Case: O(n)

CSE 5311 Saravanan Thirumuruganathan

slide-27
SLIDE 27

BST: Insert Insert: 13

CSE 5311 Saravanan Thirumuruganathan

slide-28
SLIDE 28

BST: Insert Insert: 13

CSE 5311 Saravanan Thirumuruganathan

slide-29
SLIDE 29

BST:

Analysis:

CSE 5311 Saravanan Thirumuruganathan

slide-30
SLIDE 30

BST:

Analysis: O(h) Best Case: lg n and Worst Case: O(n)

CSE 5311 Saravanan Thirumuruganathan

slide-31
SLIDE 31

BST: Minimum

CSE 5311 Saravanan Thirumuruganathan

slide-32
SLIDE 32

BST: Minimum

CSE 5311 Saravanan Thirumuruganathan

slide-33
SLIDE 33

BST: Minimum

Tree-Minimum(x) while x.left is not NULL x = x.left return x Analysis: O(h) Best Case: lg n and Worst Case: O(n)

CSE 5311 Saravanan Thirumuruganathan

slide-34
SLIDE 34

BST: Maximum

CSE 5311 Saravanan Thirumuruganathan

slide-35
SLIDE 35

BST: Maximum

CSE 5311 Saravanan Thirumuruganathan

slide-36
SLIDE 36

BST: Maximum

Tree-Maximum(x) while x.right is not NULL x = x.right return x Analysis: O(h) Best Case: lg n and Worst Case: O(n)

CSE 5311 Saravanan Thirumuruganathan

slide-37
SLIDE 37

BST: Successor Successor: 15

CSE 5311 Saravanan Thirumuruganathan

slide-38
SLIDE 38

BST: Successor Successor: 15

CSE 5311 Saravanan Thirumuruganathan

slide-39
SLIDE 39

BST: Successor Successor: 13

CSE 5311 Saravanan Thirumuruganathan

slide-40
SLIDE 40

BST: Successor Successor: 13

CSE 5311 Saravanan Thirumuruganathan

slide-41
SLIDE 41

BST: Successor

CSE 5311 Saravanan Thirumuruganathan

slide-42
SLIDE 42

BST:

Tree-Successor(x): if x.right is not NULL return Tree-Minimum(x.right) y = x.parent while y is not NULL and x == y.right x = y y = y.parent return y BST Property allowed us to find successor without comparing keys Analysis: O(h) Best Case: lg n and Worst Case: O(n)

CSE 5311 Saravanan Thirumuruganathan

slide-43
SLIDE 43

BST: Predecessor Predecessor: 6

CSE 5311 Saravanan Thirumuruganathan

slide-44
SLIDE 44

BST: Predecessor Predecessor: 6

CSE 5311 Saravanan Thirumuruganathan

slide-45
SLIDE 45

BST: Predecessor Predecessor: 17

CSE 5311 Saravanan Thirumuruganathan

slide-46
SLIDE 46

BST: Predecessor Predecessor: 17

CSE 5311 Saravanan Thirumuruganathan

slide-47
SLIDE 47

BST:

Tree-Predecessor(x): if x.left is not NULL return Tree-Maximum(x.left) y = x.parent while y is not NULL and x == y.left x = y y = y.parent return y BST Property allowed us to find predecessor without comparing keys Analysis: O(h) Best Case: lg n and Worst Case: O(n)

CSE 5311 Saravanan Thirumuruganathan

slide-48
SLIDE 48

BST: Deletion

Trickiest Operation! Suppose we want to delete node z

1 z has no children: Replace z with NULL 2 z has one children c: Promote c to z’s place 3 z has two children:

(a) Let z’s successor be y (b) y is either a leaf or has only right child (c) Promote y to z’s place (d) Fix y’s loss via Cases 1 or 2

CSE 5311 Saravanan Thirumuruganathan

slide-49
SLIDE 49

BST: Deletion Case I10

10https:

//www.cs.rochester.edu/u/gildea/csc282/slides/C12-bst.pdf

CSE 5311 Saravanan Thirumuruganathan

slide-50
SLIDE 50

BST: Deletion Case II11

11https:

//www.cs.rochester.edu/u/gildea/csc282/slides/C12-bst.pdf

CSE 5311 Saravanan Thirumuruganathan

slide-51
SLIDE 51

BST: Deletion Case III12

12https:

//www.cs.rochester.edu/u/gildea/csc282/slides/C12-bst.pdf

CSE 5311 Saravanan Thirumuruganathan

slide-52
SLIDE 52

Digression

Perfectly fine if you cannot do deletion by memory Things will become hairier in RBT As long as you remember the key ideas and operations, you will be fine

CSE 5311 Saravanan Thirumuruganathan

slide-53
SLIDE 53

BST: Traversal

Traversal: Visit all nodes in a tree Many possible traversal strategies Three are most popular: Pre-Order, In-Order, Post-Order

CSE 5311 Saravanan Thirumuruganathan

slide-54
SLIDE 54

BST: Traversals

In-Order-Walk(x): if x == NULL return In-Order-Walk(x.left) Print x.key In-Order-Walk(x.right) Analysis:

CSE 5311 Saravanan Thirumuruganathan

slide-55
SLIDE 55

BST: Traversals

In-Order-Walk(x): if x == NULL return In-Order-Walk(x.left) Print x.key In-Order-Walk(x.right) Analysis: O(n) Holds true for all three traversals

CSE 5311 Saravanan Thirumuruganathan

slide-56
SLIDE 56

BST: Traversal

In-Order:

CSE 5311 Saravanan Thirumuruganathan

slide-57
SLIDE 57

BST: Traversal

In-Order: 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 15, 19, 20. Pre-Order:

CSE 5311 Saravanan Thirumuruganathan

slide-58
SLIDE 58

BST: Traversal

In-Order: 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 15, 19, 20. Pre-Order: 7, 4, 2, 3, 6, 5, 12, 9, 8, 11, 19, 15, 20. Pre-Order:

CSE 5311 Saravanan Thirumuruganathan

slide-59
SLIDE 59

BST: Traversal

In-Order: 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 15, 19, 20. Pre-Order: 7, 4, 2, 3, 6, 5, 12, 9, 8, 11, 19, 15, 20. Pre-Order: 3, 2, 5, 6, 4, 8, 11, 9, 15, 20, 19, 12, 7.

CSE 5311 Saravanan Thirumuruganathan

slide-60
SLIDE 60

BST: Traversals

Notice anything special about In-Order traversal?

CSE 5311 Saravanan Thirumuruganathan

slide-61
SLIDE 61

BST: Traversals

Notice anything special about In-Order traversal?

Returns items in sorted order!

Successor/Predecessor can be expressed in terms on In-Order traversal

CSE 5311 Saravanan Thirumuruganathan

slide-62
SLIDE 62

Application: Tree-Sort Tree-Sort:

Construct a BST out of elements Do In-Order traversal

Analysis:

Time Complexity: Time Complexity for Constructing Tree + Time Complexity for In-Order traversal Best Case:

CSE 5311 Saravanan Thirumuruganathan

slide-63
SLIDE 63

Application: Tree-Sort Tree-Sort:

Construct a BST out of elements Do In-Order traversal

Analysis:

Time Complexity: Time Complexity for Constructing Tree + Time Complexity for In-Order traversal Best Case: n × lg n + n = O(n lg n) Worst Case:

CSE 5311 Saravanan Thirumuruganathan

slide-64
SLIDE 64

Application: Tree-Sort Tree-Sort:

Construct a BST out of elements Do In-Order traversal

Analysis:

Time Complexity: Time Complexity for Constructing Tree + Time Complexity for In-Order traversal Best Case: n × lg n + n = O(n lg n) Worst Case: n × n + n = O(n2)

CSE 5311 Saravanan Thirumuruganathan

slide-65
SLIDE 65

Balanced Binary Trees

Time complexity of BST operations depends on height Can vary between O(lg n) to O(n) BST operations do not take any special care to keep tree balanced If we can do the balancing efficiently, then all operations become faster Self-balancing - Do not run balancing algorithms periodically

CSE 5311 Saravanan Thirumuruganathan

slide-66
SLIDE 66

Notion of Balance

Maintaining perfectly balanced trees is very hard and expensive So we resort to BSTs that are approximately balanced Need to define notion of balance Ideas?

CSE 5311 Saravanan Thirumuruganathan

slide-67
SLIDE 67

Notion of Balance

Maintaining perfectly balanced trees is very hard and expensive So we resort to BSTs that are approximately balanced Need to define notion of balance Ideas?

Informally, ensure the longest path in tree is not “too” long Many ways of formally specifying it Eg: |height(left subtree) − height(right subtree)| ≤ 1

When can balance be broken?

CSE 5311 Saravanan Thirumuruganathan

slide-68
SLIDE 68

Notion of Balance

Maintaining perfectly balanced trees is very hard and expensive So we resort to BSTs that are approximately balanced Need to define notion of balance Ideas?

Informally, ensure the longest path in tree is not “too” long Many ways of formally specifying it Eg: |height(left subtree) − height(right subtree)| ≤ 1

When can balance be broken? Insertion, Deletion

CSE 5311 Saravanan Thirumuruganathan

slide-69
SLIDE 69

Balanced Search Trees

Red-Black trees AVL trees 2 − 3 and 2 − 3 − 4 trees B-trees and other variants Treaps Skip trees Splay trees and many many more

CSE 5311 Saravanan Thirumuruganathan

slide-70
SLIDE 70

Red-Black Trees

Red-Black Trees

CSE 5311 Saravanan Thirumuruganathan

slide-71
SLIDE 71

RBT: Motivation

Most important self-balancing BST Invented by Guibas-Sedgewick Simplifies/Unifies various balanced tree algorithms Became popular due to its simplicity in implementation Stores additional information about color of node (1 bit) All operations are logarithmic!

CSE 5311 Saravanan Thirumuruganathan

slide-72
SLIDE 72

RBT Property

Every node is either red or black Root and leaves are black If node is red, then its parent is black All simple paths from any node v to a descendent leaf have same number of black nodes. Aka black-height(v)

CSE 5311 Saravanan Thirumuruganathan

slide-73
SLIDE 73

RBT Example13

13MIT OCW 6-046j CSE 5311 Saravanan Thirumuruganathan

slide-74
SLIDE 74

RBT Example14

14MIT OCW 6-046j CSE 5311 Saravanan Thirumuruganathan

slide-75
SLIDE 75

RBT Property: Implications

If a red node has any children, then it must have two children and both must be black If a black node has only one child, it has to be red No root to leaf path has two consecutive red nodes No root to leaf path is more than twice as long as any other The rules bound the imbalance in the tree

CSE 5311 Saravanan Thirumuruganathan

slide-76
SLIDE 76

Major Operations

Search Insert Minimum/Maximum Successor/Predecessor Deletion

CSE 5311 Saravanan Thirumuruganathan

slide-77
SLIDE 77

Key Helper Routines

Rotations - Right and Left Case by Case Analysis:

Analysis by type of children, siblings and uncle (sibling of parent) Analysis by color of children

CSE 5311 Saravanan Thirumuruganathan

slide-78
SLIDE 78

RBT Theorem

Theorem (RBT Theorem) A red-black tree with n keys has height h ≤ 2 lg(n + 1) Corollary Operations Search, Min, Max, Successor, Predecessor all run in O(lg n) time on a Red-Black tree with n nodes.

CSE 5311 Saravanan Thirumuruganathan

slide-79
SLIDE 79

RBT - Modifying Operations

Insert and Delete need to be more complex so as to balance the tree They cause “changes” to tree

Change of color (recoloring) Change of tree structure via “rotations”

CSE 5311 Saravanan Thirumuruganathan

slide-80
SLIDE 80

Rotations15

Rotation can be done in O(1) time. Rotations maintain in-order ordering of keys: a ∈ α, b ∈ β, c ∈ γ ⇒ a ≤ A ≤ b ≤ B ≤ c

15MIT OCW 6-046j CSE 5311 Saravanan Thirumuruganathan

slide-81
SLIDE 81

RBT Insertion

Insert x in tree (based on BST property) Color x red Only red-black property 3 can be violated Fix violations by rotations and recoloring

CSE 5311 Saravanan Thirumuruganathan

slide-82
SLIDE 82

RBT : Insertion16

Case I: The parent and “uncle” of x are both red

Color parent and uncle of x as black Color grandparent of x as red Recurse of grandparent of x

16MIT OCW 6.046j CSE 5311 Saravanan Thirumuruganathan

slide-83
SLIDE 83

RBT : Insertion17

Case II: The parent of x is red, the uncle of x is black, x’s parent is a left child , x is a right child

Left rotate on x’s parent Make x’s left child the new x Solve this scenario by using by Case III

CSE 5311 Saravanan Thirumuruganathan

slide-84
SLIDE 84

RBT : Insertion18

Case III: The parent of x is Red and the uncle is black, x is a left child, and its parent is a left child

Right rotate on grandparent of x Switch colors of x’s parent and x’s sibling Done!

18MIT OCW 6.046j CSE 5311 Saravanan Thirumuruganathan

slide-85
SLIDE 85

RBT : Insertion Sorted Elements

Refer https://www.cs.utexas.edu/~scottm/cs314/ handouts/slides/Topic23RedBlackTrees.pdf for a step-by-step example of how RBT handles insertion in sorted

  • rder

CSE 5311 Saravanan Thirumuruganathan

slide-86
SLIDE 86

RBT: Deletion

Similarly complicated Refer to CLRS book for full details

CSE 5311 Saravanan Thirumuruganathan

slide-87
SLIDE 87

Summary Major Concepts:

Binary Search Trees Concept of Self-Balancing Red-Black Trees

CSE 5311 Saravanan Thirumuruganathan