The Making Change Problem (MCP) A greedy algorithm Greedy - - PowerPoint PPT Presentation

SLIDE 1

Greedy Algorithms

DPV Chapter 5, Part 1

Jim Royer

EECS

February 28, 2019

(Unless otherwise credited, all images are from DPV.)

Royer Greedy Algorithms

The Making Change Problem (MCP)

The MCP

Given: an amount a Find: the smallest collection of coins that is worth amount a. Example: For a = 0.28 the (unique) answer is 0.25, 0.01, 0.01, 0.01.

A “greedy” algorithm

// Input: a, an amount // Output: a list of coins giving // a solution to the MCP for a C ← { 1.00, 0.50, 0.25, 0.10, 0.05, 0.01 } ans ← the empty list while a > 0 do if (a < min(C)) then return the empty list // Indicates no Solution else c ← max({ c ∈ C : c ≤ a }) add c to the ans list a ← a − c return ans

Fact: US coins have denominations: 1.00, 0.50, 0.25, 0.10, 0.05, and 0.01.

Royer Greedy Algorithms

Making Change

Diagram from http://en.wikipedia.org/wiki/Greedy_algorithm Royer Greedy Algorithms

Making Change

Fiction: Martian coins have denoms: 1.00, 0.50, 0.25, 0.14, and 0.01. Example: For a = 0.28 the (unique) answer is 0.14, and 0.14. A greedy “algorithm”

// Input: a, an amount // Output: a list of coins that may or may not be a solution to the MCP for a C ← { 1.00, 0.50, 0.25, 0.14, 0.01 } ans ← the empty list while a > 0 do if (a < min(C)) then return the empty list else c ← max({ c ∈ C : c ≤ a }) add c to the ans list a ← a − c return ans

Problem: The above returns the wrong answer for a = 28. Problem: How do we know the previous version is correct?

Royer Greedy Algorithms

SLIDE 2

Standard Features of Greedy Algorithms

Greedy algorithms generally (but not always) have the following features: ◮ The goal is to build an optimal solution to a problem by selecting items from a candidate set. ◮ A feasibility function is used to check whether a candidate can be used in the solution ◮ A greedy choice function picks a “best” candidate to be added to the solution. ◮ An objective function assigns a value to partial solutions. ◮ A solution function tests whether we have a complete solution yet.

C ← { 1.00, 0.50, 0.25, 0.14, 0.01 }; ans ← [ ] while a > 0 do if (a < min(C)) then return [ ] else c ← max({ c ∈ C : c ≤ a }) ans ← cons(c, ans); a ← a − c return ans

◮ candidate set = C ◮ feasibility ≡ [coin value

?

≤ a] ◮ greedy choice ≡ c ← max({ c ∈ C : c ≤ a }) ◮ objective function ≡ the number of coins used (minimize) ◮ solution function ≡ [a ? = 0]

Royer Greedy Algorithms

Event Scheduling, 1

The Meeting Scheduling Problem (MSP)

Given: (s1, f1), . . . , (sk, fk) where, for each i, (si, fi) represents a meeting starting at time si and finishing at time fi with 0 ≤ si < fi for each i. Goal: Schedule as many of non-overlapping meetings as possible.

Definition

Two meetings overlap when their intervals intersect, i.e., (si, fi) ∩ (sj, fj) = ∅.

A maximal conflict-free schedule for a set of classes.

Image from: http://www.cs.uiuc.edu/class/fa05/cs473g/lectures/06-greedy.pdf Royer Greedy Algorithms

Event Scheduling, 2

The Meeting Scheduling Problem (MSP)

Given: (s1, f1), . . . , (sk, fk) where, for each i, (si, fi) represents a meeting starting at time si and finishing at time fi with 0 ≤ si < fi for each i. Goal: Schedule as many of non-overlapping meetings as possible.

Definition

Two meetings overlap when their intervals intersect, i.e., (si, fi) ∩ (sj, fj) = ∅. ◮ candidate set = { (s1, f1), . . . , (sk, fk) }. ◮ S is a partial solution ≡ S is a set of non-overlapping meetings ◮ (si, fi) is feasible for S ⇐ ⇒ (si, fi) does not overlap with anything in S. ◮ solution ≡ no more meetings can be added to S ◮ objective function ≡ the size of S (to be maximized) ◮ greedy choice ≡ shortest meeting?, earliest starting time?, latest finishing time?, fewest overlaps?, earliest finishing time!

Royer Greedy Algorithms

Event Scheduling, 3

function greedySch(L: a list of events) Sort L so that f1 ≤ · · · ≤ fk S ← ∅; last ← 0 for i ← 1 to k do if last ≤ si then S ← S ∪ { (si, fi) }; last ← fi return S

The same classes sorted by finish times and the greedy schedule. Image from: http://www.cs.uiuc.edu/class/fa05/cs473g/lectures/06-greedy.pdf Royer Greedy Algorithms

SLIDE 3

Event Scheduling, 4

function greedySch(L: a list of events) Sort L so that f1 ≤ · · · ≤ fk S ← ∅; last ← 0 for i ← 1 to k do if last ≤ si then S ← S ∪ { (si, fi) }; last ← fi return S // Runtime: ??

Lemma

greedySch returns a schedule with as many events as possible.

• Proof. Suppose

◮ S′ = { (s′

1, f ′ 1), . . . , (s′ m, f ′ m) } is the

schedule greedySch returns. ◮ S′′ = { (s′′

1 , f ′′ 1 ), . . . , (s′′ n, f ′′ n ) } is

an optimal schedule. (So m ≤ n.) Goal: Show m = n. Strategy: Turn S′′ into S′. Suppose f ′

1 ≤ · · · ≤ f ′ m and f ′′ 1 ≤ · · · ≤ f ′′ n .

For ℓ = 1, . . . , m, let S′′

ℓ =

{ (s′

1, f ′ 1), . . . , (s′ ℓ, f ′ ℓ), (s′′ ℓ+1, f ′′ ℓ+1), . . . , (s′′ m, f ′′ m) }.

We want to show each S′′

ℓ is feasible.

Base Case Claim 1: f ′

1 ≤ f ′′ 1 .

Proof: By greedySch’s choice, f ′

1 = the

smallest finishing time. Hence, f ′

1 ≤ f ′′ 1 .

Claim 2: S′′

1 is feasible.

Proof: Since f ′

1 ≤ f ′′ 1 (Claim 1) and f ′′ 1 ≤ s′′ 2

(S′′ has no overlaps), f ′

1 ≤ s′′ 2 . Hence, S′′ 1 has

no overlaps.

Royer Greedy Algorithms

Event Scheduling, 5

function greedySch(L: a list of events) Sort L so that f1 ≤ · · · ≤ fk S ← ∅; last ← 0 for i ← 1 to k do if last ≤ si then S ← S ∪ { (si, fi) }; last ← fi return S // Runtime: ??

Lemma

greedySch returns a schedule with as many events as possible.

• Proof. Suppose

◮ S′ = { (s′

1, f ′ 1), . . . , (s′ m, f ′ m) } is the

schedule greedySch returns. ◮ S′′ = { (s′′

1 , f ′′ 1 ), . . . , (s′′ n, f ′′ n ) } is

an optimal schedule. (So m ≤ n.) Goal: Show m = n. Strategy: Turn S′′ into S′. Suppose f ′

1 ≤ · · · ≤ f ′ m and f ′′ 1 ≤ · · · ≤ f ′′ n .

For ℓ = 1, . . . , m, let S′′

ℓ =

{ (s′

1, f ′ 1), . . . , (s′ ℓ, f ′ ℓ), (s′′ ℓ+1, f ′′ ℓ+1), . . . , (s′′ m, f ′′ m) }.

We want to show each S′′

ℓ is feasible.

Induction step Case IH: S′′

ℓ−1 is feasible, (1<ℓ≤ m).

Convention: s′′

n+1 = ∞.

Claim 3: f ′

ℓ ≤ f ′′ ℓ .

Proof: Since S′′

ℓ−1 is consistent, s′′ ℓ ≥ f ′ ℓ−1. By

greeySch’s greedy choice, f ′

ℓ is the smallest

finishing time with a corresponding starting time ≥ f ′

ℓ−1. Hence, f ′ ℓ ≤f ′′ ℓ .

Claim 4: S′′

ℓ is feasible.

Proof: Since f ′

ℓ ≤ f ′′ ℓ (Claim 3) & f ′′ ℓ ≤ s′′ ℓ+1

(S′′

ℓ has no overlaps), f ′ ℓ ≤ s′′ ℓ+1.

Hence, S′′

ℓ+1 has no overlaps.

Royer Greedy Algorithms

Event Scheduling, 6

function greedySch(L: a list of events) Sort L so that f1 ≤ · · · ≤ fk S ← ∅; last ← 0 for i ← 1 to k do if last ≤ si then S ← S ∪ { (si, fi) }; last ← fi return S // Runtime: ??

Lemma

greedySch returns a schedule with as many events as possible.

• Proof. Suppose

◮ S′ = { (s′

1, f ′ 1), . . . , (s′ m, f ′ m) } is the

schedule greedySch returns. ◮ S′′ = { (s′′

1 , f ′′ 1 ), . . . , (s′′ n, f ′′ n ) } is

an optimal schedule. (So m ≤ n.) Goal: Show m = n. Strategy: Turn S′′ into S′. Suppose f ′

1 ≤ · · · ≤ f ′ m and f ′′ 1 ≤ · · · ≤ f ′′ n .

For ℓ = 1, . . . , m, let S′′

ℓ =

{ (s′

1, f ′ 1), . . . , (s′ ℓ, f ′ ℓ), (s′′ ℓ+1, f ′′ ℓ+1), . . . , (s′′ m, f ′′ m) }.

We proved: S′′

m is feasible.

Final step Suppose by way of contradiction that m < n. Since S′′

m is feasible, (s′′ m+1, f ′′ m+1) does not

• verlap with any of (s′

1, f ′ 1), . . . , (s′ m, f ′ m).

But then by greedySch’s greedy choice, the algorithm would have chosen at least one more event to add to S, a contradiction. Therefore, m = n and S′ (= S′′

m) is optimal.

✷

Royer Greedy Algorithms

Knapsack, 1

The Knapsack Problem (KP)

Given: ◮ A knapsack with capacity W kgs. ◮ Items 1, . . . , n ◮ Item i has weight wi and value vi. Find: S ⊆ { 1, . . . , n } so that ◮ ∑i∈S wi ≤ W and ◮ ∑i∈S vi is maximized.

Greedy Heuristic

Order items so that v1/wi ≥ · · · ≥ vn/wn cap ← W; S ← ∅ for i ← 1 to n do if wi ≤ cap then S ← S ∪ { i } cap ← cap − wi return S

Image from: http://commons.wikimedia.org/wiki/File:Knapsack.svg Royer Greedy Algorithms

SLIDE 4

Knapsack, 2

The Knapsack Problem (KP)

Given: ◮ A knapsack with capacity W kgs. ◮ Items 1, . . . , n ◮ Item i has weight wi and value vi. Find: S ⊆ { 1, . . . , n } so that ◮ ∑i∈S wi ≤ W and ◮ ∑i∈S vi is maximized.

Greedy Heuristic

Order items so that v1/wi ≥ · · · ≥ vn/wn cap ← W; S ← ∅ for i ← 1 to n do if wi ≤ cap then S ← S ∪ { i } cap ← cap − wi return S

Value = 11 In general, this heurist gives suboptimal answers!!!

Image from: http://commons.wikimedia.org/wiki/File:Knapsack_greedy.svg Royer Greedy Algorithms

Knapsack, 3

The Continuous Knapsack Problem (CKP)

Given: A knapsack and items (with weights and values) as before. Find: q1, . . . , qn where qi ∈ [0, 1] is the fraction of item i taken and ◮ ∑i∈S qi · wi ≤ W and ◮ ∑i∈S qi · vi is maximized. function gk(. . . ) Order items so that v1

wi ≥ . . . vn wn

cap ← W; S ← ∅ for i ← 1 to n do if wi ≤ cap then qi ← 1; cap ← cap − wi else if cap > 0 then qi ← cap

wi ; cap ← 0

else qi ← 0 return [q1, . . . , qn] Item Wght Value Val/Wght q 1 9 10 1.11 1.0 2 12 7 0.58 0.5 3 2 1 0.50 0.0 4 7 3 0.43 0.0 5 5 2 0.40 0.0 Value = 13.50

Royer Greedy Algorithms

Knapsack, 4

• Cont. Knapsack Prob. (CKP)

Given: A knapsack and items as before. Find: q1, . . . , qn ∈ [0, 1] such that ◮ ∑i∈S qi · wi ≤ W and ◮ ∑i∈S qi · vi is maximized. function gk(. . . ) Order items so that v1

wi ≥ · · · ≥ vn wn

cap ← W; S ← ∅ for i ← 1 to n do if wi ≤ cap then qi ← 1; cap ← cap − wi else if cap > 0 then qi ← cap

wi ; cap ← 0

else qi ← 0 return [q1, . . . , qn]

Lemma

The above solves the CKP. Proof: Suppose ◮ [q1, . . . , qn] is the result of gk. ◮ [p1, . . . , pn] is an optimal solution. If qi = pi for i = 1, . . . , n we are done. Otherwise, let j = min{ i : qi = pi }. By the greedy choice, qj > pj. (Why?) Let [p′

1, . . . , p′ n] be such that

◮ p′

i = pi = qi for i < j and p′ j = qj

◮ p′

j+1, . . . , p′ n is the result of

subtracting (pi − qi) · wi much weight from items i + 1, . . . , n. Claim 1: [p′

1, . . . , p′ n] is an opt. solution.

(Why?) Claim 2: By repeating the above trick as many times as needed, [p1, . . . , pn] can be transformed into [q1, . . . , qn] maintaining optimality along the way.

Royer Greedy Algorithms

Minimal Spanning Trees, 1

Minimal Spanning Tree Problem (MSTP)

Given: A connected undirected graph (V, E) & len: E → R+. Find: A minimal cost subgraph that connects all the vertices.

Property 1

Deleting an edge from a cycle cannot disconnect a graph.

Corollary

(a) So the min-cost subgraph above must be a tree. (b) Deleting an edge from a tree disconnects it (into two trees).

Image from: http://en.wikipedia.org/wiki/Minimum_spanning_tree

[Copy Corollary (b) onto the board]

Royer Greedy Algorithms

SLIDE 5

Digression on Trees, 1

Definition

A tree is a connected, acyclic, undirected graph.

Property 2

A tree on n vertices has n − 1 edges. Notation: ◮ #v(G) = the number of vertices in G. ◮ #e(G) = the number of edges in G. Proof: Base case. n = 1. Easy! Why? Proof (continued). Induction step. n > 1. Suppose T is a tree with #v(T) = n. IH: Prop. 2 holds all trees T′ with #v(T′) < n. Pick an edge (u, v) in T and delete it. This disconnects T into trees T1 and T2 (by Cor. (b)). Note that: #v(T1), #v(T2) < n & #v(T1) + #v(T2) = n. So by the IH: #e(T1) = #v(T1) − 1. #e(T2) = #v(T2) − 1.

∴ Adding (u, v) back in results in

#e(T) = 1 + #e(T1) + #e(T2) = 1 + (#v(T1) − 1) + (#v(T2) − 1) = #v(T) − 1 = n − 1.

Royer Greedy Algorithms

Digression on Trees, 2

Definition

A tree is a connected, acyclic, undirected graph.

Property 3

Suppose G = (V, E) is a connected undirected graph with |E| = |V| − 1. Then G is a tree.

Proof.

Since G is connected, it has a spanning subtree T. #v(T) = |V| and #e(T) = #v(T) − 1 (since T is a tree). So #e(T) = |E|.

∴ T = G.

Royer Greedy Algorithms

Digression on Trees, 3

Definition

A tree is a connected, acyclic, undirected graph.

Property 4

Suppose G is an undirected graph. G is a tree ⇐ ⇒ there is a unique path between any two vertices in G.

Proof.

(⇒) By Cor. (b), in an acyclic graph, there is at most one path between any two vertices. So in a tree (= acyclic + connected), there must be exactly one path between any two vertices. (⇐) Suppose there is a unique path between any two vertices in G. So G is connected . . . and G must be acyclic (Cor. (b) again).

Royer Greedy Algorithms

How to make the Greedy Choice for MST, 1

Minimal Spanning Tree Problem (MSTP)

Given: A connected undirected graph G = (V, E) & len: E → R+. Find: A minimal cost subgraph that connects all the vertices.

Cut Property

Supppose ◮ X ⊆ E is a part of a MST of G. ◮ S ⊆ V is such that no e ∈ X crosses the S/(V − S) boundary. ◮ e is a lightest crossing this boundary. Then X ∪ { e } is part of some MST.

e S V − S e

Image from: DPV. Royer Greedy Algorithms

SLIDE 6

How to make the Greedy Choice for MST, 2

Cut Property

Supppose ◮ X ⊆ E is a part of a MST of G. ◮ S ⊆ V is such that no e ∈ X crosses the S/(V − S) boundary. ◮ e is a lightest crossing this boundary. Then X ∪ { e } is part of some MST.

e S V − S e

Proof. Supppse X ⊂ T = some MST for G. If e is part of T we are done. So suppose e is not part of T. Then T ∪ { e } has a cycle. (Why?) ∴ There is a T-edge e′ that crosses the S/(V − S) boundary. (Why?) Let T′ = T ∪ { e } − { e′ }. Claim: T′ is a tree. (Why?) Claim: T′ is a MST for G. wght(T′) = wght(T) + len(e) − len(e′) ≤ wght(T) (as len(e) ≤ len(e′)) But T is a MST. ∴ wght(T′) = wght(T). ∴ T′ is a MST ⊇ X ∪ { e }.

✷

Royer Greedy Algorithms

Minimal Spanning Trees: Basic Scheme

Cut Property

Supppose ◮ X ⊆ E is a part of a MST of G. ◮ S ⊆ V is such that no e ∈ X crosses the S/(V − S) boundary. ◮ e is a lightest crossing this boundary. Then X ∪ { e } is part of some MST. MST Scheme X ← ∅ // edges picked so far while (|X| < |V| − 1) do Pick S ⊂ V ∋ X has no edges between S and (V − S) e ← a min-weight edge between S and (V − S) X ← X ∪ { e } return X Now all we have to do is make this precise.

Royer Greedy Algorithms

Prim’s Algorithm: Growing a MST

function Prim(G, wght) // Input: G = (V, E), a connected // undir. graph & wght: E → R+ // Output: A MST given by prev for each u ∈ V do cost[u] ← ∞; prev[u] ← null u0 ← some element of V cost[u0] ← 0; S ← ∅ // Make a priority queue using // cost-vals as keys H ← makequeue(V, cost) while (H is not empty) do v ← deleteMin(H); S ← S ∪ { v } for each z adjacent to v do if cost[z] > wght(v, z) & z / ∈ S then cost[z] ← wght(v, z) prev[z] ← v decreaseKey(H, z, cost[z]) return prev

B A

6 5 3 4 2

F D C E

5 4 1 2 4

Set S A B C D E F {} 0/nil ∞/nil ∞/nil ∞/nil ∞/nil ∞/nil A 5/A 6/A 4/A ∞/nil ∞/nil A, D 2/D 2/D ∞/nil 4/D A, D, B 1/B ∞/nil 4/D A, D, B, C 5/C 3/C A, D, B, C, F 4/F

B A F D C E

Images from DPV Royer Greedy Algorithms

Prim’s Algorithm: Growing a MST

function Prim(G, wght) // Input: G = (V, E), a connected // undir. graph & wght: E → R+ // Output: A MST given by prev for each u ∈ V do cost[u] ← ∞; prev[u] ← null u0 ← some element of V cost[u0] ← 0; S ← ∅ // Make a priority queue using // cost-vals as keys H ← makequeue(V, cost) while (H is not empty) do v ← deleteMin(H); S ← S ∪ { v } for each z adjacent to v do if cost[z] > wght(v, z) & z / ∈ S then cost[z] ← wght(v, z) prev[z] ← v decreaseKey(H, z, cost[z]) return prev

Runtime Analysis ◮ Setup Θ(|V|) ◮ Iterations of the while-loop: |V| ◮ Cost of deleteMin: O(log |V|) ◮ Cost of the for-loop: O(|E| log |V|)

(over the entire while-loop)

Total: O(|V| + |V| log |V| + |E| log |V|) = O(|E| log |V|).

Royer Greedy Algorithms

SLIDE 7

Prim’s Algorithm: Growing a MST

function Prim(G, wght) // Input: G = (V, E), a connected // undir. graph & wght: E → R+ // Output: A MST given by prev for each u ∈ V do cost[u] ← ∞; prev[u] ← null u0 ← some element of V cost[u0] ← 0; S ← ∅ // Make a priority queue using // cost-vals as keys H ← makequeue(V, cost) while (H is not empty) do v ← deleteMin(H); S ← S ∪ { v } for each z adjacent to v do if cost[z] > wght(v, z) & z / ∈ S then cost[z] ← wght(v, z) prev[z] ← v decreaseKey(H, z, cost[z]) return prev

Correctness Argument

Claim 1: Invariant for the while-loop: (∀z / ∈ S)[cost[z] = minv∈S wght(v, z)]. Proof: Easy induction. Let T = (V, { (prev[v], v) | v ∈ (V − { u0 }) }). Claim 2: T is a MST for G. Proof: Another induction using Claim 1 and the Cut Property.

Royer Greedy Algorithms

Kruskal’s Algorithm, 1

Disjoint Sets Data Structure

makeset(x) make a single set { x } find(x) return the label of x’s set union(x,y) merge x’s and y’s sets function kruskal(G, wght) // Input: G and wght as before // Output: X, the edges of a MST [e1, e2, . . . , ek] ← E sorted by weight for each v ∈ V do makeset(x) X ← ∅; i ← 1 // So wght(e1) ≤ · · · ≤ wght(ek) while (i ≤ k & |X| < |V| − 1) do Suppose (u, v) = ei if (find(u) = find(v)) then X ← X ∪ { ei }; union(u, v) i ← i + 1 return X

B A

6 5 3 4 2

F D C E

5 4 1 2 4

(B, C) (C, D) (B, D) (C, F) (A, D) (D, F) (E, F) (A, B) (C, E) (A, C) Now run the algorithm

Royer Greedy Algorithms

Kruskal’s Algorithm, 2

Disjoint Sets Data Structure

makeset(x) make a single set { x } find(x) return the label of x’s set union(x,y) merge x’s and y’s sets function kruskal(G, wght) // Input: G and wght as before // Output: X, the edges of a MST [e1, e2, . . . , ek] ← E sorted by weight for each v ∈ V do makeset(x) X ← ∅; i ← 1 // So wght(e1) ≤ · · · ≤ wght(ek) while (i ≤ k & |X| < |V| − 1) do Suppose (u, v) = ei if (find(u) = find(v)) then X ← X ∪ { ei }; union(u, v) i ← i + 1 return X

Cut Property

Supppose ◮ X ⊆ E is a part of a MST of G. ◮ S ⊆ V is such that no e ∈ X crosses the S/(V − S) boundary. ◮ e is a lightest crossing this boundary. Then X ∪ { e } is part of some MST. Claim 1: In K’s Algorithm, the disjoint sets are the connected components of X. Claim 2: In K’s Algorithm, when find(u) = find(v), (u, v) is always a lightest edge jointing u’s set to v’s set.

Royer Greedy Algorithms

Union/Find, 1

Disjoint Sets Data Structure

makeset(x) make a single set { x } find(x) return the label of x’s set union(x,y) merge x’s and y’s sets Idea: vertices = cities sets = countries labels = capital cities union(x, y) = x’s and y’s countries unify (and pick a new capital) par[x] = x, if x is a capital city par[x] = y (= x), if x is not a capital city where y is x’s regional capital Representation of the sets { B, E } and { A, C, D, F, G, H }.

E H B C F A D G Royer Greedy Algorithms

SLIDE 8

Union/Find, 2

rank[x] ≈ the height of the subtree below x procedure makeset(x) par[x] ← x; rank[x] ← 0 function find(x) while (x = par[x]) do x ← par[x] return x procedure union(x, y) cx ← find(x); cy ← find(y) if (cx = cy) then return if (rank[cx] > rank[cy]) then par[cy] ← cx else // So rank[cx] ≤ rank[cy] par[cx] ← cy if (rank[cx] = rank[cy]) then rank[cy] ← rank[cy] + 1

After makeset(A), makeset(B), . . . , makeset(G):

A B C D E F0 G

After union(A, D), union(B, E), union(C, F):

A B0 C0 G0 F1 E1 D1

After union(C, G), union(E, A):

B

1

F1 C G E D2 A0

After union(B, G): A G0 F E1 C0 D2 B0

1

Royer Greedy Algorithms

Union/Find, 3

rank[x] ≈ the height of the subtree below x procedure makeset(x) par[x] ← x; rank[x] ← 0 function find(x) while (x = par[x]) do x ← par[x] return x procedure union(x, y) cx ← find(x); cy ← find(y) if (cx = cy) then return if (rank[cx] > rank[cy]) then par[cy] ← cx else // So rank[cx] ≤ rank[cy] par[cx] ← cy if (rank[cx] = rank[cy]) then rank[cy] ← rank[cy] + 1

Property 1

For any x, rank[x] < rank[par[x]] when x = par[x]. Proof: By an easy induction.

Property 2

For any x, x has at least 2rank[x] nodes in its tree. Proof: By another easy induction.

Property 3

Suppose there are n elements total. Then there can be at most n/2k nodes of rank k. Proof: By Property 2. Corollary: The maximum rank is log2 n. Corollary: find and union are O(log n) time.

Royer Greedy Algorithms

Union/Find, 4: A Wonderful Trick

function find(x) // find with path compression if (x = par[x]) then par[x] ← find(par[x]) return par[x]

B0 D0 I0 J0 K0 H0 C1

1

G1 A

3

F E2

− →

B0 D K0 J0 I0 H0 C1 F

1

G1 A3 E2

− →

B0 D H0 J 0 I0 K0 G1 C1 F

1

E2 A

3

find(I) followed by find(K)

Royer Greedy Algorithms

Union/Find, 5

function find(x) // find with path compression if (x = par[x]) then par[x] ← find(par[x]) return par[x]

Lemma

Suppose there are n elements total and there are a sequence of m many finds. Then the total time for these is O(n log∗ n).

Definition

log∗ x =

• 0,

if x ≤ 1; 1 + log∗(log2 x), if x > 1. Examples: log∗(1) = 0 log∗(2) = 1 log∗(4) = 2 log∗(16) = 3 log∗(216) = 4 log∗(2216) = 5

In fact: We can replace log∗ with α, the inverse of Ackermann’s function! See https://en.wikipedia.org/wiki/Ackermann_function.

Royer Greedy Algorithms

SLIDE 9

Back to Kruskal’s Algorithm

Disjoint Sets Data Structure

makeset(x) make a single set { x } find(x) return the label of x’s set union(x,y) merge x’s and y’s sets function kruskal(G, wght) // Input: G and wght as before // Output: X, the edges of a MST [e1, e2, . . . , ek] ← E sorted by weight for each v ∈ V do makeset(x) X ← ∅; i ← 1 // So wght(e1) ≤ · · · ≤ wght(ek) while (i ≤ k & |X| < |V| − 1) do Suppose (u, v) = ei if (find(u) = find(v)) then X ← X ∪ { ei }; union(u, v) i ← i + 1 return X

Runtime Analysis ◮ |V| many makesets: Θ(|V|) time. ◮ Sorting |E|: Θ(|E| log2 |E|). ◮ The for-loop goes through at most |E|-many iterations. ◮ In the for-loop:

◮ 2|E|-many finds ◮ (|V| − 1)-many unions

Total:

O(|V| + |E| log |E| + (|E| + |V|) log∗ |V|) = O(|E| log |V|) Claim: O(log |E|) = O(log |V|). (Why?)

Royer Greedy Algorithms