CS4102 Algorithms Summer 2020 Warm up Why is an algorithms space - - PowerPoint PPT Presentation

Warm up Why is an algorithm’s space complexity (how much memory it uses) important? Why might a memory-intensive algorithm be a “bad” one?

1

CS4102 Algorithms

Summer 2020

Why lots of memory is “bad”

2

Greedy Algorithms

• Require Optimal Substructure

– Solution to larger problem contains the solution to a smaller one – Only one subproblem to consider!

• Idea:
• 1. Identify a greedy choice property
• How to make a choice guaranteed to be included in some optimal solution
• 2. Repeatedly apply the choice property until no subproblems remain

3

Exchange argument

• Shows correctness of a greedy algorithm
• Idea:

– Show exchanging an item from an arbitrary optimal solution with your greedy choice makes the new solution no worse – How to show my sandwich is at least as good as yours:

• Show: “I can remove any item from your sandwich, and it would be no worse

by replacing it with the same item from my sandwich”

4

Von Neumann Bottleneck

• Named for John von Neumann
• Inventor of modern computer architecture
• Other notable influences include:

– Mathematics – Physics – Economics – Computer Science

5

Von Neumann Bottleneck

• Reading from memory is VERY slow
• Big memory = slow memory
• Solution: hierarchical memory
• Takeaway for Algorithms: Memory is time, more memory is a

lot more time

6

CPU, registers Cache Disk If not look here Hopefully your data in here Hope it’s not here Access time: 1 cycle Access time: 10 cycles Access time: 1,000,000 cycles

Caching Problem

• Cache misses are very expensive
• When we load something new into cache, we must eliminate

something already there

• We want the best cache “schedule” to minimize the number of

misses

7

Caching Problem Definition

• Input:

– 𝑙 = size of the cache – 𝑁 = 𝑛1, 𝑛2, … 𝑛𝑜 = memory access pattern

• Output:

– “schedule” for the cache (list of items in the cache at each time) which minimizes cache fetches

8

Example

9

A B C D A D E A D B A E C E A

A B C

Example

10

A B C D A D E A D B A E C E A

A B C A B C

Example

11

A B C D A D E A D B A E C E A

A B C A B C A B C

Example

12

A B C D A D E A D B A E C E A

A B C

We must evict something to make room for D

A B C A B C A B C

Example

13

A B C D A D E A D B A E C E A

D B C

If we evict A

A B C A B C A B C A B C

Example

14

A B C D A D E A D B A E C E A

A B D

If we evict C

A B C A B C A B C A B C

Our Problem vs Reality

• Assuming we know the entire access pattern
• Cache is Fully Associative
• Counting # of fetches (not necessarily misses)
• “Reduced” Schedule: Address only loaded on the cycle it’s required

– Reduced == Unreduced (by number of fetches)

15

A B C D A D E A D B A E C E A A B C D A D E A D B A E C E A

A B C A B C A B C A B C D B C D B C A B C A B C

Unreduced Reduced

Leaving A in longer does not save fetches

Greedy Algorithms

• Require Optimal Substructure

– Solution to larger problem contains the solution to a smaller one – Only one subproblem to consider!

• Idea:
• 1. Identify a greedy choice property
• How to make a choice guaranteed to be included in some optimal solution
• 2. Repeatedly apply the choice property until no subproblems remain

16

Greedy choice property

• Belady evict rule:

– Evict the item accessed farthest in the future

17

A B C D A D E A D B A E C E A

A B C Evict C A B C A B C A B C

Greedy choice property

• Belady evict rule:

– Evict the item accessed farthest in the future

18

A B C D A D E A D B A E C E A

A B D A B C A B C A B C A B D A B D A B D Evict B

Greedy choice property

• Belady evict rule:

– Evict the item accessed farthest in the future

19

A B C D A D E A D B A E C E A

A B D A B C A B C A B C A B D A B D A E D A E D A E D A E D Evict D

Greedy choice property

• Belady evict rule:

– Evict the item accessed farthest in the future

20

A B C D A D E A D B A E C E A

A B D A B C A B C A B C A B D A B D A E D A E D A E D A E B A E B A E B A E B Evict B

Greedy choice property

• Belady evict rule:

– Evict the item accessed farthest in the future

21

A B C D A D E A D B A E C E A

A B D A B C A B C A B C A B D A B D A E D A E D A E D A E B A E B A E B A E C A E C A E C

4 Cache Misses

Greedy Algorithms

• Require Optimal Substructure

– Solution to larger problem contains the solution to a smaller one – Only one subproblem to consider!

• Idea:
• 1. Identify a greedy choice property
• How to make a choice guaranteed to be included in some optimal solution
• 2. Repeatedly apply the choice property until no subproblems remain

22

Caching Greedy Algorithm

Initialize 𝑑𝑏𝑑ℎ𝑓= first k accesses For each 𝑛𝑗 ∈ 𝑁: if 𝑛𝑗 ∈ 𝑑𝑏𝑑ℎ𝑓: print 𝑑𝑏𝑑ℎ𝑓 else: 𝑛 = furthest-in-future from cache evict 𝑛, load 𝑛𝑗 print 𝑑𝑏𝑑ℎ𝑓

23

𝑃(𝑙) 𝑜 times 𝑃(𝑙) 𝑃(𝑙) 𝑃(𝑙𝑜) 𝑃(1) 𝑃(𝑙)

𝑃(𝑙𝑜2)

Exchange argument

• Shows correctness of a greedy algorithm
• Idea:

– Show exchanging an item from an arbitrary optimal solution with your greedy choice makes the new solution no worse – How to show my sandwich is at least as good as yours:

• Show: “I can remove any item from your sandwich, and it would be no worse

by replacing it with the same item from my sandwich”

24

Belady Exchange Lemma

Let 𝑇𝑔𝑔 be the schedule chosen by our greedy algorithm Let 𝑇𝑗 be a schedule which agrees with 𝑇𝑔𝑔 for the first 𝑗 memory accesses. We will show: there is a schedule 𝑇𝑗+1 which agrees with 𝑇𝑔𝑔 for the first 𝑗 + 1 memory accesses, and has no more misses than 𝑇𝑗 (i.e. 𝑛𝑗𝑡𝑡𝑓𝑡 𝑇𝑗+1 ≤ 𝑛𝑗𝑡𝑡𝑓𝑡(𝑇𝑗))

25

𝑇∗

Agrees with 𝑇𝑔𝑔 on first 0 accesses

𝑇1 𝑇2

Agrees with 𝑇𝑔𝑔 on first access Agrees with 𝑇𝑔𝑔 on first 2 accesses

… 𝑇𝑔𝑔

Agrees with 𝑇𝑔𝑔 on all 𝑜 accesses Lemma Lemma Lemma Lemma Optimal Greedy

Belady Exchange Proof Idea

26

𝑇𝑗 𝑇𝑔𝑔 𝑇𝑗+1

First 𝑗 accesses

Must agree with 𝑇𝑔𝑔 Need to fill in the rest

• f 𝑇𝑗+1 to have no

more misses than 𝑇𝑗

𝑇𝑗 Cache after 𝑗

Proof of Lemma

Goal: find 𝑇𝑗+1 s.t. 𝑛𝑗𝑡𝑡𝑓𝑡 𝑇𝑗+1 ≤ 𝑛𝑗𝑡𝑡𝑓𝑡(𝑇𝑗) Since 𝑇𝑗 agrees with 𝑇𝑔𝑔 for the first 𝑗 accesses, the state of the cache at access 𝑗 + 1 will be the same

27

𝑇𝑔𝑔 Cache after 𝑗

=

Consider access 𝑛𝑗+1 = 𝑒

Case 1: if 𝑒 is in the cache, then neither 𝑇𝑗 nor 𝑇𝑔𝑔 evict from the cache, use the same cache for 𝑇𝑗+1

𝑔 𝑓 𝑔 𝑓 𝑇𝑗+1 Cache after 𝑗 𝑔 𝑓 𝑒 𝑒 𝑒

𝑇𝑗 Cache after 𝑗

Proof of Lemma

Goal: find 𝑇𝑗+1 s.t. 𝑛𝑗𝑡𝑡𝑓𝑡 𝑇𝑗+1 ≤ 𝑛𝑗𝑡𝑡𝑓𝑡(𝑇𝑗) Since 𝑇𝑗 agrees with 𝑇𝑔𝑔 for the first 𝑗 accesses, the state of the cache at access 𝑗 + 1 will be the same

28

𝑇𝑔𝑔 Cache after 𝑗

=

Consider access 𝑛𝑗+1 = 𝑒

𝑔 𝑓 𝑔 𝑓

Case 2: if 𝑒 isn’t in the cache, and both 𝑇𝑗 and 𝑇𝑔𝑔 evict 𝑔 from the cache, evict 𝑔 for 𝑒 in 𝑇𝑗+1

𝑇𝑗+1 Cache after 𝑗 𝑒 𝑓

𝑇𝑗 Cache after 𝑗

Proof of Lemma

Goal: find 𝑇𝑗+1 s.t. 𝑛𝑗𝑡𝑡𝑓𝑡 𝑇𝑗+1 ≤ 𝑛𝑗𝑡𝑡𝑓𝑡(𝑇𝑗) Since 𝑇𝑗 agrees with 𝑇𝑔𝑔 for the first 𝑗 accesses, the state of the cache at access 𝑗 + 1 will be the same

29

𝑇𝑔𝑔 Cache after 𝑗

=

Consider access 𝑛𝑗+1 = 𝑒

𝑔 𝑓 𝑔 𝑓

Case 3: if 𝑒 isn’t in the cache, 𝑇𝑗 evicts 𝑓 and 𝑇𝑔𝑔 evicts 𝑔 from the cache

𝑇𝑗 Cache after 𝑗 + 1 𝑇𝑔𝑔 Cache after 𝑗 + 1

≠

𝑔 𝑒 𝑒 𝑓

Case 3

30

𝑇𝑗 𝑇𝑔𝑔 𝑇𝑗+1

First 𝑗 accesses

Must agree with 𝑇𝑔𝑔 Need to fill in the rest

• f 𝑇𝑗+1 to have no

more misses than 𝑇𝑗

Case 3

31

𝑇𝑗 𝑇𝑔𝑔 𝑇𝑗+1

𝑛𝑢

First 𝑗 accesses

First place 𝑇𝑗 involves 𝑓 or 𝑔

Copy 𝑇𝑗

𝑛𝑢 = the first access after 𝑗 + 1 in which 𝑇𝑗 deals with 𝑓 or 𝑔

3 options: 𝒏𝒖 = 𝒇 or 𝒏𝒖 = 𝒈 or 𝒏𝒖 = 𝒚 ≠ 𝒇, 𝒈

Case 3, 𝑛𝑢 = 𝑓

32

𝑇𝑗 𝑇𝑔𝑔 𝑇𝑗+1

𝑓

First 𝑗 accesses

First place 𝑇𝑗 uses 𝑓 or 𝑔

Copy 𝑇𝑗

𝑛𝑢 = the first access after 𝑗 + 1 in which 𝑇𝑗 deals with 𝑓 or 𝑔 3 options: 𝒏𝒖 = 𝒇 or 𝒏𝒖 = 𝒈 or 𝒏𝒖 = 𝒚 ≠ 𝒇, 𝒈

Case 3, 𝑛𝑢 = 𝑓

Goal: find 𝑇𝑗+1 s.t. 𝑛𝑗𝑡𝑡𝑓𝑡 𝑇𝑗+1 ≤ 𝑛𝑗𝑡𝑡𝑓𝑡(𝑇𝑗)

33

𝑇𝑗 Cache after 𝑢 − 1 𝑇𝑗+1 Cache after 𝑢 − 1

≠

𝑔 𝑒 𝑒 𝑓

𝑇𝑗 must load 𝑓 into the cache, assume it evicts 𝑦 𝑇𝑗+1 will load 𝑔 into the cache, evicting 𝑦

𝑇𝑗+1 behaved exactly the same as 𝑇𝑗 between 𝑗 and 𝑢, and has the same cache after 𝑢, therefore 𝑛𝑗𝑡𝑡𝑓𝑡 𝑇𝑗+1 = 𝑛𝑗𝑡𝑡𝑓𝑡(𝑇𝑗)

The caches now match!

𝑦 𝑦 𝑓 𝑔

Case 3, 𝑛𝑢 = 𝑔

34

𝑇𝑗 𝑇𝑔𝑔 𝑇𝑗+1

𝑔

First 𝑗 accesses

First place 𝑇𝑗 uses 𝑓 or 𝑔

Copy 𝑇𝑗

𝑛𝑢 = the first access after 𝑗 + 1 in which 𝑇𝑗 deals with 𝑓 or 𝑔 3 options: 𝒏𝒖 = 𝒇 or 𝒏𝒖 = 𝒈 or 𝒏𝒖 = 𝒚 ≠ 𝒇, 𝒈

Case 3, 𝑛𝑢 = 𝑔

Cannot Happen!

35

𝑇𝑗 𝑇𝑔𝑔 𝑇𝑗+1

𝑔 First place 𝑇𝑗 uses 𝑓 or 𝑔 “Evict 𝑔" “Evict 𝑔" Means 𝑔 not farthest future access!

Case 3, 𝑛𝑢 = 𝑦 ≠ 𝑓, 𝑔

36

𝑇𝑗 𝑇𝑔𝑔 𝑇𝑗+1

𝑦

First 𝑗 accesses

First place 𝑇𝑗 uses 𝑓 or 𝑔

Copy 𝑇𝑗

𝑛𝑢 = the first access after 𝑗 + 1 in which 𝑇𝑗 deals with 𝑓 or 𝑔 3 options: 𝒏𝒖 = 𝒇 or 𝒏𝒖 = 𝒈 or 𝒏𝒖 = 𝒚 ≠ 𝒇, 𝒈

Case 3, 𝑛𝑢 = 𝑦 ≠ 𝑓, 𝑔

Goal: find 𝑇𝑗+1 s.t. 𝑛𝑗𝑡𝑡𝑓𝑡 𝑇𝑗+1 ≤ 𝑛𝑗𝑡𝑡𝑓𝑡(𝑇𝑗)

37

𝑇𝑗 Cache after 𝑢 − 1 𝑇𝑗+1 Cache after 𝑢 − 1

≠

𝑔 𝑒 𝑒 𝑓

𝑇𝑗 loads 𝑦 into the cache, it must be evicting 𝑔 𝑇𝑗+1 will load 𝑦 into the cache, evicting 𝑓

𝑇𝑗+1 behaved exactly the same as 𝑇𝑗 between 𝑗 and 𝑢, and has the same cache after 𝑢, therefore 𝑛𝑗𝑡𝑡𝑓𝑡 𝑇𝑗+1 = 𝑛𝑗𝑡𝑡𝑓𝑡(𝑇𝑗)

𝑦 𝑦

The caches now match!

Use Lemma to show Optimality

38

𝑇∗

Agrees with 𝑇𝑔𝑔 on first 0 accesses

𝑇1 𝑇2

Agrees with 𝑇𝑔𝑔 on first access Agrees with 𝑇𝑔𝑔 on first 2 accesses

… 𝑇𝑔𝑔

Agrees with 𝑇𝑔𝑔 on all 𝑜 accesses Lemma Lemma Lemma Lemma

ARPANET

39

Problem

40

We need to connect together all these places into a network We have feasible wires to run, plus the cost of each wire Find the cheapest set of wires to run to connect all places

10 2 6 11 9 5 8 3 7 3 1 8 12 9

Find a Minimum Spanning Tree

Graphs

41

10 2 6 11 9 5 8 3 7 3 1 8 12 9 A B C D E F G I H

Definition: 𝐻 = (𝑊, 𝐹)

Vertices/Nodes Edges

𝑥 𝑓 = weight of edge 𝑓

𝑊 = {𝐵, 𝐶, 𝐷, 𝐸, 𝐹, 𝐺, 𝐻, 𝐼, 𝐽} 𝐹 = { 𝐵, 𝐶 , 𝐵, 𝐷 , 𝐶, 𝐷 , … }

Adjacency List Representation

42

10 2 6 11 9 5 8 3 7 3 1 8 12 9 A B C D E F G I H A B C D E F G H I B C A C E A B D F C E F B D G H C D G E F H I E G I G H

Tradeoffs Space: Time to list neighbors: Time to check edge (𝐵, 𝐶):

𝑊 + 𝐹 𝐸𝑓𝑕𝑠𝑓𝑓(𝐵) 𝐸𝑓𝑕𝑠𝑓𝑓(𝐵)

Adjacency Matrix Representation

43

10 2 6 11 9 5 8 3 7 3 1 8 12 9 A B C D E F G I H A B C D E F G H I

Tradeoffs Space: Time to list neighbors: Time to check edge (𝐵, 𝐶):

𝑊2 𝑊 𝑃(1)

A B C D E F G H I 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Definition: Path

44

10 2 6 11 9 5 8 3 7 3 1 8 12 9 A B C D E F G I H

A sequence of nodes (𝑤1, 𝑤2, … , 𝑤𝑙) s.t. ∀1 ≤ 𝑗 ≤ 𝑙 − 1, 𝑤𝑗, 𝑤𝑗+1 ∈ 𝐹 Simple Path: A path in which each node appears at most once Cycle: A path of > 2 nodes in which 𝑤1 = 𝑤𝑙

Definition: Connected Graph

45

A Graph 𝐻 = (𝑊, 𝐹) s.t. for any pair of nodes 𝑤1, 𝑤2 ∈ 𝑊 there is a path from 𝑤1 to 𝑤2

10 2 6 11 9 5 8 3 7 3 1 8 12 9 A B C D E F G I H

Definition: Tree

46

A connected graph with no cycles

10 11 9 5 3 7 3 12 A B C D E F G I H

Definition: Spanning Tree

47

10 2 6 11 9 5 8 3 7 3 1 8 12 9 A B C D E F G I H

A Tree 𝑈 = (𝑊𝑈, 𝐹𝑈) which connects (“spans”) all the nodes in a graph 𝐻 = (𝑊, 𝐹)

How many edges does 𝑈 have?

𝑊 − 1

Definition: Minimum Spanning Tree

48

10 2 6 11 9 5 8 3 7 3 1 8 12 9 A B C D E F G I H

A Tree 𝑈 = (𝑊𝑈, 𝐹𝑈) which connects (“spans”) all the nodes in a graph 𝐻 = (𝑊, 𝐹), that has minimal cost

𝐷𝑝𝑡𝑢 𝑈 = 𝑥(𝑓)

𝑓∈𝐹𝑈

How many edges does 𝑈 have?

𝑊 − 1

Greedy Algorithms

• Require Optimal Substructure

– Solution to larger problem contains the solution to a smaller one – Only one subproblem to consider!

• Idea:
• 1. Identify a greedy choice property
• How to make a choice guaranteed to be included in some optimal solution
• 2. Repeatedly apply the choice property until no subproblems remain

49

Kruskal’s Algorithm

50

Start with an empty tree 𝐵 Add to 𝐵 the lowest-weight edge that does not create a cycle

10 2 6 11 9 5 8 3 7 3 1 8 12 9 A B C D E F G I H

Kruskal’s Algorithm

51

Start with an empty tree 𝐵 Add to 𝐵 the lowest-weight edge that does not create a cycle

10 2 6 11 9 5 8 3 7 3 1 8 12 9 A B C D E F G I H

Kruskal’s Algorithm

52

10 2 6 11 9 5 8 3 7 3 1 8 12 9 A B C D E F G I H

Start with an empty tree 𝐵 Add to 𝐵 the lowest-weight edge that does not create a cycle

Kruskal’s Algorithm

53

10 2 6 11 9 5 8 3 7 3 1 8 12 9 A B C D E F G I H

Start with an empty tree 𝐵 Add to 𝐵 the lowest-weight edge that does not create a cycle

Kruskal’s Algorithm

54

10 2 6 11 9 5 8 3 7 3 1 8 12 9 A B C D E F G I H

Start with an empty tree 𝐵 Add to 𝐵 the lowest-weight edge that does not create a cycle

Kruskal’s Algorithm

55

10 2 6 11 9 5 8 3 7 3 1 8 12 9 A B C D E F G I H

Start with an empty tree 𝐵 Add to 𝐵 the lowest-weight edge that does not create a cycle

Definition: Cut

56

A Cut of graph 𝐻 = (𝑊, 𝐹) is a partition of the nodes into two sets, 𝑇 and 𝑊 − 𝑇

10 2 6 11 9 5 8 3 7 3 1 8 12 9 A B C D E F G I H

𝑇

Edge 𝑤1, 𝑤2 ∈ 𝐹 crosses a cut if 𝑤1 ∈ 𝑇 and 𝑤2 ∈ 𝑊 − 𝑇 (or opposite), e.g. (𝐵, 𝐷) A set of edges 𝑆 Respects a cut if no edges cross the cut e.g. 𝑆 = { 𝐵, 𝐶 , 𝐹, 𝐻 , 𝐺, 𝐻 }

Exchange argument

• Shows correctness of a greedy algorithm
• Idea:

– Show exchanging an item from an arbitrary optimal solution with your greedy choice makes the new solution no worse – How to show my sandwich is at least as good as yours:

• Show: “I can remove any item from your sandwich, and it would be no worse

by replacing it with the same item from my sandwich”

57

Cut Theorem

If a set of edges 𝐵 is a subset of a minimum spanning tree 𝑈, let (𝑇, 𝑊 − 𝑇) be any cut which 𝐵 respects. Let 𝑓 be the least-weight edge which crosses (𝑇, 𝑊 − 𝑇). 𝐵 ∪ {𝑓} is also a subset of a minimum spanning tree.

58

10 2 6 11 9 5 8 3 7 3 1 8 12 9 A B C D E F G I H

Proof of Cut Theorem

59

Claim: If 𝐵 is a subset of a MST 𝑈, and 𝑓 is the least- weight edge which crosses cut (𝑇, 𝑊 − 𝑇) (which 𝐵 respects) then 𝐵 ∪ {𝑓} is also a subset of a MST.

𝑇

𝑈 𝐵 ⊆ 𝑈 𝑓

Consider some MST 𝑈, Case 1: (the easy case) If 𝑓 ∈ 𝑈 Then claim holds

Proof of Cut Theorem

60

Claim: If 𝐵 is a subset of a MST 𝑈, and 𝑓 is the least- weight edge which crosses cut (𝑇, 𝑊 − 𝑇) (which 𝐵 respects) then 𝐵 ∪ {𝑓} is also a subset of a MST.

𝑈 𝐵 ⊆ 𝑈 Consider some MST 𝑈, Case 2: Consider if 𝑓 = (𝑤1, 𝑤2) ∉ 𝑈 Since 𝑈 is a MST, there is some path from 𝑤1 to 𝑤2. Let 𝑓′ be the first edge on this path which crosses the cut Build tree 𝑈′ by exchanging 𝑓′ for 𝑓

𝑤2 𝑤1 𝑇

𝑓 𝑓′

Proof of Cut Theorem

61

Claim: If 𝐵 is a subset of a MST 𝑈, and 𝑓 is the least- weight edge which crosses cut (𝑇, 𝑊 − 𝑇) (which 𝐵 respects) then 𝐵 ∪ {𝑓} is also a subset of a MST.

𝑈 𝐵 ⊆ 𝑈 Consider some MST 𝑈, Case 2: Consider if 𝑓 = (𝑤1, 𝑤2) ∉ 𝑈

𝑤2 𝑤1 𝑇

𝑓 𝑓′ We assumed 𝑥 𝑓 ≤ 𝑥(𝑓′) 𝑥 𝑈′ = 𝑥 𝑈 − 𝑥 𝑓′ + 𝑥(𝑓) 𝑥 𝑈′ ≤ 𝑥 𝑈 So 𝑈′ is also a MST! Thus the claim holds 𝑈′ = 𝑈 with edge 𝑓 instead of 𝑓′

Kruskal’s Algorithm

62

10 2 6 11 9 5 8 3 7 3 1 8 12 9 A B C D E F G I H

Start with an empty tree 𝐵 Repeat 𝑊 − 1 times: Add the min-weight edge that doesn’t cause a cycle

𝑇

𝑓 Keep edges in a Disjoint-set data structure (very fancy) 𝑃 𝐹 log 𝑊

General MST Algorithm

63

10 2 6 11 9 5 8 3 7 3 1 8 12 9 A B C D E F G I H

Start with an empty tree 𝐵 Repeat 𝑊 − 1 times: Pick a cut (𝑇, 𝑊 − 𝑇) which 𝐵 respects Add the min-weight edge which crosses (𝑇, 𝑊 − 𝑇)

Prim’s Algorithm

64

10 2 7 11 9 5 6 3 7 3 1 8 12 9 A B C D E F G I H

Start with an empty tree 𝐵 Repeat 𝑊 − 1 times: Pick a cut (𝑇, 𝑊 − 𝑇) which 𝐵 respects Add the min-weight edge which crosses (𝑇, 𝑊 − 𝑇) 𝑇 is all endpoint of edges in 𝐵 𝑓 is the min-weight edge that grows the tree

Prim’s Algorithm

65

10 2 7 11 9 5 6 3 7 3 1 8 12 9 A B C D E F G I H

Start with an empty tree 𝐵 Pick a start node Repeat 𝑊 − 1 times: Add the min-weight edge which connects to node in 𝐵 with a node not in 𝐵

Prim’s Algorithm

66

10 2 7 11 9 5 6 3 7 3 1 8 12 9 A B C D E F G I H

Start with an empty tree 𝐵 Pick a start node Repeat 𝑊 − 1 times: Add the min-weight edge which connects to node in 𝐵 with a node not in 𝐵

Prim’s Algorithm

67

10 2 7 11 9 5 6 3 7 3 1 8 12 9 A B C D E F G I H

Start with an empty tree 𝐵 Pick a start node Repeat 𝑊 − 1 times: Add the min-weight edge which connects to node in 𝐵 with a node not in 𝐵

Prim’s Algorithm

68

10 2 7 11 9 5 6 3 7 3 1 8 12 9 A B C D E F G I H

Start with an empty tree 𝐵 Pick a start node Repeat 𝑊 − 1 times: Add the min-weight edge which connects to node in 𝐵 with a node not in 𝐵

Prim’s Algorithm

69

10 2 7 11 9 5 6 3 7 3 1 8 12 9 A B C D E F G I H

Start with an empty tree 𝐵 Pick a start node Repeat 𝑊 − 1 times: Add the min-weight edge which connects to node in 𝐵 with a node not in 𝐵

Keep edges in a Heap 𝑃 𝐹 log 𝑊

Summary of MST results

• Fredman-Tarjan ‘84:

Θ(𝐹 + 𝑊 log 𝑊)

• Gabow et al ‘86:

Θ(𝐹 log log∗ 𝑊)

• Chazelle ‘00:

Θ(𝐹𝛽 𝑊 )

• Pettie-Ramachandran ’02:Θ(? )(optimal)
• Karger-Klein-Tarjan ‘95: Θ(𝐹) (randomized)

70