CS 310 Advanced Data Structures and Algorithms Weighted Graphs - - PowerPoint PPT Presentation

CS 310 – Advanced Data Structures and Algorithms

Weighted Graphs July 20, 2017

Tong Wang UMass Boston CS 310 July 20, 2017 1 / 34

Weighted Graphs

Each edge has a weight (cost) Edge-weighted graphs Mostly we consider only positive weights

Minimum spanning trees Shortest paths

Tong Wang UMass Boston CS 310 July 20, 2017 2 / 34

Minimum Spanning Tree

Given a connected undirected graph, we often want the cheapest way to connect all the nodes This is a minimum cost spanning tree A spanning tree is a tree that contains all the nodes in the graph The total cost is just the sum of the edge costs The minimum spanning tree is the spanning tree whose total cost is minimal among all spanning trees The kind of tree involved here has no particular root node, and such trees are called free trees, because they are not tied down by a root

Tong Wang UMass Boston CS 310 July 20, 2017 3 / 34

MST Example

Point set in the plane Geometric spanning trees The middle one is the minimum spanning tree The one on the right has shortest paths to the central node

Tong Wang UMass Boston CS 310 July 20, 2017 4 / 34

MST Example

CLRS 23.1

Tong Wang UMass Boston CS 310 July 20, 2017 5 / 34

Prim’s Algorithm for MST

A greedy algorithm: Decide what to do next by selecting the immediately best option, without regard to the global structure Pseudocode

1

Choose an arbitrary vertex s as the first tree node

2

Add s to PrimTree

3

while there are still nontree nodes

1

Choose the edge of minimum weight between a tree node and a nontree node, (u, v)

2

Add (u, v) to TreeEdge and v to PrimTree

Tong Wang UMass Boston CS 310 July 20, 2017 6 / 34

Prim’s Algorithm for MST

Choose an arbitrary node s Initialize tree T = { s } Initialize E = { } Repeat until all nodes are in T: Choose an edge (u, v) with minimum weight such that u is in T and v is not Add v to T, add (u, v) to E Output (T, E)

Tong Wang UMass Boston CS 310 July 20, 2017 7 / 34

Correctness of Prim’s Algorithm

Prim’s alogrithm creates a spanning tree

No cycle can be introduced by adding edges between tree and nontree vertices

Why does the tree has the minimal weight? Proof by contradiction Assume Prim’s algorithm does not return an MST Then there is an edge (x, y) that does not belong to the MST Instead, (x, y) is replaced by a different edge (v1, v2)

To maintain the property of spanning, when we remove an edge, we must add another edge

The cost of (x, y), however, is not greater than the cost of (v1, v2) Thus Prim’s tree is still minimal after all

Tong Wang UMass Boston CS 310 July 20, 2017 8 / 34

Correctness of Prim’s Algorithm

If we remove (x, y), a different edge must be added In the new spanning tree, let (v1, v2) be the first edge along the path from x to y that is not in the tree generated by Prim’s algorithm Cost(x, y) ≤ Cost(v1, v2) Substitute (x, y) for (v1, v2) back into the tree Repeat until the tree by Prim’s algorithm is restored

Tong Wang UMass Boston CS 310 July 20, 2017 9 / 34

Naive Implementation of Prim’s Algorithm

Prim’s algorithm grows the MST in stages, one vertex at a time Thus n − 1 iterations During each iteration, search all edges and find the minimum weight edge with one vertex in the tree and the other vertex not in the tree Thus each iteration takes O(m) time O(m ∗ n) total time

Tong Wang UMass Boston CS 310 July 20, 2017 10 / 34

Better Implementation of Prim’s Algorithm

Maintain an array Dist that stores the distances from nontree nodes to the nearest tree nodes Initialize Dist to MAXINT When vertex u is added to the tree, loop through its adjacency list Consider edge (u, v)

If v is already a tree node, ignore it If v is not a tree node, and if Cost(u, v) < Dist[v]

Dist[v] = Cost(u, v) parent[v] = u

After updating Dist, find the node with the minimum distance to be the next tree node O(n2) total time Further improvement: Use a priority queue, so that the node with the minimum distance can be found in O(log n) time, for a total time of O(m log n)

Tong Wang UMass Boston CS 310 July 20, 2017 11 / 34

Kruskal’s algorithm

This algorithm works like building the Huffman code tree Start with all the nodes separate and then join them incrementally, using a greedy approach that turns out to give the optimal solution Set up a partition, a set of sets of nodes, starting with one node in each set Then find the minimal edge to join two sets, and use that as a tree edge, and join the sets Each set in the partition is a connected component

Tong Wang UMass Boston CS 310 July 20, 2017 12 / 34

Example

v2 v0 v5 v1 v6 v4 v3 4 5 2 1 6 10 1 3 2 2 8 4 v2 v0 v5 v1 v6 v4 v3 2 1 1 2 2 4

Tong Wang UMass Boston CS 310 July 20, 2017 13 / 34

Example

1

v2 v0 v5 v1 v6 v4 v3 1

3

v2 v0 v5 v1 v6 v4 v3 2 1 1

2

v2 v0 v5 v1 v6 v4 v3 1 1

4

v2 v0 v5 v1 v6 v4 v3 2 1 1 2

Tong Wang UMass Boston CS 310 July 20, 2017 14 / 34

Example

5

v2 v0 v5 v1 v6 v4 v3 2 1 1 2 2

7

v2 v0 v5 v1 v6 v4 v3 4 2 1 1 2 2

6

v2 v0 v5 v1 v6 v4 v3 2 1 1 3 2 2

8

v2 v0 v5 v1 v6 v4 v3 2 1 1 2 2 4

Tong Wang UMass Boston CS 310 July 20, 2017 15 / 34

Running Example

The component numbers for the first few steps: Node V0 V1 V2 V3 V4 V5 V6 Set 1 2 3 4 5 6 V0-V3 1 2 4 5 6 (3 joins 0) V5-V6 1 2 4 5 5 (6 joins 5) V0-V1 2 4 5 5 V2-V3 4 5 5 and so on Edges are used in cost order If an edge has both to and from vertices with the same component number, it is skipped The resulting tree is defined by the edges used

Tong Wang UMass Boston CS 310 July 20, 2017 16 / 34

Correctness of Kruskal’s Algorithm

The algorithm adds n − 1 edges without creating a cycle Thus it must create a spanning tree Why does the tree has the minimal weight? Proof by contradiction Assume Kruskal’s algorithm does not return an MST There is a real MST Tmin Then there is an edge (x, y) in the tree by Kruskal’s that is not in Tmin Insert (x, y) in Tmin and create a cycle with the path P from x to y When Kruskal’s adds the edge (x, y), x and y are in different components That is, one of the edges in path P is not there yet; let it be (v1, v2) This means Cost(x, y) ≤ Cost(v1, v2)

Tong Wang UMass Boston CS 310 July 20, 2017 17 / 34

Correctness of Kruskal’s Algorithm

If we add (x, y) to Tmin, a cycle is formed The edge (v1, v2) is added after (x, y) Thus Cost(x, y) ≤ Cost(v1, v2) By replacing (v1, v2) with (x, y), we would not increase the cost Repeat until Tmin is transformed into the tree by Kruskal’s

Tong Wang UMass Boston CS 310 July 20, 2017 18 / 34

Kruskal’s Algorithm

Note that several edges are left unprocessed in the edge list

These are the high-cost edges we want to avoid using

Pseudocode: Put the edges in a priority queue ordered by edge weight Initialize the tree edge set T to the empty set Repeat n − 1 times

Get next edge (u, v) from PQ If component[u] != component[v]

Add (u, v) to T Merge component[u] and component[v]

In each iteration, test the connectivity of two trees plus an edge This can be done with BFS or DFS on a sparse graph with at most n edges and n vertices. Thus each iteration takes O(n) time O(m ∗ n) total time If we can determine the connectivity of two trees plus an edge in O(log n) time, then the total time for Kruskal’s algorithm is O(m log n)

Tong Wang UMass Boston CS 310 July 20, 2017 19 / 34

Data Structures for Kruskal’s Algorithm

Need two operations

1

SameComponent(u, v): returns true or false

2

MergeComponent(C1, C2): returns the merged component

Best data structure: Union-Find

Tong Wang UMass Boston CS 310 July 20, 2017 20 / 34

The Union-Find Data Structure

Let n be the number of elements in a set A partition divides the elements into a collection of disjoint subsets Examples: SCC, vertex coloring, vertices in Kruskal’s algorithm Each subset in the partition is represented by one of its elements The operation Find(u) returns the representative of the subset that u belongs to The operation Union(u, v) joins the subsets containing u and v into a new subset

Tong Wang UMass Boston CS 310 July 20, 2017 21 / 34

The Union-Find Data Structure

vertex 1 2 3 4 5 6 7 parent 1 4 3 4 3 4 2 rank 1 1 2 size 1 2 2 4 1 1 1

find(x) if (parent(x) == x) return x else return find(parent(x))

Each subset is represented by a “backwards” tree, where each child node has a pointer to its parent The rank of a tree is (without path compression) its height Initially, each vertex of the graph is in its own tree, and its parent pointer points to itself Find(i): Follow the parent pointers upwards until reaching a node that points to itself; return the label of this node as the label of the subset Union(i, j): Use the root of the higher-ranked tree as the root of the union – it becomes the parent of the root of the lower-ranked tree

Tong Wang UMass Boston CS 310 July 20, 2017 22 / 34

Runtime Analysis of Union-Find

Runtime of Union and Find depends on the heights of the trees When two trees of rank r are merged, the rank of the resulting tree is r + 1

This is the only way to increase the rank

How high can the trees become?

The smallest tree of rank 0 has 1 node The smallest tree of rank 1 has 2 nodes

Merging a rank-0 tree to a rank-1 tree is still a rank-1 tree Merging two rank-1 trees gives a rank-2 tree

The smallest tree of rank 2 has 4 nodes The smallest tree of rank k has 2k nodes – by induction The maximum height is O(log n)

O(log n) time for union or find

Tong Wang UMass Boston CS 310 July 20, 2017 23 / 34

Union-Find with Path Compression

Path compression

find(x) if (parent(x) != x) parent(x) = find(parent(x)) return parent(x)

Amortized runtime per operation of union-find with path compression is O(1)

Tong Wang UMass Boston CS 310 July 20, 2017 24 / 34

Back to Kruskal’s Algorithm

Initialize all vertices as singleton trees for Union-Find Keep all edges in a priority queue Loop through the edges out of PQ (at most m iterations)

For edge (u, v), use Union-Find to determine whether u and v are in separate trees – O(log n) time If u and v are in the same tree, move on to the next edge If not, join the two trees

Total runtime O(m log n)

Tong Wang UMass Boston CS 310 July 20, 2017 25 / 34

Unweighted Single-Source Shortest-Path Problem

Find the shortest path (measured by number of edges) from a designated vertex s to every vertex A special case of the weighted shortest-path problem by giving weight 1 to all edges This problem has an efficient solution: BFS The problem becomes more complicated when (different) weights are allowed

Tong Wang UMass Boston CS 310 July 20, 2017 26 / 34

Positive-Weighted, Single-Source, Shortest-Path Problem

Many times the edges have weights – indicating that the relations between nodes are not always equal Example: distances between cities In this case BFS does not work Goal: Find the shortest path (measured by total cost) from a designated vertex s to every vertex in a weighted graph Weights (costs) are assigned to edges All edge costs are positive

Tong Wang UMass Boston CS 310 July 20, 2017 27 / 34

Dijkstra’s Algorithm

A greedy technique that works well Let weight(u, v) be the cost for edge (u, v) If the shortest path from s to t passes through x, then the path from s to x must be shortest as well Starting from vertex s, we set up an array to hold all the running min costs Di from s to vertex i Being greedy, we visit a new node v with the least cost from s that has not been visited yet Then we update the min cost Di by passing through v

Di = min{Di, Dv + weight(v, i)}

Repeat by visiting the next node with the lowest cost Di that has not been visited

Tong Wang UMass Boston CS 310 July 20, 2017 28 / 34

Running Example, Starting From V0

1

v2 v0 v5 v1 v6 v4 v3 4 5 2 1 6 10 1 3 2 2 8 4

3

3

v2 v0

9

v5

2

v1

5

v6

3

v4

1

v3 4 5 2 1 6 10 1 3 2 2 8 4

2

v2 v0 v5

2

v1 v6 v4

1

v3 4 5 2 1 6 10 1 3 2 2 8 4

4

3

v2 v0

9

v5

2

v1

5

v6

3

v4

1

v3 4 5 2 1 6 10 1 3 2 2 8 4

Tong Wang UMass Boston CS 310 July 20, 2017 29 / 34

Running Example, Starting From V0

5

3

v2 v0

9

v5

2

v1

5

v6

3

v4

1

v3 4 5 2 1 6 10 1 3 2 2 8 4

7

3

v2 v0

6

v5

2

v1

5

v6

3

v4

1

v3 4 5 2 1 6 10 1 3 2 2 8 4

6

3

v2 v0

8

v5

2

v1

5

v6

3

v4

1

v3 4 5 2 1 6 10 1 3 2 2 8 4

8

3

v2 v0

6

v5

2

v1

5

v6

3

v4

1

v3 4 5 2 1 6 10 1 3 2 2 8 4

Tong Wang UMass Boston CS 310 July 20, 2017 30 / 34

Path with Minimum Weights from V0

v2 v0 v5 v1 v6 v4 v3 4 5 2 1 6 10 1 3 2 2 8 4

visiting D0 D1 D2 D3 D4 D5 D6 start ∞ ∞ ∞ ∞ ∞ ∞ V0 2 ∞ 1 ∞ ∞ ∞ V3 2 3 3 9 5 V1 3 3 9 5 V4 3 9 5 V2 8 5 V6 6 V5 final: 2 3 1 3 6 5

Tong Wang UMass Boston CS 310 July 20, 2017 31 / 34

Pseudocode for Dijkstra’s Algorithm

Input: a directed graph G and vertex s Di: an array of distances from s to each node Initialize Di = ∞ For each edge (s, i) in the adjacency list of s Set Di = weight(s, i) Initialize tree T = { s } Repeat until all nodes are in T Choose a node v not in T that minimizes Dv Add v to T For each edge (v, w) such that w ∈ T Dw = min(Dw, Dv + weight(v, w))

Tong Wang UMass Boston CS 310 July 20, 2017 32 / 34

Prim and Dijkstra Are Very Similar

Prim’s Algorithm: Choose an arbitrary node s Initialize tree T = { s } Initialize E = { } Repeat until all nodes are in T: Choose an edge (u, v) with minimum weight such that u is in T and v is not Add v to T, add (u, v) to E Output (T, E) Dijkstra’s Algorithm: Input: a directed graph G and source node s Di: an array of distances from s to each node Initialize Di = ∞ For each edge (s, i) in the adjacency list of s Set Di = weight(s, i) Initialize tree T = { s } Repeat until all nodes are in T Choose a node v not in T that minimizes Dv Add v to T For each edge (v, w) such that w ∈ T Dw = min(Dw, Dv + weight(v, w))

Tong Wang UMass Boston CS 310 July 20, 2017 33 / 34

Correctness of Dijkstra

The argument of the correctness of Dijkstra’s algorithm is similar to that of Prim’s algorithm By contradiction

Assume there is a shorter path An edge (u, v) in Dijkstra’s tree is absent in the shorter path Add (u, v) and get a cycle One of the edges in the cycle is not there when (u, v) is added by Dijkstra (u, v) is no worse than this edge

Tong Wang UMass Boston CS 310 July 20, 2017 34 / 34