[PPT] - Graphs Carola Wenk Slides courtesy of Charles Leiserson with PowerPoint Presentation

SLIDE 1

CS 5633 Analysis of Algorithms 1 3/25/10

CS 5633 -- Spring 2010

Graphs

Carola Wenk Slides courtesy of Charles Leiserson with changes and additions by Carola Wenk

SLIDE 2

CS 5633 Analysis of Algorithms 2 3/25/10

Graphs (review)

Definition. A directed graph (digraph)

G = (V, E) is an ordered pair consisting of

a set V of vertices (singular: vertex),
a set E ⊆ V × V of edges.

In an undirected graph G = (V, E), the edge set E consists of unordered pairs of vertices. In either case, we have |E| = O(|V| 2). Moreover, if G is connected, then |E| ≥ |V| – 1. (Review CLRS, Appendix B.4 and B.5.)

SLIDE 3

CS 5633 Analysis of Algorithms 3 3/25/10

Adjacency-matrix representation

The adjacency matrix of a graph G = (V, E), where V = {1, 2, …, n}, is the matrix A[1 . . n, 1 . . n] given by A[i, j] = 1 if (i, j) ∈ E, 0 if (i, j) ∉ E. 2 2 1 1 3 3 4 4 A 1 2 3 4 1 2 3 4 1 1 1 1 Θ(|V| 2) storage ⇒ dense representation.

SLIDE 4

CS 5633 Analysis of Algorithms 4 3/25/10

Adjacency-list representation

An adjacency list of a vertex v ∈ V is the list Adj[v]

f vertices adjacent to v.

2 2 1 1 3 3 4 4

Adj[1] = {2, 3} Adj[2] = {3} Adj[3] = {} Adj[4] = {3}

For undirected graphs, |Adj[v]| = degree(v). For digraphs, | Adj[v] | = out-degree(v).

SLIDE 5

CS 5633 Analysis of Algorithms 5 3/25/10

Adjacency-list representation

Handshaking Lemma:

Every edge is counted twice

For undirected graphs:

∑v∈V degree(v) = 2|E|

For digraphs:

∑v∈V in-degree(v) + ∑v∈V out-degree(v) = 2 | E | ⇒ adjacency lists use Θ(|V| + |E|) storage ⇒ a sparse representation ⇒ We usually use this representation, unless stated otherwise

SLIDE 6

CS 5633 Analysis of Algorithms 6 3/25/10

Graph Traversal

Let G=(V,E) be a (directed or undirected) graph, given in adjacency list representation. |V| = n , |E| = m A graph traversal visits every vertex:

Breadth-first search (BFS)
Depth-first search (DFS)

SLIDE 7

CS 5633 Analysis of Algorithms 7 3/25/10

Breadth-First Search (BFS)

BFS(G=(V,E)) Mark all vertices in G as “unvisited” // time=0 Initialize empty queue Q for each vertex v ∈ V do if v is unvisited visit v // time++ Q.enqueue(v) BFS_iter(G) BFS_iter(G) while Q is non-empty do v = Q.dequeue() for each w adjacent to v do if w is unvisited visit w // time++ Add edge (v,w) to T Q.enqueue(w)

SLIDE 8

CS 5633 Analysis of Algorithms 8 3/25/10

Example of breadth-first search

a a b b c c d d e e g g i i f f h h Q:

SLIDE 9

CS 5633 Analysis of Algorithms 9 3/25/10

Example of breadth-first search

a a b b c c d d e e g g i i f f h h Q: a

SLIDE 10

CS 5633 Analysis of Algorithms 10 3/25/10

Example of breadth-first search

a a b b c c d d e e g g i i f f h h Q: a b d

1 2 1 2

SLIDE 11

CS 5633 Analysis of Algorithms 11 3/25/10

Example of breadth-first search

a a b b c c d d e e g g i i f f h h Q: a b d c e

1 2 3 4 2 3 4

SLIDE 12

CS 5633 Analysis of Algorithms 12 3/25/10

Example of breadth-first search

a a b b c c d d e e g g i i f f h h Q: a b d c e

1 2 3 4 3 4

SLIDE 13

CS 5633 Analysis of Algorithms 13 3/25/10

Example of breadth-first search

a a b b c c d d e e g g i i f f h h Q: a b d c e

1 2 3 4 4

SLIDE 14

CS 5633 Analysis of Algorithms 14 3/25/10

Example of breadth-first search

a a b b c c d d e e g g i i f f h h Q: a b d c e g i

1 2 3 4 5 6 5 6

SLIDE 15

CS 5633 Analysis of Algorithms 15 3/25/10

Example of breadth-first search

a a b b c c d d e e g g i i f f h h Q: a b d c e g i f

1 2 3 4 5 6 7 6 7

SLIDE 16

CS 5633 Analysis of Algorithms 16 3/25/10

Example of breadth-first search

a a b b c c d d e e g g i i f f h h Q: a b d c e g i f h

1 2 3 4 5 6 7 8 7 8

a a

SLIDE 17

CS 5633 Analysis of Algorithms 17 3/25/10

Example of breadth-first search

a a b b c c d d e e g g i i f f h h Q: a b d c e g i f h

1 2 3 4 5 6 7 8 8

a a

SLIDE 18

CS 5633 Analysis of Algorithms 18 3/25/10

Example of breadth-first search

a a b b c c d d e e g g i i f f h h Q: a b d c e g i f h

1 2 3 4 5 6 7 8

a a

SLIDE 19

CS 5633 Analysis of Algorithms 19 3/25/10

Example of breadth-first search

a a b b c c d d e e g g i i f f h h Q: a b d c e g i f h

1 2 3 4 5 6 7 8

a a

SLIDE 20

CS 5633 Analysis of Algorithms 20 3/25/10

Breadth-First Search (BFS)

BFS(G=(V,E)) Mark all vertices in G as “unvisited” // time=0 Initialize empty queue Q for each vertex v ∈ V do if v is unvisited visit v // time++ Q.enqueue(v) BFS_iter(G) BFS_iter(G) while Q is non-empty do v = Q.dequeue() for each w adjacent to v do if w is unvisited visit w // time++ Add edge (v,w) to T Q.enqueue(w) O(n) O(1) O(n)

without BFS_iter

O(deg(v)) O(m)

SLIDE 21

CS 5633 Analysis of Algorithms 21 3/25/10

BFS runtime

Each vertex is marked as unvisited in the beginning ⇒ O(n) time
Each vertex is marked at most once, enqueued at most once,

and therefore dequeued at most once

The time to process a vertex is proportional to the size of its

adjacency list (its degree), since the graph is given in adjacency list representation ⇒ O(m) time

Total runtime is O(n+m) = O(|V| + |E|)

SLIDE 22

CS 5633 Analysis of Algorithms 22 3/25/10

Depth-First Search (DFS)

DFS_rec(G, v) visit v // d[v]=++time for each w adjacent to v do if w is unvisited Add edge (v,w) to tree T DFS_rec(G,w) // f[v]=++time DFS(G=(V,E)) Mark all vertices in G as “unvisited” // time=0 for each vertex v ∈ V do if v is unvisited DFS_rec(G,v)

SLIDE 23

CS 5633 Analysis of Algorithms 23 3/25/10

Example of depth-first search

a a b b c c d d e e g g i i f f h h

0/- d / f

a a π: a b c d e f g h i a

Store edges in

predecessor array

SLIDE 24

CS 5633 Analysis of Algorithms 24 3/25/10

Example of depth-first search

a a b b c c d d e e g g i i f f h h

0/- d / f

π: a b c d e f g h i a

1/-

b

Store edges in

predecessor array

SLIDE 25

CS 5633 Analysis of Algorithms 25 3/25/10

Example of depth-first search

a a b b c c d d e e g g i i f f h h

0/- d / f

π: a b c d e f g h i a

1/-

b

2/- 2/3

Store edges in

predecessor array

SLIDE 26

CS 5633 Analysis of Algorithms 26 3/25/10

Example of depth-first search

a a b b c c d d e e g g i i f f h h

0/- d / f

π: a b c d e f g h i a

1/-

b

2/3

b

Store edges in

predecessor array

SLIDE 27

CS 5633 Analysis of Algorithms 27 3/25/10

Example of depth-first search

a a b b c c d d e e g g i i f f h h

0/- d / f

π: a b c d e f g h i a

1/-

b

2/3 4/-

b e

Store edges in

predecessor array

SLIDE 28

CS 5633 Analysis of Algorithms 28 3/25/10

Example of depth-first search

a a b b c c d d e e g g i i f f h h

0/- d / f

π: a b c d e f g h i a

1/-

b

2/3 4/-

b e

5/-

g

Store edges in

predecessor array

SLIDE 29

CS 5633 Analysis of Algorithms 29 3/25/10

Example of depth-first search

a a b b c c d d e e g g i i f f h h

0/- d / f

π: a b c d e f g h i a

1/-

b

2/3 4/-

b e

5/-

g

6/-

i

Store edges in

predecessor array

SLIDE 30

CS 5633 Analysis of Algorithms 30 3/25/10

Example of depth-first search

a a b b c c d d e e g g i i f f h h

0/- d / f

π: a b c d e f g h i a

1/-

b

2/3 4/-

b e

5/-

g

6/-

i

7/- 7/8

Store edges in

predecessor array

SLIDE 31

CS 5633 Analysis of Algorithms 31 3/25/10

Example of depth-first search

a a b b c c d d e e g g i i f f h h

0/- d / f

π: a b c d e f g h i a

1/-

b

2/3 4/-

b e

5/-

g

6/-

i

7/8 6/9

Store edges in

predecessor array

SLIDE 32

CS 5633 Analysis of Algorithms 32 3/25/10

Example of depth-first search

a a b b c c d d e e g g i i f f h h

0/- d / f

π: a b c d e f g h i a

1/-

b

2/3 4/-

b e

5/-

g i

7/8 6/9

g

Store edges in

predecessor array

SLIDE 33

CS 5633 Analysis of Algorithms 33 3/25/10

Example of depth-first search

a a b b c c d d e e g g i i f f h h

0/- d / f

π: a b c d e f g h i a

1/-

b

2/3 4/-

b e

5/-

g i

7/8 6/9

g

10/-

f

Store edges in

predecessor array

SLIDE 34

CS 5633 Analysis of Algorithms 34 3/25/10

Example of depth-first search

a a b b c c d d e e g g i i f f h h

0/- d / f

π: a b c d e f g h i a

1/-

b

2/3 4/-

b e

5/-

g i

7/8 6/9

g

10/-

f

11/- 11/12

Store edges in

predecessor array

SLIDE 35

CS 5633 Analysis of Algorithms 35 3/25/10

Example of depth-first search

a a b b c c d d e e g g i i f f h h

0/- d / f

π: a b c d e f g h i a

1/-

b

2/3 4/-

b e

5/-

g i

7/8 6/9

g

10/-

f

11/12 10/13

Store edges in

predecessor array

SLIDE 36

CS 5633 Analysis of Algorithms 36 3/25/10

Example of depth-first search

a a b b c c d d e e g g i i f f h h

0/- d / f

π: a b c d e f g h i a

1/-

b

2/3 4/-

b e

5/-

g i

7/8 6/9

g f

11/12 10/13 5/14

Store edges in

predecessor array

SLIDE 37

CS 5633 Analysis of Algorithms 37 3/25/10

Example of depth-first search

a a b b c c d d e e g g i i f f h h

0/- d / f

π: a b c d e f g h i a

1/-

b

2/3 4/-

b e g i

7/8 6/9

g f

11/12 10/13 5/14 4/15

Store edges in

predecessor array

SLIDE 38

CS 5633 Analysis of Algorithms 38 3/25/10

Example of depth-first search

a a b b c c d d e e g g i i f f h h

0/- d / f

π: a b c d e f g h i a

1/-

b

2/3

b e g i

7/8 6/9

g f

11/12 10/13 5/14 4/15 1/16

Store edges in

predecessor array

SLIDE 39

CS 5633 Analysis of Algorithms 39 3/25/10

Example of depth-first search

a a b b c c d d e e g g i i f f h h

0/- d / f

π: a b c d e f g h i a b

2/3

b e g i

7/8 6/9

g f

11/12 10/13 5/14 4/15 1/16 0/17

Store edges in

predecessor array

SLIDE 40

CS 5633 Analysis of Algorithms 40 3/25/10

Depth-First Search (DFS)

DFS_rec(G, v) visit v // d[v]=++time for each w adjacent to v do if w is unvisited Add edge (v,w) to tree T DFS_rec(G,w) // f[v]=++time DFS(G=(V,E)) Mark all vertices in G as “unvisited” // time=0 for each vertex v ∈ V do if v is unvisited DFS_rec(G,v) O(n) O(n)

without DFS_rec

O(deg(v))

without recursive call

O(1) ⇒ With Handshaking Lemma, all recursive calls are O(m), for a total of O(n + m) runtime

SLIDE 41

CS 5633 Analysis of Algorithms 41 3/25/10

DFS runtime

Each vertex is visited at most once ⇒ O(n) time
The body of the for loops (except the recursive call) take constant

time per graph edge

All for loops take O(m) time
Total runtime is O(n+m) = O(|V| + |E|)

SLIDE 42

CS 5633 Analysis of Algorithms 42 3/25/10

DFS edge classification

a a b b c c d d e e g g i i f f h h

d / f 2/3 7/8 6/9 11/12 10/13 5/14 4/15 1/16 0/17

Edge u v is a:

tree edge, if it is part of the depth-first forest.
back edge, if u connects to an ancestor v in a depth-

first tree. It holds d(u)>d(v) and f(u)<f(v).

forward edge, if it connects u to a descendant v in

a depth-first tree. It holds d(u)<d(v) and f(u)>f(v).

cross edge, if it is any other edge. It holds

d(u)>d(v) and f(u)>f(v). c c f f b b b

SLIDE 43

CS 5633 Analysis of Algorithms 43 3/25/10

Paths, Cycles, Connectivity

Let G=(V,E) be a directed (or undirected) graph

A path from v1 to vk in G is a sequence of vertices v1, v2,…,vk such that

(vi,v{i+1})∈E (or {vi,v{i+1}} ∈E if G is undirected) for all i∈{1,…,k-1}.

A path is simple if all vertices in the path are distinct.
A path v1, v2,…,vk forms a cycle if v1=vk and k≥3.
A graph with no cycles is acyclic.
An undirected acyclic graph is called a tree. (Trees do not have to

have a root vertex specified.)

A directed acyclic graph is a DAG. (A DAG can have undirected

cycles if the direction of the edges is not considered.)

An undirected graph is connected if every pair of vertices is connected

by a path. A directed graph is strongly connected if for every pair u,v∈V there is a path from u to v and there is a path from v to u.

The (strongly) connected components of a graph are the equivalence

classes of vertices under this reachability relation.

SLIDE 44

CS 5633 Analysis of Algorithms 44 3/25/10

DAG Theorem

Theorem: A directed graph G is acyclic ⇔ a depth-first search of G yields no back edges. Proof: “⇒”: Suppose there is a back edge (u,v). Then by definition of a back edge there would be a cycle. “⇐”: Suppose G contains a cycle c. Let v be the first vertex to be discovered in c, and let u be the preceding vertex in c. v is an ancestor of u in the depth-first forest, hence (u,v) is a back edge.

u v

SLIDE 45

CS 5633 Analysis of Algorithms 45 3/25/10

Topological Sort

Topologically sort the vertices of a directed acyclic graph (DAG):

Determine f : V → {1, 2, …, |V|} such that (u, v) ∈ E

⇒ f(u) < f(v).

3 3 5 5 6 6 4 4 2 2 7 7 9 9 8 8 1 1 3 3 5 5 6 6 4 4 2 2 7 7 9 9 8 8 1 1

SLIDE 46

CS 5633 Analysis of Algorithms 46 3/25/10

Topological Sort Algorithm

3 3 5 5 7 7 4 4 2 2 6 6 8 8 9 9 1 1

2 2 1 1 3 1 1 1 2 1 1

Store vertices with in-degree 0 in a queue Q.
While Q is not empty
Dequeue vertex v, and give it the next number
Decrease in-degree of all adjacent vertices by 1
Enqueue all vertices with in-degree 0

Q:

a b e d c i h g f

a , b , c , d , e , f , g , i , h

SLIDE 47

CS 5633 Analysis of Algorithms 47 3/25/10

Topological Sort Runtime

Runtime:

O(|V|+|E|) because every edge is touched once, and

every vertex is enqueued and dequeued exactly

nce

SLIDE 48

CS 5633 Analysis of Algorithms 48 3/25/10

Depth-First Search Revisited

DFS_rec(G, v) visit v // d[v]=++time for each w adjacent to v do if w is unvisited Add edge (v,w) to tree T DFS_rec(G,w) // f[v]=++time DFS(G=(V,E)) Mark all vertices in G as “unvisited” // time=0 for each vertex v ∈ V do if v is unvisited DFS_rec(G,v)

SLIDE 49

CS 5633 Analysis of Algorithms 49 3/25/10

DFS-Based Topological Sort Algorithm

Call DFS on the directed acyclic graph G=(V,E)

⇒ Finish time for every vertex

Reverse the finish times (highest finish time

becomes the lowest finish time,…) ⇒ Valid function f ’: V → {1, 2, …, | V |} such that (u, v) ∈ E ⇒ f ’(u) < f’ (v) Runtime: O(|V|+|E|)

SLIDE 50

CS 5633 Analysis of Algorithms 50 3/25/10

DFS-Based Topological Sort

Run DFS:

1 2 3 4 /5 /6 7 8 /9 /10 /11 13 14 15/16 /17 /18 /12

Reverse finish times:

9 8 6 7 5 3 2 1 4

SLIDE 51

CS 5633 Analysis of Algorithms 51 3/25/10

DFS edge classification

a a b b c c d d e e g g i i f f h h

d / f 2/3 7/8 6/9 11/12 10/13 5/14 4/15 1/16 0/17

Edge u v is a:

tree edge, if it is part of the depth-first forest.
back edge, if u connects to an ancestor v in a depth-

first tree. It holds d(u)>d(v) and f(u)<f(v).

forward edge, if it connects u to a descendant v in

a depth-first tree. It holds d(u)<d(v) and f(u)>f(v).

.
cross edge, if it is any other edge. It holds

d(u)>d(v) and f(u)>f(v). c c f f b b b

SLIDE 52

CS 5633 Analysis of Algorithms 52 3/25/10

DFS-Based Top. Sort Correctness

Need to show that for any (u, v) ∈ E holds f (v) < f (u).

(since we consider reversed finish times)

Consider exploring edge (u, v) in DFS:
v cannot be visited and unfinished (and hence an ancestor in

the depth first tree), since then (u,v) would be a back edge (which by the DAG lemma cannot happen).

If v has not been visited yet, it becomes a descendant of u, and

hence f(v)<f(u) (tree edge)

If v has been finished, f(v) has been set, and u is still being

explored, hence f(u)>f(v) (forward edge, cross edge) .

SLIDE 53

CS 5633 Analysis of Algorithms 53 3/25/10

Topological Sort Runtime

Runtime:

O(|V|+|E|) because every edge is touched once, and

every vertex is enqueued and dequeued exactly

nce
DFS-based algorithm: O(|V| + |E|)