CPSC 490: Problem Solving in Computer Science Assignment 1 is due - - PowerPoint PPT Presentation

CPSC 490: Problem Solving in Computer Science

Lecture 2: Graph Traversals, Shortest Paths

Henry Xia, Brandon Zhang

based on CPSC 490 slides from 2014-2018

2019-01-08

University of British Columbia

Announcements

• Assignment 1 is due Jan 15 at noon.
• Sign up for Piazza! http://piazza.com/ubc.ca/winterterm22018/cpsc490
• Announcements and assignment hints will be posted on Piazza.
• If you don’t have a judge account or forgot your password, make a private post on

Piazza.

• Offjce hours this week: Mon, Wed, Thu 2-3 pm, Fri 1-2 pm in ICICS 005.

1

What are graphs?

• Represent relationships with nodes (vertices) and edges.
• Notation: Call the set of nodes V, and the set of edges E.
• Each edge connects some pair of nodes, edges can be
• Bidirectional (undirected)
• One-directional (directed)
• Weighted
• A tree is a connected graph with no cycles.

Figure 1: An undirected graph (Source: Wikipedia)

2

Why study graphs?

Tons of problems we can solve in terms of graphs!

• Shortest path problems
• Dependency/connectivity problems
• Cycle detection
• Minimum cost substructures (trees, matchings, etc.)
• and many many more...

Figure 2: Shortest path problems have a natural representation in graphs.

3

Graph representations

Adjacency list

• adj[u] is a list of neighbours of u
• Advantages: memory effjcient, easy & fast to iterate through neighbours
• Disadvantages: slow to check if (u, v) is an edge
• Can use a set instead of a list, but slower

Adjacency matrix

• adj[u][v] is true if there is an edge u

v

• Advantages: good for dense graphs, easy to test if u v is an edge
• Disadvantages: slow to find neighbours, not effjcient for sparse graphs

An undirected edge u v is represented as two directed edges u v, v u. In almost all problems, adjacency lists are the best representation to use.

4

Graph representations

Adjacency list

• adj[u] is a list of neighbours of u
• Advantages: memory effjcient, easy & fast to iterate through neighbours
• Disadvantages: slow to check if (u, v) is an edge
• Can use a set instead of a list, but slower

Adjacency matrix

• adj[u][v] is true if there is an edge u → v
• Advantages: good for dense graphs, easy to test if (u, v) is an edge
• Disadvantages: slow to find neighbours, not effjcient for sparse graphs

An undirected edge u v is represented as two directed edges u v, v u. In almost all problems, adjacency lists are the best representation to use.

4

Graph representations

Adjacency list

• adj[u] is a list of neighbours of u
• Advantages: memory effjcient, easy & fast to iterate through neighbours
• Disadvantages: slow to check if (u, v) is an edge
• Can use a set instead of a list, but slower

Adjacency matrix

• adj[u][v] is true if there is an edge u → v
• Advantages: good for dense graphs, easy to test if (u, v) is an edge
• Disadvantages: slow to find neighbours, not effjcient for sparse graphs

An undirected edge (u, v) is represented as two directed edges u → v, v → u. In almost all problems, adjacency lists are the best representation to use.

4

Graph representations

Adjacency list

• adj[u] is a list of neighbours of u
• Advantages: memory effjcient, easy & fast to iterate through neighbours
• Disadvantages: slow to check if (u, v) is an edge
• Can use a set instead of a list, but slower

Adjacency matrix

• adj[u][v] is true if there is an edge u → v
• Advantages: good for dense graphs, easy to test if (u, v) is an edge
• Disadvantages: slow to find neighbours, not effjcient for sparse graphs

An undirected edge (u, v) is represented as two directed edges u → v, v → u. In almost all problems, adjacency lists are the best representation to use.

4

Problem 1 – Connectivity

Input: a graph with N nodes and M edges. Output: true if node a and node b are connected, false otherwise. a b a b

5

Problem 1 – Solution Attempt

a and b are connected ⇐ ⇒ a = b, or there is a neighbour v of a such that v and b are connected. This suggests a recursive solution!

1

def connected(a, b):

2

if a == b:

3

return True

4

for v in neighbours of a:

5

if connected(v, b):

6

return True

7

return False

What’s wrong with this?

• too slow! It checks the same node multiple times.
• might never terminate if the graph has a cycle.

6

Problem 1 – Solution Attempt

a and b are connected ⇐ ⇒ a = b, or there is a neighbour v of a such that v and b are connected. This suggests a recursive solution!

1

def connected(a, b):

2

if a == b:

3

return True

4

for v in neighbours of a:

5

if connected(v, b):

6

return True

7

return False

What’s wrong with this?

• too slow! It checks the same node multiple times.
• might never terminate if the graph has a cycle.

6

Problem 1 – Solution Attempt

a and b are connected ⇐ ⇒ a = b, or there is a neighbour v of a such that v and b are connected. This suggests a recursive solution!

1

def connected(a, b):

2

if a == b:

3

return True

4

for v in neighbours of a:

5

if connected(v, b):

6

return True

7

return False

What’s wrong with this?

• too slow! It checks the same node multiple times.
• might never terminate if the graph has a cycle.

6

Problem 1 – Solution Attempt

a and b are connected ⇐ ⇒ a = b, or there is a neighbour v of a such that v and b are connected. This suggests a recursive solution!

1

def connected(a, b):

2

if a == b:

3

return True

4

for v in neighbours of a:

5

if connected(v, b):

6

return True

7

return False

What’s wrong with this?

• too slow! It checks the same node multiple times.
• might never terminate if the graph has a cycle.

6

Problem 1 – Solution Attempt 2

1

def connected(a, b):

2

if a == b:

3

return True

4

if a is visited:

5

return False

6

mark a as visited

7

for v in neighbours of a:

8

if connected(v, b):

9

return True

10

return False

This is better! We only process each node at most once. This is depth-first-search (DFS). However, this implementation might cause a stack

• verflow due to the recursive calls.

7

Problem 1 – Solution Attempt 2

1

def connected(a, b):

2

if a == b:

3

return True

4

if a is visited:

5

return False

6

mark a as visited

7

for v in neighbours of a:

8

if connected(v, b):

9

return True

10

return False

This is better! We only process each node at most once. This is depth-first-search (DFS). However, this implementation might cause a stack

• verflow due to the recursive calls.

7

Problem 1 – Solution Attempt 2

1

def connected(a, b):

2

if a == b:

3

return True

4

if a is visited:

5

return False

6

mark a as visited

7

for v in neighbours of a:

8

if connected(v, b):

9

return True

10

return False

This is better! We only process each node at most once. This is depth-first-search (DFS). However, this implementation might cause a stack

• verflow due to the recursive calls.

7

Problem 1 – Solution Attempt 3

1

def connected(a, b):

2

s = new stack containing a

3

while s is not empty:

4

u = pop(s)

5

if u == b:

6

return True

7

if u is visited:

8

continue

9

mark u as visited

10

for v in neighbours of u:

11

push(s, v)

12

return False

We simulate the call stack with an actual stack.

8

Depth-first search (DFS)

Graph traversal algorithm which starts at a given node, explores as deep as possible, and backtracks when it hits a dead end.

• Time complexity: O(|V| + |E|)

Things to consider for a DFS problem:

• What needs to be done when ...
• visiting a new node?
• backtracking?
• visiting an already visited node?
• visiting the target node?
• Do you need “shortest path”? If so, DFS is not the answer!

9

Breadth-first search (BFS)

What happens when we replace the stack in DFS with a queue? Obtain breadth-first search: explores nodes “level by level”, visiting all nodes at a certain distance from the source before moving on.

• Time complexity: O(|V| + |E|)

10

Problem 2 – Shortest path, unweighted

Input: a connected graph with N nodes and M edges, and a source node S. Output: for each v ∈ V, the distance from v to S. S 1 1 2 3 2 3 4

11

Problem 2 – Solution

Run BFS!

1

q = new queue containing (S, 0)

2

while q is not empty:

3

(u, cost) = dequeue(q)

4

if u is visited:

5

continue

6

mark u as visited

7

dist[u] = cost

8

for v in neighbours of u:

9

enqueue(q, (v, cost + 1))

10

return dist

Try implementing it without a visited array!

12

Problem 2 – Solution

Run BFS!

1

q = new queue containing (S, 0)

2

while q is not empty:

3

(u, cost) = dequeue(q)

4

if u is visited:

5

continue

6

mark u as visited

7

dist[u] = cost

8

for v in neighbours of u:

9

enqueue(q, (v, cost + 1))

10

return dist

Try implementing it without a visited array!

12

Problem 3 – Grid Maze

Input: a rectangular grid maze with obstacles, an entrance, and an exit. Output: the length of the shortest path between the entrance and exit. S . . . . . . X . . X . . . X . . E Figure: An example of a grid maze

13

Problem 3 – Solution

Run BFS!

14

Problem 4 – Grid Maze (Modified)

Input: a rectangular grid maze with obstacles, an entrance, and an exit. Output: the cells in the grid which are not on any shortest path from entrance to exit. S . . . . .

• X . . X .
• o X . . E

Figure: The same grid maze, with o marking cells not on any shortest path.

15

Problem 4 – Solution

• Let k be the distance between the entrance and exit.
• Let u be a node which has distance x to the entrance.
• If u is on a shortest path, then u should have distance k − x to the exit.

Solution:

• Run BFS from the entrance.
• Run BFS from the exit.
• Output any node which doesn’t have the above property.

16

Problem 4 – Solution

• Let k be the distance between the entrance and exit.
• Let u be a node which has distance x to the entrance.
• If u is on a shortest path, then u should have distance k − x to the exit.

Solution:

• Run BFS from the entrance.
• Run BFS from the exit.
• Output any node which doesn’t have the above property.

16

Problem 5 – Harder Grid Maze!

Input: a rectangular grid maze with obstacles, multiple entrances, and multiple exits. Output: the shortest distance between any entrance and exit. S . . . . S . X . . X . . E X . . E

17

Problem 5 – Solution

• Create new vertices S′, T′.
• Connect S′ to all the entrances, and T′ to all the exits.
• Use BFS to find the distance L from S′ to T′.
• Answer = L − 2

Time complexity: O(|V| + |E|) This is called a multi-source/multi-sink BFS. Note: no need to explicitly create S , T in an implementation.

18

Problem 5 – Solution

• Create new vertices S′, T′.
• Connect S′ to all the entrances, and T′ to all the exits.
• Use BFS to find the distance L from S′ to T′.
• Answer = L − 2

Time complexity: O(|V| + |E|) This is called a multi-source/multi-sink BFS. Note: no need to explicitly create S′, T′ in an implementation.

18

0-1 BFS

Why can we use BFS to compute shortest paths? Because we visit nodes in increasing

• rder of distance from the source.

We can generalize this idea to weighted graphs where edge weights are 0 and 1 only.

• Idea: prioritize visiting neighbours with weight-0 edges over neighbours with

weight-1 edges.

• Use a double-ended queue.
• For weight-0 edges, push the neighbour to the front of the queue. For weight-1

edges, push to the back. Time complexity: O V E

19

0-1 BFS

Why can we use BFS to compute shortest paths? Because we visit nodes in increasing

• rder of distance from the source.

We can generalize this idea to weighted graphs where edge weights are 0 and 1 only.

• Idea: prioritize visiting neighbours with weight-0 edges over neighbours with

weight-1 edges.

• Use a double-ended queue.
• For weight-0 edges, push the neighbour to the front of the queue. For weight-1

edges, push to the back. Time complexity: O(|V| + |E|)

19

Dijkstra’s algorithm

What about a graph with non-negative edge weights?

• Just need to keep the queue sorted by distance from source.
• A priority queue lets us do this. This gives Dijkstra’s algorithm.

How to implement Dijkstra’s? Just replace the queue in BFS with a priority queue, sorted by distance! Time complexity: O E V (can be improved to O E V V )

20

Dijkstra’s algorithm

What about a graph with non-negative edge weights?

• Just need to keep the queue sorted by distance from source.
• A priority queue lets us do this. This gives Dijkstra’s algorithm.

How to implement Dijkstra’s? Just replace the queue in BFS with a priority queue, sorted by distance! Time complexity: O E V (can be improved to O E V V )

20

Dijkstra’s algorithm

What about a graph with non-negative edge weights?

• Just need to keep the queue sorted by distance from source.
• A priority queue lets us do this. This gives Dijkstra’s algorithm.

How to implement Dijkstra’s? Just replace the queue in BFS with a priority queue, sorted by distance! Time complexity: O(|E| log |V|) (can be improved to O(|E| + |V| log |V|))

20

Problem 6 – Odd Path

Input: A graph G with non-negative edge weights, and nodes s, e. Output: The length of the shortest walk from s to e using an odd number of edges.

21

Problem 6 – Solution

• Make 2 copies of the graph, G1 and G2
• G1 = even number of edges used, G2 = odd number of edges used
• For each edge u → v in G, make edges u1 → v2, u2 → v1
• Answer: shortest walk from s1 to e2

22

Problem 7 – Minimum Edge Count

Input: A graph G with non-negative integer weights, and nodes s, e. Output: The minimum possible number of edges on a shortest path from s to e.

23

Problem 7 – Solution

• Idea: encode both the path weight and number of edges in the path length
• Let X = |V| + 1. For each edge weight wi, replace wi by X · wi + 1
• Run Dijkstra from s to e to get distance d
• Path weight: ⌊ d

X⌋

• Number of edges: d mod X

What if we want to get the shortest path length over all minimum-edge paths?

24

Problem 7 – Solution

• Idea: encode both the path weight and number of edges in the path length
• Let X = |V| + 1. For each edge weight wi, replace wi by X · wi + 1
• Run Dijkstra from s to e to get distance d
• Path weight: ⌊ d

X⌋

• Number of edges: d mod X

What if we want to get the shortest path length over all minimum-edge paths?

24

Negative edge weights

What’s the shortest path from A to C? What’s the output of Dijkstra’s algorithm? Dijkstra doesn’t work on graphs with negative edge weights!

25

Negative edge weights

What’s the shortest path from A to C? What’s the output of Dijkstra’s algorithm? Dijkstra doesn’t work on graphs with negative edge weights!

25

Bellman-Ford algorithm

Calculates the shortest path from source s to every other node. Idea: continually “relax” the best distance found so far.

1

dist = array of V elements, initialized to ∞

2

dist[s] = 0

3

repeat V - 1 times:

4

for each edge u → v with cost c:

5

if dist[v] > dist[u] + c:

6

dist[v] = dist[u] + c

Invariant: aħter the ith iteration, dist[u] stores the shortest path from s to u using at most i edges. If there are no negative-weight cycles, any shortest path will have V 1 edges, so we

• nly need to run the loop V

1 times. Time complexity: O V E (can improve average case; see notes on SPFA)

26

Bellman-Ford algorithm

Calculates the shortest path from source s to every other node. Idea: continually “relax” the best distance found so far.

1

dist = array of V elements, initialized to ∞

2

dist[s] = 0

3

repeat V - 1 times:

4

for each edge u → v with cost c:

5

if dist[v] > dist[u] + c:

6

dist[v] = dist[u] + c

Invariant: aħter the ith iteration, dist[u] stores the shortest path from s to u using at most i edges. If there are no negative-weight cycles, any shortest path will have |V| − 1 edges, so we

• nly need to run the loop |V| − 1 times.

Time complexity: O V E (can improve average case; see notes on SPFA)

26

Bellman-Ford algorithm

Calculates the shortest path from source s to every other node. Idea: continually “relax” the best distance found so far.

1

dist = array of V elements, initialized to ∞

2

dist[s] = 0

3

repeat V - 1 times:

4

for each edge u → v with cost c:

5

if dist[v] > dist[u] + c:

6

dist[v] = dist[u] + c

Invariant: aħter the ith iteration, dist[u] stores the shortest path from s to u using at most i edges. If there are no negative-weight cycles, any shortest path will have |V| − 1 edges, so we

• nly need to run the loop |V| − 1 times.

Time complexity: O(|V||E|) (can improve average case; see notes on SPFA)

26

Negative weight cycles

What’s the shortest path from s to e? s p q r e 2 1 2

• 4

3 How do we detect negative weight cycles?

27

Negative weight cycles

What’s the shortest path from s to e? s p q r e 2 1 2

• 4

3 How do we detect negative weight cycles?

27

Problem 8 – Arbitrage

Suppose we can trade currencies at the following rates:

• 1 CAD = 0.85 USD
• 1 USD = 0.95 EUR
• 1 EUR = 1.45 CAD

We can get rich! Trade 1 CAD → 0.85 USD → 0.81 EUR → 1.17 CAD.

28

Problem 8 – Arbitrage

Input: A set of n ≤ 100 currencies and their exchange rates. Output: Determine if we can make infinite money via trades.

29

Problem 8 – Solution

Let e(x, y) be the exchange rate from currency x to currency y. If we can make infinite money, then there exist currencies c1, . . . , ck = c1 such that:

k−1

∏

i=1

e(ci, ci+1) > 1 ⇐ ⇒

k−1

∑

i=1

log e(ci, ci+1) > 0 ⇐ ⇒

k−1

∑

i=1

− log e(ci, ci+1) < 0 Create a graph with the currencies as nodes and edges u v with weight e u v . Determine if there exists a negative weight cycle.

30

Problem 8 – Solution

Let e(x, y) be the exchange rate from currency x to currency y. If we can make infinite money, then there exist currencies c1, . . . , ck = c1 such that:

k−1

∏

i=1

e(ci, ci+1) > 1 ⇐ ⇒

k−1

∑

i=1

log e(ci, ci+1) > 0 ⇐ ⇒

k−1

∑

i=1

− log e(ci, ci+1) < 0 Create a graph with the currencies as nodes and edges u → v with weight − log e(u, v). Determine if there exists a negative weight cycle.

30

Lots of paths

Using Dijkstra/Bellman-Ford, we can find shortest paths from s to all other nodes. What if we want the shortest path between all pairs of vertices? Running Dijkstra or Bellman-Ford |V| times takes O(|V||E| log |V|) or O(|V|2|E|) time

• respectively. Slow for dense graphs!

31

Floyd-Warshall algorithm

Idea: Successively relax the distance i → j using paths i → k, k → j.

1

dist = V x V table, initialized to ∞

2

for u = 1 .. V: dist[u][u] = 0

3

for each edge u → v with cost c: dist[u][v] = c

4 5

for k = 1 .. V:

6

for i = 1 .. V:

7

for j = 1 .. V:

8

dist[i][j] = min(dist[i][j], dist[i][k] + dist[k][j])

Time complexity: O(|V|3). Fast in practice due to small constant factor. Watch out: the order of the loops matters! Works for directed/undirected, non-negative/negative edge weights. How to detect negative weight cycles?

32