CPSC 490 Problem Solving in Computer Science Trees, MST, Euler - - PowerPoint PPT Presentation

CPSC 490 Problem Solving in Computer Science

Trees, MST, Euler Tour, and Backtracking

Lucca Siaudzionis and Jack Spalding-Jamieson 2020/01/16

University of British Columbia

Announcements

• Office hours will be Tuesdays from 2-4pm in ICCS X237.
• After today’s lecture, you will be able to solve all but a few of the A1 problems.
• If you’re having any trouble with the basics (piazza, assignment submissions, IO, etc.),

please ask after class.

1

What is a Tree?

A Tree is a graph with:

• Exactly N − 1 edges (with N nodes)
• A path between every pair of nodes

2

Trees’ Properties

• There is exactly one path between each pair of nodes
• There are no cycles (you can try to prove this!)
• It is sparse (i.e. there are very few edges)

3

Spanning Tree

Given a Graph G, a Spanning Tree is a tree within the original graph that contains all its nodes

4

Minimum/Maximum Spanning Trees

Given a Graph G, a minimum/maximum spanning tree is a tree within the original graph that

• Is a spanning tree
• Sum of the weights is minimized/maximized

5

Minimum Spanning Tree

5

8 10 5 2 4 8

6

Finding Some Spanning Tree

An intuitive way to build a spanning tree:

• Start with an empty tree
• Iterate through all the edges
• Given an edge (x, y)
• If x and y are connected (i.e. there is already a path from x to y), ignore this edge
• Otherwise, add this edge to the tree

7

Kruskal’s Algorithm for MST

Sort the edges by weight

• Ascending weight if Minimum ST
• Descending weight if Maximum ST

Then, do what we did before:

• Start with an empty tree
• Iterate through all the edges in sorted order
• Given an edge (x, y)
• If x and y are connected (i.e. there is already a path from x to y), ignore this edge
• Otherwise, add this edge to the tree

8

Kruskal’s Algorithm Illustrated 3 6 10 7 8 9 12 11

9

Kruskal’s Algorithm Illustrated 3 6 10 7 8 9 12 11

10

Kruskal’s Algorithm Illustrated 3 6 10 7 8 9 12 11

11

Kruskal’s Algorithm Illustrated 3 6 10 7 8 9 12 11

12

Kruskal’s Algorithm Illustrated 3 6 10 7 8 9 12 11

13

Kruskal’s Algorithm Illustrated 3 6 10 7 8 9 12 11

14

Kruskal’s Algorithm Illustrated 3 6 10 7 8 9 12 11

15

Kruskal’s Algorithm Illustrated 3 6 10 7 8 9 12 11

16

Kruskal’s Algorithm Illustrated 3 6 10 7 8 9 12 11

17

Kruskal’s Algorithm Illustrated 3 6 10 7 8 9 12 11

18

Kruskal’s Algorithm Illustrated 3 6 10 7 8 9 12 11

19

Kruskal’s Algorithm Illustrated 3 6 10 7 8 9 12 11

20

Kruskal’s Algorithm Illustrated 3 6 10 7 8 9 12 11

21

Kruskal’s Algorithm Illustrated 3 6 10 7 8 9 12 11

22

Kruskal’s Algorithm Illustrated 3 6 10 7 8 9 12 11

This is the resulting Minimum Spanning Tree.

23

Kruskal’s Algorithm - Connectivity

Sort the edges by weight

• Ascending weight if Minimum ST
• Descending weight if Maximum ST

Then, do what we did before:

• Start with an empty tree
• Iterate through all the edges in sorted order
• Given an edge (x, y)
• If x and y are connected (i.e. there is already a path from x to y), ignore this edge
• Otherwise, add this edge to the tree

24

Kruskal’s Algorithm - Connectivity

How do we find out with vertices x and y are already connected by the edges chosen so far?

• Slow Idea: Run a traversal (DFS or BFS) from node x
• This would cost O(#Edges picked so far)
• Hence, iterating through all edges would cost O(N × M), which is too slow

25

Kruskal’s Algorithm - Connectivity

How do we find out with vertices x and y are already connected by the edges chosen so far?

• Fast Idea: Use Union-Find
• Checking whether two objects are part of the same component using Union-Find is very fast

(essentially O(1)1)

• Hence, iterating through all edges would cost O(M)

1Technically, it is not O(1). However, it is fast enough that we can pretend it is.

26

Kruskal’s Algorithm - Complexity

Sort the edges by weight – O(M log M)

• Ascending weight if Minimum ST
• Descending weight if Maximum ST

Then, do what we did before:

• Start with an empty tree
• Iterate through all the edges in sorted order – O(M)
• Given an edge (x, y)
• If x and y are connected (i.e. there is already a path from x to y), ignore this edge – O(1)
• Otherwise, add this edge to the tree

Algorithm Complexity: O(M log M)

27

Discussion Problem

Input: Given N cities. You are tasked to connect these cities with roads using (N − 1) edges. Two construction companies made proposals to build roads. Each proposal specifies two cities the road would connect and the road’s cost. You were bribed to give exactly K contracts to company A. Output: Minimum total cost to connect all cities under those requirements. Constraints: 0 < N < 50, 000, 0 ≤ K < N.

28

Discussion Problem - Insight

Choose exactly N − 1 roads with minimum cost → MST! But simply running MST cannot guarantee exactly K roads from company A. Solution:

• Pick a penalty/incentive P for MST
• Add P to the cost of each road proposed by company A
• Run MST
• MST didn’t choose K roads from company A? Tweak P and try again!
• Binary search on P

29

The n-k Queens Problem

Input: An n × n-sized chessboard, and k queen pieces. Output: Is there a placement of all the queen pieces so that no piece can attack any other? That is: No two pieces share a row, column, or diagonal. An example for 8 queens on an 8 by 8 board. (image from en.wikipedia.org/wiki/Eight queens puzzle)

30

The n-k-Queens Problem, More General

We can solve an even harder problem! Input: An n × n chessboard, and k queen pieces. Output: The number of placements where all the queen pieces cannot attack each other. Note: We say two placements are different if the set of positions is different. However, in this case is it equivalent (in some sense) to say that the exact placement of some named queen is different, since this multiplies our number of solutions by n!.

31

The n-k-Queens Problem, Backtracking Approach (1)

The easiest brute force method for this problem is to try every placement of the pieces. We can do this recursively:

21 int nkqueens(vector<vector<int>> &cur_placements, int n, int k) { 22

if (k <= 0) return 1;

23

int ans = 0;

24

for (int i = 0; i < n; ++i) for (int j = 0; j < n; ++j)

25

if (cur_placements[i][j] == 0) {

26

set_rcd(cur_placements,n,i,j,1); //place & block rcd

27

ans += nkqueens(cur_placements, n, k-1);

28

set_rcd(cur_placements,n,i,j,-1); //undo blocks

29

}

30

return ans;

31 } 32 int nkqueens(int n, int k) { 33

vector<vector<int>> cur_placements(n,vector<int>(n));

34

return nkqueens(cur_placements, n, k)/factorial(n); // note: need to implement factorial

35 }

32

The n-k-Queens Problem, Backtracking Approach (2)

1 // set row, column, and diagonal around point (i,j) to v 2 void set_rcd(vector<vector<int>> &b, int n, int i, int j, int v) { 3

for (int k = 0; k < n; ++k) {

4

b[k][j] += v;

5

b[i][k] += v;

6

}

7

int i_pos = i, j_pos = j;

8

while (i_pos >= 0 && j_pos >= 0) {

9

• -i_pos; --j_pos;

10

}

11

for (++i_pos, ++j_pos; i_pos < n && j_pos < n; ++i_pos, ++j_pos)

12

b[i_pos][j_pos] += v;

13

int i_neg = i, j_neg = j;

14

while (i_neg >= 0 && j_neg < n) {

15

• -i_neg; ++j_neg;

16

}

17

for (++i_neg, --j_neg; i_neg < n && j_neg >= 0; ++i_neg, --j_neg)

18

b[i_neg][j_neg] += v;

19 }

33

The n-k-Queens Problem, Analysis

The brute force code uses O(n2) memory, and runs in O((n2)k) time. Our run-time for our code is exponential. However, in practice, it is very fast.It is part of a more general approach, called Backtracking, which has some characteristics that make it fast in practice.

34

Backtracking - Pseudocode

Backtracking is very general solution approach that matches the following code template:

1

fun backtrack(candidate):

2

if candidate is a complete solution:

3

return 1

4

num_soln = 0

5

for every valid next extension of candidate:

6

make the extension , and record this in candidate

7

num_soln += backtrack(candidate)

8

undo the extension

9

return num_soln

10 11

create an empty candidate solution S

12

backtrack(S)

35

Backtracking - Boolean Version

This can also be altered slightly to simply determine if a solution exists at all:

1

fun backtrack(candidate):

2

if candidate is a complete solution:

3

return true

4

for every valid next extension of candidate:

5

make the extension , and record this in candidate

6

if backtrack(candidate):

7

return true;

8

undo the extension

9

return false

10 11

create an empty candidate solution S

12

backtrack(S)

36

Backtracking - Pros and Cons

Backtracking has many pros and cons: + The code is compact and easy to write.

• Backtracking is just a Depth-First-Search!

+ Very low memory usage. + After one initialisation at the beginning, this code does not allocate anything to the heap (this is called zero-alloc code). + With some modifications, we can sometimes prevent any allocations to the stack from happening too. + For small recursion depth, it can be unrolled at compile-time using C++ templates for major speedups.

• No matter how good backtracking is for memory, the complexity is often very poor.

+ Occasionally, the complexity can be very good (which we will see next lecture). + Backtracking is the foundation of many popular fast(ish) and useful algorithms for NP-Hard problems, such as Satisfiability (DPLL) and Integer Linear Programming (Branch and Bound).

37

Backtracking - How to

In order to make use of backtracking, you need to identify a few things:

• What do our candidate solutions look like? (a placement of some queens)
• How do we extend a candidate solution? (where do we place each piece?)
• Note: Different steps in the problem may have slightly different kinds of solutions (not the

case in the n-k queens problem).

• How do we identify which candidate extensions are valid?
• This is often the most complicated part of a backtracking problem, and a good solution to a

hard problem typically involves some pre-computation.

38

Discussion Problem: Lost on the Taco Islands

Jack has found himself in an unusual predicament. While visiting the Taco Islands, all of his k friends have gotten lost on different islands. Since they aren’t as familiar with the islands as Jack, they are all standing still. Unfortunately, the bridges between the islands are quite unstable, so Jack and his friends can only cross each bridge once before it breaks behind them. Additionally, Jack is very tired, and only has the energy to cross t bridges before his legs will give out. Can Jack gather all his friends? Input: There are 1 ≤ n ≤ 105 islands, and 1 ≤ m ≤ 106 bridges between islands. Jack has 2 ≤ k ≤ 15 friends, and can cross up to 1 ≤ t ≤ 7 bridges before his legs give out. Additionally, it is guaranteed that there are no more than 10 bridges connected to any particular island. Output: Can Jack find all of his friends before his legs give out?

39

Discussion Problem - Insight

Use backtracking!

• This is a graph problem.
• Our candidate solution is a path through the graph, and the set of friends collected so far.
• The extensions to the candidate are the possible next-extensions.
• If implemented with the right data structures, the run-time of backtracking for this

problem is O(10t), since the maximum degree is at most 10.

40

Preorder & Postorder Traversals

If the graph is a tree, a DFS can compute useful information:

• Preorder traversal
• Postorder traversal

41

Preorder Traversal

Algorithm: dfs pre(u) visits all nodes reachable from u in DFS order. Initialize an integer cnt = 0 before running.

1

void dfs_pre(int u, vector<vector<int>> &adj,

2

vector<int> &visited,

3

vector<int> &preorder,

4

int &cnt) {

5

if (visited[u]) return;

6

visited[u] = true;

7

preorder[u] = cnt++;

8 9

for (int v : adj[u])

10

dfs_pre(u, adj, visited, preorder, cnt);

11

}

42

Postorder Traversal

Algorithm: dfs post(u) visits all nodes reachable from u in DFS order. Initialize an integer cnt = 0 before running.

1

void dfs_post(int u, vector<vector<int>> &adj,

2

vector<int> &visited,

3

vector<int> &postorder,

4

int &cnt) {

5

if (visited[u]) return;

6

visited[u] = true;

7 8

for (int v : adj[u])

9

dfs_post(u, adj, visited, preorder, cnt);

10

postorder[u] = cnt++;

11

}

43

Euler Tours - Definition and problem statement

An Euler tour is a path through a graph that uses each edge exactly once. Problem: Find an Euler tour in a tree, or determine that none exists. Input: A simple, undirected tree. Output: An Euler tour through the graph, or −1 if none exists.

44

Euler Tour

List nodes in pre-order traversal, then subtree = subarray!

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

45

Euler Tour

List node every time you enter and exit on a “tour” then we can compute Lowest Common Ancestor!

[1, 2, 3, 3, 2, 4, 4, 2, 1, 5, 6, 7, 7, 6, 8, 8, 6, 5, 1, 9, 10, 10, 9, 1]

46

Watch out!

Common edge cases and bugs with trees

• Multiple trees (i.e. a forest)
• Tree with zero or one node
• Tree that is a very long line
• Tree with very big breadth
• “Broomstick” tree: above two cases combined

47

Fun Things to do This Weekend

Lucca’s recommendation: Drive

48

Fun Things to do This Weekend

Jack’s recommendation: ✭✭✭✭ ✭ Finish A1 Kill Bill

49