10 B: Graph Algorithms IV CS1102S: Data Structures and Algorithms - - PowerPoint PPT Presentation

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler

10 B: Graph Algorithms IV

CS1102S: Data Structures and Algorithms

Martin Henz

March 27, 2009

Generated on Friday 26th March, 2010, 10:59 CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 1

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler

1

Maximum Flow and Minimum Spanning Tree

2

Problem Difficulty

3

Another Puzzler

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 2

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

1

Maximum Flow and Minimum Spanning Tree Maximum Flow Minimum Spanning Tree

2

Problem Difficulty

3

Another Puzzler

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 3

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Maximum Flow

Interpretation of weights Every weight of an edge (v, w) represents a capacity of a flow passing from v to w.

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 4

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Maximum Flow

Interpretation of weights Every weight of an edge (v, w) represents a capacity of a flow passing from v to w. Maximum Flow Given two vertices s and t, compute the maximal flow achievable from s to t.

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 5

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Example Graph and Maximum Flow

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 6

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Idea

Identify an augmenting path a path that has a feasible flow

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 7

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Idea

Identify an augmenting path a path that has a feasible flow Add path to flow graph keeping track of a flow that is already achieved

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 8

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Idea

Identify an augmenting path a path that has a feasible flow Add path to flow graph keeping track of a flow that is already achieved Remove path from residual graph keeping track of the remaining flow that can still be exploited

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 9

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Example: Initial Setup

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 10

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Example Run

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 11

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Example Run

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 12

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Example Run

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 13

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Counterexample: Suboptimal Solution

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 14

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Idea

Allow algorithm to change its mind Add reverse edge to residual graph, allowing a flow back in the

• pposite direction.

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 15

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Idea

Allow algorithm to change its mind Add reverse edge to residual graph, allowing a flow back in the

• pposite direction.

Consequence This means that later, we can exploit an original flow which has been utilized already in the current flow graph, for a new augmenting path.

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 16

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Example

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 17

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Runtime Analysis

Assumption Edge weights are integers

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 18

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Runtime Analysis

Assumption Edge weights are integers Each augmenting path makes “progress” adding a flow of at least 1 to the flow graph.

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 19

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Runtime Analysis

Assumption Edge weights are integers Each augmenting path makes “progress” adding a flow of at least 1 to the flow graph. Finding augmenting paths can be done in O(N), using unweighted shortest path algorithm

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 20

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Runtime Analysis

Assumption Edge weights are integers Each augmenting path makes “progress” adding a flow of at least 1 to the flow graph. Finding augmenting paths can be done in O(N), using unweighted shortest path algorithm Overall O(f · |E|) where f is the maximum flow

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 21

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Bad case

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 22

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Minimum Spanning Tree Problem

Input Undirected weighted connected graph G

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 23

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Minimum Spanning Tree Problem

Input Undirected weighted connected graph G Spanning tree Tree that contains all of G’s vertices and only edges from G

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 24

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Minimum Spanning Tree Problem

Input Undirected weighted connected graph G Spanning tree Tree that contains all of G’s vertices and only edges from G Minimum spanning tree Spanning tree whose sum of edge weights is minimal

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 25

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

An Example Graph and its MST

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 26

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Algorithm Idea

Similar to Dijkstra’s Algorithm Start “growing” the MST at arbitrary vertex.

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 27

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Algorithm Idea

Similar to Dijkstra’s Algorithm Start “growing” the MST at arbitrary vertex. At each step, add an edge to MST with smallest weight.

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 28

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Algorithm Idea

Similar to Dijkstra’s Algorithm Start “growing” the MST at arbitrary vertex. At each step, add an edge to MST with smallest weight. Implementation Keep edges from “known” to “unknown” vertices in a priority queue.

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 29

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Example

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 30

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Runtime Analysis

Without priority queue O(|V|2)

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 31

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Maximum Flow Minimum Spanning Tree

Runtime Analysis

Without priority queue O(|V|2) With priority queue O(|E| log |V|)

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 32

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

1

Maximum Flow and Minimum Spanning Tree

2

Problem Difficulty Algorithmic Problems Undecidability of the Halting Problem

3

Another Puzzler

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 33

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

How Difficult Can Problems Be?

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 34

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

Examples

Finding the smallest element in a linked list:

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 35

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

Examples

Finding the smallest element in a linked list: O(N)

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 36

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

Examples

Finding the smallest element in a linked list: O(N) Inserting an element into a heap:

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 37

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

Examples

Finding the smallest element in a linked list: O(N) Inserting an element into a heap: O(log N)

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 38

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

Examples

Finding the smallest element in a linked list: O(N) Inserting an element into a heap: O(log N) Sorting an array of items using comparisons:

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 39

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

Examples

Finding the smallest element in a linked list: O(N) Inserting an element into a heap: O(log N) Sorting an array of items using comparisons: O(N log N)

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 40

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

Examples

Finding the smallest element in a linked list: O(N) Inserting an element into a heap: O(log N) Sorting an array of items using comparisons: O(N log N) Printing all sequences of N binary digits:

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 41

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

Examples

Finding the smallest element in a linked list: O(N) Inserting an element into a heap: O(log N) Sorting an array of items using comparisons: O(N log N) Printing all sequences of N binary digits: O(2N)

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 42

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

Algorithmic Problems

Given Input Clear description of input data

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 43

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

Algorithmic Problems

Given Input Clear description of input data Required Output Description of what a solution constitutes

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 44

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

Algorithmic Problems

Given Input Clear description of input data Required Output Description of what a solution constitutes Algorithmic Solution Description of a process how to arrive at the required output, given any legal input

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 45

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

Questions

Are all algorithmic problems solvable?

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 46

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

Questions

Are all algorithmic problems solvable? Are all algorithmic problems solvable in polynomial time?

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 47

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

Questions

Are all algorithmic problems solvable? Are all algorithmic problems solvable in polynomial time? Is there a k such that there is an O(Nk) algorithm for the problem?

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 48

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

Questions

Are all algorithmic problems solvable? Are all algorithmic problems solvable in polynomial time? Is there a k such that there is an O(Nk) algorithm for the problem? Clearly no! The output can have exponential size!

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 49

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

Questions

Are all algorithmic problems solvable? Are all algorithmic problems solvable in polynomial time? Is there a k such that there is an O(Nk) algorithm for the problem? Clearly no! The output can have exponential size! If we do not have a polynomial algorithm, can we always prove that there is none?

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 50

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

Halting Problem

Definition (Halting Problem) Given a description of a program and a finite input, decide whether the program finishes running or will run forever, given that input.

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 51

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

Undecidability of the Halting Problem

Theorem (Undecidability of the Halting Problem) The Halting Problem is undecidable; there cannot exist a program that returns true, if and only if a given function f terminates when applied to a given value x.

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 52

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

Proof of the Undecidability of the Halting Problem

Assume that there is a Java function halts that when applied to a representation f’ of a Java function f, and a Java object x returns true if f(x) terminates, and false if f(x) does not

• terminate. Such a function halts exists if and only if the

Halting Problem is decidable.

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 53

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

Proof of the Undecidability of the Halting Problem

With the assumption of the existence of halts, let us construct a Java function strange as follows: boolean strange(w’) { if (halts(w’,w’) while (true); return true; } Such a function strange can surely be written, if halts exists.

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 54

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

Proof of the Undecidability of the Halting Problem

boolean strange(w’) { if (halts(w’,w’) while (true); return 0; } What will strange(strange’) return?

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 55

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler Algorithmic Problems Undecidability of the Halting Problem

Proof of the Undecidability of the Halting Problem

Conclusion: strange cannot exist, and therefore halt cannot exist. The Halting Problem is undecidable.

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 56

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler

1

Maximum Flow and Minimum Spanning Tree

2

Problem Difficulty

3

Another Puzzler

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 57

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler

Remember Lecture 2 A: Parameter Passing

Java uses pass-by-value parameter passing. public static void tryChanging ( int a ) { a = 1; return ; } . . . int b = 2; tryChanging ( b ) ; System . out . p r i n t l n ( b ) ;

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 58

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler

Remember Lecture 2 A: Parameter Passing with Objects

public static void tryChanging ( SomeObject obj ) {

• bj . someField = 1;
• bj = new SomeObject ( ) ;
• bj . someField = 2;

return ; } . . . SomeObject someObj = new SomeObject ( ) ; tryChanging ( someObj ) ; System . out . p r i n t l n ( someObj . someField ) ;

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 59

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler

Remember Lecture 7 A: Sorting

Input Unsorted array of elements Behavior Rearrange elements of array such that the smallest appears first, followed by the second smallest etc, finally followed by the largest element

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 60

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler

Will This Work?

public static <AnyType extends Comparable<? super An void mergeSort ( AnyType [ ] a ) { AnyType [ ] r e t = . . . . ; / / declare helper array . . . . / / here goes a program that places . . . . / / the element

• f

” a ” i n t o ” r e t ” so . . . . / / that ” r e t ” i s sorted a = r e t ; return ; } . . . Integer [ ] myArray = . . . ; IterativeMergeSort . mergeSort ( myArray ) ;

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 61

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler

Will This Work?

public static <AnyType extends Comparable<? super An void mergeSort ( AnyType [ ] a ) { AnyType [ ] r e t = . . . . ; / / declare helper array . . . . / / here goes a program that places . . . . / / the element

• f

” a ” i n t o ” r e t ” so . . . . / / that ” r e t ” i s sorted a = r e t ; return ; } . . . Integer [ ] myArray = . . . ; IterativeMergeSort . mergeSort ( myArray ) ; Answer: No! The assignment a = ret ; has no effect on myArray!

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 62

Maximum Flow and Minimum Spanning Tree Problem Difficulty Another Puzzler

Next Week

NP-Completeness Algorithm design techniques

CS1102S: Data Structures and Algorithms 10 B: Graph Algorithms IV 63