CSE 101 Algorithm Design and Analysis Sanjoy Dasgupta Russell - - PowerPoint PPT Presentation

Algorithm Design and Analysis

Sanjoy Dasgupta Russell Impagliazzo Ragesh Jaiswal (Thanks Miles Jones) Lecture 29: Dynamic Programming (Shortest Paths)

CSE 101

Given a graph with edge weights, find paths that minimize total weight

• f edges used

Dijkstra’s shortest path algorithm assumed non-negative edge weights Another easy case: DAG

SHORTEST PATHS

What if the graph has both negative weights and cycles? If the graph has negative cycles, some distances are negative infinity. Otherwise, 𝐸𝑗𝑡𝑢 𝐷 ≡ min

! !,# $% &' ()*(}{𝐸𝑗𝑡𝑢 𝑤 + 𝑥

𝑤, 𝐷 } But this doesn’t give a consistent recursion, because the recursion graph is not a DAG, so recursion loops around cycles…… How can we make recursion acyclic when the graph is cyclic?

WHAT IF THERE ARE CYCLES?

Generalizing problems to allow recursion and hence DP Keeping recursions simple Trying multiple recursive algorithms based on different types of case analysis leading to different DP algorithms Using reductions (again)

LESSONS

WHY WE MIGHT WANT TO DO THIS: ARBITRAGE

What vertex do we visit first (after 𝑡)? What vertex do we visit last (before 𝑢)? What is the middle vertex along the path? Do we ever visit vertex 𝑤?

SOME POSSIBILITIES

FIRST VERTEX

Attempt 1: BTSP(𝑡, 𝑢, 𝐻) ▪ If 𝑡 = 𝑢 return 0 ▪ 𝑁 := ∞ ▪ For each 𝑤 ∈ 𝑂(𝑡) do: ▪ 𝑁 := min(𝑁, (𝑥(𝑡, 𝑤) + BTSP(𝑤, 𝑢, 𝐻))) ▪ return 𝑁

Cycles in graph = cycles in recursive calls

PROBLEM: LOOPING

FIRST VERTEX

Attempt 2: BTSP(𝑡, 𝑢, 𝐻) ▪ If 𝑡 = 𝑢 return 0 ▪ 𝑁 := ∞ ▪ For each 𝑤 ∈ 𝑂(𝑡) do: ▪ 𝑁 := min(𝑁, (𝑥(𝑡, 𝑤) + BTSP(𝑤, 𝑢, 𝐻 − {𝑡}))) ▪ return 𝑁

Changing G in recursive calls: exponentially many sub-graphs. Usable for longest simple path = TSP = NP-complete, so approach seems inherently exponential.

PROBLEM

Let’s put a budget on how deep we’ll recurse, 𝑈 That corresponds to looking at which paths? How large should 𝑈 be?

PREVENTING INFINITE RECURSION

Let’s put a budget on how deep we’ll recurse, 𝑈 That corresponds to looking at paths with at most T edges How large should 𝑈 be: 𝑈 = 𝑜 : search all simple paths. Not all paths we search are simple, but that’s OK

PREVENTING INFINITE RECURSION

If there is a negative cycle, we can keep going around it. But we can break up any cycle into a set of simple cycles.

MINIMUM LENGTH NEGATIVE CYCLE

If there is a negative cycle, we can keep going around it. But we can break up any cycle into a set of simple cycles.

MINIMUM LENGTH NEGATIVE CYCLE

If the overall cycle is negative, at least one simple cycle is negative. So if there is any negative cycle, there is one of size at most 𝑜

Let’s put a budget on how deep we’ll recurse, 𝑈 That corresponds to looking at paths of length at most 𝑈 If there’s no negative cycle, then the shortest paths are simple, don’t repeat vertices. So the shortest path between any two vertices is of length at most |𝑊| = 𝑜

PREVENTING INFINITE RECURSION

FIRST VERTEX

BF[𝑡, 𝑢, 𝑈]: shortest weight of a path from 𝑡 to 𝑢 with at most 𝑈 edges Note: G is also an input to recursion, but G doesn’t change. Only need changing parts as parameters in matrix

BELLMAN-FORD SUBPROBLEMS

Base case : 𝑈 = ? BF[𝑡, 𝑢,?] = 0 if ?? ?? otherwise

BASE CASES:

Base case : 𝑈 = 0 BF[𝑡, 𝑢, 0] = 0 if 𝑡 = 𝑢 ∞, otherwise

BASE CASES:

Check all first steps along path BF[𝑡, 𝑢, 𝑈] := min (BF[𝑡, 𝑢, 𝑈 − 1], min

• ∈/ %

𝐶𝐺 𝑣, 𝑢, 𝑈 − 1 + 𝑥 𝑡, 𝑣 )

RECURSIVE PART:

BELLMAN-FORD METHOD

BELLMAN-FORD ALGORITHM

BELLMAN-FORD ALGORITHM TIME ANALYSIS

We never use BF(𝑣, 𝑤, 𝑈) beyond step 𝑈 + 1. It’s OK if we’ve made BF(𝑣, 𝑤, 𝑈) smaller as long as we never make it less than the shortest path length. So we can just use one matrix BF(𝑣, 𝑤) rather than 𝑜 different matrices, and make both read and write steps from the same Matrix.

SAVING MEMORY

Bellman-Ford’s recursion based on "What is the first vertex on the path from 𝑣 to 𝑤?" What other question could we ask about the path? What is the last vertex on the path? Symmetric

ALTERNATIVE RECURSION

Bellman-Ford’s recursion based on "What is the first vertex on the path from 𝑣 to 𝑤?" What other question could we ask about the path? What is the last vertex ? (BF in reverse) What is the middle vertex of the path? (Min plus matrix multiply method) Is 𝑤' ever used in the path? (Floyd-Warshall algorithm)

ALTERNATIVE RECURSION

MIN-PLUS METHOD

MIN PLUS ALGORITHM

MIN PLUS ALGORITHM

WHICH IS BETTER?

Is 𝑤 used at all? Need to eliminate vertices in order, so that remaining vertices are consecutive. Subtlety one: need to distinguish between endpoints and set of intermediate points Subtlety two: implicitly assumes no negative cycles, so that shortest paths can be simple.

FLOYD WARSHALL

Assuming the shortest paths are simple, find the shortest path in G from 𝑡 to 𝑢 using intermediate nodes 𝑉 = {𝑣!, … , 𝑣"}, where 𝑡 and 𝑢 may or may not be in 𝑉. BTFW(𝑡, 𝑢, 𝐻, 𝑉) IF 𝑉 is empty, and 𝑡 = 𝑢 return 0 IF 𝑉 is empty and (𝑡, 𝑢) is an edge return w(𝑡, 𝑢) IF 𝑉 is empty, return infinity Usecase:= BTFW(𝑡, 𝑣", 𝐻, 𝑉 − {𝑣"}) + BTFW(𝑣", 𝑢, 𝐻, 𝑉 − {𝑣"}) //Use u_k at most once Dontusecase:=BTFW(𝑡, 𝑢, 𝐻, 𝑉 − {𝑣"}) //Don’t use 𝑣" Return max(Usecase, Dontusecase)

FLOYD WARSHALL GENERALIZATION

𝑡 can be arbitrary, 𝑢 can be arbitrary, 𝑉 = {𝑤0, … 𝑤1} for some 0 ≤ 𝑙 ≤ 𝑜 FW[𝑡, 𝑢, 𝑙] : Best path from 𝑡 to 𝑢 using only 𝑤0, … , 𝑤1 as intermediate vertices Base case: 𝑉 is empty, 𝑙 = 0, FW[𝑡, 𝑡, 0]=0, FW(𝑡, 𝑢)=w(𝑡, 𝑢) for edge (𝑡, 𝑢), FW[𝑡, 𝑢, 0]=∞ Recursion: FW[𝑡, 𝑢, 𝑙] = min ( , )

REACHABLE SUBPROBLEMS

𝑡 can be arbitrary, 𝑢 can be arbitrary, 𝑉 = {𝑤0, … 𝑤1} for some 0 ≤ 𝑙 ≤ 𝑜 FW[𝑡, 𝑢, 𝑙] : Best path from 𝑡 to 𝑢 using only 𝑤0, … , 𝑤1 as intermediate vertices Base case: 𝑉 is empty, 𝑙 = 0, FW[𝑡, 𝑡, 0]=0, FW(𝑡, 𝑢)=w(𝑡, 𝑢) for edge (𝑡, 𝑢), FW[𝑡, 𝑢, 0]=∞ Recursion: FW[𝑡, 𝑢, 𝑙] = min(F[𝑡, 𝑤1, 𝑙 − 1]+F[𝑤1, 𝑢, 𝑙 − 1], F[𝑡, 𝑢, 𝑙 − 1])) Top down: k decreases, bottom-up : k increases

REACHABLE SUBPROBLEMS

DPFW(𝐻) Initialize array FW[𝑊, 𝑊, 1 … |𝑊|] For 𝑡 in 𝑊 do: FW[𝑡, 𝑡, 0]:= 0 For edges 𝑡, 𝑢 ∈ E do: FW[𝑡, 𝑢, 0] = w(𝑡, 𝑢) For 𝑡 in 𝑊 do: For 𝑢 in 𝑊 − {𝑡} do: F[𝑡, 𝑢, 0]:= ∞ For 𝑙 = 1 to |𝑊| do: For 𝑡 in 𝑊 do: For 𝑢 in 𝑊 do:

FW[𝑡, 𝑢, 𝑙] = min(FW[𝑡, 𝑤!, 𝑙 − 1]+FW[𝑤!, 𝑢, 𝑙 − 1],FW[𝑡, 𝑢, 𝑙 − 1]))

Return the matrix FW[𝑡, 𝑢, |𝑊|] (Note: can save space by reusing single matrix)

FW ALGORITHM

WHICH IS BETTER?