Hard Problems Tyler Moore CSE 3353, SMU, Dallas, TX April 30, 2013 - - PDF document

Hard Problems

Tyler Moore

CSE 3353, SMU, Dallas, TX

April 30, 2013

The many cases of finding shortest paths

We’ve already seen how to calculate the shortest path in an unweighted graph (BFS traversal) We’ll now study how to compute the shortest path in different circumstances for weighted graphs

1

Single-source shortest path on a weighted DAG (review from last week)

2

Single-source shortest path on a weighted graph with nonnegative weights (Dijkstra’s algorithm)

3

Single-source shortest path on a weighted graph including negative weights (Bellman-Ford algorithm)

4

All-pairs shortest path on a weighted graph with nonnegative weights (Floyd-Warshall algorithm)

2 / 22

From single-source to all-pairs shortest paths

The algorithms discussed so far calculated the shortest path to all vertices from a single source We might instead be interested in the shortest path to all vertices from all sources We could run Dijkstra’s or Bellman-Ford n times, one for each source But more efficient solutions exist, particularly when graphs are dense Floyd-Warshall algorithm uses recursion or DP to compute shortest paths between all pairs of vertices

Start by placing nodes in arbitrarily sorted order d(u, v, k): length of shortest path from u to v using first k nodes as intermediaries d(u, v, k) = min(d(u, v, k − 1), d(u, k, k − 1) + d(k, v, k − 1) Repeatedly solve for all pairs with increasing k

3 / 22

All-pairs shortest path application

Let’s compare distances between airport hubs City New York Los Angeles Seattle Atlanta Dallas New York 2470 2400 760 1390 LA 2470 950 1940 1230 Seattle 2400 950 2180 1660 Atlanta 760 1940 2180 729 Dallas 1390 1230 1660 729 Total 7020 7548 8148 5609 5009 The farness of a node in a connected graph is the sum of all shortest path distances to other nodes Closeness centrality: inverse of the average length of all shortest paths from a vertex

4 / 22

Notes Notes Notes Notes

All-pairs shortest paths in a graph

280 1 2 400 320 250 400 5 650 5 5 1350 8 5 400 600 650 290 5 1200 800 1 3 9 2 2 625 675

5 / 22

Code for Floyd-Warshall

• 1. Recursive approach

def r e c f l o y d w a r s h a l l (G) : #A l l s h o r t e s t paths @memo #Store s u b s o l u t i o n s def d(u , v , k ) : #u to v v i a 1 . . k i f k==0: return G[ u ] [ v ]#Assumes v in G[ u ] return min (d(u , v , k −1) , d(u , k , k−1) + d(k , v , k−1)) #Use k or not ? return {(u , v ) : d(u , v , l e n (G)) for u in G for v in G} # D[ u , v ] = d(u , v , n)

• 2. DP approach

from copy import deepcopy def f l o y d w a r s h a l l 1 (G) : D = deepcopy (G) #No i n t e r m e d i a t e s yet for k in G: #Look f o r s h o r t c u t s with k for u in G: for v in G: D[ u ] [ v]=min (D[ u ] [ v ] ,D[ u ] [ k]+D[ k ] [ v ] ) return D

6 / 22

All-pairs shortest paths

DEN NO CHI DFW DC MSP SFO LR LAX NYC SEA TUL ATL DEN 1120 950 830 1770 650 1300 800 800 1550 1950 550 1400 NO 1120 720 400 1025 1220 2100 320 1600 1245 2520 570 400 CHI 1050 720 690 895 1150 2350 400 1850 675 2450 500 1000 DFW 830 400 680 1425 930 1700 290 1200 1355 2230 280 800 DC 1770 1545 895 1515 1120 3070 1225 2570 220 2420 1395 625 MSP 650 1220 1050 930 1120 1950 900 1450 900 1300 650 1500 SFO 1300 2100 2250 1700 3070 1950 1990 500 2850 1700 1850 2500 LR 800 320 400 290 1225 900 1990 1490 1075 2200 250 600 LAX 800 1600 1750 1200 2570 1450 500 1490 2350 1200 1350 2000 NYC 1550 1395 675 1365 220 900 2850 1075 2350 2200 1175 845 SEA 1950 2520 2350 2230 2420 1300 800 2200 1200 2200 1950 2800 TUL 550 570 400 280 1295 650 1850 250 1350 1075 1950 850 ATL 1400 920 1000 890 625 1500 2590 600 2090 845 2800 850 Farness 13670 13240 13730 12120 18370 13620 23760 11540 18260 16600 23920 11070 16110

8 / 22

Hard problems

For most of this class, we’ve discussed problems solved by efficient algorithms But for many problems, no efficient algorithm exists Formally, there are two key classes of problems: P and NP Let’s look at one such hard problem: traveling salesman problem

Input: weighted graph G = (V , E) Output: minimum-cost cycle that visits all vertices V exactly once Belongs to class NP

9 / 22

Notes Notes Notes Notes

Traveling salesman problem

280 1 2 400 320 250 400 5 650 5 5 1350 8 5 400 600 650 290 5 1200 800 1 3 9 2 2 625 675

10 / 22

Traveling salesman problem

280 1 2 400 320 250 400 5 650 5 5 1350 8 5 400 600 650 290 5 1200 800 1 3 9 2 2 625 675

Optimal solution: exhaustive search (Θ(n!) steps) Total distance: 7735 miles

10 / 22

Code for exhaustive TSP

def t s p e x (G) : #naive s o l u t i o n from i t e r t o o l s import permutations mintour =[] mincost=f l o a t ( ’ i n f ’ ) for tour in permutations (G. keys ( ) ) : t o t a l c o s t =0 nocycle=False for i in range (1 , l e n (G ) ) : try : t o t a l c o s t+= G[ tour [ i −1]][ tour [ i ] ] except KeyError : nocycle=True break i f not nocycle and t o t a l c o s t <mincost : mincost=t o t a l c o s t mintour=tour return ( mintour , mincost )

11 / 22

Traveling salesman problem

A dynamic programming solution to TSP exists that has Θ(n2 · 2n) time complexity The exhaustive TSP has Θ(n!) time complexity. Is the DP solution worth the trouble? In our example, n = 13.

n! ≈ 6 billion n2 · 2n ≈ 1.4 million

Can we do any better?

12 / 22

Notes Notes Notes Notes

One way to solve the TSP

Source: http://xkcd.com/399/ 13 / 22

Another way to approach the TSP

In general, hard problems can often be worked around in two ways

1

Constrain the input to solve a special case (think DAGs vs. general graphs)

2

Design an approximation algorithm that finds a valid answer close to the optimum

Algorithms vs. heuristics

Algorithms always produce a correct result Heuristics often work but do not guarantee correctness

Approximation heuristics and algorithms for TSP

1

Nearest-neighbor heuristic

2

Multifragment-heuristic

3

“Twice-round the tree” algorithm for metric TSP

14 / 22

TSP nearest-neighbor heuristic

Pseudocode

1

Select an arbitrary vertex, then repeatedly visit the next undiscovered node with minimum edge weight

2

Once all nodes have been visited, return to the starting node

+ Efficient execution

• No guarantee about how the cost of the cycle found by the heuristic

compares to the optimal solution

• No guarantee that a cycle will be found, even if one exists

15 / 22

TSP nearest neighbor heuristic

a b c d

3 8 3 2 4 1

Nearest-neighbor heuristic: a – d – c – b – a (cost 14) Optimal: a – b – d – c – a (cost 12)

16 / 22

Notes Notes Notes Notes

Code for TSP nearest-neighbor heuristic

def nn tsp (G) : for u in G: #t r y a l l s t a r t i n g v e r t i c e s u n t i l a c y c l e i s found C = [ u ] while True : try : #choose the cheapest path wt , nxt = min ( [ ( G[ u ] [ v ] , v ) for v in G i f v not in C ] ) except : break u = nxt

• C. append (u)

i f l e n (C)!= l e n (G) : break # i f we broke e a r l y then no c y c l e i f C [ 0 ] in G[C[ −1]]: return C else : break #i f no path back to the s t a r t then no c y c l e return ”no c y c l e found ”

17 / 22

TSP multifragment heuristic

Pseudocode

1

Select next-available edge based on minimum weight (use priority queue) until all edges have been examined

2

If the edge does not create a vertex of degree 3 and the edge does not create a cycle with fewer than n vertices, then add the edge to the tour.

+ Efficient execution (if edge addition and merging is efficiently implemented)

• In general case, no guarantee about how the cost of the cycle found

by the heuristic compares to the optimal solution

• In general case, no guarantee that a cycle will be found

+ In particular case of metric TSP (explained later), the cycle’s cost is guaranteed to be within a small constant of the optimal cost.

18 / 22

TSP multifragment heuristic

a b c d

3 8 3 2 4 1

Multifragment heuristic: c – d – a – b – c (cost 14) Optimal: a – b – d – c – a (cost 12)

19 / 22

Metric TSP problem

a b c

d(a, b) = 1 d(b, c) = 2 d(a, c) = 2.23

In a metric TSP, the distance between vertices follows the triangle inequality: d(a, c) ≤ d(a, b) + d(b, c) This means that a connected graph must be complete Let’s look at a smaller, completely connected graph of US cities

20 / 22

Notes Notes Notes Notes

TSP “Twice Around the Tree” Approximation

2 4 7 2400 760 1 3 9 950 1940 1 2 3 2180 1660 730

1: Compute minimum spanning tree

21 / 22

TSP “Twice Around the Tree” Approximation

2 4 7 2400 760 1 3 9 950 1940 1 2 3 2180 1660 730

2: DFS on MST starting in LA Total distance: 6570 miles

21 / 22

TSP “Twice Around the Tree” Approximation

2 4 7 2400 760 1 3 9 950 1940 1 2 3 2180 1660 730

Optimal cycle (from exhaustive) Total distance: 6070 miles

21 / 22

Code for “Twice Around the Tree” Approximation

from c o l l e c t i o n s import d e f a u l t d i c t def mtsp (G, r ) : #2−approx f o r metric TSP T, C = d e f a u l t d i c t ( l i s t ) , [ ] #Tree and c y c l e for c , p in prim (G, r ) . items ( ) :#Build a t r a v e r s a b l e MSP T[ p ] . append ( c ) #Child i s parent ’ s neighbor def walk ( r ) : #Recursive DFS

• C. append ( r )

#Preorder node c o l l e c t i o n for v in T[ r ] : walk ( v ) #V i s i t s u b t r e e s r e c u r s i v e l y walk ( r ) #Traverse from the root return C #At l e a s t half −optimal c y c l e

22 / 22

Notes Notes Notes Notes