Algorithms R OBERT S EDGEWICK | K EVIN W AYNE 6.5 R EDUCTIONS ‣ introduction ‣ designing algorithms ‣ establishing lower bounds Algorithms ‣ classifying problems F O U R T H E D I T I O N ‣ intractability R OBERT S EDGEWICK | K EVIN W AYNE http://algs4.cs.princeton.edu
Overview: introduction to advanced topics Main topics. [final two lectures] ・ Reduction: relationship between two problems. ・ Algorithm design: paradigms for solving problems. Shifting gears. ・ From individual problems to problem-solving models. ・ From linear/quadratic to polynomial/exponential scale. ・ From implementation details to conceptual frameworks. Goals. ・ Place algorithms and techniques we've studied in a larger context. ・ Introduce you to important and essential ideas. ・ Inspire you to learn more about algorithms! 2
6.5 R EDUCTIONS ‣ introduction ‣ designing algorithms ‣ establishing lower bounds Algorithms ‣ classifying problems ‣ intractability R OBERT S EDGEWICK | K EVIN W AYNE http://algs4.cs.princeton.edu
Bird's-eye view Desiderata. Classify problems according to computational requirements. complexity order of growth examples min, max, median, linear N Burrows-Wheeler transform, ... sorting, element distinctness, linearithmic N log N closest pair, Euclidean MST, ... quadratic N 2 ? ⋮ ⋮ ⋮ exponential c N ? Frustrating news. Huge number of problems have defied classification. 4
Bird's-eye view Desiderata. Classify problems according to computational requirements. Desiderata'. Suppose we could (could not) solve problem X efficiently. What else could (could not) we solve efficiently? “ Give me a lever long enough and a fulcrum on which to place it, and I shall move the world. ” — Archimedes 5
Reduction Def. Problem X reduces to problem Y if you can use an algorithm that solves Y to help solve X . Algorithm instance I solution to I for Y (of X) Algorithm for X Cost of solving X = total cost of solving Y + cost of reduction. perhaps many calls to Y preprocessing and postprocessing on problems of different sizes (typically less than cost of solving Y) (though, typically only one call) 6
Reduction Def. Problem X reduces to problem Y if you can use an algorithm that solves Y to help solve X . Algorithm instance I solution to I for Y (of X) Algorithm for X Ex 1. [finding the median reduces to sorting] To find the median of N items: ・ Sort N items. ・ Return item in the middle. cost of sorting cost of reduction Cost of solving finding the median. N log N + 1 . 7
Reduction Def. Problem X reduces to problem Y if you can use an algorithm that solves Y to help solve X . Algorithm instance I solution to I for Y (of X) Algorithm for X Ex 2. [element distinctness reduces to sorting] To solve element distinctness on N items: ・ Sort N items. ・ Check adjacent pairs for equality. cost of sorting cost of reduction Cost of solving element distinctness. N log N + N . 8
Reduction Def. Problem X reduces to problem Y if you can use an algorithm that solves Y to help solve X . Algorithm instance I solution to I for Y (of X) Algorithm for X 230 A. M. TUKING [Nov. 12, Novice error. Confusing X reduces to Y with Y reduces to X . ON COMPUTABLE NUMBERS, WITH AN APPLICATION TO THE ENTSCHEIDUNGSPROBLEM By A. M. TURING. [Received 28 May, 1936.—Read 12 November, 1936.] The "computable" numbers may be described briefly as the real 9 numbers whose expressions as a decimal are calculable by finite means. Although the subject of this paper is ostensibly the computable numbers. it is almost equally easy to define and investigate computable functions of an integral variable or a real or computable variable, computable predicates, and so forth. The fundamental problems involved are, however, the same in each case, and I have chosen the computable numbers for explicit treatment as involving the least cumbrous technique. I hope shortly to give an account of the relations of the computable numbers, functions, and so forth to one another. This will include a development of the theory of functions of a real variable expressed in terms of com- putable numbers. According to my definition, a number is computable if its decimal can be written down by a machine. In §§ 9, 10 I give some arguments with the intention of showing that the computable numbers include all numbers which could naturally be regarded as computable. In particular, I show that certain large classes of numbers are computable. They include, for instance, the real parts of all algebraic numbers, the real parts of the zeros of the Bessel functions, the numbers IT, e, etc. The computable numbers do not, however, include all definable numbers, and an example is given of a definable number which is not computable. Although the class of computable numbers is so great, and in many Avays similar to the class of real numbers, it is nevertheless enumerable. In § 81 examine certain arguments which would seem to prove the contrary. By the correct application of one of these arguments, conclusions are reached which are superficially similar to those of Gbdelf. These results f Godel, " Uber formal unentscheidbare Satze der Principia Mathematica und ver- •vvandter Systeme, I". Monatsheftc Math. Phys., 38 (1931), 173-198.
6.5 R EDUCTIONS ‣ introduction ‣ designing algorithms ‣ establishing lower bounds Algorithms ‣ classifying problems ‣ intractability R OBERT S EDGEWICK | K EVIN W AYNE http://algs4.cs.princeton.edu
Reduction: design algorithms Def. Problem X reduces to problem Y if you can use an algorithm that solves Y to help solve X . Design algorithm. Given algorithm for Y , can also solve X . More familiar reductions. ・ CPM reduces to topological sort. ・ Arbitrage reduces to negative cycles. ・ Bipartite matching reduces to maxflow. ・ Seam carving reduces to shortest paths in a DAG. ・ Burrows-Wheeler transform reduces to suffix sort. … Mentality. Since I know how to solve Y , can I use that algorithm to solve X ? programmer’s version: I have code for Y . Can I use it for X? 11
3-collinear 3-COLLINEAR. Given N distinct points in the plane, are there 3 (or more) that all lie on the same line? 3-collinear Brute force N 3 . For all triples of points ( p , q , r ) check if they are collinear. 12
3-collinear reduces to sorting Sorting-based algorithm. For each point p , ・ Compute the slope that each other point q makes with p . ・ Sort the remaining N – 1 points by slope. ・ Collinear points are adjacent. q 3 q 4 q 2 q 1 dy 1 dx 1 p cost of sorting (N times) cost of reduction Cost of solving 3-collinear. N 2 log N + N 2 . 13
Shortest paths on edge-weighted graphs and digraphs Proposition. Undirected shortest paths (with nonnegative weights) reduces to directed shortest path. 9 2 5 10 15 15 10 4 12 12 s 5 3 6 t Pf. Replace each undirected edge by two directed edges. 9 2 9 5 10 15 10 10 15 15 10 4 4 15 12 s 5 3 5 12 t 12 5 12 14
Shortest paths on edge-weighted graphs and digraphs Proposition. Undirected shortest paths (with nonnegative weights) reduces to directed shortest path. 9 2 5 10 15 15 10 4 12 12 s 5 3 6 t cost of shortest paths in digraph cost of reduction Cost of undirected shortest paths. E log V + ( E + V ) . 15
Shortest paths with negative weights Caveat. Reduction is invalid for edge-weighted graphs with negative weights (even if no negative cycles). –4 7 t s –4 7 7 –4 s t reduction creates negative cycles Remark. Can still solve shortest-paths problem in undirected graphs (if no negative cycles), but need more sophisticated techniques. reduces to weighted non-bipartite matching (!) 16
Some reductions in combinatorial optimization baseball bipartite undirected shortest paths seam carving elimination matching (nonnegative) shortest paths directed shortest paths arbitrage maxflow (in a DAG) (nonnegative) mincut assignment directed shortest paths problem (no neg cycles) linear programming 17
6.5 R EDUCTIONS ‣ introduction ‣ designing algorithms ‣ establishing lower bounds Algorithms ‣ classifying problems ‣ intractability R OBERT S EDGEWICK | K EVIN W AYNE http://algs4.cs.princeton.edu
Bird's-eye view Goal. Prove that a problem requires a certain number of steps. Ex. In decision tree model, any compare-based sorting algorithm requires Ω ( N log N ) compares in the worst case. a < b yes no b < c a < c no yes yes no b < c a b c a < c b a c yes no yes no argument must apply to all b c a c b a a c b c a b conceivable algorithms Bad news. Very difficult to establish lower bounds from scratch. Good news. Spread Ω ( N log N ) lower bound to Y by reducing sorting to Y . assuming cost of reduction is not too high 19
Linear-time reductions Def. Problem X linear-time reduces to problem Y if X can be solved with: ・ Linear number of standard computational steps. ・ Constant number of calls to Y . Ex. Almost all of the reductions we've seen so far. [ Exceptions? ] Establish lower bound: ・ If X takes Ω ( N log N ) steps, then so does Y . ・ If X takes Ω ( N 2 ) steps, then so does Y . Mentality. ・ If I could easily solve Y , then I could easily solve X . ・ I can’t easily solve X . ・ Therefore, I can't easily solve Y . 20
Recommend
More recommend
