CS/ECE/ISyE 524 Introduction to Optimization Spring 2017–18 18. Rounding and relaxation ❼ Decision problems ❼ Easy and hard examples ❼ Performance and the future ❼ Rounding ❼ Convex relaxation ❼ Convex hull Laurent Lessard (www.laurentlessard.com)
Decision problems A decision problem is a yes/no question. Examples ❼ Does the following sequence contain the pattern A , A , G ? { C , T , G , A , T , A , A , G , C , T } ❼ Is there a subset of these numbers that sums to zero? {− 7 , − 3 , − 1 , 5 , 8 } 18-2
NP decision problems A decision problem is in the class NP 1 if instances where the answer is “yes” have efficiently verifiable proofs of the fact that the answer is indeed “yes”. Here, “efficient” means polynomial-time in the length of the instance. Examples ❼ Does the following sequence contain the pattern A , A , G ? { C , T , G , A , T , A , A , G , C , T } ❼ Is there a subset of these numbers that sums to zero? {− 7 , − 3 , − 1 , 5 , 8 } The red elements prove that the answer is indeed “yes”. (1) “Nondeterministic Polynomial time” 18-3
NP decision problems ❼ The easiest problems are P , they can be solved efficiently. = ⇒ { C , T , G , A , T , A , A , G , C , T } . If the sequence has length n , the subsequence can be found using n comparisons (efficient). ❼ The hardest problems are NP-complete . We don’t know any better way to solve these problems aside from checking every possibility... = ⇒ {− 7 , − 3 , − 1 , 5 , 8 } . Need to check all subsets! If the sequence has length n , there are 2 n subsets (exponential). 18-4
P = NP? ❼ It’s not actually known whether there is such a thing as “hard” problems in NP! It could be possible that P=NP (all NP problems are solvable in polynomial time). ❼ This is perhaps the most famous unsolved problem in theoretical computer science. ❼ Most people believe that P � =NP. 18-5
Examples in P ❼ pattern-matching: Given a string x 1 x 2 . . . x n , does it contain a substring y 1 y 2 . . . y k ? (e.g. linear search) ❼ sorting a list: Given a set of numbers { x 1 , . . . , x n } sort it in ascending order. (e.g. bubble sort, quicksort) ❼ linear equations: Solve a system of n linear equations in n variables (e.g. Gaussian elimination) ❼ linear programming: Solve a linear program with n variables and n constraints. (e.g. ellipsoid method) 18-6
NP-complete problems ❼ Traveling salesman (TSP): Given a list of cities and the distances between each pair of cities, what is the shortest possible route that visits each city exactly once and returns to the origin city? ❼ Boolean satisfiability (SAT) : Given an expression using n boolean variables and the operators AND , OR , NOT , and parentheses, is there a choice of the variables that makes the expression true? Example: ( x 1 ∨ ¬ x 2 ) ∧ ( ¬ x 1 ∨ x 2 ∨ x 3 ) ∧ ¬ x 1 18-7
NP-complete problems ❼ ❦ -coloring: Given a graph, can we color the vertices using k colors so that each edge connects two vertices of a different color? This is in P for k = 2 only. ❼ vertex cover: Given a graph, can we select k of the vertices so that every vertex is at most one edge away from a selected vertex? 18-8
NP-complete problems ❼ Integer (linear) programs: Solving a linear program with integer constraints on the variables. Every NP problem can be represented as an integer program! 18-9
Easy instances ❼ All problems in NP can be written as IPs. ❼ (including problems in P) ❼ So some IPs must be easy to solve... maximize a 1 z 1 + · · · + a n z n z subject to: z 1 + · · · + z n = 1 z i ∈ { 0 , 1 } ❼ Same as max { a 1 , . . . , a n } , which can efficiently be solved! 18-10
Bad news Suppose you’d like to solve a SAT problem with n variables by brute force (checking all 2 n combinations), and you can check 10 9 combinations per second. ♥ time to check all combinations 10 1 microsecond 30 1 second 50 13 days 70 374 centuries 100 2908 × (current age of the universe) 18-11
Good news Very large NP-complete problems are solved in practice! ❼ SAT problems with a million variables ❼ TSP with a million variables (1000 cities) How is this possible? ❼ Instances occurring in practice have special structures that can be exploited. ❼ Efficient approximation algorithms sometimes exist. Example: get within ε of optimal in polynomial time. ❼ Computers and solvers are both getting faster... 18-12
Moore good news: processors Speedup since 1990: about 15 , 000 18-13
Moore good news: CPLEX Credit: stegua.github.io 18-14
Moore good news: Gurobi (current version: 7.5.2) Speedup since 1990 (CPLEX+Gurobi): about 850 , 000 18-15
Summary ❼ Computers have improved steadily. Single-thread computing has stalled, but now we have cloud computing, GPUs, and more... ❼ Solvers have improved steadily at a much faster rate than computers, and continue to do so. So today’s solver on yesterday’s hardware would outperform yesterday’s solver on todays hardware. ❼ Total speedup since 1990: about 12.7 billion times faster. A typical MILP that would have taken 400 years to solve in 1990 can be solved in 1 second today. ❼ Mileage may vary! 18-16
Leyffer–Linderoth–Luedtke complexity 1 Sven Leyffer (Argonne National Lab) Jeff Linderoth (UW–Madison) Jim Luedtke (UW–Madison) “How many decision variables ( n ) must a problem have before everyone would be willing to pay ✩ 50 that a state-of-the-art solver would no longer be able to solve it?” convex and convex with nonconvex continuous mixed-integer mixed or cont. 5 × 10 7 2 × 10 4 LP MILP QP 300 5 × 10 5 QP MIQP 1000 MIQP 300 10 5 SOCP MISOCP 1000 NLP 100 5 × 10 4 NLP MINLP 500 MINNLP 100 1 Not meant to be taken (too) seriously 18-17
Rounding Back to the standard IP formulation: c T x maximize x subject to: Ax ≤ b x ∈ Z Idea: ❼ Solve the problem for x ∈ R instead (a regular LP). ❼ Round each x i in the solution to the nearest integer. ❼ This usually does not work! 18-18
Rounding ❼ If LP solution is already integral, then it is also the exact solution to the original IP. (e.g. min cost flow problems) ❼ Rounding can lead to an infeasible point ❼ Rounding can produce a point far from the optimal point true optimum ( ), relaxed optimum ( ), rounded ( ) 18-19
Convex relaxation minimize f ( x ) x ∈ S Two ideas we will discuss: 1. Function relaxation: if f is troublesome, bound it with a function that is easier to work with, e.g. a convex function. 2. Constraint relaxation: If S is troublesome, find a bigger set that is easier to work with, e.g. a convex set. 18-20
Function relaxation f opt = minimize f ( x ) x ∈ S Suppose we can find g such that g ( x ) ≤ f ( x ) for all x . In other words g is a lower bound on f . f ( x ) 4 3 f ( x ) 2 g ( x ) 1 4 x - 1 1 2 3 18-21
Function relaxation f ( x ) 4 3 f ( x ) 2 g ( x ) 1 4 x - 1 1 2 3 ❼ Solve g opt = min x ∈ S g ( x ) and let ˆ x be the corresponding x . ❼ We have the bounds: g opt = g (ˆ x ) ≤ f opt ≤ f (ˆ x ). ❼ If f (ˆ x ) = g opt then the bound is tight and f opt = f (ˆ x ). Pick a convex g so that g opt and ˆ x are easy to find! 18-22
Constraint relaxation f opt = minimize f ( x ) x ∈ S Suppose we can find some set C such that S ⊆ C . In other words, C is a superset of S . 4 2 S 0 C - 2 - 4 - 5 0 5 18-23
Constraint relaxation 4 2 S 0 C - 2 - 4 - 5 0 5 ❼ Solve h opt = min x ∈ C f ( x ) and let ˜ x be the optimal x . ❼ We have the bound: h opt = f (˜ x ) ≤ f opt ≤ f ( x ) for x ∈ S . ❼ If ˜ x ∈ S then the bound is tight and f opt = f (˜ x ). Pick a convex C so that h opt and ˜ x are easy to find! 18-24
Common relaxations 1. Boolean constraint: x ∈ { 0 , 1 } = ⇒ 0 ≤ x ≤ 1 If x opt is 0 or 1, relaxation is exact. 2. Convex equality: f ( x ) = 0 = ⇒ f ( x ) ≤ 0 If f ( x opt ) = 0, relaxation is exact. 3. A constraint you don’t like: x � = 3 = ⇒ just remove the constraint! If x opt � = 3, relaxation is exact. 18-25
Convex hull The convex hull of a set S , written conv( S ) is the smallest convex set that contains S . Equivalent definitions: ❼ The set of all affine combinations of all points in S ❼ The intersection of all convex sets containing S 18-26
Recommend
More recommend
