18. Rounding and relaxation Decision problems Easy and hard - - PowerPoint PPT Presentation

CS/ECE/ISyE 524 Introduction to Optimization Spring 2017–18

• 18. Rounding and relaxation

❼ Decision problems ❼ Easy and hard examples ❼ Performance and the future ❼ Rounding ❼ Convex relaxation ❼ Convex hull

Laurent Lessard (www.laurentlessard.com)

Decision problems

A decision problem is a yes/no question. Examples

❼ Does the following sequence contain the pattern A, A, G ?

{C, T, G, A, T, A, A, G, C, T}

❼ Is there a subset of these numbers that sums to zero?

{−7, −3, −1, 5, 8}

18-2

NP decision problems

A decision problem is in the class NP1 if instances where the answer is “yes” have efficiently verifiable proofs of the fact that the answer is indeed “yes”. Here, “efficient” means polynomial-time in the length of the instance.

(1) “Nondeterministic Polynomial time”

Examples

❼ Does the following sequence contain the pattern A, A, G ?

{C, T, G, A, T, A, A, G, C, T}

❼ Is there a subset of these numbers that sums to zero?

{−7, −3, −1, 5, 8} The red elements prove that the answer is indeed “yes”.

18-3

NP decision problems

❼ The easiest problems are P, they can be solved efficiently.

= ⇒ {C, T, G, A, T, A, A, G, C, T}. If the sequence has length n, the subsequence can be found using n comparisons (efficient).

❼ The hardest problems are NP-complete. We don’t know

any better way to solve these problems aside from checking every possibility... = ⇒ {−7, −3, −1, 5, 8}. Need to check all subsets! If the sequence has length n, there are 2n subsets (exponential).

18-4

P = NP?

❼ It’s not actually known whether there is such a thing as

“hard” problems in NP! It could be possible that P=NP (all NP problems are solvable in polynomial time).

❼ This is perhaps the most famous unsolved problem in

theoretical computer science.

❼ Most people believe that P=NP.

18-5

Examples in P

❼ pattern-matching: Given a string x1x2 . . . xn, does it

contain a substring y1y2 . . . yk? (e.g. linear search)

❼ sorting a list: Given a set of numbers {x1, . . . , xn} sort it

in ascending order. (e.g. bubble sort, quicksort)

❼ linear equations: Solve a system of n linear equations in n

variables (e.g. Gaussian elimination)

❼ linear programming: Solve a linear program with n

variables and n constraints. (e.g. ellipsoid method)

18-6

NP-complete problems

❼ Traveling salesman (TSP): Given a list of cities and the

distances between each pair of cities, what is the shortest possible route that visits each city exactly once and returns to the origin city?

❼ Boolean satisfiability (SAT): Given an expression using

n boolean variables and the operators AND, OR, NOT, and parentheses, is there a choice of the variables that makes the expression true? Example: (x1 ∨ ¬x2) ∧ (¬x1 ∨ x2 ∨ x3) ∧ ¬x1

18-7

NP-complete problems

❼ ❦-coloring: Given a graph, can we color

the vertices using k colors so that each edge connects two vertices of a different color? This is in P for k = 2 only.

❼ vertex cover: Given a graph, can we

select k of the vertices so that every vertex is at most one edge away from a selected vertex?

18-8

NP-complete problems

❼ Integer (linear) programs: Solving a linear program with

integer constraints on the variables. Every NP problem can be represented as an integer program!

18-9

Easy instances

❼ All problems in NP can be written as IPs. ❼ (including problems in P) ❼ So some IPs must be easy to solve...

maximize

z

a1z1 + · · · + anzn subject to: z1 + · · · + zn = 1 zi ∈ {0, 1}

❼ Same as max{a1, . . . , an}, which can efficiently be solved!

18-10

Bad news

Suppose you’d like to solve a SAT problem with n variables by brute force (checking all 2n combinations), and you can check 109 combinations per second. ♥ time to check all combinations 10 1 microsecond 30 1 second 50 13 days 70 374 centuries 100 2908 × (current age of the universe)

18-11

Good news

Very large NP-complete problems are solved in practice!

❼ SAT problems with a million variables ❼ TSP with a million variables (1000 cities)

How is this possible?

❼ Instances occurring in practice have special structures that

can be exploited.

❼ Efficient approximation algorithms sometimes exist.

Example: get within ε of optimal in polynomial time.

❼ Computers and solvers are both getting faster...

18-12

Moore good news: processors

Speedup since 1990: about 15,000

18-13

Moore good news: CPLEX

Credit: stegua.github.io 18-14

Moore good news: Gurobi

(current version: 7.5.2)

Speedup since 1990 (CPLEX+Gurobi): about 850,000

18-15

Summary

❼ Computers have improved steadily. Single-thread

computing has stalled, but now we have cloud computing, GPUs, and more...

❼ Solvers have improved steadily at a much faster rate than

computers, and continue to do so. So today’s solver on yesterday’s hardware would outperform yesterday’s solver

• n todays hardware.

❼ Total speedup since 1990: about 12.7 billion times faster.

A typical MILP that would have taken 400 years to solve in 1990 can be solved in 1 second today.

❼ Mileage may vary!

18-16

Leyffer–Linderoth–Luedtke complexity 1

1 Not meant to be taken (too) seriously

Sven Leyffer (Argonne National Lab) Jeff Linderoth (UW–Madison) Jim Luedtke (UW–Madison) “How many decision variables (n) must a problem have before everyone would be willing to pay ✩50 that a state-of-the-art solver would no longer be able to solve it?”

convex and continuous LP 5 × 107 QP 5 × 105 SOCP 105 NLP 5 × 104 convex with mixed-integer MILP 2 × 104 MIQP 1000 MISOCP 1000 MINLP 500 nonconvex mixed or cont. QP 300 MIQP 300 NLP 100 MINNLP 100

18-17

Rounding

Back to the standard IP formulation: maximize

x

cTx subject to: Ax ≤ b x ∈ Z Idea:

❼ Solve the problem for x ∈ R instead (a regular LP). ❼ Round each xi in the solution to the nearest integer. ❼ This usually does not work!

18-18

Rounding

❼ If LP solution is already integral, then it is also the exact

solution to the original IP. (e.g. min cost flow problems)

❼ Rounding can lead to an infeasible point ❼ Rounding can produce a point far from the optimal point

true optimum ( ), relaxed optimum ( ), rounded ( )

18-19

Convex relaxation

minimize

x∈S

f (x) Two ideas we will discuss:

• 1. Function relaxation: if f is troublesome, bound it with a

function that is easier to work with, e.g. a convex function.

• 2. Constraint relaxation: If S is troublesome, find a bigger set

that is easier to work with, e.g. a convex set.

18-20

Function relaxation

fopt = minimize

x∈S

f (x) Suppose we can find g such that g(x) ≤ f (x) for all x. In other words g is a lower bound on f .

• 1

1 2 3 4 x 1 2 3 4 f(x)

f(x) g(x)

18-21

Function relaxation

• 1

1 2 3 4 x 1 2 3 4 f(x)

f(x) g(x)

❼ Solve gopt = min

x∈S g(x) and let ˆ

x be the corresponding x.

❼ We have the bounds: gopt = g(ˆ

x) ≤ fopt ≤ f (ˆ x).

❼ If f (ˆ

x) = gopt then the bound is tight and fopt = f (ˆ x). Pick a convex g so that gopt and ˆ x are easy to find!

18-22

Constraint relaxation

fopt = minimize

x∈S

f (x) Suppose we can find some set C such that S ⊆ C. In other words, C is a superset of S.

• 5

5

• 4
• 2

2 4

S C

18-23

Constraint relaxation

• 5

5

• 4
• 2

2 4

S C ❼ Solve hopt = min

x∈C f (x) and let ˜

x be the optimal x.

❼ We have the bound: hopt = f (˜

x) ≤ fopt ≤ f (x) for x ∈ S.

❼ If ˜

x ∈ S then the bound is tight and fopt = f (˜ x). Pick a convex C so that hopt and ˜ x are easy to find!

18-24

Common relaxations

• 1. Boolean constraint:

x ∈ {0, 1} = ⇒ 0 ≤ x ≤ 1 If xopt is 0 or 1, relaxation is exact.

• 2. Convex equality:

f (x) = 0 = ⇒ f (x) ≤ 0 If f (xopt) = 0, relaxation is exact.

• 3. A constraint you don’t like:

x = 3 = ⇒ just remove the constraint! If xopt = 3, relaxation is exact.

18-25

Convex hull

The convex hull of a set S, written conv(S) is the smallest convex set that contains S. Equivalent definitions:

❼ The set of all affine combinations of all points in S ❼ The intersection of all convex sets containing S

18-26