Subsection 2 NP -hardness 36 / 109 NP -Hardness Do hard problems - PowerPoint PPT Presentation

Subsection 2 NP -hardness 36 / 109

NP -Hardness ◮ Do hard problems exist? Depends on P � = NP ◮ Next best thing: define hardest problem in NP ◮ A problem P is NP -hard if Every problem Q in NP can be solved in this way: 1. given an instance q of Q transform it in polytime to an instance ρ ( q ) of P s.t. q is YES iff ρ ( q ) is YES 2. run the best algorithm for P on ρ ( q ) , get answer α ∈ { YES , NO } 3. return α ρ is called a polynomial reduction from Q to P ◮ If P is in NP and is NP -hard, it is called NP -complete ◮ Every problem in NP reduces to sat [Cook 1971] 37 / 109

Cook’s theorem Definition of TM dynamics in CNF Boolean decision variables store TM dynamics Description of a dynamical system using a declarative program- ming language ( sat ) — what MP is all about! 38 / 109

Reduction graph After Cook’s theorem To prove NP -hardness of a new problem P , pick a known NP -hard problem Q that “looks similar enough” to P and find a polynomial reduction ρ from Q to P [Karp 1972] Why it works : suppose P easier than Q , solve Q by calling ρ ◦ Alg P , conclude Q as easy as P , contradiction 39 / 109

Example of polynomial reduction ◮ stable : given G = ( V, E ) and k ∈ N , does it contain a stable set of size k ? ◮ We know k -clique is NP -complete, reduce from it ◮ Given instance ( G, k ) of clique consider the complement graph (computable in polytime) G = ( V, ¯ ¯ E = {{ i, j } | i, j ∈ V ∧ { i, j } �∈ E } ) ◮ Thm. : G has a clique of size k iff ¯ G has a stable set of size k ◮ ρ ( G ) = ¯ G is a polynomial reduction from clique to stable ◮ ⇒ stable is NP -hard ◮ stable is also in NP U ⊆ V is a stable set iff E ( G [ U ]) = ∅ (polytime verification) ◮ ⇒ stable is NP -complete 40 / 109

MILP is NP -hard ◮ sat is NP -hard by Cook’s theorem, Reduce from sat in CNF � � ℓ j i ≤ m j ∈ C i where ℓ j is either x j or ¯ x j ≡ ¬ x j ◮ Polynomial reduction ρ ¯ ∨ ∧ sat x j x j MILP 1 − x j + ≥ 1 x j ◮ E.g. ρ maps ( x 1 ∨ x 2 ) ∧ (¯ x 2 ∨ x 3 ) to min { 0 | x 1 + x 2 ≥ 1 ∧ x 3 − x 2 ≥ 0 ∧ x ∈ { 0 , 1 } 3 } ◮ sat is YES iff MILP is feasible (same solution, actually) 41 / 109

Complexity of Quadratic Programming x ⊤ Qx c ⊤ x � min + ≥ Ax b ◮ Quadratic Programming = QP ◮ Quadratic objective, linear constraints, continuous variables ◮ Many applications (e.g. portfolio selection) ◮ If Q PSD then objective is convex, problem is in P ◮ If Q has at least one negative eigenvalue, NP -hard ◮ Decision problem: “is the min. obj. fun. value = 0 ?” 42 / 109

QP is NP -hard ◮ By reduction from SAT , let σ be an instance ρ ( σ, x ) ≥ 1 : linear constraints of sat → MILP reduction ◮ ˆ ◮ Consider QP min f ( x ) = � x j (1 − x j )   j ≤ n  ( † ) ρ ( σ, x ) ≥ 1 ˆ  0 ≤ x ≤ 1  ◮ Claim : σ is YES iff val ( † ) = 0 ◮ Proof: ◮ assume σ YES with soln. x ∗ , then x ∗ ∈ { 0 , 1 } n , hence f ( x ∗ ) = 0 , since f ( x ) ≥ 0 for all x , val ( † ) = 0 ◮ assume σ NO, suppose val ( † ) = 0 , then ( † ) feasible with soln. x ′ , since f ( x ′ ) = 0 then x ′ ∈ { 0 , 1 } , feasible in sat hence σ is YES, contradiction 43 / 109

Box-constrained QP is NP -hard ◮ Add surplus vars v to sat → MILP constraints: ρ ( σ, x ) − 1 − v = 0 ˆ (denote by ∀ i ≤ m ( a ⊤ i x − b i − v i = 0) ) ◮ Now sum them on the objective ( a ⊤ i x − b i − v i ) 2 � x j (1 − x j ) + � min � j ≤ n i ≤ m 0 ≤ x ≤ 1 , v ≥ 0 ◮ Issue: v not bounded above ◮ Reduce from 3sat , get ≤ 3 literals per clause ⇒ can consider 0 ≤ v ≤ 2 44 / 109

cQKP is NP -hard ◮ continuous Quadratic Knapsack Problem (cQKP) f ( x ) = x ⊤ Qx c ⊤ x min +   � =  a j x j γ j ≤ n  [0 , 1] n , ∈ x  ◮ Reduction from subset-sum given list a ∈ Q n and γ , is there J ⊆ { 1 , . . . , n } s.t. � a j = γ ? j ∈ J reduce to f ( x ) = � j x j (1 − x j ) ◮ σ is a YES instance of subset-sum ◮ let x ∗ j = 1 iff j ∈ J , x ∗ j = 0 otherwise ◮ feasible by construction ◮ f is non-negative on [0 , 1] n and f ( x ∗ ) = 0 : optimum ◮ σ is a NO instance of subset-sum ◮ suppose opt ( cQKP ) = x ∗ s.t. f ( x ∗ ) = 0 ◮ then x ∗ ∈ { 0 , 1 } n because f ( x ∗ ) = 0 ◮ feasibility of x ∗ → supp ( x ∗ ) solves σ , contradiction, hence f ( x ∗ ) > 0 45 / 109

QP on a simplex is NP -hard f ( x ) = x ⊤ Qx c ⊤ x  min +  �  x j = 1 j ≤ n  ∀ j ≤ n ≥ 0 x j  ◮ Reduce max clique to subclass f ( x ) = − � x i x j { i,j }∈ E Motzkin-Straus formulation (MSF) ◮ Theorem [Motzkin& Straus 1964] Let C be the maximum clique of the instance G = ( V, E ) of max clique � � ∃ x ∗ ∈ opt ( MSF ) f ∗ = f ( x ∗ ) = 1 1 1 − 2 ω ( G ) 1 if j ∈ C � x ∗ ω ( G ) ∀ j ∈ V j = otherwise 0 46 / 109

Proof of the Motzkin-Straus theorem x ∗ = opt ( max x i x j ) s.t. | C = { j ∈ V | ; x ∗ j > 0 }| smallest ( ‡ ) � x ∈ [0 , 1] n ij ∈ E � j xj =1 1. C is a clique ◮ Suppose 1 , 2 ∈ C but { 1 , 2 } �∈ E [ C ] , then x ∗ 1 , x ∗ 2 > 0 , can perturb by small 2 ] , get x ǫ = ( x ∗ ǫ ∈ [ − x ∗ 1 , x ∗ 1 + ǫ, x ∗ 2 − ǫ, . . . ) , feasible w.r.t. simplex and bounds ◮ { 1 , 2 } �∈ E ⇒ x 1 x 2 does not appear in f ( x ) ⇒ f ( x ǫ ) depends linearly on ǫ ; by optimality of x ∗ , f achieves max for ǫ = 0 , in interior of its range ⇒ f ( ǫ ) constant ◮ set ǫ = − x ∗ 1 or = x ∗ 2 yields global optima with more zero components than x ∗ , against assumption ( ‡ ), hence { 1 , 2 } ∈ E [ C ] , by relabeling C is a clique 47 / 109

Proof of the Motzkin-Straus theorem x ∗ = opt ( max x i x j ) s.t. | C = { j ∈ V | ; x ∗ j > 0 }| smallest ( ‡ ) � x ∈ [0 , 1] n ij ∈ E � j xj =1 2. | C | = ω ( G ) ◮ square simplex constraint � j x j = 1 , get � � x 2 j + 2 x i x j = 1 j ∈ V i<j ∈ V ◮ by construction x ∗ j = 0 for j �∈ C ⇒ j ) 2 + 2 j ) 2 + 2 f ( x ∗ ) = 1 ψ ( x ∗ ) = � ( x ∗ � x ∗ j x ∗ � ( x ∗ j = j ∈ C i<j ∈ C j ∈ C j ) 2 is ◮ ψ ( x ) = 1 for all feasible x , so f ( x ) achieves maximum when � j ∈ C ( x ∗ minimum, i.e. x ∗ | C | for all j ∈ C 1 j = ◮ again by simplex constraint 1 1 j ) 2 = 1 − | C | f ( x ∗ ) = 1 − � ( x ∗ | C | 2 ≤ 1 − ω ( G ) j ∈ C so f ( x ∗ ) attains maximum 1 − 1 /ω ( G ) when | C | = ω ( G ) 48 / 109

Portfolio optimization You, a private investment banker, are seeing a customer. She tells you “I have 3,450,000$ I don’t need in the next three years. Invest them in low-risk assets so I get at least 2.5% re- turn per year.” Model the problem of determining the required portfolio. Missing data are part of the fun (and of real life). [Hint: what are the decision variables, objective, constraints? What data are missing?] 49 / 109