of Polynomials over a Box Georgina Hall Decision Sciences, INSEAD - - PowerPoint PPT Presentation

On Convexity

• f Polynomials over a Box

Georgina Hall Decision Sciences, INSEAD Joint work with Amir Ali Ahmadi ORFE, Princeton University

1

Convexity over a box

• A box 𝑪 is a set of the form:

𝐶 = 𝑦 ∈ ℝ𝑜 𝑚𝑗 ≤ 𝑦𝑗 ≤ 𝑣𝑗, 𝑗 = 1, … , 𝑜} where 𝑚1, … , 𝑚𝑜, 𝑣1, … , 𝑣𝑜 ∈ ℝ with 𝑚𝑗 ≤ 𝑣𝑗.

• A function 𝒈 is convex over 𝑪 if

𝑔 𝜇𝑦 + 1 − 𝜇 𝑧 ≤ 𝜇𝑔 𝑦 + 1 − 𝜇 𝑔(𝑧) for any 𝑦, 𝑧 ∈ 𝐶 and 𝜇 ∈ [0,1].

• If 𝑪 is full dimensional (i.e., 𝑚𝑗 < 𝑣𝑗, 𝑗 = 1, … , 𝑜),

this is equivalent to 𝛼2𝑔 𝑦 ≽ 0, ∀𝑦 ∈ 𝐶.

2

Complexity questions

• Restrict ourselves to polynomial functions.
• Related work:

Theorem [Ahmadi, Olshevsky, Parrilo, Tsitsiklis] It is strongly NP-hard to test (global) convexity of polynomials of degree 4.

• One may hope that adding the restriction to a box could make things easier.

3

Goal: study the complexity of testing convexity of a function over a box

Our theorem

Why are we interested in convexity over a box?

4

Theorem [Ahmadi, H.] It is strongly NP-hard to test convexity of polynomials of degree 3 over a box.

• Nonconvex optimization: branch-and-bound
• Prior work:
• Sufficient conditions for convexity [Orban et

al.], [Grant et al.]

• In practice, BARON, CVX, Gurobi check

convexity of quadratics and computationally tractable sufficient conditions for convexity

Detecting Imposing

• Control theory: convex Lyapunov functions
• Statistics: convex regression

[Ahmadi and Jungers] [Chesi and Hung]

Question: What to do a reduction from? Idea: A cubic polynomial 𝑔 is convex

• ver a (full-dimensional) box 𝐶 if and
• nly if 𝛼2𝑔 𝑦 ≽ 0, ∀𝑦 ∈ 𝐶

𝛼2𝑔(𝑦) is a matrix with entries affine in 𝒚

Proof of the theorem

5

Theorem [Ahmadi, H.] It is strongly NP-hard to test convexity of polynomials of degree 3 over a box. How to prove this? In general: Theorem [Nemirovski]: Let 𝑀(𝑦) be a matrix with entries affine in 𝑦. It is NP-hard to test whether 𝑀 𝑦 ≽ 0 for all 𝑦 in a full-dimensional box 𝐶.

Generic instance I

• f a known

NP-hard problem Instance J of problem we are interested in Construct J from I Reduction

Are we done?

No!

6

Issue 1: We want to show strong NP-hardness. Nemirovski’s result shows weak NP- hardness. Issue 2: Not every affine polynomial matrix is a valid Hessian! Example: 𝑀 𝑦1, 𝑦2 = 10 2𝑦1 + 1 2𝑦1 + 1 10 . We have

𝜖𝑀11(𝑦) 𝜖𝑦2

≠

𝜖𝑀12(𝑦) 𝜖𝑦1 .

Dealing with Issue 1 (1/5)

Reminder: weak vs strong NP-hardness

• Distinction only concerns problems where input is numerical
• Max(I): largest number in magnitude that appears in the input of instance I

(numerator or denominator)

• Length(I): number of bits it takes to write down input of instance I

7

Strong Weak

• There are instances 𝐽 that are hard with

Max(𝐽)≤ 𝑞(Length(𝐽)) (𝑞 is a polynomial)

• No pseudo-polynomial algorithm possible
• Examples:

Max-Cut Sat

• The instances that are hard may contain

numbers of large magnitude (e.g., 2𝑜).

• Pseudo-polynomial algorithms possible
• Examples:

Partition Knapsack

Dealing with Issue 1 (2/5)

Why weakly NP-hard?

8

Theorem [Nemirovski]: INTERVAL-PSDNESS Let 𝑀(𝑦) be a matrix with entries affine in 𝑦. It is (weakly) NP-hard to test whether 𝑀 𝑦 ≽ 0 for all 𝑦 in a full-dimensional box 𝐶.

PARTITION: Input: 𝑏 ∈ ℝ𝑜 such that 𝑏 2 ≤ 0.1 Test: does there exist 𝑢 ∈ −1,1 𝑜 such that σ𝑗 𝑏𝑗𝑢𝑗 = 0? INTERVAL PSDNESS Construct: 𝐷 = 𝐽𝑜 − 𝑏𝑏𝑈 −1, 𝜈 = 𝑜 − 𝑒−2 𝑏 , where 𝑒 𝑏 = smallest cd of 𝑏. Take: 𝐶 = −1,1 𝑜 and 𝑀 𝑦 = 𝐷 𝑦 𝑦𝑈 𝜈 . Test: Is 𝑀 𝑦 ≽ 0 ∀𝑦 ∈ 𝐶? Show: No to PARTITION ⇔ Yes to INTERVAL PSDNESS

REDUCTION

Weakly NP-hard Operation that can make the numbers in the instance blow up Example: 𝐵 = 1 −1 1 ⋮ −1 ⋱ −1 ⋱ −1 1 but one of the entries of 𝐵−1 is 2𝑜−2!

=

𝑏1 𝑏3 𝑏10 𝑏2 𝑏4 𝑏8

Dealing with Issue 1 (3/5)

9

Theorem [Ahmadi, H.]: INTERVAL-PSDNESS Let 𝑀(𝑦) be a matrix with entries affine in 𝑦. It is strongly NP-hard to test whether 𝑀 𝑦 ≽ 0 for all 𝑦 in a full-dimensional box 𝐶.

MAX-CUT: Input: simple graph G=(V,E) with 𝑊 = 𝑜 and adj. matrix A, and a positive integer 𝑙 ≤ 𝑜2 Test: does there exist a cut in the graph of size greater or equal to 𝑙? INTERVAL PSDNESS Construct: 𝛽 =

1 𝑜+1 3 , 𝐷 = 4𝛽(𝐽𝑜 + 𝛽𝐵)

𝜈 = 𝑜 4𝛽 + 𝑙 − 1 − 1 4 𝑓𝑈𝐵𝑓 Take: 𝐶 = −1,1 𝑜 and 𝑀 𝑦 = 𝐷 𝑦 𝑦𝑈 𝜈 . Test: Is 𝑀 𝑦 ≽ 0 ∀𝑦 ∈ 𝐶? Show: No to MAX-CUT ⇔ Yes to INTERVAL PSDNESS

REDUCTION

Strongly NP-hard Taylor series of 4𝛽 𝐽 − 𝛽𝐵 −1 truncated at the first term Scaling needed so that 𝐽𝑜 − 𝛽𝐵 −1 ≈ 𝐽𝑜 + 𝛽𝐵

Preserves strong NP-hardness

Dealing with Issue 1 (4/5)

In more detail: No to MAX-CUT ⇒ Yes to INTERVAL PSDNESS

10

No cut in 𝐻 of size ≥ 𝑙 [max

𝑦∈ −1,1 𝑜 1 4 σ𝑗,𝑘 𝐵𝑗𝑘(1 − 𝑦𝑗𝑦𝑘)] ≤ 𝑙 − 1

Size of largest cut in 𝐻

⇔ ⇔

[ max

𝑦∈ −1,1 𝑜 − 1 4 𝑦𝑈𝐵𝑦] ≤ − 1 4 𝑓𝑈𝐵𝑓 + 𝑙 − 1

⇔

[ max

𝑦∈ −1,1 𝑜 1 4 𝑦𝑈

𝑜 + 1 3𝐽𝑜 − 𝐵 𝑦] ≤

𝑜 𝑜+1 3 4

−

1 4 𝑓𝑈𝐵𝑓 + 𝑙 − 1 ≔ 𝜈

𝛽 = 𝑜 + 1 3

Convex

⇔

[ max

𝑦∈[−1,1]𝑜 1 4 𝑦𝑈 𝛽𝐽𝑜 − 𝐵 𝑦] ≤ 𝜈

⇔

1 4 𝑦𝑈 𝛽𝐽𝑜 − 𝐵 𝑦 ≤ 𝜈, ∀𝑦 ∈ −1,1 𝑜

⇒

𝑦𝑈𝐷−1𝑦 ≤ 𝜈 +

1 4, ∀𝑦 ∈ −1,1 𝑜

Approximation 𝐷−1 ≈

1 4 (𝛽𝐽 − 𝐵)

Approximation error

⇒

Schur complement

𝑀 𝑦 = 𝐷 𝑦 𝑦𝑈 𝜈 +

1 4

≽ 0, ∀𝑦 ∈ −1,1 𝑜

Dealing with Issue 1 (5/5)

For converse: Yes to MAX-CUT ⇒ No to INTERVAL PSDNESS

• Initial problem studied by Nemirovski
• Of independent interest in robust control

11

There is a cut of size ≥ 𝑙: Let ො 𝑦𝑗 = ቊ 1 if node 𝑗 on one side of cut −1 if node 𝑗 on other side of cut

⇒

Similar steps to previously

⇒

ො 𝑦𝑈𝐷−1 ො 𝑦 ≥ 𝜈 + 3 4 > 𝜈 + 1 4 ∃ ො 𝑦 ∈ −1,1 𝑜 s.t. 𝑀 ො 𝑦

⇒ Corollary [Ahmadi, H.]: Let 𝑜 be an integer and let ො 𝑟𝑗𝑘, ത 𝑟𝑗𝑘 be rational numbers with ො 𝑟𝑗𝑘 ≤ ത 𝑟𝑗𝑘 and ො 𝑟𝑗𝑘 = ො 𝑟𝑘𝑗 and ത 𝑟𝑗𝑘 = ത 𝑟𝑘𝑗 for all 𝑗 = 1, … , 𝑜 and 𝑘 = 1, … , 𝑜. It is strongly NP-hard to test whether all symmetric matrices with entries in [ො 𝑟𝑗𝑘; ത 𝑟𝑗𝑘] are positive semidefinite.

Dealing with Issue 2 (1/3)

Proof: Reduction from INTERVAL PSDNESS

12

Theorem [Ahmadi, H.] CONV3BOX It is strongly NP-hard to test convexity of polynomials of degree 3 over a box.

INTERVAL PSDNESS Input: 𝑀 𝑦 , ෠ 𝐶 Test: Is 𝑀 𝑦 ≽ 0, ∀𝑦 ∈ ෠ 𝐶? Problem: How to construct a cubic polynomial 𝑔 from 𝑀(𝑦)? Idea: Want 𝛼2𝑔 𝑦 = 𝑀 𝑦 . Issue: Not all 𝑀(𝑦) are valid Hessians! Key ideas for the construction of 𝒈:

• Start with 𝒈 𝒚, 𝒛 =

𝟐 𝟑 𝒛𝑼𝑴 𝒚 𝒛

• For 𝛼2𝑔 𝑦, 𝑧 to be able to be psd when 𝑀 𝑦 ≽ 0 , we need to have

a nonzero diagonal: add

𝜷 𝟑 𝒚𝑼𝒚 to 𝑔 𝑦, 𝑧 .

• 𝑀 𝑦 and 𝐼(𝑧) do not depend on the same variable: what if

∃(𝑦, 𝑧) s.t. 𝑀 𝑦 = 0 but 𝐼 𝑧 is not? The matrix cannot be psd: add

𝜃 2 𝑧𝑈𝑧 to 𝑔 𝑦, 𝑧 .

𝛼2𝑔 𝑦, 𝑧 = 1 2 𝐼(𝑧) 1 2 𝐼 𝑧 𝑈 𝑀 𝑦 𝛼2𝑔 𝑦, 𝑧 = 𝜷𝑱𝒐 1 2 𝐼(𝑧) 1 2 𝐼 𝑧 𝑈 𝑀 𝑦 𝛼2𝑔 𝑦, 𝑧 = 𝜷𝑱𝒐 1 2 𝐼(𝑧) 1 2 𝐼 𝑧 𝑈 𝑀 𝑦 + 𝜃𝐽𝑜+1 ⇒ 𝑔 𝑦 = 1 2 𝑧𝑈𝑀 𝑦 𝑧 + 𝛽 2 𝑦𝑈𝑦 + 𝜃 2 𝑧𝑈𝑧, 𝐶 = −1,1 2𝑜+1

Dealing with Issue 2 (2/3)

Show NO to INTERVAL PSDNESS ⇒ NO to CONV3BOX. This is equivalent to: Need to leverage extra structure of 𝑀 𝑦 : 𝑀 𝑦 = 𝐷 𝑦 𝑦𝑈 𝜈 +

1 4

13

∃ ҧ 𝑦 ∈ −1,1 𝑜 s.t. 𝑀 ҧ 𝑦 ≽ 0 ⇒ ∃ ො 𝑦, ො 𝑧 ∈ −1,1 2𝑜+1 , 𝑨 s.t. 𝑨𝑈𝛼2𝑔 ො 𝑦, ො 𝑧 𝑨 < 0 𝛼2𝑔 𝑦, 𝑧 = 𝛽𝐽𝑜 𝐼(𝑧) 𝐼 𝑧 𝑈 𝐷 + 𝜃𝐽𝑜 𝑦 𝑦𝑈 𝜈 + 1 4 + 𝜃 𝛼2𝑔 ො 𝑦, ො 𝑧 = 𝛽𝐽𝑜 𝟏 𝟏 𝟏 𝑫 + 𝜃𝐽𝑜 ഥ 𝒚 𝟏 ഥ 𝒚𝑼 𝝂 + 𝟐 𝟓 + 𝜃 𝑨𝑈𝛼2𝑔 ො 𝑦, ො 𝑧 𝑨 = −𝐷−1 ҧ 𝑦 1

𝑈

𝛽𝐽𝑜 𝟏 𝟏 𝟏 𝑫 + 𝜃𝐽𝑜 ഥ 𝒚 𝟏 ഥ 𝒚𝑼 𝝂 + 𝟐 𝟓 + 𝜃 −𝐷−1 ҧ 𝑦 1 = 𝜈 + 1 4 − ҧ 𝑦𝑈𝐷−1 ҧ 𝑦 + 𝜃(1 + 𝐷−1 ҧ 𝑦 2

2)

< 𝟏 as 𝑴 ഥ 𝒚 ≽ 𝟏 Appropriately scaled so that 𝑨𝑈𝛼2𝑔 ො 𝑦, ො 𝑧 𝑨 remains <0.

Candidates: ො 𝑦 = ҧ 𝑦, ො 𝑧 = 0, 𝑨 = −𝐷−1 ҧ 𝑦 1 Candidates: ෝ 𝒚 = ഥ 𝒚, ෝ 𝒛 = 𝟏, 𝑨 = −𝐷−1 ҧ 𝑦 1 Candidates: ො 𝑦 = ҧ 𝑦, ො 𝑧 = 0, 𝒜 = 𝟏 −𝑫−𝟐ഥ 𝒚 𝟐

Dealing with Issue 2 (3/3)

Show YES to INTERVAL PSDNESS ⇒ YES to CONV3BOX. This is equivalent to: But…

14

𝑀 𝑦 ≽ 0 ∀𝑦 ∈ −1,1 𝑜 ⇒ 𝛼2𝑔 𝑦, 𝑧 = 𝛽𝐽𝑜 1 2 𝐼(𝑧) 1 2 𝐼 𝑧 𝑈 𝑀 𝑦 + 𝜃𝐽𝑜+1 ≽ 0, ∀ 𝑦, 𝑧 ∈ −1,1 2𝑜+1

⇔

≽ 𝟏 ∀𝒚 ∈ −𝟐, 𝟐 𝒐 (Assumption)

𝑴 𝒚 + 𝜽𝑱𝒐+𝟐 −

𝟐 𝟓𝜷 𝑰 𝒛 𝑼𝑰 𝒛 ≽ 0, ∀ 𝑦, 𝑧 ∈ −1,1 2𝑜+1

𝜷 chosen large enough so that ≽ 𝟏 ∀𝒛 ∈ −𝟐, 𝟐 𝒐+𝟐

𝛼2𝑔 𝑦, 𝑧 ≽ 0, ∀ 𝑦, 𝑧 ∈ −1,1 2𝑜+1 Schur

Corollary

Completely classifies the complexity of testing convexity of a polynomial 𝑔 of degree 𝑒 over a box for any integer 𝑒 ≥ 1.

15

𝒆

𝑒 = 1

𝑔 is always convex

𝑒 = 2

𝛼2𝑔(𝑦) constant

𝑒 = 3

Previous theorem (strongly NP-hard)

𝑒 = 4 and above

Strongly NP-hard

Proof sketch:

• 𝑕 𝑦1, … , 𝑦𝑜 =cubic polynomial for which testing

convexity over a box 𝐶 is hard

• 𝑔 𝑦1, … , 𝑦𝑜, 𝑦𝑜+1 = 𝑕 𝑦1, … , 𝑦𝑜 + 𝑦𝑜+1

𝑒

• ෨

𝐶 = 𝐶 × [0,1] We have 𝛼2𝑔 𝑦, 𝑦𝑜+1 = 𝛼2𝑕(𝑦) 𝑒 𝑒 − 1 𝑦𝑜+1

𝑒−2

⇒ 𝛼2𝑔 𝑦, 𝑦𝑜+1 ≽ 0 on ෨ 𝐶 ⇔ 𝛼2𝑕 𝑦 ≽ 0 on 𝐶

Summary

• Interested in testing convexity of a polynomial over a box.
• Showed that strongly NP-hard to test convexity of cubics over a box.
• Gave a complete characterization of the complexity of testing convexity
• ver a box depending on the degree of the polynomial.
• In the process, strengthened a result on the complexity of testing

positive semidefiniteness of symmetric matrices with entries belonging to intervals.

16

Thank you for listening

Questions? Want to learn more? https://scholar.princeton.edu/ghall

17