of Polynomials over a Box Georgina Hall Decision Sciences, INSEAD - PowerPoint PPT Presentation

On Convexity of Polynomials over a Box Georgina Hall Decision Sciences, INSEAD Joint work with Amir Ali Ahmadi ORFE, Princeton University 1

Convexity over a box • A box 𝑪 is a set of the form: 𝐶 = 𝑦 ∈ ℝ 𝑜 𝑚 𝑗 ≤ 𝑦 𝑗 ≤ 𝑣 𝑗 , 𝑗 = 1, … , 𝑜} where 𝑚 1 , … , 𝑚 𝑜 , 𝑣 1 , … , 𝑣 𝑜 ∈ ℝ with 𝑚 𝑗 ≤ 𝑣 𝑗 . • A function 𝒈 is convex over 𝑪 if 𝑔 𝜇𝑦 + 1 − 𝜇 𝑧 ≤ 𝜇𝑔 𝑦 + 1 − 𝜇 𝑔(𝑧) for any 𝑦, 𝑧 ∈ 𝐶 and 𝜇 ∈ [0,1]. • If 𝑪 is full dimensional (i.e., 𝑚 𝑗 < 𝑣 𝑗 , 𝑗 = 1, … , 𝑜 ), this is equivalent to 𝛼 2 𝑔 𝑦 ≽ 0, ∀𝑦 ∈ 𝐶. 2

Complexity questions Goal: study the complexity of testing convexity of a function over a box • Restrict ourselves to polynomial functions. • Related work: Theorem [Ahmadi, Olshevsky, Parrilo, Tsitsiklis] It is strongly NP-hard to test (global) convexity of polynomials of degree 4. • One may hope that adding the restriction to a box could make things easier. 3

Our theorem Theorem [Ahmadi, H.] It is strongly NP-hard to test convexity of polynomials of degree 3 over a box. Why are we interested in convexity over a box? Detecting Imposing • Nonconvex optimization: branch-and-bound • Control theory: convex Lyapunov functions [Ahmadi and Jungers] [Chesi and Hung] • Prior work: • Sufficient conditions for convexity [Orban et • Statistics: convex regression al.], [Grant et al.] • In practice, BARON, CVX, Gurobi check convexity of quadratics and computationally tractable sufficient conditions for convexity 4

Proof of the theorem Theorem [Ahmadi, H.] It is strongly NP-hard to test convexity of polynomials of degree 3 over a box. How to prove this? Construct Generic instance I Instance J of J from I In general: of a known problem we are Reduction NP-hard problem interested in 𝛼 2 𝑔(𝑦) is a matrix Question: What to Idea: A cubic polynomial 𝑔 is convex do a reduction over a (full-dimensional) box 𝐶 if and with entries affine only if 𝛼 2 𝑔 𝑦 ≽ 0 , ∀𝑦 ∈ 𝐶 from? in 𝒚 Theorem [Nemirovski]: Let 𝑀(𝑦) be a matrix with entries affine in 𝑦 . It is NP-hard to test whether 𝑀 𝑦 ≽ 0 for all 𝑦 in a full-dimensional box 𝐶. 5

Are we done? No! Issue 1: We want to show strong NP-hardness. Nemirovski’s result shows weak NP- hardness. Issue 2: Not every affine polynomial matrix is a valid Hessian! 10 2𝑦 1 + 1 𝜖𝑀 11 (𝑦) 𝜖𝑀 12 (𝑦) Example: 𝑀 𝑦 1 , 𝑦 2 = . We have ≠ 𝜖𝑦 1 . 2𝑦 1 + 1 10 𝜖𝑦 2 6

Dealing with Issue 1 (1/5) Reminder: weak vs strong NP-hardness • Distinction only concerns problems where input is numerical • Max(I) : largest number in magnitude that appears in the input of instance I (numerator or denominator) • Length(I) : number of bits it takes to write down input of instance I Strong Weak • There are instances 𝐽 that are hard with • The instances that are hard may contain Max( 𝐽 ) ≤ 𝑞 (Length( 𝐽 )) ( 𝑞 is a polynomial) numbers of large magnitude (e.g., 2 𝑜 ) . • No pseudo-polynomial algorithm possible • Pseudo-polynomial algorithms possible • Examples: • Examples: Knapsack Sat Partition Max-Cut 7

Dealing with Issue 1 (2/5) Theorem [Nemirovski]: INTERVAL-PSDNESS Let 𝑀(𝑦) be a matrix with entries affine in 𝑦 . It is (weakly) NP-hard to test whether 𝑀 𝑦 ≽ 0 for all 𝑦 in a full-dimensional box 𝐶. Why weakly NP-hard? INTERVAL PSDNESS PARTITION: Construct : 𝐷 = 𝐽 𝑜 − 𝑏𝑏 𝑈 −1 , REDUCTION Input : 𝑏 ∈ ℝ 𝑜 such that 𝑏 2 ≤ 0.1 𝜈 = 𝑜 − 𝑒 −2 𝑏 , where 𝑒 𝑏 = smallest cd of 𝑏. 𝐷 𝑦 Take: 𝐶 = −1,1 𝑜 and 𝑀 𝑦 = Test: does there exist 𝑢 ∈ −1,1 𝑜 𝜈 . 𝑦 𝑈 such that σ 𝑗 𝑏 𝑗 𝑢 𝑗 = 0? Test: Is 𝑀 𝑦 ≽ 0 ∀𝑦 ∈ 𝐶? Show: No to PARTITION ⇔ Yes to INTERVAL PSDNESS Weakly NP-hard Operation that can make the numbers in the instance blow up 1 0 0 0 −1 1 0 0 𝑏 1 𝑏 2 𝑏 4 but one of the entries of 𝐵 −1 is 2 𝑜−2 ! 𝑏 10 = Example: 𝐵 = 𝑏 8 ⋱ ⋱ ⋮ 0 𝑏 3 8 −1 −1 −1 1

Dealing with Issue 1 (3/5) Theorem [Ahmadi, H.]: INTERVAL-PSDNESS Let 𝑀(𝑦) be a matrix with entries affine in 𝑦 . It is strongly NP-hard to test whether 𝑀 𝑦 ≽ 0 for all 𝑦 in a full-dimensional box 𝐶. INTERVAL PSDNESS MAX-CUT: 1 Construct : 𝛽 = 𝑜+1 3 , 𝐷 = 4𝛽(𝐽 𝑜 + 𝛽𝐵) 𝜈 = 𝑜 4𝛽 + 𝑙 − 1 − 1 Input : simple graph G=(V,E) with 4 𝑓 𝑈 𝐵𝑓 REDUCTION 𝑊 = 𝑜 and adj. matrix A, and a 𝐷 𝑦 positive integer 𝑙 ≤ 𝑜 2 Take: 𝐶 = −1,1 𝑜 and 𝑀 𝑦 = Preserves strong 𝜈 . 𝑦 𝑈 NP-hardness Test: does there exist a cut in the Test: Is 𝑀 𝑦 ≽ 0 ∀𝑦 ∈ 𝐶? graph of size greater or equal to 𝑙? Show: No to MAX-CUT ⇔ Yes to INTERVAL PSDNESS Taylor series of 4𝛽 𝐽 − 𝛽𝐵 −1 truncated at the first term Strongly NP-hard Scaling needed so that 𝐽 𝑜 − 𝛽𝐵 −1 ≈ 𝐽 𝑜 + 𝛽𝐵 9

Dealing with Issue 1 (4/5) In more detail: No to MAX-CUT ⇒ Yes to INTERVAL PSDNESS 1 4 σ 𝑗,𝑘 𝐵 𝑗𝑘 (1 − 𝑦 𝑗 𝑦 𝑘 )] ≤ 𝑙 − 1 [max ⇔ No cut in 𝐻 of size ≥ 𝑙 𝑦∈ −1,1 𝑜 ⇔ Convex Size of largest cut in 𝐻 𝑜 𝑜+1 3 1 1 1 4 𝑦 𝑈 𝐵𝑦 ] ≤ − 4 𝑓 𝑈 𝐵𝑓 + 𝑙 − 1 4 𝑦 𝑈 𝑜 + 1 3 𝐽 𝑜 − 𝐵 𝑦 ] ≤ ⇔ 𝑦∈ −1,1 𝑜 − max max − [ [ 𝑦∈ −1,1 𝑜 4 1 4 𝑓 𝑈 𝐵𝑓 + 𝑙 − 1 ≔ 𝜈 ⇔ 𝛽 = 𝑜 + 1 3 1 4 𝑦 𝑈 𝛽𝐽 𝑜 − 𝐵 𝑦 ] ≤ 𝜈 1 4 𝑦 𝑈 𝛽𝐽 𝑜 − 𝐵 𝑦 ≤ 𝜈 , ∀𝑦 ∈ −1,1 𝑜 ⇔ [ max 𝑦∈[−1,1] 𝑜 Approximation 𝐷 −1 ≈ 1 4 (𝛽𝐽 − 𝐵) ⇒ 𝐷 𝑦 1 ≽ 0, ∀𝑦 ∈ −1,1 𝑜 𝑦 𝑈 𝐷 −1 𝑦 ≤ 𝜈 + 4 , ∀𝑦 ∈ −1,1 𝑜 ⇒ 𝑀 𝑦 = 1 𝑦 𝑈 𝜈 + Schur 4 Approximation error complement 10

Dealing with Issue 1 (5/5) For converse: Yes to MAX-CUT ⇒ No to INTERVAL PSDNESS There is a cut of size ≥ 𝑙: 𝑦 ≥ 𝜈 + 3 4 > 𝜈 + 1 Similar steps 𝑦 𝑈 𝐷 −1 ො ⇒ ⇒ ො 𝑦 𝑗 = ቊ 1 if node 𝑗 on one side of cut Let ො to previously 4 −1 if node 𝑗 on other side of cut ⇒ 𝑦 ∈ −1,1 𝑜 s.t. 𝑀 ො ∃ ො 𝑦 0 Corollary [Ahmadi, H.]: Let 𝑜 be an integer and let ො 𝑟 𝑗𝑘 , ത 𝑟 𝑗𝑘 be rational numbers with ො 𝑟 𝑗𝑘 ≤ ത 𝑟 𝑗𝑘 and ො 𝑟 𝑗𝑘 = ො 𝑟 𝑘𝑗 and ത 𝑟 𝑗𝑘 = ത 𝑟 𝑘𝑗 for all 𝑗 = 1, … , 𝑜 and 𝑘 = 1, … , 𝑜. It is strongly NP-hard to test whether all symmetric matrices with entries in [ො 𝑟 𝑗𝑘 ; ത 𝑟 𝑗𝑘 ] are positive semidefinite. • Initial problem studied by Nemirovski • Of independent interest in robust control 11

Dealing with Issue 2 (1/3) Theorem [Ahmadi, H.] CONV3BOX It is strongly NP-hard to test convexity of polynomials of degree 3 over a box. Proof: Reduction from INTERVAL PSDNESS INTERVAL PSDNESS Problem: How to construct a cubic polynomial 𝑔 from 𝑀(𝑦) ? Input : 𝑀 𝑦 , ෠ Idea: Want 𝛼 2 𝑔 𝑦 = 𝑀 𝑦 . 𝐶 Test: Is 𝑀 𝑦 ≽ 0, ∀𝑦 ∈ ෠ 𝐶? Issue: Not all 𝑀(𝑦) are valid Hessians! Key ideas for the construction of 𝒈: 𝟐 𝟑 𝒛 𝑼 𝑴 𝒚 𝒛 • Start with 𝒈 𝒚, 𝒛 = 1 1 1 • For 𝛼 2 𝑔 𝑦, 𝑧 to be able to be psd when 𝑀 𝑦 ≽ 0 , we need to have 𝜷𝑱 𝒐 𝜷𝑱 𝒐 0 2 𝐼(𝑧) 2 𝐼(𝑧) 2 𝐼(𝑧) 𝛼 2 𝑔 𝑦, 𝑧 = 𝛼 2 𝑔 𝑦, 𝑧 = 𝛼 2 𝑔 𝑦, 𝑧 = 𝜷 𝟑 𝒚 𝑼 𝒚 to 𝑔 𝑦, 𝑧 . 1 1 1 a nonzero diagonal: add 2 𝐼 𝑧 𝑈 2 𝐼 𝑧 𝑈 2 𝐼 𝑧 𝑈 𝑀 𝑦 + 𝜃𝐽 𝑜+1 𝑀 𝑦 𝑀 𝑦 • 𝑀 𝑦 and 𝐼(𝑧) do not depend on the same variable: what if ∃(𝑦, 𝑧) s.t. 𝑀 𝑦 = 0 but 𝐼 𝑧 is not? The matrix cannot be psd: add 𝜃 2 𝑧 𝑈 𝑧 to 𝑔 𝑦, 𝑧 . ⇒ 𝑔 𝑦 = 1 2 𝑧 𝑈 𝑀 𝑦 𝑧 + 𝛽 2 𝑦 𝑈 𝑦 + 𝜃 2 𝑧 𝑈 𝑧, 𝐶 = −1,1 2𝑜+1 12

ҧ Dealing with Issue 2 (2/3) Show NO to INTERVAL PSDNESS ⇒ NO to CONV3BOX. This is equivalent to: 𝑧 ∈ −1,1 2𝑜+1 , 𝑨 s.t. 𝑨 𝑈 𝛼 2 𝑔 ො 𝑦 ∈ −1,1 𝑜 s.t. 𝑀 ∃ ҧ 𝑦 ≽ 0 ⇒ ∃ ො 𝑦, ො 𝑦, ො 𝑧 𝑨 < 0 𝐷 𝑦 Need to leverage extra structure of 𝑀 𝑦 : 𝑀 𝑦 = 1 𝑦 𝑈 𝜈 + 4 0 𝟏 0 −𝐷 −1 ҧ −𝐷 −1 ҧ −𝑫 −𝟐 ഥ Candidates: ෝ Candidates : ො Candidates: ො 𝒚 = ഥ 𝑦 = ҧ 𝑦 = ҧ 𝑦, 𝑦, 𝒚, 𝑧 = 0, 𝑧 = 0, 𝒛 = 𝟏, ෝ ො ො 𝒜 = 𝑨 = 𝑨 = 𝑦 𝒚 𝑦 1 𝟐 1 𝛽𝐽 𝑜 𝟏 𝟏 𝑈 𝛽𝐽 𝑜 𝛽𝐽 𝑜 𝟏 𝐼(𝑧) 𝟏 0 0 = 𝜈 + 1 𝟏 𝑫 + 𝜃𝐽 𝑜 𝒚 ഥ 2 ) 𝑦 𝑈 𝐷 −1 ҧ 𝑦 + 𝜃(1 + 𝐷 −1 ҧ −𝐷 −1 ҧ −𝐷 −1 ҧ 𝑨 𝑈 𝛼 2 𝑔 ො 𝑦, ො 𝑧 𝑨 = 𝟏 𝑫 + 𝜃𝐽 𝑜 𝐷 + 𝜃𝐽 𝑜 ഥ 𝒚 𝑦 4 − ҧ 𝑦 2 𝑦 𝑦 𝛼 2 𝑔 𝑦, 𝑧 = 𝛼 2 𝑔 ො 𝝂 + 𝟐 𝑦, ො 𝑧 = 𝐼 𝑧 𝑈 𝒚 𝑼 𝝂 + 𝟐 𝜈 + 1 1 1 𝟏 ഥ 𝟓 + 𝜃 𝒚 𝑼 𝑦 𝑈 𝟏 ഥ 𝟓 + 𝜃 4 + 𝜃 Appropriately scaled so that < 𝟏 as 𝑴 ഥ 𝒚 ≽ 𝟏 𝑨 𝑈 𝛼 2 𝑔 ො 𝑦, ො 𝑧 𝑨 remains <0. 13