Lecture 5: SOS Proofs and the Motzkin Polynomial Lecture Outline - PowerPoint PPT Presentation

Lecture 5: SOS Proofs and the Motzkin Polynomial

Lecture Outline • Part I: SOS proofs and examples • Part II: Motzkin Polynomial

Part I: SOS proofs and examples

SOS proofs • Fundamental question: What can we say about the pseudo-expectation values SOS gives us? • In other words, which statements that are true for any expectation of an actual distribution of solutions must also be true for pseudo- expectation values?

Non-negativity of Squares • Trivial but extremely useful: If 𝑔 is a sum of 2 then ෨ squares i.e. 𝑔 = σ 𝑘 𝑕 𝑘 𝐹 𝑔 ≥ 0 • Example: If 𝑔 = 𝑦 2 − 4𝑦 + 5 then ෨ 𝐹 𝑔 ≥ 0 as 𝑔 = 𝑦 − 2 2 + 1 . In fact, ෨ 𝐹 𝑔 ≥ 1

Single Variable Polynomials • Theorem: For a single-variable polynomial p(x) , 𝑞(𝑦) is non-negative ⬄ 𝑞(𝑦) is a sum of squares. • Proof: By induction on the degree 𝑒 • Base case 𝑒 = 0 is trivial • If 𝑒 > 0 , let 𝑑 ≥ 0 be the minimal value of 𝑞(𝑦) and let 𝑏 be a zero of 𝑞 𝑦 − 𝑑 . Since 𝑞 𝑦 − 𝑑 is non- negative, it has a zero of order 2𝑙 at 𝑏 for some integer 𝑙 ≥ 1 (the order must be even). • Write 𝑞 = 𝑦 − 𝑏 2𝑙 𝑞 ′ + 𝑑 where 𝑞 ′ = 𝑞−𝑑 𝑦−𝑏 2𝑙 is non-negative and thus a sum of squares.

Degree 2 Polynomials • Given a degree 2 polynomial 𝑔 , we can write 𝑔 𝑦 1 , 𝑦 2 , … , 𝑦 𝑜 = σ 𝑗,𝑘 𝑑 𝑗𝑘 𝑦 𝑗 𝑦 𝑘 where c 𝑘𝑗 = 𝑑 𝑗𝑘 for all 𝑗 and 𝑘 . • Taking 𝑁 to be the coefficient matrix where 𝑈 where 𝑁 𝑗𝑘 = 𝑑 𝑗𝑘 , we can write 𝑁 = σ 𝑗 𝜇 𝑗 𝑤 𝑗 𝑤 𝑗 the {𝑤 𝑗 } are orthonormal. Now 𝑔 𝑦 = 𝑦 𝑈 𝑁𝑦 . 1. 2 𝑈 𝑦 = σ 𝑗 𝜇 𝑗 σ 𝑘=1 𝑜 𝑔(𝑦) = σ 𝑗 𝜇 𝑗 𝑦 𝑈 𝑤 𝑗 𝑤 𝑗 2. 𝑤 𝑗𝑘 𝑦 𝑘

Degree 2 Polynomials • We have that 𝑈 where the {𝑤 𝑗 } are orthonormal. 𝑁 = σ 𝑗 𝜇 𝑗 𝑤 𝑗 𝑤 𝑗 1. 𝑔 𝑦 = 𝑦 𝑈 𝑁𝑦 2. 2 𝑜 𝑔 = σ 𝑗 𝜇 𝑗 σ 𝑘=1 𝑤 𝑗𝑘 𝑦 𝑘 3. • If 𝑁 ≽ 0 then ∀𝑗, 𝜇 𝑗 ≥ 0 so 𝑔 is a sum of squares • If 𝑁 is not PSD then 𝜇 𝑗 < 0 for some 𝑗 . Taking 𝑈 𝑁𝑤 𝑗 < 0 so 𝑔 is not non- 𝑦 = 𝑤 𝑗 , 𝑔 𝑦 = 𝑤 𝑗 negative. • Thus if deg 𝑔 = 2 , 𝑔 is non-negative ⬄𝑔 is SOS

Cauchy Schwarz Inequality • Cauchy-Schwarz inequality: 2 ≤ σ 𝑗 𝑔 2 2 σ 𝑗 𝑔 σ 𝑗 𝑕 𝑗 𝑗 𝑕 𝑗 𝑗 • Extremely useful • Proof: Consider 𝑔 and 𝑕 as vectors. Cauchy- Schwarz is equivalent to 𝑔 ⋅ 𝑕 2 ≤ 𝑔 2 𝑕 2 • This is true as 𝑔 ⋅ 𝑕 2 = 𝑔 2 𝑕 2 cos 2 Θ where Θ is the angle between 𝑔 and 𝑕 . • How about an SOS proof?

Cauchy Schwarz: SOS Proof 2 ≤ σ 𝑗 𝑔 2 2 • Cauchy-Schwarz: σ 𝑗 𝑔 σ 𝑗 𝑕 𝑗 𝑗 𝑕 𝑗 𝑗 • Building block: For all 𝑗 and 𝑘 , 2 = 𝑔 2 + 𝑔 2 − 2𝑔 2 𝑕 𝑘 2 𝑕 𝑗 𝑔 𝑗 𝑕 𝑘 − 𝑔 𝑘 𝑕 𝑗 𝑗 𝑕 𝑗 𝑔 𝑘 𝑕 𝑘 ≥ 0 𝑗 𝑘 • Note that: 2 + 𝑔 2 − σ 𝑗 𝑔 2 𝑕 𝑘 2 𝑕 𝑗 2 ) = σ 𝑗 𝑔 2 2 𝑕 𝑗 2 σ 𝑗<𝑘 (𝑔 σ 𝑗 𝑕 𝑗 1. 𝑗 𝑘 𝑗 𝑗 2 − σ 𝑗 𝑔 2 𝑕 𝑗 2 2 σ 𝑗<𝑘 (𝑔 𝑘 𝑕 𝑘 ) = σ 𝑗 𝑔 𝑗 𝑕 𝑗 𝑔 𝑗 𝑕 𝑗 2. 𝑗 2 = • Final proof: σ 𝑗,𝑘:𝑗<𝑘 𝑔 𝑗 𝑕 𝑘 − 𝑔 𝑘 𝑕 𝑗 2 ≥ 0 2 − σ 𝑗 𝑔 2 σ 𝑗 𝑔 σ 𝑗 𝑕 𝑗 𝑗 𝑕 𝑗 𝑗

SOS Proofs With Constraints • What if we also have constraints 𝑡 1 𝑦 1 , … , 𝑦 𝑜 = 0, 𝑡 2 𝑦 1 , … , 𝑦 𝑜 = 0 , etc.? • An SOS proof that ℎ ≥ 𝑑 now takes the form 2 ℎ = 𝑑 + σ 𝑗 𝑔 𝑗 𝑡 𝑗 + σ 𝑘 𝑕 𝑘 • Example: If 𝑦 2 = 1 then x ≥ −1 . Proof: 𝑦 + 1 = 𝑦 2 2 + 𝑦 + 1 2 = 1 2 𝑦 + 1 2 ≥ 0

Combining Proofs • If there is an SOS proof of degree 𝑒 1 that 𝑔 ≥ 0 and an SOS proof of degree 𝑒 2 that 𝑕 ≥ 0 then: 1. There is an SOS proof of degree 𝑛𝑏𝑦{𝑒 1 , 𝑒 2 } that 𝑔 + 𝑕 ≥ 0 2. There is an SOS proof of degree 𝑒 1 + 𝑒 2 that 𝑔𝑕 ≥ 0

Products of Pseudo-expectation Values • What if our statements involve products of pseudo-expectation values? • Example: We showed that 2 ≤ ෨ ෨ 2 2 σ 𝑗 𝑔 σ 𝑗 𝑔 σ 𝑗 𝑕 𝑗 𝐹 𝑗 𝑕 𝑗 𝐹 𝑗 What if we instead want to show that 2 ≤ ෨ 2 ෨ 2 ? ෨ 𝐹 σ 𝑗 𝑔 𝐹 σ 𝑗 𝑔 𝐹 σ 𝑗 𝑕 𝑗 𝑗 𝑕 𝑗 𝑗 • Requires modified proof, see problem set • Can often prove such statements by using ෨ 𝐹 values as constants in the proof.

Example: Variance • For any random variable 𝑦 , 𝐹 𝑦 2 ≥ 𝐹 𝑦 2 • Also true for pseudo-expectation values, i.e. for 2 𝐹 𝑔 2 ≥ any polynomial 𝑔 , ෨ ෨ 𝐹 𝑔 • Proof: Given ෨ 𝐹 , let 𝑑 = ෨ 𝐹[𝑔] and observe that 𝑔 − 𝑑 2 = ෨ 𝐹 𝑔 2 − 2𝑑 ෨ ෨ 𝐹 𝑔 + c 2 𝐹 2 ≥ 0 𝐹 𝑔 2 − = ෨ ෨ 𝐹 𝑔

In-class exercises 𝐹 𝑦 4 − 4𝑦 + 3 ≥ 0 1. Prove that ෨ 2. Prove that 𝐹 𝑦 2 + 2𝑧 2 + 6𝑨 2 + 2𝑦𝑧 + 2𝑦𝑨 + 6𝑧𝑨 ≥ 0 ෨ 3. Prove that if 𝑦 2 + 𝑧 2 = 1 then 𝑦 + 𝑧 ≤ 2 𝐹 𝑦 2 = 0 then for any function 𝑔 4. Prove that if ෨ of degree at most 𝑒 2 , ෨ 𝐹 𝑦𝑔 = 0 .

In-class exercise answers 𝐹 𝑦 4 − 4𝑦 + 3 ≥ 0 1. Prove that ෨ Answer: 𝑦 4 − 4𝑦 + 3 = 𝑦 − 1 2 𝑦 2 + 2𝑦 + 3 = 𝑦 − 1 2 ( 𝑦 + 1 2 + 2)

In-class exercise answers 2. Prove that 𝐹 𝑦 2 + 2𝑧 2 + 6𝑨 2 + 2𝑦𝑧 + 2𝑦𝑨 + 6𝑧𝑨 ≥ 0 ෨ Answer: The coefficient matrix for this 1 1 1 polynomial is M = 1 2 3 1 3 6 One non-orthonormal factorization is 𝑁 = 𝑈 + 𝑤 2 𝑤 2 𝑈 + 𝑤 3 𝑤 3 𝑈 = [1 𝑈 where 𝑤 1 𝑤 1 𝑤 1 1] , 1 𝑈 = [0 𝑈 = [0 𝑤 2 2] , 𝑤 3 1] , 1 0

In-class exercise answers This gives us that 𝑦 2 + 2𝑧 2 + 6𝑨 2 + 2𝑦𝑧 + 2𝑦𝑨 + 6𝑧𝑨 = 𝑦 + 𝑧 + 𝑨 2 + 𝑧 + 2𝑨 2 + 𝑨 2

In-class exercise answers 3. Prove that if we have the constraint 𝑦 2 + 𝑧 2 = 1 then ෨ 𝐹 𝑦 + 𝑧 ≤ 2 𝑦 2 +𝑧 2 1 Answer: 2 − 𝑦 − 𝑧 = − 𝑦 − 𝑧 + 2 = 2 𝑦−𝑧 2 2 2 + 𝑦+𝑧 2 2 2 − 𝑦 − 𝑧 + 1 2 = 2 ≥ 0 𝑦−𝑧 2 1 2 2 + 2 2 𝑦 + 𝑧 − 2

In-class exercise answers 𝐹 𝑦 2 = 0 then for any function 𝑔 4. Prove that if ෨ 𝑒 2 − 1 , ෨ of degree at most 𝐹 𝑦𝑔 = 0 . Answer: Observe that for any constant 𝐷 , 𝑔 − 𝐷𝑦 2 = ෨ 𝐹 𝑔 2 − 2𝐷 ෨ 𝐹 𝑦 2 = ෨ 𝐹 𝑦𝑔 + ෨ 𝐹 𝐹 𝑔 2 − 2𝐷 ෨ ෨ 𝐹 𝑦𝑔 ≥ 0 The only way this can be true for all 𝐷 us if ෨ 𝐹 𝑦𝑔 = 0 .

Part II: Motzkin Polynomial

Non-negative vs. SOS polynomials • Unfortunately, not all non-negative polynomials are SOS. • Are equivalent in the special cases where 𝑜 = 1 (single-variable polynomials), 𝑒 = 2 (quadratic polynomials), or 𝑜 = 2, 𝑒 = 4 (quartic polynomials with two variables) • Hilbert [Hil1888]: In all other cases, there are non-negative polynomials which are not sums of squares of polynomials. • Motzkin [Mot67] found the first explicit example.

Motzkin Polynomial • Motzkin Polynomial: 𝑞 𝑦, 𝑧 = 𝑦 4 𝑧 2 + 𝑦 2 𝑧 4 − 3𝑦 2 𝑧 2 + 1 • Question 1: Why is it non-negative? • Question 2: How can we show it is not a sum of squares of polynomials?

AM-GM inequality • Arithmetic mean/Geometric mean Inequality: 𝑜 ς 𝑗=1 𝑦 𝑗 ≤ 1 𝑜 𝑜 𝑜 σ 𝑗=1 𝑦 𝑗 if ∀𝑗, 𝑦 𝑗 ≥ 0 with equality if and only if all of the 𝑦 𝑗 are equal. 𝑜 ς 𝑗=1 1 𝑜 𝑜 • Proof: Minimize 𝑜 σ 𝑗=1 𝑦 𝑗 − 𝑦 𝑗 𝑜 ς 𝑗≠𝑘 𝑦 𝑗 1 • Derivative with respect to 𝑦 𝑘 is 1 − 𝑜 𝑦 𝑘 𝑜 𝑜−1 𝑜 ς 𝑗=1 𝑜 • Setting this to 0 for all 𝑘 , ∀𝑘, 𝑦 𝑘 = 𝑦 𝑗

Motzkin Polynomial Non-negativity • Motzkin Polynomial: 𝑞 𝑦, 𝑧 = 𝑦 4 𝑧 2 + 𝑦 2 𝑧 4 − 3𝑦 2 𝑧 2 + 1 • Applying AM-GM with 𝑦 4 𝑧 2 , 𝑧 2 𝑦 4 , 1 , 𝑦 4 𝑧 2 ⋅ 𝑧 2 𝑦 4 ⋅ 1 ≤ 𝑦 4 𝑧 2 + 𝑧 2 𝑦 4 + 1 𝑦 2 𝑧 2 = 3 3 • Multiplying this by 3, 𝑞 𝑦, 𝑧 ≥ 0

Newton Polytope • Given a polynomial, assign a point to each monomial based on the degree of each variable. Examples: 𝑦 2 𝑧 is assigned the point (2,1) 1. 𝑧 5 is assigned the point (0,5) 2. 𝑦𝑧 2 𝑨 3 is assigned the point (1,2,3) 3. • The Newton polytope of a polynomial is the convex hull of the points assigned to each monomial.

Newton Polytope Example • Example: Newton Polytope for the polynomial 𝑞 𝑦 = 3𝑦 2 𝑧 4 − 𝑦 4 𝑧 3 − 2𝑦 3 𝑧 + 4 • Note that the coefficients in front of the monomials don’t change the polytope. 6 5 4 3 2 1 0 0 1 2 3 4 5 6