Lecture 5: SOS Proofs and the Motzkin Polynomial Lecture Outline - - PowerPoint PPT Presentation

Lecture 5: SOS Proofs and the Motzkin Polynomial

Lecture Outline

• Part I: SOS proofs and examples
• Part II: Motzkin Polynomial

Part I: SOS proofs and examples

SOS proofs

• Fundamental question: What can we say about

the pseudo-expectation values SOS gives us?

• In other words, which statements that are true

for any expectation of an actual distribution of solutions must also be true for pseudo- expectation values?

Non-negativity of Squares

• Trivial but extremely useful: If 𝑔 is a sum of

squares i.e. 𝑔 = σ𝑘 𝑕𝑘

2 then ෨

𝐹 𝑔 ≥ 0

• Example: If 𝑔 = 𝑦2 − 4𝑦 + 5 then ෨

𝐹 𝑔 ≥ 0 as 𝑔 = 𝑦 − 2 2 + 1. In fact, ෨ 𝐹 𝑔 ≥ 1

Single Variable Polynomials

• Theorem: For a single-variable polynomial p(x),

𝑞(𝑦) is non-negative ⬄ 𝑞(𝑦) is a sum of squares.

• Proof: By induction on the degree 𝑒
• Base case 𝑒 = 0 is trivial
• If 𝑒 > 0, let 𝑑 ≥ 0 be the minimal value of 𝑞(𝑦) and

let 𝑏 be a zero of 𝑞 𝑦 − 𝑑. Since 𝑞 𝑦 − 𝑑 is non- negative, it has a zero of order 2𝑙 at 𝑏 for some integer 𝑙 ≥ 1 (the order must be even).

• Write 𝑞 = 𝑦 − 𝑏 2𝑙𝑞′ + 𝑑 where 𝑞′ =

𝑞−𝑑 𝑦−𝑏 2𝑙 is

non-negative and thus a sum of squares.

Degree 2 Polynomials

• Given a degree 2 polynomial 𝑔, we can write

𝑔 𝑦1, 𝑦2, … , 𝑦𝑜 = σ𝑗,𝑘 𝑑𝑗𝑘𝑦𝑗𝑦𝑘 where c𝑘𝑗 = 𝑑𝑗𝑘 for all 𝑗 and 𝑘.

• Taking 𝑁 to be the coefficient matrix where

𝑁𝑗𝑘 = 𝑑𝑗𝑘, we can write 𝑁 = σ𝑗 𝜇𝑗𝑤𝑗𝑤𝑗

𝑈 where

the {𝑤𝑗} are orthonormal. Now

1. 𝑔 𝑦 = 𝑦𝑈𝑁𝑦. 2. 𝑔(𝑦) = σ𝑗 𝜇𝑗 𝑦𝑈𝑤𝑗𝑤𝑗

𝑈𝑦 = σ𝑗 𝜇𝑗 σ𝑘=1 𝑜

𝑤𝑗𝑘𝑦𝑘

2

Degree 2 Polynomials

• We have that

1. 𝑁 = σ𝑗 𝜇𝑗𝑤𝑗𝑤𝑗

𝑈 where the {𝑤𝑗} are orthonormal.

2. 𝑔 𝑦 = 𝑦𝑈𝑁𝑦 3. 𝑔 = σ𝑗 𝜇𝑗 σ𝑘=1

𝑜

𝑤𝑗𝑘𝑦𝑘

2

• If 𝑁 ≽ 0 then ∀𝑗, 𝜇𝑗 ≥ 0 so 𝑔 is a sum of squares
• If 𝑁 is not PSD then 𝜇𝑗 < 0 for some 𝑗. Taking

𝑦 = 𝑤𝑗, 𝑔 𝑦 = 𝑤𝑗

𝑈𝑁𝑤𝑗 < 0 so 𝑔 is not non-

negative.

• Thus if deg 𝑔 = 2, 𝑔 is non-negative ⬄𝑔 is SOS

Cauchy Schwarz Inequality

• Cauchy-Schwarz inequality:

σ𝑗 𝑔

𝑗𝑕𝑗 2 ≤ σ𝑗 𝑔 𝑗 2

σ𝑗 𝑕𝑗

2

• Extremely useful
• Proof: Consider 𝑔 and 𝑕 as vectors. Cauchy-

Schwarz is equivalent to 𝑔 ⋅ 𝑕 2 ≤ 𝑔 2 𝑕 2

• This is true as 𝑔 ⋅ 𝑕 2 = 𝑔 2 𝑕 2 cos2 Θ

where Θ is the angle between 𝑔 and 𝑕.

• How about an SOS proof?

Cauchy Schwarz: SOS Proof

• Cauchy-Schwarz: σ𝑗 𝑔

𝑗𝑕𝑗 2 ≤ σ𝑗 𝑔 𝑗 2

σ𝑗 𝑕𝑗

2

• Building block: For all 𝑗 and 𝑘,

𝑔

𝑗𝑕𝑘 − 𝑔 𝑘𝑕𝑗 2 = 𝑔 𝑗 2𝑕𝑘 2 + 𝑔 𝑘 2𝑕𝑗 2 − 2𝑔 𝑗𝑕𝑗𝑔 𝑘𝑕𝑘 ≥ 0

• Note that:

1. σ𝑗<𝑘(𝑔

𝑗 2𝑕𝑘 2 + 𝑔 𝑘 2𝑕𝑗 2) = σ𝑗 𝑔 𝑗 2

σ𝑗 𝑕𝑗

2 − σ𝑗 𝑔 𝑗 2𝑕𝑗 2

2. 2 σ𝑗<𝑘(𝑔

𝑗𝑕𝑗𝑔 𝑘𝑕𝑘) = σ𝑗 𝑔 𝑗𝑕𝑗 2 − σ𝑗 𝑔 𝑗 2𝑕𝑗 2

• Final proof: σ𝑗,𝑘:𝑗<𝑘 𝑔

𝑗𝑕𝑘 − 𝑔 𝑘𝑕𝑗 2 =

σ𝑗 𝑔

𝑗 2

σ𝑗 𝑕𝑗

2 − σ𝑗 𝑔 𝑗𝑕𝑗 2 ≥ 0

SOS Proofs With Constraints

• What if we also have constraints

𝑡1 𝑦1, … , 𝑦𝑜 = 0, 𝑡2 𝑦1, … , 𝑦𝑜 = 0, etc.?

• An SOS proof that ℎ ≥ 𝑑 now takes the form

ℎ = 𝑑 + σ𝑗 𝑔

𝑗𝑡𝑗 + σ𝑘 𝑕𝑘 2

• Example: If 𝑦2 = 1 then x ≥ −1. Proof:

𝑦 + 1 = 𝑦2

2 + 𝑦 + 1 2 = 1 2 𝑦 + 1 2 ≥ 0

Combining Proofs

• If there is an SOS proof of degree 𝑒1 that 𝑔 ≥ 0

and an SOS proof of degree 𝑒2 that 𝑕 ≥ 0 then:

• 1. There is an SOS proof of degree 𝑛𝑏𝑦{𝑒1, 𝑒2} that

𝑔 + 𝑕 ≥ 0

• 2. There is an SOS proof of degree 𝑒1 + 𝑒2 that

𝑔𝑕 ≥ 0

Products of Pseudo-expectation Values

• What if our statements involve products of

pseudo-expectation values?

• Example: We showed that

෨ 𝐹 σ𝑗 𝑔

𝑗𝑕𝑗 2 ≤ ෨

𝐹 σ𝑗 𝑔

𝑗 2

σ𝑗 𝑕𝑗

2

What if we instead want to show that ෨ 𝐹 σ𝑗 𝑔

𝑗𝑕𝑗 2 ≤ ෨

𝐹 σ𝑗 𝑔

𝑗 2 ෨

𝐹 σ𝑗 𝑕𝑗

2 ?

• Requires modified proof, see problem set
• Can often prove such statements by using ෨

𝐹 values as constants in the proof.

Example: Variance

• For any random variable 𝑦, 𝐹 𝑦2 ≥ 𝐹 𝑦

2

• Also true for pseudo-expectation values, i.e. for

any polynomial 𝑔, ෨ 𝐹 𝑔2 ≥ ෨ 𝐹 𝑔

2

• Proof: Given ෨

𝐹, let 𝑑 = ෨ 𝐹[𝑔] and observe that ෨ 𝐹 𝑔 − 𝑑 2 = ෨ 𝐹 𝑔2 − 2𝑑 ෨ 𝐹 𝑔 + c2 = ෨ 𝐹 𝑔2 − ෨ 𝐹 𝑔

2 ≥ 0

In-class exercises

• 1. Prove that ෨

𝐹 𝑦4 − 4𝑦 + 3 ≥ 0

• 2. Prove that

෨ 𝐹 𝑦2 + 2𝑧2 + 6𝑨2 + 2𝑦𝑧 + 2𝑦𝑨 + 6𝑧𝑨 ≥ 0

• 3. Prove that if 𝑦2 + 𝑧2 = 1 then 𝑦 + 𝑧 ≤

2

• 4. Prove that if ෨

𝐹 𝑦2 = 0 then for any function 𝑔

• f degree at most 𝑒

2, ෨

𝐹 𝑦𝑔 = 0.

In-class exercise answers

• 1. Prove that ෨

𝐹 𝑦4 − 4𝑦 + 3 ≥ 0 Answer: 𝑦4 − 4𝑦 + 3 = 𝑦 − 1 2 𝑦2 + 2𝑦 + 3 = 𝑦 − 1 2( 𝑦 + 1 2 + 2)

In-class exercise answers

• 2. Prove that

෨ 𝐹 𝑦2 + 2𝑧2 + 6𝑨2 + 2𝑦𝑧 + 2𝑦𝑨 + 6𝑧𝑨 ≥ 0 Answer: The coefficient matrix for this polynomial is M = 1 1 1 1 2 3 1 3 6 One non-orthonormal factorization is 𝑁 = 𝑤1𝑤1

𝑈 + 𝑤2𝑤2 𝑈 + 𝑤3𝑤3 𝑈 where 𝑤1 𝑈 = [1

1 1], 𝑤2

𝑈 = [0

1 2], 𝑤3

𝑈 = [0

1],

In-class exercise answers

This gives us that 𝑦2 + 2𝑧2 + 6𝑨2 + 2𝑦𝑧 + 2𝑦𝑨 + 6𝑧𝑨 = 𝑦 + 𝑧 + 𝑨 2 + 𝑧 + 2𝑨 2 + 𝑨2

In-class exercise answers

• 3. Prove that if we have the constraint 𝑦2 + 𝑧2 = 1

then ෨ 𝐹 𝑦 + 𝑧 ≤ 2 Answer: 2 − 𝑦 − 𝑧 =

𝑦2+𝑧2 2

− 𝑦 − 𝑧 +

1 2 = 𝑦−𝑧 2 2 2 + 𝑦+𝑧 2 2 2 − 𝑦 − 𝑧 + 1 2 = 𝑦−𝑧 2 2 2 + 1 2 2 𝑦 + 𝑧 −

2

2 ≥ 0

In-class exercise answers

• 4. Prove that if ෨

𝐹 𝑦2 = 0 then for any function 𝑔

• f degree at most

𝑒 2 − 1, ෨

𝐹 𝑦𝑔 = 0. Answer: Observe that for any constant 𝐷, ෨ 𝐹 𝑔 − 𝐷𝑦 2 = ෨ 𝐹 𝑔2 − 2𝐷 ෨ 𝐹 𝑦𝑔 + ෨ 𝐹 𝑦2 = ෨ 𝐹 𝑔2 − 2𝐷 ෨ 𝐹 𝑦𝑔 ≥ 0 The only way this can be true for all 𝐷 us if ෨ 𝐹 𝑦𝑔 = 0.

Part II: Motzkin Polynomial

Non-negative vs. SOS polynomials

• Unfortunately, not all non-negative polynomials

are SOS.

• Are equivalent in the special cases where 𝑜 = 1

(single-variable polynomials), 𝑒 = 2 (quadratic polynomials), or 𝑜 = 2, 𝑒 = 4 (quartic polynomials with two variables)

• Hilbert [Hil1888]: In all other cases, there are

non-negative polynomials which are not sums of squares of polynomials.

• Motzkin [Mot67] found the first explicit example.

Motzkin Polynomial

• Motzkin Polynomial:

𝑞 𝑦, 𝑧 = 𝑦4𝑧2 + 𝑦2𝑧4 − 3𝑦2𝑧2 + 1

• Question 1: Why is it non-negative?
• Question 2: How can we show it is not a sum of

squares of polynomials?

AM-GM inequality

• Arithmetic mean/Geometric mean Inequality:

𝑜 ς𝑗=1

𝑜

𝑦𝑗 ≤ 1

𝑜 σ𝑗=1 𝑜

𝑦𝑗 if ∀𝑗, 𝑦𝑗 ≥ 0 with equality if and only if all of the 𝑦𝑗 are equal.

• Proof: Minimize

1 𝑜 σ𝑗=1 𝑜

𝑦𝑗 −

𝑜 ς𝑗=1

𝑜

𝑦𝑗

• Derivative with respect to 𝑦𝑘 is

1 𝑜

1 −

𝑜 ς𝑗≠𝑘 𝑦𝑗 𝑜 𝑦𝑘 𝑜−1

• Setting this to 0 for all 𝑘, ∀𝑘, 𝑦𝑘 =

𝑜 ς𝑗=1

𝑜

𝑦𝑗

Motzkin Polynomial Non-negativity

• Motzkin Polynomial:

𝑞 𝑦, 𝑧 = 𝑦4𝑧2 + 𝑦2𝑧4 − 3𝑦2𝑧2 + 1

• Applying AM-GM with 𝑦4𝑧2, 𝑧2𝑦4, 1,

𝑦2𝑧2 =

3

𝑦4𝑧2 ⋅ 𝑧2𝑦4 ⋅ 1 ≤ 𝑦4𝑧2+ 𝑧2𝑦4+ 1 3

• Multiplying this by 3, 𝑞 𝑦, 𝑧 ≥ 0

Newton Polytope

• Given a polynomial, assign a point to each

monomial based on the degree of each variable. Examples:

1. 𝑦2𝑧 is assigned the point (2,1) 2. 𝑧5 is assigned the point (0,5) 3. 𝑦𝑧2𝑨3 is assigned the point (1,2,3)

• The Newton polytope of a polynomial is the

convex hull of the points assigned to each monomial.

Newton Polytope Example

• Example: Newton Polytope for the polynomial

𝑞 𝑦 = 3𝑦2𝑧4 − 𝑦4𝑧3 − 2𝑦3𝑧 + 4

• Note that the coefficients in front of the

monomials don’t change the polytope.

0 1 2 3 4 5 6 1 2 3 4 5 6

Newton Polytope of a Sum of Squares

• Let 𝑔 be a sum of squares, i.e. 𝑔 = σ𝑘 𝑕𝑘

2

• Claim: The Newton polytope of 𝑔 is 2𝑌 where 𝑌

is the convex hull of all the points corresponding to some monomial in some 𝑕𝑘

• Proposition: If 𝑞, 𝑟 are monomials with

corresponding points 𝑏, 𝑐 then 𝑞𝑟 corresponds to the point 𝑏 + 𝑐

• One direction: Let 𝑌

𝑘 be the Newton polytope of

𝑕𝑘. The Newton polytope of 𝑕𝑘

2 ⊆ 2𝑌 𝑘 ⊆ 2𝑌.

Thus, the Newton polytope of 𝑔 ⊆ 2𝑌.

Newton Polytope of a Sum of Squares

• Other direction: If 𝑞, 𝑟, 𝑠 are monomials where

𝑞𝑠 = 𝑟2 and 𝑏, 𝑐, 𝑑 are the corresponding points, 𝑏 + 𝑑 = 2𝑐

• Corollary: If 𝑐 is a vertex of 𝑌 corresponding to a

monomial 𝑟 then if

1. 𝑞, 𝑠 are monomials appearing in some 𝑕𝑘 (and thus their corresponding points 𝑏, 𝑑 are in 𝑌) 2. 𝑞𝑠 = 𝑟2

then 𝑞 = 𝑠 = 𝑟.

Newton Polytope of a Sum of Squares

• Corollary: If 𝑐 is a vertex of 𝑌 corresponding

to a monomial 𝑟 then 𝑟2 appears with positive coefficient in 𝑔 = σ𝑘 𝑕𝑘

2.

• This implies that 2𝑌 ⊆ the Newton polytope
• f 𝑔
• Putting everthing together, the Newton

polytope of 𝑔is 2𝑌.

Motzkin Polynomial Newton Polytope

• Motzkin polynomial:

𝑞 𝑦 = 𝑦4𝑧2 + 𝑦2𝑧4 − 3𝑦2𝑧2 + 1

0 1 2 3 4 5 6 1 2 3 4 5 6

Motzkin Polynomial Newton Polytope

• If 𝑞(𝑦) were a sum of squares of polynomials,

their corresponding points would have to be inside the following polytope.

0 1 2 3 4 5 6 1 2 3 4 5 6

Motzkin is not a Sum of Squares

• If 𝑞 𝑦 = 𝑦4𝑧2 + 𝑦2𝑧4 − 3𝑦2𝑧2 + 1 were a

sum of squares of polynomials, it would have to be a sum of terms of the form 𝑏𝑦2𝑧 + 𝑐𝑦𝑧2 + 𝑑𝑦𝑧 + 𝑒

2

• However, no such term has a negative coefficient
• f 𝑦2𝑧2. Contradiction.

Showing Polynomials are not SOS

• Is there a more general way to show a

polynomial is not a sum of squares?

• Observation: By definition, if 𝑔 = σ𝑘 𝑕𝑘

2 then

for any valid pseudo-expectation values, ෨ 𝐹 𝑔 = σ𝑘 ෨ 𝐹 𝑕𝑘

2 ≥ 0

• Thus, if we can find pseudo-expectation values

such that ෨ 𝐹 𝑔 < 0, then 𝑔 is not a sum of squares of polynomials.

Motzkin is a Rational Function of Sums of Squares

• 𝑞 𝑦 = 𝑦4𝑧2 + 𝑦2𝑧4 − 3𝑦2𝑧2 + 1
• 𝑦2 + 𝑧2 + 1 𝑞 𝑦 = 𝑦6𝑧2 + 2𝑧4𝑦4 + 𝑦2𝑧6 −

2𝑦4𝑧2 − 2𝑦2𝑧4 − 3𝑦2𝑧2 + 𝑦2 + 𝑧2 + 1

• This is a sum of squares. The components are:

1. 2

1 2 𝑦3𝑧 + 1 2 𝑦𝑧3 − 𝑦𝑧 2

=

1 2 𝑦6𝑧2 + 2𝑧4𝑦4 + 𝑦2𝑧6 − 2𝑦4𝑧2 − 2𝑦2𝑧4 +

2𝑦2𝑧2 2. 𝑦2𝑧 − 𝑧

2 = 𝑦4𝑧2 − 2𝑦2𝑧2 + 𝑧2

3. 𝑦𝑧2 − 𝑦

2 = 𝑦2𝑧4 − 2𝑦2𝑧2 + 𝑦2

4.

1 2 𝑦3𝑧 − 𝑦𝑧 2 = 1 2 𝑦6𝑧2 − 𝑦4𝑧2 + 1 2 𝑦2𝑧2

5.

1 2 𝑦𝑧3 − 𝑦𝑧 2 = 1 2 𝑦2𝑧6 − 𝑦2𝑧4 + 1 2 𝑦2𝑧2

6. 𝑦2𝑧2 − 1

2 = 𝑦4𝑧4 − 2𝑦2𝑧2 + 1

Can SOS use Rational Functions?

• 𝑞 𝑦 = 𝑦4𝑧2 + 𝑦2𝑧4 − 3𝑦2𝑧2 + 1
• 𝑞 𝑦 =

σ𝑘 𝑕𝑘

2

𝑦2+𝑧2+1 ≥ 0

• Can the SOS hierarchy use such reasoning?
• Yes and no… (see problem set)

References

• [Hil1888] D. Hilbert. Uber die darstellung definiter formen als summe von formen-
• quadraten. Annals of Mathematics 32:342–350, 1888.
• [Mot67] T. Motzkin. The arithmetic-geometric inequality. In Proc. Symposium on

Inequalities p. 205–224, 1967.