The quest of efficiency and certification in polynomial optimization - PowerPoint PPT Presentation

Preliminary Comparisons f ( x ) : = x 2 x 5 + x 3 x 6 − x 2 x 3 − x 5 x 6 + x 1 ( − x 1 + x 2 + x 3 − x 4 + x 5 + x 6 ) x ∈ [ 4.00, 6.36 ] 6 , e ∈ [ − ǫ , ǫ ] 15 , ǫ = 2 − 53 Dense SDP: ( 6 + 15 + 4 6 + 15 ) = 12650 variables ❀ Out of memory Sparse SDP Real2Float tool: 15 ( 6 + 1 + 4 6 + 1 ) = 4950 ❀ 759 ǫ Victor Magron The quest of efficiency and certification in polynomial optimization 12 / 42

Preliminary Comparisons f ( x ) : = x 2 x 5 + x 3 x 6 − x 2 x 3 − x 5 x 6 + x 1 ( − x 1 + x 2 + x 3 − x 4 + x 5 + x 6 ) x ∈ [ 4.00, 6.36 ] 6 , e ∈ [ − ǫ , ǫ ] 15 , ǫ = 2 − 53 Dense SDP: ( 6 + 15 + 4 6 + 15 ) = 12650 variables ❀ Out of memory Sparse SDP Real2Float tool: 15 ( 6 + 1 + 4 6 + 1 ) = 4950 ❀ 759 ǫ Interval arithmetic: 922 ǫ (10 × less CPU) Victor Magron The quest of efficiency and certification in polynomial optimization 12 / 42

Preliminary Comparisons f ( x ) : = x 2 x 5 + x 3 x 6 − x 2 x 3 − x 5 x 6 + x 1 ( − x 1 + x 2 + x 3 − x 4 + x 5 + x 6 ) x ∈ [ 4.00, 6.36 ] 6 , e ∈ [ − ǫ , ǫ ] 15 , ǫ = 2 − 53 Dense SDP: ( 6 + 15 + 4 6 + 15 ) = 12650 variables ❀ Out of memory Sparse SDP Real2Float tool: 15 ( 6 + 1 + 4 6 + 1 ) = 4950 ❀ 759 ǫ Interval arithmetic: 922 ǫ (10 × less CPU) Symbolic Taylor FPTaylor tool: 721 ǫ (21 × more CPU) Victor Magron The quest of efficiency and certification in polynomial optimization 12 / 42

Preliminary Comparisons f ( x ) : = x 2 x 5 + x 3 x 6 − x 2 x 3 − x 5 x 6 + x 1 ( − x 1 + x 2 + x 3 − x 4 + x 5 + x 6 ) x ∈ [ 4.00, 6.36 ] 6 , e ∈ [ − ǫ , ǫ ] 15 , ǫ = 2 − 53 Dense SDP: ( 6 + 15 + 4 6 + 15 ) = 12650 variables ❀ Out of memory Sparse SDP Real2Float tool: 15 ( 6 + 1 + 4 6 + 1 ) = 4950 ❀ 759 ǫ Interval arithmetic: 922 ǫ (10 × less CPU) Symbolic Taylor FPTaylor tool: 721 ǫ (21 × more CPU) SMT-based rosa tool: 762 ǫ (19 × more CPU) Victor Magron The quest of efficiency and certification in polynomial optimization 12 / 42

Preliminary Comparisons 1,000 762 ǫ 721 ǫ 800 759 ǫ Error Bound ( ǫ ) 600 400 200 0 a r t o a s o o l r y l a F 2 T P l F a e R CPU Time Victor Magron The quest of efficiency and certification in polynomial optimization 12 / 42

Comparison with rosa Relative execution time 1 f m 0 . 5 d e x o t c y g z b i w v a − 10 0 10 100 Relative bound precision h u − 0 . 5 q k j p − 1 l r Victor Magron The quest of efficiency and certification in polynomial optimization 13 / 42

Comparison with FPTaylor Relative execution time 1 β t 0 . 5 x w f α c d u v e i b g − 10 0 10 100 a n o m Relative bound precision h − 0 . 5 γ δ q jk p − 1 r l Victor Magron The quest of efficiency and certification in polynomial optimization 14 / 42

Noncommutative (NC) Polynomials Symmetric Matrix variables X i , Y j f = X 1 ( Y 1 + Y 2 + Y 3 ) + X 2 ( Y 1 + Y 2 − Y 3 ) + X 3 ( Y 1 − Y 2 ) − X 1 − 2 Y 1 − Y 2 with X 1 X 2 � = X 2 X 1 , involution ( X 1 Y 3 ) ⋆ = Y 3 X 1 Victor Magron The quest of efficiency and certification in polynomial optimization 15 / 42

Noncommutative (NC) Polynomials Symmetric Matrix variables X i , Y j f = X 1 ( Y 1 + Y 2 + Y 3 ) + X 2 ( Y 1 + Y 2 − Y 3 ) + X 3 ( Y 1 − Y 2 ) − X 1 − 2 Y 1 − Y 2 with X 1 X 2 � = X 2 X 1 , involution ( X 1 Y 3 ) ⋆ = Y 3 X 1 Constraints K = { ( X , Y ) : X i , Y j � 0, X 2 i = X i , Y 2 j = Y j , X i Y j = Y j X i } Victor Magron The quest of efficiency and certification in polynomial optimization 15 / 42

Noncommutative (NC) Polynomials Symmetric Matrix variables X i , Y j f = X 1 ( Y 1 + Y 2 + Y 3 ) + X 2 ( Y 1 + Y 2 − Y 3 ) + X 3 ( Y 1 − Y 2 ) − X 1 − 2 Y 1 − Y 2 with X 1 X 2 � = X 2 X 1 , involution ( X 1 Y 3 ) ⋆ = Y 3 X 1 Constraints K = { ( X , Y ) : X i , Y j � 0, X 2 i = X i , Y 2 j = Y j , X i Y j = Y j X i } M INIMAL EIGENVALUE OPTIMIZATION λ min = inf {� f ( X , Y ) v , v � : ( X , Y ) ∈ K , � v � = 1 } Victor Magron The quest of efficiency and certification in polynomial optimization 15 / 42

Noncommutative (NC) Polynomials Symmetric Matrix variables X i , Y j f = X 1 ( Y 1 + Y 2 + Y 3 ) + X 2 ( Y 1 + Y 2 − Y 3 ) + X 3 ( Y 1 − Y 2 ) − X 1 − 2 Y 1 − Y 2 with X 1 X 2 � = X 2 X 1 , involution ( X 1 Y 3 ) ⋆ = Y 3 X 1 Constraints K = { ( X , Y ) : X i , Y j � 0, X 2 i = X i , Y 2 j = Y j , X i Y j = Y j X i } M INIMAL EIGENVALUE OPTIMIZATION λ min = inf {� f ( X , Y ) v , v � : ( X , Y ) ∈ K , � v � = 1 } = sup λ f ( X , Y ) − λ I � 0 , ∀ ( X , Y ) ∈ K s.t. Victor Magron The quest of efficiency and certification in polynomial optimization 15 / 42

Putinar’s Representation “ Archimedean ” constraint in K = { X : g j ( X ) � 0 } : N − ∑ i X 2 i � 0 Theorem: NC Putinar’s representation [Helton-McCullough 02] f = ∑ i s i + ∑ s ⋆ t ⋆ j ∑ f ≻ 0 on K = ⇒ ji g j t ji with s i , t ji ∈ R � X � i i Victor Magron The quest of efficiency and certification in polynomial optimization 16 / 42

Putinar’s Representation “ Archimedean ” constraint in K = { X : g j ( X ) � 0 } : N − ∑ i X 2 i � 0 Theorem: NC Putinar’s representation [Helton-McCullough 02] f = ∑ i s i + ∑ s ⋆ t ⋆ j ∑ f ≻ 0 on K = ⇒ ji g j t ji with s i , t ji ∈ R � X � i i NC variant of Lasserre’s Hierarchy for λ min : replace “ f − λ I � 0 on K ” by f − λ I = ∑ i s ⋆ i s i + ∑ j ∑ i t ⋆ ji g j t ji with s i , t ji of bounded degrees Victor Magron The quest of efficiency and certification in polynomial optimization 16 / 42

Sparse Putinar’s Representation Sparse f = f 1 + · · · + f p with f k ∈ R � X , I k � Sparse K = { X : g j ( X ) � 0 } with g j ∈ R � X , I k ( j ) � for some k ( j ) Additional constraints n k − ∑ i ∈ I k X 2 i � 0 in K Victor Magron The quest of efficiency and certification in polynomial optimization 17 / 42

Sparse Putinar’s Representation Sparse f = f 1 + · · · + f p with f k ∈ R � X , I k � Sparse K = { X : g j ( X ) � 0 } with g j ∈ R � X , I k ( j ) � for some k ( j ) Additional constraints n k − ∑ i ∈ I k X 2 i � 0 in K R UNNING I NTERSECTION P ROPERTY (RIP) � ∀ k = 1, . . . , p − 1 I k + 1 ∩ I j ⊆ I i for some i � k j � k Victor Magron The quest of efficiency and certification in polynomial optimization 17 / 42

Sparse Putinar’s Representation Sparse f = f 1 + · · · + f p with f k ∈ R � X , I k � Sparse K = { X : g j ( X ) � 0 } with g j ∈ R � X , I k ( j ) � for some k ( j ) Additional constraints n k − ∑ i ∈ I k X 2 i � 0 in K R UNNING I NTERSECTION P ROPERTY (RIP) � ∀ k = 1, . . . , p − 1 I k + 1 ∩ I j ⊆ I i for some i � k j � k Theorem: Sparse Putinar’s representation [Klep-M.-Povh 19] f = ∑ ki s ki + ∑ s ⋆ t ji ⋆ g j t ji k ∑ j ∑ f ≻ 0 on K + RIP = ⇒ i i with s ki ∈ R � X , I k � , t ji ∈ R � X , I k ( j ) � Victor Magron The quest of efficiency and certification in polynomial optimization 17 / 42

Sparse Example: I 3322 Bell Inequality Entanglement in quantum mechanics → upper bounds for violation levels of Bell inequalities Victor Magron The quest of efficiency and certification in polynomial optimization 18 / 42

Sparse Example: I 3322 Bell Inequality Entanglement in quantum mechanics → upper bounds for violation levels of Bell inequalities → upper bounds on λ max of f on K Victor Magron The quest of efficiency and certification in polynomial optimization 18 / 42

Sparse Example: I 3322 Bell Inequality Entanglement in quantum mechanics → upper bounds for violation levels of Bell inequalities → upper bounds on λ max of f on K I 3322 Bell inequality f = X 1 ( Y 1 + Y 2 + Y 3 ) + X 2 ( Y 1 + Y 2 − Y 3 ) + X 3 ( Y 1 − Y 2 ) − X 1 − 2 Y 1 − Y 2 K = { ( X , Y ) : X i , Y j � 0, X 2 i = X i , Y 2 j = Y j , X i Y j = Y j X i } Victor Magron The quest of efficiency and certification in polynomial optimization 18 / 42

Sparse Example: I 3322 Bell Inequality Entanglement in quantum mechanics → upper bounds for violation levels of Bell inequalities → upper bounds on λ max of f on K I 3322 Bell inequality f = X 1 ( Y 1 + Y 2 + Y 3 ) + X 2 ( Y 1 + Y 2 − Y 3 ) + X 3 ( Y 1 − Y 2 ) − X 1 − 2 Y 1 − Y 2 K = { ( X , Y ) : X i , Y j � 0, X 2 i = X i , Y 2 j = Y j , X i Y j = Y j X i } I k → { X 1 , X 2 , X 3 , Y k } Victor Magron The quest of efficiency and certification in polynomial optimization 18 / 42

Sparse Example: I 3322 Bell Inequality Entanglement in quantum mechanics → upper bounds for violation levels of Bell inequalities → upper bounds on λ max of f on K I 3322 Bell inequality f = X 1 ( Y 1 + Y 2 + Y 3 ) + X 2 ( Y 1 + Y 2 − Y 3 ) + X 3 ( Y 1 − Y 2 ) − X 1 − 2 Y 1 − Y 2 K = { ( X , Y ) : X i , Y j � 0, X 2 i = X i , Y 2 j = Y j , X i Y j = Y j X i } I k → { X 1 , X 2 , X 3 , Y k } level sparse dense [Pál-Vértesi 18] 2 0.2550008 0.2509397 Victor Magron The quest of efficiency and certification in polynomial optimization 18 / 42

Sparse Example: I 3322 Bell Inequality Entanglement in quantum mechanics → upper bounds for violation levels of Bell inequalities → upper bounds on λ max of f on K I 3322 Bell inequality f = X 1 ( Y 1 + Y 2 + Y 3 ) + X 2 ( Y 1 + Y 2 − Y 3 ) + X 3 ( Y 1 − Y 2 ) − X 1 − 2 Y 1 − Y 2 K = { ( X , Y ) : X i , Y j � 0, X 2 i = X i , Y 2 j = Y j , X i Y j = Y j X i } I k → { X 1 , X 2 , X 3 , Y k } level sparse dense [Pál-Vértesi 18] 2 0.2550008 0.2509397 3 0.2511592 0.2508756 Victor Magron The quest of efficiency and certification in polynomial optimization 18 / 42

Sparse Example: I 3322 Bell Inequality Entanglement in quantum mechanics → upper bounds for violation levels of Bell inequalities → upper bounds on λ max of f on K I 3322 Bell inequality f = X 1 ( Y 1 + Y 2 + Y 3 ) + X 2 ( Y 1 + Y 2 − Y 3 ) + X 3 ( Y 1 − Y 2 ) − X 1 − 2 Y 1 − Y 2 K = { ( X , Y ) : X i , Y j � 0, X 2 i = X i , Y 2 j = Y j , X i Y j = Y j X i } I k → { X 1 , X 2 , X 3 , Y k } level sparse dense [Pál-Vértesi 18] 2 0.2550008 0.2509397 3 0.2511592 0.2508756 3’ 0.2508754 Victor Magron The quest of efficiency and certification in polynomial optimization 18 / 42

Sparse Example: I 3322 Bell Inequality Entanglement in quantum mechanics → upper bounds for violation levels of Bell inequalities → upper bounds on λ max of f on K I 3322 Bell inequality f = X 1 ( Y 1 + Y 2 + Y 3 ) + X 2 ( Y 1 + Y 2 − Y 3 ) + X 3 ( Y 1 − Y 2 ) − X 1 − 2 Y 1 − Y 2 K = { ( X , Y ) : X i , Y j � 0, X 2 i = X i , Y 2 j = Y j , X i Y j = Y j X i } I k → { X 1 , X 2 , X 3 , Y k } level sparse dense [Pál-Vértesi 18] 2 0.2550008 0.2509397 3 0.2511592 0.2508756 3’ 0.2508754 4 0.2508917 Victor Magron The quest of efficiency and certification in polynomial optimization 18 / 42

Sparse Example: I 3322 Bell Inequality Entanglement in quantum mechanics → upper bounds for violation levels of Bell inequalities → upper bounds on λ max of f on K I 3322 Bell inequality f = X 1 ( Y 1 + Y 2 + Y 3 ) + X 2 ( Y 1 + Y 2 − Y 3 ) + X 3 ( Y 1 − Y 2 ) − X 1 − 2 Y 1 − Y 2 K = { ( X , Y ) : X i , Y j � 0, X 2 i = X i , Y 2 j = Y j , X i Y j = Y j X i } I k → { X 1 , X 2 , X 3 , Y k } level sparse dense [Pál-Vértesi 18] 2 0.2550008 0.2509397 3 0.2511592 0.2508756 3’ 0.2508754 4 0.2508917 5 0.2508763 Victor Magron The quest of efficiency and certification in polynomial optimization 18 / 42

Sparse Example: I 3322 Bell Inequality Entanglement in quantum mechanics → upper bounds for violation levels of Bell inequalities → upper bounds on λ max of f on K I 3322 Bell inequality f = X 1 ( Y 1 + Y 2 + Y 3 ) + X 2 ( Y 1 + Y 2 − Y 3 ) + X 3 ( Y 1 − Y 2 ) − X 1 − 2 Y 1 − Y 2 K = { ( X , Y ) : X i , Y j � 0, X 2 i = X i , Y 2 j = Y j , X i Y j = Y j X i } I k → { X 1 , X 2 , X 3 , Y k } level sparse dense [Pál-Vértesi 18] 2 0.2550008 0.2509397 3 0.2511592 0.2508756 3’ 0.2508754 4 0.2508917 5 0.2508763 6 0.2508753977180 !!!!! Victor Magron The quest of efficiency and certification in polynomial optimization 18 / 42

Exploiting Sparsity Certified Polynomial Optimization

Certified Polynomial Optimization X = ( X 1 , . . . , X n ) co-NP hard problem: check f � 0 on K NP hard problem: min { f ( x ) : x ∈ K } f ∈ Q [ X ] Victor Magron The quest of efficiency and certification in polynomial optimization 19 / 42

Certified Polynomial Optimization X = ( X 1 , . . . , X n ) co-NP hard problem: check f � 0 on K NP hard problem: min { f ( x ) : x ∈ K } f ∈ Q [ X ] 1 Unconstrained � K = R n 2 Constrained � K = { x ∈ R n : g 1 ( x ) � 0, . . . , g l ( x ) � 0 } g j ∈ Q [ X ] deg f , deg g j � d Victor Magron The quest of efficiency and certification in polynomial optimization 19 / 42

Certified Polynomial Optimization X = ( X 1 , . . . , X n ) co-NP hard problem: check f � 0 on K NP hard problem: min { f ( x ) : x ∈ K } f ∈ Q [ X ] 1 Unconstrained � K = R n 2 Constrained � K = { x ∈ R n : g 1 ( x ) � 0, . . . , g l ( x ) � 0 } g j ∈ Q [ X ] deg f , deg g j � d [Collins 75] CAD doubly exp. in n poly. in d [Grigoriev-Vorobjov 88, Basu-Pollack-Roy 98] Critical points singly exponential time ( l + 1 ) τ d O ( n ) Victor Magron The quest of efficiency and certification in polynomial optimization 19 / 42

Certified Polynomial Optimization Sums of squares (SOS) σ = h 12 + · · · + h p 2 Victor Magron The quest of efficiency and certification in polynomial optimization 20 / 42

Certified Polynomial Optimization Sums of squares (SOS) σ = h 12 + · · · + h p 2 H ILBERT 17 TH P ROBLEM : f SOS of rational functions? [Artin 27] YES ! Victor Magron The quest of efficiency and certification in polynomial optimization 20 / 42

Certified Polynomial Optimization Sums of squares (SOS) σ = h 12 + · · · + h p 2 H ILBERT 17 TH P ROBLEM : f SOS of rational functions? [Artin 27] YES ! [Lasserre/Parrilo 01] Numerical solvers compute σ Semidefinite programming (SDP) � approximate certificates Victor Magron The quest of efficiency and certification in polynomial optimization 20 / 42

Certified Polynomial Optimization Sums of squares (SOS) σ = h 12 + · · · + h p 2 H ILBERT 17 TH P ROBLEM : f SOS of rational functions? [Artin 27] YES ! [Lasserre/Parrilo 01] Numerical solvers compute σ Semidefinite programming (SDP) � approximate certificates → ≃ = The Question of Exact Certification How to go from approximate to exact certification? Victor Magron The quest of efficiency and certification in polynomial optimization 20 / 42

Motivation Positivity certificates Stability proofs of critical control systems (Lyapunov) Certified function evaluation [Chevillard et. al 11] Formal verification of real inequalities [Hales et. al 15]: C OQ H OL - LIGHT Victor Magron The quest of efficiency and certification in polynomial optimization 21 / 42

Decomposing Nonnegative Polynomials σ f = 1 Polya ’s representation ( X 2 1 + ··· + X 2 n ) D positive definite form f [Reznick 95] f = σ 2 Hilbert-Artin ’s representation h 2 f � 0 [Artin 27] f = σ 0 + σ 1 g 1 + · · · + σ l g l 3 Putinar ’s representation f > 0 on compact K deg σ i � 2 D [Putinar 93] Victor Magron The quest of efficiency and certification in polynomial optimization 22 / 42

Decomposing Nonnegative Polynomials Deciding polynomial nonnegativity f ( a , b ) = a 2 − 2 ab + b 2 � 0 � � � � � � z 1 z 2 a f ( a , b ) = a b z 2 z 3 b � �� � � 0 a 2 − 2 ab + b 2 = z 1 a 2 + 2 z 2 ab + z 3 b 2 ( A z = d ) � z 1 � � 1 � � 0 � � 0 � � 0 � z 2 0 1 0 0 = z 1 + z 2 + z 3 � z 2 z 3 0 0 1 0 0 1 0 0 � �� � � �� � � �� � � �� � F 1 F 2 F 3 F 0 Victor Magron The quest of efficiency and certification in polynomial optimization 23 / 42

Decomposing Nonnegative Polynomials Choose a cost c e.g. ( 1, 0, 1 ) and solve SDP ⊤ z min c z ∑ s.t. F i z i � F 0 , A z = d i � 1 � z 1 � � − 1 z 2 Solution = � 0 (eigenvalues 0 and 2) − 1 z 2 z 3 1 � � 1 � � a � − 1 a 2 − 2 ab + b 2 = � = ( a − b ) 2 a b − 1 1 b � �� � � 0 Solving SDP = ⇒ Finding S UMS OF S QUARES certificates Victor Magron The quest of efficiency and certification in polynomial optimization 24 / 42

From Approximate to Exact Solutions A PPROXIMATE SOLUTIONS sum of squares of a 2 − 2 ab + b 2 ? ( 1.00001 a − 0.99998 b ) 2 ! a 2 − 2 ab + b 2 ≃ ( 1.00001 a − 0.99998 b ) 2 a 2 − 2 ab + b 2 � = 1.0000200001 a 2 − 1.9999799996 ab + 0.9999600004 b 2 → = ? ≃ Victor Magron The quest of efficiency and certification in polynomial optimization 25 / 42

Rational SOS Decompositions Let f ∈ R [ X ] and f � 0 on R ( n = 1 ) Theorem There exist f 1 , f 2 ∈ R [ X ] s.t. f = f 12 + f 22 . Victor Magron The quest of efficiency and certification in polynomial optimization 26 / 42

Rational SOS Decompositions Let f ∈ R [ X ] and f � 0 on R ( n = 1 ) Theorem There exist f 1 , f 2 ∈ R [ X ] s.t. f = f 12 + f 22 . Proof. f = h 2 ( q + ir )( q − ir ) Victor Magron The quest of efficiency and certification in polynomial optimization 26 / 42

Rational SOS Decompositions Let f ∈ R [ X ] and f � 0 on R ( n = 1 ) Theorem There exist f 1 , f 2 ∈ R [ X ] s.t. f = f 12 + f 22 . Proof. f = h 2 ( q + ir )( q − ir ) Examples � √ � 2 � 2 � X + 1 3 1 + X + X 2 = + 2 2 √ � � 2 2 X + 1 + X 2 + 1 5 1 + X + X 2 + X 3 + X 4 = + 4 √ √ √ � � � � � 2 10 + 2 5 + 10 − 2 10 − 2 5 5 X + 4 4 Victor Magron The quest of efficiency and certification in polynomial optimization 26 / 42

Rational SOS Decompositions f ∈ Q [ X ] ∩ ˚ Σ [ X ] (interior of the SOS cone) Existence Question Does there exist f i ∈ Q [ X ] , c i ∈ Q > 0 s.t. f = ∑ i c i f i 2 ? Victor Magron The quest of efficiency and certification in polynomial optimization 27 / 42

Rational SOS Decompositions f ∈ Q [ X ] ∩ ˚ Σ [ X ] (interior of the SOS cone) Existence Question Does there exist f i ∈ Q [ X ] , c i ∈ Q > 0 s.t. f = ∑ i c i f i 2 ? Examples � √ � � 2 � 2 � � 2 X + 1 3 X + 1 + 3 1 + X + X 2 = 4 ( 1 ) 2 + = 1 2 2 2 √ � � 2 X 2 + 1 2 X + 1 + 5 1 + X + X 2 + X 3 + X 4 = + 4 √ √ √ � 2 � � � � 10 + 2 5 + 10 − 2 10 − 2 5 5 X + = ??? 4 4 Victor Magron The quest of efficiency and certification in polynomial optimization 27 / 42

Round & Project Algorithm [Peyrl-Parrilo 08] f f ∈ ˚ Σ Σ [ X ] with deg f = 2 D Victor Magron The quest of efficiency and certification in polynomial optimization 28 / 42

Round & Project Algorithm [Peyrl-Parrilo 08] f f ∈ ˚ Σ Σ [ X ] with deg f = 2 D Find ˜ G with SDP at tolerance ˜ δ satisfying f ( X ) ≃ v DT ( X ) ˜ ˜ G v D ( X ) G ≻ 0 v D ( X ) : vector of monomials of deg � D Victor Magron The quest of efficiency and certification in polynomial optimization 28 / 42

Round & Project Algorithm [Peyrl-Parrilo 08] f f ∈ ˚ Σ Σ [ X ] with deg f = 2 D Find ˜ G with SDP at tolerance ˜ δ satisfying f ( X ) ≃ v DT ( X ) ˜ ˜ G v D ( X ) G ≻ 0 v D ( X ) : vector of monomials of deg � D Exact G = ⇒ f γ = ∑ α ′ + β ′ = γ G α ′ , β ′ Victor Magron The quest of efficiency and certification in polynomial optimization 28 / 42

Round & Project Algorithm [Peyrl-Parrilo 08] f f ∈ ˚ Σ Σ [ X ] with deg f = 2 D Find ˜ G with SDP at tolerance ˜ δ satisfying f ( X ) ≃ v DT ( X ) ˜ ˜ G v D ( X ) G ≻ 0 v D ( X ) : vector of monomials of deg � D Exact G = ⇒ f γ = ∑ α ′ + β ′ = γ G α ′ , β ′ f α + β = ∑ α ′ + β ′ = α + β G α ′ , β ′ Victor Magron The quest of efficiency and certification in polynomial optimization 28 / 42

Round & Project Algorithm [Peyrl-Parrilo 08] f f ∈ ˚ Σ Σ [ X ] with deg f = 2 D Find ˜ G with SDP at tolerance ˜ δ satisfying f ( X ) ≃ v DT ( X ) ˜ ˜ G v D ( X ) G ≻ 0 v D ( X ) : vector of monomials of deg � D Exact G = ⇒ f γ = ∑ α ′ + β ′ = γ G α ′ , β ′ f α + β = ∑ α ′ + β ′ = α + β G α ′ , β ′ � ˜ � 1 Rounding step ˆ G , ˆ G ← round δ Victor Magron The quest of efficiency and certification in polynomial optimization 28 / 42

Round & Project Algorithm [Peyrl-Parrilo 08] f f ∈ ˚ Σ Σ [ X ] with deg f = 2 D Find ˜ G with SDP at tolerance ˜ δ satisfying f ( X ) ≃ v DT ( X ) ˜ ˜ G v D ( X ) G ≻ 0 v D ( X ) : vector of monomials of deg � D Exact G = ⇒ f γ = ∑ α ′ + β ′ = γ G α ′ , β ′ f α + β = ∑ α ′ + β ′ = α + β G α ′ , β ′ � ˜ � 1 Rounding step ˆ G , ˆ G ← round δ 2 Projection step G α , β ← ˆ η ( α + β ) ( ∑ α ′ + β ′ = α + β ˆ 1 G α , β − G α ′ , β ′ − f α + β ) Victor Magron The quest of efficiency and certification in polynomial optimization 28 / 42

Round & Project Algorithm [Peyrl-Parrilo 08] f f ∈ ˚ Σ Σ [ X ] with deg f = 2 D Find ˜ G with SDP at tolerance ˜ δ satisfying f ( X ) ≃ v DT ( X ) ˜ ˜ G v D ( X ) G ≻ 0 v D ( X ) : vector of monomials of deg � D Exact G = ⇒ f γ = ∑ α ′ + β ′ = γ G α ′ , β ′ f α + β = ∑ α ′ + β ′ = α + β G α ′ , β ′ � ˜ � 1 Rounding step ˆ G , ˆ G ← round δ 2 Projection step G α , β ← ˆ η ( α + β ) ( ∑ α ′ + β ′ = α + β ˆ 1 G α , β − G α ′ , β ′ − f α + β ) Small enough ˜ δ , ˆ ⇒ f ( X ) = v DT ( X ) G v D ( X ) and G � 0 δ = Victor Magron The quest of efficiency and certification in polynomial optimization 28 / 42

One Answer when K = { x ∈ R n : g j ( x ) � 0 } Hybrid S YMBOLIC /N UMERIC methods Magron-Allamigeon-Gaubert-Werner 14 f ≃ ˜ σ 0 + ˜ σ 1 g 1 + · · · + ˜ σ m g m u = f − ˜ σ 0 + ˜ σ 1 g 1 + · · · + ˜ σ m g m Victor Magron The quest of efficiency and certification in polynomial optimization 29 / 42

One Answer when K = { x ∈ R n : g j ( x ) � 0 } Hybrid S YMBOLIC /N UMERIC methods Magron-Allamigeon-Gaubert-Werner 14 Compact K ⊆ [ 0, 1 ] n f ≃ ˜ σ 0 + ˜ σ 1 g 1 + · · · + ˜ σ m g m u = f − ˜ σ 0 + ˜ σ 1 g 1 + · · · + ˜ σ m g m → ≃ = ∀ x ∈ [ 0, 1 ] n , u ( x ) � − ε min K f � ε when ε → 0 C OMPLEXITY ? Victor Magron The quest of efficiency and certification in polynomial optimization 29 / 42

From Approximate to Exact Solutions Win T WO -P LAYER G AME f Σ ≃ Output ! sum of squares of f ? Victor Magron The quest of efficiency and certification in polynomial optimization 30 / 42

From Approximate to Exact Solutions Win T WO -P LAYER G AME f Σ Hybrid Symbolic/Numeric Algorithms sum of squares of f − ε ? ≃ Output ! Error Compensation → ≃ = Victor Magron The quest of efficiency and certification in polynomial optimization 30 / 42

From Approximate to Exact Solutions Exact SOS Exact SONC/SAGE f f f C SONC C SAGE Σ Victor Magron The quest of efficiency and certification in polynomial optimization 31 / 42

Software: RealCertify and POEM Exact optimization via SOS : RealCertify Maple & arbitrary precision SDP solver SDPA-GMP [Nakata 10] univsos n = 1 n > 1 multivsos Exact optimization via SONC/SAGE : POEM Python ( SymPy ) & geometric programming/relative entropy ECOS [Domahidi-Chu-Boyd 13] Victor Magron The quest of efficiency and certification in polynomial optimization 31 / 42

intsos with n � 1 : Perturbation f Σ P ERTURBATION idea Approximate SOS Decomposition f ( X ) - ε ∑ α ∈P /2 X 2 α = ˜ σ + u Victor Magron The quest of efficiency and certification in polynomial optimization 32 / 42

intsos with n = 1 [Chevillard et. al 11] p p ∈ Q [ X ] , deg p = d = 2 k , p > 0 x p = 1 + X + X 2 + X 3 + X 4 Victor Magron The quest of efficiency and certification in polynomial optimization 33 / 42

intsos with n = 1 [Chevillard et. al 11] p p ∈ Q [ X ] , deg p = d = 2 k , p > 0 p ε P ERTURB : find ε ∈ Q s.t. k X 2 i > 0 ∑ p ε : = p − ε i = 0 4 ( 1 + x 2 + x 4 ) 1 x p = 1 + X + X 2 + X 3 + X 4 ε = 1 4 p > 1 4 ( 1 + X 2 + X 4 ) Victor Magron The quest of efficiency and certification in polynomial optimization 33 / 42

intsos with n = 1 [Chevillard et. al 11] p p ∈ Q [ X ] , deg p = d = 2 k , p > 0 p ε P ERTURB : find ε ∈ Q s.t. k X 2 i > 0 ∑ p ε : = p − ε i = 0 4 ( 1 + x 2 + x 4 ) 1 SDP Approximation: x k X 2 i = ˜ ∑ p − ε σ + u p = 1 + X + X 2 + X 3 + X 4 i = 0 ε = 1 4 A BSORB : small enough u i i = 0 X 2 i + u SOS p > 1 ⇒ ε ∑ k 4 ( 1 + X 2 + X 4 ) = Victor Magron The quest of efficiency and certification in polynomial optimization 33 / 42

intsos with n = 1 and SDP Approximation Input f � 0 ∈ Q [ X ] of degree d � 2 , ε ∈ Q > 0 , δ ∈ N > 0 Output : SOS decomposition with coefficients in Q h , ˜ σ , ε , u f ( p , h ) ← sqrfree ( f ) k X 2 i ∑ p ε ← p − ε σ ← sdp ( p ε , δ ) ˜ i = 0 u ← p ε − ˜ σ ε ← ε δ ← 2 δ 2 while while p ε ≤ 0 ε < | u 2 i + 1 | + | u 2 i − 1 | − u 2 i 2 Victor Magron The quest of efficiency and certification in polynomial optimization 34 / 42

intsos with n = 1 : Absorbtion � ( X + 1 ) 2 − 1 − X 2 � X = 1 2 � ( X − 1 ) 2 − 1 − X 2 � − X = 1 2 Victor Magron The quest of efficiency and certification in polynomial optimization 35 / 42

intsos with n = 1 : Absorbtion � ( X + 1 ) 2 − 1 − X 2 � X = 1 2 � ( X − 1 ) 2 − 1 − X 2 � − X = 1 2 u 2 i + 1 X 2 i + 1 = | u 2 i + 1 | � ( X i + 1 + sgn ( u 2 i + 1 ) X i ) 2 − X 2 i − X 2 i + 2 � 2 Victor Magron The quest of efficiency and certification in polynomial optimization 35 / 42

intsos with n = 1 : Absorbtion � ( X + 1 ) 2 − 1 − X 2 � X = 1 2 � ( X − 1 ) 2 − 1 − X 2 � − X = 1 2 u 2 i + 1 X 2 i + 1 = | u 2 i + 1 | � ( X i + 1 + sgn ( u 2 i + 1 ) X i ) 2 − X 2 i − X 2 i + 2 � 2 u · · · · · · 2 i − 2 2 i − 1 2 i 2 i + 1 2 i + 2 ε ∑ k i = 0 X 2 i ε ε ε Victor Magron The quest of efficiency and certification in polynomial optimization 35 / 42

intsos with n = 1 : Absorbtion � ( X + 1 ) 2 − 1 − X 2 � X = 1 2 � ( X − 1 ) 2 − 1 − X 2 � − X = 1 2 u 2 i + 1 X 2 i + 1 = | u 2 i + 1 | � ( X i + 1 + sgn ( u 2 i + 1 ) X i ) 2 − X 2 i − X 2 i + 2 � 2 u · · · · · · 2 i − 2 2 i − 1 2 i 2 i + 1 2 i + 2 ε ∑ k i = 0 X 2 i ε ε ε k ε � | u 2 i + 1 | + | u 2 i − 1 | X 2 i + u ∑ − u 2 i = ⇒ ε SOS 2 i = 0 Victor Magron The quest of efficiency and certification in polynomial optimization 35 / 42

intsos with n � 1 : Absorbtion f ( X ) - ε ∑ α ∈P /2 X 2 α = ˜ σ + u Choice of P ? y 6 ε 2 ( x + y 3 ) 2 − x 2 + y 6 xy 3 = 1 5 2 4 u 1,3 3 2 1 ε x 0 1 2 3 4 5 Victor Magron The quest of efficiency and certification in polynomial optimization 36 / 42

intsos with n � 1 : Absorbtion f ( X ) - ε ∑ α ∈P /2 X 2 α = ˜ σ + u Choice of P ? y 6 2 ( xy + y 2 ) 2 − x 2 y 2 + y 4 xy 3 = 1 5 2 ε 4 u 1,3 3 ε 2 1 x 0 1 2 3 4 5 Victor Magron The quest of efficiency and certification in polynomial optimization 36 / 42

intsos with n � 1 : Absorbtion f ( X ) - ε ∑ α ∈P /2 X 2 α = ˜ σ + u Choice of P ? y 6 2 ( xy 2 + y ) 2 − x 2 y 4 + y 2 xy 3 = 1 5 2 ε 4 3 u 1,3 ε 2 1 x 0 1 2 3 4 5 Victor Magron The quest of efficiency and certification in polynomial optimization 36 / 42

intsos with n � 1 : Absorbtion f ( X ) - ε ∑ α ∈P /2 X 2 α = ˜ σ + u Choice of P ? f = 4 x 4 y 6 + x 2 − xy 2 + y 2 spt ( f ) = { ( 4, 6 ) , ( 2, 0 ) , ( 1, 2 ) , ( 0, 2 ) } Newton Polytope P = conv ( spt ( f )) Squares in SOS decomposition ⊆ P 2 ∩ N n [Reznick 78] Victor Magron The quest of efficiency and certification in polynomial optimization 36 / 42

Algorithm intsos Input f ∈ Q [ X ] ∩ ˚ Σ [ X ] of degree d , ε ∈ Q > 0 , δ ∈ N > 0 Output : SOS decomposition with coefficients in Q h , ˜ σ , ε , u f P ← conv ( spt ( f )) f ε ← f − ε ∑ X 2 α σ ← sdp ( f ε , δ ) ˜ α ∈P /2 u ← f ε − ˜ σ ε ← ε δ ← 2 δ 2 while while f ε ≤ 0 u + ε ∑ X 2 α / ∈ Σ α ∈P /2 Victor Magron The quest of efficiency and certification in polynomial optimization 37 / 42

Algorithm intsos Theorem (Exact Certification Cost in ˚ Σ ) f ∈ Q [ X ] ∩ ˚ Σ [ X ] with deg f = d = 2 k and bit size τ ⇒ intsos terminates with SOS output of bit size τ d O ( n ) = Victor Magron The quest of efficiency and certification in polynomial optimization 38 / 42

Algorithm intsos Theorem (Exact Certification Cost in ˚ Σ ) f ∈ Q [ X ] ∩ ˚ Σ [ X ] with deg f = d = 2 k and bit size τ ⇒ intsos terminates with SOS output of bit size τ d O ( n ) = Victor Magron The quest of efficiency and certification in polynomial optimization 38 / 42

Algorithm Polyasos f positive definite form has Polya ’s representation: σ f = with σ ∈ Σ [ X ] ( X 1 + · · · + X n ) 2 D Victor Magron The quest of efficiency and certification in polynomial optimization 39 / 42

Algorithm Polyasos f positive definite form has Polya ’s representation: σ f = with σ ∈ Σ [ X ] ( X 1 + · · · + X n ) 2 D Theorem f ( X 1 + · · · + X n ) 2 D ∈ Σ [ X ] = ⇒ f ( X 1 + · · · + X n ) 2 D + 2 ∈ ˚ Σ [ X ] Victor Magron The quest of efficiency and certification in polynomial optimization 39 / 42

Algorithm Polyasos f positive definite form has Polya ’s representation: σ f = with σ ∈ Σ [ X ] ( X 1 + · · · + X n ) 2 D Theorem f ( X 1 + · · · + X n ) 2 D ∈ Σ [ X ] = ⇒ f ( X 1 + · · · + X n ) 2 D + 2 ∈ ˚ Σ [ X ] Apply Algorithm intsos on f ( X 1 + · · · + X n ) 2 D + 2 Victor Magron The quest of efficiency and certification in polynomial optimization 39 / 42