New Applications of Moment-SOS Hierarchies Victor Magron , RA - PowerPoint PPT Presentation

Primal-dual Moment-SOS [Lasserre 01] M + ( S ) : space of probability measures supported on S Q ( S ) : quadratic module Polynomial Optimization Problems (POP) (Primal) (Dual) � inf S f d µ = sup λ s.t. µ ∈ M + ( S ) s.t. λ ∈ R , f − λ ∈ Q ( S ) V. Magron New Applications of Moment-SOS Hierarchies 14 / 52

Primal-dual Moment-SOS [Lasserre 01] Finite moment sequences z of measures in M + ( S ) Truncated quadratic module Q k ( S ) : = Q ( S ) ∩ R 2 k [ x ] Polynomial Optimization Problems (POP) (Moment) (SOS) ∑ = inf f α z α sup λ α M k − v j ( g j z ) � 0 , λ ∈ R , s.t. 0 � j � l , s.t. z 1 = 1 f − λ ∈ Q k ( S ) V. Magron New Applications of Moment-SOS Hierarchies 14 / 52

Semidefinite Optimization F 0 , F α symmetric real matrices, cost vector c Primal-dual pair of semidefinite programs:  P : inf z ∑ α c α z α    ∑ α F α z α − F 0 � 0 s.t.    ( SDP )  D : sup Y Trace ( F 0 Y )     s.t. Trace ( F α Y ) = c α , Y � 0 .  Freely available SDP solvers ( CSDP , SDPA , S EDUMI ) V. Magron New Applications of Moment-SOS Hierarchies 15 / 52

Moment-SOS Hierarchies for Polynomial Optimization New Applications of Moment-SOS Hierarchies Conclusion

Moment-SOS Hierarchies for Polynomial Optimization New Applications of Moment-SOS Hierarchies Semialgebraic Maxplus Optimization Formal Nonlinear Optimization Pareto Curves Polynomial Images of Semialgebraic Sets SDP for Program Verification Ongoing: Bounding Floating-point Errors Conclusion

General informal Framework Given K a compact set and f a transcendental function, bound f ∗ = inf x ∈ K f ( x ) and prove f ∗ � 0 f is under-approximated by a semialgebraic function f sa Reduce the problem f ∗ sa : = inf x ∈ K f sa ( x ) to a polynomial optimization problem (POP) V. Magron New Applications of Moment-SOS Hierarchies 16 / 52

General informal Framework Approximation theory: Chebyshev/Taylor models mandatory for non-polynomial problems hard to combine with Sum-of-Squares techniques (degree of approximation) V. Magron New Applications of Moment-SOS Hierarchies 16 / 52

Maxplus Approximation Initially introduced to solve Optimal Control Problems [Fleming-McEneaney 00] Curse of dimensionality reduction [McEaneney Kluberg, Gaubert-McEneaney-Qu 11, Qu 13]. Allowed to solve instances of dim up to 15 (inaccessible by grid methods) In our context: approximate transcendental functions V. Magron New Applications of Moment-SOS Hierarchies 17 / 52

Maxplus Approximation Definition Let γ � 0. A function φ : R n → R is said to be γ -semiconvex if the function x �→ φ ( x ) + γ 2 � x � 2 2 is convex. y par + a 2 par + a 1 arctan par − a 2 a a 1 a 2 m M par − a 1 V. Magron New Applications of Moment-SOS Hierarchies 17 / 52

Nonlinear Function Representation Exact parsimonious maxplus representations y a V. Magron New Applications of Moment-SOS Hierarchies 18 / 52

Nonlinear Function Representation Exact parsimonious maxplus representations y a V. Magron New Applications of Moment-SOS Hierarchies 18 / 52

Nonlinear Function Representation Abstract syntax tree representations of multivariate transcendental functions: leaves are semialgebraic functions of A nodes are univariate functions of D or binary operations V. Magron New Applications of Moment-SOS Hierarchies 18 / 52

Nonlinear Function Representation For the “Simple” Example from Flyspeck: + l ( x ) arctan r ( x ) V. Magron New Applications of Moment-SOS Hierarchies 18 / 52

Maxplus Optimization Algorithm First iteration: y + arctan par − l ( x ) arctan a 1 a a 1 m M r ( x ) 1 control point { a 1 } : m 1 = − 4.7 × 10 − 3 < 0 V. Magron New Applications of Moment-SOS Hierarchies 19 / 52

Maxplus Optimization Algorithm Second iteration: y + arctan par − l ( x ) arctan a 1 a a 2 a 1 m M r ( x ) par − a 2 2 control points { a 1 , a 2 } : m 2 = − 6.1 × 10 − 5 < 0 V. Magron New Applications of Moment-SOS Hierarchies 19 / 52

Maxplus Optimization Algorithm Third iteration: y + arctan par − par − a 3 l ( x ) arctan a 1 a a 2 a 3 a 1 m M r ( x ) par − a 2 3 control points { a 1 , a 2 , a 3 } : m 3 = 4.1 × 10 − 6 > 0 OK! V. Magron New Applications of Moment-SOS Hierarchies 19 / 52

Contributions V. Magron, X. Allamigeon, S. Gaubert, and B. Werner. Certification of Real Inequalities – Templates and Sums of Squares, arxiv:1403.5899, 2014. Accepted for publication in Mathematical Programming SERIES B, volume on Polynomial Optimization . V. Magron New Applications of Moment-SOS Hierarchies 20 / 52

Moment-SOS Hierarchies for Polynomial Optimization New Applications of Moment-SOS Hierarchies Semialgebraic Maxplus Optimization Formal Nonlinear Optimization Pareto Curves Polynomial Images of Semialgebraic Sets SDP for Program Verification Ongoing: Bounding Floating-point Errors Conclusion

The General “Formal Framework” We check the correctness of SOS certificates for POP We build certificates to prove interval bounds for semialgebraic functions We bound formally transcendental functions with semialgebraic approximations V. Magron New Applications of Moment-SOS Hierarchies 21 / 52

Formal SOS bounds When q ∈ Q ( K ) , σ 0 , . . . , σ m is a positivity certificate for q Check symbolic polynomial equalities q = q ′ in C OQ Existing tactic ring [Grégoire-Mahboubi 05] Polynomials coefficients: arbitrary-size rationals bigQ [Grégoire-Théry 06] Much simpler to verify certificates using sceptical approach Extends also to semialgebraic functions V. Magron New Applications of Moment-SOS Hierarchies 22 / 52

Formal Nonlinear Optimization Inequality #boxes Time Time 9922699028 39 190 s 2218 s 3318775219 338 1560 s 19136 s Comparable with Taylor interval methods in H OL - LIGHT [Hales-Solovyev 13] Bottleneck of informal optimizer is SOS solver 22 times slower! = ⇒ Current bottleneck is to check polynomial equalities V. Magron New Applications of Moment-SOS Hierarchies 23 / 52

Contribution For more details on the formal side: V. M., X. Allamigeon, S. Gaubert and B. Werner. Formal Proofs for Nonlinear Optimization, arxiv:1404.7282, 2015. Journal of Formalized Reasoning . V. Magron New Applications of Moment-SOS Hierarchies 24 / 52

Moment-SOS Hierarchies for Polynomial Optimization New Applications of Moment-SOS Hierarchies Semialgebraic Maxplus Optimization Formal Nonlinear Optimization Pareto Curves Polynomial Images of Semialgebraic Sets SDP for Program Verification Ongoing: Bounding Floating-point Errors Conclusion

Bicriteria Optimization Problems Let f 1 , f 2 ∈ R [ x ] two conflicting criteria Let S : = { x ∈ R n : g 1 ( x ) � 0, . . . , g m ( x ) � 0 } a semialgebraic set � � x ∈ S ( f 1 ( x ) f 2 ( x )) ⊤ ( P ) min Assumption The image space R 2 is partially ordered in a natural way ( R 2 + is the ordering cone). V. Magron New Applications of Moment-SOS Hierarchies 25 / 52

Bicriteria Optimization Problems f 1 : = ( x 1 + x 2 − 7.5 ) 2 /4 + ( − x 1 + x 2 + 3 ) 2 , g 1 : = − ( x 1 − 2 ) 3 /2 − x 2 + 2.5 , g 2 : = − x 1 − x 2 + 8 ( − x 1 + x 2 + 0.65 ) 2 + 3.85 , f 2 : = ( x 1 − 1 ) 2 /4 + ( x 2 − 4 ) 2 /4 . S : = { x ∈ R 2 : g 1 ( x ) � 0, g 2 ( x ) � 0 } . V. Magron New Applications of Moment-SOS Hierarchies 25 / 52

Parametric Sublevel Set Approximations Inspired by previous research on multiobjective linear optimization [Gorissen-den Hertog 12] Workaround: reduce P to a parametric POP f ∗ ( λ ) : = min ( P λ ) : x ∈ S { f 2 ( x ) : f 1 ( x ) � λ } , variable ( x , λ ) ∈ K = S × [ 0, 1 ] V. Magron New Applications of Moment-SOS Hierarchies 26 / 52

A Hierarchy of Polynomial Approximations Moment-SOS approach [Lasserre 10]:  2 k ∑  q i / ( 1 + i ) max  ( D k ) q ∈ R 2 k [ λ ] i = 0  s.t. f 2 ( x ) − q ( λ ) ∈ Q 2 k ( K ) .  The hierarchy ( D k ) provides a sequence ( q k ) of polynomial under-approximations of f ∗ ( λ ) . � 1 0 ( f ∗ ( λ ) − q k ( λ )) d λ = 0 lim d → ∞ V. Magron New Applications of Moment-SOS Hierarchies 27 / 52

A Hierarchy of Polynomial Approximations Degree 4 V. Magron New Applications of Moment-SOS Hierarchies 28 / 52

A Hierarchy of Polynomial Approximations Degree 6 V. Magron New Applications of Moment-SOS Hierarchies 28 / 52

A Hierarchy of Polynomial Approximations Degree 8 V. Magron New Applications of Moment-SOS Hierarchies 28 / 52

Contributions Numerical schemes that avoid computing finitely many points . Pareto curve approximation with polynomials, convergence guarantees in L 1 -norm V. Magron, D. Henrion, J.B. Lasserre. Approximating Pareto Curves using Semidefinite Relaxations. Operations Research Letters . arxiv:1404.4772, April 2014. V. Magron New Applications of Moment-SOS Hierarchies 29 / 52

Moment-SOS Hierarchies for Polynomial Optimization New Applications of Moment-SOS Hierarchies Semialgebraic Maxplus Optimization Formal Nonlinear Optimization Pareto Curves Polynomial Images of Semialgebraic Sets SDP for Program Verification Ongoing: Bounding Floating-point Errors Conclusion

Polynomial Images of Semialgebraic Sets Semialgebraic set S : = { x ∈ R n : g 1 ( x ) � 0, . . . , g l ( x ) � 0 } A polynomial map f : R n → R m , x �→ f ( x ) : = ( f 1 ( x ) , . . . , f m ( x )) deg f = d : = max { deg f 1 , . . . , deg f m } F : = f ( S ) ⊆ B , with B ⊂ R m a box or a ball Tractable approximations of F ? V. Magron New Applications of Moment-SOS Hierarchies 30 / 52

Polynomial Images of Semialgebraic Sets Includes important special cases: 1 m = 1: polynomial optimization F ⊆ [ inf x ∈ S f ( x ) , sup f ( x )] x ∈ S 2 Approximate projections of S when f ( x ) : = ( x 1 , . . . , x m ) 3 Pareto curve approximations � � x ∈ S ( f 1 ( x ) f 2 ( x )) ⊤ For f 1 , f 2 two conflicting criteria: ( P ) min V. Magron New Applications of Moment-SOS Hierarchies 30 / 52

Method 1: Existential Quantifier Elimination Another point of view: F = { y ∈ B : ∃ x ∈ S s.t. f ( x ) = y } , V. Magron New Applications of Moment-SOS Hierarchies 31 / 52

Method 1: Existential Quantifier Elimination Another point of view: F = { y ∈ B : ∃ x ∈ S s.t. � y − f ( x ) � 2 2 = 0 } , V. Magron New Applications of Moment-SOS Hierarchies 31 / 52

Method 1: Existential Quantifier Elimination Another point of view: F = { y ∈ B : ∃ x ∈ S s.t. h f ( x , y ) � 0 } , with h f ( x , y ) : = −� y − f ( x ) � 2 2 . V. Magron New Applications of Moment-SOS Hierarchies 31 / 52

Method 1: Existential Quantifier Elimination Existential QE: approximate F as closely as desired [Lasserre 14] F 1 k : = { y ∈ B : q k ( y ) � 0 } , for some polynomials q k ∈ R 2 k [ y ] . V. Magron New Applications of Moment-SOS Hierarchies 31 / 52

Method 1: Outer Approximations of F Let K = S × B , Q k ( K ) be the k -truncated quadratic module REMEMBER : q − h f ∈ Q k ( K ) = ⇒ ∀ ( x , y ) ∈ K , q ( y ) − h f ( x , y ) � 0 V. Magron New Applications of Moment-SOS Hierarchies 32 / 52

Method 1: Outer Approximations of F Let K = S × B , Q k ( K ) be the k -truncated quadratic module REMEMBER : q − h f ∈ Q k ( K ) = ⇒ ∀ ( x , y ) ∈ K , q ( y ) − h f ( x , y ) � 0 Define h ( y ) : = sup x ∈ S h f ( x , y ) V. Magron New Applications of Moment-SOS Hierarchies 32 / 52

Method 1: Outer Approximations of F Let K = S × B , Q k ( K ) be the k -truncated quadratic module REMEMBER : q − h f ∈ Q k ( K ) = ⇒ ∀ ( x , y ) ∈ K , q ( y ) − h f ( x , y ) � 0 Define h ( y ) : = sup x ∈ S h f ( x , y ) Hierarchy of Semidefinite programs: � � � B ( q − h ) d y : q − h f ∈ Q k ( K )) inf . q V. Magron New Applications of Moment-SOS Hierarchies 32 / 52

Method 1: Outer Approximations of F Assuming the existence of solution q k , the sublevel sets F 1 k : = { y ∈ B : q k ( y ) � 0 } ⊇ F , provide a sequence of certified outer approximations of F . V. Magron New Applications of Moment-SOS Hierarchies 33 / 52

Method 1: Outer Approximations of F Assuming the existence of solution q k , the sublevel sets F 1 k : = { y ∈ B : q k ( y ) � 0 } ⊇ F , provide a sequence of certified outer approximations of F . It comes from the following: q k feasible solution, q k − h f ∈ Q k ( K ) ∀ ( x , y ) ∈ K , q k ( y ) � h f ( x , y ) ⇐ ⇒ ∀ y , q k ( y ) � h ( y ) . V. Magron New Applications of Moment-SOS Hierarchies 33 / 52

Method 1: Strong Convergence Property Theorem ◦ Assuming that S � = ∅ and Q k ( K ) is Archimedean, 1 The sequence of optimal solutions ( q k ) converges to h w.r.t the L 1 ( B ) -norm: � B | q k − h | d y = 0 , ( q k → L 1 h ) lim k → ∞ V. Magron New Applications of Moment-SOS Hierarchies 34 / 52

Method 1: Strong Convergence Property Theorem ◦ Assuming that S � = ∅ and Q k ( K ) is Archimedean, 1 The sequence of optimal solutions ( q k ) converges to h w.r.t the L 1 ( B ) -norm: � B | q k − h | d y = 0 , ( q k → L 1 h ) lim k → ∞ 2 k → ∞ vol ( F 1 lim k \ F ) = 0 . V. Magron New Applications of Moment-SOS Hierarchies 34 / 52

Method 2: Support of Image Measures Pushforward f # : M ( S ) → M ( B ) : f # µ 0 ( A ) : = µ 0 ( { x ∈ S : f ( x ) ∈ A } ) , ∀ A ∈ B ( B ) , ∀ µ 0 ∈ M ( S ) f # µ 0 is the image measure of µ 0 under f V. Magron New Applications of Moment-SOS Hierarchies 35 / 52

Method 2: Support of Image Measures � p ∗ : = B µ 1 sup µ 0 , µ 1 , ˆ µ 1 µ 1 + ˆ µ 1 = λ B , s.t. µ 1 = f # µ 0 , µ 0 ∈ M + ( S ) , µ 1 , ˆ µ 1 ∈ M + ( B ) . Lebesgue measure on B is λ B ( d y ) : = 1 B ( y ) d y V. Magron New Applications of Moment-SOS Hierarchies 35 / 52

Method 2: Support of Image Measures � p ∗ : = sup B µ 1 µ 0 , µ 1 , ˆ µ 1 µ 1 + ˆ µ 1 = λ B , s.t. µ 1 = f # µ 0 , µ 0 ∈ M + ( S ) , µ 1 ∈ M + ( B ) . µ 1 , ˆ Lemma Let µ ∗ 1 be an optimal solution of the above LP. 1 = λ F and p ∗ = vol F . Then µ ∗ V. Magron New Applications of Moment-SOS Hierarchies 35 / 52

Method 2: Primal-dual LP Formulation Primal LP Dual LP � p ∗ : = � d ∗ : = inf sup µ 1 w ( y ) λ B ( d y ) v , w µ 0 , µ 1 , ˆ µ 1 v ( f ( x )) � 0, ∀ x ∈ S , s.t. µ 1 + ˆ µ 1 = λ B , s.t. w ( y ) � 1 + v ( y ) , ∀ y ∈ B , µ 1 = f # µ 0 , w ( y ) � 0, ∀ y ∈ B , µ 0 ∈ M + ( S ) , v , w ∈ C ( B ) . µ 1 ∈ M + ( B ) . µ 1 , ˆ V. Magron New Applications of Moment-SOS Hierarchies 36 / 52

Method 2: Strong Convergence Property Strengthening of the dual LP: d ∗ ∑ w β z B k : = inf β v , w β ∈ N m 2 k s.t. v ◦ f ∈ Q kd ( S ) , w − 1 − v ∈ Q k ( B ) , w ∈ Q k ( B ) , v , w ∈ R 2 k [ y ] . V. Magron New Applications of Moment-SOS Hierarchies 37 / 52

Method 2: Strong Convergence Property Theorem ◦ F � = ∅ and Q k ( S ) is Archimedean, Assuming that 1 The sequence ( w k ) converges to 1 F w.r.t the L 1 ( B ) -norm: � lim B | w k − 1 F | d y = 0 . k → ∞ V. Magron New Applications of Moment-SOS Hierarchies 38 / 52

Method 2: Strong Convergence Property Theorem ◦ F � = ∅ and Q k ( S ) is Archimedean, Assuming that 1 The sequence ( w k ) converges to 1 F w.r.t the L 1 ( B ) -norm: � lim B | w k − 1 F | d y = 0 . k → ∞ 2 Let F 2 k : = { y ∈ B : w k ( y ) � 1 } . Then, k → ∞ vol ( F 2 k \ F ) = 0 . lim V. Magron New Applications of Moment-SOS Hierarchies 38 / 52

Polynomial Image of the Unit Ball Image of the unit ball S : = { x ∈ R 2 : � x � 2 2 � 1 } by f ( x ) : = ( x 1 + x 1 x 2 , x 2 − x 3 1 ) /2 F 1 F 2 1 1 V. Magron New Applications of Moment-SOS Hierarchies 39 / 52

Polynomial Image of the Unit Ball Image of the unit ball S : = { x ∈ R 2 : � x � 2 2 � 1 } by f ( x ) : = ( x 1 + x 1 x 2 , x 2 − x 3 1 ) /2 F 1 F 2 2 2 V. Magron New Applications of Moment-SOS Hierarchies 39 / 52

Polynomial Image of the Unit Ball Image of the unit ball S : = { x ∈ R 2 : � x � 2 2 � 1 } by f ( x ) : = ( x 1 + x 1 x 2 , x 2 − x 3 1 ) /2 F 1 F 2 3 3 V. Magron New Applications of Moment-SOS Hierarchies 39 / 52

Polynomial Image of the Unit Ball Image of the unit ball S : = { x ∈ R 2 : � x � 2 2 � 1 } by f ( x ) : = ( x 1 + x 1 x 2 , x 2 − x 3 1 ) /2 F 1 F 2 4 4 V. Magron New Applications of Moment-SOS Hierarchies 39 / 52

Semialgebraic Set Projections f ( x ) = ( x 1 , x 2 ) : projection on R 2 of the semialgebraic set 2 � 1, 1/4 − ( x 1 + 1/2 ) 2 − x 2 S : = { x ∈ R 3 : � x � 2 2 � 0, 1/9 − ( x 1 − 1/2 ) 4 − x 4 2 � 0 } F 1 F 2 2 2 V. Magron New Applications of Moment-SOS Hierarchies 40 / 52

Semialgebraic Set Projections f ( x ) = ( x 1 , x 2 ) : projection on R 2 of the semialgebraic set 2 � 1, 1/4 − ( x 1 + 1/2 ) 2 − x 2 S : = { x ∈ R 3 : � x � 2 2 � 0, 1/9 − ( x 1 − 1/2 ) 4 − x 4 2 � 0 } F 1 F 2 3 3 V. Magron New Applications of Moment-SOS Hierarchies 40 / 52

Semialgebraic Set Projections f ( x ) = ( x 1 , x 2 ) : projection on R 2 of the semialgebraic set 2 � 1, 1/4 − ( x 1 + 1/2 ) 2 − x 2 S : = { x ∈ R 3 : � x � 2 2 � 0, 1/9 − ( x 1 − 1/2 ) 4 − x 4 2 � 0 } F 1 F 2 4 4 V. Magron New Applications of Moment-SOS Hierarchies 40 / 52

Approximating Pareto Curves Back on our previous nonconvex example: F 1 F 2 1 1 V. Magron New Applications of Moment-SOS Hierarchies 41 / 52

Approximating Pareto Curves Back on our previous nonconvex example: F 1 F 2 2 2 V. Magron New Applications of Moment-SOS Hierarchies 41 / 52

Approximating Pareto Curves Back on our previous nonconvex example: F 1 F 2 3 3 V. Magron New Applications of Moment-SOS Hierarchies 41 / 52

Approximating Pareto Curves “Zoom” on the region which is hard to approximate: F 1 4 V. Magron New Applications of Moment-SOS Hierarchies 42 / 52

Approximating Pareto Curves “Zoom” on the region which is hard to approximate: F 1 5 V. Magron New Applications of Moment-SOS Hierarchies 42 / 52

Semialgebraic Image of Semialgebraic Sets Image of the unit ball S : = { x ∈ R 2 : � x � 2 2 � 1 } by f ( x ) : = ( min ( x 1 + x 1 x 2 , x 2 1 ) , x 2 − x 3 1 ) /3 F 1 F 2 1 1 V. Magron New Applications of Moment-SOS Hierarchies 43 / 52

Semialgebraic Image of Semialgebraic Sets Image of the unit ball S : = { x ∈ R 2 : � x � 2 2 � 1 } by f ( x ) : = ( min ( x 1 + x 1 x 2 , x 2 1 ) , x 2 − x 3 1 ) /3 F 1 F 2 2 2 V. Magron New Applications of Moment-SOS Hierarchies 43 / 52

Semialgebraic Image of Semialgebraic Sets Image of the unit ball S : = { x ∈ R 2 : � x � 2 2 � 1 } by f ( x ) : = ( min ( x 1 + x 1 x 2 , x 2 1 ) , x 2 − x 3 1 ) /3 F 1 F 2 3 3 V. Magron New Applications of Moment-SOS Hierarchies 43 / 52

Semialgebraic Image of Semialgebraic Sets Image of the unit ball S : = { x ∈ R 2 : � x � 2 2 � 1 } by f ( x ) : = ( min ( x 1 + x 1 x 2 , x 2 1 ) , x 2 − x 3 1 ) /3 F 1 F 2 4 4 V. Magron New Applications of Moment-SOS Hierarchies 43 / 52

Contributions V. Magron, D. Henrion, J.B. Lasserre. Semidefinite approximations of projections and polynomial images of semialgebraic sets. oo:2014.10.4606, October 2014. V. Magron New Applications of Moment-SOS Hierarchies 44 / 52

Moment-SOS Hierarchies for Polynomial Optimization New Applications of Moment-SOS Hierarchies Semialgebraic Maxplus Optimization Formal Nonlinear Optimization Pareto Curves Polynomial Images of Semialgebraic Sets SDP for Program Verification Ongoing: Bounding Floating-point Errors Conclusion