Automated Precision Tuning using Semidefinite Programming Victor - PowerPoint PPT Presentation

Automated Precision Tuning using Semidefinite Programming Victor Magron , RA Imperial College joint work with G. Constantinides and A. Donaldson British-French-German Conference on Optimization 15 June 2015 Victor Magron Automated Precision Tuning using Semidefinite Programming 1 / 27

Errors and Proofs Mathematicians and Computer Scientists want to eliminate all the uncertainties on their results. Why? Victor Magron Automated Precision Tuning using Semidefinite Programming 2 / 27

Errors and Proofs Mathematicians and Computer Scientists want to eliminate all the uncertainties on their results. Why? M. Lecat, Erreurs des Mathématiciens des origines à nos jours, 1935. ❀ 130 pages of errors! (Euler, Fermat, Sylvester, . . . ) Victor Magron Automated Precision Tuning using Semidefinite Programming 2 / 27

Errors and Proofs Mathematicians and Computer Scientists want to eliminate all the uncertainties on their results. Why? M. Lecat, Erreurs des Mathématiciens des origines à nos jours, 1935. ❀ 130 pages of errors! (Euler, Fermat, Sylvester, . . . ) Ariane 5 launch failure, Pentium FDIV bug Victor Magron Automated Precision Tuning using Semidefinite Programming 2 / 27

Errors and Proofs G UARANTEED O PTIMIZATION Input : Linear problem (LP), geometric, semidefinite (SDP) Output : solution + certificate numeric-symbolic ❀ formal Victor Magron Automated Precision Tuning using Semidefinite Programming 3 / 27

Errors and Proofs G UARANTEED O PTIMIZATION Input : Linear problem (LP), geometric, semidefinite (SDP) Output : solution + certificate numeric-symbolic ❀ formal V ERIFICATION OF CRITICAL SYSTEMS Reliable software/hardware embedded codes Aerospace control molecular biology, robotics, code synthesis, . . . Victor Magron Automated Precision Tuning using Semidefinite Programming 3 / 27

Errors and Proofs G UARANTEED O PTIMIZATION Input : Linear problem (LP), geometric, semidefinite (SDP) Output : solution + certificate numeric-symbolic ❀ formal V ERIFICATION OF CRITICAL SYSTEMS Reliable software/hardware embedded codes Aerospace control molecular biology, robotics, code synthesis, . . . Efficient Verification of Nonlinear Systems Automated precision tuning of systems/programs analysis/synthesis Efficiency sparsity correlation patterns Certified approximation algorithms Victor Magron Automated Precision Tuning using Semidefinite Programming 3 / 27

Rounding Error Bounds Real : p ( x ) : = x 1 × x 2 + x 3 p ( x , e ) : = [ x 1 x 2 ( 1 + e 1 ) + x 3 ]( 1 + e 2 ) Floating-point : ˆ Input variable uncertainties x ∈ S Finite precision ❀ bounds over e | e i | � 2 − m m = 24 (single) or 53 (double) Guarantees on absolute round-off error | ˆ p − p | ? Victor Magron Automated Precision Tuning using Semidefinite Programming 4 / 27

Nonlinear Programs Polynomials programs : + , − , × x 2 x 5 + x 3 x 6 + x 1 ( − x 1 + x 2 + x 3 − x 4 + x 5 + x 6 ) Victor Magron Automated Precision Tuning using Semidefinite Programming 5 / 27

Nonlinear Programs Polynomials programs : + , − , × x 2 x 5 + x 3 x 6 + x 1 ( − x 1 + x 2 + x 3 − x 4 + x 5 + x 6 ) Semialgebraic programs: | · | , √ , /, sup, inf 4 x x 1 + 1.11 Victor Magron Automated Precision Tuning using Semidefinite Programming 5 / 27

Nonlinear Programs Polynomials programs : + , − , × x 2 x 5 + x 3 x 6 + x 1 ( − x 1 + x 2 + x 3 − x 4 + x 5 + x 6 ) Semialgebraic programs: | · | , √ , /, sup, inf 4 x x 1 + 1.11 Transcendental programs: arctan, exp, log, . . . log ( 1 + exp ( x )) Victor Magron Automated Precision Tuning using Semidefinite Programming 5 / 27

Existing Frameworks Classical methods : Abstract domains [Goubault-Putot 11] F LUCTUAT : intervals, octagons, zonotopes Interval arithmetic [Daumas-Melquiond 10] G APPA : interface with C OQ proof assistant Victor Magron Automated Precision Tuning using Semidefinite Programming 6 / 27

Existing Frameworks Recent progress : Affine arithmetic + SMT [Darulova 14] rosa : sound compiler for reals (in S CALA ) Symbolic Taylor expansions [Solovyev 15] FPTaylor : certified optimization (in OC AML and H OL - LIGHT ) Victor Magron Automated Precision Tuning using Semidefinite Programming 6 / 27

Contributions Maximal Rounding error of the program implementation of f : r ⋆ : = max | ˆ f ( x , e ) − f ( x ) | Decomposition: linear term l w.r.t. e + nonlinear term h r ⋆ � max | l ( x , e ) | + max | h ( x , e ) | Sparse SDP bounds for l Coarse bound of h with interval arithmetic Victor Magron Automated Precision Tuning using Semidefinite Programming 7 / 27

Contributions 1 Comparison with SMT and linear/affine arithmetic: ❀ More Efficient optimization + � Tighter upper bounds 2 Extensions to transcendental/conditional programs 3 Formal verification of SDP bounds 4 Open source tool Real2Float (in OC AML and C OQ ) Victor Magron Automated Precision Tuning using Semidefinite Programming 7 / 27

Introduction Semidefinite Programming for Polynomial Optimization Rounding Error Bounds with Sparse SDP Conclusion

What is Semidefinite Programming? Linear Programming (LP): ⊤ z min c z s.t. A z � d . Linear cost c Polyhedron Linear inequalities “ ∑ i A ij z j � d i ” Victor Magron Automated Precision Tuning using Semidefinite Programming 8 / 27

What is Semidefinite Programming? Semidefinite Programming (SDP): ⊤ z min c z ∑ s.t. F i z i � F 0 . i Linear cost c Symmetric matrices F 0 , F i Spectrahedron Linear matrix inequalities “ F � 0” ( F has nonnegative eigenvalues) Victor Magron Automated Precision Tuning using Semidefinite Programming 9 / 27

What is Semidefinite Programming? Semidefinite Programming (SDP): ⊤ z min c z ∑ A z = d . s.t. F i z i � F 0 , i Linear cost c Symmetric matrices F 0 , F i Spectrahedron Linear matrix inequalities “ F � 0” ( F has nonnegative eigenvalues) Victor Magron Automated Precision Tuning using Semidefinite Programming 10 / 27

Applications of SDP Combinatorial optimization Control theory Matrix completion Unique Games Conjecture (Khot ’02) : “A single concrete algorithm provides optimal guarantees among all efficient algorithms for a large class of computational problems.” (Barak and Steurer survey at ICM’14) Solving polynomial optimization (Lasserre ’01) Victor Magron Automated Precision Tuning using Semidefinite Programming 11 / 27

SDP for Polynomial Optimization Prove polynomial inequalities with SDP: p ( a , b ) : = a 2 − 2 ab + b 2 � 0 . � � � � � � z 1 z 2 a Find z s.t. p ( a , b ) = a b . z 2 z 3 b � �� � � 0 Find z s.t. a 2 − 2 ab + b 2 = z 1 a 2 + 2 z 2 ab + z 3 b 2 ( A z = d ) � z 1 � � 1 � � 0 � � 0 � � 0 � z 2 0 1 0 0 = z 1 + z 2 + z 3 � z 2 z 3 0 0 1 0 0 1 0 0 � �� � � �� � � �� � � �� � F 1 F 2 F 3 F 0 Victor Magron Automated Precision Tuning using Semidefinite Programming 12 / 27

SDP for Polynomial Optimization Choose a cost c e.g. ( 1, 0, 1 ) and solve: ⊤ z min c z ∑ s.t. F i z i � F 0 , A z = d . i � 1 � z 1 � � z 2 − 1 Solution = � 0 (eigenvalues 0 and 1) − 1 z 2 z 3 1 � � 1 � � a � − 1 a 2 − 2 ab + b 2 = � = ( a − b ) 2 . a b − 1 1 b � �� � � 0 Solving SDP = ⇒ Finding S UMS OF S QUARES certificates Victor Magron Automated Precision Tuning using Semidefinite Programming 13 / 27

SDP for Polynomial Optimization General case : Semialgebraic set S : = { x ∈ R n : g 1 ( x ) � 0, . . . , g m ( x ) � 0 } p ∗ : = min x ∈ S p ( x ) : NP hard Sums of squares (SOS) Σ [ x ] (e.g. ( x 1 − x 2 ) 2 ) � � σ 0 ( x ) + ∑ m Q ( S ) : = j = 1 σ j ( x ) g j ( x ) , with σ j ∈ Σ [ x ] Fix the degree 2 k of sums of squares Q k ( S ) : = Q ( S ) ∩ R 2 k [ x ] Victor Magron Automated Precision Tuning using Semidefinite Programming 14 / 27

SDP for Polynomial Optimization Hierarchy of SDP relaxations : � � λ k : = sup λ : p − λ ∈ Q k ( S ) λ Convergence guarantees λ k ↑ p ∗ [Lasserre 01] Can be computed with SDP solvers ( CSDP , SDPA ) “No Free Lunch” Rule : ( n + 2 k n ) SDP variables � Extension to semialgebraic functions r ( x ) = p ( x ) / q ( x ) [Lasserre-Putinar 10] Victor Magron Automated Precision Tuning using Semidefinite Programming 15 / 27

Sparse SDP Optimization [Waki, Lasserre 06] Correlative sparsity pattern (csp) of variables x 2 x 5 + x 3 x 6 − x 2 x 3 − x 5 x 6 + x 1 ( − x 1 + x 2 + x 3 − x 4 + x 5 + x 6 ) 6 5 4 1 2 3 C 1 : = { 1, 4 } 1 Maximal cliques C 1 , . . . , C l C 2 : = { 1, 2, 3, 5 } 2 Average size κ ❀ ( κ + 2 k C 3 : = { 1, 3, 5, 6 } κ ) Dense SDP: 210 variables variables Sparse SDP: 115 variables Victor Magron Automated Precision Tuning using Semidefinite Programming 16 / 27

Introduction Semidefinite Programming for Polynomial Optimization Rounding Error Bounds with Sparse SDP Conclusion