NLCertify : A Tool for Formal Nonlinear Optimization Victor Magron , - - PowerPoint PPT Presentation

NLCertify: A Tool for Formal Nonlinear Optimization

Victor Magron, Postdoc LAAS-CNRS

18 September 2014

Aric Seminar Lyon

b y b → sin( √ b) par−

b1

par−

b2

par−

b3

par+

b1

par+

b2

par+

b3

1 b1 b2 b3 = 500

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 1 / 44

Errors and Proofs

Mathematicians want to eliminate all the uncertainties on their results. Why?

• M. Lecat, Erreurs des Mathématiciens des origines à

nos jours, 1935. 130 pages of errors! (Euler, Fermat, Sylvester, . . . )

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 2 / 44

Errors and Proofs

Possible workaround: proof assistants COQ (Coquand, Huet 1984) HOL-LIGHT (Harrison, Gordon 1980) Built in top of OCAML

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 2 / 44

Computer Science and Mathematics

Tool: Formal Bounds for Global Optimization Collaboration with: Benjamin Werner (LIX Polytechnique) Stéphane Gaubert (Maxplus Team CMAP/INRIA Polytechnique) Xavier Allamigeon (Maxplus Team)

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 3 / 44

Complex Proofs

Complex mathematical proofs / mandatory computation

• K. Appel and W. Haken , Every Planar Map is

Four-Colorable, 1989.

• T. Hales, A Proof of the Kepler Conjecture, 1994.
• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 4 / 44

From Oranges Stack...

Kepler Conjecture (1611): The maximal density of sphere packings in 3D-space is

π √ 18

Face-centered cubic Packing Hexagonal Compact Packing

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 5 / 44

...to Flyspeck Nonlinear Inequalities

The proof of T. Hales (1998) contains mathematical and computational parts Computation: check thousands of nonlinear inequalities Robert MacPherson, editor of The Annals of Mathematics: “[...] the mathematical community will have to get used to this state of affairs.” Flyspeck [Hales 06]: Formal Proof of Kepler Conjecture

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 6 / 44

...to Flyspeck Nonlinear Inequalities

The proof of T. Hales (1998) contains mathematical and computational parts Computation: check thousands of nonlinear inequalities Robert MacPherson, editor of The Annals of Mathematics: “[...] the mathematical community will have to get used to this state of affairs.” Flyspeck [Hales 06]: Formal Proof of Kepler Conjecture Project Completion on 10 August by the Flyspeck team!!

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 6 / 44

...to Flyspeck Nonlinear Inequalities

Nonlinear inequalities: quantified reasoning with “∀” ∀x ∈ K, f(x) 0 NP-hard optimization problem

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 7 / 44

A “Simple” Example

In the computational part: Multivariate Polynomials:

∆x := x1x4(−x1 + x2 + x3 − x4 + x5 + x6) + x2x5(x1 − x2 + x3 + x4 − x5 + x6) + x3x6(x1 + x2 − x3 + x4 + x5 − x6) − x2(x3x4 + x1x6) − x5(x1x3 + x4x6)

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 8 / 44

A “Simple” Example

In the computational part: Semialgebraic functions: composition of polynomials with | · |, √, +, −, ×, /, sup, inf, . . . p(x) := ∂4∆x q(x) := 4x1∆x r(x) := p(x)/

• q(x)

l(x) := −π 2 + 1.6294 − 0.2213 (√x2 + √x3 + √x5 + √x6 − 8.0) + 0.913 (√x4 − 2.52) + 0.728 (√x1 − 2.0)

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 8 / 44

A “Simple” Example

In the computational part: Transcendental functions T : composition of semialgebraic functions with arctan, exp, sin, +, −, ×, . . .

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 8 / 44

A “Simple” Example

In the computational part: Feasible set K := [4, 6.3504]3 × [6.3504, 8] × [4, 6.3504]2 Lemma9922699028 from Flyspeck: ∀x ∈ K, arctan p(x)

• q(x)
• + l(x) 0
• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 8 / 44

Existing Formal Frameworks

Formal proofs for Global Optimization: Bernstein polynomial methods [Zumkeller’s PhD 08] SMT methods [Gao et al. 12] Interval analysis and Sums of squares

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 9 / 44

Existing Formal Frameworks

Interval analysis Certified interval arithmetic in COQ [Melquiond 12] Taylor methods in HOL Light [Solovyev thesis 13]

Formal verification of floating-point operations

robust but subject to the Curse of Dimensionality

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 9 / 44

Existing Formal Frameworks

Lemma9922699028 from Flyspeck: ∀x ∈ K, arctan

• ∂4∆x

√4x1∆x

• + l(x) 0

Dependency issue using Interval Calculus:

One can bound ∂4∆x/√4x1∆x and l(x) separately Too coarse lower bound: −0.87 Subdivide K to prove the inequality

K = ⇒ K0 K1 K2 K3 K4

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 9 / 44

Existing Formal Frameworks

Sums of squares techniques Formalized in HOL-LIGHT [Harrison 07] COQ [Besson 07] Precise methods but scalibility and robustness issues (numerical) powerful: global optimality certificates without branching but not so robust: handles moderate size problems Restricted to polynomials

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 9 / 44

Existing Formal Frameworks

Approximation theory: Chebyshev/Taylor models mandatory for non-polynomial problems hard to combine with SOS techniques (degree of approximation)

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 9 / 44

Existing Formal Frameworks

Can we develop a new approach with both keeping the respective strength of interval and precision of SOS? Proving Flyspeck Inequalities is challenging: medium-size and tight

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 9 / 44

New Framework (in my PhD thesis)

Certificates for lower bounds of Nonlinear optimization using:

Moment-SOS hierarchies Maxplus approximation (Optimal Control)

Verification of these certificates inside COQ

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 10 / 44

New Framework (in my PhD thesis)

Software Implementation NLCertify: https://forge.ocamlcore.org/projects/nl-certify/ 15 000 lines of OCAML code 4000 lines of COQ code

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 10 / 44

Introduction Moment-SOS relaxations Semialgebraic Maxplus Optimization Formal Nonlinear Optimization

Polynomial Optimization Problems

Semialgebraic set K := {x ∈ Rn : g1(x) 0, . . . , gm(x) 0} p∗ := min

x∈K p(x): NP hard

Sums of squares Σ[x] e.g. x2

1 − 2x1x2 + x2 2 = (x1 − x2)2

Q(K) :=

• σ0(x) + ∑m

j=1 σj(x)gj(x), with σj ∈ Σ[x]

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 11 / 44

Polynomial Optimization Problems

Archimedean module The set K is compact and the polynomial N − x2

2 belongs to

Q(K) for some N > 0. Assume that K is a box: product of closed intervals Normalize the feasibility set to get K′ := [−1, 1]n K′ := {x ∈ Rn : g1 := 1 − x2

1 0, · · · , gn := 1 − x2 n 0}

n − x2

2 belongs to Q(K′)

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 12 / 44

Convexification and the K Moment Problem

Borel σ-algebra B (generated by the open sets of Rn) M+(K): set of probability measures supported on K. If µ ∈ M+(K) then

1 µ : B → [0, 1], µ(∅) = 0, µ(Rn) < ∞ 2 µ( i Bi) = ∑i µ(Bi), for any countable (Bi) ⊂ B 3 K µ(dx) = 1

supp(µ) is the smallest set K such that µ(Rn\K) = 0

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 13 / 44

Convexification and the K Moment Problem

p∗ = inf

x∈K p(x) =

inf

µ∈M+(K)

• K p dµ
• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 13 / 44

Convexification and the K Moment Problem

Let (xα)α∈Nn be the monomial basis Definition A sequence y has a representing measure on K if there exists a finite measure µ supported on K such that yα =

• K xαµ(dx) ,

∀ α ∈ Nn .

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 13 / 44

Convexification and the K Moment Problem

Ly(q) : q ∈ R[x] → ∑α qαyα Theorem [Putinar 93] Let K be compact and Q(K) be Archimedean. Then y has a representing measure on K iff Ly(σ) 0 , Ly(gj σ) 0 , ∀σ ∈ Σ[x] .

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 13 / 44

Lasserre’s Hierarchy of SDP relaxations

Moment matrix M(y)u,v := Ly(u · v), u, v monomials Localizing matrix M(gj y) associated with gj M(gj y)u,v := Ly(u · v · gj), u, v monomials

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 14 / 44

Lasserre’s Hierarchy of SDP relaxations

Mk(y) contains (n+2k

n ) variables, has size (n+k n )

Truncated matrix of order k = 2 with variables x1, x2: M2(y) =               1 | x1 x2 | x2

1

x1x2 x2

2

1 1 | y1,0 y0,1 | y2,0 y1,1 y0,2 − − − − − − − − x1 y1,0 | y2,0 y1,1 | y3,0 y2,1 y1,2 x2 y0,1 | y1,1 y0,2 | y2,1 y1,2 y0,3 − − − − − − − − − x2

1

y2,0 | y3,0 y2,1 | y4,0 y3,1 y2,2 x1x2 y1,1 | y2,1 y1,2 | y3,1 y2,2 y1,3 x2

2

y0,2 | y1,2 y0,3 | y2,2 y1,3 y0,4              

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 14 / 44

Lasserre’s Hierarchy of SDP relaxations

Consider g1(x) := 2 − x2

1 − x2

• 2. Then v1 = ⌈deg g1/2⌉ = 1.

M1(g1 y) =   1 x1 x2 1 2 − y2,0 − y0,2 2y1,0 − y3,0 − y1,2 2y0,1 − y2,1 − y0,3 x1 2y1,0 − y3,0 − y1,2 2y2,0 − y4,0 − y2,2 2y1,1 − y3,1 − y1,3 x2 2y0,1 − y2,1 − y0,3 2y1,1 − y3,1 − y1,3 2y0,2 − y2,2 − y0,4  

M1(g1 y)(3, 3) = L(g1(x) · x2 · x2) = L(2x2

2 − x2 1x2 2 − x4 2)

= 2y0,2 − y2,2 − y0,4

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 14 / 44

Lasserre’s Hierarchy of SDP relaxations

Truncation with moments of order at most 2k vj := ⌈deg gj/2⌉ Hierarchy of semidefinite relaxations:          infy Ly(p) = ∑α

• K pα xα µ(dx) = ∑α pα yα

Mk(y)

• 0 ,

Mk−vj(gj y)

• 0 ,

1 j m, y1 = 1 .

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 14 / 44

Semidefinite Optimization

F0, Fα symmetric real matrices, cost vector c Primal-dual pair of semidefinite programs: (SDP)              P : infy ∑α cαyα s.t. ∑α Fα yα − F0 0 D : supY Trace (F0 Y) s.t. Trace (Fα Y) = cα , Y 0 . Freely available SDP solvers (CSDP, SDPA, SEDUMI)

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 15 / 44

Primal-dual Moment-SOS

M+(K): space of probability measures supported on K Polynomial Optimization Problems (POP) (Primal) (Dual) inf

• K p dµ

= sup λ s.t. µ ∈ M+(K) s.t. λ ∈ R , p − λ ∈ Q(K)

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 16 / 44

Primal-dual Moment-SOS

Truncated quadratic module Qk(K) := Q(K) ∩ R2k[x] For large enough k, zero duality gap [Lasserre 01]: Polynomial Optimization Problems (POP) (Moment) (SOS) inf

∑

α

pα yα = sup λ s.t. Mk−vj(gj y) 0 , 0 j m, s.t. λ ∈ R , y1 = 1 p − λ ∈ Qk(K)

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 16 / 44

Practical Computation

Hierarchy of SOS relaxations: λk := sup

λ

• λ : p − λ ∈ Qk(K)
• Convergence guarantees λk ↑ p∗ [Lasserre 01]

If p − p∗ ∈ Qk(K) for some k then: y∗ := (1, x∗

1, x∗ 2, (x∗ 1)2, x∗ 1x∗ 2, . . . , (x∗ 1)2k, . . . , (x∗ n)2k)

is a global minimizer of the primal SDP [Lasserre 01].

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 17 / 44

Practical Computation

Caprasse Problem ∀x ∈ [−0.5, 0.5]4, −x1x3

3 + 4x2x2 3x4 + 4x1x3x2 4 + 2x2x3 4 +

4x1x3 + 4x2

3 − 10x2x4 − 10x2 4 + 5.1801 0.

scale_pol = true: scaled on [0, 1]4 relax_order = 2: SOS of degree at most 4 bound_squares_variables = true: redundant constraints x2

1 1, . . . , x2 4 1

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 17 / 44

The “No Free Lunch” Rule

Exponential dependency in

1 Relaxation order k (SOS degree) 2 number of variables n

Computing λk involves (n+2k

n ) variables

At fixed k, O(n2k) variables

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 18 / 44

Introduction Moment-SOS relaxations Semialgebraic Maxplus Optimization Formal Nonlinear Optimization

The General “Informal Framework”

Given K a compact set and f a transcendental function, bound f ∗ = inf

x∈K f(x) and prove f ∗ 0

f is underestimated by a semialgebraic function fsa Reduce the problem f ∗

sa := infx∈K fsa(x) to a polynomial

• ptimization problem (POP)
• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 19 / 44

Maxplus Approximation

Initially introduced to solve Optimal Control Problems [Fleming-McEneaney 00] Curse of dimensionality reduction [McEaneney Kluberg, Gaubert-McEneaney-Qu 11, Qu 13]. Allowed to solve instances of dim up to 15 (inaccessible by grid methods) In our context: approximate transcendental functions

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 20 / 44

Maxplus Approximation

Definition Let γ 0. A function φ : Rn → R is said to be γ-semiconvex if the function x → φ(x) + γ

2 x2 2 is convex.

a y par+

a1

par+

a2

par−

a2

par−

a1

a2 a1 arctan m M

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 20 / 44

Nonlinear Function Representation

Exact parsimonious maxplus representations

a y

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 21 / 44

Nonlinear Function Representation

Exact parsimonious maxplus representations

a y

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 21 / 44

Nonlinear Function Representation

Abstract syntax tree representations of multivariate transcendental functions: leaves are semialgebraic functions of A nodes are univariate functions of D or binary operations

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 21 / 44

Nonlinear Function Representation

For the “Simple” Example from Flyspeck:

+ l(x) arctan r(x)

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 21 / 44

Maxplus Optimization Algorithm

First iteration:

+ l(x) arctan r(x) a y par−

a1

arctan m M a1 1 control point {a1}: m1 = −4.7 × 10−3 < 0

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 22 / 44

Maxplus Optimization Algorithm

Second iteration:

+ l(x) arctan r(x) a y par−

a1

par−

a2

arctan m M a1 a2 2 control points {a1, a2}: m2 = −6.1 × 10−5 < 0

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 22 / 44

Maxplus Optimization Algorithm

Third iteration:

+ l(x) arctan r(x) a y par−

a1

par−

a2

par−

a3

arctan m M a1 a2 a3 3 control points {a1, a2, a3}: m3 = 4.1 × 10−6 > 0

OK!

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 22 / 44

Maxplus Optimization Algorithm

Input: tree t, box K, SOS relaxation order k, precision p Output: bounds m and M, approximations t−

2 and t+ 2

1: if t ∈ A then t− := t, t+ := t 2: else if u := root(t) ∈ D with child c then 3:

mc, Mc, c−, c+ := samp_approx(c, K, k, p)

4:

I := [mc, Mc]

5:

u−, u+ := unary_approx(u, I, c, p)

6:

t−, t+ := compose_approx(u, u−, u+, I, c−, c+)

7: else if bop := root(t) with children c1 and c2 then 8:

mi, Mi, c−

i , c+ i := samp_approx(ci, K, k, p) for i ∈ {1, 2}

9:

t−, t+ := compose_bop(c−

1 , c+ 1 , c− 2 , c+ 2 , bop, [m2, M2])

10: end 11: return min_sa(t−, K, k), max_sa(t+, K, k), t−, t+

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 23 / 44

Minimax Approximation / For Comparison

The precision is an integer d The best-uniform degree-d polynomial approximation of u: min

h∈Rd[x] u − h∞ = min h∈Rd[x](sup x∈I

|u(x) − h(x)|) Implementation in Sollya [Chevillard-Joldes-Lauter 10] Interface of NLCertify with Sollya

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 24 / 44

High-degree Polynomial Approximation + SOS

SWF: minx∈[1,500]n f(x) = − ∑n

i=1 xi sin(√xi)

replace sin(√·) by a degree-d Chebyshev polynomial Hard to combine with SOS

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 25 / 44

High-degree Polynomial Approximation + SOS

Indeed: Small d: lack of accuracy = ⇒ expensive Branch and Bound Large d: “No free lunch” rule with (n+d

n ) SDP variables

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 25 / 44

High-degree Polynomial Approximation + SOS

SWF with n = 10, d = 4: 38 min to compute a lower bound of −430n

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 25 / 44

Comparison on Global Optimization Problems

min

x∈[1,500]n f(x) = − ∑n i=1 xi sin(√xi)

f ∗ −418.9n Interval Arithmetic for sin + SOS n lower bound nlifting #boxes time 10 −430n 3830 129 s 10 −430n 2n 16 40 s

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 26 / 44

Comparison on Global Optimization Problems

min

x∈[1,500]n f(x) = − n

∑

i=1

xi sin(√xi) f ∗ −418.9n

a y par−

a1

par−

a2

par−

a3

par+

a1

par+

a2

par+

a3

sin 1 a1 a2 a3 = √ 500

n lower bound nlifting #boxes time 10 −430n 3830 129 s 10 −430n 2n 16 40 s

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 27 / 44

Comparison on Global Optimization Problems

min

x∈[1,500]n f(x) = − ∑n i=1 xi sin(√xi)

f ∗ −418.9n Interval Arithmetic for sin + SOS n lower bound nlifting #boxes time 100 −440n > 10000 > 10 h 100 −440n 2n 274 1.9 h

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 28 / 44

Comparison on Global Optimization Problems

min

x∈[1,500]n f(x) = − n

∑

i=1

xi sin(√xi) f ∗ −418.9n

a y par−

a1

par−

a2

par−

a3

par+

a1

par+

a2

par+

a3

sin 1 a1 a2 a3 = √ 500

n lower bound nlifting #boxes time 100 −440n > 10000 > 10 h 100 −440n 2n 274 1.9 h

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 29 / 44

Comparison on Global Optimization Problems

min

x∈[1,500]n f(x) = − n−1

∑

i=1

(xi + xi+1) sin(√xi)

a y par−

a1

par−

a2

par−

a3

par+

a1

par+

a2

par+

a3

sin 1 a1 a2 a3 = √ 500

n lower bound nlifting #boxes time 1000 −967n 2n 1 543 s 1000 −968n n 1 272 s

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 30 / 44

Comparison on Global Optimization Problems

min

x∈[1,500]n f(x) = − n−1

∑

i=1

(xi + xi+1) sin(√xi)

b y b → sin( √ b) par−

b1

par−

b2

par−

b3

par+

b1

par+

b2

par+

b3

1 b1 b2 b3 = 500

n lower bound nlifting #boxes time 1000 −967n 2n 1 543 s 1000 −968n n 1 272 s

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 31 / 44

Convergence of the Optimization Algorithm

Let f be a multivariate transcendental function Let t−

p be the underestimator of f, obtained at precision p

Let xp

• pt be a minimizer of t−

p over K

Theorem [X. Allamigeon S. Gaubert VM B. Werner 13] Every accumulation point of the sequence (xp

• pt) is a global min-

imizer of f on K. Ingredients of the proof: Convergence of Lasserre SOS hierarchy Uniform approximation schemes (Maxplus/Minimax)

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 32 / 44

Polynomial Approximations for Semialgebraic Functions

Inspired from [Lasserre - Thanh 13] Let fsa ∈ A defined on a box K ⊂ Rn Let µn be the standard Lebesgue measure on Rn Best polynomial underestimator h ∈ Rd[x] of fsa for the L1 norm: (Psa)    min

h∈Rd[x]

• K(fsa − h)dµn

s.t. fsa − h 0 on K . Lemma Problem (Psa) has a degree-d polynomial minimizer hd.

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 33 / 44

Polynomial Approximations for Semialgebraic Functions

b.s.a.l. ˆ K := {(x, z) ∈ Rn+p : g1(x, z) 0, . . . , gm(x, z) 0} The quadratic module M( ˆ K) is Archimedean The optimal solution hd of (Psa) is a maximizer of: (Pd)    max

h∈Rd[x]

• [0,1]n h dµn

s.t. (zp − h) ∈ M( ˆ K) .

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 34 / 44

Polynomial Approximations for Semialgebraic Functions

Let md be the optimal value of Problem (Psa) Let hdk be a maximizer of the SOS relaxation of (Pd) Convergence of the SOS Hierarchy The sequence (fsa − hdk1)kk0 is non-increasing and converges to md. Each accumulation point of the sequence (hdk)kk0 is an

• ptimal solution of Problem (Psa).
• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 35 / 44

Polynomial Approximations for Semialgebraic Functions

fsa(x) := ∂4∆x √4x1∆x d k Upper bound of fsa − hdk1 Bound 2 2 0.8024

• 1.171

3 0.3709

• 0.4479

4 2 1.617

• 1.056

3 0.1766

• 0.4493
• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 35 / 44

Polynomial Approximations for Semialgebraic Functions

rad2 : (x1, x2) → −64x2

1+128x1x2+1024x1−64x2 2+1024x2−4096

−8x2

1+8x1x2+128x1−8x2 2+128x2−512

Linear and quadratic underestimators for rad2 (k = 3):

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 36 / 44

Polynomial Approximations for Semialgebraic Functions

rad2 : (x1, x2) → −64x2

1+128x1x2+1024x1−64x2 2+1024x2−4096

−8x2

1+8x1x2+128x1−8x2 2+128x2−512

Linear and quadratic underestimators for rad2 (k = 3):

0.2 0.4 0.6 0.8 1 0 0.5 1 0.11 0.11 0.12 0.12 d = 1 d = 2

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 36 / 44

Contributions

Published:

• X. Allamigeon, S. Gaubert, V. Magron, and B. Werner.

Certification of inequalities involving transcendental functions: combining sdp and max-plus approximation, ECC Conference 2013.

• X. Allamigeon, S. Gaubert, V. Magron, and B. Werner.

Certification of bounds of non-linear functions: the templates method, CICM Conference, 2013.

In revision:

• X. Allamigeon, S. Gaubert, V. Magron, and B. Werner.

Certification of Real Inequalities – Templates and Sums of Squares, arxiv:1403.5899, 2014.

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 37 / 44

Introduction Moment-SOS relaxations Semialgebraic Maxplus Optimization Formal Nonlinear Optimization

The General “Formal Framework”

We check the correctness of SOS certificates for POP We build certificates to prove interval bounds for semialgebraic functions We bound formally transcendental functions with semialgebraic approximations

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 38 / 44

Formal SOS bounds

When q ∈ Q(K), σ0, . . . , σm is a positivity certificate for q Check symbolic polynomial equalities q = q′ in COQ Existing tactic ring [Grégoire-Mahboubi 05] Polynomials coefficients: arbitrary-size rationals bigQ [Grégoire-Théry 06] Much simpler to verify certificates using sceptical approach Extends also to semialgebraic functions

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 39 / 44

Bounding the Polynomial Remainder

Normalized POP (x ∈ [0, 1]n) ǫpop(x) := p(x) − λk − ∑m

j=0 σj(x)gj(x)

∀x ∈ [0, 1]n, ǫpop(x) ǫ∗

pop := ∑ ǫα0

ǫα

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 40 / 44

Formal SOS Results

POP1: ∀x ∈ K, ∂4∆x −41. POP2: ∀x ∈ K, ∆x 0. Problem n NLCertify micromega [Besson 07] POP1 6 0.08 s 9.00 s POP2 2 0.09 s 0.36 s 3 0.39 s − 6 13.2 s − Sparse SOS relaxations = ⇒ Speedup

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 41 / 44

Benchmarks for Flyspeck Inequalities

Inequality #boxes Time Time 9922699028 39 190 s 2218 s 3318775219 338 1560 s 19136 s Comparable with Taylor interval methods in HOL-LIGHT [Hales-Solovyev 13] No free lunch: SDP informal bottleneck 22 times slower than SDP: q = q′ formal bottleneck

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 42 / 44

Contribution

For more details on the formal side:

• X. Allamigeon, S. Gaubert, V. Magron and B. Werner. Formal

Proofs for Nonlinear Optimization. Submitted for publication, arxiv:1404.7282

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 43 / 44

Formal Nonlinear Optimization

Formal nonlinear optimization: NLCertify Safe solutions for challenging problems, e.g. Flyspeck

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 44 / 44

Formal Nonlinear Optimization

Further research: OCAML API Alternative polynomial bounds using geometric programming [De Wolff and Iliman] Mixed LP/SOS certificates (trade-off CPU/precision) COQ tactic Improve formal polynomial checker

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 44 / 44

Formal Nonlinear Optimization

Further research: Generalized problem of moments (Moment) (SOS) inf

• K p0 dµ
• sup

λ0 + ∑

i

λi bi s.t.

• K pi dµ bi

s.t. λ0, λi 0 , µ ∈ M+(K) p0 − λ0 − ∑

i

λi pi ∈ Qk(K) Formal bounds using SDP and ring

• V. Magron

NLCertify: A Tool for Formal Nonlinear Optimization 44 / 44

End

Thank you for your attention! http://homepages.laas.fr/vmagron/