Lower bounds certification for multivariate real functions using SDP - - PowerPoint PPT Presentation

Lower bounds certification for multivariate real functions using SDP

Joint Work with B. Werner, S. Gaubert and X. Allamigeon

Victor MAGRON

LIX/INRIA, ´ Ecole Polytechnique

LIX PhD Seminar Friday 18 t❤ January

Victor MAGRON Lower bounds certification

Two Problems

❑ ✚ ❘♥: a compact set ❢ ✿ ❑ ✦ ❘: a real multivariate function

Two challenging problems:

1

✐♥❢

①✷❑ ❢✭①✮ when ❢ is a multivariate polynomial of degree ❞

Number of variables ♥ is large, no sparsity ❂

✮ very hard to

solve using Interval Arithmetic Example:

❑ ✿❂ ❬✵❀ ✶❪♥, random numbers ✭r✐✮✶✔✐✔♥: ❢❞ ✿❂ ✭ ✶ ♥

♥

❳

✐❂✶

✹ r✷

✐

①✐✭r✐ ①✐✮✮❞❞❂✷❡, the range of ❢❞ is ❬✵❀ ✶❪

2

✐♥❢

①✷❑ ❢✭①✮ when ❢ is a multivariate real function involving

transcendental univariate functions

Victor MAGRON Lower bounds certification

Contents

Solving Polynomial Problems using Sum of Squares (SOS) and Semidefinite Programming (SDP)

1

Lower bounds of multivariate polynomial with large number of variables

2

Lower bounds of transcendental multivariate functions

Victor MAGRON Lower bounds certification

SOS and SDP Relaxations

Polynomial Optimization Problem (POP): Let ❢❀ ❣✶❀ ✁ ✁ ✁ ❀ ❣♠ ✷ ❘❬❳✶❀ ✁ ✁ ✁ ❀ ❳♥❪

❑♣♦♣ ✿❂ ❢① ✷ ❘♥ ✿ ❣✶✭①✮ ✕ ✵❀ ✁ ✁ ✁ ❀ ❣♠✭①✮ ✕ ✵❣ is the feasible set

General POP: compute ❢✄

♣♦♣ ❂

✐♥❢

①✷❑♣♦♣ ❢✭①✮

Example:

❢ ✿❂ ✶✵ ①✷

✶ ①✷ ✷❀ ❣✶ ✿❂ ✶ ①✷ ✶ ①✷ ✷

❑♣♦♣ ✿❂ ❢① ✷ ❘✷ ✿ ❣✶✭①✮ ✕ ✵❣ is the feasible set

Victor MAGRON Lower bounds certification

SOS and SDP Relaxations

Convexify the problem:

❢✄

♣♦♣ ❂

✐♥❢

①✷❑♣♦♣ ❢♣♦♣✭①✮ ❂

✐♥❢

✖✷P✭❑♣♦♣✮

❩ ❢♣♦♣ ❞✖, where P✭❑♣♦♣✮ is the

set of all probability measures ✖ supported on the set ❑♣♦♣. Equivalent formulation:

❢✄

♣♦♣ ❂ ♠✐♥ ❢▲✭❢✮

✿ ▲ ✿ ❘❬❳❪ ✦ ❘ linear, ▲✭✶✮ ❂ ✶ and

each ▲❣❥ is SDP ❣, with ❣✵

❂ ✶, ▲❣✵❀ ✁ ✁ ✁ ❀ ▲❣♠ defined by: ▲❣❥ ✿ ❘❬❳❪ ✂ ❘❬❳❪ ✦ ❘ ✭♣❀ q✮ ✼✦ ▲✭♣ ✁ q ✁ ❣❥✮

Victor MAGRON Lower bounds certification

SOS and SDP Relaxations: Lasserre Hierarchy

❇ ✿❂ ✭❳☛✮☛✷◆♥: the monomial basis and ②☛ ❂ ▲✭❳☛✮, this

identifies ▲ with the infinite series ② ❂ ✭②☛✮☛✷◆♥ Infinite moment matrix ▼:

▼✭②✮✉❀✈ ✿❂ ▲✭✉ ✁ ✈✮❀ ✉❀ ✈ ✷ ❇

Localizing matrix ▼✭❣❥②✮:

▼✭❣❥②✮✉❀✈ ✿❂ ▲✭✉ ✁ ✈ ✁ ❣❥✮❀ ✉❀ ✈ ✷ ❇ ❦ ✕ ❦✵ ✿❂ ♠❛①❢❞❞❡❣ ❢♣♦♣❡❂✷❀ ❞❞❡❣ ❣✵❂✷❡❀ ✁ ✁ ✁ ❀ ❞❞❡❣ ❣♠❂✷❡❣

Index ▼✭②✮ and ▼✭❣❥②✮ with elements in ❇ of degree at most

❦, it gives the semidefinite relaxations hierarchy: ◗❦ ✿ ✽ ❃ ❃ ❃ ❃ ❁ ❃ ❃ ❃ ❃ ✿ ✐♥❢

② ▲✭❢✮

❂ ❩ ❢☛ ①☛ ❞✖✭①✮ ❂ ❳

☛

❢☛ ②☛ ▼❦❞❞❡❣ ❣❥❂✷❡✭❣❥②✮ ❁ ✵❀ ✵ ✔ ❥ ✔ ♠❀ ②✶ ❂ ✶

Victor MAGRON Lower bounds certification

SOS and SDP Relaxations

Convergence Theorem [Lasserre]: The sequence ✐♥❢✭◗❦✮❦✕❦✵ is non-decreasing and under the SOS assumption converges to ❢✄

♣♦♣.

SDP relaxations: Many solvers (e.g. Sedumi, SDPA) solve the pair of (standard form) semidefinite programs:

✭❙❉P✮ ✽ ❃ ❃ ❃ ❃ ❃ ❃ ❃ ❃ ❃ ❁ ❃ ❃ ❃ ❃ ❃ ❃ ❃ ❃ ❃ ✿ P ✿ ♠✐♥

②

❳

☛

❝☛②☛

subject to

❳

☛

❋☛ ②☛ ❋✵ ❁ ✵ ❉ ✿ ♠❛①

❨

Trace ✭❋✵ ❨ ✮ subject to Trace ✭❋☛ ❨ ✮ ❂ ❝☛

Victor MAGRON Lower bounds certification

Large-scale POP

Complexity issues

SDP relaxation ◗❦ at order ❦ ✕ ♠❛①

❥ ❢❞❞❡❣ ❢♣♦♣❂✷❡❀ ❞❞❡❣ ❣❥❂✷❡❣:

❖✭♥✷❦✮ moment variables

linear matrix inequalities (LMIs) of size ❖✭♥❦✮ polynomial in ♥, exponential in ❦ On our example:

❑ ✿❂ ❬✵❀ ✶❪♥, random numbers ✭r✐✮✶✔✐✔♥: ❢❞ ✿❂ ✭ ✶ ♥

♥

❳

✐❂✶

✹ r✷

✐

①✐✭r✐ ①✐✮✮❞❞❂✷❡ ❞❡❣ ❣❥ ❂ ✶, ❦ ✕ ❞ ❂ ✮ at least ❖✭♥✷❞✮ moment variables with LMIs

• f size ❖✭♥❞✮!!

Victor MAGRON Lower bounds certification

Large-scale POP

Multivariate Taylor-Models Underestimators:

❢ ✿ ❑ ✦ ❘ is a multivariate polynomial

Consider a minimizer guess ①❝ obtained by heuristics Let q①❝ be the quadratic form defined by:

q①❝ ✿ ❑

• ✦

❘ ① ✼ ✦ ❢✭①❝✮ ✰ ❉❢✭①❝✮ ✭① ①❝✮ ✰✶ ✷✭① ①❝✮❚ ❉✷

❢✭①❝✮ ✭① ①❝✮ ✰ ✕✭① ①❝✮✷

with ✕ ✿❂ ♠✐♥

①✷❑❢✕♠✐♥✭❉✷ ❢✭①✮ ❉✷ ❢✭①❝✮✮❣

Theorem:

✽① ✷ ❑❀ ❢✭①✮ ✕ q①❝, that is q①❝ understimates ❢ on ❑. q①❝ is called a quadratic cut.

How to compute ✕? How to compute a lower bound of ❢?

Victor MAGRON Lower bounds certification

Large-scale POP

Computation of ✕ by Robust SDP

✕ ✿❂ ♠✐♥

①✷❑❢✕♠✐♥✭❉✷ ❢✭①✮ ❉✷ ❢✭①❝✮✮❣

Bound the Hessian difference on ❑ by POP (using SDP relaxations) to get ✖

❉✷

❢:

Define the symmetric matrix ❇ containing the bounds on the entries of ✖

❉✷

❢.

Let ❙♥ be the set of diagonal matrices of sign.

❙♥ ✿❂ ❢diag ✭s✶❀ ✁ ✁ ✁ ❀ s♥✮❀ s✶ ❂ ✝✶❀ ✁ ✁ ✁ s♥ ❂ ✝✶❣ ✕ ✿❂ ✕♠✐♥✭ ✖ ❉✷

❢ ❉✷ ❢✭①❝✮✖

■✮: minimal eigenvalue of an interval

matrix Robut Optimization with Reduced Vertex Set [Calafiore, Dabbene] The robust interval SDP problem ✕♠✐♥✭ ✖

❉✷

❢ ❉✷ ❢✭①❝✮✖

■✮ is equivalent

to the following SDP in the single variable t ✷ ❘:

✽ ❁ ✿ ♠✐♥ t

s.t.

t ■ ❉✷

❢✭①❝✮ ❙ ❇ ❙ ✗ ✵❀ ❙ ❂ diag ✭✶❀ ⑦

❙✮❀ ✽ ⑦ ❙ ✷ ❙♥✶

Victor MAGRON Lower bounds certification

Large-scale POP

Computation of ✕ by approximation and simpler SDP

Solving the previous SDP is expensive because the dimension of

❙♥ grows exponentially. Instead, we can underestimate ✕:

Write ✖

❉✷

❢ ❉✷ ❢✭①❝✮✖

■ ✿❂ ✖ ❳ ✰ ✖ ❨ with ✖ ❳✐❥ ✿❂ ❬❛✐❥ ✰ ❜✐❥ ✷ ❀ ❛✐❥ ✰ ❜✐❥ ✷ ❪ and ✖ ❨✐❥ ✿❂ ❬❜✐❥ ❛✐❥ ✷ ❀ ❜✐❥ ❛✐❥ ✷ ❪ ✕♠✐♥✭ ✖ ❳ ✰ ✖ ❨ ✮ ✕ ✕♠✐♥✭ ✖ ❳✮ ✰ ✕♠✐♥✭ ✖ ❨ ✮ ❂ ✕♠✐♥✭ ✖ ❳✮ ✕♠❛①✭ ✖ ❨ ✮ ✕♠❛①✭ ✖ ❨ ✮ ✔ ♠❛①

✐

❳

❥

❜✐❥ ❛✐❥ ✷

Computing a lower bound of ✕♠✐♥✭ ✖

❳✮ is easier because ✖ ❳ is a real

• matrix. We can do it again by SDP:

✽ ❁ ✿ ♠✐♥ t

s.t.

t ■ ✖ ❳ ✗ ✵

... and how to compute a lower bound of the polynomial ❢?

Victor MAGRON Lower bounds certification

Large-scale POP

Computing lower bounds

Input: ❢, box ❑, SDP relaxation order ❦, control points sequence s ❂ ✭①✶✮ ✷ ❑, ♥❝✉ts (final number of quadratic cuts) Output: lower bound ♠ of ❢

1: ❝✉ts ✿❂ ✶ 2: while ❝✉ts ✔ ♥❝✉ts do 3:

For ❝ ✷ ❢✶❀ ✿ ✿ ✿ ❀ ★s❣: compute ✕ using robust SDP or ✕♠✐♥ approximation and compute q①❝

4:

❢♣ ✿❂ ♠❛①

✶✔❝✔♣ q①❝, ❑♣♦♣ ✿❂ ❢① ✷ ❑ ✿ ③ ✕ q①✶✭①✮❀ ✁ ✁ ✁ ❀ ③ ✕ q①♣✭①✮❣

5:

Compute a lower bound ♠ of ❢♣ by POP at the SDP relaxation

• rder ❦: ♠ ✔

✐♥❢

①✷❑♣♦♣ ③

6:

①♦♣t ✿❂ ❣✉❡ss ❛r❣♠✐♥ ✭❢♣✮: a minimizer candidate for ❢♣

7:

s ✿❂ s ❬ ❢①♦♣t❣

8:

❝✉ts ✿❂ ❝✉ts ✰ ✶

9: done

Victor MAGRON Lower bounds certification

Large-scale POP

Comparisons w.r.t the ✕ computation

❑ ✿❂ ❬✵❀ ✶❪♥ ❢✻ ✿❂ ✭ ✶ ♥

♥

❳

✐❂✶

✹ r✷

✐

①✐✭r✐ ①✐✮✮✸

We compare the quality

• f the successive lower

bounds (previous algorithm) with different

✕ underestimators ✕robust ✕ ✕approx ❂ ✮

Better quadratic approximations when using the Robust SDP approach ♥ ❂ ✷ ♥ ❂ ✸ ♥ ❂ ✹ ♥ ❂ ✺ ♥ ❂ ✻ ❘♦❜✉st ❙❉P ❆♣♣r♦① ✕♠✐♥ ✵ ✶✵ ✷✵ ✸✵ ✹✵ ✺✵ ✽ ✻ ✹ ✷ ◗✉❛❞r❛t✐❝ ❝✉ts ▲♦✇❡r ❜♦✉♥❞s ❡rr♦rs

Victor MAGRON Lower bounds certification

Large-scale POP

Scalability Issues

When ♥ is large, Robust SDP approach is too expensive. It becomes impossible to compute ✕ and the quadratic cuts q①❝.

♥ ❂ ✷ ♥ ❂ ✶✵ ♥ ❂ ✷✵ ✵ ✺ ✶✵ ✶✺ ✷✵ ✻ ✹ ✷ ✵ ◗✉❛❞r❛t✐❝ ❝✉ts ▲♦✇❡r ❜♦✉♥❞s ❡rr♦rs

(a) ❞ ❂ ✹

♥ ❂ ✷ ♥ ❂ ✾ ✵ ✺ ✶✵ ✶✺ ✷✵ ✶✷ ✶✵ ✽ ✻ ✹ ✷ ✵ ◗✉❛❞r❛t✐❝ ❝✉ts ▲♦✇❡r ❜♦✉♥❞s ❡rr♦rs

(b) ❞ ❂ ✻

Bottleneck: computation of the ♥✭♥ ✰ ✶✮ bounds of the Hessian entries ✖

❉✷

❢ ❉✷ ❢✭①❝✮✖

■ (multivariate polynomial of

degree ❞ ✷)

Victor MAGRON Lower bounds certification

Bounding multivariate transcendental functions

Now, consider a semialgebraic compact set ❑ ✚ ❘♥ and

❢ ✿ ❑ ✦ ❘ a multivariate transcendental function

We want to compute a precise lower bound of ❢. The previous approach only gives a hierarchy of coarse bounds Motivations? How to approach the univariate transcendental functions involved in ❢?

Victor MAGRON Lower bounds certification

Flyspeck-Like Problems

The Kepler Conjecture

Kepler Conjecture (1611): The maximal density of sphere packings in 3-space is ✙

✶✽

It corresponds to the way people would intuitively stack oranges, as a pyramid shape The proof of T. Hales (1998) consists of thousands of non-linear inequalities Many recent efforts have been done to give a formal proof of these inequalities: Flyspeck Project Motivation: get positivity certificates and check them with Proof assistants like COQ

Victor MAGRON Lower bounds certification

Flyspeck-Like Problems

Lemma Example

Inequalities issued from Flyspeck non-linear part involve:

1

Semi-Algebraic functions algebra ❆: composition of polynomials with ❥ ✁ ❥, ✭✁✮

✶ ♣ ✭♣ ✷ ◆✵✮, ✰❀ ❀ ✂❀ ❂❀ s✉♣❀ ✐♥❢ 2

Transcendental functions ❚ : composition of semi-algebraic functions with ❛r❝t❛♥, ❛r❝♦s, ❛r❝s✐♥, ❡①♣, ❧♦❣, ❥ ✁ ❥,

✭✁✮

✶ ♣ ✭♣ ✷ ◆✵✮, ✰❀ ❀ ✂❀ ❂❀ s✉♣❀ ✐♥❢

Lemma✾✾✷✷✻✾✾✵✷✽ from Flyspeck

❑ ✿❂ ❬✹❀ ✻✿✸✺✵✹❪✸ ✂ ❬✻✿✸✺✵✹❀ ✽❪ ✂ ❬✹❀ ✻✿✸✺✵✹❪✷ P❀ ◗ ✷ ❘❬❳❪ ✽① ✷ ❑❀ ✙ ✷ ✰ ❛r❝t❛♥ P✭①✮ ♣ ◗✭①✮ ✰ ✶✿✻✷✾✹ ✵✿✷✷✶✸ ✭♣①✷ ✰ ♣①✸ ✰ ♣①✺ ✰ ♣①✻ ✽✿✵✮ ✰ ✵✿✾✶✸ ✭♣①✹ ✷✿✺✷✮ ✰ ✵✿✼✷✽ ✭♣①✶ ✷✿✵✮ ✕ ✵✿

Tight inequality: global optimum ✬ ✶✿✼ ✂ ✶✵✹

Victor MAGRON Lower bounds certification

Bounding multivariate transcendental functions

General Framework

Given ❑ a compact set, and ❢ a transcendental function, bound from below ❢✄ ❂ ✐♥❢

①✷❑ ❢✭①✮ and prove ❢✄ ✕ ✵

1

❢ is approximated by a semi-algebraic function ❢s❛

2

Reduce the problem ✐♥❢

①✷❑ ❢s❛✭①✮ to a polynomial optimization

problem (POP) in a lifted space ❑♣♦♣

3

Solve classically the POP problem

✐♥❢

①✷❑♣♦♣ ❢♣♦♣✭①✮ using a

sparse variant hierarchy of SDP relaxations by Lasserre

❢✄ ✕ ❢✄

s❛ ✕ ❢✄ ♣♦♣ ✕ ✵

⑤④③⑥

If the relaxations are accurate enough

Victor MAGRON Lower bounds certification

Bounding multivariate transcendental functions

General Framework

The first step is to build the abstract syntax tree from an inequality, where leaves are semi-algebraic functions and nodes are univariate transcendental functions (arctan, exp, ...)

• r basic operations (✰, ✂, , ❂).

With ❧ ✿❂ ✙

✷ ✰ ✶✿✻✷✾✹ ✵✿✷✷✶✸ ✭♣①✷ ✰ ♣①✸ ✰ ♣①✺ ✰ ♣①✻ ✽✿✵✮ ✰ ✵✿✾✶✸ ✭♣①✹ ✷✿✺✷✮ ✰ ✵✿✼✷✽ ✭♣①✶ ✷✿✵✮, the tree of the

example is:

✰ ❧✭①✮ ❛r❝t❛♥ P✭①✮ ♣ ◗✭①✮

Victor MAGRON Lower bounds certification

Bounding multivariate transcendental functions

Transcendental Functions Approximations

Let t ✷ ❚ be a transcendental univariate elementary function such as ❛r❝t❛♥, ❡①♣, ..., defined on a real interval ■. Let

❞ ✷ ◆ given.

Minimax: Best uniform degree ❞ polynomial approximation ❫

t:

solution of ❥❥✎❥❥✶ ✿❂

♠✐♥

♣✷❘❞❬❳❪ ❥❥t ♣❥❥✶

Existence and uniqueness of ❫

t

Remez algorithm implementation in Sollya: computes ❫

t for

each univariate transcendental function involved in the Flyspeck inequalities with given ■ ✿❂ ❬❛❀ ❜❪ and ❞ ✷ ◆ Also computes a certified upper bound of ❥❥✎❥❥✶ related to the minimax polynomial

Victor MAGRON Lower bounds certification

Bounding multivariate transcendental functions

General Framework

✰ ❧✭①✮ ❛r❝t❛♥ P✭①✮ ♣ ◗✭①✮

Two kinds of semialgebraic leaves: multivariate functions:

P✭①✮ ♣ ◗✭①✮

: we can get the bounds by POP using lifting variables sum of univariate functions:

❧ ✿❂ ✙ ✷ ✰ ✶✿✻✷✾✹ ✵✿✷✷✶✸ ✭♣①✷ ✰ ♣①✸ ✰ ♣①✺ ✰ ♣①✻ ✽✿✵✮ ✰ ✵✿✾✶✸ ✭♣①✹ ✷✿✺✷✮ ✰ ✵✿✼✷✽ ✭♣①✶ ✷✿✵✮: we can approximate ♣✁ by a minimax polynomial with Sollya

Victor MAGRON Lower bounds certification

Bounding multivariate transcendental functions

General Framework

P✭①✮ ♣ ◗✭①✮

• n ❑ ✿❂ ❬✹❀ ✻✿✸✺✵✹❪✸ ✂ ❬✻✿✸✺✵✹❀ ✽❪ ✂ ❬✹❀ ✻✿✸✺✵✹❪✷?

Lifting procedure by POP:

1

Get bounds of P✭①✮ by POP

2

Get bounds of ◗✭①✮ by POP and

♣ ◗✭①✮ by interval arithmetic

3

Lifting variable representing

♣ ◗✭①✮: q ✷ ■q

4

Coarse bounds of

P✭①✮ ♣ ◗✭①✮

by interval arithmetic: interval ■③

5

Lifting variable representing

P✭①✮ ♣ ◗✭①✮

: ③ ✷ ■③

6

Lifting space:

❑pop ✿❂ ❢✭①❀ q❀ ③✮ ✷ ❑ ✂ ■q ✂ ■③ ✿ q✷ ❂ ◗✭①✮❀ ③q ❂ P✭①✮❣

7

Solving

✐♥❢

✭①❀q❀③✮✷❑pop

③ by POP gives a lower bound of P✭①✮ ♣ ◗✭①✮

Victor MAGRON Lower bounds certification

Bounding multivariate transcendental functions

Univariate approximations

8

We get an interval ■ enclosing

P✭①✮ ♣ ◗✭①✮

from POP .

9

Minimax polynomials for the univariate real functions of ❢:

t ❞

Upper bound of ❥❥✎❥❥✶

❛r❝t❛♥ on ■

5

✷✿✵✶ ✂ ✶✵✹ ♣ on ❬✹❀ ✻✿✸✺✵✹❪

4

✷✿✺✵ ✂ ✶✵✺ ♣ on ❬✻✿✸✺✵✹❀ ✽❪

2

✾✿✸✹ ✂ ✶✵✽

10 We obtain a minimax polynomial for ❛r❝t❛♥. With the minimax

polynomial for ♣: we can approach ❧ by ❫

❧.

Victor MAGRON Lower bounds certification

Bounding multivariate transcendental functions

Solving the inequality

Again by using POP: Lifting variable representing

♣ ◗✭①✮: q ✷ ■q

Lifting variable representing

P✭①✮ ♣ ◗✭①✮

: ③ ✷ ■③ Lifting space:

❑pop ✿❂ ❢✭①❀ q❀ ③✮ ✷ ❑ ✂ ■q ✂ ■③ ✿ q✷ ❂ ◗✭①✮❀ ③q ❂ P✭①✮❀ ❣

Solving

✐♥❢

✭①❀q❀③✮✷❑pop

❫ ❧✭①✮ ✰ ❫ ❛r❝t❛♥✭③✮ by POP gives a lower

bound of ❫

❢ ✿❂ ❫ ❧✭①✮ ✰ ❫ ❛r❝t❛♥✭③✮

Finally, Subtract the minimax errors ❥❥✎❥❥✶ to ❫

❢ gives a lower

bound of ❢

Victor MAGRON Lower bounds certification

End

Thanks for your attention! Questions?

Victor MAGRON Lower bounds certification