Lower bounds certification for multivariate real functions using SDP Joint Work with B. Werner, S. Gaubert and X. Allamigeon Victor MAGRON LIX/INRIA, ´ Ecole Polytechnique LIX PhD Seminar Friday 18 t❤ January Victor MAGRON Lower bounds certification
Two Problems ❑ ✚ ❘ ♥ : a compact set ❢ ✿ ❑ ✦ ❘ : a real multivariate function Two challenging problems: ① ✷ ❑ ❢ ✭ ① ✮ when ❢ is a multivariate polynomial of degree ❞ ✐♥❢ 1 Number of variables ♥ is large, no sparsity ❂ ✮ very hard to solve using Interval Arithmetic Example: ❑ ✿❂ ❬✵ ❀ ✶❪ ♥ , random numbers ✭ r ✐ ✮ ✶ ✔ ✐ ✔ ♥ : ♥ ❢ ❞ ✿❂ ✭ ✶ ✹ ❳ ① ✐ ✭ r ✐ � ① ✐ ✮✮ ❞ ❞❂ ✷ ❡ , the range of ❢ ❞ is ❬✵ ❀ ✶❪ r ✷ ♥ ✐ ✐ ❂✶ ① ✷ ❑ ❢ ✭ ① ✮ when ❢ is a multivariate real function involving ✐♥❢ 2 transcendental univariate functions Victor MAGRON Lower bounds certification
Contents Solving Polynomial Problems using Sum of Squares (SOS) 0 and Semidefinite Programming (SDP) Lower bounds of multivariate polynomial with large number of 1 variables Lower bounds of transcendental multivariate functions 2 Victor MAGRON Lower bounds certification
SOS and SDP Relaxations Polynomial Optimization Problem (POP): Let ❢❀ ❣ ✶ ❀ ✁ ✁ ✁ ❀ ❣ ♠ ✷ ❘ ❬ ❳ ✶ ❀ ✁ ✁ ✁ ❀ ❳ ♥ ❪ ❑ ♣♦♣ ✿❂ ❢ ① ✷ ❘ ♥ ✿ ❣ ✶ ✭ ① ✮ ✕ ✵ ❀ ✁ ✁ ✁ ❀ ❣ ♠ ✭ ① ✮ ✕ ✵ ❣ is the feasible set General POP: compute ❢ ✄ ♣♦♣ ❂ ① ✷ ❑ ♣♦♣ ❢ ✭ ① ✮ ✐♥❢ Example: ❢ ✿❂ ✶✵ � ① ✷ ✶ � ① ✷ ✷ ❀ ❣ ✶ ✿❂ ✶ � ① ✷ ✶ � ① ✷ ✷ ❑ ♣♦♣ ✿❂ ❢ ① ✷ ❘ ✷ ✿ ❣ ✶ ✭ ① ✮ ✕ ✵ ❣ is the feasible set Victor MAGRON Lower bounds certification
SOS and SDP Relaxations Convexify the problem: ❩ ❢ ✄ ♣♦♣ ❂ ① ✷ ❑ ♣♦♣ ❢ ♣♦♣ ✭ ① ✮ ❂ ✐♥❢ ✐♥❢ ❢ ♣♦♣ ❞✖ , where P ✭ ❑ ♣♦♣ ✮ is the ✖ ✷P ✭ ❑ ♣♦♣ ✮ set of all probability measures ✖ supported on the set ❑ ♣♦♣ . Equivalent formulation: ❢ ✄ ♣♦♣ ❂ ♠✐♥ ❢ ▲ ✭ ❢ ✮ ✿ ▲ ✿ ❘ ❬ ❳ ❪ ✦ ❘ linear, ▲ ✭✶✮ ❂ ✶ and each ▲ ❣ ❥ is SDP ❣ , with ❣ ✵ ❂ ✶ , ▲ ❣ ✵ ❀ ✁ ✁ ✁ ❀ ▲ ❣ ♠ defined by: ▲ ❣ ❥ ✿ ❘ ❬ ❳ ❪ ✂ ❘ ❬ ❳ ❪ ✦ ❘ ✭ ♣❀ q ✮ ✼✦ ▲ ✭ ♣ ✁ q ✁ ❣ ❥ ✮ Victor MAGRON Lower bounds certification
SOS and SDP Relaxations: Lasserre Hierarchy ❇ ✿❂ ✭ ❳ ☛ ✮ ☛ ✷ ◆ ♥ : the monomial basis and ② ☛ ❂ ▲ ✭ ❳ ☛ ✮ , this identifies ▲ with the infinite series ② ❂ ✭ ② ☛ ✮ ☛ ✷ ◆ ♥ Infinite moment matrix ▼ : ▼ ✭ ② ✮ ✉❀✈ ✿❂ ▲ ✭ ✉ ✁ ✈ ✮ ❀ ✉❀ ✈ ✷ ❇ Localizing matrix ▼ ✭ ❣ ❥ ② ✮ : ▼ ✭ ❣ ❥ ② ✮ ✉❀✈ ✿❂ ▲ ✭ ✉ ✁ ✈ ✁ ❣ ❥ ✮ ❀ ✉❀ ✈ ✷ ❇ ❦ ✕ ❦ ✵ ✿❂ ♠❛① ❢❞ ❞❡❣ ❢ ♣♦♣ ❡ ❂ ✷ ❀ ❞ ❞❡❣ ❣ ✵ ❂ ✷ ❡ ❀ ✁ ✁ ✁ ❀ ❞ ❞❡❣ ❣ ♠ ❂ ✷ ❡❣ Index ▼ ✭ ② ✮ and ▼ ✭ ❣ ❥ ② ✮ with elements in ❇ of degree at most ❦ , it gives the semidefinite relaxations hierarchy: ❩ ✽ ❳ ❢ ☛ ① ☛ ❞✖ ✭ ① ✮ ❂ ✐♥❢ ② ▲ ✭ ❢ ✮ ❂ ❢ ☛ ② ☛ ❃ ❃ ❃ ❃ ❁ ☛ ◗ ❦ ✿ ▼ ❦ �❞ ❞❡❣ ❣ ❥ ❂ ✷ ❡ ✭ ❣ ❥ ② ✮ ✵ ❀ ✵ ✔ ❥ ✔ ♠❀ ❁ ❃ ❃ ❃ ❃ ✿ ② ✶ ❂ ✶ Victor MAGRON Lower bounds certification
SOS and SDP Relaxations Convergence Theorem [Lasserre]: The sequence ✐♥❢✭ ◗ ❦ ✮ ❦ ✕ ❦ ✵ is non-decreasing and under the SOS assumption converges to ❢ ✄ ♣♦♣ . SDP relaxations: Many solvers (e.g. Sedumi, SDPA) solve the pair of (standard form) semidefinite programs: ✽ ❳ ❃ P ✿ ♠✐♥ ❝ ☛ ② ☛ ❃ ❃ ② ❃ ❃ ☛ ❃ ❃ ❳ ❃ ❃ subject to ❋ ☛ ② ☛ � ❋ ✵ ❁ ✵ ❁ ✭ ❙❉P ✮ ☛ ❃ ❃ ❃ ❉ ✿ ♠❛① Trace ✭ ❋ ✵ ❨ ✮ ❃ ❃ ❨ ❃ ❃ ❃ ❃ Trace ✭ ❋ ☛ ❨ ✮ ❂ ❝ ☛ ✿ subject to Victor MAGRON Lower bounds certification
Large-scale POP Complexity issues SDP relaxation ◗ ❦ at order ❦ ✕ ♠❛① ❥ ❢❞ ❞❡❣ ❢ ♣♦♣ ❂ ✷ ❡ ❀ ❞ ❞❡❣ ❣ ❥ ❂ ✷ ❡❣ : ❖ ✭ ♥ ✷ ❦ ✮ moment variables linear matrix inequalities (LMIs) of size ❖ ✭ ♥ ❦ ✮ polynomial in ♥ , exponential in ❦ On our example: ❑ ✿❂ ❬✵ ❀ ✶❪ ♥ , random numbers ✭ r ✐ ✮ ✶ ✔ ✐ ✔ ♥ : ♥ ❢ ❞ ✿❂ ✭ ✶ ✹ ❳ ① ✐ ✭ r ✐ � ① ✐ ✮✮ ❞ ❞❂ ✷ ❡ r ✷ ♥ ✐ ✐ ❂✶ ✮ at least ❖ ✭ ♥ ✷ ❞ ✮ moment variables with LMIs ❞❡❣ ❣ ❥ ❂ ✶ , ❦ ✕ ❞ ❂ of size ❖ ✭ ♥ ❞ ✮ !! Victor MAGRON Lower bounds certification
Large-scale POP Multivariate Taylor-Models Underestimators: ❢ ✿ ❑ ✦ ❘ is a multivariate polynomial Consider a minimizer guess ① ❝ obtained by heuristics Let q ① ❝ be the quadratic form defined by: q ① ❝ ✿ ❑ � ✦ ❘ ① ✼� ✦ ❢ ✭ ① ❝ ✮ ✰ ❉ ❢ ✭ ① ❝ ✮ ✭ ① � ① ❝ ✮ ✰✶ ✷✭ ① � ① ❝ ✮ ❚ ❉ ✷ ❢ ✭ ① ❝ ✮ ✭ ① � ① ❝ ✮ ✰ ✕ ✭ ① � ① ❝ ✮ ✷ ① ✷ ❑ ❢ ✕ ♠✐♥ ✭ ❉ ✷ ❢ ✭ ① ✮ � ❉ ✷ with ✕ ✿❂ ♠✐♥ ❢ ✭ ① ❝ ✮✮ ❣ Theorem: ✽ ① ✷ ❑❀ ❢ ✭ ① ✮ ✕ q ① ❝ , that is q ① ❝ understimates ❢ on ❑ . q ① ❝ is called a quadratic cut. How to compute ✕ ? How to compute a lower bound of ❢ ? Victor MAGRON Lower bounds certification
Large-scale POP Computation of ✕ by Robust SDP ① ✷ ❑ ❢ ✕ ♠✐♥ ✭ ❉ ✷ ❢ ✭ ① ✮ � ❉ ✷ ✕ ✿❂ ♠✐♥ ❢ ✭ ① ❝ ✮✮ ❣ Bound the Hessian difference on ❑ by POP (using SDP relaxations) to get ✖ ❉ ✷ ❢ : Define the symmetric matrix ❇ containing the bounds on the entries of ✖ ❉ ✷ ❢ . Let ❙ ♥ be the set of diagonal matrices of sign. ❙ ♥ ✿❂ ❢ diag ✭ s ✶ ❀ ✁ ✁ ✁ ❀ s ♥ ✮ ❀ s ✶ ❂ ✝ ✶ ❀ ✁ ✁ ✁ s ♥ ❂ ✝ ✶ ❣ ✕ ✿❂ ✕ ♠✐♥ ✭ ✖ ❉ ✷ ❢ � ❉ ✷ ❢ ✭ ① ❝ ✮✖ ■ ✮ : minimal eigenvalue of an interval matrix Robut Optimization with Reduced Vertex Set [Calafiore, Dabbene] The robust interval SDP problem ✕ ♠✐♥ ✭ ✖ ❉ ✷ ❢ � ❉ ✷ ❢ ✭ ① ❝ ✮✖ ■ ✮ is equivalent to the following SDP in the single variable t ✷ ❘ : ✽ ♠✐♥ � t ❁ ❢ ✭ ① ❝ ✮ � ❙ ❇ ❙ ✗ ✵ ❀ ❙ ❂ diag ✭✶ ❀ ⑦ ❙ ✮ ❀ ✽ ⑦ � t ■ � ❉ ✷ ❙ ✷ ❙ ♥ � ✶ ✿ s.t. Victor MAGRON Lower bounds certification
Large-scale POP Computation of ✕ by approximation and simpler SDP Solving the previous SDP is expensive because the dimension of ❙ ♥ grows exponentially. Instead, we can underestimate ✕ : Write ✖ ❢ � ❉ ✷ ❢ ✭ ① ❝ ✮✖ ■ ✿❂ ✖ ❳ ✰ ✖ ❉ ✷ ❨ with ❳ ✐❥ ✿❂ ❬ ❛ ✐❥ ✰ ❜ ✐❥ ❀ ❛ ✐❥ ✰ ❜ ✐❥ ❨ ✐❥ ✿❂ ❬ � ❜ ✐❥ � ❛ ✐❥ ❀ ❜ ✐❥ � ❛ ✐❥ ✖ ❪ and ✖ ❪ ✷ ✷ ✷ ✷ ✕ ♠✐♥ ✭ ✖ ❳ ✰ ✖ ❨ ✮ ✕ ✕ ♠✐♥ ✭ ✖ ❳ ✮ ✰ ✕ ♠✐♥ ✭ ✖ ❨ ✮ ❂ ✕ ♠✐♥ ✭ ✖ ❳ ✮ � ✕ ♠❛① ✭ � ✖ ❨ ✮ ❜ ✐❥ � ❛ ✐❥ ❳ ✕ ♠❛① ✭ ✖ ❨ ✮ ✔ ♠❛① ✷ ✐ ❥ Computing a lower bound of ✕ ♠✐♥ ✭ ✖ ❳ ✮ is easier because ✖ ❳ is a real ✽ ♠✐♥ � t ❁ matrix. We can do it again by SDP: � t ■ � ✖ ✿ s.t. ❳ ✗ ✵ ... and how to compute a lower bound of the polynomial ❢ ? Victor MAGRON Lower bounds certification
Large-scale POP Computing lower bounds Input: ❢ , box ❑ , SDP relaxation order ❦ , control points sequence s ❂ ✭ ① ✶ ✮ ✷ ❑ , ♥ ❝✉ts (final number of quadratic cuts) Output: lower bound ♠ of ❢ 1: ❝✉ts ✿❂ ✶ 2: while ❝✉ts ✔ ♥ ❝✉ts do For ❝ ✷ ❢ ✶ ❀ ✿ ✿ ✿ ❀ ★ s ❣ : compute ✕ using robust SDP or ✕ ♠✐♥ 3: approximation and compute q ① ❝ ❢ ♣ ✿❂ ♠❛① ✶ ✔ ❝ ✔ ♣ q ① ❝ , ❑ ♣♦♣ ✿❂ ❢ ① ✷ ❑ ✿ ③ ✕ q ① ✶ ✭ ① ✮ ❀ ✁ ✁ ✁ ❀ ③ ✕ q ① ♣ ✭ ① ✮ ❣ 4: 5: Compute a lower bound ♠ of ❢ ♣ by POP at the SDP relaxation order ❦ : ♠ ✔ ✐♥❢ ① ✷ ❑ ♣♦♣ ③ ① ♦♣t ✿❂ ❣✉❡ss ❛r❣♠✐♥ ✭ ❢ ♣ ✮ : a minimizer candidate for ❢ ♣ 6: s ✿❂ s ❬ ❢ ① ♦♣t ❣ 7: ❝✉ts ✿❂ ❝✉ts ✰ ✶ 8: 9: done Victor MAGRON Lower bounds certification
Recommend
More recommend
