Projection and presolve in MOSEK: exponential and power cones ISMP - PowerPoint PPT Presentation

Projection and presolve in MOSEK: exponential and power cones ISMP 2018 Henrik A. Friberg www.mosek.com

❼ ❼ ❼ New cones in MOSEK Friberg, Henrik A. (2017). Power cones in second-order cone form and dual recovery . SIAM Conference on Optimization. www.mosek.com/resources/presentations. For rational numbers α 1 , . . . , α k ≥ 0: ≥ � z � e T α 2 · · · x α k K pow( α ) = { x α 1 1 x α 2 2 , x 1 , . . . x k ≥ 0 } k = P ( L ∩ Q 1 × Q 2 × · · · ) .

❼ ❼ ❼ New cones in MOSEK Friberg, Henrik A. (2017). Power cones in second-order cone form and dual recovery . SIAM Conference on Optimization. www.mosek.com/resources/presentations. For rational numbers α 1 , . . . , α k ≥ 0: ≥ � z � e T α 2 · · · x α k K pow( α ) = { x α 1 1 x α 2 2 , x 1 , . . . x k ≥ 0 } k = P ( L ∩ Q 1 × Q 2 × · · · ) . �

New cones in MOSEK Friberg, Henrik A. (2017). Power cones in second-order cone form and dual recovery . SIAM Conference on Optimization. www.mosek.com/resources/presentations. For rational numbers α 1 , . . . , α k ≥ 0: ≥ � z � e T α 2 · · · x α k K pow( α ) = { x α 1 1 x α 2 2 , x 1 , . . . x k ≥ 0 } k = P ( L ∩ Q 1 × Q 2 × · · · ) . � In MOSEK 9: 1 x 1 − α ❼ K pow( α, 1 − α ) = { x α ≥ � z � 2 , x 1 , x 2 ≥ 0 } , parametrized by 2 a real number 0 < α < 1. ❼ K exp = cl { t ≥ s exp( r / s ) , s > 0 } . ❼ The corresponding dual cones K ∗ pow( α, 1 − α ) and K ∗ exp .

Exponential cone examples 1 Exponential. t ≥ exp( x ) ⇐ ⇒ ( t , 1 , x ) ∈ K exp .

Exponential cone examples 1 Exponential. t ≥ a x ⇐ ⇒ ( t , 1 , x log( a )) ∈ K exp .

Exponential cone examples 1 Exponential.   n t ≥ a x 1 1 a x 2 2 · · · a x n �  ∈ K exp . n ⇐ ⇒  t , 1 , x j log( a j ) j =1

Exponential cone examples 2 { t ≤ log( x ) } = { ( x , 1 , t ) ∈ K exp } . 3 { t ≥ x log( x / y ) } = { ( y , x , − t ) ∈ K exp } . � (1 � t ≥ (log x ) 2 , 0 < x ≤ 1 2 , t , u ) ∈ Q 3 � � r , ( x , 1 , u ) ∈ K exp , x ≤ 1 = . 4 5 { t ≤ log log x , x > 1 } = { ( u , 1 , t ) ∈ K exp , ( x , 1 , u ) ∈ K exp } . √ � � t ≥ (log x ) − 1 , x > 1 2) ∈ Q 3 � � = ( u , t , r , ( x , 1 , u ) ∈ K exp . 6 � (1 � � � 2 , u , t ) ∈ Q 3 � t ≤ r , ( x , 1 , u ) ∈ K exp log x , x > 1 = . 7 √ � � � � � 2 t ) ∈ Q 3 t ≤ r , ( x , 1 , u ) ∈ K exp x log x , x > 1 = ( x , u , . 8 � (1 � 2 , u , x ) ∈ Q 3 9 { t ≥ x exp( x ) , x ≥ 0 } = r , ( t , x , u ) ∈ K exp .

Exponential cone examples 10 Log-sum-exponential. log(e x 1 + . . . + e x n ) ≥ t

Exponential cone examples 10 Log-sum-exponential. log(e x 1 + . . . + e x n ) ≥ t e x 1 + . . . + e x n e t ≥ �

Exponential cone examples 10 Log-sum-exponential. log(e x 1 + . . . + e x n ) ≥ t e x 1 + . . . + e x n e t ≥ � e x 1 − t + . . . + e x n − t ≥ 1 �

Exponential cone examples 10 Log-sum-exponential. log(e x 1 + . . . + e x n ) ≥ t e x 1 + . . . + e x n e t ≥ � e x 1 − t + . . . + e x n − t ≥ 1 � Geometric programming in conic form: inf t x + y 2 z inf log(e u + e 2 v + w ) ≤ t , 0 . 1 √ x + 2 y − 1 ≤ 1 , s.t. ↔ s.t. log(e 0 . 5 u +log(0 . 1) + e − v +log(2) ) ≤ 0 , z − 1 + yx − 2 ≤ 1 , log(e − w + e v − 2 u ) ≤ 0 , where ( x , y , z ) = (e u , e v , e w ).

More information Usage ❼ MOSEK Modeling Cookbook. ❼ Fri, 15:15. Micha� l Adamaszek: Exponential cone in MOSEK: overview and applications . Implementation details ❼ Wed, 8:30. Joachim Dahl: Extending MOSEK with exponential cones . Details for all of MOSEK 9 ❼ Wed, 15:15. Erling Andersen: MOSEK version 9 .

The curious case of error measuring ✞ ☎ Interior-point solution summary Problem status : PRIMAL_AND_DUAL_FEASIBLE Solution status : OPTIMAL Primal. obj: 7.4390660847e-02 nrm: 1e+00 Viol. con: 6e-09 var: 0e+00 cones: 4e-09 Dual. obj: 7.4390675795e-02 nrm: 3e-01 Viol. con: 1e-19 var: 8e-09 cones: 0e+00 ✝ ✆

❼ ❼ ❼ ❼ The curious case of error measuring Error of x = (0 , 10 8 , 1) in constraint 2 x 1 x 2 ≥ | x 3 | ?

❼ ❼ ❼ The curious case of error measuring Error of x = (0 , 10 8 , 1) in constraint 2 x 1 x 2 ≥ | x 3 | ? ❼ f ( x ) = | x 3 | − 2 x 1 x 2 ≤ 0. Error [ f ( x )] + = 1.

❼ The curious case of error measuring Error of x = (0 , 10 8 , 1) in constraint 2 x 1 x 2 ≥ | x 3 | ? ❼ f ( x ) = | x 3 | − 2 x 1 x 2 ≤ 0. Error [ f ( x )] + = 1. ❼ f ( x ) = | x 3 | / x 1 − 2 x 2 ≤ 0. Error [ f ( x )] + = Inf . ❼ f ( x ) = | x 3 | / x 2 − 2 x 1 ≤ 0. Error [ f ( x )] + = 1 e − 8.

The curious case of error measuring Error of x = (0 , 10 8 , 1) in constraint 2 x 1 x 2 ≥ | x 3 | ? ❼ f ( x ) = | x 3 | − 2 x 1 x 2 ≤ 0. Error [ f ( x )] + = 1. ❼ f ( x ) = | x 3 | / x 1 − 2 x 2 ≤ 0. Error [ f ( x )] + = Inf . ❼ f ( x ) = | x 3 | / x 2 − 2 x 1 ≤ 0. Error [ f ( x )] + = 1 e − 8. ❼ dist ( x , Q 3 r ) = 5 e − 9.

The curious case of error measuring Error of x = (0 , 10 8 , 1) in constraint 2 x 1 x 2 ≥ | x 3 | ? ❼ f ( x ) = | x 3 | − 2 x 1 x 2 ≤ 0. Error [ f ( x )] + = 1. ❼ f ( x ) = | x 3 | / x 1 − 2 x 2 ≤ 0. Error [ f ( x )] + = Inf . ❼ f ( x ) = | x 3 | / x 2 − 2 x 1 ≤ 0. Error [ f ( x )] + = 1 e − 8. ❼ dist ( x , Q 3 r ) = 5 e − 9. The power and exponential cones are also representation sensitive: x 0 . 3333 x 0 . 6666 x 1 1 x 2 2 ≥ || z || 3 ≥ || z || 2 ⇐ ⇒ 1 2 2 y ≥ exp ( x ) ⇐ ⇒ x ≤ log ( y ) This sensitivity is a well-known caveat of forward error. Projection is an example of backwards error.

The curious case of error measuring ✞ ☎ Interior-point solution summary Problem status : PRIMAL_AND_DUAL_FEASIBLE Solution status : OPTIMAL Primal. obj: 7.4390660847e-02 nrm: 1e+00 Viol. con: 6e-09 var: 0e+00 cones: 4e-09 Dual. obj: 7.4390675795e-02 nrm: 3e-01 Viol. con: 1e-19 var: 8e-09 cones: 0e+00 ✝ ✆ Variable domains are measured with backwards error: � dist ( x 1 , K 1 ) , dist ( x 2 , K 2 ) , . . . � ∞ .

In need of projections! dist (˜ x , K ) = min � x − ˜ x � x ∈K � � ˜ = arg min � x − ˜ x � x K x ∈K What is the hype about? ❼ Set membership conditions ( x ∈ K ). ❼ Representation-free error measures. ❼ Maximal separating hyperplanes. ❼ First-order methods for feasible point searches (e.g., looking for specific properties). ...basically a useful low cost operation (time+memory).

Projection theory Moreau’s decomposition theorem All matrices/vectors are uniquely decomposable as v 0 = [ v 0 ] K + [ v 0 ] K ◦ , for all nonempty, closed, convex cones K (and in any norm). Trivial example: All scalars are uniquely decomposable as v 0 = [ v 0 ] + + [ v 0 ] − , where [ • ] + = [ • ] R + = max(0 , • ), and [ • ] − = [ • ] R − = min(0 , • ).

Projection theory Moreau’s decomposition theorem All matrices/vectors are uniquely decomposable as v 0 = [ v 0 ] K + [ v 0 ] K ◦ , for all nonempty, closed, convex cones K (and in any norm). Dual cone projection: � � � � [ v 0 ] K ∗ = − − v 0 −K ∗ = − − v 0 K ◦ .

Projection theory Moreau’s decomposition theorem All matrices/vectors are uniquely decomposable as v 0 = [ v 0 ] K + [ v 0 ] K ◦ , for all nonempty, closed, convex cones K (and in any norm). Reflection (intrepid projection for obtuse cones): Ref K ( v 0 ) = [ v 0 ] K − [ v 0 ] K ◦ .

Separation For nonempty closed convex cones, K = { x | a T x ≤ 0 , ∀ a ∈ K ◦ } . x �∈ K are points of { a ∈ K ◦ | a T ˜ ❼ Separators of ˆ x > 0 } .

Separation For nonempty closed convex cones, K = { x | a T x ≤ 0 , ∀ a ∈ K ◦ } . x �∈ K are points of { a ∈ K ◦ | a T ˜ ❼ Separators of ˆ x > 0 } . Gradient separator For positively homogeneous convex functions, the cone K = { x | f ( x ) ≤ 0 } , has separator a = ∇ f (ˆ x ) for ˆ x �∈ K . Lubin, Miles (2017). “Mixed-integer convex optimization: outer approximation algorithms and modeling power”. PhD thesis. Massachusetts Institute of Technology.

Separation For nonempty closed convex cones, K = { x | a T x ≤ 0 , ∀ a ∈ K ◦ } . x �∈ K are points of { a ∈ K ◦ | a T ˜ ❼ Separators of ˆ x > 0 } . a ∈K ◦ , � a � 2 ≤ 1 a T ˆ ❼ The maximal separator solves max x .

Separation For nonempty closed convex cones, K = { x | a T x ≤ 0 , ∀ a ∈ K ◦ } . x �∈ K are points of { a ∈ K ◦ | a T ˜ ❼ Separators of ˆ x > 0 } . a ∈K ◦ , � a � 2 ≤ 1 a T ˆ ❼ The maximal separator solves max x . ❼ Its dual problem is min x ∈K � x − ˆ x � 2 . ❼ Maximal separator is dual solution to projection problem.

Projection theory Moreau’s decomposition theorem All matrices/vectors are uniquely decomposable as v 0 = [ v 0 ] K + [ v 0 ] K ◦ , for all nonempty, closed, convex cones K (and in any norm). Maximal separation: [ v 0 ] K ◦ maxsep K ( v 0 ) = � [ v 0 ] K ◦ � 2