Conic Programming in GAMS Armin Pruessner, Michael Bussieck, Steven - - PowerPoint PPT Presentation

Conic Programming in GAMS

Armin Pruessner, Michael Bussieck, Steven Dirkse, Alex Meeraus

GAMS Development Corporation

INFORMS 2003, Atlanta October 19-22

2

Direction

• What this talk is about

– Overview: the class of conic programs – Conic constraints and examples using GAMS – Solvers in GAMS and numerical results – CONE World - a forum for conic programming

• What this talk is not about:

– Conic programming algorithms – Detailed applications

3

Overview

What are conic programs?

• Generalized linear programs with the addition
• f nonlinear convex cones
• Class includes, for example,
• Linear program (LP)
• (Convex) Quadratic program (QP)
• (Convex) Quadratically constrained QP (QCQP)
• Recently much activity in this area!

4

Areas and Applications

Conic programming used for:

• Engineering

– Truss topology design – FIR filter design

• Finance

– Portfolio optimization

• Statistics and Numerical Linear Algebra

– Robust linear programming – Norm minimization problems 7th DIMACS Implementation Challenge on SDP and Second Order Cone Programming (SOCP)

5

Modeling and Solving

Less General More General

SDP SOCP (Convex) QCQP (Convex) QP LP

Less Difficult More Difficult Model Generality Solution Difficulty

6

Cone Programs

• General form of conic program

where C is a second order cone (dim=k):

      ≤       =

− − k k k k k

x x x x C

2 ) 1 ( : 1 ) 1 ( : 1

:

[ ]

, , . . min

x x T

u l x C x b Ax t s x f ∈ ∈ ≤

7

Example (quadratic cone)

Quadractic cone C sometimes also called Lorentz cone (or ice cream cone)

Trivial Quadratic Cone:

2 2

y x z + ≥

8

Second Order Cone Programs

ℜ ∈ ℜ ∈ ℜ ∈ ℜ ∈ ℜ ∈ = + ≤ +

− × − i k i n k i n i n i T i i i T

b d C a f x N 1 i b x a d x C t s x f

i i

1 ) 1 (

, ,..., , . . min

Equivalent to conic program

• Linear constraints: cone dimension k=1
• Cone constraints:

change of variables

, (vector)

i T i i i

b x a z d x C y + = + =

9

Types of Cones

• Quadratic Cone
• Rotated Quadratic Cone

Sometimes preferable for modeling quadratic inequalities

∑ ≥

≠i j i i

x x

2

∑

≠ ≠ ≥ j k i k 2 k

x j x i 2x

,

10

Rotated Quadratic Cone

• Show equivalence to quadratic cone:

( )

      − ≥ + + + − ≥ ≥

∑

) ( 2

2 2 2 2 2

y x z y x y x xy z xy z xy

i i

Rotated quadratic cone Quadratic cone

11

General Transformation

• If where
• C r rotated quadratic cone
• C q quadratic cone

A =

                    − 1 1 2 1 2 1 2 1 2 1 L M O M M L L

q r

C Ax C x ∈ ⇔ ∈

12

Conic Constraints in GAMS

In the GAMS modeling language:

• Conic constraints denoted by =C=
• Conic programs result in “linear

programs” in GAMS

13

Quadratic Cone in GAMS

Set s /s1,s2,…,sn/; Set t(s) / s2,…,sn/; Variable x(s);

Conic “LP” formulation

x(‘s1’) =C= sum(t(s), x(s));

Equivalent NLP formulation

Note: Summation on right hand side

x(‘s1’) =G= sqrt[sum(t(s),sqr(x(s)))];

14

Rotated Quadratic Cone

Set s /s1,s2,s3,…,sn/; Set t(s) / s3,…,sn/; Variable x(s);

Conic “LP” formulation

x(‘s1’)+x(‘s2’) =C= sum(t(s), x(s));

Equivalent NLP formulation

Note: Summation on right hand side 2*x(‘s1’)*x(‘s2’)=G= sum[t(s),sqr(x(s))];

15

Example: Complex L1 Norm

Complex L1 Norm Minimization

• Minimize ||Ax-b||1 where A,x,b are complex

valued

• We can write this as

2

) Im( ) Re( ) Im( ) Re( ) Re( ) Im( ) Im( ) Re( min         −               − b b x x A A A A

16

Example (Continued)

Complex L1 Norm Minimization

cone) (quadratic ) ( ) ( ) ( ) Im( ) Re( ) Im( ) Re( ) Re( ) Im( ) Im( ) Re( s.t. ) ( min

2 2

i z i z i t b b x x A A A A (i) z (i) z i t

im re im re i

+ ≥         −               − =        

∑

17

GAMS Model (Quadratic Cone)

Objective..

• bj =E= sum(i, t(i));

reseq_re(i).. res_re(i) =E= sum(j, A_re(i,j)*x_re(j))

• sum(j,A_im(i,j)*x_im(j))
• b_re(i);

reseq_im(i).. res_im(i) =E= … coneeq(i).. t(i) =C= res_re(i) + res_im(i); Model conemodel /objective, reseq_re, reseq_im, coneeq/; Solve conemodel using lp minimizing obj;

18

Portfolio Optimization

∑ ∑ ∑

≥ ≥ = =

j j j i j j j j j j j j

r x p x x t s Dx x x min

min 2 2 ' , ' ' ,

, 1 . . α σ

Objective is to minimize variance (risk), subject to an expected return σj,j’ = covariance =1 / (numdays-1) xj = % of investment in stock j pj = price change (return) for stock j rmin = minimum expected return Dj,d = Deviation per day d of stock j wrt to mean return

19

Portfolio Optimization (Cont.)

Can rewrite:

2 2

Dx min α

By introducing intermediate variables p,q, and w:

2

2 1 ) ( ) , ( ) ( subject to 2 minimize

∑ ∑

≥ = =

d j

w(d) qr q j x d j D d w r α

20

GAMS Model (Rotated Cone)

Objective..

• bj =E= a*(2*r);

Budget.. sum(j, x(j)) =E= 1; Return.. Sum(j, p(j)*x(j)) =G= rmin; Wcone(days).. w(d) = sum(j, D(j,d)*x(j); cone_eq1.. q =E= 1; cone_eq2.. q + r =C= sum(d, w(d)); Model conemodel / all /; Solve conemodel using lp minimizing obj;

21

GAMS/MOSEK

Solving Conic Models in GAMS:

• Newest addition is MOSEK

– LP (simplex or interior point) – MIP (branch and bound) – Conic Programs (conic interior point): – Convex NLP

• Solver CONOPT also accepts conic constraints

22

Numerical Examples

DIMACS Challenge Models (SDP and SOCP)

• Chose subset of models from DIMACS (only

SOCP models)

• SOCP models:

– Sum of norms – Antenna array weight design – Scheduling problems MOSEK solves all SOCP models and is the most efficient

23

Numerical Examples

DIMACS Benchmarks by Hans Mittelmann

• Solvers: MOSEK 2.5.1 (ext MPS), LOQO

6.03, SDPT3 3.01, SeDuMi 1.05R4

• 18 SOCP problems

– In SeDuMi (MATLAB) format – MOSEK: extended QPS format (based on MPS)

24

DIMACS Results (H. Mittelmann)

5 9 32 32 75 75 6 13 13 22 22 68 68 1 2 5 10 10 7 28 28 107 107 445 445 Sched_50_50_orig Sched_100_50_orig Sched_100_100_orig Sched_200_100_orig 6 13 13 29 29 138 138 6 17 17 28 28 95 95 1 2 4 10 10 5 22 22 94 94 409 409 Sched_50_50_orig Sched_100_50_orig Sched_100_100_orig Sched_200_100_orig 3 18 18 780 780 6 29 29 504 504 2 9 355 355 15 15 221 221 MM MM Qssp30 Qssp60 Qssp180 4 9 459 459 4 14 14 232 232 2 9 182 182 11 11 151 151 MM MM Nql30 Nql60 Nql180 9 11 11 24 24 12 12 11 11 20 20 18 18 11 11 3 3 8 2 11 11 9 16 16 7 Nb Nb_L1 Nb_L2 Nb_L2_Bessel

SeDuMi SeDuMi SDPT3 SDPT3 MOS MOSEK LO LOQO QO Problem Problem

MM: memory problems

25

DIMACS Results (Cont.)

Use Performance Profiles (Dolan and Moré, 2002) to visualize results:

• Cumulative distribution function for a

performance metric

• Performance metric: ratio τ of current solver

time over best time of all solvers for “success”

• Intuitively: probability of success if given τ

times fastest time (τ=ratio)

26

Profiles (Data: H. Mittelmann)

Figure 1

27

• An online forum for

discussion and information on cone programming

• CONELib library of

models

– GAMS cone format – NLP formulation

• Conic programming

solvers

• Links and lists

CONE World

28

ConeLib – Conic Models in GAMS

29

ConeLib – Conic Models in GAMS

30

Cone Versus NLP Formulation

• Modeling conic constraints can be tedious
• NLP form of quadratic cone is more natural
• Rotated quadratic cone for QP is cumbersome

BUT

• Potentially significant computational advantages
• Compare cone vs. NLP formulation on CONELib

(currently 28 models)

• NLP formulation: substitute into conic constraint (converted

using CONVERT utility in GAMS)

• In practice “smarter” NLP formulations may exist depending
• n model

31

Cone Vs. NLP (Efficiency)

Figure 2

10 20 30 40 50 60 70 80 90 100 1 10 100 1000 Percent Of Models Solved Time Factor Performance Profile MOSEK NLP-1 NLP-2 NLP-3

32

Reproducibility of Results

• Reproducibility of results is key to validity in scientific

disciplines (but sometimes neglected)

– Open data source (models, solvers, solver options) – Should be easily reproducible

• As part of the GAMS World, we provide downloadable

scripts to reproduce results

• Currently only for NLP/Cone models (Global World)

– See www.gamsworld.org/global/reproduce

33

Reproducibility of Results

34

Reproducibility of Results

35

Conclusions

• Addition of conic programming capability within

GAMS

• MOSEK as state-of-the-art conic programming

solver

• Although modeling still cumbersome, significant

computational advantages using cones

• CONELib: growing collection of conic

programming models

• Presentation will be available under

http://www.gams.com/presentations

36

References

• Lobo, M.S., Vandenberghe, L., Boyd, S. and Lebret, H.

Applications of Second Order Cone Programming, Linear Algebra and its Applications, 284:193-228, November 1998.

• MOSEK Optimization Tools Help Desk, Version 2, online

at www.mosek.com/documentation, 2003.

• H.D. Mittelmann, An independent benchmarking of SDP

and SOCP solvers, Math Program., Series B, 2002 (appeared electronically).

• E. D. Dolan and J. J. More, Benchmarking optimization

software with performance profiles, Math. Programming, 91 (2),

201-213, 2002.