Dr. Ampl A Meta Solver for Optimization Dominique Orban Bob Fourer - - PowerPoint PPT Presentation

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 1/36

• Dr. Ampl

A Meta Solver for Optimization

Dominique Orban

École Polytechnique de Montréal Dominique.Orban@polymtl.ca

Bob Fourer

Northwestern University 4er@iems.northwestern.edu

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 2/36

Outline

■ Numerical Optimization

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 2/36

Outline

■ Numerical Optimization ■ Current software challenges

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 2/36

Outline

■ Numerical Optimization ■ Current software challenges ■ The NEOS Server

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 2/36

Outline

■ Numerical Optimization ■ Current software challenges ■ The NEOS Server ■ The AMPL modeling language

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 2/36

Outline

■ Numerical Optimization ■ Current software challenges ■ The NEOS Server ■ The AMPL modeling language ■ The DrAmpl meta solver

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 2/36

Outline

■ Numerical Optimization ■ Current software challenges ■ The NEOS Server ■ The AMPL modeling language ■ The DrAmpl meta solver ■ Numerical results

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 2/36

Outline

■ Numerical Optimization ■ Current software challenges ■ The NEOS Server ■ The AMPL modeling language ■ The DrAmpl meta solver ■ Numerical results ■ Future directions of research

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 3/36

Numerical Optimization

A general problem, representable in a modeling language may be written as minimize f(x)

• bjective

subject to cL ≤ c(x) ≤ cU general constraints xL ≤ x ≤ xU explicit bounds where f : Rn → R, c : Rn → Rm ∈ C2. Bounds may be equal or infinite, some ci(x) may be linear.

• Outline
• Numerical Optimization

Software

• Current software challenges
• Consequences
• More software challenges
• The NEOS Optimization Server
• The AMPL modeling language
• Some typical AMPL commands
• DrAmpl is a meta solver

Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 4/36

Current software challenges

■ Myriad of solvers available for most classes of optimization

problems

■ Competing solvers designed to tackle the same class of

problems, with varying subtelties

■ Competing general-purpose solvers ■ Competing free and commercial codes ■ Competing modeling languages and environments ■ Often, solvers have limited compatibility with modeling

environments or interfaces do not exist

• Outline
• Numerical Optimization

Software

• Current software challenges
• Consequences
• More software challenges
• The NEOS Optimization Server
• The AMPL modeling language
• Some typical AMPL commands
• DrAmpl is a meta solver

Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 5/36

Consequences

■ Much confusion arising from so much choice ■ Much room for remote or local software offering guidance

and bridging the abyssal gap between problem classes and solvers

■ Much need for tools to help bridge this gap, by compiling lists

• f available software, classifying the possible problem

instances, linking them together

■ Much need for software able to associate a given problem

instance with a general class and a general class with a few solvers.

• Outline
• Numerical Optimization

Software

• Current software challenges
• Consequences
• More software challenges
• The NEOS Optimization Server
• The AMPL modeling language
• Some typical AMPL commands
• DrAmpl is a meta solver

Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 6/36

More software challenges

■ Modeling languages are often commercial ■ Optimization software is often commercial ■ Software may be problematic to install locally ■ Need to learn how to use the software ■ Trial versions are often available with various limitations

• Outline
• Numerical Optimization

Software

• Current software challenges
• Consequences
• More software challenges
• The NEOS Optimization Server
• The AMPL modeling language
• Some typical AMPL commands
• DrAmpl is a meta solver

Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 7/36

The NEOS Optimization Server

■ Remote optimization software repository ■ Problems modelled locally, solved remotely ■ Solution may be returned for later local examination ■ Accepts problems in several modeling languages; AMPL,

GAMS, SIF, SeDuMi, Sparse SDPA, C and Fortran

■ 43 solvers for 12 classes of optimization problems ■ Optimization tree... last updated in March 1996. ■ 2723 submissions from May 2–May 9, up to 5430/week ■ Alleviates the need to install some solvers locally ■ Allows to test solvers before purchasing or downloading ■ Free of charge ■ E-mail, Web, TCP/IP submission clients

How to best make use of all those resources?

• Outline
• Numerical Optimization

Software

• Current software challenges
• Consequences
• More software challenges
• The NEOS Optimization Server
• The AMPL modeling language
• Some typical AMPL commands
• DrAmpl is a meta solver

Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 8/36

The AMPL modeling language

■ Fourer, Gay, Kernighan book ■ Flexible and powerful language to model linear, nonlinear,

mixed-integer and constraint programming

■ Provides some level of preprocessing ■ Provides automatic differentiation ■ Size-limited student version available free of charge ■ Source code of AMPL library available on NetLib to facilitate

interfacing with solvers and exploring the AMPL data structures

• Outline
• Numerical Optimization

Software

• Current software challenges
• Consequences
• More software challenges
• The NEOS Optimization Server
• The AMPL modeling language
• Some typical AMPL commands
• DrAmpl is a meta solver

Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 9/36

Some typical AMPL commands

Select a model, a solver and solve. ampl: model mymodel.mod; ampl: data mymodel.dat; ampl: option solver lancelot; ampl: solve; What is a solver to AMPL? Any program which can receive a model, possibly manipulate it and return some result. Make a distinction between a solver and a meta solver.

• Outline
• Numerical Optimization

Software

• Current software challenges
• Consequences
• More software challenges
• The NEOS Optimization Server
• The AMPL modeling language
• Some typical AMPL commands
• DrAmpl is a meta solver

Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 10/36

DrAmpl is a meta solver

The purposes of DrAmpl are to

■ Analyze a given model ■ Provide bounds on expressions appearing in the model ■ Assess convexity of expressions appearing in the model ■ Classify the model ■ Provide some level of "nonlinear preprocessing" ■ Assist in the choice of an appropriate solver

• Outline
• Numerical Optimization

Software Model analysis

• Analysis of a model
• Analysis of a model
• Analysis of a model
• Analysis of a model

The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 11/36

Analysis of a model

Problem statistics

• # Objectives:

[1 ] # Nonlinear objectives: # Linear objectives: 1 Sparsity statistics

• # nnz in Jacobian:

4000 # nnz in obj. gradients: 800

• Outline
• Numerical Optimization

Software Model analysis

• Analysis of a model
• Analysis of a model
• Analysis of a model
• Analysis of a model

The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 12/36

Analysis of a model

# Constraints: [1603 ] # Range: 801 # Equality: # Inequality: 802 # Spurious:

• # Nonlinear general:

801 # Quadratic: # Linear general: 802 # Nonlinear network: # Linear network: # Complementarity cond: [0 ] # Linear: # Nonlinear: Provides initial point: Yes Provides initial dual point: Yes

• Outline
• Numerical Optimization

Software Model analysis

• Analysis of a model
• Analysis of a model
• Analysis of a model
• Analysis of a model

The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 13/36

Analysis of a model

Variables statistics

• # Variables:

[800 ] # Continuous: 800 # Binary: # Other integer: # Variables bounded above only: | below only: |_ above and below: 800 fixed: free :

• Outline
• Numerical Optimization

Software Model analysis

• Analysis of a model
• Analysis of a model
• Analysis of a model
• Analysis of a model

The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 14/36

Analysis of a model

Nonlinearity statistics

• # Variables appearing nonlinearly:

[800 ] # in objectives only: # in constraints only: 800 # in both: # Integer vars appearing nonlinearly: [0 ] # in objectives only: # in constraints only:

• Outline
• Numerical Optimization

Software Model analysis The DAG

• Internals: The DAG
• Walking the DAG

Bounds Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 15/36

Internals: The DAG

AMPL stores linear expressions in linked lists. Nonlinear expressions are stored in a Directed Acyclic Graph for evaluation and automatic differentiation.

• Outline
• Numerical Optimization

Software Model analysis The DAG

• Internals: The DAG
• Walking the DAG

Bounds Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 15/36

Internals: The DAG

AMPL stores linear expressions in linked lists. Nonlinear expressions are stored in a Directed Acyclic Graph for evaluation and automatic differentiation. x2

1(1 + x2)3

• 2 #*
• 5 #ˆ

v0 #x[1] n2

• 5 #ˆ
• 0 # +

n1 v1 #x[2] n3

* ˆ 2 x[1] ˆ 3 + 1 x[2]

2 * x[1] 3 + 1 x[2] ^ ^

• Outline
• Numerical Optimization

Software Model analysis The DAG

• Internals: The DAG
• Walking the DAG

Bounds Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 15/36

Internals: The DAG

AMPL stores linear expressions in linked lists. Nonlinear expressions are stored in a Directed Acyclic Graph for evaluation and automatic differentiation. AMPL creates a DAG per objective and constraint function. The recursive nature of this data structure allows us to infer useful information on nonlinear expressions. The key ingredient: xL ≤ x ≤ xU.

• Outline
• Numerical Optimization

Software Model analysis The DAG

• Internals: The DAG
• Walking the DAG

Bounds Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 16/36

Walking the DAG

By recursively exploring each node of the DAG, we are able to infer bounds on nonlinear expressions and assess their

• convexity. A prototype function looks like

int OperatorProcessor( *node ) { switch( node->op ) { /* Represents operator */ case leaf: /* Variable or constant */ return appropriate_value; case other; OperatorProcessor( node->left ); /* left child */ OperatorProcessor( node->right ); /* right child */ process( node->op ); } }

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds

• Bounds on expressions
• Simple sample AMPL example
• Objective bounds in COPS
• Redundant constraints
• Fragment of camshape.mod

Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 17/36

Bounds on expressions

Use interval arithmetic and the prototype function to propagate bounds. In the example x2

1(1 + x2)3, assume we have the specified

bounds 1 ≤ x1 ≤ 2 and −3 ≤ x2 ≤ 3.

■ 1 ≤ x2 1 ≤ 4 2 * x[1] 3 + 1 x[2] ^ ^

Bounds on the variables are crucial!

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds

• Bounds on expressions
• Simple sample AMPL example
• Objective bounds in COPS
• Redundant constraints
• Fragment of camshape.mod

Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 17/36

Bounds on expressions

Use interval arithmetic and the prototype function to propagate bounds. In the example x2

1(1 + x2)3, assume we have the specified

bounds 1 ≤ x1 ≤ 2 and −3 ≤ x2 ≤ 3.

■ 1 ≤ x2 1 ≤ 4 ■ −2 ≤ 1 + x2 ≤ 4 2 * x[1] 3 + 1 x[2] ^ ^

Bounds on the variables are crucial!

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds

• Bounds on expressions
• Simple sample AMPL example
• Objective bounds in COPS
• Redundant constraints
• Fragment of camshape.mod

Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 17/36

Bounds on expressions

Use interval arithmetic and the prototype function to propagate bounds. In the example x2

1(1 + x2)3, assume we have the specified

bounds 1 ≤ x1 ≤ 2 and −3 ≤ x2 ≤ 3.

■ 1 ≤ x2 1 ≤ 4 ■ −2 ≤ 1 + x2 ≤ 4 ■ −8 ≤ (1 + x2)3 ≤ 64 2 * x[1] 3 + 1 x[2] ^ ^

Bounds on the variables are crucial!

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds

• Bounds on expressions
• Simple sample AMPL example
• Objective bounds in COPS
• Redundant constraints
• Fragment of camshape.mod

Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 17/36

Bounds on expressions

Use interval arithmetic and the prototype function to propagate bounds. In the example x2

1(1 + x2)3, assume we have the specified

bounds 1 ≤ x1 ≤ 2 and −3 ≤ x2 ≤ 3.

■ 1 ≤ x2 1 ≤ 4 ■ −2 ≤ 1 + x2 ≤ 4 ■ −8 ≤ (1 + x2)3 ≤ 64 ■ −32 ≤ x2 1(1 + x2)3 ≤ 256. 2 * x[1] 3 + 1 x[2] ^ ^

Bounds on the variables are crucial!

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds

• Bounds on expressions
• Simple sample AMPL example
• Objective bounds in COPS
• Redundant constraints
• Fragment of camshape.mod

Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 18/36

Simple sample AMPL example

model; var x >= -6, <= 8; minimize objective: exp(sin(1/x)); DrAmpl deduces the bounds 0.36789 ≤ esin(1/x) ≤ 2.718282

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds

• Bounds on expressions
• Simple sample AMPL example
• Objective bounds in COPS
• Redundant constraints
• Fragment of camshape.mod

Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 19/36

Objective bounds in COPS

f ∗ = lowest value given in the COPS report inf f = inferred lower bound See www.mcs.anl.gov/˜more/cops Problem Parameter f ∗ inf f Bearing ny = 50

• 1.54824e-1

−∞ Camshape n = 800 4.23739e+0 3.974 Catmix nh = 800

• 4.80559e-2

−∞ Chain nh = 200 5.06891e+0 −∞ Channel nh = 200 1.00000e+0 1 Elec np = 100 4.44835e+3

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds

• Bounds on expressions
• Simple sample AMPL example
• Objective bounds in COPS
• Redundant constraints
• Fragment of camshape.mod

Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 19/36

Objective bounds in COPS

f ∗ = lowest value given in the COPS report inf f = inferred lower bound See www.mcs.anl.gov/˜more/cops Problem Parameter f ∗ inf f Gasoil nh = 200 5.23659e-3 Glider nh = 200 1.24880e+3 Marine nh = 200 1.97465e+7 Methanol nh = 100 9.00563e-3 Minsurf ny = 50 2.51488e+0 1.02594 Pinene nh = 50 1.93937e+1

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds

• Bounds on expressions
• Simple sample AMPL example
• Objective bounds in COPS
• Redundant constraints
• Fragment of camshape.mod

Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 19/36

Objective bounds in COPS

f ∗ = lowest value given in the COPS report inf f = inferred lower bound See www.mcs.anl.gov/˜more/cops Problem Parameter f ∗ inf f Polygon nv = 50 6.57163e-1 Robot nh = 200 9.14138e+0 Rocket nh = 400 1.01238e+0 1 Steering nh = 100 5.54594e-1 Torsion ny = 50

• 4.18087e-1
• 0.833013

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds

• Bounds on expressions
• Simple sample AMPL example
• Objective bounds in COPS
• Redundant constraints
• Fragment of camshape.mod

Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 20/36

Redundant constraints

If the inferred bounds γL ≤ ci(x) ≤ γU are tighter than those given in the constraint cL

i ≤ ci(x) ≤ cU i, this constraint is

redundant. Redundant constraints are detected in camshape. Nature Imposed lower Inferred lower

• Inequality

0.997647 < 1 Inequality

• 2.00235

< -2 Found 2 redundant constraints.

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds

• Bounds on expressions
• Simple sample AMPL example
• Objective bounds in COPS
• Redundant constraints
• Fragment of camshape.mod

Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 21/36

Fragment of camshape.mod

param n := 800; param R min := 1.0; param R max := 2.0; param alpha := 1.5; let d theta := 2*pi/(5*(n+1)); var r{1..n} <= R max, >= R min; subject to curvature edge1:

• alpha*d theta <= (r[1] - R min) <= alpha*d theta;

subject to curvature edge2:

• alpha*d theta <= (R max - r[n]) <= alpha*d theta;

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds

• Bounds on expressions
• Simple sample AMPL example
• Objective bounds in COPS
• Redundant constraints
• Fragment of camshape.mod

Convexity Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 21/36

Fragment of camshape.mod

param n := 800; param R min := 1.0; param R max := 2.0; param alpha := 1.5; let d theta := 2*pi/(5*(n+1)); var r{1..n} <= R max, >= R min; subject to curvature edge1:

• alpha*d theta <= (r[1] - R min) <= alpha*d theta;

subject to curvature edge2:

• alpha*d theta <= (R max - r[n]) <= alpha*d theta;

Offending constraints

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity

• Convexity of expressions
• Two sides to convexity
• Disproving convexity
• Final verdict on convexity
• Convexity in COPS

Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 22/36

Convexity of expressions

Similarly, using the prototype function, convexity may be assessed. Constants and variables are convex expression. Recursively, use rules such as

■ f and g convex implies f + g convex ■ fg is convex when both have the same monotonicity and

f, g ≥ 0 and convex, or f, g ≤ 0 and concave

■ ef is convex if f is convex ■ cosh(f) is convex if f is linear or f is convex and

nonnegative, or f is concave and nonpositive

■ √f nonconvex in general; however

√ ex is convex

■ etc. for every operator implemented in AMPL

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity

• Convexity of expressions
• Two sides to convexity
• Disproving convexity
• Final verdict on convexity
• Convexity in COPS

Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 23/36

Two sides to convexity

Recursively walking the dag allows to prove convexity/concavity symbolically. It is an absolute statement.

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity

• Convexity of expressions
• Two sides to convexity
• Disproving convexity
• Final verdict on convexity
• Convexity in COPS

Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 23/36

Two sides to convexity

Recursively walking the dag allows to prove convexity/concavity symbolically. It is an absolute statement. What if no rule applies and convexity cannot be proved?

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity

• Convexity of expressions
• Two sides to convexity
• Disproving convexity
• Final verdict on convexity
• Convexity in COPS

Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 23/36

Two sides to convexity

Recursively walking the dag allows to prove convexity/concavity symbolically. It is an absolute statement. What if no rule applies and convexity cannot be proved? In such a case, attempt to disprove convexity/concavity

• numerically. Use an iterative method to attempt to discover

negative curvature in the Hessian matrix of the function being examined.

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity

• Convexity of expressions
• Two sides to convexity
• Disproving convexity
• Final verdict on convexity
• Convexity in COPS

Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 24/36

Disproving convexity

Use GLTR [Gould et al.] to minimized ∇f(x)T d + 1

2dT ∇2f(x)d

subject to d ≤ ∆. ∆ truncates the Lanczos iterations to keep the work within reasonable limits. By default, select only x = x0 but several xi can be chosen on a latin hypercube in xL ≤ xi ≤ xU. If f is nonconvex at x, negative curvature may be discovered inside the trust region. If not, convexity cannot be disproved. We choose the radius ∆ = max(10, 1

10∇f(x)).

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity

• Convexity of expressions
• Two sides to convexity
• Disproving convexity
• Final verdict on convexity
• Convexity in COPS

Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 25/36

Final verdict on convexity

Proving Disproving Verdict √ X Convex X √ Nonconvex X X Inconclusive √ √ Error!

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity

• Convexity of expressions
• Two sides to convexity
• Disproving convexity
• Final verdict on convexity
• Convexity in COPS

Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 26/36

Convexity in COPS

Problem Pre n m nb Class Objective Time (s) Bearing 2704 208 2704

QP

Convex 0.86 10 2500 2500

BQP

Convex 0.80 Camshape 800 1603 800

QCLP

Linear 0.74 10 800 1600 800

QCLP

Linear 0.76 Catmix 2403 1602 801

QCLP

Linear 4.87 10 2401 1600 801

QCLP

Linear 4.87 Chain 402 203

ECNLP

Inconclusive 0.11 10 400 201

ECNLP

Inconclusive 0.09 Channel 1600 1600

QE

Constant 1.70 10 1598 1598

QE

Constant 1.74 Elec 300 100

QCNLP

Nonconvex 3.13 10 300 100

QCNLP

Nonconvex 3.15

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity

• Convexity of expressions
• Two sides to convexity
• Disproving convexity
• Final verdict on convexity
• Convexity in COPS

Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 26/36

Convexity in COPS

Problem Pre n m nb Class Objective Time (s) Gasoil 2003 2003

QCQP

Convex 2.01 10 2001 1998 3

QCQP

Convex 1.80 Glider 1006 1411

QCLP

Linear 0.65 10 999 800 601

QCLP

Linear 0.73 Marine 4815 4807

QCQP

Convex 9.40 10 4815 4792 15

QCQP

Convex 9.17 Methanol 1205 1205

NCQP

Convex 1.02 10 1202 1197 5

NCQP

Convex 0.75 Minsurf 2704 3696

LCNLP

Inconclusive 0.76 10 2500 2500

LCNLP

Inconclusive 1.01 Pinene 1005 1005

QCQP

Convex 0.85 10 1000 995 5

QCQP

Convex 0.70

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity

• Convexity of expressions
• Two sides to convexity
• Disproving convexity
• Final verdict on convexity
• Convexity in COPS

Classification Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 26/36

Convexity in COPS

Problem Pre n m nb Class Objective Time (s) Polygon 100 1376

NLP

Nonconvex 4.25 10 98 1273 98

ICNLP

Nonconvex 3.17 Robot 1811 1213 1207

QCLP

Linear 2.44 10 1799 1201 1202

NCLP

Linear 2.51 Rocket 1605 2408 401

NCLP

Linear 1.62 10 1601 1200 1601

NCLP

Linear 1.99 Steering 507 510

NCLP

Linear 0.21 10 500 401 103

NCLP

Linear 0.25 Torsion 2704 2704

QP

Convex 0.70 10 2500 2500

BQP

Convex 0.70

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification

• Problem classification
• Problem classification

Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 27/36

Problem classification

Build a richer optimization tree for differentiable problems

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification

• Problem classification
• Problem classification

Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 27/36

Problem classification

NLPs Constrained NLPs Unconstrained QPs Convex Noncnvx Linearly− −constrained NLPs Bound− −constr. NLPs Bound− −constr. QPs Bound− −constr. LPs LPs Linearly− −constr. QPs Quadratically− −constr. QPs Convex Noncnvx Convex Noncnvx NCPs LCPs NLEs NLLS LLS Nonlinearly− −constrained NLPs MINLPs MPECs Equality− −constrained NLPs Inequality− −constrained NLPs Unconstrained LPs Unconstrained NLPs Quadratically− −constrained NLPs Quadratically− −constrained LPs

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification

• Problem classification
• Problem classification

Preprocessing Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 28/36

Problem classification

Specializing a problem moves it down the tree. Problem type

• Problem has bounded variables
• Problem has linear constraints
• Problem has quadratic constraints

Analyzing problem using objective #0

• This objective is linear
• Problem is a constrained NLP
• Problem seems to be an LP
• Problem seems to be a quadratically-constrained LP

with inequality constraints with bounds

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing

• "Nonlinear preprocessing"
• "Nonlinear preprocessing" in COPS

Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 29/36

"Nonlinear preprocessing"

■ Build upon AMPL

’s linear primal presolve

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing

• "Nonlinear preprocessing"
• "Nonlinear preprocessing" in COPS

Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 29/36

"Nonlinear preprocessing"

■ Build upon AMPL

’s linear primal presolve

■ Attempt to tighten bounds on variables

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing

• "Nonlinear preprocessing"
• "Nonlinear preprocessing" in COPS

Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 29/36

"Nonlinear preprocessing"

■ Build upon AMPL

’s linear primal presolve

■ Attempt to tighten bounds on variables ■ Examine each constraint in turn and decompose

ci(x) = ℓ(x) + αjxj + n(x)

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing

• "Nonlinear preprocessing"
• "Nonlinear preprocessing" in COPS

Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 29/36

"Nonlinear preprocessing"

■ Build upon AMPL

’s linear primal presolve

■ Attempt to tighten bounds on variables ■ Examine each constraint in turn and decompose

ci(x) = ℓ(x) + αjxj + n(x)

■ Find bounds on ℓ(x) and n(x) and obtain

c

L

i − ℓ

U − n U ≤ αjxj ≤ c U

i − ℓ

L − n L

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing

• "Nonlinear preprocessing"
• "Nonlinear preprocessing" in COPS

Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 29/36

"Nonlinear preprocessing"

■ Build upon AMPL

’s linear primal presolve

■ Attempt to tighten bounds on variables ■ Examine each constraint in turn and decompose

ci(x) = ℓ(x) + αjxj + n(x)

■ Find bounds on ℓ(x) and n(x) and obtain

c

L

i − ℓ

U − n U ≤ αjxj ≤ c U

i − ℓ

L − n L

■ May return tighter bounds than [xL j, xU j]

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing

• "Nonlinear preprocessing"
• "Nonlinear preprocessing" in COPS

Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 29/36

"Nonlinear preprocessing"

■ Build upon AMPL

’s linear primal presolve

■ Attempt to tighten bounds on variables ■ Examine each constraint in turn and decompose

ci(x) = ℓ(x) + αjxj + n(x)

■ Find bounds on ℓ(x) and n(x) and obtain

c

L

i − ℓ

U − n U ≤ αjxj ≤ c U

i − ℓ

L − n L

■ May return tighter bounds than [xL j, xU j] ■ Keep an updated "tight" vector of bounds

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing

• "Nonlinear preprocessing"
• "Nonlinear preprocessing" in COPS

Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 29/36

"Nonlinear preprocessing"

■ Build upon AMPL

’s linear primal presolve

■ Attempt to tighten bounds on variables ■ Examine each constraint in turn and decompose

ci(x) = ℓ(x) + αjxj + n(x)

■ Find bounds on ℓ(x) and n(x) and obtain

c

L

i − ℓ

U − n U ≤ αjxj ≤ c U

i − ℓ

L − n L

■ May return tighter bounds than [xL j, xU j] ■ Keep an updated "tight" vector of bounds ■ Command-line option for (finite) number of passes

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing

• "Nonlinear preprocessing"
• "Nonlinear preprocessing" in COPS

Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 29/36

"Nonlinear preprocessing"

■ Build upon AMPL

’s linear primal presolve

■ Attempt to tighten bounds on variables ■ Examine each constraint in turn and decompose

ci(x) = ℓ(x) + αjxj + n(x)

■ Find bounds on ℓ(x) and n(x) and obtain

c

L

i − ℓ

U − n U ≤ αjxj ≤ c U

i − ℓ

L − n L

■ May return tighter bounds than [xL j, xU j] ■ Keep an updated "tight" vector of bounds ■ Command-line option for (finite) number of passes ■ Tighter bounds on x means "no-looser" bounds on functions

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing

• "Nonlinear preprocessing"
• "Nonlinear preprocessing" in COPS

Solver choice Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 30/36

"Nonlinear preprocessing" in COPS

passes fmin fmax n/nvar 3.14159 6.28319 0/800 5 3.16109 5.44318 431/800 Problem camshape with presolve = 0 passes fmin fmax n/nvar 3.974 6.24433 0/800 5 3.974 5.44305 425/800 Problem camshape with presolve = 10

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice

• Assistance in choosing a solver
• Recommendation example

Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 31/36

Assistance in choosing a solver

■ Once problem class is identified, query solver database ■ SQL queries assembled as classification is carried out ■ Database contains 28 solvers for linear and nonlinear

• ptimization

■ Solvers are/were on NEOS ■ Database links solvers to problem classes ■ Implemented with SQLite

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice

• Assistance in choosing a solver
• Recommendation example

Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 32/36

Recommendation example

Assume problem was deemed convex ### First query:: Soft parameters only ### SBB TRON BLMVM MINLP FortMP PathNLP L-BFGS-B First query ignores convexity requirement

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice

• Assistance in choosing a solver
• Recommendation example

Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 32/36

Recommendation example

Assume problem was deemed convex ### Second query:: Include hard parameters ### OOQP MOSEK Second query enforces convexity requirement, if detected

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice

• Assistance in choosing a solver
• Recommendation example

Final notes Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 32/36

Recommendation example

Assume problem was deemed convex ### Matching general-purpose solvers ### Loqo Knitro Lancelot Last query returns matching general-purpose solvers

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes

• Statistics on DrAmpl
• Other features
• Future features
• Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 33/36

Statistics on DrAmpl

■ About 7000 lines of code ■ Uses the AMPL library extensively ■ Written in ANSI C, should install on any platform where the

AMPL library may be installed

■ Being developed and tested in Linux ■ Will (hopefully quite soon) be available for testing on NEOS

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes

• Statistics on DrAmpl
• Other features
• Future features
• Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 34/36

Other features

The following features are being implemented

■ Prove quadraticity

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes

• Statistics on DrAmpl
• Other features
• Future features
• Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 34/36

Other features

The following features are being implemented

■ Prove quadraticity ■ Prove that a nonlinear function is a polynomial

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes

• Statistics on DrAmpl
• Other features
• Future features
• Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 34/36

Other features

The following features are being implemented

■ Prove quadraticity ■ Prove that a nonlinear function is a polynomial ■ Prove that a function is DC

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes

• Statistics on DrAmpl
• Other features
• Future features
• Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 34/36

Other features

The following features are being implemented

■ Prove quadraticity ■ Prove that a nonlinear function is a polynomial ■ Prove that a function is DC ■ Assess convexity of feasible set

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes

• Statistics on DrAmpl
• Other features
• Future features
• Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36

Future features

■ Detect infeasible problems

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes

• Statistics on DrAmpl
• Other features
• Future features
• Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36

Future features

■ Detect infeasible problems ■ Detect unbounded problems

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes

• Statistics on DrAmpl
• Other features
• Future features
• Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36

Future features

■ Detect infeasible problems ■ Detect unbounded problems ■ Recommend a solver based on past experience

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes

• Statistics on DrAmpl
• Other features
• Future features
• Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36

Future features

■ Detect infeasible problems ■ Detect unbounded problems ■ Recommend a solver based on past experience ■ Include many more solvers in database

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes

• Statistics on DrAmpl
• Other features
• Future features
• Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36

Future features

■ Detect infeasible problems ■ Detect unbounded problems ■ Recommend a solver based on past experience ■ Include many more solvers in database ■ Treat problems with integer variables

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes

• Statistics on DrAmpl
• Other features
• Future features
• Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36

Future features

■ Detect infeasible problems ■ Detect unbounded problems ■ Recommend a solver based on past experience ■ Include many more solvers in database ■ Treat problems with integer variables ■ Link with mProbe

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes

• Statistics on DrAmpl
• Other features
• Future features
• Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36

Future features

■ Detect infeasible problems ■ Detect unbounded problems ■ Recommend a solver based on past experience ■ Include many more solvers in database ■ Treat problems with integer variables ■ Link with mProbe ■ Minimize the work in nonlinear presolve

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes

• Statistics on DrAmpl
• Other features
• Future features
• Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36

Future features

■ Detect infeasible problems ■ Detect unbounded problems ■ Recommend a solver based on past experience ■ Include many more solvers in database ■ Treat problems with integer variables ■ Link with mProbe ■ Minimize the work in nonlinear presolve ■ Output updated nl file

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes

• Statistics on DrAmpl
• Other features
• Future features
• Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36

Future features

■ Detect infeasible problems ■ Detect unbounded problems ■ Recommend a solver based on past experience ■ Include many more solvers in database ■ Treat problems with integer variables ■ Link with mProbe ■ Minimize the work in nonlinear presolve ■ Output updated nl file ■ Compare solving times with[out] presolving

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes

• Statistics on DrAmpl
• Other features
• Future features
• Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36

Future features

■ Detect infeasible problems ■ Detect unbounded problems ■ Recommend a solver based on past experience ■ Include many more solvers in database ■ Treat problems with integer variables ■ Link with mProbe ■ Minimize the work in nonlinear presolve ■ Output updated nl file ■ Compare solving times with[out] presolving ■ IEEE directed roundings

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes

• Statistics on DrAmpl
• Other features
• Future features
• Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36

Future features

■ Detect infeasible problems ■ Detect unbounded problems ■ Recommend a solver based on past experience ■ Include many more solvers in database ■ Treat problems with integer variables ■ Link with mProbe ■ Minimize the work in nonlinear presolve ■ Output updated nl file ■ Compare solving times with[out] presolving ■ IEEE directed roundings ■ Bounds on dual variables from KKT conditions

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes

• Statistics on DrAmpl
• Other features
• Future features
• Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36

Future features

■ Detect infeasible problems ■ Detect unbounded problems ■ Recommend a solver based on past experience ■ Include many more solvers in database ■ Treat problems with integer variables ■ Link with mProbe ■ Minimize the work in nonlinear presolve ■ Output updated nl file ■ Compare solving times with[out] presolving ■ IEEE directed roundings ■ Bounds on dual variables from KKT conditions ■ Hand model off to Kestrel

• Outline
• Numerical Optimization

Software Model analysis The DAG Bounds Convexity Classification Preprocessing Solver choice Final notes

• Statistics on DrAmpl
• Other features
• Future features
• Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 36/36

Personal data

Contact

Dominique.Orban@polymtl.ca

Homepage www.mgi.polymtl.ca/dominique.orban Reports ${HOME}/reports.html