[PPT] - Lus F. D. Brs Alvaro F. M. Azevedo Faculty of Engineering PowerPoint Presentation

SLIDE 1

28-30 May 2001 OPTI 2001 Bologna - Italy 1

Luís F. D. Brás Alvaro F. M. Azevedo

Faculty of Engineering University of Porto PORTUGAL

OBJECT ORIENTED IMPLEMENTATION OF A SECOND-ORDER OPTIMIZATION METHOD

SLIDE 2

28-30 May 2001 OPTI 2001 Bologna - Italy 2

OPTIMIZATION APPROACH

Nonlinear program
Second-order approximation
Integrated formulation
All the problem variables are present in

the nonlinear program

No sensitivity analysis

SLIDE 3

28-30 May 2001 OPTI 2001 Bologna - Italy 3

OPTIMIZATION SOFTWARE

General purpose code
Lagrange-Newton method
Symbolic manipulation of all the functions
Exact 1st and 2nd derivatives
Object oriented approach
Language: C++

SLIDE 4

28-30 May 2001 OPTI 2001 Bologna - Italy 4

OBJECT ORIENTED PROGRAMMING

What we gain
What we lose

♦ Higher abstraction level ♦ Encapsulation of lower level complexities ♦ Code maintenance and reuse is facilitated ♦ Performance ♦ Straightforward coding

SLIDE 5

28-30 May 2001 OPTI 2001 Bologna - Italy 5

OBJECT ORIENTED FEATURES

Classes
Function and operator overloading
Inheritance
Polymorphism
Templates
Exception handling

SLIDE 6

28-30 May 2001 OPTI 2001 Bologna - Italy 6

NONLINEAR PROGRAMMING

Variables / functions real and continuous
Generic functions can be treated

M inim ize f x ( )

~

subject to

g x

~ ~ ~

( ) ≤ 0

→ = g x s

i i

( ) +

~ 2

h x

~ ~ ~

( ) = 0

SLIDE 7

28-30 May 2001 OPTI 2001 Bologna - Italy 7

LAGRANGIAN

( ) ( ) ( ) ( )

L X f x g x s h x

k g k k k m k h k k p ~ ~ ~ ~

= + +       +

= =

∑ ∑

λ λ

2 1 1

VARIABLES SOLUTION

Stationary point of the Lagrangian

( )

h g

s x X

~ ~ ~ ~ ~

, , , λ λ =

SLIDE 8

28-30 May 2001 OPTI 2001 Bologna - Italy 8

SYSTEM OF NONLINEAR EQUATIONS

2 si

i g

λ =

( )

i m = 1,...,

( )

∇ = ⇒ L X

~ ~

g s

i i

+ =

2

( )

i m = 1,..., ∂ ∂ λ ∂ ∂ λ ∂ ∂ f x g x h x

i k g k m k i k h k p k i

+ + =

= =

∑ ∑

1 1

( )

i n = 1,..., hi = 0

( )

i p = 1,...,

The solution of the system is a KKT solution when

λg

~

≥ 0

SLIDE 9

28-30 May 2001 OPTI 2001 Bologna - Italy 9

LAGRANGE-NEWTON METHOD

∇ = L X ( )

~ ~

The system of nonlinear equations

is solved by the Newton method

( ) ( )

H X X L X

q q q ~ ~ ~ ~ ~ − −

+ ∇ =

1 1

∆

In each iteration the following system of linear equations has

to be solved

H is the Hessian of the Lagrangian
Second derivatives of all the functions are required

SLIDE 10

28-30 May 2001 OPTI 2001 Bologna - Italy 10

SYSTEM OF LINEAR EQUATIONS

Gaussian elimination
Conjugate gradients

♦ adapted to the sparsity pattern of the Hessian matrix ♦ diagonal preconditioning ♦ adapted to an indefinite Hessian matrix

SLIDE 11

28-30 May 2001 OPTI 2001 Bologna - Italy 11

LINE SEARCH

X X X

q q q ~ ~ ~

= +

−1

α ∆

The value of α minimizes the error in

direction

The value of α is made considerably smaller (e.g. α = 0.1)

♦ stable convergence ♦ more iterations - slower ♦ the value of α is often close to one ♦ faster convergence ♦ process may fail

∆ X q

~

SLIDE 12

28-30 May 2001 OPTI 2001 Bologna - Italy 12

AUTOMATIC DIFFERENTIATION

Expression evaluation
Partial derivative calculation (first, second, ...)
Each function is parsed and stored as a tree of tokens

(constants, variables and operators)

Automatic differentiation is based on Rall numbers

SLIDE 13

28-30 May 2001 OPTI 2001 Bologna - Italy 13

RALL NUMBERS

A Rall number is a class that encapsulates the numerical value
f the function, its gradient vector and its Hessian matrix
All the operators are overloaded in order to apply the

differentiation rules

With Rall numbers automatic differentiation can be efficiently

performed

SLIDE 14

28-30 May 2001 OPTI 2001 Bologna - Italy 14

RALL NUMBERS

Example: Functions and

( )

1 1 1

x g f g x f g f x ∂ ∂ + ∂ ∂ = ∂ ∂

( )

2 1 2 2 1 1 2 2 1 2 2 1 2

x x g f x g x f x g x f g x x f g f x x ∂ ∂ ∂ + ∂ ∂ ∂ ∂ + ∂ ∂ ∂ ∂ + ∂ ∂ ∂ = ∂ ∂ ∂

( )

2 1, x

x f

( )

2 1, x

x g Derivatives of the product:

SLIDE 15

28-30 May 2001 OPTI 2001 Bologna - Italy 15

RALL NUMBERS

class CRall { double x; // Operand value double v[2]; // df/dx1, df/dx2 double m[2][2]; // d2f/dxi dxj public: CRall CRall::operator* (const CRall & g) const { CRall t; t.x = x * g.x; t.v[0] = v[0]*g.x + x*g.v[0]; t.v[1] = v[1]*g.x + x*g.v[1]; t.m[0][0] = m[0][0]*g.x+v[0]*g.v[0]+v[0]*g.v[0]+x*g.m[0][0]; t.m[0][1] = m[0][1]*g.x+v[0]*g.v[1]+v[1]*g.v[0]+x*g.m[0][1]; t.m[1][0] = m[1][0]*g.x+v[1]*g.v[0]+v[0]*g.v[1]+x*g.m[1][0]; t.m[1][1] = m[1][1]*g.x+v[1]*g.v[1]+v[1]*g.v[1]+x*g.m[1][1]; return t; } };

SLIDE 16

28-30 May 2001 OPTI 2001 Bologna - Italy 16

RALL NUMBERS

x = constant value; v = [0,0]; m = [[0,0],[0,0]] x = value of x1; v = [1,0]; m = [[0,0],[0,0]] x = value of x2; v = [0,1]; m = [[0,0],[0,0]]

SLIDE 17

28-30 May 2001 OPTI 2001 Bologna - Italy 17

EXPRESSION PARSER

A binary tree is constructed according to the operator

precedence

Each tree node is a Rall number
A symbol table is initialized with the values of the

variables and constants

The tree traversal causes an evaluation of the function,

gradient and Hessian

SLIDE 18

28-30 May 2001 OPTI 2001 Bologna - Italy 18

EXPRESSION PARSER

Example:

( )

) 6 ( / ) 8 ( , ,

2 3 2 1 3 2 1

x x x x x x f − + =

8

x2

*

x1

+

x3

2 ^

6 − /

SLIDE 19

28-30 May 2001 OPTI 2001 Bologna - Italy 19

EXPRESSION PARSER

Exp_Obj Binary_Obj Mul_Obj Binary_Add_Obj Binary_Sub_Obj Div_Obj Pow_Obj Unary_Obj Unary_Log10 Unary_Loge Unary_Cos Unary_Exp Unary_Minus Unary_Sin Unary_Sqr Unary_Sqrt Base_Expression CExpression Close_Paren_Obj Literal_Obj Open_Paren_Obj Start_Obj Stop_Obj Variable_Obj ExpStack Symbol_Table

SLIDE 20

28-30 May 2001 OPTI 2001 Bologna - Italy 20

SCALING

500 10 2 . 200 2000 .

2 1 2 4 2 2 3 1 1

= + − = + − = + + − x x x x x x to subject x Min 707 . 707 . 447 . 894 . 447 . 384 . 256 . 640 . .

2 1 2 4 2 2 3 1 1

= + − = + − = + + − y y y y y y to subject y Min

Variable substitution:
Constraint normalization:

i i

x c x =

i i

g c g =

SLIDE 21

28-30 May 2001 OPTI 2001 Bologna - Italy 21

x2 x1 x x 100 cm A (cross-sectional area) A F = (f1,f2) 1 2 3

σ m ax = 10 0

2

kN cm r F kN = 2 0 0

f f

2 1

8 =

NUMERICAL EXAMPLE

Variables: A , x
Svanberg’s solution confirmed

SLIDE 22

28-30 May 2001 OPTI 2001 Bologna - Italy 22

( ) ( ) ( )

6 . 1 1 . ; . 4 2 . 1 1 8 1 , 1 1 8 1 , 1 , .

2 1 2 1 1 2 2 2 2 1 2 2 1 1 2 2 2 2 1 1 2 2 1 1 2 1

≤ ≤ ≤ ≤ ≤         − + = ≤         + + = + = x x x x x x C x x x x x x C x x to subject x x C x x w Min σ σ

NONLINEAR PROGRAM

SLIDE 23

28-30 May 2001 OPTI 2001 Bologna - Italy 23 # Main title Shape optimization of a two bar truss # N. of eq. constr.; N. of ineq. constr. 0 6 # Objective Function C1 * x1 * sqrt(1+x2^2); # Allowable stress - bar 1 C2 * sqrt(1+x2^2) * (8/x1+1/x1/x2) - 1; # Allowable stress - bar 2 C2 * sqrt(1+x2^2) * (8/x1-1/x1/x2) - 1; # Minimum x1

x1 + 0.2;

# Maximum x1 x1 - 4.0; # Minimum x2

x2 + 0.1;

# Maximum x2 x2 - 1.6; # N. of variables 4 SUBSTITUTED, 1.000, C1; SUBSTITUTED, 0.124, C2; INDEPENDENT, 1.5, x1; INDEPENDENT, 0.5, x2;

DATA FILE

SLIDE 24

28-30 May 2001 OPTI 2001 Bologna - Italy 24

Stress and displacement constraints 38 bar truss - member sizing

SLIDE 25

28-30 May 2001 OPTI 2001 Bologna - Italy 25

CONCLUSIONS

Code maintenance
Efficiency and accuracy in the evaluation of derivatives
Friendly user interface is still required
Not efficient in the OO manipulation of the Hessian matrix
Easy inclusion of alternative numerical techniques