[PPT] - Null Space Gradient Flows for Constrained Optimization with PowerPoint Presentation

SLIDE 1

Null Space Gradient Flows for Constrained Optimization with Applications to Shape Optimization

Florian Feppon Gr´ egoire Allaire, Charles Dapogny Julien Cortial, Felipe Bordeu SIAM CSE – Spokane – February 26, 2019

SLIDE 2

Shape optimization

Multiphysics, non parametric, shape and topology optimization, in

2D. . .

SLIDE 3

Shape optimization

And in 3D. . .

SLIDE 4

Shape optimization

And in 3D. . . . . . Nonlinear, non convex, infinite dimensional optimization problems featuring multiple and arbitrary constraints!

SLIDE 5

Outline

1. Gradient flows for equality and inequality constrained
ptimizations
2. Demonstration on topology optimization test cases

SLIDE 6

1. Constrained optimization

min

x∈X

J(x) s.t.

❣(x) = 0

❤(x) ≤ 0, with J : X → R, ❣ : X → Rp and ❤ : X → Rq Fr´ echet differentiable. The set X can be ◮ a finite dimensional vector space, X = Rn

SLIDE 7

1. Constrained optimization

min

x∈X

J(x) s.t.

❣(x) = 0

❤(x) ≤ 0, with J : X → R, ❣ : X → Rp and ❤ : X → Rq Fr´ echet differentiable. The set X can be ◮ a finite dimensional vector space, X = Rn ◮ a Hilbert space equipped with a scalar product a(·, ·), X = V

SLIDE 8

1. Constrained optimization

min

x∈X

J(x) s.t.

❣(x) = 0

❤(x) ≤ 0, with J : X → R, ❣ : X → Rp and ❤ : X → Rq Fr´ echet differentiable. The set X can be ◮ a finite dimensional vector space, X = Rn ◮ a Hilbert space equipped with a scalar product a(·, ·), X = V ◮ a “manifold”, as in topology optimization: X = {Ω ⊂ D | Ω Lipschitz }

SLIDE 9

1. A generic optimization algorithm

From a current guess xn, how to select the descent direction ξn given objective J and constraints ❣, ❤?

SLIDE 10

1. A generic optimization algorithm

From a current guess xn, how to select the descent direction ξn given objective J and constraints ❣, ❤? ◮ Many “iteratives” methods in literature:

◮ Penalty methods (like Augmented Lagrangian Method) ◮ Linearization methods : SLP, SQP, MMA, MFD

SLIDE 11

1. A generic optimization algorithm

From a current guess xn, how to select the descent direction ξn given objective J and constraints ❣, ❤? ◮ Many “iteratives” methods in literature:

◮ Penalty methods (like Augmented Lagrangian Method) ◮ Linearization methods : SLP, SQP, MMA, MFD

These methods suffer from: ◮ the need for tuning unintuitive parameters.

SLIDE 12

1. A generic optimization algorithm

From a current guess xn, how to select the descent direction ξn given objective J and constraints ❣, ❤? ◮ Many “iteratives” methods in literature:

◮ Penalty methods (like Augmented Lagrangian Method) ◮ Linearization methods : SLP, SQP, MMA, MFD

These methods suffer from: ◮ the need for tuning unintuitive parameters. ◮ “inconsistencies” when ∆t → 0: SLP, SQP, MFD subproblems may not have a solution if ∆t too small. ALG does not guarantee reducing the objective function if constraints are satisfied.

SLIDE 13

1. A generic optimization algorithm

Dynamical systems approaches: ❣ ❣ ❣ ❣ ❣ ❣ ❣ ❣ ❣ ❣ ❣ ❣ ❣ ❣ ❣ ❣

SLIDE 14

1. A generic optimization algorithm

Dynamical systems approaches: ◮ For unconstrained optimization, the celebrated gradient flow: ˙ x = −∇J(x) J(x(t)) decreases necessarily! ❣ ❣ ❣ ❣ ❣ ❣ ❣ ❣ ❣ ❣ ❣ ❣ ❣ ❣ ❣ ❣

SLIDE 15

1. A generic optimization algorithm

Dynamical systems approaches: ◮ For unconstrained optimization, the celebrated gradient flow: ˙ x = −∇J(x) J(x(t)) decreases necessarily! ◮ For equality constrained optimization, projected gradient flow (Tanabe (1980)): ˙ x = −(I − D❣ T(D❣D❣ T)−1D❣)∇J(x) (gradient flow on X = {x ∈ V | ❣(x) = 0}) ❣ ❣ ❣ ❣ ❣ ❣ ❣ ❣ ❣ ❣ ❣

SLIDE 16

1. A generic optimization algorithm

Dynamical systems approaches: ◮ For unconstrained optimization, the celebrated gradient flow: ˙ x = −∇J(x) J(x(t)) decreases necessarily! ◮ For equality constrained optimization, projected gradient flow (Tanabe (1980)): ˙ x = −(I − D❣ T(D❣D❣ T)−1D❣)∇J(x) (gradient flow on X = {x ∈ V | ❣(x) = 0}) Then Yamashita (1980) added a Gauss-Newton direction: ˙ x = −αJ(I − D❣ T(D❣D❣ T)−1D❣)∇J(x) −αCD❣ T(D❣D❣ T)−1❣(x) ❣ ❣ ❣

SLIDE 17

1. A generic optimization algorithm

Dynamical systems approaches: ◮ For unconstrained optimization, the celebrated gradient flow: ˙ x = −∇J(x) J(x(t)) decreases necessarily! ◮ For equality constrained optimization, projected gradient flow (Tanabe (1980)): ˙ x = −(I − D❣ T(D❣D❣ T)−1D❣)∇J(x) (gradient flow on X = {x ∈ V | ❣(x) = 0}) Then Yamashita (1980) added a Gauss-Newton direction: ˙ x = −αJ(I − D❣ T(D❣D❣ T)−1D❣)∇J(x) −αCD❣ T(D❣D❣ T)−1❣(x) ❣(x(t)) = ❣(x(0))e−αC t and J(x(t)) is guaranteed to decrease if ❣(x(t)) = 0.

SLIDE 18

1. A generic optimization algorithm

For both equality constraints ❣(x) = 0 and inequality constraints ❤(x) ≤ 0, consider I(x) the set of violated constraints:

I(x) = {i ∈ {1, . . . , q} | hi(x) 0}.

❈

I(x) =

❣(x)

| (hi(x))i∈

I(x)

T ❈ ❈ ❈ ❈

SLIDE 19

1. A generic optimization algorithm

For both equality constraints ❣(x) = 0 and inequality constraints ❤(x) ≤ 0, consider I(x) the set of violated constraints:

I(x) = {i ∈ {1, . . . , q} | hi(x) 0}.

❈

I(x) =

❣(x)

| (hi(x))i∈

I(x)

T We propose ˙ x = −αJξJ(x(t)) − αCξC(x(t)) with −ξJ(x) :=

the “best” descent direction

with respect to the constraints I(x) −ξC(x) :=

the Gauss-Newton direction

−D❈ T

I(x)(D❈

I(x)D❈ T

I(x))−1❈

I(x)(x)

SLIDE 20

1. A generic optimization algorithm

min

(x1,x2)∈R2

J(x1, x2) = x2

1 + (x2 + 3)2

s.t.

h1(x1, x2) = −x2

1 + x2

≤ 0 h2(x1, x2) = −x1 − x2 − 2 ≤ 0

SLIDE 21

1. A generic optimization algorithm

We propose: ˙ x = −αJξJ(x(t)) − αCξC(x(t)) with ξJ(x) := (I − D❈ T

I(x)(D❈

I(x)D❈ T

I(x))−1D❈

I(x))(∇J(x))

ξC(x) := D❈ T

I(x)(D❈

I(x)D❈ T

I(x))−1❈

I(x)(x).

SLIDE 22

1. A generic optimization algorithm

We propose: ˙ x = −αJξJ(x(t)) − αCξC(x(t)) with ξJ(x) := (I − D❈ T

I(x)(D❈

I(x)D❈ T

I(x))−1D❈

I(x))(∇J(x))

ξC(x) := D❈ T

I(x)(D❈

I(x)D❈ T

I(x))−1❈

I(x)(x).

I(x) ⊂

I(x) is a subset of the active or violated constraints which can be computed by mean of a dual subproblem.

SLIDE 23

1. A generic optimization algorithm

The best descent direction −ξJ(x) must be proportional to ξ∗ = arg min

ξ∈V

DJ(x)ξ s.t.      D❣(x)ξ = 0 D❤

I(x)(x)ξ ≤ 0

||ξ||V ≤ 1. where ❤

I(x)(x) = (hi(x))i∈ I(x)

SLIDE 24

1. A generic optimization algorithm

Proposition

Let (λ∗(x), µ∗(x)) ∈ Rp × RCard

I(x) the solutions of the following

dual minimization problem: (λ∗(x), µ∗(x)) := arg min

λ∈Rp µ∈R

q(x), µ0

||∇J(x)+D❣(x)T λ+D❤

I(x)(x)T µ||V .

Then, unless x is a KKT point, the best descent direction ξ∗(x) is given by ξ∗(x) = − ∇J(x) + D❣(x)T λ∗(x) + D❤

I(x)(x)T µ∗(x)

||∇J(x) + D❣(x)T λ∗(x) + D❤

I(x)(x)T µ∗(x)||V

.

SLIDE 25

1. A generic optimization algorithm

Proposition

Let I(x) the set obtained by collecting the non zero components of µ∗(x):

I(x) := {i ∈

I | µ∗

i (x) > 0}.

Then ξ∗(x) is explicitly given by: ξ∗(x) = − Π❈

I(x)(∇J(x))

||Π❈

I(x)(∇J(x))||V

, with Π❈

I(x)(∇J(x)) = (I − D❈ T

I(x)(D❈

I(x)D❈ T

I(x))−1D❈

I(x))(∇J(x))

SLIDE 26

1. A generic optimization algorithm

We can prove:

1. Constraints are asymptotically satisfied:

❣(x(t)) = e−αC t❣(x(0)) and ❤

I(x(t)) ≤ e−αC t❤(x(0))

2. J decreases as soon as the violation ❈

I(x(t)) is sufficiently

small

3. All stationary points x∗ of the ODE are KKT points

SLIDE 27

2. Applications to shape optimization