Sensitivity analysis for optimal control problems. - - PowerPoint PPT Presentation

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Sensitivity analysis for optimal control problems. Chance-constrained stochastic optimal control.

PhD defense Laurent Pfeiffer

Inria-Saclay and Ecole Polytechnique Advisor: J. Fr´ ed´ eric Bonnans

November 5th, 2013

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

About optimal control

A theory which aims at optimizing the behavior of a dynamical system over time. We distinguish: the state variable the control variable: the decision variable. The typical models: an ODE or a SDE in a stochastic framework. Two main approaches, based on

• ptimality conditions (Pontryagin’s principle)

dynamic programming (HJB equation).

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

1 Introduction to sensitivity analysis of optimal control problems 2 Necessary conditions in Pontryagin form 3 Sufficient conditions in Pontryagin form 4 Sensitivity analysis 5 Chance-constrained stochastic optimal control

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

1 Introduction to sensitivity analysis of optimal control problems 2 Necessary conditions in Pontryagin form 3 Sufficient conditions in Pontryagin form 4 Sensitivity analysis 5 Chance-constrained stochastic optimal control

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Formulation

Let U := L∞(0, T; Rm) and Y := W 1,∞(0, T; Rn) be respectively the control and state spaces. The optimization problem is inf

u∈U, y∈Y φ(yT) subject to:

(P) the state equation: ˙ yt = f (ut, yt), y0 = y 0 final-state inequality constraints: Φ(yT) ≤ 0 mixed inequality constraints: c(ut, yt) ≤ 0, for a.a. t ∈ (0, T) to simplify, no state constraints: g(yt) ≤ 0, ∀t ∈ [0, T]. From an abstract point of view: infu∈U J (u) s.t. G(u) ∈ K. From now on, we fix a feasible trajectory (¯ u, ¯ y).

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Multipliers

We define the end-point Lagrangian Φ: Φ[β, Ψ](yT ) = βφ(yT ) + ΨΦ(yT), the Hamiltonian H and augmented Hamiltonian Ha H[p](u, y) = pf (u, y), Ha[p, ν](u, y) = pf (u, y) + νc(u, y), the normal cone to the constraints N(¯ u): N(¯ u) =

• λ = (β, Ψ, ν) ∈ R+ × RnΦ × L∞(0, T; Rnc ),

Ψ ∈ NR

nΦ − (Φ(¯

yT)), νt ∈ NRnc

− (c(¯

ut, ¯ yt)), for a.a. t

• and the costate pλ associated with λ ∈ N(¯

u), solution to −˙ pt = DyHa[pλ

t , νt](¯

ut, ¯ yt), pT = DΦ[β, Ψ](¯ yT).

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Multipliers

We define the set of (generalized) Lagrange multipliers ΛL(¯ u):

• λ ∈ N(¯

u) : DuHa[pλ

t , νt](¯

ut, ¯ yt) = 0, for a.a. t

• \{(0, 0, 0)}.

We consider the multimapping: U(t) = cl{u ∈ Rm : c(u, ¯ yt) < 0}, the set of feasible controls for the mixed constraints (at ¯ y). We define the set of Pontryagin multipliers ΛP(¯ u):

• λ ∈ ΛL(¯

u) : H[pλ

t ](¯

ut, ¯ yt) ≤ H[pλ

t ](u, ¯

yt), for a.a. t, ∀u ∈ U(t)

• .

Informal statement: ΛL(¯ u) associated with the L∞ local optimality, ΛP(¯ u) with the L1 local optimality.

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Main issues

The results that will presented deal with: Necessary second-order conditions satisfied if ¯ u is L1-locally

• ptimal,

Sufficient second-order conditions ensuring the L1-optimality, Sensitivity analysis in a L1-neighborhood: V (θ) = inf

u∈U J (u, θ) s.t. G(u, θ) ∈ K, u − ¯

u1 ≤ η, where η > 0 is arbitrarily small and ¯ u the solution for θ = 0. Results:

a second-order expansion of V (θ) near 0, first-order information for the optimal solution.

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

1 Introduction to sensitivity analysis of optimal control problems 2 Necessary conditions in Pontryagin form 3 Sufficient conditions in Pontryagin form 4 Sensitivity analysis 5 Chance-constrained stochastic optimal control

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Abstract approach

Given v and w, we say that u(θ) is an associated path if u(θ) = ¯ u + θv + θ2w + o(θ2), θ ≥ 0. If ¯ u is locally optimal and u(θ) feasible, then J (u(θ)) − J (¯ u) ≥ 0. Taking the best possible path leads to necessary conditions. Denote by C(¯ u) the critical cone in U2 := L2(0, T; Rm), defined by

• v ∈ U2 : DJ (¯

u)v ≤ 0, DG(¯ u)v ∈ TK(G(¯ u))

• and define the Lagrangian and its Hessian:

L[λ](u) = J (u) + λ, G(u), Ω[λ](v) = D2

uuL[λ](u)v 2.

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Second-order tools

For v ∈ U2, denote by z[v] the linearized state : ˙ zt[v] = Df (¯ ut, ¯ yt)(vt, zt[v]), z0[v] = 0. Then, C(¯ u) =

• v ∈ U2 : Dφ(¯

yT )zT[v] ≤ 0, DΦ(¯ yT)zT[v] ∈ TR

nΦ − (Φ(yT )),

Dc(¯ ut, ¯ yt)(vt, z[v]t) ∈ TRnc

− (c(¯

ut, ¯ yt)), for a.a. t

• .

and Ω[λ](v) = T D2Ha[pλ

t , νt](¯

ut, ¯ yt)(vt, zt[v])2dt + D2Φ[β, Ψ](¯ yT )zT[v]2.

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Technical assumptions

For all i = 1, ..., nc, let ∆δ

c,i =

• t ∈ [0, T] : ci(¯

ut, ¯ yt) ≥ −δ

• .

Assumption Inward pointing condition: ∃v in U and ε > 0 such that c(¯ ut, ¯ yt) + Duc(¯ ut, ¯ yt)vt ≤ −ε. Surjectivity condition: there exists δ > 0 such that the following linear mapping is onto: v ∈ U2 →

• Dci(¯

ut, ¯ yt)(vt, zt[v])|∆δ

c,i

• 1≤i≤nc ∈ nc

i=1 L2(∆δ c,i).

Strict complementarity: there exists λ = (β, Ψ, ν) ∈ ΛL(¯ u) such that for all i = 1, ..., nc, for a.a. t ∈ ∆0

c,i, νt,i > 0.

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Results

Definition The control ¯ u is: a weak minimum iff it is locally optimal for the L∞ norm. a Pontryagin minimum iff ∀R > ¯ u∞, ∃ε > 0, G(u) ∈ K, u∞ ≤ R, and u − ¯ u1 ≤ ε = ⇒ J (u) ≥ J (¯ u). Theorem (Second-order necessary conditions) Let the three technical assumptions hold. If ¯ u is a weak minimum, then for all v ∈ C(¯ u), ∃λ ∈ ΛL(¯ u) such that Ω[λ](v) ≥ 0. If ¯ u is a Pontryagin minimum, then for all v ∈ C(¯ u), ∃λ ∈ ΛP(¯ u) such that Ω[λ](v) ≥ 0.

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Elements of proof

There exists a sequence (uk)k of controls which is such that ∀k, ∃δ > 0, for a.a. t, c(uk

t , ¯

yt) < −δ, for a.a. t, {uk

t }k is dense into U(t).

Let K, α ∈ L∞(0, T; RK

+), u ∈ U, the relaxed state equation is

˙ yt =

• 1 − K

k=1 αk t

• f (ut, yt) + K

k=1 αk t f (uk t , yt).

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Elements of proof

The relaxed problem PK is the same as P, with a supplementary control α and the relaxed state equation. The set Lagrange multipliers of PK at (¯ u, α = 0) is:

• λ ∈ ΛL(¯

u) : H[pλ

t ](¯

ut, ¯ yt) ≤ H[pλ

t ](uk t , ¯

yt), ∀k = 1, ...K, for a.a.t

• .

If ¯ u is a Pontryagin minimum, then (¯ u, α = 0) is a weak minimum of PK. Let v ∈ C(¯ u), by the weak conditions, there exists a Lagrange multiplier λK to PK such that Ω[λK](v) ≥ 0 The sequence λK/|λK| has a weak limit point λ ∈ ΛP(¯ u), satisfying Ω[λ](v) ≥ 0.

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Bibliography

On necessary conditions for a weak minimum:

J.F. Bonnans, A. Hermant. Second-order analysis for optimal control problems with pure and mixed constraints, ANIHP, 2009.

On necessary conditions for a Pontryagin minimum (without state constraints):

N.P. Osmolovskii. Quadratic extremality conditions for broken extremals in the general problem of the calculus of variations, J. Math. Sci., 2004.

Our result:

J.F. Bonnans, X. Dupuis, L.P. Second-order necessary conditions in Pontryagin form for optimal control problems. Submitted, Inria Research Report 3806.

Additional results: Simplification of the proof of the weak conditions Qualification condition equivalent to the non-degeneracy of Pontryagin multipliers.

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

1 Introduction to sensitivity analysis of optimal control problems 2 Necessary conditions in Pontryagin form 3 Sufficient conditions in Pontryagin form 4 Sensitivity analysis 5 Chance-constrained stochastic optimal control

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Sufficient conditions

Definition We say the second-order sufficient conditions in Pontryagin form hold iff there exists α > 0 such that for some λ∗ =(β∗,Ψ∗,ν∗)∈ΛP(¯ u), for a.a. t, for all u ∈ U(t), H[pλ∗

t ](u, ¯

yt) − H[pλ∗

t ](¯

ut, ¯ yt) ≥ α|u − ¯ ut|2, for all v ∈ C(¯ u)\{0}, ∃λ ∈ ΛP(¯ u) such that Ω[λ](v) > 0.

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Supplementary assumptions

Assumption (A metric regularity property) There exists ε > 0 such that ∀(u, y) ∈ U ×Y with y − ¯ y∞ ≤ ε, if (u, y) satisfies the mixed constraints, then there exists ˆ u such that c(ˆ ut, ¯ yt) ≤ 0, for a.a. t and u − ˆ u∞ = O(y − ¯ y∞). Assumption (Legendre-Clebsch) For all λ ∈ ΛP(¯ u), Ω[λ] is a Legendre form. This is satisfied if and only if for all λ ∈ ΛP(¯ u), ∃ γ > 0 such that for a.a. t, γId ≤ D2

uuHa[pλ t , νt](¯

ut, ¯ yt). We still need the inward pointing condition.

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Quadratic growth

Definition We say that the quadratic growth for bounded strong solutions holds iff ∀R > ¯ u∞, ∃ε > 0, α > 0 such that for all feasible (u, y), u∞ ≤ R and y − ¯ y∞ ≤ ε = ⇒ J (u) − J (¯ u) ≥ αu − ¯ u2

2.

Theorem Assume that the following assumptions hold: the sufficient second-order conditions the last two extra conditions. Then, the quadratic growth for bounded strong solutions holds.

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Elements of proof

Proof (by contradiction). Let R > 0. Assume that there exists a sequence of feasible trajectories (uk, y k) such that: uk − ¯ u∞ ≤ R and y k − ¯ y∞ → 0 J (uk) − J (¯ u) ≤ o(uk − ¯ u2

2).

With the quadratic growth of the Hamiltonian, and the metric regularity assumption, we derive that: uk − ¯ u2 → 0. Moreover, for all λ ∈ ΛP(¯ u), J (uk) − J (¯ u) ≥ J (uk) − J (¯ u) + λ, G(uk) − G(¯ u) = L[λ](uk) − L[λ](¯ u) = Ω[λ](uk − ¯ u) + O(uk − ¯ u3

3).

But we need a finer expansion, with a remainder: o(uk − ¯ u2

2).

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Decomposition principle

Let (Ak, Bk) be a partition of [0, T] such that meas(Bk) → 0 and uA,k − ¯ u∞ → 0. There exists a control ˜ uk such that for a.a. t, c(˜ uk

t , y k t ) ≤ 0,

˜ uk −¯ u∞ = O(y k −¯ y∞).

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Conclusion of the proof

Theorem (Decomposition principle) For all λ ∈ ΛP(¯ u), the following expansion holds: J (uk) − J (¯ u) ≥

• Bk
• H[pλ

t ](uk t , ¯

yt) − H[pλ

t ](˜

uk

t , ¯

yt)

• dt

+Ω[λ](uA,k − ¯ u) + o(u − ¯ u2

2).

The small perturbations (on Ak) are not negligible compared to the large ones. Let v be a weak limit point of the ratio

uA,k−¯ u uA,k−¯ u2.

By the Legendre-Clebsch assumption, for all λ ∈ ΛP(¯ u), Ω[λ](v) ≤ 0 and v ∈ C(¯ u), thus v = 0. By the Legendre-Clebsch assumption, the convergence is strong: contradiction.

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Bibliography

On the decomposition principle:

J.F. Bonnans, N.P. Osmolovskii. Second-order analysis of optimal control problems with control and initial-final state constraints, J. Conv. Anal., 2010.

Another result of quadratic growth property:

N.P. Osmolovskii. Second-order sufficient optimality conditions for control problems with linearly independent gradients of control constraints, ESAIM: COCV, 2012.

Our results:

J.F. Bonnans, X. Dupuis, L. Pfeiffer. Second-order sufficient conditions for strong solutions to optimal control problems. To appear in ESAIM: COCV.

Open question: result without the Legendre-Clebsch assumption ?

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

1 Introduction to sensitivity analysis of optimal control problems 2 Necessary conditions in Pontryagin form 3 Sufficient conditions in Pontryagin form 4 Sensitivity analysis 5 Chance-constrained stochastic optimal control

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Setting

The family under study, parameterized by θ ≥ 0 is inf

u∈U, y∈Y φ(yT , θ), s.t. Φ(yT, θ) ≤ 0, ˙

yt = f (ut, yt, θ), y0 = y 0. Let (¯ u, ¯ y) be feasible for θ = 0, let R > ¯ u∞, let PR be the set of probability measures on the ball of radius R. We consider the set

• f Young measures MR := L∞(0, T; P(BR)).

For η > 0, we define V η(θ) =          infµ∈U, y∈Y φ(yT , θ), s.t. Φ(yT, θ) ≤ 0, ˙ yt =

• BR

f (u, yt, θ)dµt(u), y0 = y 0, y − ¯ y∞ ≤ η. (Pη(θ)) We assume that for η small enough, ¯ u is a solution to Pη(0).

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

General approach

Two possible approaches for sensitivity analysis:

• ne based on the local stability of 1st-order optimality

conditions: adapted for weak solutions, requires uniqueness of multipliers the one we use is based on a variational analysis. We focus on the derivation of a second-order expansion of V . Two main steps: find an upper estimate, thanks to paths of feasible points. prove that it is also a lower estimate, thanks to an expansion

• f the Lagrangian.

The sets ΛL(¯ u) and ΛP(¯ u) are considered for θ = 0 and u ∈ BR. The constraints are supposed to be qualified (β = 1).

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

First-order upper estimate

A possible path is: ¯ u + θv, where v ∈ U2. The best v solves: inf

v∈U2 DJ (¯

u, 0)(v, 1), s.t. DG(¯ u, 0)(v, 1) ∈ TK(G(¯ u, 0)). (LP) Another possible path is: µ(θ) = (1 − θ)¯ u + θµ. They resp. lead to V η(θ) ≤ V η(0) +

• supλ∈ΛL(¯

u)DθL[λ](¯

u, θ)

• θ + o(θ),

V η(θ) ≤ V η(0) +

• supλ∈ΛP(¯

u)DθL[λ](¯

u, θ)

• θ + o(θ),

where DθL[λ](¯ u, θ) = DθΦ[Ψ](¯ yT , 0) + T DθH[pλ

t ](¯

ut, ¯ yt, 0)dt. The second estimate is better, but cannot be used at the second

• rder. Thus, we assume that

supλ∈ΛL(¯

u) DθL[λ](¯

u, 0) = supλ∈ΛP(¯

u) DθL[λ](¯

u, 0). (H) Let Λθ

P be the maximizers of the r.h.s. of the previous equality.

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Second-order upper estimate

The linearized problem can be extended to the space M2 of square integrable Young measures, that we call LP′. Let v ∈ S(LP′), we study a path of this form: (1 − θ2)(¯ u + θv) + θ2µ, where µ ∈ MR has to be optimized. This leads to the following upper second-order estimate of V η(θ) − V η(0):

• sup

λ∈ΛP

DθL[λ](¯ u, 0)

• θ + 1

2

• inf

v∈S(LP′)

sup

λ∈Λθ

P

Ωθ(v)

• θ2 + o(θ2).

where Ωθ(v) = D2

(u,θ),(u,θ)L[λ](¯

u, 0)(v, 1).

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Result

Theorem Assume that a qualification condition holds, a strong second-order sufficient condition in Pontryagin form (involving Λθ

P) holds,

the equality of linearized problems holds (H), then, there exists ¯ η > 0 such that ∀η ∈ (0, ¯ η], V η(θ) − V η(0) =

• sup

λ∈ΛP

DθL[λ](¯ u, 0)

• θ + 1

2

• inf

v∈S(LP′)

sup

λ∈Λθ

P

Ωθ(v)

• θ2 + o(θ2).

Proof of the lower estimate: an extension of the decomposition

• principle. Conclusion by contradiction.

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Bibliography

On sensitivity for weak solutions:

• K. Malanowski. Second-order conditions in stability analysis for

state-constrained optimal control. J. Global Analysis, 2008.

With a second-order state constraint:

• A. Hermant. Stability analysis of optimal control problems with a

second-order state constraint. SIAM J. on Optim., 2009.

Our results:

J.F. Bonnans, L. Pfeiffer, O.S. Serea. Sensitivity analysis for relaxed

• ptimal control problems with a final-state constraint. Nonlinear Analysis

T.M.A., 2013.

Open question: sensitivity analysis for strong solutions to state-constrained problems ? An application problem has been considered:

• K. Barty, J.F. Bonnans, L. Pfeiffer. Sensitivity analysis for the outages of

nuclear power plants. To appear in Energy Systems.

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

1 Introduction to sensitivity analysis of optimal control problems 2 Necessary conditions in Pontryagin form 3 Sufficient conditions in Pontryagin form 4 Sensitivity analysis 5 Chance-constrained stochastic optimal control

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Setting

Let ξ1,...,ξT be T i.i.d. random variables taking values in {1, ..., I} with probabilities p1,...,pI . Let t, let Ut be the set of adapted controls u = (ut, ..., uT−1) in a compact U: ∀s, us depends on ξt+1, ..., ξs. Let F : Rn → Rn, for all t, x ∈ Rn, u ∈ Ut, let X t,x,u be the solution to X t,x,u

t

= x, X t,x,u

s+1 = F(X t,x,u s

, us, ξs+1), ∀s ∈ {t, ..., T − 1}. Let φ, g : Rn → R, we consider the family of problems: Vt(x, z) = inf

u∈Ut E

• φ(X t,x,u

T

)

• , s.t. E
• g(X t,x,u

T

)

• ≥ z
• (⋆)

.

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Setting

Case of a probability constraint: if g(x) =

• 1

if h(x) ≥ 0

• therwise,

then E

• g(X t,x,u

T

)

• = P
• h(X t,x,u

T

)

• .

Proposition The constraint (⋆) holds if and only if there exists a martingale Z = (Zt, ..., ZT ) (called associated martingale) such that Zt = z and ZT ≤ g(X t,x,u

T

) a.s. If (⋆) is active, Z is the conditional expectation of g(X t,x,u

T

).

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Dynamic programming

Proposition The following dynamic programming equation holds: ∀t, x, z, Vt(x, z) = inf

u∈U, (zi)i∈RI s.t. I

i=1 pizi=z

I

i=1piVt+1(F(x, u, i), zi)

• ,

VT(x, z) =

• φ(x)

if z ≤ g(x), +∞

• therwise.

The boundary of the domain of V can be described as follows: Zt(x) = sup

z∈R

• z, Vt(x, z) < +∞
• = sup

u∈Ut

• E
• g(X t,x,u

T

)

• ,

thus, a dynamic programming principle is also available.

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Relaxation and convexity

We denote now by Ur

t the set of relaxed strategies, and by V r the

associated value function. Denote by ∂z the subdifferential w.r.t. z. Theorem For all t, x, V r

t (x, ·) is the convex enveloppe of Vt(x, ·).

Let u be an optimal control, Z and associated martingale and λ ∈ ∂zV r

t (x, z). Then, for all s ≥ t, λ ∈ ∂zV r s (X t,x,u s

, Zs).

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Lagrange relaxation

Let t, x ∈ Rn. By Fenchel-Moreau-Rockafellar theorem, V r

t (x, z) = sup λ≥0

• λz + inf

u∈Ut E

• φ(X t,x,u

T

) − λg(X t,x,u

T

)

• Problem D(λ)
• ,

since∀λ≥ 0,−Val(D(λ))=V ∗

t (x, λ),the Fencheltransform w.r.t. z.

Let u be an optimal solution to D(λ), let z ∈ E

• g(X t,x,u

T

)

• . Then,

u is a solution to the constrained problem with the level z. Moreover, z ∈ ∂λV ∗

t (x, λ).

We derive a method to compute V r: dichotomy (or sub-gradients methods) for the maximization w.r.t. to λ.

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Continuous-time

Setting: Ut is the set of adapted controls to a Brownian filtration. The state variable is a solution to the SDE: X t,x,u

t

= x, dXs = f (Xs, us)ds + σ(Xs, us)dWs, ∀s ≥ t. The family of problems is given by Vt(x, z) = inf

u∈Ut E

• φ(X t,x,u

T

)

• s.t. E
• g(Xt,x,u)
• ≥ z.

Theorem We assume that ∃L > 0 such that ∀x, y ∈ Rn, ∀u ∈ U, |f (x, u)| ≤ L(1 + |x|), |f (x, u) − f (y, u)| ≤ L|y − x|, and that the same holds for σ and φ. If g is Lipschitz, then Vt(x, z) is convex w.r.t. to z.

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Numerical results

We can compute: The boundary Zt(x) and V ∗

t (x, λ) with a semi-Lagrangian

scheme. We deduce V r

t (x, z) as follows: supλ∈Λ{λz − V ∗ t (x, λ)},

where Λ is a sampling of R+. Toy example: dXt = utdt + dWt, ut ∈ [0, 1], inf

u∈Ut E

T

t

u2

s ds

• , s.t. P
• X t,x,u

T

≥ 0

• ≥ z.

Remark A direct approach with the martingale is possible but delicate. Curse of dimensionality.

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Graphs

Figure: Maximum of probability

Time steps: 20; T = 10 Discretized control space: {0, 1/5, ...1} Number of space steps: 40, state space [-20,20] Probability steps: 40, Λ = {0, 1, ..., 100}.

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Graphs

Figure: Graph of the (relaxed) value function

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Bibliography

On a subgradient method for the dual approach:

• L. Andrieu, G. Cohen, F.J. V´

asquez-Abad. Gradient-based simulation

• ptimization under probability constraints. EJOR, 2011.

On the HJB equation for problems with a target constraint:

• B. Bouchard, R. Elie, C. Imbert. Optimal control under stochastic target
• constraints. SICON, 2010.

On stochastic target problems:

• N. Touzi. Optimal Stochastic control, Stochastic Target Problems, and

Backward SDE. Fields Institute Monographs, 2012.

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Bibliography

On the HJB equation (+ numerical scheme) for a close problem:

• O. Bokanowski, B. Bruder, S. Maroso, H. Zidani. Numerical

approximation for a superreplication problem under γ-constraints. SICON, 2010.

Open questions: Convexity of the value function in a general setting ? Convergence of the numerical scheme ?

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control

Thank you for your attention!