Nonlinear Programming Formulation of Chance-Constraints Andreas - - PowerPoint PPT Presentation

Nonlinear Programming Formulation of Chance-Constraints

Andreas Wächter joint with Alejandra Peña-Ordieres and Jim Luedtke

Department of Industrial Engineering and Management Sciences Northwestern University Evanston, IL, USA andreas.waechter@northwestern.edu

US & Mexico Workshop on Optimization and Its Applications Huatulco, Mexico January 9, 2018

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Problem Statement

min

x∈X f (x)

s.t. Pξ[c(x, ξ) ≤ 0] ≥ 1 − α f (x) : Rn → R c(x, ξ) : Rn × Ξ → Rm

◮ Random variable ξ with support Ξ ◮ Assume f (x) and c(x, ξ) are sufficiently smooth for all ξ ∈ Ξ ◮ X ⊆ Rn: Captures additional constraints

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Problem Statement

min

x∈X f (x)

s.t. Pξ[c(x, ξ) ≤ 0] ≥ 1 − α f (x) : Rn → R c(x, ξ) : Rn × Ξ → Rm

◮ Random variable ξ with support Ξ ◮ Assume f (x) and c(x, ξ) are sufficiently smooth for all ξ ∈ Ξ ◮ X ⊆ Rn: Captures additional constraints ◮ Goal: Formulate as continuous NLP

◮ Do not want to assume particular probability distribution ◮ Do not want to assume convexity (or convex approximations) ◮ Want to avoid combinatorial approach ◮ Use powerful NLP algorithms and techniques Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Constraint in Probability Space

Impose p(x) = P[c(x, ξ) ≤ 0] ≥ 1 − α

• 2.5
• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2 2.5

• 0.5

0.5 1 1.5 (

• 1) - p(x)

y = 0

c(x, ξ) = x2 − 2 + ξ ξ ∼ N(0, 1) Issues

◮ Linearization is poor approximation ◮ Always nonconvex ◮ Used in [Hu, Hong, Zhang 13], [Bremer, Henrion, Möller 15]

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Quantile Formulation

• 3
• 2
• 1

1 2 3

x

0.2 0.4 0.6 0.8 1

Cumulative Distribution Function Nomal CDF

◮ Let Y be a real-valued random variable ◮ (1 − α)-quantile:

QY

1−α = inf{y ∈ R : P[Y ≤ y] ≥ 1 − α}

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Quantile Formulation

• 3
• 2
• 1

1 2 3

x

0.2 0.4 0.6 0.8 1

Cumulative Distribution Function Nomal CDF

◮ Let Y be a real-valued random variable (Y = c(x, ξ)) ◮ (1 − α)-quantile:

Qc(x,ξ)

1−α

= inf{y ∈ R : P[c(x, ξ) ≤ y] ≥ 1 − α}

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Quantile Formulation

• 3
• 2
• 1

1 2 3

x

0.2 0.4 0.6 0.8 1

Cumulative Distribution Function Nomal CDF

◮ Let Y be a real-valued random variable (Y = c(x, ξ)) ◮ (1 − α)-quantile:

Qc(x,ξ)

1−α

= inf{y ∈ R : P[c(x, ξ) ≤ y] ≥ 1 − α} p(x) ≥ 1 − α ⇐ ⇒ q1−α(x) = Qc(x,ξ)

1−α

≤ 0

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Choice of Formulation

p(x) ≥ 1 − α ⇐ ⇒ q1−α(x) ≤ 0

• 2.5
• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2 2.5

• 0.5

0.5 1 1.5 (

• 1) - p(x)

q 1 - (x) y = 0

◮ Quantile formulation more suitable for NLP solver

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Sample Average Approximation (SAA)

min

x∈X f (x)

s.t. Pξ[c(x, ξ) ≤ 0] ≥ 1 − α f (x) : Rn → R c(x, ξ) : Rn × Ξ → R

◮ Finite scenario set

ΞN = { ξ1, . . . , ξN} with ξi ∈ Ξ chosen i.i.d.

◮ For simplicity: ci(x) = c(x,

ξi)

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Sample Average Approximation (SAA)

min

x∈X f (x)

s.t. Pξ[c(x, ξ) ≤ 0] ≥ 1 − α f (x) : Rn → R c(x, ξ) : Rn × Ξ → R

◮ Finite scenario set

ΞN = { ξ1, . . . , ξN} with ξi ∈ Ξ chosen i.i.d.

◮ For simplicity: ci(x) = c(x,

ξi) SAA approximation of quantile

◮ Ensure that M = ⌈(1 − α)N⌉ constraints are satisfied

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Sample Average Approximation (SAA)

min

x∈X f (x)

s.t. Pξ[c(x, ξ) ≤ 0] ≥ 1 − α f (x) : Rn → R c(x, ξ) : Rn × Ξ → R

◮ Finite scenario set

ΞN = { ξ1, . . . , ξN} with ξi ∈ Ξ chosen i.i.d.

◮ For simplicity: ci(x) = c(x,

ξi) SAA approximation of quantile

◮ Ensure that M = ⌈(1 − α)N⌉ constraints are satisfied ◮ Scenario vector:

C(x) = (c1(x), . . . , cN(x))

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Sample Average Approximation (SAA)

min

x∈X f (x)

s.t. Pξ[c(x, ξ) ≤ 0] ≥ 1 − α f (x) : Rn → R c(x, ξ) : Rn × Ξ → R

◮ Finite scenario set

ΞN = { ξ1, . . . , ξN} with ξi ∈ Ξ chosen i.i.d.

◮ For simplicity: ci(x) = c(x,

ξi) SAA approximation of quantile

◮ Ensure that M = ⌈(1 − α)N⌉ constraints are satisfied ◮ Scenario vector:

C(x) = (c1(x), . . . , cN(x))

◮ Order constraint values: c[1](x) ≤ c[2](x) ≤ . . . ≤ c[N](x)

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Sample Average Approximation (SAA)

min

x∈X f (x)

s.t. Pξ[c(x, ξ) ≤ 0] ≥ 1 − α f (x) : Rn → R c(x, ξ) : Rn × Ξ → R

◮ Finite scenario set

ΞN = { ξ1, . . . , ξN} with ξi ∈ Ξ chosen i.i.d.

◮ For simplicity: ci(x) = c(x,

ξi) SAA approximation of quantile

◮ Ensure that M = ⌈(1 − α)N⌉ constraints are satisfied ◮ Scenario vector:

C(x) = (c1(x), . . . , cN(x))

◮ Order constraint values: c[1](x) ≤ c[2](x) ≤ . . . ≤ c[N](x) ◮ Empirical quantile:

• q1−α(C(x)) = c[M](x) ≤ 0

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Empirical Quantile

◮ Feasible region for c(x, ξ) = ξ1x1 + ξ2x2 − 1

ξ1, ξ1 ∼ N(0, 1)

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.1 0.2 0.3 0.4 0.5 0.6 0.7

N = 200

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.1 0.2 0.3 0.4 0.5 0.6 0.7

N = 500

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.1 0.2 0.3 0.4 0.5 0.6 0.7

N = 2000

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.1 0.2 0.3 0.4 0.5 0.6 0.7

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Empirical Quantile

◮ Feasible region for c(x, ξ) = ξ1x1 + ξ2x2 − 1

ξ1, ξ1 ∼ N(0, 1)

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.1 0.2 0.3 0.4 0.5 0.6 0.7

N = 200

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.1 0.2 0.3 0.4 0.5 0.6 0.7

N = 500

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.1 0.2 0.3 0.4 0.5 0.6 0.7

N = 2000 Observations:

◮ Approximation improves as N increases ◮ Rough boundary of feasible region results in spurious local minima ◮ A lot of variance for small N as ξi are resampled ◮

q1−α(C(x)) is non-differentiable

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Variance Reduction using Kernels

0.9 0.91 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 1 0.5 1 1.5 2

◮ Y[1], . . . , Y[N] ordered realizations of a random variable (c(x, ξi)) ◮ Empirical quantiles: For p ∈ [0, 1] define

• QY

p = Y[j], where j ∈ {1, .., N} with j−1 N < p ≤ j N

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Variance Reduction using Kernels

0.9 0.91 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 1 0.5 1 1.5 2

◮ Y[1], . . . , Y[N] ordered realizations of a random variable (c(x, ξi)) ◮ Empirical quantiles: For p ∈ [0, 1] define

• QY

p = Y[j], where j ∈ {1, .., N} with j−1 N < p ≤ j N ◮ Kernel smoothing [Parson 79]

QY ,N,h

1−α

=

1

• QY

p 1 hK

• α−p

h

• dp

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Variance Reduction using Kernels

0.9 0.91 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 1 0.5 1 1.5 2

◮ Y[1], . . . , Y[N] ordered realizations of a random variable (c(x, ξi)) ◮ Empirical quantiles: For p ∈ [0, 1] define

• QY

p = Y[j], where j ∈ {1, .., N} with j−1 N < p ≤ j N ◮ Kernel smoothing [Parson 79]

QY ,N,h

1−α

=

1

• QY

p 1 hK

• α−p

h

• dp =

N

• j=1

Y[j]

=:wN,h

j

• j

N j−1 N

1 hK

• α−p

h

• dp

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Kernel Estimation of Quantiles

◮ Approximate q1−α(C(x)) = QY 1−α with Yi = c(x; ξi) by kernel

estimate qN,h

1−α(C(x)) := N

• i=1

wN,h

i

c[i](x) ≤ 0

◮ The weights depend only on N and h and can be precomputed

✶

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Kernel Estimation of Quantiles

◮ Approximate q1−α(C(x)) = QY 1−α with Yi = c(x; ξi) by kernel

estimate qN,h

1−α(C(x)) := N

• i=1

wN,h

i

c[i](x) ≤ 0

◮ The weights depend only on N and h and can be precomputed ◮ Epanechnikov kernel: K(u) = 3 4(1 − u2)✶|u|<1

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Feasible Region of Kernel-Quantile Formulation

c(x, ξ) = ξ1x1 + ξ2x2 − 1 ξ1, ξ1 ∼ N(0, 1)

h ∈ {0.05, 0.01}

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.1 0.2 0.3 0.4 0.5 0.6 0.7

N = 200

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.1 0.2 0.3 0.4 0.5 0.6 0.7

N = 500

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.1 0.2 0.3 0.4 0.5 0.6 0.7

N = 2000

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.1 0.2 0.3 0.4 0.5 0.6 0.7

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Feasible Region of Kernel-Quantile Formulation

c(x, ξ) = ξ1x1 + ξ2x2 − 1 ξ1, ξ1 ∼ N(0, 1)

h ∈ {0.05, 0.01}

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.1 0.2 0.3 0.4 0.5 0.6 0.7

N = 200

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.1 0.2 0.3 0.4 0.5 0.6 0.7

N = 500

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.1 0.2 0.3 0.4 0.5 0.6 0.7

N = 2000 Observations:

◮ “Looks” smoother and convex ◮ Reduces variance as ξi are resampled ◮ Large h creates bias ◮ qN,h 1−α(C(x)) has more points of non-differentiability ◮ Usually “solved” well with Knitro, but no proper termination

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Smoothed Empirical CDF

• 3
• 2
• 1

1 2 3

x

0.2 0.4 0.6 0.8 1

Cumulative Distribution Function Nomal CDF

◮ Quantile QY p = inf{y ∈ R : Φ(y) ≥ p} where Φ(y) is cdf of Y

✶

✶

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Smoothed Empirical CDF

• 2
• 1.5
• 1
• 0.5

c(x, )

0.2 0.4 0.6 0.8 1

Cumulative Distribution Function Smoothing of empirical CDF Empirical CDF

◮ Quantile QY p = inf{y ∈ R : Φ(y) ≥ p} where Φ(y) is cdf of Y ◮ Y1, . . . , YN (unordered) realizations of a random variable ◮ Empirical cdf:

• Φ(y) = 1

N

N

• i=1

✶≥0(y − Yi)

✶

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Smoothed Empirical CDF

• 2
• 1.5
• 1
• 0.5

c(x, )

0.2 0.4 0.6 0.8 1

Cumulative Distribution Function Smoothing of empirical CDF Empirical CDF epsilon = 0.1 epsilon = 0.5

◮ Quantile QY p = inf{y ∈ R : Φ(y) ≥ p} where Φ(y) is cdf of Y ◮ Y1, . . . , YN (unordered) realizations of a random variable ◮ Empirical cdf:

• Φ(y) = 1

N

N

• i=1

✶≥0(y − Yi)

◮ Smoothed cdf [Azzalini 81]:

(Kǫ(t) ≈ ✶≥0(t) differentiable)

ΦN,ǫ(y) = 1 N

N

• i=1

Kǫ(y − Yi)

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Differentiable Quantile Estimates

• 2
• 1.5
• 1
• 0.5

c(x, )

0.2 0.4 0.6 0.8 1

Cumulative Distribution Function Smoothing of empirical CDF Empirical CDF epsilon = 0.1 epsilon = 0.5

◮ Yi = c(x, ξi) ◮ Smoothed cdf: ΦN,ǫ(y) = 1 N

N

i=1 Kǫ(y − ci(x)) ◮ Resulting quantile estimate

qN,ǫ

p

(C(x)) is the solution y∗ of p = ΦN,ǫ(y)

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Differentiable Quantile Estimates

• 2
• 1.5
• 1
• 0.5

c(x, )

0.2 0.4 0.6 0.8 1

Cumulative Distribution Function Smoothing of empirical CDF Empirical CDF epsilon = 0.1 epsilon = 0.5

◮ Yi = c(x, ξi) ◮ Smoothed cdf: ΦN,ǫ(y) = 1 N

N

i=1 Kǫ(y − ci(x)) ◮ Resulting quantile estimate

qN,ǫ

p

(C(x)) is the solution y∗ of p = 1 N

N

• i=1

Kǫ(y − ci(x))

◮ Implicit function theorem guarantees that

qN,ǫ

p

(C(x)) exists and is differentiable

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Differentiable Quantile Estimates

• 2
• 1.5
• 1
• 0.5

c(x, )

0.2 0.4 0.6 0.8 1

Cumulative Distribution Function Smoothing of empirical CDF Empirical CDF epsilon = 0.1 epsilon = 0.5

◮ Yi = c(x, ξi) ◮ Smoothed cdf: ΦN,ǫ(y) = 1 N

N

i=1 Kǫ(y − ci(x)) ◮ Resulting quantile estimate

qN,ǫ

p

(C(x)) is the solution y∗ of p = 1 N

N

• i=1

Kǫ(y − ci(x)) + 1 2N

◮ Implicit function theorem guarantees that

qN,ǫ

p

(C(x)) exists and is differentiable (for p ∈ {0, 1

N , 2 N , . . . , N−1 N })

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Final Estimate

• 1. Kernel estimate using empirical quantiles c[i](x)

qN,h

1−α(C(x)) = N

• i=1

wN,h

i

c[i](x)

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Final Estimate

• 1. Kernel estimate using empirical quantiles c[i](x)

qN,h

1−α(C(x)) = N

• i=1

wN,h

i

c[i](x)

• 2. Smoothed empirical quantile

qN,ǫ

p

(C(x)), solution of p = 1 N

N

• i=1

Kǫ(y − ci(x)) + 1 2N

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Final Estimate

• 1. Kernel estimate using empirical quantiles c[i](x)

qN,h

1−α(C(x)) = N

• i=1

wN,h

i

c[i](x)

• 2. Smoothed empirical quantile

qN,ǫ

p

(C(x)), solution of p = 1 N

N

• i=1

Kǫ(y − ci(x)) + 1 2N

• 3. Final estimate

qN,h,ǫ

1−α (C(x)) := N

• i=1

wN,h

i

• qN,ǫ

i N (C(x)) Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Feasible Region with Smoothing (N = 100)

Smoothing with =1, 0.5, 0.1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.1 0.2 0.3 0.4 0.5 0.6 0.7 Kernel h=0.01 with smoothing = 0.06 0.1 0.2 0.3 0.4 0.5 0.6 0.7 x 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 x 2 Smoothing with =1, 0.5, 0.1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 x 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 x 2 Kernel h=0.01 with smoothing = 0.06 0.1 0.2 0.3 0.4 0.5 0.6 0.7 x 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 x 2

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Example: Portfolio Optimization

max

x∈Rn E[r Tx] = µTx

s.t. Pr[r Tx ≥ −0.05] ≥ 0.95

n

• i=1

xi = 1, x ≥ 0

◮ r ∼ N(µ, Σ), fixed µ ∈ R100 and Σ ∈ R100×100 pos. def. ◮ 100 replications

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Example: Portfolio Optimization

max

x∈Rn E[r Tx] = µTx

s.t. Pr[r Tx ≥ −0.05] ≥ 0.95

n

• i=1

xi = 1, x ≥ 0

◮ r ∼ N(µ, Σ), fixed µ ∈ R100 and Σ ∈ R100×100 pos. def. ◮ 100 replications

Iterations with Knitro N h min iter mean iter max iter 500 0.05 15 26.7 54 10,000 0.005 17 29.3 45

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Portfolio Optimization

100 500 1000 5000 10000

Number of samples

0.89 0.9 0.91 0.92 0.93 0.94 0.95 0.96 0.97

P(c(x, )) h = 5.00e-02

100 500 1000 5000 10000

Number of samples

• 3
• 2
• 1

1 2 3

Relative function error (%)

10 -3

h = 5.00e-02

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Portfolio Optimization

100 500 1000 5000 10000

Number of samples

0.87 0.88 0.89 0.9 0.91 0.92 0.93 0.94 0.95 0.96 0.97

P(c(x, )) h = 1.00e-02

100 500 1000 5000 10000

Number of samples

• 5
• 4
• 3
• 2
• 1

1 2 3

Relative function error (%)

10 -3

h = 1.00e-02

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Portfolio Optimization

100 500 1000 5000 10000

Number of samples

0.87 0.88 0.89 0.9 0.91 0.92 0.93 0.94 0.95 0.96 0.97

P(c(x, )) h = 1.00e-03

100 500 1000 5000 10000

Number of samples

• 3
• 2
• 1

1 2 3

Relative function error (%)

10 -3

h = 1.00e-03

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Portfolio Optimization

100 500 1000 5000 10000

Number of samples

0.87 0.88 0.89 0.9 0.91 0.92 0.93 0.94 0.95 0.96 0.97

P(c(x, )) h = 1.00e-04

100 500 1000 5000 10000

Number of samples

• 3
• 2
• 1

1 2 3

Relative function error (%)

10 -3

h = 1.00e-04

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Joint Chance Constraints

◮ Original Problem

min

x∈X f (x)

s.t. Pξ[c(x, ξ) ≤ 0] ≥ 1 − α

◮ Scenario vector

C(x) = (c(x, ξ1), . . . , c(x, ξN))

◮ NLP formulation

min

x∈X f (x)

s.t. qN,h,ǫ

1−α (C(x)) ≤ 0

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Joint Chance Constraints

◮ Original Problem

min

x∈X f (x)

s.t. Pξ[cj(x, ξ) ≤ 0, j = 1, . . . , m] ≥ 1 − α

◮ Scenario vector

C(x) = (c(x, ξ1), . . . , c(x, ξN))

◮ NLP formulation

min

x∈X f (x)

s.t. qN,h,ǫ

1−α (C(x)) ≤ 0

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Joint Chance Constraints

◮ Original Problem

min

x∈X f (x)

s.t. Pξ[ c(x, ξ) ≤ 0] ≥ 1 − α

• c(x, ξ) := maxj cj(x, ξi)

◮ Scenario vector

C(x) = (c(x, ξ1), . . . , c(x, ξN))

◮ NLP formulation

min

x∈X f (x)

s.t. qN,h,ǫ

1−α (C(x)) ≤ 0

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Joint Chance Constraints

◮ Original Problem

min

x∈X f (x)

s.t. Pξ[ c(x, ξ) ≤ 0] ≥ 1 − α

• c(x, ξ) := maxj cj(x, ξi)

◮ Scenario vector

• C(x) = (

c(x, ξ1), . . . , c(x, ξN))

◮ NLP formulation

min

x∈X f (x)

s.t. qN,h,ǫ

1−α (

C(x)) ≤ 0 Note: Constraint no longer differentiable

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Joint Chance Constraints

◮ Original Problem

min

x∈X f (x)

s.t. Pξ[ c(x, ξ) ≤ 0] ≥ 1 − α

• c(x, ξ) := maxj cj(x, ξi)

◮ Scenario vector

• C(x) = (

c(x, ξ1), . . . , c(x, ξN))

◮ NLP formulation

min

x∈X f (x)

s.t. q( C(x)) ≤ 0 q( C(x)) := qN,h,ǫ

1−α (

C(x)) Note: Constraint no longer differentiable

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Exact Penalty Function

min

x∈X f (x)

s.t. q( C(x)) ≤ 0

◮ Exact penalty function:

φρ(x) = f (x) + ρ max{q( C(x)), 0} (+penalty for X)

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Exact Penalty Function

min

x∈X f (x)

s.t. q( C(x)) ≤ 0

◮ Exact penalty function:

φρ(x) = f (x) + ρ max{q( C(x)), 0} (+penalty for X)

◮ Model of scenario vector

C(x) = (. . . , maxj cj(x, ξi), . . .) m

C(xk, d) =

• . . . , max

j

{cj(xk, ξi) + ∇cj(xk, ξi)Td}, . . .

• Andreas Wächter

Nonlinear Programming Formulation of Chance-Constraints

Exact Penalty Function

min

x∈X f (x)

s.t. q( C(x)) ≤ 0

◮ Exact penalty function:

φρ(x) = f (x) + ρ max{q( C(x)), 0} (+penalty for X)

◮ Model of scenario vector

C(x) = (. . . , maxj cj(x, ξi), . . .) m

C(xk, d) =

• . . . , max

j

{cj(xk, ξi) + ∇cj(xk, ξi)Td}, . . .

• ◮ Model of penalty function

mφ(xk, d) =f (xk) + ∇f (xk)Td + 1

2dTBkd+

ρ max{q( C(xk)) + ∇q( C(xk))T(m

C(xk, d) −

C(xk)), 0}

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Trust Region Sℓ1QP Algorithm

mφ(xk, d) =f (xk) + ∇f (xk)Td + 1

2dTBkd+

ρ max{q( C(xk)) + ∇q( C(xk))T(m

C(xk, d) −

C(xk)), 0}

◮ Can show: |φρ(xk) − mφ(xk, d)| = O(d2) ◮ Standard Sℓ1QP-type trust region algorithm can be applied

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Trust Region Sℓ1QP Algorithm

mφ(xk, d) =f (xk) + ∇f (xk)Td + 1

2dTBkd+

ρ max{q( C(xk)) + ∇q( C(xk))T(m

C(xk, d) −

C(xk)), 0}

◮ Can show: |φρ(xk) − mφ(xk, d)| = O(d2) ◮ Standard Sℓ1QP-type trust region algorithm can be applied ◮ QP Subproblem

min

d,z,t f (xk) + ∇f (xk)Td + 1 2dTBkd + ρ t

s.t. q( C(xk)) + ∇q( C(xk))TZ ≤ t [Z = (z1, . . . , zn)] cj(xk, ξi) + ∇cj(xk, ξi)Td ≤ zi for all i, j t ≥ 0, xk + d ∈ ˜ X, d∞ ≤ ∆k

◮ Size of QP proportional to number of nonzeros in ∇q(

C(xk)), not N

Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints

Summary

◮ Goal: Formulate chance-constrained problem as NLP ◮ Constrain the quantile q1−α(x) ≤ 0

◮ Better linearization than p(x) ≥ (1 − α)

◮ Sample-based empirical quantile

◮ No need to know probability distribution

◮ Variance reduction using kernel

◮ Improves approximation of feasible region, “smoother boundary”

◮ Quantile estimates using smoothed empirical cdf

◮ Constraint becomes differentiable

◮ Joint chance constraints

◮ Exact merit function ◮ Sℓ1QP-type trust region algorithm Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints