Dualit sans contraintes en optimisation globale Simon Boulmier - - PowerPoint PPT Presentation
Dualit sans contraintes en optimisation globale Simon Boulmier www.localsolver.com 1 | 28 Global optimization and lower bounds Mixed-integer nonlinear problem : Dual search Primal search v v f 2 | 28 min s.c. x R n f ( x ) l
Simon Boulmier
www.localsolver.com
1 | 28
Mixed-integer nonlinear problem : min
x∈Rn f(x)
s.c. l ≤ x ≤ u, g(x) = 0, h(x) ≤ 0, xi ∈ Z, i ∈ I. f v∗ v v Primal search Dual search
2 | 28
I - Reformulation II - Bound tightening techniques III - Convex relaxations IV - Branch-and-bound
I - Solving convex relaxations II - Unconstrained Wolfe duality
3 | 28
4 | 28
h R r
max
x∈R3
π 3 x3 ( x2
1 + x1x2 + x2 2
) s.c. 0 ≤ x1 ≤ 1 0 ≤ x2 ≤ 1 0 ≤ x3 ≤ 1 x2
1 + (x1 + x2)
√ (x1 − x2)2 + x2
3 ≤ 1
5 | 28
w 6 | 28
7 | 28
8 | 28
9 | 28
y
x z
10 | 28
x y lx ux uy ly x y lx ux uy ly x y lx ux uy ly x y lx ux uy ly
11 | 28
12 | 28
13 | 28
14 | 28
15 | 28
y
x z
16 | 28
y
x z
16 | 28
Reformulation Convex relaxations Bound tightening Branch-and-bound
17 | 28
18 | 28
◮ Smooth convex problem : min { f(x) | x ∈ Rn, g(x) ≤ 0 ∈ Rm} ◮ KKT certifjcate depends on L(x, µ) = f(x) + µTg(x) : Condition Exact Approximate Stationarity
x
x
x
x
statio
Feasibility gi x gi x
feas
Complementarity
igi x igi x compl
19 | 28
◮ Smooth convex problem : min { f(x) | x ∈ Rn, g(x) ≤ 0 ∈ Rm} ◮ KKT certifjcate depends on L(x, µ) = f(x) + µTg(x) : Condition Exact Approximate Stationarity ∇xL(x, µ) = 0
x
x
statio
Feasibility gi(x) ≤ 0 gi x
feas
Complementarity µigi(x) = 0
igi x compl
19 | 28
◮ Smooth convex problem : min { f(x) | x ∈ Rn, g(x) ≤ 0 ∈ Rm} ◮ KKT certifjcate depends on L(x, µ) = f(x) + µTg(x) : Condition Exact Approximate Stationarity ∇xL(x, µ) = 0 ∥∇xL(x, µ)∥ ≤ ϵstatio Feasibility gi(x) ≤ 0 gi(x) ≤ ϵfeas Complementarity µigi(x) = 0 |µigi(x)| ≤ ϵcompl
19 | 28
◮ Smooth convex problem : min { f(x) | x ∈ Rn, g(x) ≤ 0 ∈ Rm} ◮ KKT certifjcate depends on L(x, µ) = f(x) + µTg(x) : Condition Exact Approximate Stationarity ∇xL(x, µ) = 0 ∥∇xL(x, µ)∥ ≤ ϵstatio Feasibility gi(x) ≤ 0 gi(x) ≤ ϵfeas Complementarity µigi(x) = 0 |µigi(x)| ≤ ϵcompl
19 | 28
20 | 28
20 | 28
20 | 28
20 | 28
Theorem
The Wolfe dual of the convex differentiable problem (P) min
x∈Rn f(x)
s.c. { ℓ ≤ x ≤ u is the unconstrained maximisation problem D max
x
x wih the dual function : x f x
i i
xi
if x
ui xi
if x
f x x T f x u x T f x
21 | 28
Theorem
The Wolfe dual of the convex differentiable problem (P) min
x∈Rn f(x)
s.c. { ℓ ≤ x ≤ u is the unconstrained maximisation problem (D) max
x
θ(x) wih the dual function : x f x
i i
xi
if x
ui xi
if x
f x x T f x u x T f x
21 | 28
Theorem
The Wolfe dual of the convex differentiable problem (P) min
x∈Rn f(x)
s.c. { ℓ ≤ x ≤ u is the unconstrained maximisation problem (D) max
x
θ(x) wih the dual function : θ(x) = f(x) + ∑
i
(ℓi − xi)∇if(x)+ + (ui − xi)∇if(x)− = f(x) + (ℓ − x)T∇f(x)+ + (u − x)T∇f(x)−
21 | 28
22 | 28
Theorem
The Wolfe dual of the convex differentiable problem (P) min
x∈Rn f(x)
s.c. { ℓ ≤ x ≤ u, g(x) ≤ 0 is the bound constrained maximisation problem D max
x
x wih the dual function : x x x T x u x T x
23 | 28
Theorem
The Wolfe dual of the convex differentiable problem (P) min
x∈Rn f(x)
s.c. { ℓ ≤ x ≤ u, g(x) ≤ 0 is the bound constrained maximisation problem (D) max
x,µ≥0 θ(x, µ)
wih the dual function : x x x T x u x T x
23 | 28
Theorem
The Wolfe dual of the convex differentiable problem (P) min
x∈Rn f(x)
s.c. { ℓ ≤ x ≤ u, g(x) ≤ 0 is the bound constrained maximisation problem (D) max
x,µ≥0 θ(x, µ)
wih the dual function : θ(x, µ) = L(x, µ) + (ℓ − x)T∇L(x, µ)+ + (u − x)T∇L(x, µ)−
23 | 28
◮ Robustness of dual bounds and inconsistency certifjcates Performance of some bound tightening techniques Solve min f x g x with augmented lagrangian
Sequence of box-constrained subproblems min x x u Approximate KKT : not enough, also use Defjne min x xk and max x xk % solver fails, % function evaluations, % newton systems solved
24 | 28
◮ Robustness of dual bounds and inconsistency certifjcates ◮ Performance of some bound tightening techniques Solve min f x g x with augmented lagrangian
Sequence of box-constrained subproblems min x x u Approximate KKT : not enough, also use Defjne min x xk and max x xk % solver fails, % function evaluations, % newton systems solved
24 | 28
◮ Robustness of dual bounds and inconsistency certifjcates ◮ Performance of some bound tightening techniques ◮ Solve min { f(x) | g(x) ≤ 0 } with augmented lagrangian
Sequence of box-constrained subproblems min x x u Approximate KKT : not enough, also use Defjne min x xk and max x xk % solver fails, % function evaluations, % newton systems solved
24 | 28
◮ Robustness of dual bounds and inconsistency certifjcates ◮ Performance of some bound tightening techniques ◮ Solve min { f(x) | g(x) ≤ 0 } with augmented lagrangian
◮ Sequence of box-constrained subproblems min { Lρ(x) | ℓ ≤ x ≤ u } Approximate KKT : not enough, also use Defjne min x xk and max x xk % solver fails, % function evaluations, % newton systems solved
24 | 28
◮ Robustness of dual bounds and inconsistency certifjcates ◮ Performance of some bound tightening techniques ◮ Solve min { f(x) | g(x) ≤ 0 } with augmented lagrangian
◮ Sequence of box-constrained subproblems min { Lρ(x) | ℓ ≤ x ≤ u } ◮ Approximate KKT : not enough, also use Lρ − Lρ ≤ ϵ Defjne min x xk and max x xk % solver fails, % function evaluations, % newton systems solved
24 | 28
◮ Robustness of dual bounds and inconsistency certifjcates ◮ Performance of some bound tightening techniques ◮ Solve min { f(x) | g(x) ≤ 0 } with augmented lagrangian
◮ Sequence of box-constrained subproblems min { Lρ(x) | ℓ ≤ x ≤ u } ◮ Approximate KKT : not enough, also use Lρ − Lρ ≤ ϵ ◮ Defjne Lρ = min (Lρ(x1), · · · , Lρ(xk)) and Lρ = max (θ(x1), · · · , θ(xk)) % solver fails, % function evaluations, % newton systems solved
24 | 28
◮ Robustness of dual bounds and inconsistency certifjcates ◮ Performance of some bound tightening techniques ◮ Solve min { f(x) | g(x) ≤ 0 } with augmented lagrangian
◮ Sequence of box-constrained subproblems min { Lρ(x) | ℓ ≤ x ≤ u } ◮ Approximate KKT : not enough, also use Lρ − Lρ ≤ ϵ ◮ Defjne Lρ = min (Lρ(x1), · · · , Lρ(xk)) and Lρ = max (θ(x1), · · · , θ(xk)) ◮ −68% solver fails, −30% function evaluations, −15% newton systems solved
24 | 28
25 | 28
A global optimization method, αbb, for general twice-differentiable constrained nlps—i. theoretical advances. Computers & Chemical Engineering, 22(9):1137–1158, 1998.
Branching and bounds tightening techniques for non-convex minlp. Optimization Methods & Software, 24(4-5):597–634, 2009.
Improving the performance of standard solvers for quadratic 0-1 programs by a tight convex reformulation: The qcr method. Discrete Applied Mathematics, 157(6):1185–1197, 2009.
Convex envelopes of monomials of odd degree. Journal of Global Optimization, 25(2):157–168, 2003.
Safe bounds in linear and mixed-integer linear programming. Mathematical Programming, 99(2):283–296, 2004.
A branch-and-reduce approach to global optimization. Journal of global optimization, 8(2):107–138, 1996.
A symbolic reformulation/spatial branch-and-bound algorithm for the global optimisation of nonconvex minlps. Computers & Chemical Engineering, 23(4-5):457–478, 1999. 26 | 28
Convexifjcation and global optimization in continuous and mixed-integer nonlinear programming: theory, algorithms, software, and applications, volume 65. Springer Science & Business Media, 2002.
A duality theorem for non-linear programming. Quarterly of applied mathematics, 19(3):239–244, 1961. 27 | 28
◮ Smooth convex problem : min { f(x) | x ∈ Rn, g(x) ≤ 0 ∈ Rm} ◮ KKT certifjcate depends on L(x, µ) = f(x) + µTg(x) : Condition Exact Approximate Stationarity ∇xL(x, µ) = 0 ∥∇xL(x, µ)∥ ≤ ϵstatio Feasibility gi(x) ≤ 0 gi(x) ≤ ϵfeas Complementarity µigi(x) = 0 |µigi(x)| ≤ ϵcompl Primal Lagrangian dual Wolfe dual [9] min
x
f x s.c. g x max min
x
f x
Tg x
max
x
f x
Tg x
s.c. f x
i i
gi x
28 | 28
◮ Smooth convex problem : min { f(x) | x ∈ Rn, g(x) ≤ 0 ∈ Rm} ◮ KKT certifjcate depends on L(x, µ) = f(x) + µTg(x) : Condition Exact Approximate Stationarity ∇xL(x, µ) = 0 ∥∇xL(x, µ)∥ ≤ ϵstatio Feasibility gi(x) ≤ 0 gi(x) ≤ ϵfeas Complementarity µigi(x) = 0 |µigi(x)| ≤ ϵcompl Primal Lagrangian dual Wolfe dual [9] min
x
f(x) s.c. { g(x) ≤ 0 max
µ≥0 min x
f(x)+µTg(x) max
x,µ≥0 f(x) + µTg(x)
s.c. { ∇f(x) + ∑
i µi∇gi(x) = 0
28 | 28