Inner Regions and Interval Linearizations for Global Optimization - - PowerPoint PPT Presentation

Background Main contribution New framework for constrained global optimization Experimental evaluation

Inner Regions and Interval Linearizations for Global Optimization

• G. Trombettoni, I. Araya, B. Neveu, G. Chabert

INRIA, I3S, Univ. Nice-Sophia (France); UTFSM (Chile); Imagine, LIGM, Univ. Paris-Est (France); LINA, EMN (France) SWIM, Bourges, June 14th, 2011

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

Plan

1

Background

2

Main contribution

3

New framework for constrained global optimization

4

Experimental evaluation

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

Plan

1

Background

2

Main contribution

3

New framework for constrained global optimization

4

Experimental evaluation

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

Handled Problem

Reliable continuous constrained global optimization: Constrained optimization: argminx∈[x]⊂Rnf(x) s.t. g(x) ≤ 0 ∧ h(x) = 0 Reliability: the best solution is guaranteed with a bounded error

• n the cost.

Constraints Objective function

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

Example (Coconut: ex 6 1 1)

Variables x2, x3, x4, x5 in [1e-7,0.5]; x6 in [0,0.901]; x7 in [0,0.274]; x8 in [0,0.69]; x9 in [0,0.998]; Minimize x2*(log(x2) - log(x2 + x4)) + x4*(log(x4) - log(x2 + x4)) + x3*(log(x3) - log(x3 + x5)) + x5*(log(x5) - log(x3 + x5)) + 0.92*x2*x8 + 0.746*x4*x6 + 0.92*x3*x9 + 0.746*x5*x7; Subject to x6*(x2 + 0.159*x4) - x2 = 0; x7*(x3 + 0.159*x5) - x3 = 0; x8*(0.308*x2 + x4) - x4 = 0; x9*(0.308*x3 + x5) - x5 = 0; x2 + x3 = 0.5; x4 + x5 = 0.5;

Operators (+,−,∗,/,power,sinus,exp,etc) are piecewise continuous and differentiable ⇒ non convex optimization

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

Application domains

Chemistry Robotics: design, proof of properties Signal processing: source separation ...

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

Intervals and boxes

Intervals Interval [x] = [x, x] {x ∈ R, x ≤ x ≤ x} x et x Floating-point bounds IR Set of all the intervals m([x]) Midpoint of [x] w([x]) := x − x Width or size of [x] Boxes Box [x] [x1] × ... × [xi] × ... × [xn]. w([x]) maxn(w([xi])) Outer box search space containing sols & non sols Inner box search space containing only solutions

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

Interval Branch & Bound

Constrained global optimization: Vocabulary Minimize an objective function f in a given box under inequality (g) and equality (h) constraints. A vector/point x satisfying the constraints is called feasible. Interval Branch & Bound Explore a box [x] in subdividing it in sub-boxes (of width greater than ǫsol). During the search, for every node/box: contract the current box, without loss of solution update a (generally non feasible) lower bound lb of the objective function cost, and update a feasible upper bound ub of the cost, until ub − lb < ǫobj. Store the minimum cost associated to every box/node of the search tree. Perform a best-first search: select first the box with the lowest cost (possible memory overflow, utilization of a heap)

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

Illustration

Objective function

Lower bound Upper bound

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

Plan

1

Background

2

Main contribution

3

New framework for constrained global optimization

4

Experimental evaluation

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

Upper Bounding and reliability

Constraints Objective function

Upper bounding and reliability: issue A point obtained during local search is rarely feasible: it must be

• ften corrected and certified by costly interval analysis methods.

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

Alternative: Upper Bounding in inner regions

Principles For each node, management of an outer box [x]out and of an inner region r in extracted from [x]out. Inner region ≡ box or polytope in which all points are feasible. Upper Bounding essentially in r in. Equations are relaxed: h(x) ∈ [−ǫeq, +ǫeq]

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

Plan

1

Background

2

Main contribution

3

New framework for constrained global optimization

4

Experimental evaluation

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

General schema

System: S := {g(x) ≤ 0, h(x) ∈ [−ǫeq, +ǫeq], y = f(x), y ≤ ub} Branching heuristic Principle: impact of a variable xi on a function fj Kearfott 1990: smear(xi, fj) =

• ∂fj

∂xi

• N([x])
• ∗ w([xi])

Variant: smearRelative(xi, fj) =

smear(xi,fj) P

xk ∈x smear(xk,fj) ∈ [0, 1]

At every node of the search tree

1

y ← ub

2

OuterContractLB(...) // Contract [x] and improve y

3

InnerExtractUB(...) // Upper bounding in inner regions

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

OuterContractLB

Mohc Contract [x]: interval constraint propagation algorithm Mohc. Mohc exploits monotonicity of functions Side effect: improve y Improve y with safe linearization (lower bounding) Main difficulty: Compute a safe outer linearization (round-off errors) Existing approaches: Reliable reformulation-linearization-technique (Quad by Lebbah et al.), affine arithmetic Proposition: interval first-order taylor-based linearization (by Lin & Stadtherr)

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

Convex interval Taylor

f’ f’ f’ f’ x x

f(m([x]))+f'.(x-m([x])) f(m([x]))+f'.(x-m([x])) f(x)+f'.(x-x) f(x)+f'.(x-x) f(x) f(x)

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

Convex interval Taylor

∀x ∈ [x], f(x) + a1 ∗ xl

1 + ... + an ∗ xl n ≤ f(x) with xl i = xi − xi

Linear program LPlb = min f(x) + a1 ∗ xl

1 + ... + an ∗ xl n

subject to : ∀j gj(x) + aj

1 ∗ xl 1 + ... + aj n ∗ xl n ≤ 0

∀i 0 ≤ xl

i , xl i ≤ w([xi])

where : xl

i = xi − xi

Using two corners x and x ⇒ tighter polytope f(x) +

i ai(xi − xi) = f(x) + i aixi − ai xi =

• i aixi + f(x) −

i ai xi

f(x) +

i ai(xi − xi) = f(x) + i aixi − aixi =

• i aixi + f(x) −

i ai xi

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

InnerExtractUB

Inner region extraction based on InHC4 (inner box) and InnerLinearization (inner polytope). Using InHC4 [x]in ← InHC4 (S, [x]out) /* See [Chabert, Beldiceanu, CP 2010] */ if [x]in = ∅ then [x]in ← MonotonicityAnalysis (f, [x]in) x ← RandomProbing([x]in) else x ← RandomProbing([x]out) end if Update the upper bound

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

One word about InnerLinearization

Inner interval linearization based on corner-based Taylor ∀x ∈ [x], f(x) ≤ f l(x) = f(x) +

i ai ∗ (xi − xi) ≤ 0

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

Plan

1

Background

2

Main contribution

3

New framework for constrained global optimization

4

Experimental evaluation

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

Protocol, overview

Implementation with the free C++ interval library Ibex (maintained by Chabert) ⇒ IbexOpt Precision of the solutions/boxes: 1e-9 Precision on the cost: 1e-8 Qualitative study Comparison between IbexOpt and

GlobSol (Kearfott) Icos (Lebbah, Rueher, Michel), IBBA+ (Ninin, Messine, Hansen), the non reliable Baron (Sahinidis, Tawarmalani);

• n a benchmark of 74 systems proposed by Ninin.

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

Qualitative study

Gain 0.02 [0.1, 0.5] [0.5, 2] [2, 10] [10, 100] >100 Mohc/HC4 1 62 5 2 OuterLinear. 1 35 9 5 20 Inner/Probing 33 24 9 4 InnerLinear. 62 7 1 InHC4 66 4 SSR/SM 2 59 4 1 4 SSR/RR 1 42 13 11 3 SSR/LF 1 40 9 16 4 SSR (SmearRelative), SM (standard smear), RR (Round Robin), LF (largest-first)

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

Performance profile

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

Comparison on difficult instances

System n Baron GlobSol IBBA+ Icos IbexOpt IbexOpt ǫobj 1.e-8 1.e-8 1.e-8 1.e-3 1.e-3 1.e-8 ex2 1 9 10 1.52 154 59.9 13 30 2050 60007 1549 13370 30444 ex6 1 1 8 7.64 3203 >3600 >600 13 17 5616 12811 14725 ex6 1 3 12 19.2 >3600 >600 46.74 540 11217 26137 204439 ex6 2 6 3 26 306 1575 >600 36.75 173 26765 922664 34318 163227 ex6 2 8 3 19 220 458 >600 29.40 111 29469 265276 27513 97554 ex6 2 9 4 170 465 522 >600 12.94 37 92143 203775 9873 27461

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

Comparison on difficult instances

System n Baron GlobSol IBBA+ Icos IbexOpt IbexOpt ǫobj 1.e-8 1.e-8 1.e-8 1.e-3 1.e-3 1.e-8 ex6 2 10 6 >1000 >3600 >3600 >600 431 1955 2.e-3 224484 820902 ex6 2 11 3 55 273 140 >600 4.02 22 45085 83457 4487 24264 ex6 2 12 4 30 193 113 > 600 4.37 122 19182 58231 4173 86722 ex6 2 13 6 >1000 >3600 >3600 >600 1099 > 3600 2.e-2 545676 2.e-4 ex7 3 5 13 1.11 >3600 136 50.50 55 309 3699 40936 44147 ex14 1 7 10 1.27 >3600 >600 451 464 181 177464 181136 ex14 2 7 6 0.03 >3600 >600 84.73 85 1 17463 16759

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

Sum up: new schema for constrained global

• ptimization

Does not use : local search, interval analysis operators (interval Newton) for upper bounding the cost. Relaxes equalities: h(X) ∈ [−ǫeq, +ǫeq]. Ingredients: Mohc (contraction with constraint propagation), convex interval Taylor (lower bounding), two inner regions’ extraction algorithms. Outperforms significantly the three existing interval B&Bs. Far from Baron performances... except in (highly nonlinear) 10% of the tested systems.

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization

Background Main contribution New framework for constrained global optimization Experimental evaluation

A few words about InHC4

InHC4 Single constraint: InHC4-Revise extracts a box inner to that constraint System of constraints: intersection of individual inner boxes Like HC4-Revise, InHC4-Revise traverses twice the syntactical tree of the constraint. During projection, one interval is chosen among a union of intervals ⇒ heuristic Details in [Chabert, Beldiceanu, CP 2010].

Trombettoni, Araya, Neveu, Chabert Inner Regions for Global Optimization