GLOBAL OPTIMIZATION WITH BRANCH-AND-REDUCE: Algorithms, Software, - - PowerPoint PPT Presentation

GLOBAL OPTIMIZATION WITH BRANCH-AND-REDUCE: Algorithms, Software, and Applications

Nick Sahinidis University of Illinois at Urbana-Champaign Chemical and Biomolecular Engineering

CHALLENGES IN GLOBAL OPTIMIZATION

p n

Z y R x y x g y x f ∈ ∈ ≤ , ) , ( s.t. ) , ( min

NP-HARD PROBLEM

Multimodal objective

) , ( y x f

Integrality conditions

) , ( y x f

Nonconvex constraints

) , ( y x f

AUTOMOTIVE REFRIGERANT DESIGN (Joback and Stephanopoulos, 1990)

• Higher enthalpy of vaporization (ΔHve) reduces the

amount of refrigerant

• Lower liquid heat capacity (Cpla) reduces amount of

vapor generated in expansion valve

• Maximize ΔHve/ Cpla, subject to: ΔHve ≥ 18.4, Cpla ≤ 32.2

FUNCTIONAL GROUPS CONSIDERED

PROPERTY PREDICTION

BRANCH-AND-BOUND

MOLECULAR DESIGN AFTER 150 CPU HOURS IN 1995

• One feasible solution identified
• Optimality not proved
• First attempt:

– IBM RS/6000 43P with 128 MB RAM

• Second attempt:

– IBM SP/2 Single Processor with 2 GB RAM

MOLECULAR DESIGN IN 2000

In 30 CPU minutes

BREAST CANCER DIAGNOSIS

• 200,000 cases diagnosed in the U.S. a year
• 40,000 deaths a year
• Most breast cancers are first diagnosed by

the patient as a lump in the breast

• Majority of breast lumps are benign
• Available diagnosis methods:

– Mammography (68% to 79% correct) – Surgical biopsy (100% correct but invasive and costly) – Fine needle aspirate (FNA) » With visual inspection: 65% to 98% correct » Automated diagnosis: 95% correct

• Linear programming techniques
• Mangasarian and Wolberg in 1990s

WISCONSIN DIAGNOSTIC BREAST CANCER (WDBC) DATABASE

• 653 patients
• 9 cytological characteristics:

– Clump thickness – Uniformity of cell size – Uniformity of cell shape – Marginal adhesion – Single epithelial cell size – Bare nuclei – Bland chromatin – Normal nucleoli – Mitoses

• Biopsy classified these 653

patients in two classes:

– Benign – Malignant

From Wolberg, Street, & Mangasarian, 1993

BILINEAR (IN-)SEPARABILITY OF TWO SETS IN Rn

Requires the solution of three nonconvex bilinear programs

GLOBAL OPTIMIZATION ALGORITHMS

• Stochastic and deterministic

algorithms

• Branch-and-Bound

– Bound problem over successively refined partitions » Falk and Soland, 1969 » McCormick, 1976

• Convexification

– Outer-approximate with increasingly tighter convex programs – Tuy, 1964 – Sherali and Adams, 1994

• Horst and Tuy, Global

Optimization: Deterministic Approaches, 1996

– Over 800 citations

• Our approach

– Branch-and-Reduce » Ryoo and Sahinidis, 1995, 1996 » Shectman and Sahinidis, 1998 – Constraint Propagation & Duality-Based Reduction » Ryoo and Sahinidis, 1995, 1996 » Tawarmalani and Sahinidis, 2002 – Convexification » Tawarmalani and Sahinidis, 2001, 2002, 2003, 2005

• Tawarmalani and Sahinidis,

Convexification and Global Optimization in Continuous and Mixed-Integer Nonlinear Programming, 2002

BOUNDING SEPARABLE PROGRAMS

10 4 6 8 s.t. min

3 2 1 2 1 3 2 2 2 1 2 3 2 1

≤ ≤ ≤ ≤ ≤ ≤ + = ≤ − − − − = x x x x x x x x x x x f

TIGHT RELAXATIONS

Convex/concave envelopes often finitely generated

x x

Concave

• ver-estimator

Convex under-estimator

) (x f ) (x f

Concave envelope Convex envelope

) (x f x

RATIO: THE GENERATING SET

DIFFERENCE BETWEEN ENVELOPE AND TRADITIONAL RELAXATION

ENVELOPES OF MULTILINEAR FUNCTIONS

• Multilinear function over a box
• Generating set
• Polyhedral convex encloser follows trivially from

polyhedral representation theorems

n i U x L x a x x M

i i i p i i t t n

t

, , 1 , , ) ,..., (

1 1

K = +∞ < ≤ ≤ < ∞ − =

∏ ∑

=

⎟ ⎟ ⎠ ⎞ ⎜ ⎝ ⎛∏

=

] , [ vert

1 i i n i

U L

BOUNDING FACTORABLE PROGRAMS

Introduce variables for intermediate quantities whose envelopes are not known

• Local NLP solvers essential for local search
• Linear programs can be solved very efficiently
• Outer-approximate convex relaxation by polyhedron

Tawarmalani and Sahinidis (Math. Progr., 2004, 2005)

POLYHEDRAL OUTER-APPROXIMATION

• Quadratically convergent sandwich algorithm
• Cutting planes for functional compositions

RECURSIVE FUNCTIONAL COMPOSITIONS

• Consider h=g(f), where

– g and f are multivariate convex functions – g is non-decreasing in the range of each nonlinear component of f

• h is convex
• Two outer approximations of the composite

function h:

– S1: a single-step procedure that constructs supporting hyperplanes of h at a predetermined number of points – S2: a two-step procedure that constructs supporting hyperplanes for g and f at corresponding points

• Two-step is sharper than one-step

– If f is affine, S2=S1 – In general, the inclusion is strict

OUTER APPROXIMATION OF x2+y2

+

AUTOMATIC DETECTION AND EXPLOITATION OF CONVEXITY

• Composition rule: h = g(f), where

– g and f are multivariate convex functions – g is non-decreasing in the range of each nonlinear component of f

• Subsumes many known rules for detecting

convexity/concavity

– g univariate convex, f linear – g=max{f1(x), …, fm(x)}, each fi convex – g=exp(f(x)) – …

• Automatic exploitation of convexity is not

essential for constructing polyhedral outer approximations in these cases

– However, logexp(x) = log(ex1 + … + exn) – CONVEX_EQUATIONS modeling language construct

MARGINALS-BASED RANGE REDUCTION

If a variable goes to its upper bound at the relaxed problem solution, this variable’s lower bound can be improved Relaxed Value Function

z x xU xL U L

f. c. e. b. a. d.

REDUCTION VIA CONSTRAINT PROPAGATION

FINITE VERSUS CONVERGENT BRANCH-AND-BOUND ALGORITHMS

Finite sequences A potentially infinite sequence

FINITE BRANCHING RULE

• Variable selection:

– Typically, select variable with largest underestimating gap – Occasionally, select variable corresponding to largest edge

• Point selection:

– Typically, at the midpoint (exhaustiveness) – When possible, at the best currently known solution

• Finite isolation of global optimum
• Finite termination in many cases

– Concave minimization over polytopes – 2-Stage stochastic integer programming x*

x x

∗

f(x)

BRANCH-AND-REDUCE

STOP START Multistart search and reduction Nodes? N Y Select Node Lower Bound Inferior? Delete Node Y N Preprocess Upper Bound Postprocess Reduced? N Y Branch

Feasibility-based reduction Optimality-based reduction

Branch-And-Reduce Optimization Navigator

Components

• Modeling language
• Preprocessor
• Data organizer
• I/O handler
• Range reduction
• Solver links
• Interval arithmetic
• Sparse matrix routines
• Automatic differentiator
• IEEE exception handler
• Debugging facilities

Capabilities

• Core module

– Application-independent – Expandable

• Fully automated MINLP

solver

• Application modules

– Multiplicative programs – Indefinite QPs – Fixed-charge programs – Mixed-integer SDPs – …

• Solve relaxations using

– CPLEX, MINOS, SNOPT, OSL, SDPA, …

• Available under GAMS and AIMMS
• Available on NEOS server

26 PROBLEMS FROM globallib AND minlplib

93 6 76 CPU hrs 98 13,772 622,339 Nodes in memory 99 253,754 23,031,434 Nodes % reduction With cuts Without cuts 63 432 Discrete variables 115 1030 4 Variables 76 513 2 Constraints Average Maximum Minimum

EFFECT OF CUTTING PLANES

POOLING PROBLEM: p-FORMULATION

POOLING PROBLEM: q-FORMULATION

POOLING PROBLEM: pq-FORMULATION

PRODUCT DISAGGREGATION

Consider the function: Let Then

∑ ∑

= =

+ + + =

n k k k n k k k n

y b x b x y a a y y x

1 1 1

) , , ; ( K φ ] , [ ] , [

1 U k L k n k U L

y y x x H

Π

=

× =

∑ ∑

= × =

+ + + =

n k k k x x y y n k k k H

x y b b x y a a

U L U k L k

1 ] , [ ] , [ 1

) ( convenv convenv φ

Disaggregated formulations are tighter

LOCAL SEARCH WITH CONOPT

• 4330.78

Infeasible rt97

• 750

haverly3

• 400

haverly2

• 400

haverly1

• 6.5
• 7

foulds5

• 6.5
• 6

foulds4

• 6.5
• 6.5

foulds3

• 600
• 1000

foulds2

• 2700
• 2900

bental5 bental4

• 470.83
• 470.83

adhya4

• 57.74
• 65

adhya3 adhya2

• 56.67
• 68.74

adhya1 pq-formulation

• bjective

q-formulation

• bjective

Problem

GLOBAL SEARCH WITH BARON

0.5 6 174 5629 rt97 1 3 haverly3 1 17 haverly2 1 25 haverly1 1

• 1

>1200 >389 foulds5 1

• 1

>1200 >326 foulds4 5

• 1

>1200 >348 foulds3

• 1

16 1061 foulds2

• 1

>1200 >6445 bental5 0.5 1 0.5 101 bental4 1 1 >1200 >6129 adhya4 1.5 31 >1200 >9248 adhya3 0.5 17 20 501 adhya2 0.5 24 17 573 adhya1 CPU sec Nodes CPU sec Nodes pq-formulation p-formulation Problem

ONGOING DEVELOPMENT OF BARON

Structural Bioinformatics Systems biology X-ray imaging

U E(r)

Portfolio optimization

BARON IN APPLICATIONS

• Development of new Runge-Kutta methods

for partial differential equations

– Ruuth and Spiteri, SIAM J. Numerical Analysis, 2004

• Energy policy making

– Manne and Barreto, Energy Economics, 2004

• Design of metabolic pathways

– Grossmann, Domach and others, Computers & Chemical Engineering, 2005

• Model estimation for automatic control

– Bemporand and Ljung, Automatica, 2004

• Agricultural economics

– Cabrini et al., Manufacturing and Service Operations Management, 2005

GLOBAL/MINLP SOFTWARE

• AlphaECP—Exploits pseudoconvexity
• BARON—Branch-And-Reduce
• BONMIN—Integer programming technology (CMU/IBM)
• DICOPT—Decomposition
• GlobSol—Interval arithmetic
• Interval Solver (Frontline)—Interval solver; Excel
• LaGO—Lagrangian relaxations (COIN/OR)
• LGO—Stochastic search; black-box optimization
• LINGO—Trigonometric functions; IF-THEN-ELSE; …
• MSNLP, OQNLP—Stochastic search
• SBB—Simple branch-and-bound

NLP/MINLP NLP MINLP

COMPARISONS ON MINLPLIB

GAMS SALES Commercial and academic users

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006

Global (BARON, LGO, MSNLP, OQNLP) MINLP (DICOPT, SBB) Local NLP (CONOPT, KNITRO, MINOS, PATH, SNOPT) Data courtesy of Alex Meeraus

x*

Convexification Range Reduction Finiteness

BRANCH-AND-REDUCE

Engineering design Management and Finance Chem-, Bio-, Medical Informatics

ACKNOWLEDGEMENTS

• N. Adhya (i2)
• S. Ahmed

– Georgia Institute of Technology

• Y. Chang (Penn State)
• J. Elble
• K. Furman (ExxonMobil)
• V. Ghildyal (Sabre)
• M. L. Liu

– National Chengchi University

• G. Nanda (USAir)
• H. Ryoo

– Korea University

• J. Shectman (Northfield)
• A. Smith
• M. Tawarmalani

– Purdue University

• A. Vaia (BPAmoco)
• R. Vander Wiel (3M)
• Y. Voudouris (Merck)
• W. Xie
• M. Yu (Marconi Telecomm)
• American Chemical Society
• DuPont
• ExxonMobil
• Lucent Technologies
• Mitsubishi Chemicals
• National Institutes of Health

– General Medical Sciences

• National Science Foundation

– Bioengineering and Environmental Sciences – Chemical and Thermal Systems – Design and Manufacturing – Electrical and Communication Systems – Operations Research

• TAPPI
• University of Illinois at U-C

– Chemical Engineering – Computational Science and Engineering – Mechanical and Industrial Engineering – Research Board