Optimizing Convex Functions over Non-Convex Domains Dan Bienstock - - PowerPoint PPT Presentation

▶
optimizing convex functions over non convex domains
SMART_READER_LITE
LIVE PREVIEW

Optimizing Convex Functions over Non-Convex Domains Dan Bienstock - - PowerPoint PPT Presentation

Mar 27, 2023 97 likes •1.13k views

Optimizing Convex Functions over Non-Convex Domains Dan Bienstock and Alex Michalka (adm2148@columbia.edu) Columbia University Aussois 2012 Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 1 / 33 Introduction

slide-1
SLIDE 1

Optimizing Convex Functions over Non-Convex Domains

Dan Bienstock and Alex Michalka (adm2148@columbia.edu)

Columbia University

Aussois 2012

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 1 / 33

slide-2
SLIDE 2

Introduction

Generic Problem: min Q(x), s.t. x ∈ F,

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 2 / 33

slide-3
SLIDE 3

Introduction

Generic Problem: min Q(x), s.t. x ∈ F, Q(x) convex, especially: convex quadratic

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 2 / 33

slide-4
SLIDE 4

Introduction

Generic Problem: min Q(x), s.t. x ∈ F, Q(x) convex, especially: convex quadratic F nonconvex

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 2 / 33

slide-5
SLIDE 5

Introduction

Generic Problem: min Q(x), s.t. x ∈ F, Q(x) convex, especially: convex quadratic F nonconvex Examples: F is a mixed-integer set

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 2 / 33

slide-6
SLIDE 6

Introduction

Generic Problem: min Q(x), s.t. x ∈ F, Q(x) convex, especially: convex quadratic F nonconvex Examples: F is a mixed-integer set F is constrained in a nasty way, e.g. x1 − 3 sin(x2) + 2 cos(x3) = 4

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 2 / 33

slide-7
SLIDE 7

Exclude-and-cut

min z, s.t. z ≥ Q(x), x ∈ F

  • 0. Let ˆ

F be a convex relaxation of F

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 3 / 33

slide-8
SLIDE 8

Exclude-and-cut

min z, s.t. z ≥ Q(x), x ∈ F

  • 0. Let ˆ

F be a convex relaxation of F

  • 1. Let (x∗, z∗) = argmin{ z : z ≥ Q(x), x ∈ ˆ

F}

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 3 / 33

slide-9
SLIDE 9

Exclude-and-cut

min z, s.t. z ≥ Q(x), x ∈ F

  • 0. Let ˆ

F be a convex relaxation of F

  • 1. Let (x∗, z∗) = argmin{ z : z ≥ Q(x), x ∈ ˆ

F}

  • 2. Find an open set S s.t. x∗ ∈ S and S ∩ F = ∅.

Examples: lattice-free sets, geometry

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 3 / 33

slide-10
SLIDE 10

Exclude-and-cut

min z, s.t. z ≥ Q(x), x ∈ F

  • 0. Let ˆ

F be a convex relaxation of F

  • 1. Let (x∗, z∗) = argmin{ z : z ≥ Q(x), x ∈ ˆ

F}

  • 2. Find an open set S s.t. x∗ ∈ S and S ∩ F = ∅.

Examples: lattice-free sets, geometry

  • 3. Add to the formulation an inequality az + αTx ≥ α0 valid for

{ (x, z) : x ∈ Rn − S, z ≥ Q(x) } but violated by (x∗, z∗).

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 3 / 33

slide-11
SLIDE 11

The SUV problem

given full-dimensional polyhedra P1, . . . , PK in Rd, find a point closest to the origin not contained inside any of the Ph.

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 4 / 33

slide-12
SLIDE 12

The SUV problem

given full-dimensional polyhedra P1, . . . , PK in Rd, find a point closest to the origin not contained inside any of the Ph. min x2 s.t. x ∈ Rd −

K

  • h=1

int(Ph),

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 4 / 33

slide-13
SLIDE 13

The SUV problem

given full-dimensional polyhedra P1, . . . , PK in Rd, find a point closest to the origin not contained inside any of the Ph. min x2 s.t. x ∈ Rd −

K

  • h=1

int(Ph), (application: X-ray lythography; see Ahmadia (2010))

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 4 / 33

slide-14
SLIDE 14

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 5 / 33

slide-15
SLIDE 15

Typical values for d (dimension): less than 20; usually even smaller Typical values for K (number of polyhedra): possibly hundreds, but

  • ften less than 50

Very hard problem

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 5 / 33

slide-16
SLIDE 16

First problem setting - polyhedral case

Let Q(x) be a positive definite quadratic on Rd, Let P = {x ∈ Rd : aT

i x ≤ bi, 1 ≤ i ≤ m} full dimensional

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 6 / 33

slide-17
SLIDE 17

First problem setting - polyhedral case

Let Q(x) be a positive definite quadratic on Rd, Let P = {x ∈ Rd : aT

i x ≤ bi, 1 ≤ i ≤ m} full dimensional

Want to produce a linear inequality description for: S = conv

  • (x, q) ∈ Rd+1 : Q(x) ≤ q,

x ∈ Rd − int(P)

  • .

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 6 / 33

slide-18
SLIDE 18

First problem setting - polyhedral case

Let Q(x) be a positive definite quadratic on Rd, Let P = {x ∈ Rd : aT

i x ≤ bi, 1 ≤ i ≤ m} full dimensional

Want to produce a linear inequality description for: S = conv

  • (x, q) ∈ Rd+1 : Q(x) ≤ q,

x ∈ Rd − int(P)

  • .

change in coordinates → S = conv

  • (x, q) ∈ Rd+1 : x2 ≤ q,

x ∈ Rd − int(P)

  • .

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 6 / 33

slide-19
SLIDE 19

{(x, q) ∈ Rd+1 : x2 ≤ q, x ∈ Rd − int(P)}

Write P = {x ∈ Rd : aT

i x ≤ bi, 1 ≤ i ≤ m}

Pick y with aT

i y = bi but aT j y < bj ∀ j = i.

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 7 / 33

slide-20
SLIDE 20

{(x, q) ∈ Rd+1 : x2 ≤ q, x ∈ Rd − int(P)}

Write P = {x ∈ Rd : aT

i x ≤ bi, 1 ≤ i ≤ m}

Pick y with aT

i y = bi but aT j y < bj ∀ j = i.

The “first-order” inequality q ≥ 2yT(x − y) + y2 = 2yTx − y2 is valid for all x ∈ Rd

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 7 / 33

slide-21
SLIDE 21

{(x, q) ∈ Rd+1 : x2 ≤ q, x ∈ Rd − int(P)}

Write P = {x ∈ Rd : aT

i x ≤ bi, 1 ≤ i ≤ m}

Pick y with aT

i y = bi but aT j y < bj ∀ j = i.

The “first-order” inequality q ≥ 2yT(x − y) + y2 = 2yTx − y2 is valid for all x ∈ Rd For real α > 0 small enough the inequality q ≥ (2y − αai)T(x − y) + y2 = 2yTx − y2 − α (aT

i x − bi)

is valid. It cuts off (some) points (x, q) with x2 ≤ q and x ∈ int(P).

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 7 / 33

slide-22
SLIDE 22

{(x, q) ∈ Rd+1 : x2 ≤ q, x ∈ Rd − int(P)}

Write P = {x ∈ Rd : aT

i x ≤ bi, 1 ≤ i ≤ m}

Pick y with aT

i y = bi but aT j y < bj ∀ j = i.

The “first-order” inequality q ≥ 2yT(x − y) + y2 = 2yTx − y2 is valid for all x ∈ Rd For real α > 0 small enough the inequality q ≥ (2y − αai)T(x − y) + y2 = 2yTx − y2 − α (aT

i x − bi)

is valid. It cuts off (some) points (x, q) with x2 ≤ q and x ∈ int(P). Largest possible α: “lifted first-order inequality”

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 7 / 33

slide-23
SLIDE 23

{(x, q) ∈ Rd+1 : x2 ≤ q, x ∈ Rd − int(P)}

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 8 / 33

slide-24
SLIDE 24

{(x, q) ∈ Rd+1 : x2 ≤ q, x ∈ Rd − int(P)}

Theorem. (a) Any linear inequality valid for S is dominated by a lifted first-order

  • inequality. More precisely,

conv

  • (x, q) ∈ Rd+1 : x2 ≤ q,

x ∈ Rd − int(P)

  • =

{(x, q) ∈ Rd+1 : x2 ≤ q, and LFO inequalities }

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 8 / 33

slide-25
SLIDE 25

{(x, q) ∈ Rd+1 : x2 ≤ q, x ∈ Rd − int(P)}

Theorem. (a) Any linear inequality valid for S is dominated by a lifted first-order

  • inequality. More precisely,

conv

  • (x, q) ∈ Rd+1 : x2 ≤ q,

x ∈ Rd − int(P)

  • =

{(x, q) ∈ Rd+1 : x2 ≤ q, and LFO inequalities }

(b) Let (ˆ x, ˆ q) ∈ Rd+1 with ˆ x ∈ int(P). We can compute a lifted first-order inequality maximally violated by (ˆ x, ˆ q), by solving m linearly constrained convex quadratic programs

  • n O(d) variables.

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 8 / 33

slide-26
SLIDE 26

{(x, q) ∈ Rd+1 : x2 ≤ q, x ∈ Rd − int(P)}

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 9 / 33

slide-27
SLIDE 27

{(x, q) ∈ Rd+1 : x2 ≤ q, x ∈ Rd − int(P)}

Sketch proof of (a).

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 9 / 33

slide-28
SLIDE 28

{(x, q) ∈ Rd+1 : x2 ≤ q, x ∈ Rd − int(P)}

Sketch proof of (a). Suppose γTx − q ≤ δ is valid and holds with equality at some feasible point y.

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 9 / 33

slide-29
SLIDE 29

{(x, q) ∈ Rd+1 : x2 ≤ q, x ∈ Rd − int(P)}

Sketch proof of (a). Suppose γTx − q ≤ δ is valid and holds with equality at some feasible point y. If y is strictly feasible, the inequality is just a first-order inequality. So assume y is on the ith facet of P.

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 9 / 33

slide-30
SLIDE 30

{(x, q) ∈ Rd+1 : x2 ≤ q, x ∈ Rd − int(P)}

Sketch proof of (a). Suppose γTx − q ≤ δ is valid and holds with equality at some feasible point y. If y is strictly feasible, the inequality is just a first-order inequality. So assume y is on the ith facet of P. Using Farkas’s Lemma, we can show that γ = 2y − αai and δ = y2 − αbi for some α ≥ 0.

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 9 / 33

slide-31
SLIDE 31

{(x, q) ∈ Rd+1 : x2 ≤ q, x ∈ Rd − int(P)}

Sketch proof of (a). Suppose γTx − q ≤ δ is valid and holds with equality at some feasible point y. If y is strictly feasible, the inequality is just a first-order inequality. So assume y is on the ith facet of P. Using Farkas’s Lemma, we can show that γ = 2y − αai and δ = y2 − αbi for some α ≥ 0. This means the inequality is a (partially) lifted first-order inequality.

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 9 / 33

slide-32
SLIDE 32

{(x, q) ∈ Rd+1 : x2 ≤ q, x ∈ Rd − int(P)}

Sketch proof of (a). Suppose γTx − q ≤ δ is valid and holds with equality at some feasible point y. If y is strictly feasible, the inequality is just a first-order inequality. So assume y is on the ith facet of P. Using Farkas’s Lemma, we can show that γ = 2y − αai and δ = y2 − αbi for some α ≥ 0. This means the inequality is a (partially) lifted first-order inequality. (“Partially” since α may not be as large as possible)

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 9 / 33

slide-33
SLIDE 33

Deriving the Lifting Coefficient

We want the lifted inequality from a point y on the ith facet of P.

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 10 / 33

slide-34
SLIDE 34

Deriving the Lifting Coefficient

We want the lifted inequality from a point y on the ith facet of P. Amount of violation at point (x, x2) is: x2 − (2y − αai)Tx − αbi + y2

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 10 / 33

slide-35
SLIDE 35

Deriving the Lifting Coefficient

We want the lifted inequality from a point y on the ith facet of P. Amount of violation at point (x, x2) is: x2 − (2y − αai)Tx − αbi + y2 For the lifted inequality to be valid, this must be non-negative for all feasible x.

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 10 / 33

slide-36
SLIDE 36

Deriving the Lifting Coefficient

We want the lifted inequality from a point y on the ith facet of P. Amount of violation at point (x, x2) is: x2 − (2y − αai)Tx − αbi + y2 For the lifted inequality to be valid, this must be non-negative for all feasible x. If the lifted inequality is as strong as possible, the violation will be 0 at some point x on another facet of P.

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 10 / 33

slide-37
SLIDE 37

Deriving the Lifting Coefficient

We find a lifting coefficient for every facet other than the ith. Taking the smallest one gives a valid lifted inequality.

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 11 / 33

slide-38
SLIDE 38

Deriving the Lifting Coefficient

We find a lifting coefficient for every facet other than the ith. Taking the smallest one gives a valid lifted inequality. To find the lifting coefficient for jth facet, we find αj satisfying:

  • minimize:

x2 − (2y − αjai)Tx − αjbi + y2 subject to: aT

j x = bj

  • = 0

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 11 / 33

slide-39
SLIDE 39

Deriving the Lifting Coefficient

We find a lifting coefficient for every facet other than the ith. Taking the smallest one gives a valid lifted inequality. To find the lifting coefficient for jth facet, we find αj satisfying:

  • minimize:

x2 − (2y − αjai)Tx − αjbi + y2 subject to: aT

j x = bj

  • = 0

We can solve this QP in closed form to obtain αj = 2(bj − aT

j y)

aiaj − aT

i aj

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 11 / 33

slide-40
SLIDE 40

Deriving the Lifting Coefficient

We find a lifting coefficient for every facet other than the ith. Taking the smallest one gives a valid lifted inequality. To find the lifting coefficient for jth facet, we find αj satisfying:

  • minimize:

x2 − (2y − αjai)Tx − αjbi + y2 subject to: aT

j x = bj

  • = 0

We can solve this QP in closed form to obtain αj = 2(bj − aT

j y)

aiaj − aT

i aj

Final lifting coefficient is: min

j=i αj

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 11 / 33

slide-41
SLIDE 41

Finding the Strongest Cut

We have derived an expression for the lifting coefficient α : α = min

j=i

2(bj − aT

j y)

aiaj − aT

i aj

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 12 / 33

slide-42
SLIDE 42

Finding the Strongest Cut

We have derived an expression for the lifting coefficient α : α = min

j=i

2(bj − aT

j y)

aiaj − aT

i aj

Given a point ˆ x ∈ int(P), solving m convex QPs of the following form gives the strongest cut at ˆ x: maximize: −y2 + ˆ xTy + α(bi − aT

i ˆ

x) subject to: aT

i y = bi

aT

j y ≤ bj

∀j = i α ≤

2(bj−aT

j y)

aiaj−aT

i aj

∀j = i

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 12 / 33

slide-43
SLIDE 43

Disjunctive Approach

P = {x ∈ Rd : aT

i x ≤ bi, 1 ≤ i ≤ m}

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 13 / 33

slide-44
SLIDE 44

Disjunctive Approach

P = {x ∈ Rd : aT

i x ≤ bi, 1 ≤ i ≤ m}

S = conv (Q1 ∪ Q2 ∪ . . . ∪ Qm) , where Qi =

  • (x, q) ∈ Rd+1 : x2 ≤ q, aT

i x ≥ bi

  • .

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 13 / 33

slide-45
SLIDE 45

Disjunctive Approach

P = {x ∈ Rd : aT

i x ≤ bi, 1 ≤ i ≤ m}

S = conv (Q1 ∪ Q2 ∪ . . . ∪ Qm) , where Qi =

  • (x, q) ∈ Rd+1 : x2 ≤ q, aT

i x ≥ bi

  • .

Ceria and Soares (1999) ⇒ min { q : (ˆ x, q) ∈ S} is an SOCP

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 13 / 33

slide-46
SLIDE 46

Disjunctive Approach

P = {x ∈ Rd : aT

i x ≤ bi, 1 ≤ i ≤ m}

S = conv (Q1 ∪ Q2 ∪ . . . ∪ Qm) , where Qi =

  • (x, q) ∈ Rd+1 : x2 ≤ q, aT

i x ≥ bi

  • .

Ceria and Soares (1999) ⇒ min { q : (ˆ x, q) ∈ S} is an SOCP with O(md) variables and m conic constraints

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 13 / 33

slide-47
SLIDE 47

Disjunctive Approach

P = {x ∈ Rd : aT

i x ≤ bi, 1 ≤ i ≤ m}

S = conv (Q1 ∪ Q2 ∪ . . . ∪ Qm) , where Qi =

  • (x, q) ∈ Rd+1 : x2 ≤ q, aT

i x ≥ bi

  • .

Ceria and Soares (1999) ⇒ min { q : (ˆ x, q) ∈ S} is an SOCP with O(md) variables and m conic constraints Separation: solve SOCP, use SOCP “Farkas Lemma”, get linear cut

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 13 / 33

slide-48
SLIDE 48

Disjunctive Approach

P = {x ∈ Rd : aT

i x ≤ bi, 1 ≤ i ≤ m}

S = conv (Q1 ∪ Q2 ∪ . . . ∪ Qm) , where Qi =

  • (x, q) ∈ Rd+1 : x2 ≤ q, aT

i x ≥ bi

  • .

Ceria and Soares (1999) ⇒ min { q : (ˆ x, q) ∈ S} is an SOCP with O(md) variables and m conic constraints Separation: solve SOCP, use SOCP “Farkas Lemma”, get linear cut Generally harder than m QPs with d + 1 variables, 2m linear constraints each

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 13 / 33

slide-49
SLIDE 49

Baby Example

A simple example: d = 1 and P = [−1, 2]. Feasible region is blue. Weak relaxation gives optimal point of (0, 0).

−3 −2 −1 1 2 3 2 4 6 8 x q

  • Dan Bienstock, Alex Michalka (Columbia)

Convex obj non-convex domain Aussois 2012 14 / 33

slide-50
SLIDE 50

Baby Example

First-order inequality at x = −1 is: q ≥ −2x − 1

−3 −2 −1 1 2 3 2 4 6 8 x q

  • Dan Bienstock, Alex Michalka (Columbia)

Convex obj non-convex domain Aussois 2012 15 / 33

slide-51
SLIDE 51

Baby Example

Lifted first-order inequality at x = −1 is: q ≥ x + 2. In this case, a single cut gives the convex hull of feasible region.

−3 −2 −1 1 2 3 2 4 6 8 x q

  • Dan Bienstock, Alex Michalka (Columbia)

Convex obj non-convex domain Aussois 2012 16 / 33

slide-52
SLIDE 52

Geometrical characterization

When does a point

  • ˆ

x, ˆ x2 violate a lifted first-order inequality q ≥ 2yTx − y2 − α (aT

i x − bi) ?

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 17 / 33

slide-53
SLIDE 53

Geometrical characterization

When does a point

  • ˆ

x, ˆ x2 violate a lifted first-order inequality q ≥ 2yTx − y2 − α (aT

i x − bi) ?

It does, if and only if: ˆ x2 − 2yT ˆ x + αaT

i ˆ

x < − y2 + αbi

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 17 / 33

slide-54
SLIDE 54

Geometrical characterization

When does a point

  • ˆ

x, ˆ x2 violate a lifted first-order inequality q ≥ 2yTx − y2 − α (aT

i x − bi) ?

It does, if and only if: ˆ x2 − 2yT ˆ x + αaT

i ˆ

x < − y2 + αbi This describes the interior of a ball,

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 17 / 33

slide-55
SLIDE 55

Geometrical characterization

When does a point

  • ˆ

x, ˆ x2 violate a lifted first-order inequality q ≥ 2yTx − y2 − α (aT

i x − bi) ?

It does, if and only if: ˆ x2 − 2yT ˆ x + αaT

i ˆ

x < − y2 + αbi This describes the interior of a ball, which must be contained in int(P)

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 17 / 33

slide-56
SLIDE 56

Geometrical characterization

y

int(P) cut−off region

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 18 / 33

slide-57
SLIDE 57

Geometrical characterization

y

int(P) cut−off region

Characterization: x ∈ Rd − int(P) iff, for each ball B(µ, R) ⊆ P,

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 18 / 33

slide-58
SLIDE 58

Geometrical characterization

y

int(P) cut−off region

Characterization: x ∈ Rd − int(P) iff, for each ball B(µ, R) ⊆ P, x − µ2 ≥ R2

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 18 / 33

slide-59
SLIDE 59

Geometrical characterization - Example 1

−4 −2 2 4 −3 −2 −1 1 2 3

  • Separating (0, 0). Excluded region in red.

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 19 / 33

slide-60
SLIDE 60

Geometrical characterization - Example 2

  • −6

−4 −2 2 4 6 −2 −1 1 2

  • Separating (0, 0). Excluded region in red.

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 20 / 33

slide-61
SLIDE 61

Ellipsoidal Case

For A 0, polynomially separable linear inequality description for: conv{(x, q) ∈ Rd+1 : x2 ≤ q, xTAx − 2cTx + b ≥ 0}

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 21 / 33

slide-62
SLIDE 62

Ellipsoidal Case

For A 0, polynomially separable linear inequality description for: conv{(x, q) ∈ Rd+1 : x2 ≤ q, xTAx − 2cTx + b ≥ 0}

ρ µ

S

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 21 / 33

slide-63
SLIDE 63

Ellipsoidal Case

For A 0, polynomially separable linear inequality description for: conv{(x, q) ∈ Rd+1 : x2 ≤ q, xTAx − 2cTx + b ≥ 0}

ρ µ

S

Separation problem: given(ˆ x, ˆ q) ∈ Rd+1, Find a ball B(µ, √ρ) ⊆ Rd − S so as to maximize ρ − (ˆ q − 2µTˆ x + µTµ)

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 21 / 33

slide-64
SLIDE 64

Ellipsoidal Case

For A 0, polynomially separable linear inequality description for: conv{(x, q) ∈ Rd+1 : x2 ≤ q, xTAx − 2cTx + b ≥ 0}

ρ µ

S

Separation problem: given(ˆ x, ˆ q) ∈ Rd+1, Find a ball B(µ, √ρ) ⊆ Rd − S so as to maximize ρ − (ˆ q − 2µTˆ x + µTµ) = ρ − ˆ x − µ2 − ˆ q + ˆ x2

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 21 / 33

slide-65
SLIDE 65

S-Lemma (Yakubovich, 1971)

Let f and g be quadratic functions on Rd, and suppose that there is ¯ x with g(¯ x) < 0.

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 22 / 33

slide-66
SLIDE 66

S-Lemma (Yakubovich, 1971)

Let f and g be quadratic functions on Rd, and suppose that there is ¯ x with g(¯ x) < 0. Then: f (x) ≥ 0 whenever g(x) ≤ 0 if and only if There is τ ≥ 0 such that f (x) + τg(x) ≥ 0 for all x ∈ Rd

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 22 / 33

slide-67
SLIDE 67

S-Lemma (Yakubovich, 1971)

Let f and g be quadratic functions on Rd, and suppose that there is ¯ x with g(¯ x) < 0. Then: f (x) ≥ 0 whenever g(x) ≤ 0 if and only if There is τ ≥ 0 such that f (x) + τg(x) ≥ 0 for all x ∈ Rd Proof relies on convexity of {(f (x), g(x)) : x ∈ Rd} for homogenous f and g (Dines 1941).

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 22 / 33

slide-68
SLIDE 68

{ (x, q) ∈ Rd+1 : x2 ≤ q, xTAx − 2cTx + b ≥ 0}

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 23 / 33

slide-69
SLIDE 69

{ (x, q) ∈ Rd+1 : x2 ≤ q, xTAx − 2cTx + b ≥ 0}

→ Fix ρ > 0: ⇒ B(µ, √ρ) ⊆ Rd − S is SOCP-representable

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 23 / 33

slide-70
SLIDE 70

{ (x, q) ∈ Rd+1 : x2 ≤ q, xTAx − 2cTx + b ≥ 0}

→ Fix ρ > 0: ⇒ B(µ, √ρ) ⊆ Rd − S is SOCP-representable

x2 − ρ ≥ 0, ∀ x s.t. (x + µ)TA(x + µ) − 2cT(x + µ) + b ≥ 0}

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 23 / 33

slide-71
SLIDE 71

{ (x, q) ∈ Rd+1 : x2 ≤ q, xTAx − 2cTx + b ≥ 0}

→ Fix ρ > 0: ⇒ B(µ, √ρ) ⊆ Rd − S is SOCP-representable

x2 − ρ ≥ 0, ∀ x s.t. (x + µ)TA(x + µ) − 2cT(x + µ) + b ≥ 0} iff (S-Lemma) there exists τ > 0 such that, for all x ∈ Rd xT

  • I − 1

τ A

  • x + 2

τ (cT − µTA)x + 1 τ (−µTAµ + 2cTµ − b) − ρ ≥ 0 → A = UΛUT (eigenspace), y . = UTx, v . = UT(c − Aµ)

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 23 / 33

slide-72
SLIDE 72

{ (x, q) ∈ Rd+1 : x2 ≤ q, xTAx − 2cTx + b ≥ 0}

→ Fix ρ > 0: ⇒ B(µ, √ρ) ⊆ Rd − S is SOCP-representable

x2 − ρ ≥ 0, ∀ x s.t. (x + µ)TA(x + µ) − 2cT(x + µ) + b ≥ 0} iff (S-Lemma) there exists τ > 0 such that, for all x ∈ Rd xT

  • I − 1

τ A

  • x + 2

τ (cT − µTA)x + 1 τ (−µTAµ + 2cTµ − b) − ρ ≥ 0 → A = UΛUT (eigenspace), y . = UTx, v . = UT(c − Aµ) ⇒ ∀ y ∈ Rd y T

  • I − 1

τ Λ

  • y + 2

τ v Ty + 1 τ (−µTAµ + 2cTµ − b) − ρ ≥ 0

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 23 / 33

slide-73
SLIDE 73

{ (x, q) ∈ Rd+1 : x2 ≤ q, xTAx − 2cTx + b ≥ 0}

→ Fix ρ > 0: ⇒ B(µ, √ρ) ⊆ Rd − S is SOCP-representable

x2 − ρ ≥ 0, ∀ x s.t. (x + µ)TA(x + µ) − 2cT(x + µ) + b ≥ 0} iff (S-Lemma) there exists τ > 0 such that, for all x ∈ Rd xT

  • I − 1

τ A

  • x + 2

τ (cT − µTA)x + 1 τ (−µTAµ + 2cTµ − b) − ρ ≥ 0 → A = UΛUT (eigenspace), y . = UTx, v . = UT(c − Aµ) ⇒ ∀ y ∈ Rd y T

  • I − 1

τ Λ

  • y + 2

τ v Ty + 1 τ (−µTAµ + 2cTµ − b) − ρ ≥ 0       I − 1

τ Λ 1 τ v 1 τ v T 1 τ (−µTAµ + 2cTµ − b) − ρ

     

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 23 / 33

slide-74
SLIDE 74

y T I − 1

τ Λ

  • y + 2

τ v Ty + 1 τ (−µTAµ + 2cTµ − b) − ρ ≥ 0

∀ y Using Schur Complement gives the equivalent conditions: − 1

τ 2

d

j=1 v 2

j

1−λj/τ + 1 τ (−µTAµ + 2cTµ − b) − ρ ≥ 0

τ ≥ λmax(A)

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 24 / 33

slide-75
SLIDE 75

y T I − 1

τ Λ

  • y + 2

τ v Ty + 1 τ (−µTAµ + 2cTµ − b) − ρ ≥ 0

∀ y Using Schur Complement gives the equivalent conditions: − 1

τ 2

d

j=1 v 2

j

1−λj/τ + 1 τ (−µTAµ + 2cTµ − b) − ρ ≥ 0

τ ≥ λmax(A) The first inequality is equivalent to: −

d

  • j=1

v 2

j

τ − λj − µTAµ + 2cTµ − b − ρτ ≥ 0

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 24 / 33

slide-76
SLIDE 76

y T I − 1

τ Λ

  • y + 2

τ v Ty + 1 τ (−µTAµ + 2cTµ − b) − ρ ≥ 0

∀ y Using Schur Complement gives the equivalent conditions: − 1

τ 2

d

j=1 v 2

j

1−λj/τ + 1 τ (−µTAµ + 2cTµ − b) − ρ ≥ 0

τ ≥ λmax(A) The first inequality is equivalent to: −

d

  • j=1

v 2

j

τ − λj − µTAµ + 2cTµ − b − ρτ ≥ 0 which is SOCP-representable.

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 24 / 33

slide-77
SLIDE 77

Separation problem: given(ˆ x, ˆ q) ∈ Rd+1, Find a ball B(µ, √ρ) ⊆ Rd − S so as to maximize ρ − (ˆ q − 2µTˆ x + µTµ)

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 25 / 33

slide-78
SLIDE 78

Separation problem: given(ˆ x, ˆ q) ∈ Rd+1, Find a ball B(µ, √ρ) ⊆ Rd − S so as to maximize ρ − (ˆ q − 2µTˆ x + µTµ) = ρ − ˆ x − µ2 − ˆ q + ˆ x2

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 25 / 33

slide-79
SLIDE 79

Separation problem: given(ˆ x, ˆ q) ∈ Rd+1, Find a ball B(µ, √ρ) ⊆ Rd − S so as to maximize ρ − (ˆ q − 2µTˆ x + µTµ) = ρ − ˆ x − µ2 − ˆ q + ˆ x2 Given ρ > 0, ∆(ρ) . = max

µ

ρ − ˆ x − µ2 − ˆ q + ˆ x2 s.t. B(µ, √ρ) ⊆ Rd − S

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 25 / 33

slide-80
SLIDE 80

Separation problem: given(ˆ x, ˆ q) ∈ Rd+1, Find a ball B(µ, √ρ) ⊆ Rd − S so as to maximize ρ − (ˆ q − 2µTˆ x + µTµ) = ρ − ˆ x − µ2 − ˆ q + ˆ x2 Given ρ > 0, ∆(ρ) . = max

µ

ρ − ˆ x − µ2 − ˆ q + ˆ x2 s.t. B(µ, √ρ) ⊆ Rd − S Separation problem: maxρ>0 ∆(ρ)

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 25 / 33

slide-81
SLIDE 81

Separation problem: given(ˆ x, ˆ q) ∈ Rd+1, Find a ball B(µ, √ρ) ⊆ Rd − S so as to maximize ρ − (ˆ q − 2µTˆ x + µTµ) = ρ − ˆ x − µ2 − ˆ q + ˆ x2 Given ρ > 0, ∆(ρ) . = max

µ

ρ − ˆ x − µ2 − ˆ q + ˆ x2 s.t. B(µ, √ρ) ⊆ Rd − S Separation problem: maxρ>0 ∆(ρ) Lemma: ∆(ρ) is a concave function of ρ. Separation problem can be solved using a series of SOCPs.

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 25 / 33

slide-82
SLIDE 82

Indefinite Quadratics

Next we consider an objective of the form min xTMx + vTx + c where M is symmetric but indefinite.

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 26 / 33

slide-83
SLIDE 83

Indefinite Quadratics

Next we consider an objective of the form min xTMx + vTx + c where M is symmetric but indefinite. By changing coordinates and lifting we obtain the equivalent problem: min q − p + vTx + c s.t. q ≥

d

  • i=1

x2

i ,

p ≤

d

  • i=1

λix2

i

where λi > 1 for each i.

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 26 / 33

slide-84
SLIDE 84

Indefinite Quadratics

Define P = { (x, p) ∈ Rd × R : p ≥

d

  • i=1

λix2

i }

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 27 / 33

slide-85
SLIDE 85

Indefinite Quadratics

Define P = { (x, p) ∈ Rd × R : p ≥

d

  • i=1

λix2

i }

The problem can then be stated as: min q − p + vTx + c s.t. q ≥

d

  • i=1

x2

i ,

(x, p) ∈ Rd+1 − int(P)

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 27 / 33

slide-86
SLIDE 86

Indefinite Quadratics

Fix β = maxi λi, and for µ ∈ Rd and φ ∈ R, define M(β, µ, φ) = {(x, p) ∈ Rd+1 : p ≥ βx − µ2 + φ}

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 28 / 33

slide-87
SLIDE 87

Indefinite Quadratics

Fix β = maxi λi, and for µ ∈ Rd and φ ∈ R, define M(β, µ, φ) = {(x, p) ∈ Rd+1 : p ≥ βx − µ2 + φ} This is a paraboloid in the (x, p) space.

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 28 / 33

slide-88
SLIDE 88

Indefinite Quadratics

Fix β = maxi λi, and for µ ∈ Rd and φ ∈ R, define M(β, µ, φ) = {(x, p) ∈ Rd+1 : p ≥ βx − µ2 + φ} This is a paraboloid in the (x, p) space. Then (x, p) ∈ Rd+1 − int(P) if and only if (x, p) ∈ Rd+1 − int(M(β, µ, φ)) for all (β, µ, φ) with M(β, µ, φ) ⊆ P.

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 28 / 33

slide-89
SLIDE 89

Indefinite Quadratics - Separation

Given (ˆ x, ˆ p, ˆ q) ∈ Rd ×R×R, the separation problem can be formulated as: min β ˆ q − 2βµT ˆ x + βµ2 + φ s.t. M(β, µ, φ) ⊆ P

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 29 / 33

slide-90
SLIDE 90

Indefinite Quadratics - Separation

Given (ˆ x, ˆ p, ˆ q) ∈ Rd ×R×R, the separation problem can be formulated as: min β ˆ q − 2βµT ˆ x + βµ2 + φ s.t. M(β, µ, φ) ⊆ P If (µ∗, φ∗) is optimal for this problem, then we get the cut βq − 2βµ∗Tx + βµ∗2 + φ∗ − p ≥ 0

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 29 / 33

slide-91
SLIDE 91

Indefinite Quadratics - Separation

We can reformulate the separation problem as: min β ˆ q − 2

  • βµT ˆ

x + µ2 + φ s.t. M(β, µ/

  • β, φ) ⊆ P

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 30 / 33

slide-92
SLIDE 92

Indefinite Quadratics - Separation

We can reformulate the separation problem as: min β ˆ q − 2

  • βµT ˆ

x + µ2 + φ s.t. M(β, µ/

  • β, φ) ⊆ P

But M(β, µ/√β, φ) ⊆ P iff min

x∈Rd

  • βx − µ/
  • β2 + φ −

d

  • i=1

λix2

i

  • ≥ 0

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 30 / 33

slide-93
SLIDE 93

Indefinite Quadratics - Separation

We need to enforce the constraint min

x∈Rd

  • βx − µ/
  • β2 + φ −

d

  • i=1

λix2

i

  • ≥ 0

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 31 / 33

slide-94
SLIDE 94

Indefinite Quadratics - Separation

We need to enforce the constraint min

x∈Rd

  • βx − µ/
  • β2 + φ −

d

  • i=1

λix2

i

  • ≥ 0

The optimal solution to the minimization problem is any point ¯ x where ¯ xi = µi √β β − λi for i s.t. λi = β

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 31 / 33

slide-95
SLIDE 95

Indefinite Quadratics - Separation

We need to enforce the constraint min

x∈Rd

  • βx − µ/
  • β2 + φ −

d

  • i=1

λix2

i

  • ≥ 0

The optimal solution to the minimization problem is any point ¯ x where ¯ xi = µi √β β − λi for i s.t. λi = β Plugging ¯ x into the objective value and simplifying results in the constraint: φ ≥

  • {i:λi=β}

µ2

i λi

β − λi

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 31 / 33

slide-96
SLIDE 96

Indefinite Quadratics - Separation

We need to enforce the constraint min

x∈Rd

  • βx − µ/
  • β2 + φ −

d

  • i=1

λix2

i

  • ≥ 0

The optimal solution to the minimization problem is any point ¯ x where ¯ xi = µi √β β − λi for i s.t. λi = β Plugging ¯ x into the objective value and simplifying results in the constraint: φ ≥

  • {i:λi=β}

µ2

i λi

β − λi which is SOCP-representable.

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 31 / 33

slide-97
SLIDE 97

Indefinite Quadratics - Separation

The final separation problem is the SOCP: min β ˆ q − 2

  • βµT ˆ

x + µ2 + φ s.t. φ ≥

d

  • {i:λi=β}

µ2

i λi

β − λi

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 32 / 33

slide-98
SLIDE 98

Application: the pooling problem Example: min 2

  • j

xjyj + . . . s.t. . . .

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 33 / 33

slide-99
SLIDE 99

Application: the pooling problem Example: min 2

  • j

xjyj + . . . s.t. . . . But: 2

j xjyj

=

  • j(xj + yj)2 −

j(x2 j + y 2 j )

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 33 / 33

slide-100
SLIDE 100

Application: the pooling problem Example: min 2

  • j

xjyj + . . . s.t. . . . But: 2

j xjyj

=

  • j(xj + yj)2 −

j(x2 j + y 2 j )

General case where α > 0: min

  • j

u2

j

−

  • j

αju2

j

s.t. . . .

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 33 / 33

slide-101
SLIDE 101

Application: the pooling problem Example: min 2

  • j

xjyj + . . . s.t. . . . But: 2

j xjyj

=

  • j(xj + yj)2 −

j(x2 j + y 2 j )

General case where α > 0: min

  • j

u2

j

−

  • j

αju2

j

s.t. . . . equivalently min q − p s.t.

  • j

u2

j ≤ q,

p ≤

  • j

αju2

j

. . .

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 33 / 33

slide-102
SLIDE 102

Application: the pooling problem Example: min 2

  • j

xjyj + . . . s.t. . . . But: 2

j xjyj

=

  • j(xj + yj)2 −

j(x2 j + y 2 j )

General case where α > 0: min

  • j

u2

j

−

  • j

αju2

j

s.t. . . . equivalently min q − p s.t.

  • j

u2

j ≤ q,

p ≤

  • j

αju2

j

. . . → get linear cuts involving all variables

Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 33 / 33