Primary objectives: Convex optimization Ellipsoid method A - - PowerPoint PPT Presentation

Primary objectives:

Convex optimization Ellipsoid method A polynomial algorithm for linear programming

141

PART 6 CONVEX OPTIMIZATION

142

Reminder: Convex functions

Convex function

f : Rn − → R is convex function, if domain of f is convex and for each x,y ∈ dom(f ) and 0 λ 1 one has f (λx+(1−λ)y) λf (x)+(1−λ)f (y)

(x, f(x)) (y, f(y))

Example

· (any norm) is a convex function, since α·x = |α|·x and x+y x+y. Thus λx+(1−λ)y λx+(1−λ)y.

143

Sublevel sets

Definition Cα

f : Rn − → R convex and α ∈ R, Cα = {x ∈ Rn : f (x) α} is α-sublevel set of f .

Lemma 6.1

If f is convex, then Cα is a convex set for each α ∈ R.

Epigraph

f : Rn − → R convex, epi(f ) = {(x,t): x ∈ dom(f ), f (x) t} is epigraph

• f f .

Lemma 6.2

f is convex if and only if epi(f ) is convex set.

144

Convex optimization problem

Convex optimization problem

A convex optimization problem is of the form minimize f0(x) subject to fi(x) bi for i = 1,...,m, where fi, i = 0,...,m are convex functions.

Example: Quadratic programming

Q ∈ Rn×n positive semidefinite, c ∈ Rn A ∈ Rm×n and b ∈ Rm. Convex quadratic program minxTQx+cTx Ax = b x

• 0,

is convex optimization problem.

145

Binary search for minimum

Suppose we can efficiently test whether convex set is empty or

not

Search smallest β ∈ R such that convex set

Cβ = {x ∈ Rn : f0(x) β,f1(x) b1,...,fm(x) bm} is non-empty.

Keep upper bound U and lower bound L Test: Whether C(L+U)/2 = . If yes, then L := (L+U)/2. If no,

then U := (L+U)/2.

After O(log((U −L)/ε) tests, one obtains a value of distance ε

from the optimum value.

146

Separating hyperplane

Theorem 6.3

If S ⊆ Rn is closed and convex and x∗ ∉ S, then there exists a hyperplane cTx = δ such that cTs < δ for each s ∈ S and cTx∗ > δ. x∗

147

Balls and ellipsoids

The unit ball is the set B = {x ∈ Rn | x 1}. An ellipsoid E(A,b) is the image of the unit ball under an affine map t : Rn → Rn with t(x) = Ax+b, where A ∈ Rn×n is an invertible matrix and b ∈ Rn is a vector. Clearly E(A,b) = {x ∈ Rn | A−1x−A−1b 1}. (13)

Exercise

Consider the mapping t(x) = 1 3

2 5

x(1)

x(2)

• . Draw the ellipsoid which is

defined by t. What are the axes of the ellipsoid?

Volume of unit ball

The volume of the unit ball is Vn ∼

1 πn

2eπ

n

n/2. Volume of ellipsoid E(A,b) is equal to |det(A)|·Vn.

148

Lemma 6.4 (Half-Ball Lemma)

The half-ball H = {x ∈ Rn | x 1,x(1) 0} is contained in the ellipsoid E =

• x ∈ Rn |

n+1 n 2 x(1)− 1 n+1 2 + n2 −1 n2

n

• i=2

x(i)2 1

• (14)

x(1) 0

149

Proof

Let x be contained in the unit ball, i.e., x 1 and suppose further that 0 x(1) holds. We need to show that n+1 n 2 x(1)− 1 n+1 2 + n2 −1 n2

n

• i=2

x(i)2 1 (15)

• holds. Since n

i=2 x(i)2 1−x(1)2 holds we have

n+1 n 2 x(1)− 1 n+1 2 + n2 −1 n2

n

• i=2

x(i)2

• n+1

n 2 x(1)− 1 n+1 2 + n2 −1 n2 (1−x(1)2) (16) This shows that (15) holds if x is contained in the half-ball and x(1) = 0 or x(1) = 1.

150

Proof cont.

Now consider the right-hand-side of (16) as a function of x(1), i.e., consider f (x(1)) = n+1 n 2 x(1)− 1 n+1 2 + n2 −1 n2 (1−x(1)2). (17) The first derivative is f ′(x(1)) = 2· n+1 n 2 x(1)− 1 n+1

• −2· n2 −1

n2 x(1). (18) We have f ′(0) < 0 and since both f (0) = 1 and f (1) = 1, we have f (x(1)) 1 for all 0 x(1) 1 and the assertion follows.

151

Corollary 6.5

The half-ball {x ∈ Rn | x(1) 0, x 1} is contained in an ellipsoid E, whose volume is bounded by e−

1 2(n+1) ·Vn.

Ellipsoids: Convenient notation

An ellipsoid E (A,a) is the set

E (A,a) = {x ∈ Rn | (x−a)T A−1(x−a) 1}, where A ∈ Rn×n is a

symmetric positive definite matrix and a ∈ Rn is a vector. Half-ellipsoid: E (A,a)∩(cT x cTa) where c ∈ Rn

152

Half-ellipsoid theorem

Proof of the correctness of next formula can be found in book of Grötschel, Lovász and Schrijver: Geometric algorithms and combinatorial optimization.

Lemma 6.6 (Half-Ellipsoid-Theorem)

The half-ellipsoid E (A,b)∩(cTx cTa) is contained in the ellipsoid

E ′(A′,a′) and one has vol(E ′)/vol(E ) e−1/(2n).

Here E ′(A′,a′) is defined by a′ = a− 1 n+1b (19) A′ = n2 n2 −1

• A−

2 n+1bbT

• ,

(20) where b is the vector b = Ac/

• cTAc.

153

Ellipsoid method

S ⊆ Rn convex compact set. Suppose the following:

I) We have an ellipsoid Einit which contains S. II) We have separation oracle for S Ellipsoid method decides whether vol(S) < L or computes a point x∗ ∈ S

Ellipsoid method

a) (Initialize): Set E (A,a) := Einit b) If vol(E (A,a)) < L, then stop. c) If a ∈ S, then assert S = and stop d) Otherwise, compute inequality cTx β which is valid for S and satisfies cTa > β and replace E (A,a) by E (A′,a) computed with formula (19) and goto step c).

154

Theorem 6.7

The ellipsoid method computes a point in S or asserts that vol(S) < L. The number of iterations is bounded by 2·nln(vol(Einit)/L).

Further remarks

The ellipsoid method can be used to solve convex

programming problems in polynomial time under certain

• conditions. The exact formulation of the result involves some

rounding arguments and is beyond the scope of a lecture on Optimization Methods in Finance. Instead we refer to the book

• f Grötschel, Lovász and Schrijver: Geometric algorithms and

combinatorial optimization for a thorough account.

The ellipsoid algorithm was in particular the first polynomial

time method for linear programming.

155

Primary objectives:

Convex optimization ✔ Ellipsoid method ✔ A polynomial algorithm for linear programming

156