Convex optimization problems (II) Lijun Zhang zlj@nju.edu.cn - - PowerPoint PPT Presentation

Convex optimization problems (II)

Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj

Outline

 Linear Optimization Problems  Quadratic Optimization Problems  Geometric Programming  Generalized Inequality Constraints  Vector Optimization

Linear Optimization Problems

 Linear Program (LP)



and

•  It is common to omit the constant

 Maximization problem with affine objective and constraint functions is also an LP  The feasible set of LP is a polyhedron

Linear Optimization Problems

 Geometric Interpretation of an LP

 The objective 𝑑𝑦 is linear, so its level curves are hyperplanes orthogonal to 𝑑  𝑦∗ is as far as possible in the direction 𝑑

Two Special Cases of LP

 Standard Form LP

 The only inequalities are

 Inequality Form LP

 No equality constraint

Converting to Standard Form

 Conversion

 To use an algorithm for standard LP

 Introduce Slack Variables

Converting to Standard Form

 Decompose  Standard Form LP

𝑦 𝑦 𝑦, 𝑦, 𝑦 ≽ 0

Example

 Diet Problem

 Choose nonnegative quantities 𝑦, … , 𝑦 of 𝑜 foods  One unit of food 𝑘 contains amount 𝑏 of nutrient 𝑗, and costs 𝑑

• 

Healthy diet requires nutrient 𝑗 in quantities at least 𝑐  Determine the cheapest diet that satisfies the nutritional requirements.

Example

 Chebyshev Center of a Polyhedron

 Find the largest Euclidean ball that lies in the polyhedron  The center of the optimal ball is called the Chebyshev center of the polyhedron  Represent the ball as ℬ 𝑦 𝑣| 𝑣 𝑠  𝑦 ∈ 𝐒 and 𝑠 are variables, and we wish to maximize 𝑠 subject to ℬ ∈ 𝒬  ∀𝑦 ∈ ℬ, 𝑏

𝑦 𝑐 ⟺ 𝑏 𝑦 𝑣 𝑐, 𝑣 𝑠 ⟺

𝑏

𝑦 𝑠 𝑏 𝑐

𝒬 𝑦 ∈ 𝐒|𝑏

𝑦 𝑐, 𝑗 1, … , 𝑛

Example

 Chebyshev Center of a Polyhedron

 Find the largest Euclidean ball that lies in the polyhedron  The center of the optimal ball is called the Chebyshev center of the polyhedron  Represent the ball as ℬ 𝑦 𝑣| 𝑣 𝑠  𝑦 ∈ 𝐒 and 𝑠 are variables, and we wish to maximize 𝑠 subject to ℬ ∈ 𝒬 𝒬 𝑦 ∈ 𝐒|𝑏

𝑦 𝑐, 𝑗 1, … , 𝑛

max 𝑠

• s. t.

𝑏

𝑦 𝑠 𝑏 𝑐,

𝑗 1, … , 𝑛

Example

 Chebyshev Inequalities

 is a random variable on

• 
• ,
• 
• is a linear function of

 Prior knowledge is given as  To find a lower bound of

• 𝛽 𝑏

𝑞 𝛾,

𝑗 1, … , 𝑛 min 𝑏

𝑞

• s. t.

𝑞 ≽ 0, 𝟐𝑞 1 𝛽 𝑏

𝑞 𝛾,

𝑗 1, … , 𝑛

Example

 Piecewise-linear Minimization

 Consider the (unconstrained) problem  The epigraph problem  An LP problem

,…,

• ,…,

Linear-fractional Programming

 Linear-fractional Program

 The objective function is a ratio of affine functions  The domain is  A quasiconvex optimization problem

min 𝑔

𝑦

• s. t.

𝐻𝑦 ≼ ℎ 𝐵𝑦 𝑐 𝑔

𝑦 𝑑𝑦 𝑒

𝑓𝑦 𝑔 dom 𝑔

𝑦|𝑓𝑦 𝑔 0

Linear-fractional Programming

 Transforming to a linear program

 Proof

min 𝑑𝑧 𝑒𝑨

• s. t.

𝐻𝑧 ℎ𝑨 ≼ 0 𝐵𝑧 𝑐𝑨 0 𝑓𝑧 𝑔𝑨 1 𝑨 0 min 𝑔

𝑦 𝑑𝑦 𝑒

𝑓𝑦 𝑔

• s. t.

𝐻𝑦 ≼ ℎ 𝐵𝑦 𝑐

𝑦 is feasible in LFP ⇒ 𝑧

• , 𝑨
• is feasible

in LP , 𝑑𝑧 𝑒𝑨 𝑔

𝑦 ⇒ the optimal value of LFP is

greater than or equal to the optimal value of LP

Linear-fractional Programming

 Transforming to a linear program

 Proof

min 𝑑𝑧 𝑒𝑨

• s. t.

𝐻𝑧 ℎ𝑨 ≼ 0 𝐵𝑧 𝑐𝑨 0 𝑓𝑧 𝑔𝑨 1 𝑨 0 min 𝑔

𝑦 𝑑𝑦 𝑒

𝑓𝑦 𝑔

• s. t.

𝐻𝑦 ≼ ℎ 𝐵𝑦 𝑐

𝑧, 𝑨 is feasible in LP and 𝑨 0 ⇒ 𝑦 𝑧 𝑨 ⁄ is feasible in LFP , 𝑔

𝑦 𝑑𝑧 𝑒𝑨 ⇒ the optimal value of FLP is

less than or equal to the optimal value of LP 𝑧, 𝑨 is feasible in LP , 𝑨 0 and 𝑦 is feasible in LFP ⇒ 𝑦 𝑦 𝑢𝑧 is feasible in LFP for all 𝑢 0, lim

→ 𝑔 𝑦 𝑢𝑧 𝑑𝑧 𝑒𝑨

Generalized Linear-fractional Programming

 Generalized Linear-fractional Program



•  A quasiconvex optimization problem

 Von Neumann Growth Problem

𝑔

𝑦 max ,…,

𝑑

𝑦 𝑒

𝑓

𝑦 𝑔

• max

min

,…, 𝑦 𝑦

⁄

• s. t.

𝑦 ≽ 0 𝐶𝑦 ≼ 𝐵𝑦

Generalized Linear-fractional Programming

 Von Neumann Growth Problem

 𝑦, 𝑦 ∈ 𝐒: activity levels of 𝑜 sectors, in current and next period  𝐵𝑦 , 𝐶𝑦 : produced and consumed amounts of good 𝑗  𝐶𝑦 ≼ 𝐵𝑦: goods consumed in the next period cannot exceed the goods produced in the current period  𝑦

𝑦

⁄ growth rate of sector 𝑗

max min

,…, 𝑦 𝑦

⁄

• s. t.

𝑦 ≽ 0 𝐶𝑦 ≼ 𝐵𝑦

Outline

 Linear Optimization Problems  Quadratic Optimization Problems  Geometric Programming  Generalized Inequality Constraints  Vector Optimization

Quadratic Optimization Problems

 Quadratic Program (QP)



• and
•  The objective function is (convex)

quadratic  The constraint functions are affine  When , QP becomes LP

min 1 2 ⁄ 𝑦𝑄𝑦 𝑟𝑦 𝑠

• s. t.

𝐻𝑦 ≼ ℎ 𝐵𝑦 𝑐

Quadratic Optimization Problems

 Geometric Illustration of QP

 The feasible set 𝒬 is a polyhedron  The contour lines of the objective function are shown as dashed curves.

Quadratic Optimization Problems

 Quadratically Constrained Quadratic Program (QCQP)

 𝑄 ∈ 𝐓

, 𝑗 0, … , 𝑛

 The inequality constraint functions are (convex) quadratic  The feasible set is the intersection of ellipsoids (when 𝑄 ≻ 0) and an affine set  Include QP as a special case min 1 2 ⁄ 𝑦𝑄𝑦 𝑟

𝑦 𝑠

• s. t.

1 2 ⁄ 𝑦𝑄𝑦 𝑟

𝑦 𝑠 0,

𝑗 1, … , 𝑛 𝐵𝑦 𝑐

Examples

 Least-squares and Regression

 Analytical solution: 𝑦 𝐵𝑐  Can add linear constraints, e.g., 𝑚 ≼ 𝑦 ≼ 𝑣

 Distance Between Polyhedra

 Find the distance between the polyhedra 𝒬

𝑦|𝐵𝑦 ≼ 𝑐 and 𝒬 𝑦|𝐵𝑦 ≼ 𝑐

min 𝐵𝑦 𝑐

𝑦𝐵𝐵𝑦 2𝑐𝐵𝑦 𝑐𝑐

• dist 𝒬

, 𝒬 inf

𝑦 𝑦 𝑦 ∈ 𝒬

, 𝑦 ∈ 𝒬

Example

 Bounding Variance

 is a random variable on

• 
• ,
•  The variance of a random variable

 Maximize the variance

𝐅𝑔 𝐅𝑔 𝑔

• 𝑞
• 𝑔

𝑞

• max
• 𝑔
• 𝑞
• 𝑔

𝑞

• s. t.

𝑞 ≽ 0, 𝟐𝑞 1 𝛽 𝑏

𝑞 𝛾, 𝑗 1, … , 𝑛

Second-order Cone Programming

 Second-order Cone Program (SOCP)

 𝐵 ∈ 𝐒, 𝐺 ∈ 𝐒  Second-order Cone (SOC) constraint: 𝐵𝑦 𝑐 𝑑𝑦 𝑒 where 𝐵 ∈ 𝐒, is same as requiring 𝐵𝑦 𝑐, 𝑑𝑦 𝑒 ∈ SOC in 𝐒 min 𝑔𝑦

• s. t.

𝐵𝑦 𝑐

𝑑 𝑦 𝑒,

𝑗 1, … , 𝑛 𝐺𝑦 𝑕 SOC

• 𝑦, 𝑢 ∈ 𝐒| 𝑦 𝑢
• 𝑦

𝑢 | 𝑦 𝑢

𝐽

1 𝑦 𝑢 0, 𝑢 0

Second-order Cone Programming

 Second-order Cone Program (SOCP)

 𝐵 ∈ 𝐒, 𝐺 ∈ 𝐒  Second-order Cone (SOC) constraint: 𝐵𝑦 𝑐 𝑑𝑦 𝑒 where 𝐵 ∈ 𝐒, is same as requiring 𝐵𝑦 𝑐, 𝑑𝑦 𝑒 ∈ SOC in 𝐒  If 𝑑 0, 𝑗 1, … , 𝑛, it reduces to QCQP by squaring each inequality constraint  More general than QCQP and LP min 𝑔𝑦

• s. t.

𝐵𝑦 𝑐

𝑑 𝑦 𝑒,

𝑗 1, … , 𝑛 𝐺𝑦 𝑕

Example

 Robust Linear Programming

 There can be uncertainty in 𝑏  Assume 𝑏 are known to lie in ellipsoids  The constraints must hold for all 𝑏 ∈ ℰ

• ,

Example

 Note that  Robust linear constraint  QCQP

sup 𝑏

𝑦 𝑏 ∈ ℰ 𝑏

• 𝑦 sup𝑣𝑄

𝑦| 𝑣 1

𝑏

• 𝑦 𝑄

𝑦

𝑏

• 𝑦 𝑄

𝑦 𝑐

min 𝑑𝑦

• s. t.

𝑏

• 𝑦 𝑄

𝑦 𝑐,

𝑗 1, … , 𝑛

Example

 Linear Programming with Random Constraints

 Suppose that

is independent

Gaussian random vectors with mean

and covariance

•  Require each constraint
• holds

with probability exceeding

min 𝑑𝑦

• s. t.

prob 𝑏

𝑦 𝑐 𝜃,

𝑗 1, … , 𝑛

Example

 Linear Programming with Random Constraints

 Analysis

where Φ 𝑨 1 2𝜌 ⁄

• 𝑓

⁄ 𝑒𝑢

• min

𝑑𝑦

• s. t.

𝑏

• 𝑦 Φ 𝜃

Σ

⁄ 𝑦 𝑐,

𝑗 1, … , 𝑛

prob 𝑏

𝑦 𝑐 prob

• ⁄
• ⁄
• 𝜃 ⟺
• ⁄
• Φ 𝜃 ⟺ 𝑏
• 𝑦 Φ 𝜃

Σ

⁄ 𝑦 𝑐

Outline

 Linear Optimization Problems  Quadratic Optimization Problems  Geometric Programming  Generalized Inequality Constraints  Vector Optimization

Definitions

 Monomial Function



• ,
• ,

and

•  Closed under multiplication, division, and

nonnegative scaling.

 Posynomial Function

 Closed under addition, multiplication, and nonnegative scaling

• 𝑔 𝑦 𝑑𝑦

𝑦 … 𝑦

Geometric Programming (GP)

 The Problem



• are posynomials



• are monomials

 Domain of the problem  Implicit constraint:

• 𝒠 𝐒

Extensions of GP

 is a posynomial and is a monomial 

and are nonzero monomials

 Maximize a nonzero monomial objective function by minimizing its inverse

𝑔 𝑦 ℎ 𝑦 ⇔ 𝑔 𝑦 ℎ 𝑦 1 ℎ 𝑦 ℎ 𝑦 ⇔ ℎ 𝑦 ℎ 𝑦 1

max 𝑦 𝑧 ⁄

• s. t.

2 𝑦 3 𝑦 3𝑧 𝑨 ⁄ 𝑧 𝑦 𝑧 ⁄ 𝑨 ⇔ min 𝑦𝑧

• s. t.

2𝑦 1, 1 3 ⁄ 𝑦 1 𝑦𝑧

⁄ 𝑧 ⁄ 𝑨 1

𝑦𝑧𝑨 1

GP in Convex Form

 Change of Variables

• 

is the monomial function  is the posynomial function

𝑔 𝑦 𝑑𝑦

𝑦 … 𝑦 ,

𝑦 𝑓 𝑔 𝑦 𝑔 𝑓, … , 𝑓 𝑑 𝑓 ⋯ 𝑓 𝑓⋯ 𝑓 𝑔 𝑦 𝑑𝑦

𝑦 … 𝑦

• 𝑔 𝑦

𝑓

GP in Convex Form

 New Form

 Taking the Logarithm

min

• 𝑓
• s. t.
• 𝑓
• 1,

𝑗 1, … , 𝑛 𝑓

1,

𝑗 1, … , 𝑞 min 𝑔

• 𝑧 log

𝑓

• s. t.

𝑔

• 𝑧 log

𝑓

• 0,

𝑗 1, … , 𝑛 ℎ 𝑧 𝑕

𝑧 ℎ 0,

𝑗 1, … , 𝑞

Example

 Frobenius Norm Diagonal Scaling

 Given a matrix

•  Choose a diagonal matrix

such that

is small

 Unconstrained GP

𝐸𝑁𝐸

• tr

𝐸𝑁𝐸 𝐸𝑁𝐸 𝐸𝑁𝐸

• ,

𝑁

𝑒 𝑒

• ,

min 𝑁

𝑒 𝑒

• ,

Outline

 Linear Optimization Problems  Quadratic Optimization Problems  Geometric Programming  Generalized Inequality Constraints  Vector Optimization

Generalized Inequality Constraints

 Convex Optimization Problem with Generalized Inequality Constraints



• is convex;



• are proper cones



• is
• convex w.r.t. proper

cone

Generalized Inequality Constraints

 Convex Optimization Problem with Generalized Inequality Constraints

 The feasible set, any sublevel set, and the optimal set are convex  Any locally optimal is globally optimal  The optimality condition for differentiable holds without change

Conic Form Problems

 Conic Form Problems

 A linear objective  One inequality constraint function which is affine  A generalization of linear programs

min 𝑑𝑦

• s. t.

𝐺𝑦 𝑕 ≼ 0 𝐵𝑦 𝑐

Conic Form Problems

 Conic Form Problems  Standard Form  Inequality Form

min 𝑑𝑦

• s. t.

𝐺𝑦 𝑕 ≼ 0 𝐵𝑦 𝑐 min 𝑑𝑦

• s. t.

𝑦 ≽ 0 𝐵𝑦 𝑐 min 𝑑𝑦

• s. t.

𝐺𝑦 𝑕 ≼ 0

Semidefinite Programming

 Semidefinite Program (SDP)



• 
• and
•  Linear matrix inequality (LMI)

 If

• are all diagonal, LMI is

equivalent to a set of linear inequalities, and SDP reduces to LP

min 𝑑𝑦

• s. t.

𝑦𝐺

⋯ 𝑦𝐺 𝐻 ≼ 0

𝐵𝑦 𝑐

Semidefinite Programming

 Standard From SDP

 𝑌 ∈ 𝐓 is the variable and 𝐷, 𝐵, … , 𝐵 ∈ 𝐓  𝑞 linear equality constraints  A nonnegativity constraint

 Inequality Form SDP

 𝐶, 𝐵, … , 𝐵 ∈ 𝐓 and no equality constraint min tr 𝐷𝑌

• s. t.

tr 𝐵𝑌 𝑐, 𝑗 1, … , 𝑞 𝑌 ≽ 0 min 𝑑𝑦

• s. t.

𝑦𝐵 ⋯ 𝑦𝐵 ≼ 𝐶

Semidefinite Programming

 Multiple LMIs and Linear Inequalities

 It is referred as SDP as well

 Be transformed as

 A standard SDP

min 𝑑𝑦

• s. t.

𝐺 𝑦 𝑦𝐺

• ⋯ 𝑦𝐺
• 𝐻 ≼ 0, 𝑗 1, … , 𝐿

𝐻𝑦 ≼ ℎ, 𝐵𝑦 𝑐 min 𝑑𝑦

• s. t.

diag 𝐻𝑦 ℎ, 𝐺 𝑦 , … , 𝐺 𝑦 ≼ 0 𝐵𝑦 𝑐

Examples

 Second-order Cone Programming

 A conic form problem in which

min 𝑑𝑦

• s. t.

𝐵𝑦 𝑐

𝑑 𝑦 𝑒,

𝑗 1, … , 𝑛 𝐺𝑦 𝑕 min 𝑑𝑦

• s. t.

𝐵𝑦 𝑐, 𝑑

𝑦 𝑒 ≼ 0,

𝑗 1, … , 𝑛 𝐺𝑦 𝑕 𝐿 𝑧, 𝑢 ∈ 𝐒| 𝑧 𝑢

Example

 Matrix Norm Minimization



• and
•  Fact:

 A New Problem



• is matrix convex

min 𝐵𝑦 𝜇𝐵 𝑦 𝐵𝑦

⁄

𝐵 𝑡 ⇔ 𝐵𝐵 ≼ 𝑡𝐽 min 𝑡

• s. t.

𝐵 𝑦 𝐵𝑦 ≼ 𝑡𝐽 ⇔ min 𝑡

• s. t.

𝐵 𝑦 𝐵𝑦 𝑡𝐽 ≼ 0

Example

 Matrix Norm Minimization



• and
•  Fact:

 SDP

 A single linear matrix inequality

min 𝐵𝑦 𝜇𝐵 𝑦 𝐵𝑦

⁄

𝐵 𝑢 ⇔ 𝐵𝐵 ≼ 𝑢𝐽 ⇔ 𝑢𝐽 𝐵 𝐵 𝑢𝐽 ≽ 0 min 𝑢

• s. t.

𝑢𝐽 𝐵 𝑦 𝐵 𝑦 𝑢𝐽 ≽ 0

Outline

 Linear Optimization Problems  Quadratic Optimization Problems  Geometric Programming  Generalized Inequality Constraints  Vector Optimization

General and Convex Vector Optimization Problems

 General Vector Optimization Problem

 𝑔

: 𝐒 → 𝐒 is a vector-valued objective

function  𝐿 ∈ 𝐒 is a proper cone, which is used to compare objective values  𝑔

: 𝐒 → 𝐒 are the inequality constraint

functions  ℎ: 𝐒 → 𝐒 are the equality constraint functions min w. r. t. 𝐿 𝑔

𝑦

• s. t.

𝑔

𝑦 0,

𝑗 1, … , 𝑛 ℎ 𝑦 0, 𝑗 1, … , 𝑞

General and Convex Vector Optimization Problems

 Convex Vector Optimization Problem



• is
• convex



• are convex



• are affine

 is better than or equal to

 Could be incomparable

min w. r. t. 𝐿 𝑔

𝑦

• s. t.

𝑔

𝑦 0,

𝑗 1, … , 𝑛 ℎ 𝑦 0, 𝑗 1, … , 𝑞 𝑔

𝑦 ≼ 𝑔 𝑧

Optimal Points and Values

 Achievable Objective Values  If has a minimum element

 is optimal and is the optimal value



∗ is optimal if and only if it is feasible

and

𝒫 𝑔

𝑦 |∃𝑦 ∈ 𝒠, 𝑔 𝑦 0, 𝑗 1, … , 𝑛, ℎ 𝑦 0, 𝑗 1, … , 𝑞

𝒫 ⊆ 𝑔

𝑦⋆ 𝐿

Optimal Points and Values

 Achievable Objective Values  If has a minimum element

 is optimal and is the optimal value



• 𝒫 𝑔

𝑦 |∃𝑦 ∈ 𝒠, 𝑔 𝑦 0, 𝑗 1, … , 𝑛, ℎ 𝑦 0, 𝑗 1, … , 𝑞

𝒫 ⊆ 𝑔

𝑦⋆ 𝐿

Example

 Best Linear Unbiased Estimator

 Suppose that , where

• is noise,

and

•  Estimate

from and  Assume that has rank , and

•  A linear estimator

 If , is an unbiased linear estimator of , i.e.,

Example

 Best Linear Unbiased Estimator

 The error covariance of an unbiased estimator

 Minimize the covariance

 Solution

𝐅 𝑦 𝑦 𝑦 𝑦 𝐅𝐺𝑤𝑤𝐺 𝐺𝐺 min w. r. t. 𝐓

• 𝐺𝐺
• s. t.

𝐺𝐵 𝐽 𝐺⋆ 𝐵 𝐵𝐵 𝐵 𝐺⋆𝐺⋆ 𝐵𝐵

Pareto Optimal Points and Values

 Achievable Objective Values 

• is a minimal element of

 is Pareto optimal 

• is a Pareto optimal value

 is Pareto optimal if and only if it is feasible and

𝒫 𝑔

𝑦 |∃𝑦 ∈ 𝒠, 𝑔 𝑦 0, 𝑗 1, … , 𝑛, ℎ 𝑦 0, 𝑗 1, … , 𝑞

𝑔

𝑦 𝐿 ⋂𝒫 𝑔 𝑦

Pareto Optimal Points and Values

 Achievable Objective Values 

• is a minimal element of

 is Pareto optimal 

• is a Pareto optimal value



• 𝒫 𝑔

𝑦 |∃𝑦 ∈ 𝒠, 𝑔 𝑦 0, 𝑗 1, … , 𝑛, ℎ 𝑦 0, 𝑗 1, … , 𝑞

𝑔

𝑦 𝐿 ⋂𝒫 𝑔 𝑦

Pareto Optimal Points and Values

 Achievable Objective Values 

• is a minimal element of

 is Pareto optimal 

• is a Pareto optimal value

 is Pareto optimal if and only if it is feasible and  Let be the set of Pareto optimal values

𝒫 𝑔

𝑦 |∃𝑦 ∈ 𝒠, 𝑔 𝑦 0, 𝑗 1, … , 𝑛, ℎ 𝑦 0, 𝑗 1, … , 𝑞

𝑔

𝑦 𝐿 ⋂𝒫 𝑔 𝑦

𝑄 ⊆ 𝒫 ∩ bd𝒫

Scalarization

 A standard technique for finding Pareto optimal (or optimal) points  Find Pareto optimal points for any vector optimization problem by solving the ordinary scalar

• ptimization problem

 Characterization of minimum and minimal points via dual generalized inequalities

Dual Characterization of Minimal Elements (1)

 If

∗

, and minimizes

• ver

, then is minimal.

Scalarization

 Choose any

∗

 The optimal point for this scalar problem is Pareto optimal for the vector

• ptimization problem

 is called the weight vector  By varying we obtain (possibly) different Pareto optimal solutions

min 𝜇𝑔

𝑦

• s. t.

𝑔

𝑦 0,

𝑗 1, … , 𝑛 ℎ 𝑦 0, 𝑗 1, … , 𝑞

Scalarization



•  Scalarization cannot find

Scalarization of Convex Vector Optimization Problems

 Choose any

∗

 A convex optimization problem  The optimal point for this scalar problem is Pareto optimal for the vector

• ptimization problem

 is called the weight vector  By varying we obtain (possibly) different Pareto optimal solutions

min 𝜇𝑔

𝑦

• s. t.

𝑔

𝑦 0,

𝑗 1, … , 𝑛 ℎ 𝑦 0, 𝑗 1, … , 𝑞

Dual Characterization of Minimal Elements (2)

 If is convex, for any minimal element there exists a nonzero

∗

such that minimizes over .

Scalarization of Convex Vector Optimization Problems

 For every Pareto optimal point

,

there is some nonzero

∗

such that

is a solution of the scalarized

problem  It is not true that every solution of the scalarized problem, with

∗

and , is a Pareto optimal point for the vector problem

min 𝜇𝑔

𝑦

• s. t.

𝑔

𝑦 0,

𝑗 1, … , 𝑛 ℎ 𝑦 0, 𝑗 1, … , 𝑞

Scalarization of Convex Vector Optimization Problems

• 1. Consider all

∗

 Solve the above problem

• 2. Consider all

∗

, ,

∗

 Solve the above problem  Verify the solution

min 𝜇𝑔

𝑦

• s. t.

𝑔

𝑦 0,

𝑗 1, … , 𝑛 ℎ 𝑦 0, 𝑗 1, … , 𝑞

Example

 Minimal Upper Bound on a Set of Matrices



•  The constraints mean that

is an upper bound on

•  A Pareto optimal solution is a minimal

upper bound on the matrices

min w. r. t. 𝐓

• 𝑌
• s. t.

𝑌 ≽ 𝐵, 𝑗 1. … , 𝑛

Example

 Scalarization



•  An SDP

 If is Pareto optimal for the vector problem then it is optimal for the SDP, for some nonzero weight matrix .

min tr𝑋𝑌

• s. t.

𝑌 ≽ 𝐵, 𝑗 1. … , 𝑛

Example

 A Simple Geometric Interpretation

 Define an ellipsoid centered at the

• rigin as
• 

Multicriterion Optimization



•  𝑔

consists of 𝑟 different objectives 𝐺 and we

want to minimize all 𝐺

•  It is convex if 𝑔

, … , 𝑔 are convex, ℎ, … , ℎ

are affine, and 𝐺

, … , 𝐺 are convex

 Feasible 𝑦⋆ is optimal if  Feasible 𝑦 is Pareto optimal if 𝑔

𝑦 𝐺 𝑦 , … , 𝐺 𝑦

𝑧 is feasible ⇒ 𝑔

𝑦⋆ ≼ 𝑔 𝑧

𝑧 is feasible, 𝑔

𝑧 ≼ 𝑔 𝑦 ⇒ 𝑔 𝑦 𝑔 𝑧

Example

 Regularized Least-Squares



• measures the misfit



• measures the size

 Our goal is to find that gives a good fit and that is not large

 Scalarization

min w. r. t. 𝐒

𝑔 𝑦 𝐺 𝑦 , 𝐺 𝑦

𝜇𝑔

𝑦 𝜇𝐺 𝑦 𝜇𝐺 𝑦

𝑦 𝜇𝐵𝐵 𝜇𝐽 𝑦 2𝜇𝑐𝐵𝑦 𝜇𝑐𝑐

Example

 Solution



• 

, we

get  With , we get

• 𝑦 𝜈 𝜇𝐵𝐵 𝜇𝐽 𝜇𝐵𝑐 𝐵𝐵 𝜈𝐽 𝐵𝑐

Summary

 Linear Optimization Problems  Quadratic Optimization Problems  Geometric Programming  Generalized Inequality Constraints  Vector Optimization