Convex Optimization 2. Convex Sets Prof. Ying Cui Department of - - PowerPoint PPT Presentation

convex optimization
SMART_READER_LITE
LIVE PREVIEW

Convex Optimization 2. Convex Sets Prof. Ying Cui Department of - - PowerPoint PPT Presentation

Convex Optimization 2. Convex Sets Prof. Ying Cui Department of Electrical Engineering Shanghai Jiao Tong University 2018 SJTU Ying Cui 1 / 33 Outline Affine and convex sets Some important examples Operations that preserve convexity


slide-1
SLIDE 1

Convex Optimization

  • 2. Convex Sets
  • Prof. Ying Cui

Department of Electrical Engineering Shanghai Jiao Tong University

2018

SJTU Ying Cui 1 / 33

slide-2
SLIDE 2

Outline

Affine and convex sets Some important examples Operations that preserve convexity Generalized inequalities Separating and supporting hyperplanes Dual cones and generalized inequalities

SJTU Ying Cui 2 / 33

slide-3
SLIDE 3

Lines and line segments

◮ line passing through two distinct points x1, x2 ∈ Rn, x1 = x2: points of the form

◮ sum of x1 scaled by θ and x2 scaled by 1 − θ y = θx1 + (1 − θ)x2, θ ∈ R ◮ sum of base point x2 and direction x1 − x2 scaled by θ y = x2 + θ(x1 − x2), θ ∈ R

◮ line segment between two distinct points x1, x2 ∈ Rn, x1 = x2: points of the form y = θx1 + (1 − θ)x2, θ ∈ [0, 1]

x1 x2 θ = 1.2 θ = 1 θ = 0.6 θ = 0 θ = −0.2 Figure 2.1 The line passing through x1 and x2 is described parametrically by θx1 + (1 − θ)x2, where θ varies over R. The line segment between x1 and x2, which corresponds to θ between 0 and 1, is shown darker.

SJTU Ying Cui 3 / 33

slide-4
SLIDE 4

Affine sets

◮ affine set: an affine set C ⊆ Rn contains the line through any two distinct points in C

◮ ∀x1, x2 ∈ C and θ ∈ R, θx1 + (1 − θ)x2 ∈ C

◮ affine combination of points x1, · · · , xk: a point of the form θ1x1 + · · · + θkxk, where θ1, · · · , θk ∈ R and θ1 + · · · + θk = 1

◮ an affine set contains every affine combination of its points

◮ affine hull of set C ⊆ Rn: the set of all affine combinations of points in C, i.e., affC = {θ1x1 + · · · + θkxk|x1, · · · , xk ∈ C, θ1 + · · · + θk = 1}

◮ affC is the smallest affine set that contains C: if S is any affine set with C ⊆ S, then affC ⊆ S

◮ example: empty set ∅, any single point (i.e., singleton) {x0}, line, hyperplane, whole space Rn, solution set of linear equations C = {x|Ax = b}

◮ solution set of a system of linear equations is an affine set ◮ every affine set can be expressed as the solution set of a system of linear equations

SJTU Ying Cui 4 / 33

slide-5
SLIDE 5

Convex sets

◮ convex set: a convex set C ⊆ Rn contains the line segment between any two distinct points in C

◮ ∀x1, x2 ∈ C and θ ∈ [0, 1], θx1 + (1 − θ)x2 ∈ C ◮ every affine set is also convex

◮ convex combination of points x1, · · · , xk: a point of the form θ1x1 + · · · + θkxk, where θ1, · · · , θk ≥ 0 and θ1 + · · · + θk = 1

◮ can be generalized to include infinite sums, integrals, and probability distributions ◮ a convex set contains every convex combination of its points

◮ convex hull of set C ⊆ Rn: the set of all convex combinations

  • f points in C, i.e., convC = {θ1x1 + · · · + θkxk|xi ∈ C, θi ≥

0, i = 1, · · · , k, θ1 + · · · + θk = 1}

◮ convC is the smallest convex set that contains C: if B is any convex set with C ⊆ B, then convC ⊆ B

◮ example: line segment, ray

Figure 2.2 Some simple convex and nonconvex sets. Left. The hexagon, which includes its boundary (shown darker), is convex. Middle. The kidney shaped set is not convex, since the line segment between the two points in the set shown as dots is not contained in the set. Right. The square contains some boundary points but not others, and is not convex. Figure 2.3 The convex hulls of two sets in R2. Left. The convex hull of a set of fifteen points (shown as dots) is the pentagon (shown shaded). Right. The convex hull of the kidney shaped set in figure 2.2 is the shaded set.

SJTU Ying Cui 5 / 33

slide-6
SLIDE 6

Cones

◮ cone (or nonnegative homogeneous): a set C ⊆ Rn satisfies

◮ ∀x ∈ C and θ ≥ 0, θx ∈ C

◮ convex cone: a convex cone C ⊆ Rn is convex and a cone

◮ ∀x1, x2 ∈ C and θ1, θ2 ≥ 0, θ1x1 + θ2x2 ∈ C

◮ conic combination (or nonnegative linear combination) of points x1, · · · , xk: a point of the form θ1x1 + · · · + θkxk, where θ1, · · · , θk ≥ 0

◮ can be generalized to include infinite sums and integrals ◮ a convex cone contains all conic combinations of its elements

◮ conic hull of set C ⊆ Rn: set of all conic combinations of points in C, {θ1x1 + · · · + θkxk|xi ∈ C, θi ≥ 0, i = 1, · · · , k}

◮ the smallest convex cone that contains C

◮ example: subspace, line passing through origin, ray with base

  • rigin

x1 x2 Figure 2.4 The pie slice shows all points of the form θ1x1 + θ2x2, where θ1, θ2 ≥ 0. The apex of the slice (which corresponds to θ1 = θ2 = 0) is at 0; its edges (which correspond to θ1 = 0 or θ2 = 0) pass through the points x1 and x2. Figure 2.5 The conic hulls (shown shaded) of the two sets of figure 2.3.

SJTU Ying Cui 6 / 33

slide-7
SLIDE 7

Hyperplanes and halfspaces

◮ hyperplane: set of form {x|aTx = b} (a ∈ Rn, a = 0, b ∈ R)

◮ analytical interpretation: solution set of a nontrivial linear equation

◮ hyperplane is an affine set

◮ geometric interpretation:

◮ set of points with constant inner product b to given vector a ◮ hyperplane with normal vector a and offset from the origin determined by b: an offset x0 plus all vectors orthogonal to normal vector a {x|aT (x − x0) = 0} = x0 + {v|aT v = 0}

  • a⊥

where x0 is any point in the hyperplane satisfying aTx0 = b and a⊥ denotes the orthogonal complement of a

a x aT x = b x0 Figure 2.6 Hyperplane in R2, with normal vector a and a point x0 in the

  • hyperplane. For any point x in the hyperplane, x − x0 (shown as the darker

arrow) is orthogonal to a.

SJTU Ying Cui 7 / 33

slide-8
SLIDE 8

Hyperplanes and halfspaces

◮ halfspace: set of form {x|aT x ≤ b} (a ∈ Rn, a = 0, b ∈ R)

◮ analytical interpretation: solution set of a nontrivial linear inequality

◮ halfspace is not an affine set, but a convex set

◮ geometric interpretation: an offset x0 plus all vectors making an obtuse (or right) angle with outward normal vector a {x|aT(x − x0) ≤ 0} = x0 + {v|aTv ≤ 0} where x0 is any point in the hyperplane satisfying aTx0 = b

a aT x ≥ b aT x ≤ b x0 Figure 2.7 A hyperplane defined by aT x = b in R2 determines two halfspaces. The halfspace determined by aT x ≥ b (not shaded) is the halfspace extending in the direction a. The halfspace determined by aT x ≤ b (which is shown shaded) extends in the direction −a. The vector a is the outward normal of this halfspace.

◮ a hyperplane divides Rn into two halfspaces ◮ the boundary of a halfspace is a hyperplane

SJTU Ying Cui 8 / 33

slide-9
SLIDE 9

Euclidean balls and ellipsoids

◮ (Euclidean) ball with center xc and radius r: B(xc, r) = {x| ||x − xc||2 ≤ r} = {xc + ru| ||u||2 ≤ 1}

◮ ||u||2 = (uT u)1/2 denotes the Euclidean norm (l2 norm) ◮ convex set

◮ ellipsoid with center xc: E = {x| (x − xc)T P−1(x − xc) ≤ 1}, where P ∈ Sn

++

E = {xc + Au| ||u||2 ≤ 1}, where A ∈ Sn

++

◮ P determines how far the ellipsoid extends in every direction from xc, the lengths of the semi-axes of the ellipsoid are given by √λi, where λi are eigenvalues of P

◮ when P = r 2I, ellipsoid becomes ball

◮ when A = P1/2, two representations are the same ◮ convex set

xc Figure 2.9 An ellipsoid in R2, shown shaded. The center xc is shown as a dot, and the two semi-axes are shown as line segments.

SJTU Ying Cui 9 / 33

slide-10
SLIDE 10

Norm balls and norm cones

◮ norm: a function || · || : Rn → R (measure of length of vector)

◮ nonnegative: ||x|| ≥ 0 for all x ∈ Rn ◮ definite: ||x|| = 0 only if x = 0 ◮ homogeneous: ||tx|| = |t|||x||, for all x ∈ Rn and t ∈ R ◮ triangle inequality: ||x + y|| ≤ ||x|| + ||y||, for all x, y ∈ Rn

◮ norm ball with center xc and radius r: {x| ||x − xc|| ≤ r}

◮ convex set

◮ norm cone: {(x, t)| ||x|| ≤ t} ⊆ Rn+1

◮ convex cone ◮ second-order (Euclidean norm) cone , i.e., {(x, t)| ||x||2 ≤ t}

x1 x2 t −1 1 −1 1 0.5 1 Figure 2.10 Boundary of second-order cone in R3, {(x1, x2, t) | (x2

1+x2 2)1/2 ≤

t}.

SJTU Ying Cui 10 / 33

slide-11
SLIDE 11

Polyhedra

◮ polyhedron: solution set of a finite number of linear inequalities and equalities (can be bounded or unbounded) P ={x|aT

j x ≤ bj, j = 1, · · · , m, cT j x = dj, j = 1, · · · , p}

={x|Ax b, Cx = d} where A =    aT

1

. . . aT

m

  , C =    cT

1

. . . cT

p

  , denotes vector inequality

◮ intersection of a finite number of halfspaces and hyperplanes

◮ affine sets (e.g., subspaces, hyperplanes, lines), rays, line segments, and halfspaces are all polyhedra

a1 a2 a3 a4 a5 P Figure 2.11 The polyhedron P (shown shaded) is the intersection of five halfspaces, with outward normal vectors a1, . . . . , a5.

SJTU Ying Cui 11 / 33

slide-12
SLIDE 12

Positive semidefinite cone

◮ Sn = {X ∈ Rn×n|X = X T}: set of symmetric n × n matrices

◮ vector space with dimension n(n + 1)/2 ◮ convex cone

◮ Sn

+ = {X ∈ Sn|X 0}: set of symmetric positive semidefinite

n × n matrices, X ∈ Sn

+ ⇐

⇒ zTXz ≥ 0 for all z ∈ Rn

◮ convex cone, referred to as positive semidefinite cone ◮ example: X = x y y z

  • ∈ S2

+ ⇐

⇒ x ≥ 0, z ≥ 0, xz ≥ y 2

x y z 0.5 1 −1 1 0.5 1 Figure 2.12 Boundary of positive semidefinite cone in S2.

◮ Sn

++ = {X ∈ Sn|X ≻ 0}: set of symmetric positive definite

n × n matrices, X ∈ Sn

++ ⇐

⇒ zTXz > 0 for all z ∈ Rn, z = 0

◮ not cone (as origin not in it)

SJTU Ying Cui 12 / 33

slide-13
SLIDE 13

Determine or establish convexity of sets

◮ apply definition: a set C is convex if for all x1, x2 ∈ C and θ ∈ [0, 1], we have θx1 + (1 − θ)x2 ∈ C ◮ use operations that preserve convexity: show that C is

  • btained from simple convex sets (hyperplanes, halfspaces,

norm balls, . . . ) by operations that preserve convexity

◮ intersection ◮ affine functions ◮ perspective functions ◮ linear-fractional functions

SJTU Ying Cui 13 / 33

slide-14
SLIDE 14

Intersection

◮ the intersection of (any number of) convex sets is convex

◮ if Sa is convex for every a ∈ A, then ∩a∈ASa is convex

◮ examples:

◮ a polyhedron is the intersection of halfspaces and hyperplanes (which are convex), and so is convex P ={x|aT

j x ≤ bj, j = 1, · · · , m, cT j x = dj, j = 1, · · · , p}

=(∩m

j=1{x|aT j x ≤ bj}) ∩ (∩p j=1{x|cT j x = dj})

{x|aT

j x ≤ bj}: halfspace; {x|cT j x = dj}: hyperplane

◮ the positive semidefinite cone is the intersection of an infinite number of halfspaces, and so is convex Sn

+ ={X ∈ Sn|X 0} = ∩z∈Rn,z=0{X ∈ Sn : zT Xz ≥ 0}

zTXz, z = 0: linear function of X; {X ∈ Sn : zT Xz ≥ 0}: halfspace in Sn

SJTU Ying Cui 14 / 33

slide-15
SLIDE 15

Affine functions

◮ affine function f : Rn → Rm: f (x) = Ax + b where A ∈ Rm×n and b ∈ Rm ◮ the image of a convex set under an affine function is convex

◮ S ⊆ Rn convex = ⇒ f (S) = {f (x)|x ∈ S} convex

◮ the inverse image of a convex set under an affine function is convex

◮ S ⊆ Rn convex = ⇒ f −1(S) = {x|f (x) ∈ S} convex

SJTU Ying Cui 15 / 33

slide-16
SLIDE 16

Affine functions

Examples: ◮ the scaling and translation of a convex set are convex

◮ S ⊆ Rn convex, α ∈ R, a ∈ Rn = ⇒ αS = {αx|x ∈ S} convex, S + a = {x + a|x ∈ S} convex

◮ the projection of a convex set onto some coordinates is convex

◮ S ⊆ Rm × Rn convex = ⇒ {x1 ∈ Rm|(x1, x2) ∈ S, x2 ∈ Rn} convex

◮ the sum of two convex sets is convex

◮ S1 and S2 convex = ⇒ Cartesian produce S1 × S2 = {(x1, x2)|x1 ∈ S1, x2 ∈ S2} convex, and hence S1 + S2 = {x1 + x2|x1 ∈ S1, x2 ∈ S2} convex

◮ polyhedron {x|Ax b, Cx = d} is convex

◮ the inverse image of Rm

+ × {0} under affine function

f : Rn → Rm+p given by f = (b − Ax, d − Cx)

◮ the solution set of linear matrix inequality {x|A(x) B} is convex, where A(x) = x1A1 + · · · + xnAn, B, Ai ∈ Sm

◮ the inverse image of the positive semidefinite cone under affine function f : Rn → Sm given by f (x) = B − A(x)

SJTU Ying Cui 16 / 33

slide-17
SLIDE 17

Perspective functions

◮ perspective function P : Rn+1 → Rn: P(z, t) = z/t, domP = Rn × R++

◮ scales or normalizes vectors so the last component is one, and then drops the last component

◮ the image of a convex set under a perspective function is convex

◮ C ⊆ domP convex = ⇒ P(C) = {P(x)|x ∈ C} convex

◮ the inverse image of a convex set under a perspective function is convex

◮ C ⊆ Rn convex = ⇒ P−1(C) = {(x, t) ∈ Rn+1| x

t ∈ C, t > 0}

convex

SJTU Ying Cui 17 / 33

slide-18
SLIDE 18

Linear-fractional functions

◮ linear-fractional (or projective) function f : Rn → Rm: f (x) = (Ax + b)/(cT x + d), domf = {x|cT x + d > 0} where A ∈ Rm×n, b ∈ Rm, c ∈ Rn, d ∈ R

◮ composes the perspective function with an affine function: f = P ◦ g

◮ perspective function P : Rm+1 → Rm, i.e., P(z, t) = z/t ◮ affine function g : Rn → Rm+1, i.e,. g(x) = A cT

  • x +

b d

  • ◮ example: conditional probabilities

◮ u ∈ {1, · · · , n} and v ∈ {1, · · · , m} are r.v.s, and let pij = Pr(u = i, v = j) and fij = Pr(u = i|v = j) ◮ fij =

pij n

k=1 pkj is a linear-fractional function

◮ the image of a convex set under a perspective function is convex

◮ C ⊆ domf convex = ⇒ f (C) = {f (x)|x ∈ C} convex

◮ the inverse image of a convex set under a perspective function is convex

◮ C ⊆ Rm convex = ⇒ f −1(C) = {x|f (x) ∈ C} convex

SJTU Ying Cui 18 / 33

slide-19
SLIDE 19

Proper cones and generalized inequalities

◮ a cone K ⊆ Rn is a proper cone if K is

◮ convex ◮ closed (contains its boundary) ◮ solid (has nonempty interior) ◮ pointed (contains no line, i.e., x, −x ∈ K = ⇒ x = 0)

◮ (nonstrict) generalized inequality defined by a proper cone K ⊆ Rn is a partial ordering on Rn x K y (y K x) ⇐ ⇒ y − x ∈ K ◮ strict generalized inequality defined by a proper cone K ⊆ Rn is a strict partial ordering on Rn x ≺K y (y ≻K x) ⇐ ⇒ y − x ∈ intK ◮ (nonstrict and strict) generalized inequalities include as a special case ordinary (nonstrict and strict) inequality in R

◮ when K = R+, partial ordering K is usual ordering ≤ on R, and strict partial ordering ≺K is usual strict ordering < on R

SJTU Ying Cui 19 / 33

slide-20
SLIDE 20

Proper cones and generalized inequalities

Examples ◮ nonnegative orthant and componentwise inequality:

◮ Rn

+ = {x ∈ Rn|xi ≥ 0, i = 1, · · · , n} is a proper cone

◮ Rn

+ corresponds to componentwise inequality between vectors:

x Rn

+ y ⇐

⇒ xi ≤ yi, i = 1, · · · , n ◮ drop subscript Rn

+, understood when appears between

vectors

◮ positive semidefinite cone and matrix inequality

◮ Sn

+ is a proper cone

◮ Sn

+ corresponds to matrix inequality:

X Sn

+ Y ⇐

⇒ Y − X ∈ Sn

+

◮ drop subscript Sn

+, understood when appears between

symmetric matrices

SJTU Ying Cui 20 / 33

slide-21
SLIDE 21

Proper cones and generalized inequalities

Properties of generalized inequalities ◮ preserved under addition: x K y, u K v = ⇒ x + u K y + v ◮ transitive: x K y, y K z = ⇒ x K z ◮ preserved under nonnegative scaling: x K y, α ≥ 0 = ⇒ αx K αy ◮ reflexive: x K x ◮ antisymmetric: x K y, y K x = ⇒ x = y ◮ preserved under limits: xi K yi for i = 1, 2, . . . , xi → x, yi → y as i → ∞ = ⇒ x K y

SJTU Ying Cui 21 / 33

slide-22
SLIDE 22

Minimum and minimal elements

◮ ≤ on R is a linear ordering: any two points are comparable, i.e., either x ≤ y or y ≤ x ◮ K is not in general a linear ordering: not any two points are comparable, i.e., we can have x K y and y K x ◮ concepts like minimum and maximum are more complicated in the context of generalized inequalities ◮ when K = R+ (partial ordering K is usual ordering ≤ on R), the concepts of minimal and minimum are the same, i.e., usual definition of the minimum element of a set

SJTU Ying Cui 22 / 33

slide-23
SLIDE 23

Minimum and minimal elements

◮ x ∈ S is the minimum element of S with respect to K

◮ if for every y ∈ S, we have x K y ◮ iff S ⊆ x + K, where x + K denotes all the points that are comparable to x and greater than or equal to x (w.r.t. K)

◮ if a set has a minimum element, then it is unique ◮ x ∈ S is a minimal element of S with respect to K

◮ if y ∈ S, y K x only if y = x ◮ iff (x − K) ∩ S = {x}, where x − K denotes all the points that are comparable to x and less than or equal to x (w.r.t. K)

SJTU Ying Cui 23 / 33

slide-24
SLIDE 24

Minimum and minimal elements

example ◮ K = Rn

+: Rn

+ corresponds to componentwise inequality

◮ x ∈ S is the minimum element of a set S means all other points of S lie above and to the right of x ◮ x ∈ S is a minimal element of a set S means that no other point of S lies to the left and below x

x1 x2 S1 S2 Figure 2.17 Left. The set S1 has a minimum element x1 with respect to componentwise inequality in R2. The set x1 + K is shaded lightly; x1 is the minimum element of S1 since S1 ⊆ x1 + K. Right. The point x2 is a minimal point of S2. The set x2 − K is shown lightly shaded. The point x2 is minimal because x2 − K and S2 intersect only at x2.

SJTU Ying Cui 24 / 33

slide-25
SLIDE 25

Separating hyperplane theorem

◮ separating hyperplane theorem: Suppose C and D are two nonempty convex sets that do not intersect, i.e., C ∩ D = ∅. Then there exist a = 0 and b such that aTx ≤ b for all x ∈ C, aTx ≥ b for all x ∈ D i.e., affine function f (x) = aTx − b is nonpositive on C and nonnegative on D ◮ separating hyperplane for sets C and D: {x|aT x = b}

D C a aT x ≥ b aT x ≤ b Figure 2.19 The hyperplane {x | aT x = b} separates the disjoint convex sets C and D. The affine function aT x − b is nonpositive on C and nonnegative

  • n D.

SJTU Ying Cui 25 / 33

slide-26
SLIDE 26

Supporting hyperplane theorem

◮ supporting hyperplane to set C at x0 ∈ bdC = clC \ intC: {x|aT x = aTx0} if a = 0 satisfies aTx ≤ aTx0 for all x ∈ C

◮ geometric interpretation: hyperplane {x|aTx = aTx0} is tangent to C at x0, and halfspace {x|aTx ≤ aTx0} contains C

C a x0

Figure 2.21 The hyperplane {x | aT x = aT x0} supports C at x0.

◮ supporting hyperplane theorem: for any nonempty convex set C, and any x0 ∈ bdC, there exists a supporting hyperplane to C at x0

SJTU Ying Cui 26 / 33

slide-27
SLIDE 27

Dual cones

◮ dual cone of a cone K: K ∗ = {y | y Tx ≥ 0 for all x ∈ K} ◮ K ∗ is a cone and is always convex (even when original cone K is not), i.e., convex cone ◮ geometric interpretation: y ∈ K ∗ iff −y is the normal of a hyperplane that supports K at the origin

K K y z Figure 2.22 Left. The halfspace with inward normal y contains the cone K, so y ∈ K∗. Right. The halfspace with inward normal z does not contain K, so z ∈ K∗.

SJTU Ying Cui 27 / 33

slide-28
SLIDE 28

Dual cones

examples ◮ nonnegative orthant: K = Rn

+, K ∗ = Rn +

◮ positive semidefinite cone: K = Sn

+, K ∗ = Sn +

◮ dual of a l2 norm cone: K = {(x, t) | ||x||2 ≤ t}, K ∗ = {(x, t) | ||x||2 ≤ t} ◮ dual of a l1 norm cone: K = {(x, t) | ||x||1 ≤ t}, K ∗ = {(x, t) | ||x||∞ ≤ t} first three examples are self-dual cones

SJTU Ying Cui 28 / 33

slide-29
SLIDE 29

Dual cones

properties ◮ K ∗ is closed and convex ◮ if K is solid (has nonempty interior), then K ∗ is pointed (contains no line) ◮ if the closure of K is pointed, then K ∗ is solid ◮ K ∗∗ is the closure of the convex hull of K ( = ⇒ if K is convex and closed, K ∗∗ = K) ◮ K1 ⊆ K2 implies K ∗

2 ⊆ K ∗ 1

Suppose K is a proper cone (convex, closed, solid and pointed). By the first three properties, its dual K ∗ is a proper cone, and moreover, by the fourth property, K ∗∗ = K.

SJTU Ying Cui 29 / 33

slide-30
SLIDE 30

Dual generalized inequalities

◮ dual cone K ∗ of a proper cone K is proper

◮ K is proper, inducing generalized inequality K ◮ K ∗ is proper, inducing generalized inequality K ∗ ◮ refer to K ∗ as the dual of K

◮ properties for proper cone K:

◮ x K y iff λT x ≤ λT y for all λ K ∗ 0 ◮ x ≺K y iff λT x < λT y for all λ K ∗ 0, λ = 0 ◮ hold if K and K ∗ are swapped (as K = K ∗∗)

SJTU Ying Cui 30 / 33

slide-31
SLIDE 31

Minimum and minimal elements via dual inequalities

Dual characterization of minimum element x is the minimum element of S w.r.t. K iff for all λ ≻K ∗ 0, x is the unique minimizer of λT z over z ∈ S ◮ geometric interpretation: for all λ ≻K ∗ 0, the hyperplane {z|λT(z − x) = 0} is a strict supporting hyperplane to S at x (i.e., intersects S only at the point x)

x S Figure 2.23 Dual characterization of minimum element. The point x is the minimum element of the set S with respect to R2

+. This is equivalent to:

for every λ ≻ 0, the hyperplane {z | λT (z − x) = 0} strictly supports S at x, i.e., contains S on one side, and touches it only at x.

SJTU Ying Cui 31 / 33

slide-32
SLIDE 32

Minimum and minimal elements via dual inequalities

Dual characterization of minimal elements ◮ if x minimizes λTz over z ∈ S for some λ ≻K ∗ 0, then x is minimal

S x1 x2 λ1 λ2 Figure 2.24 A set S ⊆ R2. Its set of minimal points, with respect to R2

+, is

shown as the darker section of its (lower, left) boundary. The minimizer of λT

1 z over S is x1, and is minimal since λ1 ≻ 0. The minimizer of λT 2 z over

S is x2, which is another minimal point of S, since λ2 ≻ 0.

◮ converse is in general false, i.e., x can be minimal in S, but not a minimizer of λT z over z ∈ S for any λ

◮ if x is a minimal element of a convex set S, then there exists a nonzero λ K ∗ 0 such that x minimizes λT z over z ∈ S

◮ convexity plays an important role in the converse

SJTU Ying Cui 32 / 33

slide-33
SLIDE 33

Optimal production frontier ◮ different production methods use different amounts of resources x ∈ Rn ◮ production set P: resource vectors x for all possible production methods ◮ efficient (Pareto optimal) methods correspond to resource vectors x that are minimal w.r.t. Rn

+

◮ example (n = 2): x1, x2, x3 are efficient; x4, x5 are not

x4 x2 x1 x5 x3 λ P labor fuel Figure 2.27 The production set P, for a product that requires labor and fuel to produce, is shown shaded. The two dark curves show the efficient production frontier. The points x1, x2 and x3 are efficient. The points x4 and x5 are not (since in particular, x2 corresponds to a production method that uses no more fuel, and less labor). The point x1 is also the minimum cost production method for the price vector λ (which is positive). The point x2 is efficient, but cannot be found by minimizing the total cost λT x for any price vector λ 0.

SJTU Ying Cui 33 / 33