Convex optimization problems (II) Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj
Outline Linear Optimization Problems Quadratic Optimization Problems Geometric Programming Generalized Inequality Constraints Vector Optimization
Linear Optimization Problems Linear Program (LP) � ��� and ��� It is common to omit the constant Maximization problem with affine objective and constraint functions is also an LP The feasible set of LP is a polyhedron
Linear Optimization Problems Geometric Interpretation of an LP The objective 𝑑 � 𝑦 is linear, so its level curves are hyperplanes orthogonal to 𝑑 𝑦 ∗ is as far as possible in the direction �𝑑
Two Special Cases of LP Standard Form LP � The only inequalities are Inequality Form LP � No equality constraint
Converting to Standard Form Conversion � � To use an algorithm for standard LP Introduce Slack Variables � �
Converting to Standard Form Decompose 𝑦 � 𝑦 � � 𝑦 � , 𝑦 � , 𝑦 � ≽ 0 Standard Form LP � � � � � � � � � � �
Example Diet Problem Choose nonnegative quantities 𝑦 � , … , 𝑦 � of 𝑜 foods One unit of food 𝑘 contains amount 𝑏 �� of nutrient 𝑗 , and costs 𝑑 � Healthy diet requires nutrient 𝑗 in quantities at least 𝑐 � Determine the cheapest diet that satisfies the nutritional requirements. �
Example Chebyshev Center of a Polyhedron Find the largest Euclidean ball that lies in the polyhedron � 𝑦 � 𝑐 � , 𝑗 � 1, … , 𝑛� 𝒬 � �𝑦 ∈ 𝐒 � |𝑏 � The center of the optimal ball is called the Chebyshev center of the polyhedron Represent the ball as ℬ � �𝑦 � � 𝑣| 𝑣 � � 𝑠� 𝑦 � ∈ 𝐒 � and 𝑠 are variables, and we wish to maximize 𝑠 subject to ℬ ∈ 𝒬 � 𝑦 � � 𝑣 � 𝑐 � , 𝑣 � � 𝑠 ⟺ � 𝑦 � 𝑐 � ⟺ 𝑏 � ∀𝑦 ∈ ℬ, 𝑏 � � 𝑦 � � 𝑠 𝑏 � 𝑏 � � � 𝑐 �
Example Chebyshev Center of a Polyhedron Find the largest Euclidean ball that lies in the polyhedron � 𝑦 � 𝑐 � , 𝑗 � 1, … , 𝑛� 𝒬 � �𝑦 ∈ 𝐒 � |𝑏 � The center of the optimal ball is called the Chebyshev center of the polyhedron Represent the ball as ℬ � �𝑦 � � 𝑣| 𝑣 � � 𝑠� 𝑦 � ∈ 𝐒 � and 𝑠 are variables, and we wish to maximize 𝑠 subject to ℬ ∈ 𝒬 max 𝑠 � 𝑦 � � 𝑠 𝑏 � s. t. 𝑏 � � � 𝑐 � , 𝑗 � 1, … , 𝑛
Example Chebyshev Inequalities is a random variable on � � � � , � � is a linear function of � � ��� Prior knowledge is given as � 𝑞 � 𝛾 � , 𝛽 � � 𝑏 � 𝑗 � 1, … , 𝑛 � To find a lower bound of � � � 𝑞 min 𝑏 � 𝑞 ≽ 0, 𝟐 � 𝑞 � 1 s. t. � 𝑞 � 𝛾 � , 𝛽 � � 𝑏 � 𝑗 � 1, … , 𝑛
Example Piecewise-linear Minimization Consider the (unconstrained) problem � � � ���,…,� The epigraph problem � � � ���,…,� An LP problem � � �
Linear-fractional Programming Linear-fractional Program min 𝑔 � �𝑦� s. t. 𝐻𝑦 ≼ ℎ 𝐵𝑦 � 𝑐 The objective function is a ratio of affine � 𝑦 � 𝑑 � 𝑦 � 𝑒 functions 𝑔 𝑓 � 𝑦 � 𝑔 The domain is � � �𝑦|𝑓 � 𝑦 � 𝑔 � 0� dom 𝑔 A quasiconvex optimization problem
Linear-fractional Programming Transforming to a linear program 𝑑 � 𝑧 � 𝑒𝑨 min � 𝑦 � 𝑑 � 𝑦 � 𝑒 min 𝑔 s. t. 𝐻𝑧 � ℎ𝑨 ≼ 0 𝑓 � 𝑦 � 𝑔 𝐵𝑧 � 𝑐𝑨 � 0 s. t. 𝐻𝑦 ≼ ℎ 𝑓 � 𝑧 � 𝑔𝑨 � 1 𝐵𝑦 � 𝑐 𝑨 � 0 Proof � � 𝑦 is feasible in LFP ⇒ 𝑧 � � � ��� , 𝑨 � � � ��� is feasible , 𝑑 � 𝑧 � 𝑒𝑨 � 𝑔 � �𝑦� ⇒ the optimal value of LFP is in LP greater than or equal to the optimal value of LP
Linear-fractional Programming Transforming to a linear program 𝑑 � 𝑧 � 𝑒𝑨 min � 𝑦 � 𝑑 � 𝑦 � 𝑒 min 𝑔 s. t. 𝐻𝑧 � ℎ𝑨 ≼ 0 𝑓 � 𝑦 � 𝑔 𝐵𝑧 � 𝑐𝑨 � 0 s. t. 𝐻𝑦 ≼ ℎ 𝑓 � 𝑧 � 𝑔𝑨 � 1 𝐵𝑦 � 𝑐 𝑨 � 0 Proof ⁄ �𝑧, 𝑨� is feasible in LP and 𝑨 � 0 ⇒ 𝑦 � 𝑧 𝑨 is feasible � 𝑦 � 𝑑 � 𝑧 � 𝑒𝑨 ⇒ the optimal value of FLP is , 𝑔 in LFP less than or equal to the optimal value of LP �𝑧, 𝑨� is feasible in LP , 𝑨 � 0 and 𝑦 � is feasible in LFP ⇒ 𝑦 � 𝑦 � � 𝑢𝑧 is feasible in LFP for all 𝑢 � 0 , � �𝑦 � � 𝑢𝑧� � 𝑑 � 𝑧 � 𝑒𝑨 �→� 𝑔 lim
Generalized Linear-fractional Programming Generalized Linear-fractional Program � 𝑦 � 𝑒 � 𝑑 � 𝑔 � 𝑦 � max � 𝑦 � 𝑔 𝑓 � ���,…,� � � � � � A quasiconvex optimization problem Von Neumann Growth Problem � 𝑦 � ⁄ max ���,…,� 𝑦 � min 𝑦 � ≽ 0 s. t. 𝐶𝑦 � ≼ 𝐵𝑦
Generalized Linear-fractional Programming Von Neumann Growth Problem � 𝑦 � ⁄ max ���,…,� 𝑦 � min 𝑦 � ≽ 0 s. t. 𝐶𝑦 � ≼ 𝐵𝑦 𝑦, 𝑦 � ∈ 𝐒 � : activity levels of 𝑜 sectors, in current and next period 𝐵𝑦 � , 𝐶𝑦 � � : produced and consumed amounts of good 𝑗 𝐶𝑦 � ≼ 𝐵𝑦 : goods consumed in the next period cannot exceed the goods produced in the current period � 𝑦 � ⁄ 𝑦 � growth rate of sector 𝑗
Outline Linear Optimization Problems Quadratic Optimization Problems Geometric Programming Generalized Inequality Constraints Vector Optimization
Quadratic Optimization Problems Quadratic Program (QP) ⁄ �𝑦 � 𝑄𝑦 � 𝑟 � 𝑦 � 𝑠 min �1 2 s. t. 𝐻𝑦 ≼ ℎ 𝐵𝑦 � 𝑐 ��� and � ��� � The objective function is (convex) quadratic The constraint functions are affine When , QP becomes LP
Quadratic Optimization Problems Geometric Illustration of QP The feasible set 𝒬 is a polyhedron The contour lines of the objective function are shown as dashed curves.
Quadratic Optimization Problems Quadratically Constrained Quadratic Program (QCQP) � 𝑦 � 𝑠 𝑦 � 𝑄 � 𝑦 � 𝑟 � min 1 2 ⁄ � � 𝑦 � 𝑠 𝑦 � 𝑄 � 𝑦 � 𝑟 � ⁄ s. t. 1 2 � � 0, 𝑗 � 1, … , 𝑛 𝐵𝑦 � 𝑐 � , 𝑗 � 0, … , 𝑛 𝑄 � ∈ 𝐓 � The inequality constraint functions are (convex) quadratic The feasible set is the intersection of ellipsoids (when 𝑄 � ≻ 0 ) and an affine set Include QP as a special case
Examples Least-squares and Regression � � 𝑦 � 𝐵 � 𝐵𝑦 � 2𝑐 � 𝐵𝑦 � 𝑐 � 𝑐 min 𝐵𝑦 � 𝑐 � Analytical solution: 𝑦 � 𝐵 � 𝑐 Can add linear constraints, e.g., 𝑚 ≼ 𝑦 ≼ 𝑣 Distance Between Polyhedra � � � � � � � � � � Find the distance between the polyhedra 𝒬 � � �𝑦|𝐵 � 𝑦 ≼ 𝑐 � � and 𝒬 � � �𝑦|𝐵 � 𝑦 ≼ 𝑐 � � dist 𝒬 � , 𝒬 � � inf 𝑦 � � 𝑦 � � 𝑦 � ∈ 𝒬 � , 𝑦 � ∈ 𝒬 �
Example Bounding Variance is a random variable on � � � � , � The variance of a random variable � � � 𝐅𝑔 � � 𝐅𝑔 � � � � 𝑞 � 𝑔 � � 𝑔 � 𝑞 � � ��� ��� Maximize the variance � � � � 𝑞 � max � 𝑔 � � 𝑔 � 𝑞 � � ��� ��� 𝑞 ≽ 0, 𝟐 � 𝑞 � 1 s. t. � 𝑞 � 𝛾 � , 𝑗 � 1, … , 𝑛 𝛽 � � 𝑏 �
Second-order Cone Programming Second-order Cone Program (SOCP) 𝑔 � 𝑦 min � 𝑦 � 𝑒 � , s. t. 𝐵 � 𝑦 � 𝑐 � � � 𝑑 � 𝑗 � 1, … , 𝑛 𝐺𝑦 � 𝐵 � ∈ 𝐒 � � �� , 𝐺 ∈ 𝐒 ��� Second-order Cone (SOC) constraint: 𝐵𝑦 � 𝑐 � � 𝑑 � 𝑦 � 𝑒 where 𝐵 ∈ 𝐒 ��� , is same as requiring 𝐵𝑦 � 𝑐, 𝑑 � 𝑦 � 𝑒 ∈ SOC in 𝐒 ��� 𝑦, 𝑢 ∈ 𝐒 ��� | 𝑦 � � 𝑢 SOC � � 𝐽 𝑦 𝑢 | 𝑦 𝑦 0 � 𝑢 � 0, 𝑢 � 0 𝑢 0 �1
Second-order Cone Programming Second-order Cone Program (SOCP) 𝑔 � 𝑦 min � 𝑦 � 𝑒 � , s. t. 𝐵 � 𝑦 � 𝑐 � � � 𝑑 � 𝑗 � 1, … , 𝑛 𝐺𝑦 � 𝐵 � ∈ 𝐒 � � �� , 𝐺 ∈ 𝐒 ��� Second-order Cone (SOC) constraint: 𝐵𝑦 � 𝑐 � � 𝑑 � 𝑦 � 𝑒 where 𝐵 ∈ 𝐒 ��� , is same as requiring 𝐵𝑦 � 𝑐, 𝑑 � 𝑦 � 𝑒 ∈ SOC in 𝐒 ��� If 𝑑 � � 0, 𝑗 � 1, … , 𝑛 , it reduces to QCQP by squaring each inequality constraint More general than QCQP and LP
Recommend
More recommend
