13. Cones and semidefinite constraints Geometry of cones Second - PowerPoint PPT Presentation

CS/ECE/ISyE 524 Introduction to Optimization Spring 2017–18 13. Cones and semidefinite constraints ❼ Geometry of cones ❼ Second order cone programs ❼ Example: robust linear program ❼ Semidefinite constraints Laurent Lessard (www.laurentlessard.com)

What is a cone? ❼ A set of points C ∈ R n is called a cone if it satisfies: ◮ α x ∈ C whenever x ∈ C and α > 0. ◮ x + y ∈ C whenever x ∈ C and y ∈ C . ❼ Similar to a subspace, but α > 0 instead of α ∈ R . (this is a critical difference!) ❼ Simple examples: | x | ≤ y and y ≥ 0 1.5 1.5 1.0 1.0 0.5 0.5 - 1.5 - 1.0 - 0.5 0.5 1.0 1.5 - 1.5 - 1.0 - 0.5 0.5 1.0 1.5 13-2

What is a cone? ❼ A slice of a cone is its intersection with a subspace. ❼ We are interested in convex cones (all slices are convex). ❼ Can be polyhedral, ellipsoidal, or something else... 13-3

What is a cone? Polyhedral cone recipe: 1. Begin with your favorite polyhedron Ax ≤ b where x ∈ R n 2. { Ax ≤ bt , t ≥ 0 } is a polyhedral cone in ( x , t ) ∈ R n +1 3. The slice t = 1 is the original polyhedron. 2.5 2.0 1.5 1.0 0.5 0.5 1.0 1.5 2.0 2.5 13-4

What is a cone? Ellipsoidal cone recipe: 1. Ellipsoid x T Px + q T x + r ≤ 0 where P ≻ 0 and x ∈ R n 2. Complete the square ⇐ ⇒ � Ax + b � ≤ c 3. {� Ax + bt � ≤ ct } is an ellipsoidal cone in ( x , t ) ∈ R n +1 4. The slice t = 1 is the original ellipsoid. 2.5 2.0 1.5 1.0 0.5 0.5 1.0 1.5 2.0 2.5 13-5

Second-order cone constraint A second-order cone constraint is the set of points x ∈ R n : � Ax + b � ≤ c T x + d Every SOC constraint is a slice (set t = 1) of the cone � Ax + bt � ≤ c T x + dt . It’s not always a cone itself! Special cases: ❼ If A = 0, we have a linear inequality (hyperplane) ❼ If c = 0, it’s a slice of an ellipsoidal cone Every SOC constraint describes a convex set. 13-6

Second-order cone constraint A second-order cone constraint is the set of points x ∈ R n : � Ax + b � ≤ c T x + d If you square both sides... � Ax + b � 2 ≤ ( c T x + d ) 2 � � Ax + b � ≤ c T x + d ⇐ ⇒ c T x + d ≥ 0 The quadratic inequality is: x T ( A T A − cc T ) x + 2( b T A − dc T ) x + ( b T b − d 2 ) ≤ 0 This may be nonconvex ! 13-7

Second-order cone constraint A second-order cone constraint is the set of points x ∈ R n : � Ax + b � ≤ c T x + d Example: 1.5 1.0 � 0 � � � If A = 1 0 and c = and b = d = 0: 0.5 1 - 1.5 - 1.0 - 0.5 0.5 1.0 1.5 - 0.5 | x | ≤ y 1.5 1.0 Squaring both sides leads to: 0.5 x 2 − y 2 ≤ 0 - 1.5 - 1.0 - 0.5 0.5 1.0 1.5 and y ≥ 0 - 0.5 13-8

Special case: rotated second-order cone A rotated second-order cone is the set x ∈ R n , y , z ∈ R : x T x ≤ yz , y ≥ 0 , z ≥ 0 With n = 1, this looks like: 13-9

Special case: rotated second-order cone A rotated second-order cone is the set x ∈ R n , y , z ∈ R : x T x ≤ yz , y ≥ 0 , z ≥ 0 Can put into standard form: 4 x T x ≤ 4 yz 4 x T x + y 2 + z 2 ≤ 4 yz + y 2 + z 2 4 x T x + ( y − z ) 2 ≤ ( y + z ) 2 4 x T x + ( y − z ) 2 ≤ y + z � � 2 x � �� � � � ≤ y + z � � y − z � 13-10

SOCPs A second-order cone program (SOCP) has the form: c T x minimize x � A i x + b i � ≤ c T subject to: i x + d i for i = 1 , . . . , m ❼ Every LP is an SOCP (just make each A i = 0) ❼ Every convex QP and QCQP is an SOCP ◮ convert quadratic cost to epigraph form (add a variable) ◮ convert quadratic constraints to SOCP (complete square) 13-11

Implementation details A second-order cone program (SOCP) has the form: c T x minimize x � A i x + b i � ≤ c T subject to: i x + d i for i = 1 , . . . , m ❼ In JuMP, you can specify SOCP using: @constraint(m, norm(A*x+b) <= dot(c,x)+d) works with ECOS , SCS , Mosek , Gurobi , Ipopt . ❼ Can also specify rotated cones directly in Mosek , Ipopt . 13-12

Example: robust LP Consider a linear program with each linear constraint separately written out: c T x maximize x a T subject to: i x ≤ b i for i = 1 , . . . , m Suppose there is uncertainty in some of the a i vectors. Say for example that a i = ¯ a i + ρ u where ¯ a i is a nominal value and u is the uncertainty. ❼ box constraint: � u � ∞ ≤ 1 ❼ ball constraints: � u � 2 ≤ 1 13-13

Robust LP with box constraint a i + ρ u into a T Substituting a i = ¯ i x ≤ b i , obtain: a T i x + ρ u T x ≤ b i ¯ for all uncertain u box constraint: If this must hold for all u with � u � ∞ ≤ 1, then it holds for the worst-case u . Therefore: n n n � � � u T x = u i x i ≤ | u i || x i | ≤ | x i | = � x � 1 i =1 i =1 i =1 Then we have a T ¯ i x + ρ � x � 1 ≤ b i 13-14

Robust LP with box constraint With a box constraint a i = ¯ a i + ρ u with � u � ∞ ≤ 1 c T x maximize x a T subject to: i x ≤ b i for i = 1 , . . . , m Is equivalent to the optimization problem c T x maximize x a T subject to: ¯ i x + ρ � x � 1 ≤ b i for i = 1 , . . . , m 13-15

Robust LP with box constraint With a box constraint a i = ¯ a i + ρ u with � u � ∞ ≤ 1 c T x maximize x a T subject to: i x ≤ b i for i = 1 , . . . , m ... which is equivalent to the linear program: c T x maximize x , t n � a T subject to: ¯ i x + ρ t j ≤ b i for i = 1 , . . . , m j =1 − t j ≤ x j ≤ t j for j = 1 , . . . , n 13-16

Robust LP with box constraint 1.5 1.5 1.0 1.0 0.5 0.5 - 1.5 - 1.0 - 0.5 0.5 1.0 1.5 2.0 - 1.5 - 1.0 - 0.5 0.5 1.0 1.5 2.0 - 0.5 - 0.5 - 1.0 - 1.0 - 1.5 - 1.5 a T a T i x ≤ b i i x + 0 . 2 � x � 1 ≤ b i ❼ New region is smaller, still a polyhedron ❼ More robust to uncertain constraints 13-17

Robust LP with ball constraint a i + ρ u into a T Substituting a i = ¯ i x ≤ b i , obtain: a T i x + ρ u T x ≤ b i ¯ for all uncertain u ball constraint: If this must hold for all u with � u � 2 ≤ 1, then it holds for the worst-case u . Using Cauchy-Schwarz inequality: u T x ≤ � u � 2 � x � 2 ≤ � x � 2 Then we have a T ¯ i x + ρ � x � 2 ≤ b i (a second-order cone constraint!) 13-18

Robust LP with ball constraint With a ball constraint a i = ¯ a i + ρ u with � u � 2 ≤ 1 c T x maximize x a T subject to: i x ≤ b i for i = 1 , . . . , m Is equivalent to the optimization problem c T x maximize x a T subject to: ¯ i x + ρ � x � 2 ≤ b i for i = 1 , . . . , m which is an SOCP 13-19

Robust LP with ball constraint 1.5 1.5 1.0 1.0 0.5 0.5 - 1.5 - 1.0 - 0.5 0.5 1.0 1.5 2.0 - 1.5 - 1.0 - 0.5 0.5 1.0 1.5 2.0 - 0.5 - 0.5 - 1.0 - 1.0 - 1.5 - 1.5 a T a T i x ≤ b i i x + 0 . 2 � x � 2 ≤ b i ❼ New region is smaller, no longer a polyhedron ❼ More robust to uncertain constraints 13-20

Matrix variables Sometimes, the decision variable is a matrix X . ❼ Can always just think of X ∈ R m × n as x ∈ R mn . ❼ Linear functions: mn � c k x k = c T x k =1 m n � � C ij X ij = trace( C T X ) i =1 j =1 ❼ Linear program: trace( C T X ) maximize X trace( A T subject to: i X ) ≤ b i for i = 1 , . . . , k 13-21

Matrix variables If a decision variable is a symmetric matrix X = X T ∈ R n × n , we can represent it as a vector x ∈ R n ( n +1) / 2 .   x 1 x 2     x 1 x 2 x 3   x 3   x 2 x 4 x 5 ⇐ ⇒     x 4   x 3 x 5 x 6   x 5   x 6 The constraint X � 0 is called a semidefinite constraint. What does it look like geometrically? 13-22

The PSD cone The set of matrices X � 0 are a convex cone in R n ( n +1) / 2 � x � y � 0 of points in R 3 satisfy: Example: The set y z xz ≥ y 2 , x ≥ 0 , z ≥ 0 This is a rotated second-order cone! Equivalent to: � 2 y � �� � � � ≤ x + z � � x − z � 13-23

More complicated example � 1 � x y The set of ( x , y , z ) satisfying � 0 is the solution of: x 1 z y z 1 X ∈ R 3 × 3 , � � X � 0 , X 11 = 1 , X 22 = 1 , X 33 = 1 13-24

Spectrahedra ❼ Two common set representations: ◮ variables x 1 , . . . , x k , constants Q i = Q T i , and constraint: Q 0 + x 1 Q 1 + . . . x k Q k � 0 (linear matrix inequality) ◮ variable X � 0 and the constraints: trace( A T i X ) ≤ b i (linear constraint form) ❼ These sets are called spectrahedra . ❼ Very rich set, lots of possible shapes. 13-25

Semidefinite program (SDP) Standard form #1: (looks like the standard form for an LP) trace( C T X ) maximize X trace( A T subject to: i X ) ≤ b i for i = 1 , . . . , m X � 0 Standard form #2: c T x maximize x m � subject to: Q 0 + x i Q i � 0 i =1 13-26

Relationship with other programs Every LP is an SDP: � a 11 � � x 1 � � b 1 � a 12 ≤ a 21 a 22 x 2 b 2 is the same as: � a 11 � � a 12 � � b 1 � 0 0 0 x 1 + x 2 � 0 a 21 0 a 22 0 b 2 (polyhedra are special cases of spectrahedra) 13-27

Relationship with other programs Every SOCP is an SDP: � Ax + b � ≤ c T x + d is the same as: � ( c T x + d ) I � Ax + b � 0 ( Ax + b ) T c T x + d This isn’t obvious — proof requires use of Schur complement. (second-order cones are special cases of spectrahedra) 13-28