Duality in linear optimization Example with both primal and dual infeasibility Consider minimize − x 1 − x 2 subject to x 1 = − 1 x 1 , x 2 ≥ 0 with a dual problem maximize − y � 1 � 1 � � subject to − y ≥ . 0 1 • y = − 1 is a certificate of primal infeasibility, p ⋆ = ∞ • x = (0 , 1) is a certificate of dual infeasibility, d ⋆ = −∞ .
Section 2 Conic optimization
Proper convex cones We consider proper convex cones K in R n : • Closed. • Pointed: K ∩ ( − K ) = { 0 } . • Non-empty interior. Dual-cone: K ∗ = { v ∈ R n | u T v ≥ 0 , ∀ u ∈ K } . If K is a proper cone, then K ⋆ is also proper. We use the notation: x � K y ⇐ ⇒ ( x − y ) ∈ K x ≻ K y ⇐ ⇒ ( x − y ) ∈ int K
Example of cones Quadratic cone (second-order cone, Lorenz cone) � Q n = { x ∈ R n | x 1 ≥ x 2 2 + x 2 3 + · · · + x 2 n } . Q n is self-dual: ( Q n ) ∗ = Q n .
Examples of quadratic cones • Epigraph of absolute value: ( t , x ) ∈ Q 2 . | x | ≤ t ⇐ ⇒ • Epigraph of Euclidean norm: ( t , x ) ∈ Q n − 1 , � x � 2 ≤ t ⇐ ⇒ � where x ∈ R n and � x � = x 2 1 + · · · + x 2 n . • Second-order cone inequality: � Ax + b � 2 ≤ c T x + d ( c T x + d , Ax + b ) ∈ Q m +1 ⇐ ⇒ for A ∈ R m × n , b ∈ R m , c ∈ R n , d ∈ R .
Examples of quadratic cones • Epigraph of absolute value: ( t , x ) ∈ Q 2 . | x | ≤ t ⇐ ⇒ • Epigraph of Euclidean norm: ( t , x ) ∈ Q n − 1 , � x � 2 ≤ t ⇐ ⇒ � where x ∈ R n and � x � = x 2 1 + · · · + x 2 n . • Second-order cone inequality: � Ax + b � 2 ≤ c T x + d ( c T x + d , Ax + b ) ∈ Q m +1 ⇐ ⇒ for A ∈ R m × n , b ∈ R m , c ∈ R n , d ∈ R .
Examples of quadratic cones • Epigraph of absolute value: ( t , x ) ∈ Q 2 . | x | ≤ t ⇐ ⇒ • Epigraph of Euclidean norm: ( t , x ) ∈ Q n − 1 , � x � 2 ≤ t ⇐ ⇒ � where x ∈ R n and � x � = x 2 1 + · · · + x 2 n . • Second-order cone inequality: � Ax + b � 2 ≤ c T x + d ( c T x + d , Ax + b ) ∈ Q m +1 ⇐ ⇒ for A ∈ R m × n , b ∈ R m , c ∈ R n , d ∈ R .
Examples of quadratic cones Robust optimization with ellipsoidal uncertainty Ellipsoidal set: E = { x ∈ R n | � P ( x − a ) � 2 ≤ 1 } � x ∈ R n | x = P − 1 y + a , � y � 2 ≤ 1 � = . Worst-case realization of a linear function over E : c T x = a T x + sup y T P − 1 x = a T x + � P − 1 x � 2 . sup c ∈E � y � 2 ≤ 1 Robust LP: a T x + t minimize c T x minimize sup subject to Ax = b c ∈E subject to Ax = b ( t , P − 1 x ) ∈ Q n +1 x ≥ 0 , x ≥ 0 .
Examples of quadratic cones Robust optimization with ellipsoidal uncertainty Ellipsoidal set: E = { x ∈ R n | � P ( x − a ) � 2 ≤ 1 } � x ∈ R n | x = P − 1 y + a , � y � 2 ≤ 1 � = . Worst-case realization of a linear function over E : c T x = a T x + sup y T P − 1 x = a T x + � P − 1 x � 2 . sup c ∈E � y � 2 ≤ 1 Robust LP: a T x + t minimize c T x minimize sup subject to Ax = b c ∈E subject to Ax = b ( t , P − 1 x ) ∈ Q n +1 x ≥ 0 , x ≥ 0 .
Examples of quadratic cones Robust optimization with ellipsoidal uncertainty Ellipsoidal set: E = { x ∈ R n | � P ( x − a ) � 2 ≤ 1 } � x ∈ R n | x = P − 1 y + a , � y � 2 ≤ 1 � = . Worst-case realization of a linear function over E : c T x = a T x + sup y T P − 1 x = a T x + � P − 1 x � 2 . sup c ∈E � y � 2 ≤ 1 Robust LP: a T x + t minimize c T x minimize sup subject to Ax = b c ∈E subject to Ax = b ( t , P − 1 x ) ∈ Q n +1 x ≥ 0 , x ≥ 0 .
Example of cones Rotated quadratic cone Rotated quadratic cone: r = { x ∈ R n | 2 x 1 x 2 ≥ x 2 Q n 3 + . . . x 2 n , x 1 , x 2 ≥ 0 } . Related to standard quadratic cone: x ∈ Q n ( T n x ) ∈ Q n ⇐ ⇒ r for √ √ 1 / 2 1 / 2 0 √ √ . T n := 1 / 2 − 1 / 2 0 0 0 I n − 2 r ) ∗ = Q n Q n r is self-dual: ( Q n r .
Examples of rotated quadratic cones • Epigraph of squared Euclidean norm: � x � 2 (1 / 2 , t , x ) ∈ Q n +2 2 ≤ t ⇐ ⇒ . r • Convex quadratic inequality: (1 / 2) x T Qx ≤ c T x + d (1 / 2 , c T x + d , F T x ) ∈ Q k +2 ⇐ ⇒ r with Q = F T F , F ∈ R n × k . So we can write QCQPs as conic problems.
Examples of rotated quadratic cones • Epigraph of squared Euclidean norm: � x � 2 (1 / 2 , t , x ) ∈ Q n +2 2 ≤ t ⇐ ⇒ . r • Convex quadratic inequality: (1 / 2) x T Qx ≤ c T x + d (1 / 2 , c T x + d , F T x ) ∈ Q k +2 ⇐ ⇒ r with Q = F T F , F ∈ R n × k . So we can write QCQPs as conic problems.
Examples of rotated quadratic cones • Convex hyperbolic function: √ 1 2) ∈ Q 3 x ≤ t , x > 0 ⇐ ⇒ ( x , t , r . • Square roots: √ x ≥ t , x ≥ 0 (1 2 , x , t ) ∈ Q 3 ⇐ ⇒ r . • Convex positive rational power: x 3 / 2 ≤ t , x ≥ 0 ( s , t , x ) , ( x , 1 / 8 , s ) ∈ Q 3 ⇐ ⇒ r . • Convex negative rational power: √ 1 ( t , 1 2) ∈ Q 3 x 2 ≤ t , x > 0 ⇐ ⇒ 2 , s ) , ( x , s , r .
Examples of rotated quadratic cones • Convex hyperbolic function: √ 1 2) ∈ Q 3 x ≤ t , x > 0 ⇐ ⇒ ( x , t , r . • Square roots: √ x ≥ t , x ≥ 0 (1 2 , x , t ) ∈ Q 3 ⇐ ⇒ r . • Convex positive rational power: x 3 / 2 ≤ t , x ≥ 0 ( s , t , x ) , ( x , 1 / 8 , s ) ∈ Q 3 ⇐ ⇒ r . • Convex negative rational power: √ 1 ( t , 1 2) ∈ Q 3 x 2 ≤ t , x > 0 ⇐ ⇒ 2 , s ) , ( x , s , r .
Examples of rotated quadratic cones • Convex hyperbolic function: √ 1 2) ∈ Q 3 x ≤ t , x > 0 ⇐ ⇒ ( x , t , r . • Square roots: √ x ≥ t , x ≥ 0 (1 2 , x , t ) ∈ Q 3 ⇐ ⇒ r . • Convex positive rational power: x 3 / 2 ≤ t , x ≥ 0 ( s , t , x ) , ( x , 1 / 8 , s ) ∈ Q 3 ⇐ ⇒ r . • Convex negative rational power: √ 1 ( t , 1 2) ∈ Q 3 x 2 ≤ t , x > 0 ⇐ ⇒ 2 , s ) , ( x , s , r .
Examples of rotated quadratic cones • Convex hyperbolic function: √ 1 2) ∈ Q 3 x ≤ t , x > 0 ⇐ ⇒ ( x , t , r . • Square roots: √ x ≥ t , x ≥ 0 (1 2 , x , t ) ∈ Q 3 ⇐ ⇒ r . • Convex positive rational power: x 3 / 2 ≤ t , x ≥ 0 ( s , t , x ) , ( x , 1 / 8 , s ) ∈ Q 3 ⇐ ⇒ r . • Convex negative rational power: √ 1 ( t , 1 2) ∈ Q 3 x 2 ≤ t , x > 0 ⇐ ⇒ 2 , s ) , ( x , s , r .
Semidefinite matrices Basic definitions • We denote n × n symmetric matrices by S n . • Standard inner product for matrices: � � V , W � := tr ( V T W ) = V ij W ij = vec ( V ) T vec ( W ) . ij • X is semidefinite if and only if 1 z T Xz ≥ 0, ∀ z ∈ R n . 2 All the eigenvalues of X are nonnegative. 3 X is a Grammian matrix, X = V T V . • The (semi)definite matrices form a cone ( S + ) S ++ . Exercise: Show the three definitions are equivalent.
Semidefinite matrices Basic definitions • We denote n × n symmetric matrices by S n . • Standard inner product for matrices: � � V , W � := tr ( V T W ) = V ij W ij = vec ( V ) T vec ( W ) . ij • X is semidefinite if and only if 1 z T Xz ≥ 0, ∀ z ∈ R n . 2 All the eigenvalues of X are nonnegative. 3 X is a Grammian matrix, X = V T V . • The (semi)definite matrices form a cone ( S + ) S ++ . Exercise: Show the three definitions are equivalent.
Semidefinite matrices Basic definitions • We denote n × n symmetric matrices by S n . • Standard inner product for matrices: � � V , W � := tr ( V T W ) = V ij W ij = vec ( V ) T vec ( W ) . ij • X is semidefinite if and only if 1 z T Xz ≥ 0, ∀ z ∈ R n . 2 All the eigenvalues of X are nonnegative. 3 X is a Grammian matrix, X = V T V . • The (semi)definite matrices form a cone ( S + ) S ++ . Exercise: Show the three definitions are equivalent.
Semidefinite matrices Basic definitions • We denote n × n symmetric matrices by S n . • Standard inner product for matrices: � � V , W � := tr ( V T W ) = V ij W ij = vec ( V ) T vec ( W ) . ij • X is semidefinite if and only if 1 z T Xz ≥ 0, ∀ z ∈ R n . 2 All the eigenvalues of X are nonnegative. 3 X is a Grammian matrix, X = V T V . • The (semi)definite matrices form a cone ( S + ) S ++ . Exercise: Show the three definitions are equivalent.
Semidefinite matrices Basic definitions Dual cone: + ) ∗ = { Z ∈ R n × n | � X , Z � ≥ 0 , ∀ X ∈ S n ( S n + } . + ) ∗ = S n The semidefinite is self-dual: ( S n + . Easy to prove: Assume Z � 0 so that Z = U T U and X = V T V . � X , Z � = � V T V , U T U � = tr ( UV T )( UV T ) T = � UV T � 2 F ≥ 0 . Conversely assume Z �� 0. Then ∃ w ∈ R n such that w T Zw = � ww T , Z � = � X , Z � < 0 .
Positive semidefinite matrices Schur’s lemma Schur’s lemma: � � C T B B − C T D − 1 C ≻ 0 , C ≻ 0 , D ≻ 0 . ≻ 0 ⇐ ⇒ C D Example: � � x T 1 t t x T x < t ≻ 0 ⇐ ⇒ ⇐ ⇒ � x � < t , x tI i.e., quadratic cone can be embedded in a semidefinite cone.
A geometric example The pillow spectrahedron The convex set � � � � 1 x y ( x , y , z ) ∈ R 3 | S = ≻ 0 , x 1 z y z 1 is called a pillow . Exercise: Characterize the restriction S | z =0 .
Eigenvalue optimization Symmetric matrices F ( x ) = F 0 + x 1 F 1 + · · · + x m F m , F i ∈ S m . • Minimize largest eigenvalue λ 1 ( F ( x )): minimize γ subject to γ I � F ( x ) , • Maximize smallest eigenvalue λ n ( F ( x )): maximize γ subject to F ( x ) � γ I , • Minimize eigenvalue spread λ 1 ( F ( x )) − λ n ( F ( x )): minimize γ − λ subject to γ I � F ( x ) � λ I ,
Eigenvalue optimization Symmetric matrices F ( x ) = F 0 + x 1 F 1 + · · · + x m F m , F i ∈ S m . • Minimize largest eigenvalue λ 1 ( F ( x )): minimize γ subject to γ I � F ( x ) , • Maximize smallest eigenvalue λ n ( F ( x )): maximize γ subject to F ( x ) � γ I , • Minimize eigenvalue spread λ 1 ( F ( x )) − λ n ( F ( x )): minimize γ − λ subject to γ I � F ( x ) � λ I ,
Eigenvalue optimization Symmetric matrices F ( x ) = F 0 + x 1 F 1 + · · · + x m F m , F i ∈ S m . • Minimize largest eigenvalue λ 1 ( F ( x )): minimize γ subject to γ I � F ( x ) , • Maximize smallest eigenvalue λ n ( F ( x )): maximize γ subject to F ( x ) � γ I , • Minimize eigenvalue spread λ 1 ( F ( x )) − λ n ( F ( x )): minimize γ − λ subject to γ I � F ( x ) � λ I ,
Matrix norms Nonsymmetric matrices F i ∈ R n × p . F ( x ) = F 0 + x 1 F 1 + · · · + x m F m , � • Frobenius norm: � F ( x ) � F := � F ( x ) , F ( x ) � , ( t , vec ( F ( x ))) ∈ Q np +1 , � F ( x ) � F ≤ t ⇔ • Induced ℓ 2 norm: � F ( x ) � 2 := max σ k ( F ( x )), k minimize t � � F ( x ) T tI subject to � 0 , F ( x ) tI corresponds to the largest eigenvalue for F ( x ) ∈ S n + .
Matrix norms Nonsymmetric matrices F i ∈ R n × p . F ( x ) = F 0 + x 1 F 1 + · · · + x m F m , � • Frobenius norm: � F ( x ) � F := � F ( x ) , F ( x ) � , ( t , vec ( F ( x ))) ∈ Q np +1 , � F ( x ) � F ≤ t ⇔ • Induced ℓ 2 norm: � F ( x ) � 2 := max σ k ( F ( x )), k minimize t � � F ( x ) T tI subject to � 0 , F ( x ) tI corresponds to the largest eigenvalue for F ( x ) ∈ S n + .
Nearest correlation matrix Consider S = { X ∈ S n + | X ii = 1 , i = 1 , . . . , n } . For a symmetric A ∈ R n × n , the nearest correlation matrix is X ⋆ = arg min X ∈ S � A − X � F , which corresponds to a mixed SOCP/SDP, minimize t subject to � vec ( A − X ) � 2 ≤ t diag ( X ) = e X � 0 . MOSEK is limited by the many constraints to, say n < 200.
Nearest correlation matrix Consider S = { X ∈ S n + | X ii = 1 , i = 1 , . . . , n } . For a symmetric A ∈ R n × n , the nearest correlation matrix is X ⋆ = arg min X ∈ S � A − X � F , which corresponds to a mixed SOCP/SDP, minimize t subject to � vec ( A − X ) � 2 ≤ t diag ( X ) = e X � 0 . MOSEK is limited by the many constraints to, say n < 200.
Combinatorial relaxations Consider a binary problem x T Qx + c T x minimize subject to x i ∈ { 0 , 1 } , i = 1 , . . . , n . where Q ∈ S n can be indefinite. • Rewrite binary constraints x i ∈ { 0 , 1 } : x 2 X = xx T , i = x i ⇐ ⇒ diag ( X ) = x . • Semidefinite relaxation: X � xx T , diag ( X ) = x .
Combinatorial relaxations Consider a binary problem x T Qx + c T x minimize subject to x i ∈ { 0 , 1 } , i = 1 , . . . , n . where Q ∈ S n can be indefinite. • Rewrite binary constraints x i ∈ { 0 , 1 } : x 2 X = xx T , i = x i ⇐ ⇒ diag ( X ) = x . • Semidefinite relaxation: X � xx T , diag ( X ) = x .
Combinatorial relaxations Consider a binary problem x T Qx + c T x minimize subject to x i ∈ { 0 , 1 } , i = 1 , . . . , n . where Q ∈ S n can be indefinite. • Rewrite binary constraints x i ∈ { 0 , 1 } : x 2 X = xx T , i = x i ⇐ ⇒ diag ( X ) = x . • Semidefinite relaxation: X � xx T , diag ( X ) = x .
Combinatorial relaxations Lifted non-convex problem: � Q , X � + c T x minimize subject to diag ( X ) = x X = xx T . Semidefinite relaxation: � Q , X � + c T x minimize subject to diag ( X ) = x � X � x � 0 . x T 1 • Relaxation is exact if X = xx T . • Otherwise can be strengthened, e.g., by adding X ij ≥ 0.
Combinatorial relaxations Lifted non-convex problem: � Q , X � + c T x minimize subject to diag ( X ) = x X = xx T . Semidefinite relaxation: � Q , X � + c T x minimize subject to diag ( X ) = x � X � x � 0 . x T 1 • Relaxation is exact if X = xx T . • Otherwise can be strengthened, e.g., by adding X ij ≥ 0.
Combinatorial relaxations Lifted non-convex problem: � Q , X � + c T x minimize subject to diag ( X ) = x X = xx T . Semidefinite relaxation: � Q , X � + c T x minimize subject to diag ( X ) = x � X � x � 0 . x T 1 • Relaxation is exact if X = xx T . • Otherwise can be strengthened, e.g., by adding X ij ≥ 0.
Combinatorial relaxations Lifted non-convex problem: � Q , X � + c T x minimize subject to diag ( X ) = x X = xx T . Semidefinite relaxation: � Q , X � + c T x minimize subject to diag ( X ) = x � X � x � 0 . x T 1 • Relaxation is exact if X = xx T . • Otherwise can be strengthened, e.g., by adding X ij ≥ 0.
Relaxations for boolean optimization Same approach used for boolean constraints x i ∈ {− 1 , +1 } . Lifting of boolean constraints Rewrite boolean constraints x i ∈ {− 1 , 1 } : x 2 X = xx T , i = 1 ⇐ ⇒ diag ( X ) = e . Semidefinite relaxation of boolean constraints X � xx T , diag ( X ) = e .
Relaxations for boolean optimization Example: MAXCUT Undirected graph G with vertices V and edges E . A cut partitions V into disjoint sets S and T with cut-set I = { ( u , v ) ∈ E | u ∈ S , v ∈ T } . The capacity of a cut is | I | . The cut { v 2 , v 4 , v 5 } has capacity 9.
Relaxations for boolean optimization Example: MAXCUT Let � +1 , v i ∈ S x i = − 1 , v i / ∈ S and assume x i ∈ S . Then � 2 , v j ∈ S 1 − x i x j = ∈ S . 0 , v j / If A is the adjancency matrix for G , then the capacity is cap( x ) = 1 (1 − x i x j ) = 1 � � (1 − x i x j ) A ij , 2 4 ( i , j ) ∈ E i , j i.e, the MAXCUT problem is 1 4 e T Ae − 1 4 x T Ax maximize x ∈ {− 1 , +1 } n . subject to Exercise: Implement a SDP relaxation for G on the previous slide.
Sums-of-squares relaxations • f : multivariate polynomial of degree 2 d . • v d = (1 , x 1 , x 2 , . . . , x n , x 2 1 , x 1 x 2 , . . . , x 2 n , . . . , x d n ). Vector of monomials of degree d or less. Sums-of-squares representation f is a sums-of-squares (SOS) iff f ( x 1 , . . . , x n ) = v T d Qv d , Q � 0 . If Q = LL T then m � f ( x 1 , . . . , x n ) = v T d LL T v d = ( l T i v d ) 2 . i =1 Sufficient condition for f ( x 1 , . . . , x n ) ≥ 0.
Sums-of-squares relaxations • f : multivariate polynomial of degree 2 d . • v d = (1 , x 1 , x 2 , . . . , x n , x 2 1 , x 1 x 2 , . . . , x 2 n , . . . , x d n ). Vector of monomials of degree d or less. Sums-of-squares representation f is a sums-of-squares (SOS) iff f ( x 1 , . . . , x n ) = v T d Qv d , Q � 0 . If Q = LL T then m � f ( x 1 , . . . , x n ) = v T d LL T v d = ( l T i v d ) 2 . i =1 Sufficient condition for f ( x 1 , . . . , x n ) ≥ 0.
A simple example Consider f ( x , z ) = 2 x 4 + 2 x 3 z − x 2 z 2 + 5 z 4 , homogeneous of degree 4, so we only need � z 2 � x 2 v = . xz Comparing cofficients of f ( x , z ) and v T Qv = � Q , vv T � , x 4 x 3 z x 2 z 2 q 00 q 01 q 02 , x 3 z x 2 z 2 xz 3 � Q , vv T � = � q 10 q 11 q 12 � x 2 z 2 xz 3 z 4 q 20 q 21 q 22 we see that f ( x , z ) is SOS iff Q � 0 and q 00 = 2 , 2 q 10 = 2 , 2 q 20 + q 11 = − 1 , 2 q 21 = 0 , q 22 = 5 .
A simple example Consider f ( x , z ) = 2 x 4 + 2 x 3 z − x 2 z 2 + 5 z 4 , homogeneous of degree 4, so we only need � z 2 � x 2 v = . xz Comparing cofficients of f ( x , z ) and v T Qv = � Q , vv T � , x 4 x 3 z x 2 z 2 q 00 q 01 q 02 , x 3 z x 2 z 2 xz 3 � Q , vv T � = � q 10 q 11 q 12 � x 2 z 2 xz 3 z 4 q 20 q 21 q 22 we see that f ( x , z ) is SOS iff Q � 0 and q 00 = 2 , 2 q 10 = 2 , 2 q 20 + q 11 = − 1 , 2 q 21 = 0 , q 22 = 5 .
Applications in polynomial optimization f ( x , z ) = 4 x 2 − 21 10 x 4 + 1 3 x 6 + xz − 4 z 2 + 4 z 4 Global lower bound Replace non-tractable problem, 6 5 minimize f ( x , z ) 4 3 f ( x, z ) 2 by a tractable lower bound 1 0 maximize t -1 -2 subject to f ( x , z ) − t is SOS . -1.0 -0.5 -2.0 -1.5 -1.0 0.0 -0.5 z 0.0 0.5 0.5 1.0 x 1.5 Relaxation finds the global optimum t = − 1 . 031.
f ( x , z ) − t = 4 x 2 − 21 10 x 4 + 1 3 x 6 + xz − 4 z 2 + 4 z 4 − t x 2 z 2 x 3 x 2 z xz 2 z 3 1 x z xz x 2 x 3 x 2 z xz 2 x 4 x 3 z x 2 z 2 xz 3 x xz z 2 x 2 z xz 2 z 3 x 3 z x 2 z 2 xz 3 z 4 z xz x 2 x 3 x 2 z x 4 x 3 z x 2 z 2 x 5 x 4 z x 3 z 2 x 2 z 3 x 2 z xz 2 x 3 z x 2 z 2 xz 3 x 4 z x 3 z 2 x 2 z 3 xz 4 xz vv T = z 2 xz 2 z 3 x 2 z 2 xz 3 z 4 x 3 z 2 x 2 z 3 xz 4 y 5 x 3 x 4 x 3 z x 5 x 4 z x 3 z 2 x 6 x 5 z x 4 z 2 x 3 z 3 x 2 z x 3 z x 2 z 2 x 4 z x 3 z 2 x 2 z 3 x 5 z x 4 z 2 x 3 z 3 x 2 z 4 xz 2 x 2 z 2 xz 3 x 3 z 2 x 2 z 3 xz 4 x 4 z 2 x 3 z 3 x 2 z 4 xz 5 z 3 xz 3 z 4 x 2 z 3 xz 4 z 5 x 3 z 3 x 2 z 4 xz 5 z 6 By comparing cofficients of v T Qv and f ( x , z ) − t : (2 q 72 + q 44 ) = − 21 q 77 = 1 q 00 = − t , (2 q 30 + q 11 ) = 4 , 10 , 3 2( q 51 + q 32 ) = 1 , (2 q 61 + q 33 ) = − 4 , (2 q 10 , 3 + q 66 ) = 4 2 q 10 = 0 , 2 q 20 = 0 , 2( q 71 + q 42 ) = 0 , . . . A standard SDP with a 10 × 10 variable and 28 constraints.
Nonnegative polynomials • Univariate polynomial of degree 2 n : f ( x ) = c 0 + c 1 x + · · · + c 2 n x 2 n . • Nonnegativity is equivalent to SOS, i.e., f ( x ) = v T Qv , f ( x ) ≥ 0 ⇐ ⇒ Q � 0 with v = (1 , x , . . . , x n ). • Simple extensions for nonnegativity on a subinterval I ⊂ R .
Nonnegative polynomials • Univariate polynomial of degree 2 n : f ( x ) = c 0 + c 1 x + · · · + c 2 n x 2 n . • Nonnegativity is equivalent to SOS, i.e., f ( x ) = v T Qv , f ( x ) ≥ 0 ⇐ ⇒ Q � 0 with v = (1 , x , . . . , x n ). • Simple extensions for nonnegativity on a subinterval I ⊂ R .
Nonnegative polynomials • Univariate polynomial of degree 2 n : f ( x ) = c 0 + c 1 x + · · · + c 2 n x 2 n . • Nonnegativity is equivalent to SOS, i.e., f ( x ) = v T Qv , f ( x ) ≥ 0 ⇐ ⇒ Q � 0 with v = (1 , x , . . . , x n ). • Simple extensions for nonnegativity on a subinterval I ⊂ R .
Polynomial interpolation Fit a polynomial of degree n to a set of points ( x j , y j ), f ( x j ) = y j , j = 1 , . . . , m , i.e., linear equality constraints in c , x 2 x n 1 x 1 . . . c 0 y 1 1 1 x 2 x n 1 . . . x 2 c 1 y 2 2 2 = . . . . . . . . . . . . . . . . . . x 2 x n 1 x m . . . c n y m m m Semidefinite shape constraints: • Nonnegativity f ( x ) ≥ 0. • Monotonicity f ′ ( x ) ≥ 0. • Convexity f ′′ ( x ) ≥ 0.
Polynomial interpolation Fit a polynomial of degree n to a set of points ( x j , y j ), f ( x j ) = y j , j = 1 , . . . , m , i.e., linear equality constraints in c , x 2 x n 1 x 1 . . . c 0 y 1 1 1 x 2 x n 1 . . . x 2 c 1 y 2 2 2 = . . . . . . . . . . . . . . . . . . x 2 x n 1 x m . . . c n y m m m Semidefinite shape constraints: • Nonnegativity f ( x ) ≥ 0. • Monotonicity f ′ ( x ) ≥ 0. • Convexity f ′′ ( x ) ≥ 0.
Polynomial interpolation A specific example Smooth interpolation f 2 1 Minimize largest derivative, x ∈ [ − 1 , 1] | f ′ ( x ) | minimize max 1 2 subject to f ( − 1) = 1 f (0) = 0 f (1) = 1 x − 1 1 or equivalently f 2 ( x ) = x 2 f ′ 2 (1) = 2 minimize z subject to − z ≤ f ′ ( x ) ≤ z f ( − 1) = 1 f (0) = 0 f (1) = 1 .
Polynomial interpolation A specific example Smooth interpolation 1 Minimize largest derivative, x ∈ [ − 1 , 1] | f ′ ( x ) | minimize max f 4 1 2 subject to f ( − 1) = 1 f (0) = 0 f (1) = 1 x − 1 1 or equivalently f 2 ( x ) = x 2 f ′ 2 (1) = 2 minimize z subject to − z ≤ f ′ ( x ) ≤ z √ f 4 ( x ) = 3 2 x 2 − 1 4 ( 1 2 x 4 f ′ √ ) = 2 f ( − 1) = 1 2 f (0) = 0 f (1) = 1 .
Polynomial interpolation A specific example Smooth interpolation 1 Minimize largest derivative, x ∈ [ − 1 , 1] | f ′ ( x ) | minimize max 1 2 subject to f ( − 1) = 1 f 16 f (0) = 0 f (1) = 1 x − 1 1 or equivalently f 2 ( x ) = x 2 f ′ 2 (1) = 2 minimize z subject to − z ≤ f ′ ( x ) ≤ z √ f 4 ( x ) = 3 2 x 2 − 1 4 ( 1 2 x 4 f ′ √ ) = 2 f ( − 1) = 1 2 f (0) = 0 f (1) = 1 .
Polynomial interpolation A specific example Smooth interpolation f 2 1 Minimize largest derivative, x ∈ [ − 1 , 1] | f ′ ( x ) | minimize max f 4 1 2 subject to f ( − 1) = 1 f 16 f (0) = 0 f (1) = 1 x − 1 1 or equivalently f 2 ( x ) = x 2 f ′ 2 (1) = 2 minimize z subject to − z ≤ f ′ ( x ) ≤ z √ f 4 ( x ) = 3 2 x 2 − 1 4 ( 1 2 x 4 f ′ √ ) = 2 f ( − 1) = 1 2 f (0) = 0 f (1) = 1 .
Optimizing over Hermitian semidefinite matrices Let X ∈ H n + be a Hermitian semidefinite matrix of order n with inner product � � V , W � := tr ( V H W ) = V ∗ ij W ij = vec ( V ) H vec ( W ) . ij Then z H Xz = ( ℜ z − i ℑ z ) T ( ℜ X + i ℑ X )( ℜ z + i ℑ z ) � ℜ z � T � ℜ X � � ℜ z � −ℑ X ∀ z ∈ C n . = ≥ 0 , ℑ z ℑ X ℜ X ℑ z In other words, � ℜ X � −ℑ X ∈ S 2 n X ∈ H n ⇐ ⇒ + . + ℑ X ℜ X Note skew-symmetry ℑ X = −ℑ X T .
Optimizing over Hermitian semidefinite matrices Let X ∈ H n + be a Hermitian semidefinite matrix of order n with inner product � � V , W � := tr ( V H W ) = V ∗ ij W ij = vec ( V ) H vec ( W ) . ij Then z H Xz = ( ℜ z − i ℑ z ) T ( ℜ X + i ℑ X )( ℜ z + i ℑ z ) � ℜ z � T � ℜ X � � ℜ z � −ℑ X ∀ z ∈ C n . = ≥ 0 , ℑ z ℑ X ℜ X ℑ z In other words, � ℜ X � −ℑ X ∈ S 2 n X ∈ H n ⇐ ⇒ + . + ℑ X ℜ X Note skew-symmetry ℑ X = −ℑ X T .
Nonnegative trigonometric polynomials Consider a trigonometric polynomial: n � x i z − i ) , f ( z ) = x 0 + 2 ℜ ( | z | = 1 i =1 parametrized by x ∈ R × C n . Let T i be Toeplitz matrices with � 1 , k − l = i [ T i ] kl = i = 0 , . . . , n . 0 , otherwise Then f ( z ) ≥ 0 on the unit-circle iff X ∈ H n +1 , x i = � X , T i � , i = 0 , . . . , n . + Proved by Nesterov. Simple extensions for nonnegativity on subintervals.
Cones of nonnegative trigonometric polynomials Filter design example Consider a transfer function : n � x k e − j ω k ) . H ( ω ) = x 0 + 2 ℜ ( k =1 We can design a lowpass filter by solving minimize t subject to 0 ≤ H ( ω ) ∀ ω ∈ [0 , π ] 1 − δ ≤ H ( ω ) ≤ 1 + δ ∀ ω ∈ [0 , ω p ] H ( ω ) ≤ ∀ ω ∈ [ ω s , π ] , t where ω s and ω s are design parameters. The constraints all have simple semidefinite characterizations.
Cones of nonnegative trigonometric polynomials Filter design example Transfer function for n = 10 , δ = 0 . 05 , ω p = π/ 4 , ω s = ω p + π/ 8.
Power cone The ( n + 1)-dimensional power-cone is � x ∈ R n +1 | x α 1 � 1 x α 2 2 · · · x α n K α = ≥ | x n +1 | , x 1 , . . . , x n ≥ 0 n for α > 0 , e T α = 1. Dual cone: � s ∈ R n +1 | ( s 1 /α 1 ) α 1 · · · ( s n /α n ) α n ≥ | s n +1 | , s 1 , . . . , s n ≥ 0 � K ∗ α = The power cone is self-dual: T α K ∗ α = K α where T α := Diag ( α 1 , . . . , α n , 1) ≻ 0.
Power cone Simple examples Three dimensional power cone: Q α = { x ∈ R 3 | x α 1 x 1 − α ≥ | x 3 | , x 1 , x 2 ≥ 0 } . 2 • Epigraph of convex power p ≥ 1: | x | p ≤ t ⇐ ⇒ ( t , 1 , x ) ∈ Q 1 / p . • Epigraph of p -norm: e T z = t . � x � p ≤ t ⇐ ⇒ ( z i , t , x i ) ∈ Q 1 / p , �� � 1 / p | x i | p where � x � p := . i
Power cone 0.5 x 3 0.0 0 0.5 x 1 0 x 2 1 1 Q 3 / 4
Power cone 0.5 x 3 0.0 0 0.5 x 1 0 x 2 1 1 Q 1 / 2
Power cone 0.5 x 3 0.0 0 0.5 x 1 0 x 2 1 1 Q 1 / 4
Exponential cone Exponential cone: K exp = cl { x ∈ R 3 | x 1 ≥ x 2 e x 3 / x 2 , x 2 > 0 } = { x ∈ R 3 | x 1 ≥ x 2 e x 3 / x 2 , x 2 > 0 } ∪ ( R + × { 0 } × R − ) Dual cone: � s 3 − s 2 � exp = cl { s ∈ R 3 | s 1 ≥ ( − s 3 ) exp K ∗ , s 3 < 0 } − s 3 � s 3 − s 2 � = { s ∈ R 3 | s 1 ≥ ( − s 3 ) exp , s 3 < 0 } ∪ ( R 2 + × { 0 } ) . − s 3 Not a self-dual cone.
Exponential cone 0 x 3 2 4 0.0 0.5 0.0 x 2 0.5 x 1 1.0 1.0 K exp
Exponential cone Simple examples • Epigraph of negative logarithm: − log( x ) ≤ t ⇐ ⇒ ( x , 1 , − t ) ∈ K exp . • Epigraph of negative entropy: x log x ≤ t ⇐ ⇒ (1 , x , − t ) ∈ K exp . • Epigraph of Kullback-Leibler divergence (with variable p ): p i log p i � D ( p � q ) = ≤ t ⇐ ⇒ q i i � p i log p i ≤ p i log q i , p i log q i ≤ t i
Exponential cone Simple examples • Epigraph of exponential: e x ≤ t ⇐ ⇒ ( t , 1 , x ) ∈ K exp . • Epigraph of log of sum of exponentials: � e a T i x + b i ≤ t ( z i , 1 , a T e T z = 1 . log ⇐ ⇒ i x + b i − t ) ∈ K exp , i
Recommend
More recommend
