Facial Reduction in Cone Optimization with Applications to Matrix Completions Henry Wolkowicz Dept. Combinatorics and Optimization, University of Waterloo, Canada Wed. July 27, 2016, 2-3:20 PM at: DIMACS Workshop on Distance Geometry: Theory and Applications 1
** Motivation: Loss of Slater CQ/Facial reduction Slater condition – existence of a strictly feasible solution – is at the heart of convex optimization. Without Slater: first-order optimality conditions may fail; dual problem may yield little information; small perturbations may result in infeasibility; many software packages can behave poorly. a pronounced phenomenon: though Slater holds generically, surprisingly many models arising from relaxations of hard nonconvex problems show loss of strict feasibility, e.g., Matrix completions/compressive sensing, sensor network localization, SNL, EDM, POP , Molecular Conformation, QAP , GP , strengthened Max-Cut We concentrate on Semidefinite Programming, SDP. We look at various reasons and how to take advantage using two views of FACIAL REDUCTION, FR Main Ref: (in progress) “The many faces of degeneracy in conic optimization”, Drusvyatskiy, Wolkowicz ’16 ; 2
** Facial Reduction/Preprocessing for LP Primal-Dual Pair: A onto, m × n , P = { 1 , . . . , n } c ⊤ x min b ⊤ y max (LP-P) (LP-D) s.t. Ax = b , A ⊤ y ≤ c s.t. x ≥ 0 . Slater’s CQ for (LP-D) / Theorem of alternative Exactly One is True: ∃ ˆ x s.t. A ˆ x = b , ˆ (ˆ (I) x > 0 x ∈ ri F ) Slater point 0 � = z = A ⊤ y ≥ 0 , b ⊤ y = 0 ( � z , F � = 0 ) (II) exposing vector 3
Linear Programming Example, x ∈ R 5 � � min 2 6 − 1 − 2 7 x � 1 � 1 � � 1 1 1 0 s.t. x = − 1 − 1 − 1 1 0 1 x ≥ 0 (multiply by: y T = ( 1 1 ) ): Sum the two constraints get: 2 x 1 + x 4 + x 5 = 0 = ⇒ x 1 = x 4 = x 5 = 0 i.e., equiv. simplified problem/smaller face/ fewer constr. min 6 x 2 − x 3 s.t. x 2 + x 3 = 1 , x 2 , x 3 ≥ 0 , ( x 1 = x 4 = x 5 = 0 ) 4
Linear Programming, LP Slater’s CQ for (LP-P) / Theorem of alternative y s.t. c − A ⊤ ˆ c − A ⊤ ˆ i > 0 , ∀ i ∈ P =: P l � ∃ ˆ �� � y > 0 , y iff Ad = 0 , c ⊤ d = 0 , d ≥ 0 = ⇒ d = 0 ( ∗ ) i ∈ P e implicit equality constraints: Find 0 � = d ∗ to ( ∗ ) with max number of non-zeros (exposes minimal face containing feasible slacks) d ∗ ⇒ ( c − A ⊤ y ) i = 0 , ∀ y ∈ F y i ∈ P e ) i > 0 = (where F y is primal feasible set) 5
Make implicit-equalities explicit/ Regularizes LP Facial Reduction: A ⊤ y ≤ f c ; minimal face f � R n + proper primal-dual pair; dual of dual is primal ( c l ) ⊤ x l + ( c e ) ⊤ x e min b ⊤ y max x l A e � � � ( A l ) ⊤ y ≤ c l � A l s.t. = b (LP reg -P) s.t. (LP reg -D) x e ( A e ) ⊤ y = c e x l ≥ 0 , x e free Generalized Slater CQ holds - And! after deleting redundant equality constraints! Mangasarian-Fromovitz CQ (MFCQ) holds � y = c e � ( A e ) ⊤ is onto ( A l ) ⊤ ˆ y < c l , ( A e ) ⊤ ˆ ∃ ˆ y : MFCQ holds iff dual optimal set is compact Numerical difficulties if MFCQ fails; in particular for interior point methods! Modelling issue! 6
** General convex programming Ordinary convex programming, (OCP) b ⊤ y subject to g ( y ) ≤ 0 sup (CP) y ∈ R n , g i : R m → R convex, ∀ i ∈ P b ∈ R m ; g ( y ) = � � g i ( y ) Slater’s CQ; strict feasibility ∃ ˆ g i (ˆ y s.t. y ) < 0 , ∀ i (implies MFCQ) Slater’s CQ fails ⇐ ⇒ implicit equality constraints exist P e := { i ∈ P : g ( y ) ≤ 0 = ⇒ g i ( y ) = 0 } � = ∅ Let P l := P\P e and g l := ( g i ) i ∈P l , g e := ( g i ) i ∈P e 7
implicit equalities to equalities / Regularize OCP Minimal face f f = { z ∈ R m + : z i = 0 , ∀ i ∈ P e } � R m + (OCP) is equivalent to g ( y ) ≤ f 0 b ⊤ y sup g l ( y ) ≤ 0 (OCP reg ) s.t. y ∈ F e where F e := { y : g e ( y ) = 0 } . Then F e = { y : g e ( y ) ≤ 0 } , so is a convex set!! y ∈ F e : g l (ˆ ∃ ˆ Slater’s CQ holds for (OCP reg ) y ) < 0 modelling issue again! (BBZ Conditions ’80) 8
FYI Aside: Faithfully convex case Faithfully convex function f (Rockafellar’70 ) f affine on a line segment only if affine on complete line containing the segment (e.g. analytic convex functions) F e = { y : g e ( y ) = 0 } is an affine set Then: F e = { y : Vy = V ˆ y } for some ˆ y and full-row-rank matrix V . Then MFCQ holds for regularized b ⊤ y sup g l ( y ) ≤ (OCP reg ) s.t. 0 V ˆ Vy = y 9
* (FYI - full generality) Abstract convex program inf x f ( x ) s.t. g ( x ) � K 0 , x ∈ Ω (ACP) where: f : R n → R convex; g : R n → R m is K -convex K ⊂ R m closed convex cone; Ω ⊆ R n convex set a � K b ⇐ ⇒ b − a ∈ K , a ≺ K b ⇐ ⇒ b − a ∈ int K g ( α x + ( 1 − α y )) � K α g ( x ) + ( 1 − α ) g ( y ) , ∀ x , y ∈ R n , ∀ α ∈ [ 0 , 1 ] ∃ ˆ x ∈ Ω s.t. g (ˆ x ) ∈ − int K ( g ( x ) ≺ K 0 ) Slater’s CQ: guarantees strong duality (zero duality gap AND dual attainmment) (near) loss of strict feasibility, nearness to infeasibility, correlates with number of iterations & loss of accuracy Recall that Slater (M-F) is equivalent to a nonempty bounded dual optimal set. 10
Faces of Convex Sets - Useful for Charact. of Opt. Face of C , F � C F ⊆ C is a face of C if F contains any line segment in C whose relative interior intersects F . A convex cone F ⊆ K is a face of a convex cone K , F � K , if (simplified) x , y ∈ K and x + y ∈ F = ⇒ x , y ∈ F Polar (Dual) Cone/Conjugate Face K ∗ := { φ : � φ, k � ≥ 0 , ∀ k ∈ K } polar cone If F � K , the conjugate face of F is F c := F ⊥ ∩ K ∗ � K ∗ 11
Properties of Faces General case A face of a face is a face intersection of a face with a face is a face. Let C ⊆ K , then face ( C ) denotes the minimal face (intersection of faces) containing C . F � K is an exposed face if there exists φ ∈ K ∗ with F = K ∩ φ ⊥ F c is always exposed by x ∈ ri F . The SDP cone is facially exposed, all its faces are exposed. (In fact like R n + : S n + is a proper closed convex cone, self-dual and facially exposed.) 12
Regularize abstract convex program (full generality) inf x f ( x ) s.t. g ( x ) � K 0 , x ∈ Ω (ACP) (Borwein-W.’81 ) inf x f ( x ) s.t. g ( x ) � K f 0 , x ∈ Ω (ACP R ) K f where: is the minimal face , it is simple if we use the minimal face K f . Like LP We get a proper primal-dual pair? 13
Recall: (ACP) inf x f ( x ) s.t. g ( x ) � K 0 , x ∈ Ω polar cone: K ∗ = { φ : � φ, y � ≥ 0 , ∀ y ∈ K } . K f := face ( F ) minimal face containing feasible set F . Lemma (Facial Reduction (FR); find EXPOSING vector φ ) Suppose ¯ x is feasible. Then the LHS system � (Ω − ¯ x ) ∗ ∩ ∂ � φ, g (¯ � x ) � � = ∅ K f ⊆ φ ⊥ ∩ K , implies φ ∈ K ∗ , � φ, g (¯ x ) � = 0 where: ∂ is subgradient; �·� is inner-product. Proof line 1 of system implies ¯ x global min for convex function � φ, g ( · ) � on Ω ; i.e., 0 = � φ, g (¯ x ) � ≤ � φ, g ( x ) � ≤ 0 , ∀ x ∈ F ; implies − g ( F ) ⊆ φ ⊥ ∩ K . 14
* SDP Case/Replicating Cone/Faces SDP case/Replicating cone Let X ∈ S n + with spectral decomposition, � D + � 0 [ P Q ] T , D + ∈ S r X = [ P Q ] ( rank X = r ) ++ 0 0 Then Range ( X ) = Range ( P ) , Null ( X ) = Range ( Q ) + P T = ( QQ T ) ⊥ ∩ S n face ( X ) = P S r + . ( Z = QQ T exposing vector/matrix for face.) face ( X ) c = Q S n − r + Q T Range/Nullspace representations � Y ∈ S n � face ( X ) = + : Range ( Y ) ⊆ Range ( X ) Y ∈ S n � � face ( X ) = + : Null ( Y ) ⊇ Null ( X ) � Y ∈ S n � ri face ( X ) = + : Range ( Y ) = Range ( X ) 15
, S n Semidefinite Programming, SDP + K = S n + = K ∗ : nonpolyhedral, self-polar, facially exposed y ∈ R m b ⊤ y s.t. g ( y ) := A ∗ y − c � S n v P = sup + 0 (SDP-P) (SDP-D) v D = inf x ∈S n � c , x � s.t. A x = b , x � S n + 0 where: + ⊂ S n symm. matrices PSD cone S n c ∈ S n , b ∈ R m A : S n → R m is an onto linear map, with adjoint A ∗ A x = ( trace A i x ) = ( � A i , x � ) ∈ R m , A i ∈ S n A ∗ y = � m i = 1 A i y i ∈ S n 16
Slater’s CQ/Theorem of Alternative simplifies for SDP y s.t. c − A ∗ ˜ Assume feasibility: ∃ ˜ y � 0. Exactly one of the following alternatives holds/is consistent: y s.t. s = c − A ∗ ˆ ∃ ˆ ( I ) y ≻ 0 ( Slater ) or ( II ) A d = 0 , � c , d � = 0 , 0 � = d � 0 ( ∗ ) In case (II): - finds exposing vector: 0 � = d � 0 d exposes a proper face containing all the feasible slacks z = c − A ∗ y � 0 = ⇒ zd = 0 . ( equiv. trace zd = 0 ) 17
Regularization Using Minimal Face Borwein-W.’81 , f P = face F s P ; min. face of feasible slacks (SDP-P) is equivalent to the regularized {� b , y � : A ∗ y � f P c } v RP := sup (SDP reg -P) y f p is minimal face of primal feasible slacks { s � 0 : s = c − A ∗ y } ⊆ f p � S n + Lagrangian dual of regularized problem satisfies strong duality: v DRP := inf x {� c , x � : A x = b , x � f ∗ P 0 } (SDP reg -D) v P = v RP = v DRP and v DRP is attained. regularized primal-dual pair (dual of dual is primal) If we take the dual of (SDP reg -D) we recover the primal regularized problem (SDP reg -P) . 18
Recommend
More recommend
