A structural geometrical analysis of ill-conditioned semidefinite - PowerPoint PPT Presentation

A structural geometrical analysis of ill-conditioned semidefinite programs Takashi Tsuchiya National Graduate Institute for Policy Studies (Tokyo, Japan) (Joint work with Bruno F. Loren¸ co (Tokyo Institute of Technology) and Masakazu Muramatsu (The University of Electro-Communications)) Symmetric Positive Semidefinite Matrices and Related Modeling, Mathematics and Algorithms National Graduate Institute for Policy Studies Jaunary 14, 2014 1 / 1

Main topic of this talk — Weakly infeasible SDP Outline • SDP feasibility problem • Basic instance of weakly infeasible SDP • Preliminary and notations • Reduction of problem • Hyper feasible decomposition • Detection of feasibility status • Construction of a weakly infeasible sequence • Main results 2 / 1

SDP feasibility problem Consider the following SDP feasibility (SDPF) problem: find x ∈ ( L + c ) ∩ K , where: • L : Linear subspace of n × n symmetric matrices, c : a n × n symmetric matrix. • K : The cone of n × n symmetric positive semidefinite matrices (PSD cone). 3 / 1

Feasibility status of SDP There are four feasibility status of semidefinite programming. • Strongly feasible: There exists an interior feasible solution, ( L + c ) ∩ int ( K ) � = ∅ • Weakly feasible: There exists no interior feasible solution but there exists a feasible solution on the surface of PSD cone ( L + c ) ∩ int ( K ) = ∅ but ( L + c ) ∩ K is nonempty. • Weakly infeasible: There exists no feasible solution but the distance between the affine space and PSD cone can be arbitrarily close to zero, ( L + c ) ∩ K = ∅ and dist ( L + c , K ) = 0. • Strongly infeasible: There exists no feasible solution and the distance between the affine space and PSD cone is strictly positive. ( L + c ) ∩ K = ∅ and dist ( L + c , K ) > 0 Weak infeasibilty is difficult to detect, since behavior at infinity is involved! 4 / 1

If we denote by λ min ( x ) the minimum eigenvalue of x , • Strongly feasible: sup x ∈ L + c λ min ( x ) > 0 • Weakly feasible: sup x ∈ L + c λ min ( x ) = 0, and there exists x ∈ L + c . • Weakly infeasible: sup x ∈ L + c λ min ( x ) = 0, but no x ∈ L + c attains sup. • Strongly infeasible: sup x ∈ L + c λ min ( x ) < 0. 5 / 1

Previous Works Analysis of SDP under irregular conditions. • Borwein and Wolkowicz (Facial reduction) • Ramana (Extended Lagrangian Dual Problem) • Ramana, Tuncel and Wolkowicz (ELDP and Facial reduction) • Porkolab and Khachiyan (Complexity) • Sturm, Luo and Zhang (Facial Expansion) • Pataki (Facial Structure, Common Structure of badly bahaved SDP) • Waki and Muramatsu (Facial Reduction and Facial Expansion) • Waki (Construction of weak infeasible SDP) 6 / 1

Notation and Terminology • S n , S n + : the set of n × n symmetric matrices and symmetric positive semidefinite matrices, respectively. • Let W be a set of matrices and P be a matrix. Then, we define P T WP = { x | x = P T yP , P ∈ W } . • Let W be a set of n × n matrices. Then, we define � W 11 � W 12 L m ( W ) = W 22 , where W = W T W 22 12 ���� ���� n − m m We define U m ( W ) analogously, i.e., U n − m ( W ) = W 11 . 7 / 1

• For a SDP feasibility problem ( K , L , c ), we denote the SDP feasibility problem when we restrict to the lower-right m × m submatrix by L m ( K , L , c ) = ( L m ( K ) , L m ( L ) , L m ( c )) . L m ( K , L , c ) is referred to as the m -lower subproblem of ( K , L , c ). We use an analogous notation U m ( K , L , c ) to define m -upper-left subproblem. 8 / 1

Basic example of weakly infeasible problem �� x � � �� − 1 y t L + c = = , K = (2 × 2 PSD cone) y z − 1 0 x -axis is on the boundary of K . 9 / 1

As t goes to infinity, the surface of the cone gets close to the xy -plane. This implies weak infeasibility. 10 / 1

Basic question Is every weakly infeasibility in SDP arises in the way as above? 11 / 1

Answer Weak infeasibility in SDP: • The existence of a submatrix of the form � tA C � , A ≻ 0 , C � = 0 : independent of t 0 C and • Nested hierarchical structure of such submatrices (level is at most n − 1). 12 / 1

Related question Let F be an affine space in S n , and suppose that dist ( F , S n F ∩ S n + ) = 0 , + = ∅ . (Touching but not intersect!) Then, what is the minimum dimension of the affine subspace F ′ ⊆ F such that F ′ ∩ S n dist ( F ′ , S n + = ∅ . + ) = 0 , (Touching but not intersect!) 13 / 1

• The most optimistic position: dim ( F ′ ) = 1, i.e., we can always find a nice direction and displacement which would realize weak infeasibility in one shot! • The most pessimistic position: dim ( F ′ ) = dim ( F ), we cannot do much. (Note that dim ( F ) can be up to n ( n + 1) / 2. 14 / 1

Basic observation • If ( K , L , c ) is weakly infeasible, then there exists a ( � = 0) ∈ K ∩ L (recession cone exists). • Feasibility status is invariant under congruence transformation, i.e., let P be an invertible matrix. Then, the feasibility status of ( K , L , c ) and ( PKP T , PLP T , PcP T ) = ( K , PLP T , PcP T ) are the same. 15 / 1

Theorem Let ( K , L , c ) be a SDP feasibility problem and suppose that there is a matrix ˜ A with rank k, and ˜ A ∈ K ∩ L (recession cone). Let P be any non-singular matrix such that � A 11 � 0 P T ˜ AP = , A 11 ≻ 0 . 0 0 ���� ���� k n − k Let us consider ( n − k ) -lower SDP feasibility subproblem L = L n − k ( K , P T LP , P T cP ) of ( K , P T LP , P T cP ) . Then, i. ( K , L , c ) is strongly feasible if and only if L is, ii. ( K , L , c ) is in weakly status if and only if L is, iii. ( K , L , c ) is strongly infeasible if and only if L is. Here, weakly status means either weakly feasible of weakly infeasible. 16 / 1

Let us represent ( K , P T LP , P T cP ) as � X 11 � X 12 � 0 , X = Each element of X is an affine function . X T X 22 12 Then the theorem says that 1. X � 0 is strongly feasible if and only if X 22 is. 2. X � 0 is in weakly status if and only if X 22 is. 3. X � 0 is strongly infeasible if and only if X 22 is. Applying this theorem repeatedly, we can determine completely feasibility status of SDP. (Detail omitted.) 17 / 1

Sketch of the proof: Let us represent ( K , P T LP , P T cP ) as � X 11 � X 12 � 0 , X T X 22 12 we have � A 11 � 0 PAP T = ∈ PLP T . 0 0 We can always add tPAP T , t ≥ 0 to X . � X 11 + tA 11 � X 12 ∈ P ( L + c ) P T . X T X 22 12 � X 11 + tA 11 � X 12 → λ min ( X 22 ) − O ( t − 1 ) . λ min X T X 22 12 18 / 1

� tA � C C T D � � tA � � � � A − 1 C I 0 0 I = t C T A − 1 D − C T A − 1 C I 0 0 I t t � � I 0 → I ( t → ∞ ) C T A − 1 I t � tA � � � D − C T A − 1 C C ∼ λ min λ min C T D t 19 / 1

Hyper-feasible partition Consider the following procedure to produce repeatedly find “recession direction” of lower subproblems followed by rotation to make the direction upper left corner. � X ′ � A 11 � � ∗ 0 PXP T = 11 X ′ = with : ∗ X ′ 0 0 22 recession direction � I � � X ′ � � I � 0 ∗ 0 X ′′ 11 = P ′ T 0 P ′ ∗ X ′ 0 22 with � X ′′ 11 � A ′ 11 � � ∗ 0 P ′ X ′ 22 P ′ T = X ′′ 22 = with : ∗ X ′′ 22 0 0 recession direction for X ′′ 22 20 / 1

This process can be continued until no recession direction is found. Then the last lower subproblem X ( ∞ ) is either strongly infeasible, 22 weakly feasible or strongly feasible. If X ( ∞ ) is strongly feasible 22 (infeasible), then so is the original problem. If X ( ∞ ) is weakly 22 feasible, then the original problem can be weakly feasible or weakly infeasible due to the above mentioned theorm. 21 / 1

Detecting feasibility status We explain how to determine feasibility status by making use of hyper-feasible partition by illustrative example. Consider a weakly infeasible SDP ( K , L , c ) whose hyper-feasible partition is as follows:   X 11 X 12 X 13 X 14 X 15 X T 12 X 22 X 23 X 24 X 25     X T 13 X T 23 X 33 X 34 X 35 .     X T 14 X T 24 X T 34 X 44 X 45   X T 15 X T 25 X T 35 X T 45 X 55 22 / 1

L admits the following matrices (*: wild card):     A 11 0 0 0 0 ∗ ∗ ∗ ∗ ∗ 0 0 0 0 0 ∗ A 22 0 0 0         ˆ ˆ A 1 = 0 0 0 0 0 , A 2 = ∗ 0 0 0 0 ,         ∗ 0 0 0 0 0 0 0 0 0     0 0 0 0 0 ∗ 0 0 0 0   ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗     ˆ A 3 = ∗ ∗ A 33 0 0 .     ∗ ∗ 0 0 0   ∗ ∗ 0 0 0 with A 11 , A 22 , A 33 ≻ 0. 23 / 1

� X 44 � X 45 � 0 does not admit a recession direction. X T X 55 45 24 / 1

� X 44 � X 45 Then, � 0 determines the original problem X T X 55 45 feasibility status. If strongly feasible or strongly infeasible, then done. In the case of weakly feasible, we do as follows. By appropriate rotation we can assume, without loss of generality, there exist a feasible element for the last block such that: � X 44 � 0 X 44 ≻ 0 , , 0 0 25 / 1