A structural geometrical analysis of ill-conditioned semidefinite - - PowerPoint PPT Presentation

A structural geometrical analysis of ill-conditioned semidefinite programs

Takashi Tsuchiya

National Graduate Institute for Policy Studies (Tokyo, Japan) (Joint work with Bruno F. Loren¸ co (Tokyo Institute of Technology) and Masakazu Muramatsu (The University of Electro-Communications)) Symmetric Positive Semidefinite Matrices and Related Modeling, Mathematics and Algorithms National Graduate Institute for Policy Studies

Jaunary 14, 2014

1 / 1

Main topic of this talk — Weakly infeasible SDP

Outline

• SDP feasibility problem
• Basic instance of weakly infeasible SDP
• Preliminary and notations
• Reduction of problem
• Hyper feasible decomposition
• Detection of feasibility status
• Construction of a weakly infeasible sequence
• Main results

2 / 1

SDP feasibility problem

Consider the following SDP feasibility (SDPF) problem: find x ∈ (L + c) ∩ K, where:

• L: Linear subspace of n × n symmetric matrices, c: a n × n

symmetric matrix.

• K: The cone of n × n symmetric positive semidefinite

matrices (PSD cone).

3 / 1

Feasibility status of SDP

There are four feasibility status of semidefinite programming.

• Strongly feasible: There exists an interior feasible solution,

(L + c) ∩ int(K) = ∅

• Weakly feasible: There exists no interior feasible solution but

there exists a feasible solution on the surface of PSD cone (L + c) ∩ int(K) = ∅ but (L + c) ∩ K is nonempty.

• Weakly infeasible: There exists no feasible solution but the

distance between the affine space and PSD cone can be arbitrarily close to zero, (L + c) ∩ K = ∅ and dist(L + c, K) = 0.

• Strongly infeasible: There exists no feasible solution and the

distance between the affine space and PSD cone is strictly

• positive. (L + c) ∩ K = ∅ and dist(L + c, K) > 0

Weak infeasibilty is difficult to detect, since behavior at infinity is involved!

4 / 1

If we denote by λmin(x) the minimum eigenvalue of x,

• Strongly feasible: supx∈L+c λmin(x) > 0
• Weakly feasible: supx∈L+c λmin(x) = 0, and there exists

x ∈ L + c.

• Weakly infeasible: supx∈L+c λmin(x) = 0, but no x ∈ L + c

attains sup.

• Strongly infeasible: supx∈L+c λmin(x) < 0.

5 / 1

Previous Works

Analysis of SDP under irregular conditions.

• Borwein and Wolkowicz (Facial reduction)
• Ramana (Extended Lagrangian Dual Problem)
• Ramana, Tuncel and Wolkowicz (ELDP and Facial reduction)
• Porkolab and Khachiyan (Complexity)
• Sturm, Luo and Zhang (Facial Expansion)
• Pataki (Facial Structure, Common Structure of badly bahaved

SDP)

• Waki and Muramatsu (Facial Reduction and Facial Expansion)
• Waki (Construction of weak infeasible SDP)

6 / 1

Notation and Terminology

• Sn, Sn

+: the set of n × n symmetric matrices and symmetric

positive semidefinite matrices, respectively.

• Let W be a set of matrices and P be a matrix. Then, we

define PTWP = {x|x = PTyP, P ∈ W }.

• Let W be a set of n × n matrices. Then, we define

Lm(W ) = W22, where W = W11 W12 W T

12

W22

• n−m

m

• We define Um(W ) analogously, i.e., Un−m(W ) = W11.

7 / 1

• For a SDP feasibility problem (K, L, c), we denote the SDP

feasibility problem when we restrict to the lower-right m × m submatrix by Lm(K, L, c) = (Lm(K), Lm(L), Lm(c)). Lm(K, L, c) is referred to as the m-lower subproblem of (K, L, c). We use an analogous notation Um(K, L, c) to define m-upper-left subproblem.

8 / 1

Basic example of weakly infeasible problem

L + c = x y y z

• =
• t

−1 −1

• ,

K = (2 × 2 PSD cone) x-axis is on the boundary of K.

9 / 1

As t goes to infinity, the surface of the cone gets close to the xy-plane. This implies weak infeasibility.

10 / 1

Basic question Is every weakly infeasibility in SDP arises in the way as above?

11 / 1

Answer Weak infeasibility in SDP:

• The existence of a submatrix of the form

tA C C

• , A ≻ 0, C = 0 : independent of t

and

• Nested hierarchical structure of such

submatrices (level is at most n − 1).

12 / 1

Related question Let F be an affine space in Sn, and suppose that dist(F, Sn

+) = 0,

F ∩ Sn

+ = ∅.

(Touching but not intersect!) Then, what is the minimum dimension of the affine subspace F′ ⊆ F such that dist(F′, Sn

+) = 0,

F′ ∩ Sn

+ = ∅.

(Touching but not intersect!)

13 / 1

• The most optimistic position: dim(F′) = 1, i.e.,

we can always find a nice direction and displacement which would realize weak infeasibility in one shot!

• The most pessimistic position:

dim(F′) = dim(F), we cannot do much. (Note that dim(F) can be up to n(n + 1)/2.

14 / 1

Basic observation

• If (K, L, c) is weakly infeasible, then there exists

a(= 0) ∈ K ∩ L (recession cone exists).

• Feasibility status is invariant under congruence

transformation, i.e., let P be an invertible

• matrix. Then, the feasibility status of (K, L, c)

and (PKPT, PLPT, PcPT) = (K, PLPT, PcPT) are the same.

15 / 1

Theorem

Let (K, L, c) be a SDP feasibility problem and suppose that there is a matrix ˜ A with rank k, and ˜ A ∈ K ∩ L (recession cone). Let P be any non-singular matrix such that PT ˜ AP = A11

• k

n−k

• ,

A11 ≻ 0. Let us consider (n − k)-lower SDP feasibility subproblem L = Ln−k(K, PTLP, PTcP) of (K, PTLP, PTcP). Then,

• i. (K, L, c) is strongly feasible if and only if L is,
• ii. (K, L, c) is in weakly status if and only if L is,
• iii. (K, L, c) is strongly infeasible if and only if L is.

Here, weakly status means either weakly feasible of weakly infeasible.

16 / 1

Let us represent (K, PTLP, PTcP) as X = X11 X12 X T

12

X22

• 0,

Each element of X is an affine function. Then the theorem says that

• 1. X 0 is strongly feasible if and only if X22 is.
• 2. X 0 is in weakly status if and only if X22 is.
• 3. X 0 is strongly infeasible if and only if X22 is.

Applying this theorem repeatedly, we can determine completely feasibility status of SDP. (Detail omitted.)

17 / 1

Sketch of the proof: Let us represent (K, PTLP, PTcP) as X11 X12 X T

12

X22

• 0,

we have PAPT = A11

• ∈ PLPT.

We can always add tPAPT, t ≥ 0 to X. X11 + tA11 X12 X T

12

X22

• ∈ P(L + c)PT.

λmin X11 + tA11 X12 X T

12

X22

• → λmin(X22) − O(t−1).

18 / 1

tA C C T D

• =
• I

C T A−1 t

I tA D − C T A−1C

t

I

A−1C t

I

• I

C T A−1 t

I

• → I

(t → ∞) λmin tA C C T D

• ∼ λmin
• D − C TA−1C

t

• 19 / 1

Hyper-feasible partition

Consider the following procedure to produce repeatedly find “recession direction” of lower subproblems followed by rotation to make the direction upper left corner. X ′ = PXPT = X ′

11

∗ ∗ X ′

22

• with

A11

• :

recession direction X ′′ = I P′ X ′

11

∗ ∗ X ′

22

I P′T

• with

X ′′22 = P′X ′22P′T = X ′′11 ∗ ∗ X ′′22

• with

A′11

• :

recession direction for X ′′

22

20 / 1

This process can be continued until no recession direction is found. Then the last lower subproblem X (∞)

22

is either strongly infeasible, weakly feasible or strongly feasible. If X (∞)

22

is strongly feasible (infeasible), then so is the original problem. If X (∞)

22

is weakly feasible, then the original problem can be weakly feasible or weakly infeasible due to the above mentioned theorm.

21 / 1

Detecting feasibility status We explain how to determine feasibility status by making use of hyper-feasible partition by illustrative

• example. Consider a weakly infeasible SDP (K, L, c)

whose hyper-feasible partition is as follows:       X11 X12 X13 X14 X15 X T

12 X22 X23 X24 X25

X T

13 X T 23 X33 X34 X35

X T

14 X T 24 X T 34 X44 X45

X T

15 X T 25 X T 35 X T 45 X55

      .

22 / 1

L admits the following matrices (*: wild card): ˆ A1 =       A11       , ˆ A2 =       ∗ ∗ ∗ ∗ ∗ ∗ A22 ∗ ∗ ∗       , ˆ A3 =       ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ A33 ∗ ∗ ∗ ∗       . with A11, A22, A33 ≻ 0.

23 / 1

X44 X45 X T

45

X55

• 0 does not admit a recession direction.

24 / 1

Then, X44 X45 X T

45

X55

• 0 determines the original problem

feasibility status. If strongly feasible or strongly infeasible, then

• done. In the case of weakly feasible, we do as follows.

By appropriate rotation we can assume, without loss of generality, there exist a feasible element for the last block such that: X44

• ,

X44 ≻ 0,

25 / 1

In this setting, we impose (X15, X25, X35, X45, X55) = 0 as necessary condition for feasibility. Now, we try to check a reduced problem.       X11 X12 X13 X14 X T

12

X22 X23 X24 X T

13

X T

23

X33 X34 X T

14

X T

24

X T

34

X44       0. Continuing this way, the problem dimension is reduced to one dimensional feasibility problem, which is trivial. (Feasibility or infeasibility can also be detected in the middle. Anyway, feasibility

• r infeasibility of a reduced problem means weak feasibility or weak

infeasibility of the original problem.)

26 / 1

Construction of weakly infeasible sequence We explain how to construct a weakly infeasible sequence in a weakly infeasible problem by making use of hyper-feasible partition by illustrative

• example. Consider a weakly infeasible SDP (K, L, c)

whose hyper-feasible partition is as follows:       X11 X12 X13 X14 X15 X T

12 X22 X23 X24 X25

X T

13 X T 23 X33 X34 X35

X T

14 X T 24 X T 34 X44 X45

X T

15 X T 25 X T 35 X T 45 X55

      .

27 / 1

L admits the following matrices (*: wild card): ˆ A1 =       A11       , ˆ A2 =       ∗ ∗ ∗ ∗ ∗ ∗ A22 ∗ ∗ ∗       , ˆ A3 =       ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ A33 ∗ ∗ ∗ ∗       . with A11, A22, A33 ≻ 0.

28 / 1

X44 X45 X T

45

X55

• 0 does not admit a recession direction.

29 / 1

Then, X44 X45 X T

45

X55

• 0 is weakly feasible.

By appropriate rotation we can assume, without loss of generality, there exist a feasible element for the last block such that: X44

• ,

X44 ≻ 0, In the following, we fix such an element.

30 / 1

Construction of a weakly infeasible sequence

Now, we can find in L + c a point which is close to as we like by adding to X the matrices ˆ A3, ˆ A2, ˆ A1 ∈ L as follows: X+t1ˆ A3 =       X11 X12 X13 X14 X15 X T

12

X22 X23 X24 X25 X T

13

X T

23

X33 X34 X35 X T

14

X T

24

X T

34

X44 X T

15

X T

25

X T

35

      +t1       ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ A33 ∗ ∗ ∗ ∗      

31 / 1

Construction of a weakly infeasible sequence

This will control the minimum eigenvalue of the lower 3 × 3 matrix, but addition of ˆ A3 contaminates the remaining part of the

• matrix. But this will be fixed by adding t2ˆ

A2 and t1ˆ

• A1. Intuitively,

t1 << t2 << t3. X + t1ˆ A3 =       ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ X33 + t1A33 X34 X35 ∗ ∗ X34 X44 ∗ ∗ X T

35

      (∗ = O(t1).)

32 / 1

Construction of a weakly infeasible sequence

This will control the minimum eigenvalue of the lower 3 × 3 matrix, but addition of ˆ A3 contaminates the remaining part of the

• matrix. But this will be fixed by adding t2ˆ

A2 and t1ˆ

• A1. Intuitively,

t1 << t2 << t3. X + t1ˆ A3 + t2ˆ A2 =       ∗∗ ∗∗ ∗∗ ∗∗ ∗∗ ∗∗ t2A2 ∗ ∗ ∗ ∗∗ ∗ X33 + t1A33 X34 X35 ∗∗ ∗ X34 X44 ∗∗ ∗ X T

35

      (∗ = O(t1), ∗∗ = O(t2).)

33 / 1

X + t1ˆ A3 + t2ˆ A2 + t3ˆ A1 =       ∗ ∗ +t3A11 ∗∗ ∗∗ ∗∗ ∗∗ ∗∗ ∗ + t2A22 ∗ ∗ ∗ ∗∗ ∗ X33 + t1A33 X34 X35 ∗∗ ∗ X34 X44 ∗∗ ∗ X T

35

      (∗ = O(t1), ∗∗ = O(t2).)

34 / 1

(Note that X15, X25, X35 = 0 for X44 ≻ 0, if all of them are zero, then the problem is feasible.)

35 / 1

Answer (again) Weakly infeasibility in SDP:

• The existence of a principal submatrix of the

form tA C C

• , A ≻ 0, C = 0 : independent of t

and

• Nested hierarchical structure of such principal

submatrices (level is at most n − 1).

35 / 1

Main results

Theorem

If the last lower-subproblem in the hyper-feasible partition is weakly feasible and the original problem is infeasible, then the

• riginal problem is weakly infeasible.

Theorem

Let K = Sn

+ and F ∈ Sn be an affine space. If dist(K, F) = 0 and

K ∩ F = ∅, then, there always exists an affine subspace G ⊆ F whose dimension is at most n − 1 such that dist(K, G) = 0 and K ∩ G = ∅. Remark: (Optimistic) 1 ≤ n − 1 ≤ n(n + 1)/2 (Pessimistic)

36 / 1

Thank you!

37 / 1