http://www.stanford.edu/yyye Joint work with Anthony So and Jiawei - PowerPoint PPT Presentation

SDP Rank Reduction Yinyu Ye, EURO XXII 1 A Unified Theorem on SDP Rank Reduction Yinyu Ye Department of Management Science and Engineering and Institute of Computational and Mathematical Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/˜yyye Joint work with Anthony So and Jiawei Zhang

SDP Rank Reduction Yinyu Ye, EURO XXII 2 Outline • Problem Statement • Application • New SDP Rank Reduction Theorem and Algorithm • Sketch of Proof • Extension and Question

SDP Rank Reduction Yinyu Ye, EURO XXII 3 Problem Statement • Consider the system of Semidefinite Programming constraints: A i • X = b i i = 1 , . . . , m, X � 0 where given A 1 , . . . , A m are n × n symmetric positive semidefinite i,j a ij x ij = Tr A T X . matrices, and b 1 , . . . , b m ≥ 0 , and A • X = � • Clearly, the feasibility of the above system can be “decided” by using SDP interior-point algorithms (Alizadeh 91, Nesterov and Nemirovskii 93, etc). • More precisely, find an ǫ -approximate solution where solution time is linear in log(1 /ǫ ) .

SDP Rank Reduction Yinyu Ye, EURO XXII 4 Problem Statement (Cont’d) • However, we are interested in finding a low–rank solution to the above system. • The low–rank problem arises in many applications, e.g.: – localizing sensor network (e.g., Biswas and Y 03, So and Y 04) – metric embedding/dimension reduction (e.g., Johnson and Lindenstrauss 84, Matousek 90) – approximating non-convex (complex, quaternion) quadratic optimization (e.g., Nemirovskii, Roos and Terlaky 99, Luo, Sidiropoulos, Tseng and Zhang 06, Faybusovich 07) – graph rigidity/distance matix (e.g., Alfakih, Khandani and Wolkowicz 99, etc.)

SDP Rank Reduction Yinyu Ye, EURO XXII 5 Graph Realization Given a graph G = ( V, E ) and sets of non–negative weights, say { d ij : ( i, j ) ∈ E } and { θ ilj : ( i, l, j ) ∈ Θ } , the goal is to compute a realization of G in the Euclidean space R d for a given low dimension d , i.e. • to place the vertices of G in R d such that • the Euclidean distance between every pair of adjacent vertices ( i, j ) equals (or bounded) by the prescribed weight d ij ∈ E , and • the angle between edges ( i, l ) and ( j, l ) equals (or bounded) by the prescribed weight θ ilj ∈ Θ .

SDP Rank Reduction Yinyu Ye, EURO XXII 6 0.5 0.4 0.3 0.2 0.1 0 −0.1 −0.2 −0.3 −0.4 −0.5 −0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.5 Figure 1: 50-node 2-D Sensor Localization

SDP Rank Reduction Yinyu Ye, EURO XXII 7 Figure 2: A 3-D Tensegrity Graph Realization; provided by Anstreicher

SDP Rank Reduction Yinyu Ye, EURO XXII 8 Figure 3: Tensegrity Graph: A Needle Tower; provided by Anstreicher

SDP Rank Reduction Yinyu Ye, EURO XXII 9 Figure 4: Molecular Conformation: 1F39(1534 atoms) with 85% of distances below 6 ˚ A and 10% noise on upper and lower bounds

SDP Rank Reduction Yinyu Ye, EURO XXII 10 Math Programming: Rank-Constrained SDP Given a k ∈ R d , d ij ∈ N x , ˆ d kj ∈ N a , and v ilj ∈ Θ , find x i ∈ R d such that � x i − x j � 2 d 2 ( ≤ ) = ( ≥ ) ij , ∀ ( i, j ) ∈ N x , i < j, ˆ � a k − x j � 2 d 2 ( ≤ ) = ( ≥ ) kj , ∀ ( k, j ) ∈ N a , ( x i − x l ) T ( x j − x l ) ( ≤ ) = ( ≥ ) v ilj , ∀ ( i, l, j ) ∈ Θ , which lead to A i • X = b i i = 1 , . . . , m, X � 0 , rank ( X ) ≤ d ; and relaxed to A i • X = b i X � 0 . i = 1 , . . . , m,

SDP Rank Reduction Yinyu Ye, EURO XXII 11 Some Background • Barvinok 95 showed that if the system is feasible, then there exists a solution √ X whose rank is at most 2 m (also see Carath´ eodorys theorem). Moreover, Pataki 98 showed how to construct such an X efficiently. • Unfortunately, for the applications mentioned above, this is not enough. – We want a fixed-low-rank (say d ) solution! • However, there are some issues: – Such a solution may not exist! – Even if it does, one may not be able to find it efficiently. • So we consider an approximation of the problem.

SDP Rank Reduction Yinyu Ye, EURO XXII 12 Approximating the Problem We consider the problem of finding an ˆ X � 0 of rank at most d that satisfies the system approximately: β ( m, n, d ) · b i ≤ A i • ˆ X ≤ α ( m, n, d ) · b i ∀ i = 1 , . . . , m Here, distortion factors α ≥ 1 and β ∈ (0 , 1] . Clearly, the closer are both to 1 , the better.

SDP Rank Reduction Yinyu Ye, EURO XXII 13 Our Result Theorem 1. Suppose that the original system is feasible. Let r = max i { Rank ( A i ) } . Then, for any d ≥ 1 , there exists an ˆ X � 0 of rank at most d such that: 1 + 12 log(4 mr )  for 1 ≤ d ≤ 12 log(4 mr )   d  α ( m, n, d ) = � 12 log(4 mr )  1 + for d > 12 log(4 mr )   d

SDP Rank Reduction Yinyu Ye, EURO XXII 14 Our Result Theorem 1. Suppose that the original system is feasible. Let r = max i { Rank ( A i ) } . Then, for any d ≥ 1 , there exists an ˆ X � 0 of rank at most d such that: 1 + 12 log(4 mr )  for 1 ≤ d ≤ 12 log(4 mr )  d   α ( m, n, d ) = � 12 log(4 mr )  1 + for d > 12 log(4 mr )   d 1 1 2 log m  for 1 ≤ d ≤ 5 e ·   m 2 /d log log(2 m )      1 1 2 log m   log log(2 m ) < d ≤ 4 log(4 mr ) 4 e · for β ( m, n, d ) = log f ( m ) /d (2 m )     �  4 log(4 mr )   1 − for d > 4 log(4 mr )   d where f ( m ) = 3 log m/ log log(2 m ) .

SDP Rank Reduction Yinyu Ye, EURO XXII 15 Our Result Theorem 1. Suppose that the original system is feasible. Let r = max i { Rank ( A i ) } . Then, for any d ≥ 1 , there exists an ˆ X � 0 of rank at most d such that: 1 + 12 log(4 mr )  for 1 ≤ d ≤ 12 log(4 mr )  d   α ( m, n, d ) = � 12 log(4 mr )  1 + for d > 12 log(4 mr )   d  1 1 2 log m for 1 ≤ d ≤ 5 e ·   m 2 /d log log(2 m )      1 1 2 log m   log log(2 m ) < d ≤ 4 log(4 mr ) 4 e · for β ( m, n, d ) = log f ( m ) /d (2 m )     �  4 log(4 mr )   1 − for d > 4 log(4 mr )   d where f ( m ) = 3 log m/ log log(2 m ) . Moreover, such an ˆ X can be found in randomized polynomial time.

SDP Rank Reduction Yinyu Ye, EURO XXII 16 Some Remarks √ In general, the data parameter r can be bounded by 2 m , so that � log m � α ( m, n, d ) = 1 + O d and  � log m � � m − 2 /d � Ω for d = O   log log m  β ( m, n, d ) = � (log m ) − 3 log m/ ( d log log m ) �  Ω  otherwise 

SDP Rank Reduction Yinyu Ye, EURO XXII 17 Some Remarks (Cont’d) • In the region 1 ≤ d ≤ 2 log m/ log log(2 m ) , the lower bound β is independent of the ranks of A 1 , . . . , A m . 2 log m • f ( m ) /d ≤ 3 / 2 in the region d > log log(2 m ) . � 4 log(4 mr ) • 1 − is a constant in the region d > 4 log(4 mr ) d • Our result contains as special cases several well-known results in the literature.

SDP Rank Reduction Yinyu Ye, EURO XXII 18 Early Result: Metric Embedding • Given an n –point set V = { v 1 , . . . , v n } in R l , we would like to embed it into a low–dimensional Euclidean space as faithfully as possible. • Specifically, a map f : V → R d is an α –embedding (where α ≥ 1 ) if � u − v � 2 ≤ � f ( u ) − f ( v ) � 2 ≤ α · � u − v � 2 The goal is to find an f such that α is as small as possible. • It is known that: – for any ǫ > 0 , an (1 + ǫ ) –embedding into R O ( ǫ − 2 log n ) exists (Johnson–Lindenstrauss); – for any fixed d ≥ 1 , an O ( n 2 /d d − 1 / 2 log 1 / 2 n ) –embedding into R d exists (Matousek).

SDP Rank Reduction Yinyu Ye, EURO XXII 19 Early Result: Metric Embedding (Cont’d) We can get these results from our Theorem. We focus on the fixed d case. • Let { e i } m i =1 be the standard basis vectors, and set E ij = ( e i − e j )( e i − e j ) T . • Let U be the m × n matrix whose i –th column is v i . Then, X = U T U satisfies the system E ij • X = � v i − v j � 2 2 for 1 ≤ i < j ≤ n . • By our Theorem, we can find an ˆ X � 0 of rank at most d such that: 2 ≤ E ij • ˆ Ω( n − 4 /d ) · � v i − v j � 2 X ≤ O (log n/d ) · � v i − v j � 2 2 U T ˆ • Upon decomposing ˆ X = ˆ U , where ˆ U is d × n , we recover points u n ∈ R d such that: u 1 , . . . , ˆ ˆ Ω( n − 2 /d ) · � v i − v j � 2 ≤ � ˆ � u i − ˆ u j � 2 ≤ O ( log n/d ) · � v i − v j � 2 . The embedding results imply only a weaker version ( r = 1 ) of our theorem.

SDP Rank Reduction Yinyu Ye, EURO XXII 20 Early Result: Approximating QPs • Let A 1 , . . . , A m be positive semidefinite. Consider the following QP: v ∗ = maximize x T Ax subject to x T A i x ≤ 1 i = 1 , . . . , m and its natural SDP relaxation: v ∗ sdp = maximize A • X subject to A i • X ≤ 1 i = 1 , . . . , m ; X � 0 • Let X ∗ be an optimal solution to the SDP . • Nemirovskii et al. showed that one can randomly extract a rank– 1 matrix ˆ X from X ∗ such that it is feasible for the SDP and that X ] ≥ Ω(log − 1 m ) v ∗ . E [ A • ˆ