OUTLINE l p -Norm Constrained Quadratic Programming: Conic Approximation Methods Wenxun Xing Department of Mathematical Sciences Tsinghua University, Beijing Email: wxing@math.tsinghua.edu.cn thu-bell W. Xing Sept. 2-4, 2014, Peking University
OUTLINE OUTLINE l p -Norm Constrained Quadratic Programming 1 Linear Conic Programming Reformulation 2 Complexity 3 Approximation Scheme 4 Questions 5 thu-bell W. Xing Sept. 2-4, 2014, Peking University
lp -Norm Constrained Quadratic Programming Linear Conic Programming Reformulation Complexity Approximation Scheme Questions Data fitting l 2 -norm: least-square data fitting min � Ax − b � 2 x ∈ R n . s . t . When A is full rank in column, then x ∗ = ( A T A ) − 1 A T b . A 2nd-order conic programming formulation min t � Ax − b � 2 ≤ t s . t . x ∈ R n . Experts in numerical analysis prefer the direct calculation thu-bell much more than the optimal solution method. W. Xing Sept. 2-4, 2014, Peking University
lp -Norm Constrained Quadratic Programming Linear Conic Programming Reformulation Complexity Approximation Scheme Questions l 1 - norm problem l 1 -norm. min � x � 1 s . t . Ax = b x ∈ R n . A linear programming formulation � n min i = 1 t i − t i ≤ x i ≤ t i , i = 1 , 2 , . . . , n s . t . Ax = b t , x ∈ R n . thu-bell W. Xing Sept. 2-4, 2014, Peking University
lp -Norm Constrained Quadratic Programming Linear Conic Programming Reformulation Complexity Approximation Scheme Questions Heuristic method for finding a sparse solution Regressor selection problem: A potential regressors, b to be fit by a linear combination of A min � Ax − b � 2 s . t . card ( x ) ≤ k x ∈ Z n + . � n It is NP-hard. Let m = 1, A = ( a 1 , a 2 , . . . , a n ) , b = 1 i = 1 a i , 2 k ≤ n 2 . It is a partition problem. Heuristic method. min � Ax − b � 2 + γ � x � 1 x ∈ R n . s . t . Ref. S. Boyd and L. Vandenberghe, Convex Optimization, thu-bell Cambridge University Press, 2004. W. Xing Sept. 2-4, 2014, Peking University
lp -Norm Constrained Quadratic Programming Linear Conic Programming Reformulation Complexity Approximation Scheme Questions Regularized approximation � Ax − b � 2 + γ � x � 1 min x ∈ R n . s . t . l 1 -norm and l 2 -norm constrained programming min t 1 + γ t 2 � Ax − b � 2 ≤ t 1 s . t . � x � 1 ≤ t 2 x ∈ R n , t 1 , t 2 ∈ R . The objective function is linear, the first constraint is a 2nd-order cone and the 2nd is a 1st-order cone. thu-bell It is a convex optimization problem of polynomially solvable. W. Xing Sept. 2-4, 2014, Peking University
lp -Norm Constrained Quadratic Programming Linear Conic Programming Reformulation Complexity Approximation Scheme Questions p -norm domain thu-bell Black: 1-norm. Red: 2-norm. Green: 3-norm. Yellow: 8-norm. W. Xing Sept. 2-4, 2014, Peking University
lp -Norm Constrained Quadratic Programming Linear Conic Programming Reformulation Complexity Approximation Scheme Questions Convex l p -norm problems p -norm domain is convex ( p ≥ 1). For set { x | � x � p ≤ 1 } , the smallest one is the domain with p = 1, which is the smallest convex set containing integer points {− 1 , 1 } n . For p ≥ 1, the l p -norm problems with linear objective or linear constraints are polynomially solvable. Variants of l p -norm problems should be considered. thu-bell W. Xing Sept. 2-4, 2014, Peking University
lp -Norm Constrained Quadratic Programming Linear Conic Programming Reformulation Complexity Approximation Scheme Questions Variants of l p -norm problems l 2 -norm constrained quadratic problem x T Qx + q T x min � Ax − b � 2 ≤ c T x s . t . c T x = d ≥ 0 x ∈ R n . l 1 -norm constrained quadratic problem x T Qx + q T x min s . t . � x � 1 ≤ k x ∈ R n , thu-bell where Q is a general symmetric matrix. W. Xing Sept. 2-4, 2014, Peking University
lp -Norm Constrained Quadratic Programming Linear Conic Programming Reformulation Complexity Approximation Scheme Questions l p -Norm Constrained Quadratic Programming 2 x T Qx + q T x 1 min 1 2 x T Q i x + q T i x + c i ≤ 0 , i = 1 , 2 , . . . , m s . t . � Ax − b � p ≤ c T x x ∈ R n , where p ≥ 1. thu-bell W. Xing Sept. 2-4, 2014, Peking University
lp -Norm Constrained Quadratic Programming Linear Conic Programming Reformulation Complexity Approximation Scheme Questions QCQP reformulation 2 x T Q 0 x + q T 1 min 0 x + c 0 1 2 x T Q i x + q T i x + c i ≤ 0 , i = 1 , 2 , . . . , m s . t . x ∈ D , where D ⊆ R n . thu-bell W. Xing Sept. 2-4, 2014, Peking University
lp -Norm Constrained Quadratic Programming Linear Conic Programming Reformulation Complexity Approximation Scheme Questions p -norm form l 1 -norm problem x T Qx + q T x min s . t . � x � 1 ≤ k x ∈ R n . Denote D = { x ∈ R n | � x � 1 ≤ k } . QCQP form x T Qx + q T x min s . t . x ∈ D . thu-bell W. Xing Sept. 2-4, 2014, Peking University
lp -Norm Constrained Quadratic Programming Linear Conic Programming Reformulation Complexity Approximation Scheme Questions 2-norm form 2-norm problem x T Qx + q T x min � Ax − b � 2 ≤ c T x s . t . c T x = d ≥ 0 x ∈ R n . x ∈ R n | � Ax − b � 2 ≤ c T x Denote D = � � QCQP form x T Qx + q T x min c T x = d s . t . x ∈ D . thu-bell W. Xing Sept. 2-4, 2014, Peking University
lp -Norm Constrained Quadratic Programming Linear Conic Programming Reformulation Complexity Approximation Scheme Questions Lifting reformulation f ( x ) = 1 2 x T Q 0 x + q T min 0 x + c 0 g i ( x ) = 1 2 x T Q i x + q T i x + c i ≤ 0 , i = 1 , 2 , . . . , m s . t . ( QCQP ) x ∈ D . Denote: F = { x ∈ D | g i ( x ) ≤ 0 , i = 1 , 2 , . . . , m } . Lifting � � q T 2 c 0 1 0 min • X 2 q 0 Q 0 � � q T 2 c i 1 i • X ≤ 0 , i = 1 , 2 , . . . , m s . t . 2 q i Q i � T � � � 1 1 thu-bell X = , x ∈ F . x x W. Xing Sept. 2-4, 2014, Peking University
lp -Norm Constrained Quadratic Programming Linear Conic Programming Reformulation Complexity Approximation Scheme Questions Convex reformulation � � q T 2 c 0 1 0 • X min 2 q 0 Q 0 � � q T 2 c i 1 i • X ≤ 0 , i = 1 , 2 , . . . , m s . t . 2 q i Q i � � 1 0 • X = 1 0 0 � T � � � 1 1 X ∈ cl ( conv ( | x ∈ F )) . x x thu-bell W. Xing Sept. 2-4, 2014, Peking University
lp -Norm Constrained Quadratic Programming Linear Conic Programming Reformulation Complexity Approximation Scheme Questions Linear conic programming reformulation � � q T 2 c 0 1 0 • X min 2 q 0 Q 0 � � q T 2 c i 1 s . t . i • X ≤ 0 , i = 1 , 2 , . . . , m 2 q i Q i � � 1 0 • X = 1 0 0 � T � � � 1 1 X ∈ cl ( cone ( | x ∈ F )) . x x It is a linear conic programming and has the same optimal thu-bell value with QCQP . W. Xing Sept. 2-4, 2014, Peking University
lp -Norm Constrained Quadratic Programming Linear Conic Programming Reformulation Complexity Approximation Scheme Questions Quadratic-Function Conic Programming PRIMAL � � q T 2 c 0 1 0 • V min 2 q 0 Q 0 � � q T 2 c i 1 i • V ≤ 0 , i = 1 , 2 , . . . , m s . t . ( QFCP ) 2 q i Q i � � 1 0 • V = 1 0 0 � T � � � 1 1 . V ∈ D ∗ , x ∈ F F = cl cone x x F ⊆ R n , A • B = trace ( AB T ) , thu-bell W. Xing Sept. 2-4, 2014, Peking University
lp -Norm Constrained Quadratic Programming Linear Conic Programming Reformulation Complexity Approximation Scheme Questions Quadratic-Function Conic Programming DUAL max σ � � − 2 σ + 2 c 0 + 2 � m ( q 0 + � m i = 1 λ i q i ) T i = 1 λ i c i ∈ D F s . t . q 0 + � m Q 0 + � m i = 1 λ i q i i = 1 λ i Q i σ ∈ R , λ ∈ R m + , F ⊆ R n , � T � � � 1 1 U ∈ S n + 1 | D F = U ≥ 0 , ∀ x ∈ F . x x thu-bell W. Xing Sept. 2-4, 2014, Peking University
lp -Norm Constrained Quadratic Programming Linear Conic Programming Reformulation Complexity Approximation Scheme Questions Properties of the Quadratic-Function Cone Cone of nonnegative quadratic functions (Sturm and Zhang, MOR 28, 2003). � T � � � 1 1 U ∈ S n + 1 | D F = U ≥ 0 , ∀ x ∈ F . x x If F � = ∅ , then D ∗ F is the dual cone of D F and vice versa. If F is a bounded nonempty set, then � T � � � 1 1 D ∗ F = cone , x ∈ F . x x If int ( F ) � = ∅ , then D ∗ F and D F are proper. thu-bell W. Xing Sept. 2-4, 2014, Peking University
lp -Norm Constrained Quadratic Programming Linear Conic Programming Reformulation Complexity Approximation Scheme Questions Properties The complexity of checking whether V ∈ D ∗ F or U ∈ D F depends on F . F = S n + 1 When F = R n , D ∗ . + When F = R n + , D ∗ F is the copositive cone! Ref: recent survey papers (I. M. Bomze, EJOR, 2012 216(3); Mirjam D¨ ur, Recent Advances in Optimization and its Applications in Engineering, 2010; J.-B. Hiriart-Urruty and A. Seeger, SIAM Review 52(4), 2010.) Relaxation or restriction D ∗ F ⊆ S n + ⊆ D F . Approximation: Computable cover of F . thu-bell W. Xing Sept. 2-4, 2014, Peking University
Recommend
More recommend
