Convex Optimization 8. Geometric Problems Prof. Ying Cui - PowerPoint PPT Presentation

Convex Optimization 8. Geometric Problems Prof. Ying Cui Department of Electrical Engineering Shanghai Jiao Tong University 2018 SJTU Ying Cui 1 / 26

Outline Extremal volume ellipsoids Centering Classification Placement and facility location SJTU Ying Cui 2 / 26

Minimum volume ellipsoid covering a set owner-John ellipsoid of a set C ⊆ R n (denoted by E lj ): L¨ minimum volume ellipsoid that contains set C ◮ parametrize a general ellipsoid E as E = { v | � Av + b � 2 ≤ 1 } ( A ∈ S n , b ∈ R n ) ◮ inverse image of Euclidean unit ball under an affine mapping ◮ w.l.o.g. assume A ∈ S n ++ ◮ volume of E is proportional to det A − 1 ◮ find minimum volume ellipsoid containing C log det A − 1 min A , b s . t . sup v ∈ C � Av + b � 2 ≤ 1 ◮ convex problem (objective and constraint functions are both convex in A and b ) ◮ tractable only in certain special cases (evaluating constraint for general C can be hard) SJTU Ying Cui 3 / 26

Minimum volume ellipsoid covering a finite set finite set : ◮ find minimum volume ellipsoid containing finite set { x 1 , . . . , x m } log det A − 1 min A , b s . t . � Ax i + b � 2 ≤ 1 , i = 1 , . . . , m ◮ norm constraints can be replaced with squared versions which are convex quadratic inequalities ◮ equivalent to finding minimum volume ellipsoid containing polyhedron conv { x 1 , . . . , x m } ◮ an ellipsoid covers C iff it covers its convex hull SJTU Ying Cui 4 / 26

Maximum volume inscribed ellipsoid ◮ parametrize a general ellipsoid E as E = { Bu + d | � u � 2 ≤ 1 } ( B ∈ S n , d ∈ R n ) ◮ image of Euclidean unit ball under an affine mapping ◮ w.l.o.g. assume B ∈ S n ++ ◮ volume of E is proportional to det B ◮ find maximum volume ellipsoid inside a convex set C ⊆ R n max log det B B , d s . t . sup � u � 2 ≤ 1 I C ( Bu + d ) ≤ 0 where I C ( x ) = 0 for x ∈ C and I C ( x ) = ∞ for x / ∈ C ◮ convex problem (log det is concave and I C is convex, in B and d ) SJTU Ying Cui 5 / 26

Maximum volume ellipsoid in a polyhedron polyhedron : ◮ find maximum volume ellipsoid inside polyhedron C = { x | a T i x ≤ b i , i = 1 , . . . , m } max log det B B , d � Ba i � 2 + a T s . t . i d ≤ b i , i = 1 , . . . , m ◮ constraint follows from sup � u � 2 ≤ 1 a T i ( Bu + d ) = � Ba i � 2 + a T i d SJTU Ying Cui 6 / 26

Efficiency of ellipsoidal approximations C ⊆ R n convex, bounded, with nonempty interior ◮ L¨ owner-John ellipsoid of C , shrunk by a factor n , lies inside C ◮ maximum volume inscribed ellipsoid of C , expanded by a factor n , covers C ◮ factor n can be improved to √ n if C is symmetric example (for two polyhedra P = conv { x 1 , . . . , x 6 } in R 2 ) SJTU Ying Cui 7 / 26

Affine invariance of extremal volume ellipsoids L¨ owner-John ellipsoid ellipsoid and maximum volume inscribed ellipsoid are both affinely invariant owner-John ellipsoid of C and T ∈ R n × n is ◮ if E is L¨ nonsingular, then L¨ owner-John ellipsoid of TC is T E ◮ if E is maximum volume inscribed ellipsoid of C and T ∈ R n × n is nonsingular, then maximum volume inscribed ellipsoid of TC is T E SJTU Ying Cui 8 / 26

Chebyshev center Chebyshev center x cheb ( C ) of set C ⊆ R n (bounded, with nonempty interior) is ◮ any point of maximum depth in C ◮ any point inside C that is farthest from exterior of C ◮ the center of the largest ball that lies inside C SJTU Ying Cui 9 / 26

Chebyshev center of a convex set ◮ C is convex and defined by a set of convex inequalities: C = { x | f 1 ( x ) ≤ 0 , · · · , f m ( x ) ≤ 0 } ◮ find a Chebyshev center of convex set C : max R x , R s . t . g i ( x , R ) ≤ 0 , i = 1 , · · · , m where g i ( x , R ) = sup � u �≤ 1 f i ( x + Ru ) ◮ convex problem ( g i is the pointwise maximum of a family of convex functions of x and R , hence convex), x cheb ( C ) = x ∗ ◮ tractable only in certain special cases (evaluate g i involving solving a convex problem which can be hard) SJTU Ying Cui 10 / 26

Chebyshev center of a polyhedron ◮ C is a polyhedron defined by a set of linear inequalities: C = { x | a T i x ≤ b i , i = 1 , · · · , m } ◮ find a Chebyshev center of polyhedron C : max R x , R a T s . t . i x + R � a i � ∗ ≤ b i , i = 1 , · · · , m R ≥ 0 where each constraint follows from � � a T = a T g i ( x , R ) = sup i ( x + Ru ) − b i i x + R � a i � ∗ − b i � u �≤ 1 ◮ LP SJTU Ying Cui 11 / 26

Maximum volume ellipsoid center ◮ maximum volume ellipsoid center x mve ( C ) of set C is center of maximum volume ellipsoid in C ◮ find maximum volume ellipsoid center of set C max log det B B , d s . t . sup � u � 2 ≤ 1 I C ( Bu + d ) ≤ 0 ◮ convex problem, x mve ( C ) = d ∗ ◮ maximum volume ellipsoid center is affine invariant, as maximum volume inscribed ellipsoid is affine invariant SJTU Ying Cui 12 / 26

Analytic center of a set of inequalities analytic center x ac of a set of convex inequalities and linear equations f i ( x ) ≤ 0 , i = 1 , . . . , m , Fx = g is an optimal point of � m min − i =1 log( − f i ( x )) x s . t . Fx = g ◮ objective called logarithmic barrier associated with inequalities ◮ interpretation of f i ( x ) at a feasible point x : margin or slack in i -th inequality ◮ x ac is point that maximizes product (or geometric mean) of these margins or slacks subject to linear equality and implicit convex inequality constraints ◮ x ac is not a function of the set: two sets of inequalities can describe the same set, but have different analytic centers ◮ x ac is independent of affine changes of coordinates ◮ x ac is invariant under (positive) scalings of inequality functions, and any reparametrization of equality constraints SJTU Ying Cui 13 / 26

Analytic center of a set of linear inequalities analytic center of a set of linear inequalities a T i x ≤ b i , i = 1 , . . . , m is an optimal point of unconstrained minimization problem m � log( b i − a T min φ ( x ) = − i x ) x i =1 ◮ x ac is the point that maximizes product of distances to hyperplanes H i = { x | a T i x = b i } , assuming � a i � 2 = 1 w.l.o.g. ◮ x ac is independent of positive scaling of constraint functions SJTU Ying Cui 14 / 26

Inner and outer ellipsoids from analytic center of linear inequalities analytic center of a set of linear inequalities P implicitly defines an inscribed and a covering ellipsoid, i.e., E inner ⊆ P ⊆ E outer , where P = { x | a T i x ≤ b i , i = 1 , . . . , m } E inner = { x | ( x − x ac ) T ∇ 2 φ ( x ac )( x − x ac ) ≤ 1 } E outer = { x | ( x − x ac ) T ∇ 2 φ ( x ac )( x − x ac ) ≤ m ( m − 1) } ◮ E inner and E outer are related by scale factor ( m ( m − 1)) 1 / 2 , which is always at least n SJTU Ying Cui 15 / 26

Classification given two sets of points in R n , { x 1 , · · · , x N } and { y 1 , · · · , y N } , and find a function f : R n → R (within a given family of functions) such that f ( x i ) > 0 , i = 1 , . . . , N , f ( y i ) < 0 , i = 1 , . . . , M 0-level set of f { z | f ( z ) = 0 } separates, classifies, or discriminates two sets of points ◮ linear discrimination: f is linear (or affine) ◮ nonlinear discrimination: otherwise SJTU Ying Cui 16 / 26

Linear discrimination ◮ find an affine function f ( z ) = a T z − b (i.e., a hyperplane) a T x i + b > 0 , i = 1 , . . . , N , a T y i + b < 0 , i = 1 , . . . , M ◮ strict inequalities homogeneous in a and b , hence equivalent to a T x i + b ≥ 1 , i = 1 , . . . , N , a T y i + b ≤ − 1 , i = 1 , . . . , M ◮ problem: find a , b a T x i − b ≥ 1 , s . t . i = 1 , · · · , N a T y i − b ≤ − 1 , i = 1 , · · · , M SJTU Ying Cui 17 / 26

Robust linear discrimination ◮ existence of an affine classifying function f ( z ) = a T z − b is equivalent to a set of linear inequalities in a and b that define f and choose one that optimizes some measure of robustness ◮ seek function giving maximum possible gap between (positive) values at points x i and (negative) values at points y i ◮ normalize a and b , since otherwise scaling a and b by a positive constant can make gap arbitrarily large SJTU Ying Cui 18 / 26

Robust linear discrimination ◮ primal problem: max t a , b , t a T x i − b ≥ t , s . t . i = 1 , · · · , N a T y i − b ≤ − t , i = 1 , · · · , M � a � 2 ≤ 1 ◮ convex problem ◮ t ∗ is positive iff two sets of points can be linearly discriminated and in this case � a ∗ � 2 = 1 ◮ dual problem: � N � M max − � i =1 u i x i − i =1 v i y i � 2 u , v 1 T u = 1 / 2 s . t . u � 0 , 1 T v = 1 / 2 v � 0 , ◮ find (half) distance between convex hulls of two sets SJTU Ying Cui 19 / 26

Approximate linear separation of non-separable sets ◮ when two sets of points cannot be linearly separated, seek an affine function that approximately classifies them ◮ a heuristic for minimizing number of misclassified points 1 T u + 1 T v min a , b , u , v a T x i + b ≥ 1 − u i , s . t . i = 1 , . . . , N a T y i + b ≤ − 1 + v i , i = 1 , . . . , M u � 0 , v � 0 ◮ LP ◮ u i ( v i ): a measure of how much a T x i + b ≥ 1 ( a T y i + b ≤ − 1) is violated ◮ objective: a convex relaxation of number of points x i violating a T x i − b ≥ 1 plus number of points y i violating a T y i − b ≤ − 1 ◮ u ∗ i = max { 0 , 1 − a ∗ T x i − b ∗ } , v ∗ i = max { 0 , 1 + a ∗ T y ∗ i + b } SJTU Ying Cui 20 / 26