8. Geometric problems extremal volume ellipsoids centering - PowerPoint PPT Presentation

Convex Optimization — Boyd & Vandenberghe 8. Geometric problems • extremal volume ellipsoids • centering • classification • placement and facility location 8–1

Minimum volume ellipsoid around a set L¨ owner-John ellipsoid of a set C : minimum volume ellipsoid E s.t. C ⊆ E • parametrize E as E = { v | � Av + b � 2 ≤ 1 } ; w.l.o.g. assume A ∈ S n ++ • vol E is proportional to det A − 1 ; to compute minimum volume ellipsoid, log det A − 1 minimize (over A , b ) subject to sup v ∈ C � Av + b � 2 ≤ 1 convex, but evaluating the constraint can be hard (for general C ) finite set C = { x 1 , . . . , x m } : log det A − 1 minimize (over A , b ) subject to � Ax i + b � 2 ≤ 1 , i = 1 , . . . , m also gives L¨ owner-John ellipsoid for polyhedron conv { x 1 , . . . , x m } Geometric problems 8–2

Maximum volume inscribed ellipsoid maximum volume ellipsoid E inside a convex set C ⊆ R n • parametrize E as E = { Bu + d | � u � 2 ≤ 1 } ; w.l.o.g. assume B ∈ S n ++ • vol E is proportional to det B ; can compute E by solving maximize log det B subject to sup � u � 2 ≤ 1 I C ( Bu + d ) ≤ 0 (where I C ( x ) = 0 for x ∈ C and I C ( x ) = ∞ for x �∈ C ) convex, but evaluating the constraint can be hard (for general C ) polyhedron { x | a T i x ≤ b i , i = 1 , . . . , m } : maximize log det B � Ba i � 2 + a T subject to i d ≤ b i , i = 1 , . . . , m (constraint follows from sup � u � 2 ≤ 1 a T i ( Bu + d ) = � Ba i � 2 + a T i d ) Geometric problems 8–3

Efficiency of ellipsoidal approximations C ⊆ R n convex, bounded, with nonempty interior • L¨ owner-John ellipsoid, shrunk by a factor n , lies inside C • maximum volume inscribed ellipsoid, expanded by a factor n , covers C example (for two polyhedra in R 2 ) factor n can be improved to √ n if C is symmetric Geometric problems 8–4

Centering some possible definitions of ‘center’ of a convex set C : • center of largest inscribed ball (’Chebyshev center’) for polyhedron, can be computed via linear programming (page 4–19) • center of maximum volume inscribed ellipsoid (page 8–3) x cheb x cheb x mve MVE center is invariant under affine coordinate transformations Geometric problems 8–5

Analytic center of a set of inequalities the analytic center of set of convex inequalities and linear equations f i ( x ) ≤ 0 , i = 1 , . . . , m, Fx = g is defined as the optimal point of − � m minimize i =1 log( − f i ( x )) subject to Fx = g • more easily computed than MVE or Chebyshev center (see later) • not just a property of the feasible set: two sets of inequalities can describe the same set, but have different analytic centers Geometric problems 8–6

analytic center of linear inequalities a T i x ≤ b i , i = 1 , . . . , m x ac is minimizer of x ac m � log( b i − a T φ ( x ) = − i x ) i =1 inner and outer ellipsoids from analytic center: E inner ⊆ { x | a T i x ≤ b i , i = 1 , . . . , m } ⊆ E outer where { x | ( x − x ac ) T ∇ 2 φ ( x ac )( x − x ac ) ≤ 1 } E inner = { x | ( x − x ac ) T ∇ 2 φ ( x ac )( x − x ac ) ≤ m ( m − 1) } E outer = Geometric problems 8–7

Linear discrimination separate two sets of points { x 1 , . . . , x N } , { y 1 , . . . , y M } by a hyperplane: a T x i + b > 0 , a T y i + b < 0 , i = 1 , . . . , N, i = 1 , . . . , M homogeneous in a , b , hence equivalent to a T x i + b ≥ 1 , a T y i + b ≤ − 1 , i = 1 , . . . , N, i = 1 , . . . , M a set of linear inequalities in a , b Geometric problems 8–8

Robust linear discrimination (Euclidean) distance between hyperplanes { z | a T z + b = 1 } H 1 = { z | a T z + b = − 1 } H 2 = is dist ( H 1 , H 2 ) = 2 / � a � 2 to separate two sets of points by maximum margin, minimize (1 / 2) � a � 2 a T x i + b ≥ 1 , subject to i = 1 , . . . , N (1) a T y i + b ≤ − 1 , i = 1 , . . . , M (after squaring objective) a QP in a , b Geometric problems 8–9

Lagrange dual of maximum margin separation problem (1) 1 T λ + 1 T µ maximize � � �� N i =1 λ i x i − � M subject to 2 i =1 µ i y i 2 ≤ 1 � � (2) � 1 T λ = 1 T µ, λ � 0 , µ � 0 from duality, optimal value is inverse of maximum margin of separation interpretation • change variables to θ i = λ i / 1 T λ , γ i = µ i / 1 T µ , t = 1 / ( 1 T λ + 1 T µ ) • invert objective to minimize 1 / ( 1 T λ + 1 T µ ) = t minimize t � � � N i =1 θ i x i − � M subject to i =1 γ i y i 2 ≤ t � � � � 1 T θ = 1 , 1 T γ = 1 θ � 0 , γ � 0 , optimal value is distance between convex hulls Geometric problems 8–10

Approximate linear separation of non-separable sets 1 T u + 1 T v minimize a T x i + b ≥ 1 − u i , subject to i = 1 , . . . , N a T y i + b ≤ − 1 + v i , i = 1 , . . . , M u � 0 , v � 0 • an LP in a , b , u , v • at optimum, u i = max { 0 , 1 − a T x i − b } , v i = max { 0 , 1 + a T y i + b } • can be interpreted as a heuristic for minimizing #misclassified points Geometric problems 8–11

Support vector classifier � a � 2 + γ ( 1 T u + 1 T v ) minimize a T x i + b ≥ 1 − u i , subject to i = 1 , . . . , N a T y i + b ≤ − 1 + v i , i = 1 , . . . , M u � 0 , v � 0 produces point on trade-off curve between inverse of margin 2 / � a � 2 and classification error, measured by total slack 1 T u + 1 T v same example as previous page, with γ = 0 . 1 : Geometric problems 8–12

Nonlinear discrimination separate two sets of points by a nonlinear function: f ( x i ) > 0 , i = 1 , . . . , N, f ( y i ) < 0 , i = 1 , . . . , M • choose a linearly parametrized family of functions f ( z ) = θ T F ( z ) F = ( F 1 , . . . , F k ) : R n → R k are basis functions • solve a set of linear inequalities in θ : θ T F ( x i ) ≥ 1 , θ T F ( y i ) ≤ − 1 , i = 1 , . . . , N, i = 1 , . . . , M Geometric problems 8–13

quadratic discrimination : f ( z ) = z T Pz + q T z + r x T i Px i + q T x i + r ≥ 1 , y T i Py i + q T y i + r ≤ − 1 can add additional constraints ( e.g. , P � − I to separate by an ellipsoid) polynomial discrimination : F ( z ) are all monomials up to a given degree separation by ellipsoid separation by 4th degree polynomial Geometric problems 8–14

Placement and facility location • N points with coordinates x i ∈ R 2 (or R 3 ) • some positions x i are given; the other x i ’s are variables • for each pair of points, a cost function f ij ( x i , x j ) placement problem � minimize i � = j f ij ( x i , x j ) variables are positions of free points interpretations • points represent plants or warehouses; f ij is transportation cost between facilities i and j • points represent cells on an IC; f ij represents wirelength Geometric problems 8–15

example: minimize � ( i,j ) ∈A h ( � x i − x j � 2 ) , with 6 free points, 27 links optimal placement for h ( z ) = z , h ( z ) = z 2 , h ( z ) = z 4 1 1 1 0 0 0 − 1 − 1 − 1 − 1 0 1 − 1 0 1 − 1 0 1 histograms of connection lengths � x i − x j � 2 6 4 4 5 3 3 4 3 2 2 2 1 1 1 0 0 0 0 0 . 5 1 1 . 5 2 0 0 . 5 1 1 . 5 0 0 . 5 1 1 . 5 Geometric problems 8–16