The minimum Euclidean norm point in a polytope: Wolfe's method is exponential Luis Rademacher, UC Davis Simons Institute Join work with JesΓΊs de Loera, Jamie Haddock
Minimum norm point in a polytope β’ Given π 1 , β¦ , π π β π π , determine argmin π¦βπ π¦ for π = conv(π 1 , β¦ , π π ) . β’ A convex quadratic program.
Projection onto a simplex: not so easy β’ Only polynomial time algorithms I know are general purpose convex programming algorithms like the ellipsoid method. β’ No strongly polynomial time algorithm known.
Wolfeβs method (Wolfe β74, Lawson Hanson β74) β’ A combinatorial algorithm to find the minimum norm point in a polytope. β’ Not the same as Frank-Wolfe
Motivation β’ In machine learning: β’ Fujishige-Wolfe algorithm β’ one of the most practical algorithms for submodular function minimization . β’ Wolfeβs method is a subroutine in it β’ Optimal loading of recursive neural networks [CDKSV β95] β’ Non-negative least squares
This talk: β’ Complexity of Wolfeβs method β’ Relationship between Linear Programming (LP) and minimum norm point problem.
Related work β’ [Lawson, Hanson β74] An algorithm very similar to Wolfeβs. β’ [Fujishige, Hayashi, Isotani β06] Polynomial time reduction from LP to minimum norm point in a polytope. β’ [Chakrabarty, Jain, Kothari β14] [Lacoste-Julien, Jaggi β15] Rates of convergence of Wolfeβs method.
Our results β’ LP reduces in strongly polynomial time to βminimum norm point in a simplex β. β’ A (strongly) polynomial time algorithm for minimum norm point in a simplex would give a (strongly) polynomial time algorithm for general LPs. β’ Step 1: LP reduc es to βmembership in a V - polytopeβ. β’ Step 2: βMembership in a V - polytopeβ reduces to βdistance to a simplexβ.
Our results β’ Wolfeβs method takes exponential time in the worst case. β’ We construct explicit sets of points in every dimension. β’ Similar in spirit to Klee-Minty cubes for LP.
Idea of Wolfeβs method β’ Given points π β π π , maintain the following invariant : A subset π β π whose vertices determine a simplex, and the minimum norm point, π¦ , in the simplex. β’ Start with any point in π as π and π¦ . β’ Alternate between the following two steps: β’ Find point π β π such that π β ππππ€ π βͺ π contains a better point (entering rule). Add π to π . β’ Let π¦ β follow gravity β (towards 0) within ππππ€ π while dimension of current face decreases. Let π be the (vertices of the) current face of π¦ .
Inefficiency of Wolfeβs method with β minnorm β entering rule β’ A point, π , enters current set π , leaves, and then re-enters. β’ π < π < π < π < |π‘| S . a ap apq pq pqr qr qrs rs rsa
Exponential lower bound β’ Replace point π and 1 st coordinate by subspace and a set of points in it, constructed recursively. β’ Seq π : π π β 2 β β― β π π π‘ π β π π β 2 π π π‘ π
Open questions β’ Find exponential lower bound for other βentering rules.β ( linopt) β’ For submodular function minimization, polytopes are so- called βbase polytopes.β β’ Our exponential example is not of that kind. β’ Could Wolfeβs method be faster on base polytopes? β’ Smoothed analysis of Wolfeβs method.
Recommend
More recommend
