Greedy Algorithms, Frank-Wolfe and Friends a modern perspective - - PowerPoint PPT Presentation
Greedy Algorithms, Frank-Wolfe and Friends a modern perspective NIPS 2013 Workshop sites.google.com/site/nips13greedyfrankwolfe Zaid Harchaoui, Martin Jaggi, Federico Pierucci, INRIA Simons Institute, LJK lab, Grenoble (France) Berkeley
NIPS 2013 Workshop
sites.google.com/site/nips13greedyfrankwolfe
Zaid Harchaoui, INRIA Grenoble (France) Martin Jaggi, Simons Institute, Berkeley (USA) Federico Pierucci, LJK lab, Grenoble (France)
convex (or affine) hull
[ Chandrasekaran et al. 2012 ]
x∈D f(x)
Examples of Atomic Domains Suitable for Frank-Wolfe
Optimization Domain Atoms A D = conv(A) Rn Sparse vectors k.k1-ball Rn Sign-vectors k.k1-ball Rn `p-Sphere k.kp-ball Rn Sparse non-neg. vectors Simplex ∆n Rn Latent group sparse vect. k.kG-ball Rm⇥n Matrix trace norm k.ktr-ball Rm⇥n Matrix operator norm k.kop-ball Rm⇥n Schatten matrix norms k(i(.))kp-ball Rm⇥n Matrix max-norm k.kmax-ball Rn⇥n Permutation matrices Birkhoff polytope Rn⇥n Rotation matrices Sn⇥n
Rank-1 PSD matrices
{x ⌫ 0, Tr(x) = 1} Sn⇥n
PSD matrices
{x ⌫ 0, xii 1}
and many more...
(away steps and variants)
(for submodular minimization)
Time Speaker Title
7:30 - 7:40am Organizers Introduction 7:40 - 8:20am Robert M. Freund Remarks on Frank-Wolfe and Structural Friends 8:20 - 9:00am Ben Recht The Algorithmic Frontiers of Atomic Norm Minimization: Relaxation, Discretization, and Greedy Pursuit
9:00 - 9:30pm Coffee Break
9:30 - 9:45am Nikhil Rao, Parikshit Shah and Stephen Wright Conditional Gradient with Enhancement and Truncation for Atomic Norm Regularization (canceled) Hector Allende, Emanuele Frandi, Ricardo Nanculef, Claudio Sartori Pairwise Away Steps for the Frank-Wolfe Algorithm 9:55 - 10:05am Simon Lacoste-Julien and Martin Jaggi An Affine Invariant Linear Convergence Analysis for Frank-Wolfe Algorithms 10:05 - 10:15am Vamsi Potluru, Jonathan Le Roux, Barak Pearlmutter, John Hershey and Matthew Brand Pairwise Coordinate Descent for Sparse NMF 10:15 - 10:30am Robert M. Freund and Paul Grigas New Analysis and Results for the Conditional Gradient Method
10:30 - 3:30pm Lunch Break
3:30 - 3:45pm Marguerite Frank Honorary Guest 3:45 - 4:25pm Shai Shalev-Schwartz Efficiently Training Sum-Product Neural Networks using Forward Greedy Selection 4:25 - 4:40pm Xiaocheng Tang and Katya Scheinberg Complexity of Inexact Proximal Newton methods 4:40 - 4:50pm Jacob Steinhardt and Jonathan Huggins A Greedy Framework for First-Order Optimization 4:50 - 5:00pm Ahmed Farahat, Ali Ghodsi and Mohamed Kamel A Fast Greedy Algorithm for Generalized Column Subset Selection
5:00 - 5:30pm! Coffee Break
5:30 - 6:10pm Francis Bach Duality between Subgradient and Conditional Gradient Methods 6:10 - 6:25pm David Belanger, Dan Sheldon and Andrew McCallum Marginal Inference in MRFs using Frank-Wolfe
A N ALGORITHM FOR QUADRATIC PROGRAMMING
M a r g u e r i t e
F r a n k a n d P h i l i p
W
f e l
Pr in
ce t o n Un i v e r s i t
y
A finite iteration method for calculating the solution of quadratic E x t e n s i
s t
e g e n e r a l n
r
r a m m i n g p r
l e m s is d e s c r i b e d .
l i n e a r D r
l e m s a r e
suggested.
1 . INTRODUCTION
The problem of maximizing a concave quadratic function whose variables are subject linear inequality constraints has been the subject of several recent s t u d i e s , from both the side and the theoretical
( s e e
Bibliography). Our aim here has been to develop a solving this non-linear programming problem which should be particularly well high-speed machine computation. quadratic programming problem
a s
such, called PI, is set forth in Section aid of generalized Lagrange multipliers the'solutions the solutions of a new quadratic programming maximum sought in PI1 from the fact that the boundedness (linear) constraints of computation
1 9 5 6 today 3:30pm
[ Frank & Wolfe 1956 ]