Optimal Minimal Margin Maximization with Boosting ndr 2. linje i - PowerPoint PPT Presentation

Optimal Minimal Margin Maximization with Boosting Ændr 2. linje i overskriften til AU Passata Light Allan Grønlund Kasper Green Larsen Alexander Mathiasen (me) AU AARHUS UNIVERSITY DENMARK

What is boosting? Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

No Overfitting? Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998 No Overfitting? Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998 No Overfitting? Overskrift én linje Bold eller Regular # Hypotheses AU AARHUS UNIVERSITY DENMARK

Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998 No Overfitting? Overskrift én linje Bold eller Regular Classification Error # Hypotheses AU AARHUS UNIVERSITY DENMARK

Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998 No Overfitting? Overskrift én linje Bold eller Regular Classification Error Training Error # Hypotheses AU AARHUS UNIVERSITY DENMARK

Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998 No Overfitting? Overskrift én linje Bold eller Regular Classification Error Test Error Training Error # Hypotheses AU AARHUS UNIVERSITY DENMARK

Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998 No Overfitting? Overskrift én linje Bold eller Regular Classification Error Test Error Training Error # Hypotheses Perfectly classify training data AU AARHUS UNIVERSITY DENMARK

Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998 No Overfitting? Overskrift én linje Bold eller Regular Classification Error Test error still improves! Test Error Training Error # Hypotheses Perfectly classify training data AU AARHUS UNIVERSITY DENMARK

Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998 No Overfitting? Overskrift én linje Bold eller Regular How do we explain this? Classification Error Test error still improves! Test Error Training Error # Hypotheses Perfectly classify training data AU AARHUS UNIVERSITY DENMARK

An explanation by the minimal margin Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

An explanation by the minimal margin Overskrift én linje Bold eller Regular * Not technically correct, see paper for definition. AU AARHUS UNIVERSITY DENMARK

Breiman 1998 An explanation by the minimal margin Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

Breiman 1998 An explanation by the minimal margin Overskrift én linje Bold eller Regular Generalization error AU AARHUS UNIVERSITY DENMARK

Breiman 1998 An explanation by the minimal margin Overskrift én linje Bold eller Regular Generalization error Minimal margin AU AARHUS UNIVERSITY DENMARK

Breiman 1998 An explanation by the minimal margin Overskrift én linje Bold eller Regular Number of hypotheses Generalization error Minimal margin AU AARHUS UNIVERSITY DENMARK

Breiman 1998 An explanation by the minimal margin Overskrift én linje Bold eller Regular Number of points Number of hypotheses Generalization error Minimal margin AU AARHUS UNIVERSITY DENMARK

Breiman 1998 An explanation by the minimal margin Overskrift én linje Bold eller Regular Number of points Number of hypotheses Generalization error Minimal margin AdaBoostV: Finds a classifier with a provable bound on the trade-off between the number of hypotheses and the minimal margin. [Rätsch and Warmuth 2005] AU AARHUS UNIVERSITY DENMARK

Breiman 1998 An explanation by the minimal margin Overskrift én linje Bold eller Regular Number of points Number of hypotheses Generalization error Minimal margin AdaBoostV: Finds a classifier with a provable bound on the trade-off between the number of hypotheses and the minimal margin. [Rätsch and Warmuth 2005] JMLR: Conjecture: there is a lower bound which matches AdaBoostV. [Nie et al. 2013] AU AARHUS UNIVERSITY DENMARK

Breiman 1998 An explanation by the minimal margin Overskrift én linje Bold eller Regular Number of points Number of hypotheses Generalization error Minimal margin AdaBoostV: Finds a classifier with a provable bound on the trade-off between the number of hypotheses and the minimal margin. [Rätsch and Warmuth 2005] JMLR: Conjecture: there is a lower bound which matches AdaBoostV. [Nie et al. 2013] FALSE. AU AARHUS UNIVERSITY DENMARK

Breiman 1998 An explanation by the minimal margin Overskrift én linje Bold eller Regular Number of points Number of hypotheses Generalization error Minimal margin AdaBoostV: Finds a classifier with a provable bound on the trade-off between the number of hypotheses and the minimal margin. [Rätsch and Warmuth 2005] JMLR: Conjecture: there is a lower bound which matches AdaBoostV. [Nie et al. 2013] FALSE. SparsiBoost: Obtains a slightly better bound than AdaBoostV. AU AARHUS UNIVERSITY DENMARK

Breiman 1998 An explanation by the minimal margin Overskrift én linje Bold eller Regular Number of points Number of hypotheses Generalization error Minimal margin AdaBoostV: Finds a classifier with a provable bound on the trade-off between the number of hypotheses and the minimal margin. [Rätsch and Warmuth 2005] JMLR: Conjecture: there is a lower bound which matches AdaBoostV. [Nie et al. 2013] FALSE. SparsiBoost: Obtains a slightly better bound than AdaBoostV. Proved matching lower bound, so SparsiBoost is optimal. AU AARHUS UNIVERSITY DENMARK

Experiments Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

AU AARHUS UNIVERSITY DENMARK

Optimal Minimal Margin Maximization with Boosting ndr 2. linje i - PowerPoint PPT Presentation

Optimal Minimal Margin Maximization with Boosting ndr 2. linje i overskriften til AU Passata Light Allan Grnlund Kasper Green Larsen Alexander Mathiasen (me) AU AARHUS UNIVERSITY DENMARK What is boosting? Overskrift n linje Bold

lti Is convex Perceptron Boosting Max-Margin Conditional Likelihood MIRA Based on Uses a

Minimal Sufficient Conditions for a Primal Optimizer in Nonsmooth Utility Maximization Harry

Optimal and Adaptive Algorithms for Online Boosting Alina Beygelzimer 1 Satyen Kale 1 Haipeng Luo 2

Optimal Continuous DR-Submodular Maximization and Applications to Provable Mean Field Inference

Non-monotone Submodular Maximization with Nearly Optimal Adaptivity and Query Complexity Matthew

Boosting, Min-Norm Interpolated Classifiers, and Overparametrization: a precise asymptotic theory

Optimal Left and Right Additive Schwarz Preconditioning for Minimal Residual Methods with

Optimal ranking of online search requests for long-term revenue maximization Pierre LEcuyer

Minimal Dataset to obtain optimal registration data from Pathologists Dr. Mia Slabbaert BWP

Learning near-optimal hyperparameters with minimal overhead Gellrt Weisz Andrs Gyrgy

Optimal Composition of Characteristic Modes For Minimal Quality Factor Q Miloslav Capek Luk

Geometric and numerical methods in optimal control for the time minimal saturation in Magnetic

Regularization Paths Boosting fits a regularization path toward a max-margin classifier.

Round Optimal Secure Multiparty Computation from Minimal Assumptions Arka Rai Choudhuri Michele

Optimal Supply Networks Introduction Optimal branching Complex Networks, Course 295A, Spring,

Expectation Maximization CMSC 691 UMBC Outline EM (Expectation Maximization) Basic idea Three

Life Between Systems Martin Brynskov AU Smart Cities @brynskov pit.au.dk AARHUS GOTO 2013

Receding Horizon Control Mar a M. Seron September 2004 Centre for Complex Dynamic Systems

Section 18.3 Learning Decision Trees CS4811 - Artificial Intelligence Nilufer Onder Department

Basic SQL Queries 1 Why SQL? SQL is a very-high-level language Say what to do

Measurements of Non-photonic Electrons Production and Elliptic Flow in s NN = 39, 62 and 200

Dynamical evolution of near-Sun objects V.V.EMEL'YANENKO INSTITUTE OF ASTRONOMY RAS Disruption

THE RHODES SCHOLARSHIPS AUSTRALIA Information seminar for Queensland applicants applying in

We have rights, you know An Inclusive Research Approach to Exploring the Views of People