SLIDE 1
Ændr 2. linje i overskriften til AU Passata Light AARHUS UNIVERSITY DENMARK
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
Optimal Minimal Margin Maximization with Boosting ndr 2. linje i - - PowerPoint PPT Presentation
Optimal Minimal Margin Maximization with Boosting ndr 2. linje i - - PowerPoint PPT Presentation
Optimal Minimal Margin Maximization with Boosting ndr 2. linje i overskriften til AU Passata Light Allan Grnlund Kasper Green Larsen Alexander Mathiasen (me) AU AARHUS UNIVERSITY DENMARK What is boosting? Overskrift n linje Bold
SLIDE 2
SLIDE 3
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
What is boosting?
SLIDE 4
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
What is boosting?
SLIDE 5
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
What is boosting?
SLIDE 6
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
What is boosting?
SLIDE 7
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
What is boosting?
SLIDE 8
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
What is boosting?
SLIDE 9
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
What is boosting?
SLIDE 10
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
What is boosting?
SLIDE 11
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
What is boosting?
SLIDE 12
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
What is boosting?
SLIDE 13
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
What is boosting?
SLIDE 14
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
What is boosting?
SLIDE 15
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
No Overfitting?
SLIDE 16
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
No Overfitting?
Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998
SLIDE 17
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
No Overfitting?
# Hypotheses
Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998
SLIDE 18
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
No Overfitting?
Classification Error # Hypotheses
Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998
SLIDE 19
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
No Overfitting?
Training Error Classification Error # Hypotheses
Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998
SLIDE 20
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
No Overfitting?
Training Error Test Error Classification Error # Hypotheses
Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998
SLIDE 21
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
No Overfitting?
Training Error Test Error Perfectly classify training data Classification Error # Hypotheses
Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998
SLIDE 22
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
No Overfitting?
Training Error Test Error Perfectly classify training data Test error still improves! Classification Error # Hypotheses
Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998
SLIDE 23
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
No Overfitting?
Training Error Test Error Perfectly classify training data Test error still improves! How do we explain this? Classification Error # Hypotheses
Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998
SLIDE 24
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
An explanation by the minimal margin
SLIDE 25
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
An explanation by the minimal margin
SLIDE 26
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
An explanation by the minimal margin
SLIDE 27
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
An explanation by the minimal margin
* Not technically correct, see paper for definition.
SLIDE 28
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
An explanation by the minimal margin
Breiman 1998
SLIDE 29
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
An explanation by the minimal margin
Generalization error
Breiman 1998
SLIDE 30
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
An explanation by the minimal margin
Generalization error Minimal margin
Breiman 1998
SLIDE 31
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
An explanation by the minimal margin
Generalization error Minimal margin Number of hypotheses
Breiman 1998
SLIDE 32
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
An explanation by the minimal margin
Generalization error Minimal margin Number of points Number of hypotheses
Breiman 1998
SLIDE 33
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
An explanation by the minimal margin
Generalization error Minimal margin Number of points Number of hypotheses
Breiman 1998
AdaBoostV: Finds a classifier with a provable bound on the trade-off between the number of hypotheses and the minimal margin. [Rätsch and Warmuth 2005]
SLIDE 34
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
An explanation by the minimal margin
Generalization error Minimal margin Number of points Number of hypotheses
Breiman 1998
AdaBoostV: Finds a classifier with a provable bound on the trade-off between the number of hypotheses and the minimal margin. [Rätsch and Warmuth 2005] JMLR: Conjecture: there is a lower bound which matches AdaBoostV. [Nie et al. 2013]
SLIDE 35
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
An explanation by the minimal margin
Generalization error Minimal margin Number of points Number of hypotheses
Breiman 1998
AdaBoostV: Finds a classifier with a provable bound on the trade-off between the number of hypotheses and the minimal margin. [Rätsch and Warmuth 2005] JMLR: Conjecture: there is a lower bound which matches AdaBoostV. [Nie et al. 2013] FALSE.
SLIDE 36
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
An explanation by the minimal margin
Generalization error Minimal margin Number of points Number of hypotheses
Breiman 1998
AdaBoostV: Finds a classifier with a provable bound on the trade-off between the number of hypotheses and the minimal margin. [Rätsch and Warmuth 2005] JMLR: Conjecture: there is a lower bound which matches AdaBoostV. [Nie et al. 2013] FALSE. SparsiBoost: Obtains a slightly better bound than AdaBoostV.
SLIDE 37
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
An explanation by the minimal margin
AdaBoostV: Finds a classifier with a provable bound on the trade-off between the number of hypotheses and the minimal margin. [Rätsch and Warmuth 2005] JMLR: Conjecture: there is a lower bound which matches AdaBoostV. [Nie et al. 2013] FALSE. SparsiBoost: Obtains a slightly better bound than AdaBoostV. Proved matching lower bound, so SparsiBoost is optimal.
Generalization error Minimal margin Number of points Number of hypotheses
Breiman 1998
SLIDE 38
AARHUS UNIVERSITY DENMARK
AU
Overskrift én linje Bold eller Regular
Experiments
SLIDE 39