Toy Example Toy Example Toy Example Toy Example Toy Example D 1 - - PowerPoint PPT Presentation

Toy Example Toy Example Toy Example Toy Example Toy Example

D1

weak classifiers = vertical or horizontal half-planes

Round 1 Round 1 Round 1 Round 1 Round 1

• h1

α ε1 1 =0.30 =0.42 2 D

Round 2 Round 2 Round 2 Round 2 Round 2

• α

ε2 2 =0.21 =0.65 h2 3 D

Round 3 Round 3 Round 3 Round 3 Round 3

• h3

α ε3 3=0.92 =0.14

Final Classifier Final Classifier Final Classifier Final Classifier Final Classifier

• H

final + 0.92 + 0.65 0.42 sign = =

Actual Typical Run Actual Typical Run Actual Typical Run Actual Typical Run Actual Typical Run

10 100 1000 5 10 15 20

# of rounds (T C4.5 test error ) train test error

(boosting C4.5 on “letter” dataset)

not not not not increase, even after 1000 rounds

# rounds

train error

test error

wrongly wrongly wrongly wrongly predicts “simpler” rule is better

Empirical Evidence: The Margin Distribution Empirical Evidence: The Margin Distribution Empirical Evidence: The Margin Distribution Empirical Evidence: The Margin Distribution Empirical Evidence: The Margin Distribution

margin distribution margin distribution margin distribution margin distribution = cumulative distribution of margins of training examples

10 100 1000 5 10 15 20

error test train ) T # of rounds (

• 1
• 0.5

0.5 1 0.5 1.0

cumulative distribution 1000 100 margin 5

# rounds

train error

test error

% margins

• 0:5

minimum margin