CSE 571 A simple algorithm for learning robust Probabilistic - - PowerPoint PPT Presentation

11/13/2007 1

SA-1

CSE 571 Probabilistic Robotics

Boosting

Some slides taken from: Wolfram Burgard, Hongbo Deng, Antonio Torralba

• A simple algorithm for learning robust

classifiers

• Freund & Shapire, 1995
• Friedman, Hastie, Tibshhirani, 1998
• Provides efficient algorithm for sparse

feature selection

• Tieu & Viola, 2000
• Viola & Jones, 2003
• Easy to implement, doesn’t requires

external optimization tools.

Why boosting?

SA-1

Motivation

• Indoor mapping is an important task in mobile robotics.

SA-1

Motivation

• Semantic mapping:
• Human-Robot interaction:

User: “Go to the corridor” Room Room Corridor Corridor Doorway Doorway

11/13/2007 2

SA-1

Goal

• Classification of the position of the robot using
• ne single observation: a 360o laser range data.

SA-1

Observations

Room Room

SA-1

Observations

Room Room Doorway Doorway

SA-1

Observations

Room Room Corridor Corridor Doorway Doorway

11/13/2007 3

SA-1

Simple Features

• Gap = d > θ
• f = # Gaps

Minimum

• f =Area
• f =Perimeter
• f = d

d di

N 1 f

d

• f = d
• d

Σ di

SA-1

Combining Features

• Observation:

There are many simple features fi.

• Problem:

Each single feature fi gives poor classification rates.

• Solution:

Combine multiple simple features to form a final classifier using AdaBoost.

Idea

12

Example

11/13/2007 4

13

Example

14

Example

15

Example

16

Example

11/13/2007 5

17

Example

18

Example

AdaBoost Algorithm Sequential Optimization

• Original motivation in learning theory
• Can be derived as sequential optimization
• f exponential error function

with

• Fixing first m-1 classifiers and minimizing

wrt m-th classifier gives AdaBoost equations!

1

exp ( )

N n m n n

E t f x

1

1 ( ) ( ) 2

m m l l l

f x y x

11/13/2007 6

Exponential loss

• 1.5
• 1
• 0.5

0.5 1 1.5 2 0.5 1 1.5 2 2.5 3 3.5 4

Squared error Exponential loss

Misclassification error

Loss

Squared error Exponential loss

Further Observations

• Test error often decreases even when

training error is already zero

• Many other boosting algorithms possible

by changing error function

• Regression
• Multiclass

SA-1

Multiple Classes

• Sequential AdaBoost for K different classes.

Binary Classifier 1 Binary Classifier K-1 . . . Class K Class 1 Class K-1 H(x)=0 H(x)=0 H(x)=1 H(x)=1

SA-1

Summary of the Approach (1)

• Training: learn a strong classifier using

previously labeled observations:

AdaBoost Multi-class Classifier Room Room Corridor Corridor Doorway Doorway

features features features features features features

11/13/2007 7

SA-1

Summary of the Approach (2)

• Test: using the classifier to classify new
• bservations:

Multi-class Classifier

New observation

features Corridor Corridor

SA-1

Experiments

Training (top) # examples: 16045 Test (bottom) # examples: 18726 classification:

93.94%

Building 079

• Uni. Freiburg

Room Room Corridor Corridor Doorway Doorway

SA-1

Experiments

Training (left) # examples: 13906 Test (right) # examples: 10445 classification:

89.52%

Building 101

• Uni. Freiburg

Room Room Corridor Corridor Doorway Doorway Hallway Hallway

SA-1

Application to a New Environment

Training map

Intel Research Lab in Seattle

11/13/2007 8

SA-1

Application to a New Environment

Training map

Intel Research Lab in Seattle

Room Room Corridor Corridor Doorway Doorway

SA-1

Sequential AdaBoost vs AdaBoost.M2

Corridor Corridor Room Room Doorway Doorway

Sequential AdaBoost classification: 92.10 % AdaBoost.M2 classification: 91.83 %

Boosting

• Extremely flexible framework
• Handles high-dimensional continuous data
• Easy to implement
• Limitation:
• Only models local classification problems