- " " AGGREGATION " " BOOTSTRAP BAGGING - - PowerPoint PPT Presentation

SLIDE 1

ENSEMBLE

METHODS

• BAGGING

" "

BOOTSTRAP

AGGREGATION

" "

• RANDOM

FOREST

• BOOSTING

SLIDE 2

ATAmlLIARSETc

X.

, X . . X . . . . . .

X ,

Efx

;]=u

Efx

. .)

• n

Var [Xi)

• r

'

Efx]

• a

varlet

.

?

. "= . .

SLIDE 3 , X y

!

Fine

,

I

^

: D:

I

/

!

""

. ..

.." " . . }

÷÷÷÷÷

.

• ¥71

"⇒.

#HgI*q=

's .IS:c,

ya 7 - i ' :

/

: i ,

• ' '

BAGGED

' ' MODEL X Y

• AN

ENSEMBLE MODEL

tar . )

I:c's

5 5

D*

6 B- BOOTSTRAP BESAMPLES BOOTSTRAP REPLICATES

SLIDE 4 B

Naoise

. = & Ifi # Db)

→ #

BESA - PLES THAT D- to contain

I -th

DATA POINT b

• I
• ops

prenotion

• III. (Xi)

=

noo÷ E FI ( xi)

i -4dB#

WHICH RE SAMPLE 's Do but CONTAIN X

=

0013 BMSE

[ (yi

• II. ( x:))

' c- =/

|

f#

yea , , ,u , D # a pa , .org

LESS

COM POTATO -

→

• way

Bootstrap

• nce

→

1/5

TRAINING TIME

THAN CV

↳

us 5- Foa's c-

SLIDE 5

f)

SIMPLE

SILLY

EXAMPLE

KNN

Is

• l

ppm II Fro - II

it

s

:* ,

• II. Air

.

+1--6

I

'

"" '

=f÷÷÷

,

• II. 1*41=47--5

Bout z z i 6 From IT

'

n' T

:

i¥H÷÷÷

.

it :c"".÷¥÷÷:*

B'

I

x):{÷÷÷ .

• oo .nse=÷iiInui=µ⇒

SLIDE 6

varlet

. ÷÷÷⇒ .

. .

N

↳

Bootstrap REPLICATES

✓

ARE CORRELATED

• HOW

TO REDUCE THIS

CORRELATION ? ADD

RANDOMNESS

to Fc TANG

PROCEDURE !

with trees

• "

←

s." .

• IEIIEII

SLIDE 7

f)

OFTEN TONED

RANDOM

FOREST

SAMPLE AT EACH SPLIT

• m try

→

#

PREDICTORS TO # TREES , THAT IS BOOTSTRAP REPLICATES

TUNING

PARAMETERS ↳

\

n tree →

↳

usually

FAXED

w/

n tree

• 500
• y

> BOOT STRAP TEESAMPLE

BF

w/

mtry =p

is

For

b

in 1 : ntree , p , , , , , ,,⇐ , , y,

D,

\

a gauge, tree . model

• AT

EACH SPLIT

{

a SAMPLE Mt-J VA" " " " F" "

{ × "

' × " "

• "

" " "P} "

• FIND

BEST SPLIT From THESE mt- y UAREIABCES

II

y

. Grow TREE ° "" - "t!!

"

"µ→⇐s

w.tn size

1

• CLASSIFICATION

← REGRESSION ALL NODES AT MOST 5 OBSERVATION ntnee RETURN

£ppG)=

n! I £ ( x)

→

FOR REGRESSION

• b. =L

↳

SEE NEXT PAGE tow CLASSIFICATION

SLIDE 8

CLASSIFICATION

RANDOM

FIRES 5

• y

t.cn#eiEii:cxsntree

" " MAJORITY VOTE ' '

P'

* [y

• KIX
• x]

=

¥ b.FI (

( x)

= k) use , in ramson Forest

Jk , ,a= ( x)

=

Thtkaoentree

pie . ..cn

• II. [* Hx
• D=

EEE . ( x )

" 'II: n b - l l . ESTIMATED PROBABIC 'T -1 From D - th TREE

THEN

CLASSIFY TO LARGEST

PROBABILITY

SLIDE 9

TINI - 6

A

RANDO -

F-zest

• n try

C-

{ I

, 2,3 ,

• -

. .

p)

(

#

FEATURES

( or si

DE

• n

Try

i - w

• J

"

Smale

" m try = I

y

' ' ne .im "

nts

• f"÷

" . . . } '

' " " '

' a . . .

BY DEFAULT

L . ( i 1 BIG

mtg =p

SLIDE 10

STILL

NEEDS

Discussion

• STRENGTHS

OF

RF

→

IT WORKS RIGHT

• ut

OF THE B - X

EXTREMELY

RANDOMIZED TREES ?

Boost .no

!

• gbn
• xgbaost