Introduction to Machine Learning Classification and Regression Trees - - PowerPoint PPT Presentation

Introduction to Machine Learning Classification and Regression Trees (CART): Basics

compstat-lmu.github.io/lecture_i2ml

TREE MODEL AND PREDICTION

Classification and Regression Trees, introduced by Breiman Binary splits are constructed top-down Constant prediction in each terminal node (leaf): either a numerical value, a class label or a probability vector over class labels.

c

• Introduction to Machine Learning – 1 / 5

TREE MODEL AND PREDICTION

For predictions, observations are passed down the tree, according to the splitting rules in each node An observation will end up in exactly one leaf node All observations in a leaf node are assigned the same prediction for the target

c

• Introduction to Machine Learning – 2 / 5

TREE MODEL AND PREDICTION

For predictions, observations are passed down the tree, according to the splitting rules in each node An observation will end up in exactly one leaf node All observations in a leaf node are assigned the same prediction for the target

• 1.20
• 0.42

0.98

• 0.20
• 0.01

c

• Introduction to Machine Learning – 3 / 5

TREES AS AN ADDITIVE MODEL

Each point in X is assigned to exactly one leaf, and each leaf has a set

• f input points leading to it, through axis-parallel splits.

Hence, trees divide the feature space X into rectangular regions: f(x) =

M

• m=1

cmI(x ∈ Qm), where a tree with M leaf nodes defines M “rectangles” Qm. cm is the predicted numerical response, class label or class distribution in the respective leaf node.

• ●
• ●
• ●
• ●
• ●
• ●
• ● ●
• ●
• 0.0

0.5 1.0 1.5 2.0 2.5 2 4 6

Petal.Length Petal.Width Species

• setosa

versicolor virginica

Iris Data

c

• Introduction to Machine Learning – 4 / 5

TREES

The hypothesis space of a CART is the set of all step functions over rectangular partitions of X : f(x) =

M

• m=1

cmI(x ∈ Qm), Classification:

• ●
• ●
• ●
• ●
• 0.0

0.5 1.0 1.5 2.0 2.5 2 4 6

Petal.Length Petal.Width Species

• setosa

versicolor virginica

Iris Data

Regression:

• 1.20
• 0.42

0.98

• 0.20
• 0.01

c

• Introduction to Machine Learning – 5 / 5