Decision Theory, and Loss Functions CMSC 691 UMBC Some slides - - PowerPoint PPT Presentation

Decision Theory, and Loss Functions

CMSC 691 UMBC

Some slides adapted from Hamed Pirsiavash

argmin

h

෍

𝑗=1 𝑂

ℓ 𝑧𝑗, ℎ𝜄 𝒚𝑗

F

Today’s Goal: learn about empirical risk minimization

Set t = 0 Pick a starting value θt Until converged:

• 1. Get value y t = F(θ t)
• 2. Get derivative g t = F’(θ t)
• 3. Get scaling factor ρ t
• 4. Set θ t+1 = θ t + ρ t *g t
• 5. Set t += 1

Outline

Decision Theory Loss Functions Multiclass vs. Multilabel Prediction

Decision Theory

“Decision theory is trivial, apart from the computational details” – MacKay, ITILA, Ch 36 Input: x (“state of the world”) Output: a decision ỹ

Decision Theory

“Decision theory is trivial, apart from the computational details” – MacKay, ITILA, Ch 36 Input: x (“state of the world”) Output: a decision ỹ Requirement 1: a decision (hypothesis) function h(x) to produce ỹ

Decision Theory

“Decision theory is trivial, apart from the computational details” – MacKay, ITILA, Ch 36 Input: x (“state of the world”) Output: a decision ỹ Requirement 1: a decision (hypothesis) function h(x) to produce ỹ Requirement 2: a function ℓ(y, ỹ) telling us how wrong we are

Decision Theory

“Decision theory is trivial, apart from the computational details” – MacKay, ITILA, Ch 36 Input: x (“state of the world”) Output: a decision ỹ Requirement 1: a decision (hypothesis) function h(x) to produce ỹ Requirement 2: a loss function ℓ(y, ỹ) telling us how wrong we are Goal: minimize our expected loss across any possible input

score

Requirement 1: Decision Function

instance 1 instance 2 instance 3 instance 4

Evaluator

Gold/correct labels

h(x) is our predictor (classifier, regression model, clustering model, etc.)

Machine Learning Predictor Extra-knowledge

h(x)

Requirement 2: Loss Function

ℓ 𝑧, ො 𝑧 ≥ 0

“correct” label/result predicted label/result “ell” (fancy l character)

loss: A function that tells you how much to penalize a prediction ỹ from the correct answer y

• ptimize ℓ?
• minimize
• maximize

Requirement 2: Loss Function

ℓ 𝑧, ො 𝑧 ≥ 0

“correct” label/result predicted label/result “ell” (fancy l character)

loss: A function that tells you how much to penalize a prediction ỹ from the correct answer y

Negative ℓ (−ℓ) is called a utility or reward function

Decision Theory

minimize expected loss across any possible input

arg min

ො 𝑧 𝔽[ℓ(𝑧, ො

𝑧)]

Risk Minimization

minimize expected loss across any possible input

a particular, unspecified input pair (x,y)… but we want any possible pair

arg min

ො 𝑧 𝔽[ℓ(𝑧, ො

𝑧)] = arg min

ℎ 𝔽[ℓ(𝑧, ℎ(𝒚))]

Decision Theory

minimize expected loss across any possible input input

arg min

ො 𝑧 𝔽[ℓ(𝑧, ො

𝑧)] = arg min

ℎ 𝔽[ℓ(𝑧, ℎ(𝒚))] =

argmin

h

𝔽 𝒚,𝑧 ∼𝑄 ℓ 𝑧, ℎ 𝒚

Assumption: there exists some true (but likely unknown) distribution P over inputs x and outputs y

Risk Minimization

minimize expected loss across any possible input

arg min

ො 𝑧 𝔽[ℓ(𝑧, ො

𝑧)] = arg min

ℎ 𝔽[ℓ(𝑧, ℎ(𝒚))] =

argmin

h

𝔽 𝒚,𝑧 ∼𝑄 ℓ 𝑧, ℎ 𝒚 = argmin

h ∫ ℓ 𝑧, ℎ 𝒚

𝑄 𝒚, 𝑧 𝑒(𝒚, 𝑧)

Risk Minimization

minimize expected loss across any possible input

arg min

ො 𝑧 𝔽[ℓ(𝑧, ො

𝑧)] = arg min

ℎ 𝔽[ℓ(𝑧, ℎ(𝒚))] =

argmin

h

𝔽 𝒚,𝑧 ∼𝑄 ℓ 𝑧, ℎ 𝒚 = argmin

h ∫ ℓ 𝑧, ℎ 𝒚

𝑄 𝒚, 𝑧 𝑒(𝒚, 𝑧)

we don’t know this distribution*!

*we could try to approximate it analytically

(Posterior) Empirical Risk Minimization

minimize expected (posterior) loss across our observed input

arg min

ො 𝑧 𝔽[ℓ(𝑧, ො

𝑧)] = arg min

ℎ 𝔽[ℓ(𝑧, ℎ(𝒚))] =

argmin

h

𝔽 𝒚,𝑧 ∼𝑄 ℓ 𝑧, ℎ 𝒚 ≈ argmin

h

1 𝑂 ෍

𝑗=1 𝑂

𝔽𝑧∼𝑄(⋅|𝒚𝒋) ℓ 𝑧, ℎ 𝒚𝒋

Empirical Risk Minimization

minimize expected loss across our observed input (& output)

arg min

ො 𝑧 𝔽[ℓ(𝑧, ො

𝑧)] = arg min

ℎ 𝔽[ℓ(𝑧, ℎ(𝒚))] =

argmin

h

𝔽 𝒚,𝑧 ∼𝑄 ℓ 𝑧, ℎ 𝒚 ≈ argmin

h

1 𝑂 ෍

𝑗=1 𝑂

ℓ 𝑧𝑗, ℎ 𝒚𝑗

Empirical Risk Minimization

minimize expected loss across our observed input (& output)

argmin

h

෍

𝑗=1 𝑂

ℓ 𝑧𝑗, ℎ 𝒚𝑗

• ur

classifier/predictor controlled by our parameters θ

change θ → change the behavior of the classifier

Best Case: Optimize Empirical Risk with Gradients

argmin

h

෍

𝑗=1 𝑂

ℓ 𝑧𝑗, ℎ𝜄 𝒚𝑗 argmin

𝜄

෍

𝑗=1 𝑂

ℓ 𝑧𝑗, ℎ𝜄 𝒚𝑗

change θ → change the behavior of the classifier

Best Case: Optimize Empirical Risk with Gradients

differentiating might not always work: “… apart from the computational details”

argmin

𝜄

෍

𝑗=1 𝑂

ℓ 𝑧𝑗, ℎ𝜄 𝒚𝑗

change θ → change the behavior of the classifier

How? Use Gradient Descent on 𝐺(𝜄)!

𝐺(𝜄)

Best Case: Optimize Empirical Risk with Gradients

𝛼𝜄𝐺 = ෍

𝑗

𝜖ℓ 𝑧𝑗, ො 𝑧 = ℎ𝜄 𝒚𝑗 𝜖 ො 𝑧 𝛼𝜄ℎ𝜄 𝒚𝒋

differentiating might not always work: “… apart from the computational details”

argmin

𝜄

෍

𝑗=1 𝑂

ℓ 𝑧𝑗, ℎ𝜄 𝒚𝑗

change θ → change the behavior of the classifier

Best Case: Optimize Empirical Risk with Gradients

𝛼𝜄𝐺 = ෍

𝑗

𝜖ℓ 𝑧𝑗, ො 𝑧 = ℎ𝜄 𝒚𝑗 𝜖 ො 𝑧 𝛼𝜄ℎ𝜄 𝒚𝒋

differentiating might not always work: “… apart from the computational details”

argmin

𝜄

෍

𝑗=1 𝑂

ℓ 𝑧𝑗, ℎ𝜄 𝒚𝑗

change θ → change the behavior of the classifier

Step 1: compute the gradient of the loss wrt the predicted value

Best Case: Optimize Empirical Risk with Gradients

𝛼𝜄𝐺 = ෍

𝑗

𝜖ℓ 𝑧𝑗, ො 𝑧 = ℎ𝜄 𝒚𝑗 𝜖 ො 𝑧 𝛼𝜄ℎ𝜄 𝒚𝒋

differentiating might not always work: “… apart from the computational details”

argmin

𝜄

෍

𝑗=1 𝑂

ℓ 𝑧𝑗, ℎ𝜄 𝒚𝑗

change θ → change the behavior of the classifier

Step 1: compute the gradient of the loss wrt the predicted value Step 2: compute the gradient of the predicted value wrt 𝜄.

Outline

Decision Theory Loss Functions Multiclass vs. Multilabel Prediction

Loss Functions Serve a Task

Classification Regression Clustering Fully-supervised Semi-supervised Un-supervised

Probabilistic Generative Conditional Spectral Neural Memory- based Exemplar …

the data: amount of human input/number

• f labeled examples

the approach: how any data are being used the task: what kind

• f problem are you

solving?

Classification: Supervised Machine Learning

Assigning subject categories, topics, or genres Spam detection Authorship identification Age/gender identification Language Identification Sentiment analysis …

Input:

an instance d a fixed set of classes C = {c1, c2,…, cJ} A training set of m hand-labeled instances (d1,c1),....,(dm,cm)

Output:

a learned classifier γ that maps instances to classes

γ learns to associate certain features of instances with their labels

Classification Loss Function Example: 0-1 Loss

ℓ 𝑧, ො 𝑧 = ቊ0, if 𝑧 = ො 𝑧 1, if 𝑧 ≠ ො 𝑧

Classification Loss Function Example: 0-1 Loss

ℓ 𝑧, ො 𝑧 = ቊ0, if 𝑧 = ො 𝑧 1, if 𝑧 ≠ ො 𝑧

Problem 1: not differentiable wrt ො 𝑧 (or θ)

Surrogate loss: replace Zero/one loss by a smooth function Easier to optimize if the surrogate loss is convex

Convex surrogate loss functions

Courtesy Hamed Pirsiavash, CIML

ෝ 𝑧𝑗

Example: ERM with Exponential loss

Courtesy Hamed Pirsiavash

• bjective

Example: ERM with Exponential loss

Courtesy Hamed Pirsiavash

gradient

• bjective

Example: ERM with Exponential loss

loss term

high for misclassified points

Courtesy Hamed Pirsiavash

gradient

• bjective

update

Structured Classification: Sequence & Structured Prediction

Courtesy Hamed Pirsiavash

Classification Loss Function Example: 0-1 Loss

ℓ 𝑧, ො 𝑧 = ቊ0, if 𝑧 = ො 𝑧 1, if 𝑧 ≠ ො 𝑧

Problem 1: not differentiable wrt ො 𝑧 (or θ) Problem 2: too strict. Structured Prediction involves many individual decisions Solution 1: Specialize 0-1 to the structured problem at hand

Regression

Like classification, but real-valued

Regression Example: Stock Market Prediction

Courtesy Hamed Pirsiavash

Regression Loss Function Examples

ℓ 𝑧, ො 𝑧 = y − ො 𝑧 2 squared loss/MSE (Mean squared error)

ො 𝑧 is a real value → nicely differentiable (generally) ☺

Regression Loss Function Examples

ℓ 𝑧, ො 𝑧 = y − ො 𝑧 2 ℓ 𝑧, ො 𝑧 = |𝑧 − ො 𝑧| squared loss/MSE (Mean squared error) absolute loss

ො 𝑧 is a real value → nicely differentiable (generally) ☺ Absolute value is mostly differentiable

Regression Loss Function Examples

ℓ 𝑧, ො 𝑧 = y − ො 𝑧 2 ℓ 𝑧, ො 𝑧 = |𝑧 − ො 𝑧| squared loss/MSE (Mean squared error) absolute loss

ො 𝑧 is a real value → nicely differentiable (generally) ☺ Absolute value is mostly differentiable

These loss functions prefer different behavior in the predictions (hint: look at the gradient of each)… we’ll get back to this

Outline

Decision Theory Loss Functions Multiclass vs. Multilabel Prediction

Multi-class Classification

Given input 𝑦, predict discrete label 𝑧

Multi-label Classification

Multi-class Classification

Given input 𝑦, predict discrete label 𝑧

If 𝑧 ∈ {0,1} (or 𝑧 ∈ {True, False}), then a binary classification task

Multi-label Classification

Multi-class Classification

Given input 𝑦, predict discrete label 𝑧

If 𝑧 ∈ {0,1} (or 𝑧 ∈ {True, False}), then a binary classification task If 𝑧 ∈ {0,1, … , 𝐿 − 1} (for finite K), then a multi-class classification task Q: What are some examples

• f multi-class classification?

Multi-label Classification

Multi-class Classification

Given input 𝑦, predict discrete label 𝑧

If 𝑧 ∈ {0,1} (or 𝑧 ∈ {True, False}), then a binary classification task If 𝑧 ∈ {0,1, … , 𝐿 − 1} (for finite K), then a multi-class classification task Q: What are some examples

• f multi-class classification?

A: Many possibilities. See A2, Q{1,2,4-7}

Multi-label Classification

Multi-class Classification

Given input 𝑦, predict discrete label 𝑧

If 𝑧 ∈ {0,1} (or 𝑧 ∈ {True, False}), then a binary classification task If 𝑧 ∈ {0,1, … , 𝐿 − 1} (for finite K), then a multi-class classification task

Multi-label Classification

Single

• utput

Multi-

• utput

If multiple 𝑧𝑚 are predicted, then a multi- label classification task

Multi-class Classification

Given input 𝑦, predict discrete label 𝑧

If 𝑧 ∈ {0,1} (or 𝑧 ∈ {True, False}), then a binary classification task If 𝑧 ∈ {0,1, … , 𝐿 − 1} (for finite K), then a multi-class classification task

Multi-label Classification

Single

• utput

Multi-

• utput

Given input 𝑦, predict multiple discrete labels 𝑧 = (𝑧1, … , 𝑧𝑀)

If multiple 𝑧𝑚 are predicted, then a multi- label classification task

Multi-class Classification

Given input 𝑦, predict discrete label 𝑧

If 𝑧 ∈ {0,1} (or 𝑧 ∈ {True, False}), then a binary classification task If 𝑧 ∈ {0,1, … , 𝐿 − 1} (for finite K), then a multi-class classification task

Multi-label Classification

Single

• utput

Multi-

• utput

Given input 𝑦, predict multiple discrete labels 𝑧 = (𝑧1, … , 𝑧𝑀)

If multiple 𝑧𝑚 are predicted, then a multi- label classification task Each 𝑧𝑚 could be binary or multi-class

Outline

Decision Theory Loss Functions Multiclass vs. Multilabel Prediction

Bring it all together: MAP, 0/1 loss, cross-entropy, log- likelihood likelihood

• 1. Show that a MAP estimation 𝑞 𝑧 𝑦)

minimizes 0/1 loss

• 2. Show that minimizing cross-entropy loss is

the same as maximizing the (conditional) log- likelihood

• 3. Consider: what is cross-entropy in a multi-

label setting?