[PPT] - Classification March 19, 2020 Data Science CSCI 1951A Brown PowerPoint Presentation

SLIDE 1

Classification

March 19, 2020 Data Science CSCI 1951A Brown University Instructor: Ellie Pavlick HTAs: Josh Levin, Diane Mutako, Sol Zitter

1

SLIDE 2

Today

Generative vs. Discriminative Models
KNN, Naive Bayes, Logistic Regression
SciKit Learn Demo

2

SLIDE 3

Supervised vs. Unsupervised Learning

Supervised: Explicit data labels
Sentiment analysis—review text -> star ratings
Image tagging—image -> caption
Unsupervised: No explicit labels
Clustering—find groups similar customers
Dimensionality Reduction—find features that

differentiate individuals

3

SLIDE 4

Classification

One Goal: P(Y|X)

SLIDE 5

Classification

One Goal: P(Y|X)

Label

SLIDE 6

Classification

One Goal: P(Y|X)

Label Features

SLIDE 7

Classification

One Goal: P(Y|X)

P(email is spam | words in the message) P(genre of song|tempo, harmony, lyrics…) P(article clicked | title, font, photo…)

SLIDE 8

tempo harmonic complexity

K Means

8

SLIDE 9

tempo harmonic complexity

X X

K Means

9

SLIDE 10

tempo harmonic complexity

K Nearest Neighbors

10

Blue or Red?

SLIDE 11

tempo harmonic complexity

K Nearest Neighbors

11

K = 1

SLIDE 12

tempo harmonic complexity

K Nearest Neighbors

12

K = 5

SLIDE 13

K Nearest Neighbors

13

tempo harmonic complexity

SLIDE 14

K Nearest Neighbors

14

tempo harmonic complexity K = 5

SLIDE 15

K Nearest Neighbors

15

tempo harmonic complexity K = 5

SLIDE 16

K Nearest Neighbors

16

tempo harmonic complexity K = 5

SLIDE 17

Arguably the simplest ML algorithm
“Non-Parametric” — no assumptions about the

form of the classification model

All the work is done at classification time
Works with tiny amounts of training data (single

example per class)

The best classification model ever???

K Nearest Neighbors

SLIDE 18

Arguably the simplest ML algorithm
“Non-Parametric” — no assumptions about the

form of the classification model

All the work is done at classification time
Works with tiny amounts of training data (single

example per class)

The best classification model ever???

K Nearest Neighbors

SLIDE 19

Arguably the simplest ML algorithm
“Non-Parametric” — no assumptions about the

form of the classification model

All the work is done at classification time
Works with tiny amounts of training data (single

example per class)

The best classification model ever???

K Nearest Neighbors

SLIDE 20

Arguably the simplest ML algorithm
“Non-Parametric” — no assumptions about the

form of the classification model

All the work is done at classification time
Works with tiny amounts of training data (single

example per class)

The best classification model ever???

K Nearest Neighbors

SLIDE 21

Arguably the simplest ML algorithm
“Non-Parametric” — no assumptions about the

form of the classification model

All the work is done at classification time
Works with tiny amounts of training data (single

example per class)

The best classification model ever???

K Nearest Neighbors

SLIDE 22

Arguably the simplest ML algorithm
“Non-Parametric” — no assumptions about the

form of the classification model

All the work is done at classification time
Works with tiny amounts of training data (single

example per class)

The best classification model ever???

K Nearest Neighbors

SLIDE 23

Supervised Classification

https://scikit-learn.org/stable/auto_examples/classification/plot_classifier_comparison.html

SLIDE 24

Generative Models Discriminative Models

SLIDE 25

Generative Models Discriminative Models

estimate P(X, Y) first

SLIDE 26

Generative Models Discriminative Models

estimate P(Y | X) directly estimate P(X, Y) first

SLIDE 27

Generative Models Discriminative Models

estimate P(Y | X) directly estimate P(X, Y) first /no explicit probability model

SLIDE 28

Generative Models Discriminative Models

estimate P(Y | X) directly estimate P(X, Y) first /no explicit probability model Can assign probability to

bservations, generate

new observations

SLIDE 29

Generative Models Discriminative Models

estimate P(Y | X) directly estimate P(X, Y) first /no explicit probability model Can assign probability to

bservations, generate

new observations Only supports classification, less flexible

SLIDE 30

Generative Models Discriminative Models

estimate P(Y | X) directly estimate P(X, Y) first /no explicit probability model Can assign probability to

bservations, generate

new observations Only supports classification, less flexible Often more parameters, but more flexible

SLIDE 31

Generative Models Discriminative Models

estimate P(Y | X) directly estimate P(X, Y) first /no explicit probability model Can assign probability to

bservations, generate

new observations Only supports classification, less flexible Often more parameters, but more flexible Often fewer parameters, better performance on small data

SLIDE 32

Generative Models Discriminative Models

estimate P(Y | X) directly estimate P(X, Y) first /no explicit probability model Can assign probability to

bservations, generate

new observations Only supports classification, less flexible Often more parameters, but more flexible Naive Bayes, Bayes Nets, VAEs, GANs Often fewer parameters, better performance on small data

SLIDE 33

Generative Models Discriminative Models

estimate P(Y | X) directly estimate P(X, Y) first /no explicit probability model Can assign probability to

bservations, generate

new observations Only supports classification, less flexible Often more parameters, but more flexible Naive Bayes, Bayes Nets, VAEs, GANs Logistic Regression, SVMs, Perceptrons Often fewer parameters, better performance on small data

SLIDE 34

Generative Models Discriminative Models

estimate P(Y | X) directly estimate P(X, Y) first /no explicit probability model Can assign probability to

bservations, generate

new observations Only supports classification, less flexible Often more parameters, but more flexible Naive Bayes, Bayes Nets, VAEs, GANs Logistic Regression, SVMs, Perceptrons KNN Often fewer parameters, better performance on small data

SLIDE 35

Generative Models Discriminative Models

estimate P(Y | X) directly estimate P(X, Y) first /no explicit probability model Can assign probability to

bservations, generate

new observations Only supports classification, less flexible Often more parameters, but more flexible Naive Bayes, Bayes Nets, VAEs, GANs Logistic Regression, SVMs, Perceptrons KNN Often fewer parameters, better performance on small data

SLIDE 36

Supervised Classification

SLIDE 37

Supervised Classification

Lovely mushroomy nose and good length. ***** Good if not dramatic fizz. *** Rubbery - rather oxidised. * Gamy, succulent tannins. Lovely. **** Quite raw finish. A bit rubbery. ** Provence herbs, creamy, lovely. ****

SLIDE 38

Supervised Classification

Lovely mushroomy nose and good length. 1 Good if not dramatic fizz. 0 Rubbery - rather oxidised. 0 Gamy, succulent tannins. Lovely. 1 Quite raw finish. A bit rubbery. 0 Provence herbs, creamy, lovely. 1

SLIDE 39

Supervised Classification

Lovely mushroomy nose and good length. 1 Good if not dramatic fizz. 0 Rubbery - rather oxidised. 0 Gamy, succulent tannins. Lovely. 1 Quite raw finish. A bit rubbery. 0 Provence herbs, creamy, lovely. 1

Label lovely good raw rubbery rather mushroomy gamy … 1 1 1 … 1 1 1 … 1 1 … 1 1 …

SLIDE 40

Supervised Classification

Lovely mushroomy nose and good length. 1 Good if not dramatic fizz. 0 Rubbery - rather oxidised. 0 Gamy, succulent tannins. Lovely. 1 Quite raw finish. A bit rubbery. 0 Provence herbs, creamy, lovely. 1

Label lovely good raw rubbery rather mushroomy gamy … 1 1 1 … 1 1 1 … 1 1 … 1 1 …

y

SLIDE 41

Supervised Classification

Lovely mushroomy nose and good length. 1 Good if not dramatic fizz. 0 Rubbery - rather oxidised. 0 Gamy, succulent tannins. Lovely. 1 Quite raw finish. A bit rubbery. 0 Provence herbs, creamy, lovely. 1

Label lovely good raw rubbery rather mushroomy gamy … 1 1 1 … 1 1 1 … 1 1 … 1 1 …

y X

SLIDE 42

Supervised Classification

Lovely mushroomy nose and good length. 1 Good if not dramatic fizz. 0 Rubbery - rather oxidised. 0 Gamy, succulent tannins. Lovely. 1 Quite raw finish. A bit rubbery. 0 Provence herbs, creamy, lovely. 1

Label lovely good raw rubbery rather mushroomy gamy … 1 1 1 … 1 1 1 … 1 1 … 1 1 … ??? 1 1 1 1 …

y X

SLIDE 43

Bayes Rule

SLIDE 44

P(Y|X) = P(X|Y)P(Y) P(X)

Bayes Rule

SLIDE 45

P(Y|X) = P(X|Y)P(Y) P(X)

Bayes Rule

SLIDE 46

Label lovely good raw rubbery rather mushroomy gamy … 1 1 1 … 1 1 1 … 1 1 … 1 1 …

Bayes Rule

SLIDE 47

Label lovely good raw rubbery rather mushroomy gamy … 1 1 1 … 1 1 1 … 1 1 … 1 1 …

Bayes Rule

P(Y=1|lovely, good,…) =P(lovely, good,…|Y=1)P(Y=1) =P(Y=1, lovely, good,…) =P(lovely|Y=1, good,…)P(Y=1, good,…)

SLIDE 48

Label lovely good raw rubbery rather mushroomy gamy … 1 1 1 … 1 1 1 … 1 1 … 1 1 …

Bayes Rule

P(Y=1|lovely, good,…) =P(lovely, good,…|Y=1)P(Y=1) =P(Y=1, lovely, good,…) =P(lovely|Y=1, good,…)P(Y=1, good,…)

SLIDE 49

Label lovely good raw rubbery rather mushroomy gamy … 1 1 1 … 1 1 1 … 1 1 … 1 1 …

Bayes Rule

P(Y=1|lovely, good,…) =P(lovely, good,…|Y=1)P(Y=1) =P(Y=1, lovely, good,…) =P(lovely|Y=1, good,…)P(Y=1, good,…)

SLIDE 50

Label lovely good raw rubbery rather mushroomy gamy … 1 1 1 … 1 1 1 … 1 1 … 1 1 …

Bayes Rule

P(Y=1|lovely, good,…) =P(lovely, good,…|Y=1)P(Y=1) =P(Y=1, lovely, good,…) =P(lovely|Y=1, good,…)P(Y=1, good,…)

SLIDE 51

Label lovely good raw rubbery rather mushroomy gamy … 1 1 1 … 1 1 1 … 1 1 … 1 1 …

Bayes Rule

P(C|x1, x2, …, xk) =P(x1|x2, …, xk, C)P(x2|x3, …, xk, C)…P(xk|C)P(C) =P(x1|C)P(x2|C)…P(xk|C)P(C)

SLIDE 52

Label lovely good raw rubbery rather mushroomy gamy … 1 1 1 … 1 1 1 … 1 1 … 1 1 …

Naive Bayes

Assume features are independent!

P(C|x1, x2, …, xk) =P(x1|x2, …, xk, C)P(x2|x3, …, xk, C)…P(xk|C)P(C) =P(x1|C)P(x2|C)…P(xk|C)P(C)

SLIDE 53

Label lovely good raw rubbery rather mushroomy gamy … 1 1 1 … 1 1 1 … 1 1 … 1 1 …

Naive Bayes

P(C|x1, x2, …, xk) =P(x1|x2, …, xk, C)P(x2|x3, …, xk, C)…P(xk|C)P(C) =P(x1|C)P(x2|C)…P(xk|C)P(C)

Assume features are independent!

SLIDE 54

Naive Bayes

x P(x|Y=1) P(x|Y=0) lovely ?? ?? good ?? ?? raw ?? ?? rubbery ?? ??

Lovely mushroomy nose and good length. 1 Good if not dramatic fizz. 0 Rubbery - rather oxidised. 0 Gamy, succulent tannins. Lovely. 1 Quite raw finish. A bit rubbery. 0 Provence herbs, creamy, lovely. 1

SLIDE 55

Lovely mushroomy nose and good length. 1 Good if not dramatic fizz. 0 Rubbery - rather oxidised. 0 Gamy, succulent tannins. Lovely. 1 Quite raw finish. A bit rubbery. 0 Provence herbs, creamy, lovely. 1

Clicker Question!

x P(x|Y=1) P(x|Y=0) lovely ?? ?? good ?? ?? raw ?? ?? rubbery ?? ??

(a)1.0, 0.0 (b)1/2, 1/2 (c) 1/3 , 1/3

SLIDE 56

Lovely mushroomy nose and good length. 1 Good if not dramatic fizz. 0 Rubbery - rather oxidised. 0 Gamy, succulent tannins. Lovely. 1 Quite raw finish. A bit rubbery. 0 Provence herbs, creamy, lovely. 1

Clicker Question!

x P(x|Y=1) P(x|Y=0) lovely ?? ?? good 1/3 1/3 raw ?? ?? rubbery ?? ??

(a)1.0, 0.0 (b)1/2, 1/2 (c) 1/3 , 1/3

SLIDE 57

Quite mushroomy, a bit dramatic. ???

Naive Bayes

x P(x|Y=1) P(x|Y=0) a 0.9 0.9 bit 0.2 0.4 dramatic 0.6 0.4 gamy 0.1 0.0 good 0.2 0.2 lovely 0.5 0.1 mushroomy 0.2 0.2 quite 0.7 0.8

SLIDE 58

x P(x|Y=1) P(x|Y=0) a 0.9 0.9 bit 0.2 0.4 dramatic 0.6 0.4 gamy 0.1 0.0 good 0.2 0.2 lovely 0.5 0.1 mushroomy 0.2 0.2 quite 0.7 0.8

Quite mushroomy, a bit dramatic. ???

Naive Bayes

What do we do now?

SLIDE 59

x P(x|Y=1) P(x|Y=0) a 0.9 0.9 bit 0.2 0.4 dramatic 0.6 0.4 gamy 0.1 0.0 good 0.2 0.2 lovely 0.5 0.1 mushroomy 0.2 0.2 quite 0.7 0.8

Quite mushroomy, a bit dramatic. ???

Naive Bayes

P(Y|X) = P(X|Y)P(Y)

SLIDE 60

Quite mushroomy, a bit dramatic. ???

Naive Bayes

P(Y|X) = P(X|Y)P(Y)

x P(x|Y=1) P(x|Y=0) a 0.9 0.9 bit 0.2 0.4 dramatic 0.6 0.4 gamy 0.1 0.0 good 0.2 0.2 lovely 0.5 0.1 mushroomy 0.2 0.2 quite 0.7 0.8

???

SLIDE 61

Quite mushroomy, a bit dramatic. ???

Naive Bayes

P(Y|X) = P(X|Y)P(Y)

x P(x|Y=1) P(x|Y=0) a 0.9 0.9 bit 0.2 0.4 dramatic 0.6 0.4 gamy 0.1 0.0 good 0.2 0.2 lovely 0.5 0.1 mushroomy 0.2 0.2 quite 0.7 0.8

Domain knowledge

r estimate from data

SLIDE 62

Quite mushroomy, a bit dramatic. ???

Naive Bayes

P(Y|X) = P(X|Y)P(Y)

x P(x|Y=1) P(x|Y=0) a 0.9 0.9 bit 0.2 0.4 dramatic 0.6 0.4 gamy 0.1 0.0 good 0.2 0.2 lovely 0.5 0.1 mushroomy 0.2 0.2 quite 0.7 0.8

Decision rule: argmax_y P(Y=y|X)

SLIDE 63

Quite mushroomy, a bit dramatic. ???

x P(x|Y=1) P(x|Y=0) a 0.9 0.9 bit 0.2 0.4 dramatic 0.6 0.4 gamy 0.1 0.0 good 0.2 0.2 lovely 0.5 0.1 mushroomy 0.2 0.2 quite 0.7 0.8 P(Y=1) P(Y=0) 0.3 0.7

Clicker Question! (a) Positive (b) Negative

SLIDE 64

Quite mushroomy, a bit dramatic. ???

x P(x|Y=1) P(x|Y=0) a 0.9 0.9 bit 0.2 0.4 dramatic 0.6 0.4 gamy 0.1 0.0 good 0.2 0.2 lovely 0.5 0.1 mushroomy 0.2 0.2 quite 0.7 0.8 P(Y=1) P(Y=0) 0.3 0.7

Clicker Question! (a) Positive (b) Negative

0.9 x 0.2 x 0.6 x 0.2 x 0.7 x 0.3 = 0.005

SLIDE 65

Quite mushroomy, a bit dramatic. ???

x P(x|Y=1) P(x|Y=0) a 0.9 0.9 bit 0.2 0.4 dramatic 0.6 0.4 gamy 0.1 0.0 good 0.2 0.2 lovely 0.5 0.1 mushroomy 0.2 0.2 quite 0.7 0.8 P(Y=1) P(Y=0) 0.3 0.7

Clicker Question! (a) Positive (b) Negative

0.9 x 0.2 x 0.6 x 0.2 x 0.7 x 0.3 = 0.005 0.9 x 0.4 x 0.4 x 0.2 x 0.8 x 0.7 = 0.016

SLIDE 66

Quite mushroomy, a bit dramatic. ???

x P(x|Y=1) P(x|Y=0) a 0.9 0.9 bit 0.2 0.4 dramatic 0.6 0.4 gamy 0.1 0.0 good 0.2 0.2 lovely 0.5 0.1 mushroomy 0.2 0.2 quite 0.7 0.8 P(Y=1) P(Y=0) 0.3 0.7

Clicker Question! (a) Positive (b) Negative

0.9 x 0.2 x 0.6 x 0.2 x 0.7 x 0.3 = 0.005 0.9 x 0.4 x 0.4 x 0.2 x 0.8 x 0.7 = 0.016

SLIDE 67

Naive Bayes

x P(x|Y=1) P(x|Y=0) a 0.9 0.9 bit 0.2 0.4 dramatic 0.6 0.4 gamy 0.1 0.0 good 0.2 0.2 lovely 0.5 0.1 mushroomy 0.2 0.2 quite 0.7 0.8

SLIDE 68

Naive Bayes

x P(x|Y=1) P(x|Y=0) a 0.9 0.9 bit 0.2 0.4 dramatic 0.6 0.4 gamy 0.1 0.0 good 0.2 0.2 lovely 0.5 0.1 mushroomy 0.2 0.2 quite 0.7 0.8

… 1

SLIDE 69

Naive Bayes

x P(x|Y=1) P(x|Y=0) a 0.9 0.9 bit 0.2 0.4 dramatic 0.6 0.4 gamy 0.1 0.0 good 0.2 0.2 lovely 0.5 0.1 mushroomy 0.2 0.2 quite 0.7 0.8

A … 0.9

SLIDE 70

Naive Bayes

x P(x|Y=1) P(x|Y=0) a 0.9 0.9 bit 0.2 0.4 dramatic 0.6 0.4 gamy 0.1 0.0 good 0.2 0.2 lovely 0.5 0.1 mushroomy 0.2 0.2 quite 0.7 0.8

A quite … 0.63

SLIDE 71

Naive Bayes

x P(x|Y=1) P(x|Y=0) a 0.9 0.9 bit 0.2 0.4 dramatic 0.6 0.4 gamy 0.1 0.0 good 0.2 0.2 lovely 0.5 0.1 mushroomy 0.2 0.2 quite 0.7 0.8

A quite dramatic … 0.38

SLIDE 72

Naive Bayes

x P(x|Y=1) P(x|Y=0) a 0.9 0.9 bit 0.2 0.4 dramatic 0.6 0.4 gamy 0.1 0.0 good 0.2 0.2 lovely 0.5 0.1 mushroomy 0.2 0.2 quite 0.7 0.8

A quite dramatic gamy … 0.04

SLIDE 73

73

SLIDE 74

Linear Regression

SLIDE 75

Linear Regression

y x

SLIDE 76

Linear Regression

y = w1x1 + w2x2 + · · · + wkxk

<latexit sha1_base64="g1kuJCNP+kOmhk9MaU6EkxSH5cY=">ACEHicbZDLSsNAFIZPvNZ6i7p0M1hEQShJEXQjFN24rGAv0IYwmU7aoZMLMxM1hD6CG1/FjQtF3Lp059s4TbPQ1gMHPv5zmTm/F3MmlWV9GwuLS8srq6W18vrG5ta2ubPbklEiCG2SiEei42FJOQtpUzHFaScWFAcep21vdDWpt+okCwKb1UaUyfAg5D5jGClJdc8StEFundt9KDzRFNU01Trx8pmSsjrYxcs2JVrTzQPNgFVKCIhmt+6Q0kCWioCMdSdm0rVk6GhWKE03G5l0gaYzLCA9rVGOKASifLDxqjQ630kR8JnaFCufp7IsOBlGng6c4Aq6GcrU3E/2rdRPnTsbCOFE0JNOH/IQjFaGJO6jPBCWKpxowEUz/FZEhFpgo7WFZm2DPnjwPrVrVtqr2zWmlflnYUYJ9OIBjsOEM6nANDWgCgUd4hld4M56MF+Pd+Ji2LhjFzB78CePzBxz4mYE=</latexit><latexit sha1_base64="g1kuJCNP+kOmhk9MaU6EkxSH5cY=">ACEHicbZDLSsNAFIZPvNZ6i7p0M1hEQShJEXQjFN24rGAv0IYwmU7aoZMLMxM1hD6CG1/FjQtF3Lp059s4TbPQ1gMHPv5zmTm/F3MmlWV9GwuLS8srq6W18vrG5ta2ubPbklEiCG2SiEei42FJOQtpUzHFaScWFAcep21vdDWpt+okCwKb1UaUyfAg5D5jGClJdc8StEFundt9KDzRFNU01Trx8pmSsjrYxcs2JVrTzQPNgFVKCIhmt+6Q0kCWioCMdSdm0rVk6GhWKE03G5l0gaYzLCA9rVGOKASifLDxqjQ630kR8JnaFCufp7IsOBlGng6c4Aq6GcrU3E/2rdRPnTsbCOFE0JNOH/IQjFaGJO6jPBCWKpxowEUz/FZEhFpgo7WFZm2DPnjwPrVrVtqr2zWmlflnYUYJ9OIBjsOEM6nANDWgCgUd4hld4M56MF+Pd+Ji2LhjFzB78CePzBxz4mYE=</latexit><latexit sha1_base64="g1kuJCNP+kOmhk9MaU6EkxSH5cY=">ACEHicbZDLSsNAFIZPvNZ6i7p0M1hEQShJEXQjFN24rGAv0IYwmU7aoZMLMxM1hD6CG1/FjQtF3Lp059s4TbPQ1gMHPv5zmTm/F3MmlWV9GwuLS8srq6W18vrG5ta2ubPbklEiCG2SiEei42FJOQtpUzHFaScWFAcep21vdDWpt+okCwKb1UaUyfAg5D5jGClJdc8StEFundt9KDzRFNU01Trx8pmSsjrYxcs2JVrTzQPNgFVKCIhmt+6Q0kCWioCMdSdm0rVk6GhWKE03G5l0gaYzLCA9rVGOKASifLDxqjQ630kR8JnaFCufp7IsOBlGng6c4Aq6GcrU3E/2rdRPnTsbCOFE0JNOH/IQjFaGJO6jPBCWKpxowEUz/FZEhFpgo7WFZm2DPnjwPrVrVtqr2zWmlflnYUYJ9OIBjsOEM6nANDWgCgUd4hld4M56MF+Pd+Ji2LhjFzB78CePzBxz4mYE=</latexit><latexit sha1_base64="g1kuJCNP+kOmhk9MaU6EkxSH5cY=">ACEHicbZDLSsNAFIZPvNZ6i7p0M1hEQShJEXQjFN24rGAv0IYwmU7aoZMLMxM1hD6CG1/FjQtF3Lp059s4TbPQ1gMHPv5zmTm/F3MmlWV9GwuLS8srq6W18vrG5ta2ubPbklEiCG2SiEei42FJOQtpUzHFaScWFAcep21vdDWpt+okCwKb1UaUyfAg5D5jGClJdc8StEFundt9KDzRFNU01Trx8pmSsjrYxcs2JVrTzQPNgFVKCIhmt+6Q0kCWioCMdSdm0rVk6GhWKE03G5l0gaYzLCA9rVGOKASifLDxqjQ630kR8JnaFCufp7IsOBlGng6c4Aq6GcrU3E/2rdRPnTsbCOFE0JNOH/IQjFaGJO6jPBCWKpxowEUz/FZEhFpgo7WFZm2DPnjwPrVrVtqr2zWmlflnYUYJ9OIBjsOEM6nANDWgCgUd4hld4M56MF+Pd+Ji2LhjFzB78CePzBxz4mYE=</latexit>

y x

SLIDE 77

Linear Regression

y x

y = ~ w · ~ x

<latexit sha1_base64="FBMv2wyAHxkmlGRKqF3+j4ktAn4=">ACAnicbVDLSgMxFM3UV62vUVfiJlgEV2VGBN0IRTcuK9gHtEPJZDJtaCYZkx1GAY3/obF4q49Svc+Tem7Sy09cCFk3PuJfceP2ZUacf5tkpLyura+X1ysbm1vaOvbvXUiKRmDSxYEJ2fKQIo5w0NdWMdGJUOQz0vZH1xO/PSZSUcHvdBoTL0IDTkOKkTZS3z5I4SXsjQnO7nPYw4HQs9dD3rerTs2ZAi4StyBVUKDRt796gcBJRLjGDCnVdZ1YexmSmJG8kovUSRGeIQGpGsoRxFRXjY9IYfHRglgKQpruFU/T2RoUipNPJNZ4T0UM17E/E/r5vo8MLKI8TiefRQmDGoBJ3nAgEqCNUsNQVhSsyvEQyQR1ia1ignBnT95kbROa65Tc2/PqvWrIo4yOARH4AS4BzUwQ1ogCbA4BE8g1fwZj1ZL9a79TFrLVnFzD74A+vzB6XrlvQ=</latexit><latexit sha1_base64="FBMv2wyAHxkmlGRKqF3+j4ktAn4=">ACAnicbVDLSgMxFM3UV62vUVfiJlgEV2VGBN0IRTcuK9gHtEPJZDJtaCYZkx1GAY3/obF4q49Svc+Tem7Sy09cCFk3PuJfceP2ZUacf5tkpLyura+X1ysbm1vaOvbvXUiKRmDSxYEJ2fKQIo5w0NdWMdGJUOQz0vZH1xO/PSZSUcHvdBoTL0IDTkOKkTZS3z5I4SXsjQnO7nPYw4HQs9dD3rerTs2ZAi4StyBVUKDRt796gcBJRLjGDCnVdZ1YexmSmJG8kovUSRGeIQGpGsoRxFRXjY9IYfHRglgKQpruFU/T2RoUipNPJNZ4T0UM17E/E/r5vo8MLKI8TiefRQmDGoBJ3nAgEqCNUsNQVhSsyvEQyQR1ia1ignBnT95kbROa65Tc2/PqvWrIo4yOARH4AS4BzUwQ1ogCbA4BE8g1fwZj1ZL9a79TFrLVnFzD74A+vzB6XrlvQ=</latexit><latexit sha1_base64="FBMv2wyAHxkmlGRKqF3+j4ktAn4=">ACAnicbVDLSgMxFM3UV62vUVfiJlgEV2VGBN0IRTcuK9gHtEPJZDJtaCYZkx1GAY3/obF4q49Svc+Tem7Sy09cCFk3PuJfceP2ZUacf5tkpLyura+X1ysbm1vaOvbvXUiKRmDSxYEJ2fKQIo5w0NdWMdGJUOQz0vZH1xO/PSZSUcHvdBoTL0IDTkOKkTZS3z5I4SXsjQnO7nPYw4HQs9dD3rerTs2ZAi4StyBVUKDRt796gcBJRLjGDCnVdZ1YexmSmJG8kovUSRGeIQGpGsoRxFRXjY9IYfHRglgKQpruFU/T2RoUipNPJNZ4T0UM17E/E/r5vo8MLKI8TiefRQmDGoBJ3nAgEqCNUsNQVhSsyvEQyQR1ia1ignBnT95kbROa65Tc2/PqvWrIo4yOARH4AS4BzUwQ1ogCbA4BE8g1fwZj1ZL9a79TFrLVnFzD74A+vzB6XrlvQ=</latexit><latexit sha1_base64="FBMv2wyAHxkmlGRKqF3+j4ktAn4=">ACAnicbVDLSgMxFM3UV62vUVfiJlgEV2VGBN0IRTcuK9gHtEPJZDJtaCYZkx1GAY3/obF4q49Svc+Tem7Sy09cCFk3PuJfceP2ZUacf5tkpLyura+X1ysbm1vaOvbvXUiKRmDSxYEJ2fKQIo5w0NdWMdGJUOQz0vZH1xO/PSZSUcHvdBoTL0IDTkOKkTZS3z5I4SXsjQnO7nPYw4HQs9dD3rerTs2ZAi4StyBVUKDRt796gcBJRLjGDCnVdZ1YexmSmJG8kovUSRGeIQGpGsoRxFRXjY9IYfHRglgKQpruFU/T2RoUipNPJNZ4T0UM17E/E/r5vo8MLKI8TiefRQmDGoBJ3nAgEqCNUsNQVhSsyvEQyQR1ia1ignBnT95kbROa65Tc2/PqvWrIo4yOARH4AS4BzUwQ1ogCbA4BE8g1fwZj1ZL9a79TFrLVnFzD74A+vzB6XrlvQ=</latexit>

SLIDE 78

Linear Regression

y x

y = ~ w · ~ x

<latexit sha1_base64="FBMv2wyAHxkmlGRKqF3+j4ktAn4=">ACAnicbVDLSgMxFM3UV62vUVfiJlgEV2VGBN0IRTcuK9gHtEPJZDJtaCYZkx1GAY3/obF4q49Svc+Tem7Sy09cCFk3PuJfceP2ZUacf5tkpLyura+X1ysbm1vaOvbvXUiKRmDSxYEJ2fKQIo5w0NdWMdGJUOQz0vZH1xO/PSZSUcHvdBoTL0IDTkOKkTZS3z5I4SXsjQnO7nPYw4HQs9dD3rerTs2ZAi4StyBVUKDRt796gcBJRLjGDCnVdZ1YexmSmJG8kovUSRGeIQGpGsoRxFRXjY9IYfHRglgKQpruFU/T2RoUipNPJNZ4T0UM17E/E/r5vo8MLKI8TiefRQmDGoBJ3nAgEqCNUsNQVhSsyvEQyQR1ia1ignBnT95kbROa65Tc2/PqvWrIo4yOARH4AS4BzUwQ1ogCbA4BE8g1fwZj1ZL9a79TFrLVnFzD74A+vzB6XrlvQ=</latexit><latexit sha1_base64="FBMv2wyAHxkmlGRKqF3+j4ktAn4=">ACAnicbVDLSgMxFM3UV62vUVfiJlgEV2VGBN0IRTcuK9gHtEPJZDJtaCYZkx1GAY3/obF4q49Svc+Tem7Sy09cCFk3PuJfceP2ZUacf5tkpLyura+X1ysbm1vaOvbvXUiKRmDSxYEJ2fKQIo5w0NdWMdGJUOQz0vZH1xO/PSZSUcHvdBoTL0IDTkOKkTZS3z5I4SXsjQnO7nPYw4HQs9dD3rerTs2ZAi4StyBVUKDRt796gcBJRLjGDCnVdZ1YexmSmJG8kovUSRGeIQGpGsoRxFRXjY9IYfHRglgKQpruFU/T2RoUipNPJNZ4T0UM17E/E/r5vo8MLKI8TiefRQmDGoBJ3nAgEqCNUsNQVhSsyvEQyQR1ia1ignBnT95kbROa65Tc2/PqvWrIo4yOARH4AS4BzUwQ1ogCbA4BE8g1fwZj1ZL9a79TFrLVnFzD74A+vzB6XrlvQ=</latexit><latexit sha1_base64="FBMv2wyAHxkmlGRKqF3+j4ktAn4=">ACAnicbVDLSgMxFM3UV62vUVfiJlgEV2VGBN0IRTcuK9gHtEPJZDJtaCYZkx1GAY3/obF4q49Svc+Tem7Sy09cCFk3PuJfceP2ZUacf5tkpLyura+X1ysbm1vaOvbvXUiKRmDSxYEJ2fKQIo5w0NdWMdGJUOQz0vZH1xO/PSZSUcHvdBoTL0IDTkOKkTZS3z5I4SXsjQnO7nPYw4HQs9dD3rerTs2ZAi4StyBVUKDRt796gcBJRLjGDCnVdZ1YexmSmJG8kovUSRGeIQGpGsoRxFRXjY9IYfHRglgKQpruFU/T2RoUipNPJNZ4T0UM17E/E/r5vo8MLKI8TiefRQmDGoBJ3nAgEqCNUsNQVhSsyvEQyQR1ia1ignBnT95kbROa65Tc2/PqvWrIo4yOARH4AS4BzUwQ1ogCbA4BE8g1fwZj1ZL9a79TFrLVnFzD74A+vzB6XrlvQ=</latexit><latexit sha1_base64="FBMv2wyAHxkmlGRKqF3+j4ktAn4=">ACAnicbVDLSgMxFM3UV62vUVfiJlgEV2VGBN0IRTcuK9gHtEPJZDJtaCYZkx1GAY3/obF4q49Svc+Tem7Sy09cCFk3PuJfceP2ZUacf5tkpLyura+X1ysbm1vaOvbvXUiKRmDSxYEJ2fKQIo5w0NdWMdGJUOQz0vZH1xO/PSZSUcHvdBoTL0IDTkOKkTZS3z5I4SXsjQnO7nPYw4HQs9dD3rerTs2ZAi4StyBVUKDRt796gcBJRLjGDCnVdZ1YexmSmJG8kovUSRGeIQGpGsoRxFRXjY9IYfHRglgKQpruFU/T2RoUipNPJNZ4T0UM17E/E/r5vo8MLKI8TiefRQmDGoBJ3nAgEqCNUsNQVhSsyvEQyQR1ia1ignBnT95kbROa65Tc2/PqvWrIo4yOARH4AS4BzUwQ1ogCbA4BE8g1fwZj1ZL9a79TFrLVnFzD74A+vzB6XrlvQ=</latexit>

1

SLIDE 79

Logistic Regression

y x 1

y = 1 1 + e−(~

w·~ x)

<latexit sha1_base64="xjF7J23uMAifTYmCo9gPQZs6Uv0=">ACFXicbVDLSgMxFM3UV62vqks3wSJU1DIjgm6EohuXFewDOmPJpHdqaOZBkqmWYX7Cjb/ixoUibgV3/o1pOwtPXDh5Jx7yb3HjTiTyjS/jdzc/MLiUn65sLK6tr5R3NxqyDAWFOo05KFouUQCZwHUFVMcWpEA4rscm7/cuQ3ByAkC4MbNYzA8UkvYB6jRGmpUzwc4nNse4LQxEoT6wBuk6OyPQCa3KfYpt1Q4fHrId1P06xZFbMfAsTJSQhlqneKX3Q1p7EOgKCdSti0zUk5ChGKUQ1qwYwkRoX3Sg7amAfFBOsn4qhTvaWLvVDoChQeq78nEuJLOfRd3ekTdSenvZH4n9eOlXfmJCyIYgUBnXzkxRyrEI8iwl0mgCo+1IRQwfSumN4RHZHSQRZ0CNb0ybOkcVyxzIp1fVKqXmRx5NEO2kVlZKFTVEVXqIbqiKJH9Ixe0ZvxZLwY78bHpDVnZDPb6A+Mzx9o4Z5g</latexit><latexit sha1_base64="xjF7J23uMAifTYmCo9gPQZs6Uv0=">ACFXicbVDLSgMxFM3UV62vqks3wSJU1DIjgm6EohuXFewDOmPJpHdqaOZBkqmWYX7Cjb/ixoUibgV3/o1pOwtPXDh5Jx7yb3HjTiTyjS/jdzc/MLiUn65sLK6tr5R3NxqyDAWFOo05KFouUQCZwHUFVMcWpEA4rscm7/cuQ3ByAkC4MbNYzA8UkvYB6jRGmpUzwc4nNse4LQxEoT6wBuk6OyPQCa3KfYpt1Q4fHrId1P06xZFbMfAsTJSQhlqneKX3Q1p7EOgKCdSti0zUk5ChGKUQ1qwYwkRoX3Sg7amAfFBOsn4qhTvaWLvVDoChQeq78nEuJLOfRd3ekTdSenvZH4n9eOlXfmJCyIYgUBnXzkxRyrEI8iwl0mgCo+1IRQwfSumN4RHZHSQRZ0CNb0ybOkcVyxzIp1fVKqXmRx5NEO2kVlZKFTVEVXqIbqiKJH9Ixe0ZvxZLwY78bHpDVnZDPb6A+Mzx9o4Z5g</latexit><latexit sha1_base64="xjF7J23uMAifTYmCo9gPQZs6Uv0=">ACFXicbVDLSgMxFM3UV62vqks3wSJU1DIjgm6EohuXFewDOmPJpHdqaOZBkqmWYX7Cjb/ixoUibgV3/o1pOwtPXDh5Jx7yb3HjTiTyjS/jdzc/MLiUn65sLK6tr5R3NxqyDAWFOo05KFouUQCZwHUFVMcWpEA4rscm7/cuQ3ByAkC4MbNYzA8UkvYB6jRGmpUzwc4nNse4LQxEoT6wBuk6OyPQCa3KfYpt1Q4fHrId1P06xZFbMfAsTJSQhlqneKX3Q1p7EOgKCdSti0zUk5ChGKUQ1qwYwkRoX3Sg7amAfFBOsn4qhTvaWLvVDoChQeq78nEuJLOfRd3ekTdSenvZH4n9eOlXfmJCyIYgUBnXzkxRyrEI8iwl0mgCo+1IRQwfSumN4RHZHSQRZ0CNb0ybOkcVyxzIp1fVKqXmRx5NEO2kVlZKFTVEVXqIbqiKJH9Ixe0ZvxZLwY78bHpDVnZDPb6A+Mzx9o4Z5g</latexit><latexit sha1_base64="xjF7J23uMAifTYmCo9gPQZs6Uv0=">ACFXicbVDLSgMxFM3UV62vqks3wSJU1DIjgm6EohuXFewDOmPJpHdqaOZBkqmWYX7Cjb/ixoUibgV3/o1pOwtPXDh5Jx7yb3HjTiTyjS/jdzc/MLiUn65sLK6tr5R3NxqyDAWFOo05KFouUQCZwHUFVMcWpEA4rscm7/cuQ3ByAkC4MbNYzA8UkvYB6jRGmpUzwc4nNse4LQxEoT6wBuk6OyPQCa3KfYpt1Q4fHrId1P06xZFbMfAsTJSQhlqneKX3Q1p7EOgKCdSti0zUk5ChGKUQ1qwYwkRoX3Sg7amAfFBOsn4qhTvaWLvVDoChQeq78nEuJLOfRd3ekTdSenvZH4n9eOlXfmJCyIYgUBnXzkxRyrEI8iwl0mgCo+1IRQwfSumN4RHZHSQRZ0CNb0ybOkcVyxzIp1fVKqXmRx5NEO2kVlZKFTVEVXqIbqiKJH9Ixe0ZvxZLwY78bHpDVnZDPb6A+Mzx9o4Z5g</latexit>

SLIDE 80

Logistic Regression

x 1

y = 1 1 + e−(~

w·~ x)

<latexit sha1_base64="xjF7J23uMAifTYmCo9gPQZs6Uv0=">ACFXicbVDLSgMxFM3UV62vqks3wSJU1DIjgm6EohuXFewDOmPJpHdqaOZBkqmWYX7Cjb/ixoUibgV3/o1pOwtPXDh5Jx7yb3HjTiTyjS/jdzc/MLiUn65sLK6tr5R3NxqyDAWFOo05KFouUQCZwHUFVMcWpEA4rscm7/cuQ3ByAkC4MbNYzA8UkvYB6jRGmpUzwc4nNse4LQxEoT6wBuk6OyPQCa3KfYpt1Q4fHrId1P06xZFbMfAsTJSQhlqneKX3Q1p7EOgKCdSti0zUk5ChGKUQ1qwYwkRoX3Sg7amAfFBOsn4qhTvaWLvVDoChQeq78nEuJLOfRd3ekTdSenvZH4n9eOlXfmJCyIYgUBnXzkxRyrEI8iwl0mgCo+1IRQwfSumN4RHZHSQRZ0CNb0ybOkcVyxzIp1fVKqXmRx5NEO2kVlZKFTVEVXqIbqiKJH9Ixe0ZvxZLwY78bHpDVnZDPb6A+Mzx9o4Z5g</latexit><latexit sha1_base64="xjF7J23uMAifTYmCo9gPQZs6Uv0=">ACFXicbVDLSgMxFM3UV62vqks3wSJU1DIjgm6EohuXFewDOmPJpHdqaOZBkqmWYX7Cjb/ixoUibgV3/o1pOwtPXDh5Jx7yb3HjTiTyjS/jdzc/MLiUn65sLK6tr5R3NxqyDAWFOo05KFouUQCZwHUFVMcWpEA4rscm7/cuQ3ByAkC4MbNYzA8UkvYB6jRGmpUzwc4nNse4LQxEoT6wBuk6OyPQCa3KfYpt1Q4fHrId1P06xZFbMfAsTJSQhlqneKX3Q1p7EOgKCdSti0zUk5ChGKUQ1qwYwkRoX3Sg7amAfFBOsn4qhTvaWLvVDoChQeq78nEuJLOfRd3ekTdSenvZH4n9eOlXfmJCyIYgUBnXzkxRyrEI8iwl0mgCo+1IRQwfSumN4RHZHSQRZ0CNb0ybOkcVyxzIp1fVKqXmRx5NEO2kVlZKFTVEVXqIbqiKJH9Ixe0ZvxZLwY78bHpDVnZDPb6A+Mzx9o4Z5g</latexit><latexit sha1_base64="xjF7J23uMAifTYmCo9gPQZs6Uv0=">ACFXicbVDLSgMxFM3UV62vqks3wSJU1DIjgm6EohuXFewDOmPJpHdqaOZBkqmWYX7Cjb/ixoUibgV3/o1pOwtPXDh5Jx7yb3HjTiTyjS/jdzc/MLiUn65sLK6tr5R3NxqyDAWFOo05KFouUQCZwHUFVMcWpEA4rscm7/cuQ3ByAkC4MbNYzA8UkvYB6jRGmpUzwc4nNse4LQxEoT6wBuk6OyPQCa3KfYpt1Q4fHrId1P06xZFbMfAsTJSQhlqneKX3Q1p7EOgKCdSti0zUk5ChGKUQ1qwYwkRoX3Sg7amAfFBOsn4qhTvaWLvVDoChQeq78nEuJLOfRd3ekTdSenvZH4n9eOlXfmJCyIYgUBnXzkxRyrEI8iwl0mgCo+1IRQwfSumN4RHZHSQRZ0CNb0ybOkcVyxzIp1fVKqXmRx5NEO2kVlZKFTVEVXqIbqiKJH9Ixe0ZvxZLwY78bHpDVnZDPb6A+Mzx9o4Z5g</latexit><latexit sha1_base64="xjF7J23uMAifTYmCo9gPQZs6Uv0=">ACFXicbVDLSgMxFM3UV62vqks3wSJU1DIjgm6EohuXFewDOmPJpHdqaOZBkqmWYX7Cjb/ixoUibgV3/o1pOwtPXDh5Jx7yb3HjTiTyjS/jdzc/MLiUn65sLK6tr5R3NxqyDAWFOo05KFouUQCZwHUFVMcWpEA4rscm7/cuQ3ByAkC4MbNYzA8UkvYB6jRGmpUzwc4nNse4LQxEoT6wBuk6OyPQCa3KfYpt1Q4fHrId1P06xZFbMfAsTJSQhlqneKX3Q1p7EOgKCdSti0zUk5ChGKUQ1qwYwkRoX3Sg7amAfFBOsn4qhTvaWLvVDoChQeq78nEuJLOfRd3ekTdSenvZH4n9eOlXfmJCyIYgUBnXzkxRyrEI8iwl0mgCo+1IRQwfSumN4RHZHSQRZ0CNb0ybOkcVyxzIp1fVKqXmRx5NEO2kVlZKFTVEVXqIbqiKJH9Ixe0ZvxZLwY78bHpDVnZDPb6A+Mzx9o4Z5g</latexit>

P(Y|X)

SLIDE 81

Linear Regression

81

=

n

X

i=1

(Yi − ˆ Y )2

<latexit sha1_base64="r/S0kziZA4N57u3eiAn1RfFZ4w=">ACDHicbVDLSsNAFJ3UV62vqks3g0WoC0tSBN0Uim5ctmBfNGmYTCft0MkzEyEvIBbvwVNy4UcesHuPNvnLZaOuBgcM53LnHi9iVCrT/DZya+sbm1v57cLO7t7+QfHwqC3DWGDSwiELRdDkjDKSUtRxUg3EgQFHiMdb3I78zsPREga8ns1jYgToBGnPsVIacktlpqwBm0ZB25Ca1Y6SHha7rkUXkB7jFTS8HVZ0yK+YcJVYGSmBDA23+GUPQxwHhCvMkJR9y4yUkyChKGYkLdixJBHCEzQifU05Coh0kvkxKTzTyhD6odCPKzhXf08kKJByGng6GSA1lsveTPzP68fKv3YSyqNYEY4Xi/yYQRXCWTNwSAXBik01QVhQ/VeIx0grHR/BV2CtXzyKmlXK5ZsZqXpfpNVkcenIBTUAYWuAJ1cAcaoAUweATP4BW8GU/Gi/FufCyiOSObOQZ/YHz+ADvfmdM=</latexit><latexit sha1_base64="r/S0kziZA4N57u3eiAn1RfFZ4w=">ACDHicbVDLSsNAFJ3UV62vqks3g0WoC0tSBN0Uim5ctmBfNGmYTCft0MkzEyEvIBbvwVNy4UcesHuPNvnLZaOuBgcM53LnHi9iVCrT/DZya+sbm1v57cLO7t7+QfHwqC3DWGDSwiELRdDkjDKSUtRxUg3EgQFHiMdb3I78zsPREga8ns1jYgToBGnPsVIacktlpqwBm0ZB25Ca1Y6SHha7rkUXkB7jFTS8HVZ0yK+YcJVYGSmBDA23+GUPQxwHhCvMkJR9y4yUkyChKGYkLdixJBHCEzQifU05Coh0kvkxKTzTyhD6odCPKzhXf08kKJByGng6GSA1lsveTPzP68fKv3YSyqNYEY4Xi/yYQRXCWTNwSAXBik01QVhQ/VeIx0grHR/BV2CtXzyKmlXK5ZsZqXpfpNVkcenIBTUAYWuAJ1cAcaoAUweATP4BW8GU/Gi/FufCyiOSObOQZ/YHz+ADvfmdM=</latexit><latexit sha1_base64="r/S0kziZA4N57u3eiAn1RfFZ4w=">ACDHicbVDLSsNAFJ3UV62vqks3g0WoC0tSBN0Uim5ctmBfNGmYTCft0MkzEyEvIBbvwVNy4UcesHuPNvnLZaOuBgcM53LnHi9iVCrT/DZya+sbm1v57cLO7t7+QfHwqC3DWGDSwiELRdDkjDKSUtRxUg3EgQFHiMdb3I78zsPREga8ns1jYgToBGnPsVIacktlpqwBm0ZB25Ca1Y6SHha7rkUXkB7jFTS8HVZ0yK+YcJVYGSmBDA23+GUPQxwHhCvMkJR9y4yUkyChKGYkLdixJBHCEzQifU05Coh0kvkxKTzTyhD6odCPKzhXf08kKJByGng6GSA1lsveTPzP68fKv3YSyqNYEY4Xi/yYQRXCWTNwSAXBik01QVhQ/VeIx0grHR/BV2CtXzyKmlXK5ZsZqXpfpNVkcenIBTUAYWuAJ1cAcaoAUweATP4BW8GU/Gi/FufCyiOSObOQZ/YHz+ADvfmdM=</latexit><latexit sha1_base64="r/S0kziZA4N57u3eiAn1RfFZ4w=">ACDHicbVDLSsNAFJ3UV62vqks3g0WoC0tSBN0Uim5ctmBfNGmYTCft0MkzEyEvIBbvwVNy4UcesHuPNvnLZaOuBgcM53LnHi9iVCrT/DZya+sbm1v57cLO7t7+QfHwqC3DWGDSwiELRdDkjDKSUtRxUg3EgQFHiMdb3I78zsPREga8ns1jYgToBGnPsVIacktlpqwBm0ZB25Ca1Y6SHha7rkUXkB7jFTS8HVZ0yK+YcJVYGSmBDA23+GUPQxwHhCvMkJR9y4yUkyChKGYkLdixJBHCEzQifU05Coh0kvkxKTzTyhD6odCPKzhXf08kKJByGng6GSA1lsveTPzP68fKv3YSyqNYEY4Xi/yYQRXCWTNwSAXBik01QVhQ/VeIx0grHR/BV2CtXzyKmlXK5ZsZqXpfpNVkcenIBTUAYWuAJ1cAcaoAUweATP4BW8GU/Gi/FufCyiOSObOQZ/YHz+ADvfmdM=</latexit>

minimize

SLIDE 82

Logistic Regression

82

minimize−logP(Y | ˆ

Y )

<latexit sha1_base64="QJTVqFTqbLs0IDXvizP93NjwUpI=">AB+nicbVBNS8NAEN3Ur1q/Uj16WSxCPVgSEfRY9OKxgv2iDWz3bRLN5uwO1FK2p/ixYMiXv0l3vw3btsctPXBwO9GWbm+bHgGhzn28qtrW9sbuW3Czu7e/sHdvGwoaNEUVankYhUyeaCS5ZHTgI1oVI6EvWNMf3c785iNTmkfyAcYx80IykDzglICRenbxXEQDXCu3J90hgbQ9PevZJafizIFXiZuREspQ69lf3X5Ek5BJoIJo3XGdGLyUKOBUsGmhm2gWEzoiA9YxVJKQaS+dnz7Fp0bp4yBSpiTgufp7IiWh1uPQN50hgaFe9mbif14ngeDaS7mME2CSLhYFicAQ4VkOuM8VoyDGhCquLkV0yFRhIJq2BCcJdfXiWNi4rVNz7y1L1Josj47RCSojF12hKrpDNVRHFD2hZ/SK3qyJ9WK9Wx+L1pyVzRyhP7A+fwAjp5NE</latexit><latexit sha1_base64="QJTVqFTqbLs0IDXvizP93NjwUpI=">AB+nicbVBNS8NAEN3Ur1q/Uj16WSxCPVgSEfRY9OKxgv2iDWz3bRLN5uwO1FK2p/ixYMiXv0l3vw3btsctPXBwO9GWbm+bHgGhzn28qtrW9sbuW3Czu7e/sHdvGwoaNEUVankYhUyeaCS5ZHTgI1oVI6EvWNMf3c785iNTmkfyAcYx80IykDzglICRenbxXEQDXCu3J90hgbQ9PevZJafizIFXiZuREspQ69lf3X5Ek5BJoIJo3XGdGLyUKOBUsGmhm2gWEzoiA9YxVJKQaS+dnz7Fp0bp4yBSpiTgufp7IiWh1uPQN50hgaFe9mbif14ngeDaS7mME2CSLhYFicAQ4VkOuM8VoyDGhCquLkV0yFRhIJq2BCcJdfXiWNi4rVNz7y1L1Josj47RCSojF12hKrpDNVRHFD2hZ/SK3qyJ9WK9Wx+L1pyVzRyhP7A+fwAjp5NE</latexit><latexit sha1_base64="QJTVqFTqbLs0IDXvizP93NjwUpI=">AB+nicbVBNS8NAEN3Ur1q/Uj16WSxCPVgSEfRY9OKxgv2iDWz3bRLN5uwO1FK2p/ixYMiXv0l3vw3btsctPXBwO9GWbm+bHgGhzn28qtrW9sbuW3Czu7e/sHdvGwoaNEUVankYhUyeaCS5ZHTgI1oVI6EvWNMf3c785iNTmkfyAcYx80IykDzglICRenbxXEQDXCu3J90hgbQ9PevZJafizIFXiZuREspQ69lf3X5Ek5BJoIJo3XGdGLyUKOBUsGmhm2gWEzoiA9YxVJKQaS+dnz7Fp0bp4yBSpiTgufp7IiWh1uPQN50hgaFe9mbif14ngeDaS7mME2CSLhYFicAQ4VkOuM8VoyDGhCquLkV0yFRhIJq2BCcJdfXiWNi4rVNz7y1L1Josj47RCSojF12hKrpDNVRHFD2hZ/SK3qyJ9WK9Wx+L1pyVzRyhP7A+fwAjp5NE</latexit><latexit sha1_base64="QJTVqFTqbLs0IDXvizP93NjwUpI=">AB+nicbVBNS8NAEN3Ur1q/Uj16WSxCPVgSEfRY9OKxgv2iDWz3bRLN5uwO1FK2p/ixYMiXv0l3vw3btsctPXBwO9GWbm+bHgGhzn28qtrW9sbuW3Czu7e/sHdvGwoaNEUVankYhUyeaCS5ZHTgI1oVI6EvWNMf3c785iNTmkfyAcYx80IykDzglICRenbxXEQDXCu3J90hgbQ9PevZJafizIFXiZuREspQ69lf3X5Ek5BJoIJo3XGdGLyUKOBUsGmhm2gWEzoiA9YxVJKQaS+dnz7Fp0bp4yBSpiTgufp7IiWh1uPQN50hgaFe9mbif14ngeDaS7mME2CSLhYFicAQ4VkOuM8VoyDGhCquLkV0yFRhIJq2BCcJdfXiWNi4rVNz7y1L1Josj47RCSojF12hKrpDNVRHFD2hZ/SK3qyJ9WK9Wx+L1pyVzRyhP7A+fwAjp5NE</latexit>

SLIDE 83

Logistic Regression

83

minimize −Y log ˆ Y + (1 − Y )log(1 − ˆ Y )

<latexit sha1_base64="M/NyrZTFl/wZ6128vLqkIxJ8urs=">ACFHicbZDLSsNAFIZPvNZ6i7p0M1iElmJRNBl0Y3LCvZG8pkOm2HTjJhZiKU0Idw46u4caGIWxfufBunaRbaemDg4/P4cz5/YgzpR3n21pZXVvf2Mxt5bd3dvf27YPDhKxJLROBey5WNFOQtpXTPNaSuSFAc+p01/fDPzmw9UKibCez2JqBfgYcgGjGBtpJ5dPkNtxMUQdUdYJ+0pKqOi4xYStWUM6vUswtOxUkLYObQGyqvXsr25fkDigoSYcK9VxnUh7CZaEU6n+W6saITJGA9px2CIA6q8JD1qik6N0kcDIc0LNUrV3xMJDpSaBL7pDLAeqUVvJv7ndWI9uPISFkaxpiGZLxrEHGmBZgmhPpOUaD4xgIlk5q+IjLDERJsc8yYEd/HkZWicV1yn4t5dFKrXWRw5OIYTKILl1CFW6hBHQg8wjO8wpv1ZL1Y79bHvHXFymaO4E9Znz8c+Zp5</latexit><latexit sha1_base64="M/NyrZTFl/wZ6128vLqkIxJ8urs=">ACFHicbZDLSsNAFIZPvNZ6i7p0M1iElmJRNBl0Y3LCvZG8pkOm2HTjJhZiKU0Idw46u4caGIWxfufBunaRbaemDg4/P4cz5/YgzpR3n21pZXVvf2Mxt5bd3dvf27YPDhKxJLROBey5WNFOQtpXTPNaSuSFAc+p01/fDPzmw9UKibCez2JqBfgYcgGjGBtpJ5dPkNtxMUQdUdYJ+0pKqOi4xYStWUM6vUswtOxUkLYObQGyqvXsr25fkDigoSYcK9VxnUh7CZaEU6n+W6saITJGA9px2CIA6q8JD1qik6N0kcDIc0LNUrV3xMJDpSaBL7pDLAeqUVvJv7ndWI9uPISFkaxpiGZLxrEHGmBZgmhPpOUaD4xgIlk5q+IjLDERJsc8yYEd/HkZWicV1yn4t5dFKrXWRw5OIYTKILl1CFW6hBHQg8wjO8wpv1ZL1Y79bHvHXFymaO4E9Znz8c+Zp5</latexit><latexit sha1_base64="M/NyrZTFl/wZ6128vLqkIxJ8urs=">ACFHicbZDLSsNAFIZPvNZ6i7p0M1iElmJRNBl0Y3LCvZG8pkOm2HTjJhZiKU0Idw46u4caGIWxfufBunaRbaemDg4/P4cz5/YgzpR3n21pZXVvf2Mxt5bd3dvf27YPDhKxJLROBey5WNFOQtpXTPNaSuSFAc+p01/fDPzmw9UKibCez2JqBfgYcgGjGBtpJ5dPkNtxMUQdUdYJ+0pKqOi4xYStWUM6vUswtOxUkLYObQGyqvXsr25fkDigoSYcK9VxnUh7CZaEU6n+W6saITJGA9px2CIA6q8JD1qik6N0kcDIc0LNUrV3xMJDpSaBL7pDLAeqUVvJv7ndWI9uPISFkaxpiGZLxrEHGmBZgmhPpOUaD4xgIlk5q+IjLDERJsc8yYEd/HkZWicV1yn4t5dFKrXWRw5OIYTKILl1CFW6hBHQg8wjO8wpv1ZL1Y79bHvHXFymaO4E9Znz8c+Zp5</latexit><latexit sha1_base64="M/NyrZTFl/wZ6128vLqkIxJ8urs=">ACFHicbZDLSsNAFIZPvNZ6i7p0M1iElmJRNBl0Y3LCvZG8pkOm2HTjJhZiKU0Idw46u4caGIWxfufBunaRbaemDg4/P4cz5/YgzpR3n21pZXVvf2Mxt5bd3dvf27YPDhKxJLROBey5WNFOQtpXTPNaSuSFAc+p01/fDPzmw9UKibCez2JqBfgYcgGjGBtpJ5dPkNtxMUQdUdYJ+0pKqOi4xYStWUM6vUswtOxUkLYObQGyqvXsr25fkDigoSYcK9VxnUh7CZaEU6n+W6saITJGA9px2CIA6q8JD1qik6N0kcDIc0LNUrV3xMJDpSaBL7pDLAeqUVvJv7ndWI9uPISFkaxpiGZLxrEHGmBZgmhPpOUaD4xgIlk5q+IjLDERJsc8yYEd/HkZWicV1yn4t5dFKrXWRw5OIYTKILl1CFW6hBHQg8wjO8wpv1ZL1Y79bHvHXFymaO4E9Znz8c+Zp5</latexit>

SLIDE 84

84

minimize −Y log ˆ Y + (1 − Y )log(1 − ˆ Y )

<latexit sha1_base64="M/NyrZTFl/wZ6128vLqkIxJ8urs=">ACFHicbZDLSsNAFIZPvNZ6i7p0M1iElmJRNBl0Y3LCvZG8pkOm2HTjJhZiKU0Idw46u4caGIWxfufBunaRbaemDg4/P4cz5/YgzpR3n21pZXVvf2Mxt5bd3dvf27YPDhKxJLROBey5WNFOQtpXTPNaSuSFAc+p01/fDPzmw9UKibCez2JqBfgYcgGjGBtpJ5dPkNtxMUQdUdYJ+0pKqOi4xYStWUM6vUswtOxUkLYObQGyqvXsr25fkDigoSYcK9VxnUh7CZaEU6n+W6saITJGA9px2CIA6q8JD1qik6N0kcDIc0LNUrV3xMJDpSaBL7pDLAeqUVvJv7ndWI9uPISFkaxpiGZLxrEHGmBZgmhPpOUaD4xgIlk5q+IjLDERJsc8yYEd/HkZWicV1yn4t5dFKrXWRw5OIYTKILl1CFW6hBHQg8wjO8wpv1ZL1Y79bHvHXFymaO4E9Znz8c+Zp5</latexit><latexit sha1_base64="M/NyrZTFl/wZ6128vLqkIxJ8urs=">ACFHicbZDLSsNAFIZPvNZ6i7p0M1iElmJRNBl0Y3LCvZG8pkOm2HTjJhZiKU0Idw46u4caGIWxfufBunaRbaemDg4/P4cz5/YgzpR3n21pZXVvf2Mxt5bd3dvf27YPDhKxJLROBey5WNFOQtpXTPNaSuSFAc+p01/fDPzmw9UKibCez2JqBfgYcgGjGBtpJ5dPkNtxMUQdUdYJ+0pKqOi4xYStWUM6vUswtOxUkLYObQGyqvXsr25fkDigoSYcK9VxnUh7CZaEU6n+W6saITJGA9px2CIA6q8JD1qik6N0kcDIc0LNUrV3xMJDpSaBL7pDLAeqUVvJv7ndWI9uPISFkaxpiGZLxrEHGmBZgmhPpOUaD4xgIlk5q+IjLDERJsc8yYEd/HkZWicV1yn4t5dFKrXWRw5OIYTKILl1CFW6hBHQg8wjO8wpv1ZL1Y79bHvHXFymaO4E9Znz8c+Zp5</latexit><latexit sha1_base64="M/NyrZTFl/wZ6128vLqkIxJ8urs=">ACFHicbZDLSsNAFIZPvNZ6i7p0M1iElmJRNBl0Y3LCvZG8pkOm2HTjJhZiKU0Idw46u4caGIWxfufBunaRbaemDg4/P4cz5/YgzpR3n21pZXVvf2Mxt5bd3dvf27YPDhKxJLROBey5WNFOQtpXTPNaSuSFAc+p01/fDPzmw9UKibCez2JqBfgYcgGjGBtpJ5dPkNtxMUQdUdYJ+0pKqOi4xYStWUM6vUswtOxUkLYObQGyqvXsr25fkDigoSYcK9VxnUh7CZaEU6n+W6saITJGA9px2CIA6q8JD1qik6N0kcDIc0LNUrV3xMJDpSaBL7pDLAeqUVvJv7ndWI9uPISFkaxpiGZLxrEHGmBZgmhPpOUaD4xgIlk5q+IjLDERJsc8yYEd/HkZWicV1yn4t5dFKrXWRw5OIYTKILl1CFW6hBHQg8wjO8wpv1ZL1Y79bHvHXFymaO4E9Znz8c+Zp5</latexit><latexit sha1_base64="M/NyrZTFl/wZ6128vLqkIxJ8urs=">ACFHicbZDLSsNAFIZPvNZ6i7p0M1iElmJRNBl0Y3LCvZG8pkOm2HTjJhZiKU0Idw46u4caGIWxfufBunaRbaemDg4/P4cz5/YgzpR3n21pZXVvf2Mxt5bd3dvf27YPDhKxJLROBey5WNFOQtpXTPNaSuSFAc+p01/fDPzmw9UKibCez2JqBfgYcgGjGBtpJ5dPkNtxMUQdUdYJ+0pKqOi4xYStWUM6vUswtOxUkLYObQGyqvXsr25fkDigoSYcK9VxnUh7CZaEU6n+W6saITJGA9px2CIA6q8JD1qik6N0kcDIc0LNUrV3xMJDpSaBL7pDLAeqUVvJv7ndWI9uPISFkaxpiGZLxrEHGmBZgmhPpOUaD4xgIlk5q+IjLDERJsc8yYEd/HkZWicV1yn4t5dFKrXWRw5OIYTKILl1CFW6hBHQg8wjO8wpv1ZL1Y79bHvHXFymaO4E9Znz8c+Zp5</latexit>

Logistic Regression

SLIDE 85

85

minimize −Y log ˆ Y + (1 − Y )log(1 − ˆ Y )

<latexit sha1_base64="M/NyrZTFl/wZ6128vLqkIxJ8urs=">ACFHicbZDLSsNAFIZPvNZ6i7p0M1iElmJRNBl0Y3LCvZG8pkOm2HTjJhZiKU0Idw46u4caGIWxfufBunaRbaemDg4/P4cz5/YgzpR3n21pZXVvf2Mxt5bd3dvf27YPDhKxJLROBey5WNFOQtpXTPNaSuSFAc+p01/fDPzmw9UKibCez2JqBfgYcgGjGBtpJ5dPkNtxMUQdUdYJ+0pKqOi4xYStWUM6vUswtOxUkLYObQGyqvXsr25fkDigoSYcK9VxnUh7CZaEU6n+W6saITJGA9px2CIA6q8JD1qik6N0kcDIc0LNUrV3xMJDpSaBL7pDLAeqUVvJv7ndWI9uPISFkaxpiGZLxrEHGmBZgmhPpOUaD4xgIlk5q+IjLDERJsc8yYEd/HkZWicV1yn4t5dFKrXWRw5OIYTKILl1CFW6hBHQg8wjO8wpv1ZL1Y79bHvHXFymaO4E9Znz8c+Zp5</latexit><latexit sha1_base64="M/NyrZTFl/wZ6128vLqkIxJ8urs=">ACFHicbZDLSsNAFIZPvNZ6i7p0M1iElmJRNBl0Y3LCvZG8pkOm2HTjJhZiKU0Idw46u4caGIWxfufBunaRbaemDg4/P4cz5/YgzpR3n21pZXVvf2Mxt5bd3dvf27YPDhKxJLROBey5WNFOQtpXTPNaSuSFAc+p01/fDPzmw9UKibCez2JqBfgYcgGjGBtpJ5dPkNtxMUQdUdYJ+0pKqOi4xYStWUM6vUswtOxUkLYObQGyqvXsr25fkDigoSYcK9VxnUh7CZaEU6n+W6saITJGA9px2CIA6q8JD1qik6N0kcDIc0LNUrV3xMJDpSaBL7pDLAeqUVvJv7ndWI9uPISFkaxpiGZLxrEHGmBZgmhPpOUaD4xgIlk5q+IjLDERJsc8yYEd/HkZWicV1yn4t5dFKrXWRw5OIYTKILl1CFW6hBHQg8wjO8wpv1ZL1Y79bHvHXFymaO4E9Znz8c+Zp5</latexit><latexit sha1_base64="M/NyrZTFl/wZ6128vLqkIxJ8urs=">ACFHicbZDLSsNAFIZPvNZ6i7p0M1iElmJRNBl0Y3LCvZG8pkOm2HTjJhZiKU0Idw46u4caGIWxfufBunaRbaemDg4/P4cz5/YgzpR3n21pZXVvf2Mxt5bd3dvf27YPDhKxJLROBey5WNFOQtpXTPNaSuSFAc+p01/fDPzmw9UKibCez2JqBfgYcgGjGBtpJ5dPkNtxMUQdUdYJ+0pKqOi4xYStWUM6vUswtOxUkLYObQGyqvXsr25fkDigoSYcK9VxnUh7CZaEU6n+W6saITJGA9px2CIA6q8JD1qik6N0kcDIc0LNUrV3xMJDpSaBL7pDLAeqUVvJv7ndWI9uPISFkaxpiGZLxrEHGmBZgmhPpOUaD4xgIlk5q+IjLDERJsc8yYEd/HkZWicV1yn4t5dFKrXWRw5OIYTKILl1CFW6hBHQg8wjO8wpv1ZL1Y79bHvHXFymaO4E9Znz8c+Zp5</latexit><latexit sha1_base64="M/NyrZTFl/wZ6128vLqkIxJ8urs=">ACFHicbZDLSsNAFIZPvNZ6i7p0M1iElmJRNBl0Y3LCvZG8pkOm2HTjJhZiKU0Idw46u4caGIWxfufBunaRbaemDg4/P4cz5/YgzpR3n21pZXVvf2Mxt5bd3dvf27YPDhKxJLROBey5WNFOQtpXTPNaSuSFAc+p01/fDPzmw9UKibCez2JqBfgYcgGjGBtpJ5dPkNtxMUQdUdYJ+0pKqOi4xYStWUM6vUswtOxUkLYObQGyqvXsr25fkDigoSYcK9VxnUh7CZaEU6n+W6saITJGA9px2CIA6q8JD1qik6N0kcDIc0LNUrV3xMJDpSaBL7pDLAeqUVvJv7ndWI9uPISFkaxpiGZLxrEHGmBZgmhPpOUaD4xgIlk5q+IjLDERJsc8yYEd/HkZWicV1yn4t5dFKrXWRw5OIYTKILl1CFW6hBHQg8wjO8wpv1ZL1Y79bHvHXFymaO4E9Znz8c+Zp5</latexit>

Logistic Regression

SLIDE 86

86

minimize −Y log ˆ Y + (1 − Y )log(1 − ˆ Y )

<latexit sha1_base64="M/NyrZTFl/wZ6128vLqkIxJ8urs=">ACFHicbZDLSsNAFIZPvNZ6i7p0M1iElmJRNBl0Y3LCvZG8pkOm2HTjJhZiKU0Idw46u4caGIWxfufBunaRbaemDg4/P4cz5/YgzpR3n21pZXVvf2Mxt5bd3dvf27YPDhKxJLROBey5WNFOQtpXTPNaSuSFAc+p01/fDPzmw9UKibCez2JqBfgYcgGjGBtpJ5dPkNtxMUQdUdYJ+0pKqOi4xYStWUM6vUswtOxUkLYObQGyqvXsr25fkDigoSYcK9VxnUh7CZaEU6n+W6saITJGA9px2CIA6q8JD1qik6N0kcDIc0LNUrV3xMJDpSaBL7pDLAeqUVvJv7ndWI9uPISFkaxpiGZLxrEHGmBZgmhPpOUaD4xgIlk5q+IjLDERJsc8yYEd/HkZWicV1yn4t5dFKrXWRw5OIYTKILl1CFW6hBHQg8wjO8wpv1ZL1Y79bHvHXFymaO4E9Znz8c+Zp5</latexit><latexit sha1_base64="M/NyrZTFl/wZ6128vLqkIxJ8urs=">ACFHicbZDLSsNAFIZPvNZ6i7p0M1iElmJRNBl0Y3LCvZG8pkOm2HTjJhZiKU0Idw46u4caGIWxfufBunaRbaemDg4/P4cz5/YgzpR3n21pZXVvf2Mxt5bd3dvf27YPDhKxJLROBey5WNFOQtpXTPNaSuSFAc+p01/fDPzmw9UKibCez2JqBfgYcgGjGBtpJ5dPkNtxMUQdUdYJ+0pKqOi4xYStWUM6vUswtOxUkLYObQGyqvXsr25fkDigoSYcK9VxnUh7CZaEU6n+W6saITJGA9px2CIA6q8JD1qik6N0kcDIc0LNUrV3xMJDpSaBL7pDLAeqUVvJv7ndWI9uPISFkaxpiGZLxrEHGmBZgmhPpOUaD4xgIlk5q+IjLDERJsc8yYEd/HkZWicV1yn4t5dFKrXWRw5OIYTKILl1CFW6hBHQg8wjO8wpv1ZL1Y79bHvHXFymaO4E9Znz8c+Zp5</latexit><latexit sha1_base64="M/NyrZTFl/wZ6128vLqkIxJ8urs=">ACFHicbZDLSsNAFIZPvNZ6i7p0M1iElmJRNBl0Y3LCvZG8pkOm2HTjJhZiKU0Idw46u4caGIWxfufBunaRbaemDg4/P4cz5/YgzpR3n21pZXVvf2Mxt5bd3dvf27YPDhKxJLROBey5WNFOQtpXTPNaSuSFAc+p01/fDPzmw9UKibCez2JqBfgYcgGjGBtpJ5dPkNtxMUQdUdYJ+0pKqOi4xYStWUM6vUswtOxUkLYObQGyqvXsr25fkDigoSYcK9VxnUh7CZaEU6n+W6saITJGA9px2CIA6q8JD1qik6N0kcDIc0LNUrV3xMJDpSaBL7pDLAeqUVvJv7ndWI9uPISFkaxpiGZLxrEHGmBZgmhPpOUaD4xgIlk5q+IjLDERJsc8yYEd/HkZWicV1yn4t5dFKrXWRw5OIYTKILl1CFW6hBHQg8wjO8wpv1ZL1Y79bHvHXFymaO4E9Znz8c+Zp5</latexit><latexit sha1_base64="M/NyrZTFl/wZ6128vLqkIxJ8urs=">ACFHicbZDLSsNAFIZPvNZ6i7p0M1iElmJRNBl0Y3LCvZG8pkOm2HTjJhZiKU0Idw46u4caGIWxfufBunaRbaemDg4/P4cz5/YgzpR3n21pZXVvf2Mxt5bd3dvf27YPDhKxJLROBey5WNFOQtpXTPNaSuSFAc+p01/fDPzmw9UKibCez2JqBfgYcgGjGBtpJ5dPkNtxMUQdUdYJ+0pKqOi4xYStWUM6vUswtOxUkLYObQGyqvXsr25fkDigoSYcK9VxnUh7CZaEU6n+W6saITJGA9px2CIA6q8JD1qik6N0kcDIc0LNUrV3xMJDpSaBL7pDLAeqUVvJv7ndWI9uPISFkaxpiGZLxrEHGmBZgmhPpOUaD4xgIlk5q+IjLDERJsc8yYEd/HkZWicV1yn4t5dFKrXWRw5OIYTKILl1CFW6hBHQg8wjO8wpv1ZL1Y79bHvHXFymaO4E9Znz8c+Zp5</latexit>

∂loss ∂w

<latexit sha1_base64="m0Pz6WxCBdaUNP7zyXpySngOXOM=">ACEHicbVDLSsNAFJ34rPUVdelmsIiuSiKCLotuXFawD2hKmUwn7dDJMzcqCXkE9z4K25cKOLWpTv/xkbUFsPXDic+/MvcePBdfgOF/WwuLS8spqa28vrG5tW3v7DZ1lCjKGjQSkWr7RDPBJWsAB8HasWIk9AVr+aPL3G/dMqV5JG9gHLNuSAaSB5wSMFLPvICRWjqxUQBJwJ7wO4hFZHWfaj3mU9u+JUnQnwPHELUkEF6j370+tHNAmZBCqI1h3XiaGb5g9SwbKyl2gWEzoiA9YxVJKQ6W46OSjDh0bp4yBSpiTgifp7IiWh1uPQN50hgaGe9XLxP6+TQHDeTbmME2CSTj8KEoEhwnk6uM8VoyDGhCquNkV0yExCYHJsGxCcGdPnifNk6rVN3r0rtoijhPbRATpGLjpDNXSF6qiBKHpAT+gFvVqP1rP1Zr1PWxesYmYP/YH18Q2rtp48</latexit><latexit sha1_base64="m0Pz6WxCBdaUNP7zyXpySngOXOM=">ACEHicbVDLSsNAFJ34rPUVdelmsIiuSiKCLotuXFawD2hKmUwn7dDJMzcqCXkE9z4K25cKOLWpTv/xkbUFsPXDic+/MvcePBdfgOF/WwuLS8spqa28vrG5tW3v7DZ1lCjKGjQSkWr7RDPBJWsAB8HasWIk9AVr+aPL3G/dMqV5JG9gHLNuSAaSB5wSMFLPvICRWjqxUQBJwJ7wO4hFZHWfaj3mU9u+JUnQnwPHELUkEF6j370+tHNAmZBCqI1h3XiaGb5g9SwbKyl2gWEzoiA9YxVJKQ6W46OSjDh0bp4yBSpiTgifp7IiWh1uPQN50hgaGe9XLxP6+TQHDeTbmME2CSTj8KEoEhwnk6uM8VoyDGhCquNkV0yExCYHJsGxCcGdPnifNk6rVN3r0rtoijhPbRATpGLjpDNXSF6qiBKHpAT+gFvVqP1rP1Zr1PWxesYmYP/YH18Q2rtp48</latexit><latexit sha1_base64="m0Pz6WxCBdaUNP7zyXpySngOXOM=">ACEHicbVDLSsNAFJ34rPUVdelmsIiuSiKCLotuXFawD2hKmUwn7dDJMzcqCXkE9z4K25cKOLWpTv/xkbUFsPXDic+/MvcePBdfgOF/WwuLS8spqa28vrG5tW3v7DZ1lCjKGjQSkWr7RDPBJWsAB8HasWIk9AVr+aPL3G/dMqV5JG9gHLNuSAaSB5wSMFLPvICRWjqxUQBJwJ7wO4hFZHWfaj3mU9u+JUnQnwPHELUkEF6j370+tHNAmZBCqI1h3XiaGb5g9SwbKyl2gWEzoiA9YxVJKQ6W46OSjDh0bp4yBSpiTgifp7IiWh1uPQN50hgaGe9XLxP6+TQHDeTbmME2CSTj8KEoEhwnk6uM8VoyDGhCquNkV0yExCYHJsGxCcGdPnifNk6rVN3r0rtoijhPbRATpGLjpDNXSF6qiBKHpAT+gFvVqP1rP1Zr1PWxesYmYP/YH18Q2rtp48</latexit><latexit sha1_base64="m0Pz6WxCBdaUNP7zyXpySngOXOM=">ACEHicbVDLSsNAFJ34rPUVdelmsIiuSiKCLotuXFawD2hKmUwn7dDJMzcqCXkE9z4K25cKOLWpTv/xkbUFsPXDic+/MvcePBdfgOF/WwuLS8spqa28vrG5tW3v7DZ1lCjKGjQSkWr7RDPBJWsAB8HasWIk9AVr+aPL3G/dMqV5JG9gHLNuSAaSB5wSMFLPvICRWjqxUQBJwJ7wO4hFZHWfaj3mU9u+JUnQnwPHELUkEF6j370+tHNAmZBCqI1h3XiaGb5g9SwbKyl2gWEzoiA9YxVJKQ6W46OSjDh0bp4yBSpiTgifp7IiWh1uPQN50hgaGe9XLxP6+TQHDeTbmME2CSTj8KEoEhwnk6uM8VoyDGhCquNkV0yExCYHJsGxCcGdPnifNk6rVN3r0rtoijhPbRATpGLjpDNXSF6qiBKHpAT+gFvVqP1rP1Zr1PWxesYmYP/YH18Q2rtp48</latexit>

Logistic Regression

SLIDE 87

87

∂loss ∂w

<latexit sha1_base64="m0Pz6WxCBdaUNP7zyXpySngOXOM=">ACEHicbVDLSsNAFJ34rPUVdelmsIiuSiKCLotuXFawD2hKmUwn7dDJMzcqCXkE9z4K25cKOLWpTv/xkbUFsPXDic+/MvcePBdfgOF/WwuLS8spqa28vrG5tW3v7DZ1lCjKGjQSkWr7RDPBJWsAB8HasWIk9AVr+aPL3G/dMqV5JG9gHLNuSAaSB5wSMFLPvICRWjqxUQBJwJ7wO4hFZHWfaj3mU9u+JUnQnwPHELUkEF6j370+tHNAmZBCqI1h3XiaGb5g9SwbKyl2gWEzoiA9YxVJKQ6W46OSjDh0bp4yBSpiTgifp7IiWh1uPQN50hgaGe9XLxP6+TQHDeTbmME2CSTj8KEoEhwnk6uM8VoyDGhCquNkV0yExCYHJsGxCcGdPnifNk6rVN3r0rtoijhPbRATpGLjpDNXSF6qiBKHpAT+gFvVqP1rP1Zr1PWxesYmYP/YH18Q2rtp48</latexit><latexit sha1_base64="m0Pz6WxCBdaUNP7zyXpySngOXOM=">ACEHicbVDLSsNAFJ34rPUVdelmsIiuSiKCLotuXFawD2hKmUwn7dDJMzcqCXkE9z4K25cKOLWpTv/xkbUFsPXDic+/MvcePBdfgOF/WwuLS8spqa28vrG5tW3v7DZ1lCjKGjQSkWr7RDPBJWsAB8HasWIk9AVr+aPL3G/dMqV5JG9gHLNuSAaSB5wSMFLPvICRWjqxUQBJwJ7wO4hFZHWfaj3mU9u+JUnQnwPHELUkEF6j370+tHNAmZBCqI1h3XiaGb5g9SwbKyl2gWEzoiA9YxVJKQ6W46OSjDh0bp4yBSpiTgifp7IiWh1uPQN50hgaGe9XLxP6+TQHDeTbmME2CSTj8KEoEhwnk6uM8VoyDGhCquNkV0yExCYHJsGxCcGdPnifNk6rVN3r0rtoijhPbRATpGLjpDNXSF6qiBKHpAT+gFvVqP1rP1Zr1PWxesYmYP/YH18Q2rtp48</latexit><latexit sha1_base64="m0Pz6WxCBdaUNP7zyXpySngOXOM=">ACEHicbVDLSsNAFJ34rPUVdelmsIiuSiKCLotuXFawD2hKmUwn7dDJMzcqCXkE9z4K25cKOLWpTv/xkbUFsPXDic+/MvcePBdfgOF/WwuLS8spqa28vrG5tW3v7DZ1lCjKGjQSkWr7RDPBJWsAB8HasWIk9AVr+aPL3G/dMqV5JG9gHLNuSAaSB5wSMFLPvICRWjqxUQBJwJ7wO4hFZHWfaj3mU9u+JUnQnwPHELUkEF6j370+tHNAmZBCqI1h3XiaGb5g9SwbKyl2gWEzoiA9YxVJKQ6W46OSjDh0bp4yBSpiTgifp7IiWh1uPQN50hgaGe9XLxP6+TQHDeTbmME2CSTj8KEoEhwnk6uM8VoyDGhCquNkV0yExCYHJsGxCcGdPnifNk6rVN3r0rtoijhPbRATpGLjpDNXSF6qiBKHpAT+gFvVqP1rP1Zr1PWxesYmYP/YH18Q2rtp48</latexit><latexit sha1_base64="m0Pz6WxCBdaUNP7zyXpySngOXOM=">ACEHicbVDLSsNAFJ34rPUVdelmsIiuSiKCLotuXFawD2hKmUwn7dDJMzcqCXkE9z4K25cKOLWpTv/xkbUFsPXDic+/MvcePBdfgOF/WwuLS8spqa28vrG5tW3v7DZ1lCjKGjQSkWr7RDPBJWsAB8HasWIk9AVr+aPL3G/dMqV5JG9gHLNuSAaSB5wSMFLPvICRWjqxUQBJwJ7wO4hFZHWfaj3mU9u+JUnQnwPHELUkEF6j370+tHNAmZBCqI1h3XiaGb5g9SwbKyl2gWEzoiA9YxVJKQ6W46OSjDh0bp4yBSpiTgifp7IiWh1uPQN50hgaGe9XLxP6+TQHDeTbmME2CSTj8KEoEhwnk6uM8VoyDGhCquNkV0yExCYHJsGxCcGdPnifNk6rVN3r0rtoijhPbRATpGLjpDNXSF6qiBKHpAT+gFvVqP1rP1Zr1PWxesYmYP/YH18Q2rtp48</latexit>

minimize −Y log ˆ Y + (1 − Y )log(1 − ˆ Y )

<latexit sha1_base64="M/NyrZTFl/wZ6128vLqkIxJ8urs=">ACFHicbZDLSsNAFIZPvNZ6i7p0M1iElmJRNBl0Y3LCvZG8pkOm2HTjJhZiKU0Idw46u4caGIWxfufBunaRbaemDg4/P4cz5/YgzpR3n21pZXVvf2Mxt5bd3dvf27YPDhKxJLROBey5WNFOQtpXTPNaSuSFAc+p01/fDPzmw9UKibCez2JqBfgYcgGjGBtpJ5dPkNtxMUQdUdYJ+0pKqOi4xYStWUM6vUswtOxUkLYObQGyqvXsr25fkDigoSYcK9VxnUh7CZaEU6n+W6saITJGA9px2CIA6q8JD1qik6N0kcDIc0LNUrV3xMJDpSaBL7pDLAeqUVvJv7ndWI9uPISFkaxpiGZLxrEHGmBZgmhPpOUaD4xgIlk5q+IjLDERJsc8yYEd/HkZWicV1yn4t5dFKrXWRw5OIYTKILl1CFW6hBHQg8wjO8wpv1ZL1Y79bHvHXFymaO4E9Znz8c+Zp5</latexit><latexit sha1_base64="M/NyrZTFl/wZ6128vLqkIxJ8urs=">ACFHicbZDLSsNAFIZPvNZ6i7p0M1iElmJRNBl0Y3LCvZG8pkOm2HTjJhZiKU0Idw46u4caGIWxfufBunaRbaemDg4/P4cz5/YgzpR3n21pZXVvf2Mxt5bd3dvf27YPDhKxJLROBey5WNFOQtpXTPNaSuSFAc+p01/fDPzmw9UKibCez2JqBfgYcgGjGBtpJ5dPkNtxMUQdUdYJ+0pKqOi4xYStWUM6vUswtOxUkLYObQGyqvXsr25fkDigoSYcK9VxnUh7CZaEU6n+W6saITJGA9px2CIA6q8JD1qik6N0kcDIc0LNUrV3xMJDpSaBL7pDLAeqUVvJv7ndWI9uPISFkaxpiGZLxrEHGmBZgmhPpOUaD4xgIlk5q+IjLDERJsc8yYEd/HkZWicV1yn4t5dFKrXWRw5OIYTKILl1CFW6hBHQg8wjO8wpv1ZL1Y79bHvHXFymaO4E9Znz8c+Zp5</latexit><latexit sha1_base64="M/NyrZTFl/wZ6128vLqkIxJ8urs=">ACFHicbZDLSsNAFIZPvNZ6i7p0M1iElmJRNBl0Y3LCvZG8pkOm2HTjJhZiKU0Idw46u4caGIWxfufBunaRbaemDg4/P4cz5/YgzpR3n21pZXVvf2Mxt5bd3dvf27YPDhKxJLROBey5WNFOQtpXTPNaSuSFAc+p01/fDPzmw9UKibCez2JqBfgYcgGjGBtpJ5dPkNtxMUQdUdYJ+0pKqOi4xYStWUM6vUswtOxUkLYObQGyqvXsr25fkDigoSYcK9VxnUh7CZaEU6n+W6saITJGA9px2CIA6q8JD1qik6N0kcDIc0LNUrV3xMJDpSaBL7pDLAeqUVvJv7ndWI9uPISFkaxpiGZLxrEHGmBZgmhPpOUaD4xgIlk5q+IjLDERJsc8yYEd/HkZWicV1yn4t5dFKrXWRw5OIYTKILl1CFW6hBHQg8wjO8wpv1ZL1Y79bHvHXFymaO4E9Znz8c+Zp5</latexit><latexit sha1_base64="M/NyrZTFl/wZ6128vLqkIxJ8urs=">ACFHicbZDLSsNAFIZPvNZ6i7p0M1iElmJRNBl0Y3LCvZG8pkOm2HTjJhZiKU0Idw46u4caGIWxfufBunaRbaemDg4/P4cz5/YgzpR3n21pZXVvf2Mxt5bd3dvf27YPDhKxJLROBey5WNFOQtpXTPNaSuSFAc+p01/fDPzmw9UKibCez2JqBfgYcgGjGBtpJ5dPkNtxMUQdUdYJ+0pKqOi4xYStWUM6vUswtOxUkLYObQGyqvXsr25fkDigoSYcK9VxnUh7CZaEU6n+W6saITJGA9px2CIA6q8JD1qik6N0kcDIc0LNUrV3xMJDpSaBL7pDLAeqUVvJv7ndWI9uPISFkaxpiGZLxrEHGmBZgmhPpOUaD4xgIlk5q+IjLDERJsc8yYEd/HkZWicV1yn4t5dFKrXWRw5OIYTKILl1CFW6hBHQg8wjO8wpv1ZL1Y79bHvHXFymaO4E9Znz8c+Zp5</latexit>

Logistic Regression

SLIDE 88

x P(x|Y=1) a 0.9 bit 0.2 dramatic 0.6 gamy 0.1 good 0.2 lovely 0.5 mushroo my 0.2 quite 0.7

Logistic Regression

Naive Bayes

SLIDE 89

Logistic Regression

x ??? a 0.9 bit 0.4 dramatic 1.0 gamy 0.7 good 0.2 lovely 0.4 mushroom y 0.8 quite 0.7

Logistic Regression

SLIDE 90

(d)

x ??? a 0.9 bit 0.4 dramatic 1.0 gamy 0.7 good 0.2 lovely 0.4 mushroom y 0.8 quite 0.7

Logistic Regression

Clicker Question! (a) WTF does this mean?

There is a 1.0 probability of

bserving “dramatic” given Y = 1

(b)

There is a 1.0 probability that Y = 1 given we observe “dramatic” 1 is the co-efficient on the “dramatic” variable in linear regression that minimizes the log loss.

(c)

1 is the co-efficient on the “dramatic” variable in the best fit linear regression.

SLIDE 91

(d)

x ??? a 0.9 bit 0.4 dramatic 1.0 gamy 0.7 good 0.2 lovely 0.4 mushroom y 0.8 quite 0.7

Logistic Regression

Clicker Question! (a) WTF does this mean?

There is a 1.0 probability of

bserving “dramatic” given Y = 1

(b)

There is a 1.0 probability that Y = 1 given we observe “dramatic” 1 is the co-efficient on the “dramatic” variable in linear regression that minimizes the log loss.

(c)

1 is the co-efficient on the “dramatic” variable in the best fit linear regression.

SLIDE 92

y x Quite mushroomy, a bit dramatic. ???

Logistic Regression

SLIDE 93

y x Quite mushroomy, a bit dramatic. ???

Logistic Regression

What do we do now?

SLIDE 94

y x Quite mushroomy, a bit dramatic. ???

Logistic Regression

y = ~ w · ~ x

<latexit sha1_base64="FBMv2wyAHxkmlGRKqF3+j4ktAn4=">ACAnicbVDLSgMxFM3UV62vUVfiJlgEV2VGBN0IRTcuK9gHtEPJZDJtaCYZkx1GAY3/obF4q49Svc+Tem7Sy09cCFk3PuJfceP2ZUacf5tkpLyura+X1ysbm1vaOvbvXUiKRmDSxYEJ2fKQIo5w0NdWMdGJUOQz0vZH1xO/PSZSUcHvdBoTL0IDTkOKkTZS3z5I4SXsjQnO7nPYw4HQs9dD3rerTs2ZAi4StyBVUKDRt796gcBJRLjGDCnVdZ1YexmSmJG8kovUSRGeIQGpGsoRxFRXjY9IYfHRglgKQpruFU/T2RoUipNPJNZ4T0UM17E/E/r5vo8MLKI8TiefRQmDGoBJ3nAgEqCNUsNQVhSsyvEQyQR1ia1ignBnT95kbROa65Tc2/PqvWrIo4yOARH4AS4BzUwQ1ogCbA4BE8g1fwZj1ZL9a79TFrLVnFzD74A+vzB6XrlvQ=</latexit><latexit sha1_base64="FBMv2wyAHxkmlGRKqF3+j4ktAn4=">ACAnicbVDLSgMxFM3UV62vUVfiJlgEV2VGBN0IRTcuK9gHtEPJZDJtaCYZkx1GAY3/obF4q49Svc+Tem7Sy09cCFk3PuJfceP2ZUacf5tkpLyura+X1ysbm1vaOvbvXUiKRmDSxYEJ2fKQIo5w0NdWMdGJUOQz0vZH1xO/PSZSUcHvdBoTL0IDTkOKkTZS3z5I4SXsjQnO7nPYw4HQs9dD3rerTs2ZAi4StyBVUKDRt796gcBJRLjGDCnVdZ1YexmSmJG8kovUSRGeIQGpGsoRxFRXjY9IYfHRglgKQpruFU/T2RoUipNPJNZ4T0UM17E/E/r5vo8MLKI8TiefRQmDGoBJ3nAgEqCNUsNQVhSsyvEQyQR1ia1ignBnT95kbROa65Tc2/PqvWrIo4yOARH4AS4BzUwQ1ogCbA4BE8g1fwZj1ZL9a79TFrLVnFzD74A+vzB6XrlvQ=</latexit><latexit sha1_base64="FBMv2wyAHxkmlGRKqF3+j4ktAn4=">ACAnicbVDLSgMxFM3UV62vUVfiJlgEV2VGBN0IRTcuK9gHtEPJZDJtaCYZkx1GAY3/obF4q49Svc+Tem7Sy09cCFk3PuJfceP2ZUacf5tkpLyura+X1ysbm1vaOvbvXUiKRmDSxYEJ2fKQIo5w0NdWMdGJUOQz0vZH1xO/PSZSUcHvdBoTL0IDTkOKkTZS3z5I4SXsjQnO7nPYw4HQs9dD3rerTs2ZAi4StyBVUKDRt796gcBJRLjGDCnVdZ1YexmSmJG8kovUSRGeIQGpGsoRxFRXjY9IYfHRglgKQpruFU/T2RoUipNPJNZ4T0UM17E/E/r5vo8MLKI8TiefRQmDGoBJ3nAgEqCNUsNQVhSsyvEQyQR1ia1ignBnT95kbROa65Tc2/PqvWrIo4yOARH4AS4BzUwQ1ogCbA4BE8g1fwZj1ZL9a79TFrLVnFzD74A+vzB6XrlvQ=</latexit><latexit sha1_base64="FBMv2wyAHxkmlGRKqF3+j4ktAn4=">ACAnicbVDLSgMxFM3UV62vUVfiJlgEV2VGBN0IRTcuK9gHtEPJZDJtaCYZkx1GAY3/obF4q49Svc+Tem7Sy09cCFk3PuJfceP2ZUacf5tkpLyura+X1ysbm1vaOvbvXUiKRmDSxYEJ2fKQIo5w0NdWMdGJUOQz0vZH1xO/PSZSUcHvdBoTL0IDTkOKkTZS3z5I4SXsjQnO7nPYw4HQs9dD3rerTs2ZAi4StyBVUKDRt796gcBJRLjGDCnVdZ1YexmSmJG8kovUSRGeIQGpGsoRxFRXjY9IYfHRglgKQpruFU/T2RoUipNPJNZ4T0UM17E/E/r5vo8MLKI8TiefRQmDGoBJ3nAgEqCNUsNQVhSsyvEQyQR1ia1ignBnT95kbROa65Tc2/PqvWrIo4yOARH4AS4BzUwQ1ogCbA4BE8g1fwZj1ZL9a79TFrLVnFzD74A+vzB6XrlvQ=</latexit>

SLIDE 95

y x Quite mushroomy, a bit dramatic. ???

Logistic Regression

y = ~ w · ~ x

<latexit sha1_base64="FBMv2wyAHxkmlGRKqF3+j4ktAn4=">ACAnicbVDLSgMxFM3UV62vUVfiJlgEV2VGBN0IRTcuK9gHtEPJZDJtaCYZkx1GAY3/obF4q49Svc+Tem7Sy09cCFk3PuJfceP2ZUacf5tkpLyura+X1ysbm1vaOvbvXUiKRmDSxYEJ2fKQIo5w0NdWMdGJUOQz0vZH1xO/PSZSUcHvdBoTL0IDTkOKkTZS3z5I4SXsjQnO7nPYw4HQs9dD3rerTs2ZAi4StyBVUKDRt796gcBJRLjGDCnVdZ1YexmSmJG8kovUSRGeIQGpGsoRxFRXjY9IYfHRglgKQpruFU/T2RoUipNPJNZ4T0UM17E/E/r5vo8MLKI8TiefRQmDGoBJ3nAgEqCNUsNQVhSsyvEQyQR1ia1ignBnT95kbROa65Tc2/PqvWrIo4yOARH4AS4BzUwQ1ogCbA4BE8g1fwZj1ZL9a79TFrLVnFzD74A+vzB6XrlvQ=</latexit><latexit sha1_base64="FBMv2wyAHxkmlGRKqF3+j4ktAn4=">ACAnicbVDLSgMxFM3UV62vUVfiJlgEV2VGBN0IRTcuK9gHtEPJZDJtaCYZkx1GAY3/obF4q49Svc+Tem7Sy09cCFk3PuJfceP2ZUacf5tkpLyura+X1ysbm1vaOvbvXUiKRmDSxYEJ2fKQIo5w0NdWMdGJUOQz0vZH1xO/PSZSUcHvdBoTL0IDTkOKkTZS3z5I4SXsjQnO7nPYw4HQs9dD3rerTs2ZAi4StyBVUKDRt796gcBJRLjGDCnVdZ1YexmSmJG8kovUSRGeIQGpGsoRxFRXjY9IYfHRglgKQpruFU/T2RoUipNPJNZ4T0UM17E/E/r5vo8MLKI8TiefRQmDGoBJ3nAgEqCNUsNQVhSsyvEQyQR1ia1ignBnT95kbROa65Tc2/PqvWrIo4yOARH4AS4BzUwQ1ogCbA4BE8g1fwZj1ZL9a79TFrLVnFzD74A+vzB6XrlvQ=</latexit><latexit sha1_base64="FBMv2wyAHxkmlGRKqF3+j4ktAn4=">ACAnicbVDLSgMxFM3UV62vUVfiJlgEV2VGBN0IRTcuK9gHtEPJZDJtaCYZkx1GAY3/obF4q49Svc+Tem7Sy09cCFk3PuJfceP2ZUacf5tkpLyura+X1ysbm1vaOvbvXUiKRmDSxYEJ2fKQIo5w0NdWMdGJUOQz0vZH1xO/PSZSUcHvdBoTL0IDTkOKkTZS3z5I4SXsjQnO7nPYw4HQs9dD3rerTs2ZAi4StyBVUKDRt796gcBJRLjGDCnVdZ1YexmSmJG8kovUSRGeIQGpGsoRxFRXjY9IYfHRglgKQpruFU/T2RoUipNPJNZ4T0UM17E/E/r5vo8MLKI8TiefRQmDGoBJ3nAgEqCNUsNQVhSsyvEQyQR1ia1ignBnT95kbROa65Tc2/PqvWrIo4yOARH4AS4BzUwQ1ogCbA4BE8g1fwZj1ZL9a79TFrLVnFzD74A+vzB6XrlvQ=</latexit><latexit sha1_base64="FBMv2wyAHxkmlGRKqF3+j4ktAn4=">ACAnicbVDLSgMxFM3UV62vUVfiJlgEV2VGBN0IRTcuK9gHtEPJZDJtaCYZkx1GAY3/obF4q49Svc+Tem7Sy09cCFk3PuJfceP2ZUacf5tkpLyura+X1ysbm1vaOvbvXUiKRmDSxYEJ2fKQIo5w0NdWMdGJUOQz0vZH1xO/PSZSUcHvdBoTL0IDTkOKkTZS3z5I4SXsjQnO7nPYw4HQs9dD3rerTs2ZAi4StyBVUKDRt796gcBJRLjGDCnVdZ1YexmSmJG8kovUSRGeIQGpGsoRxFRXjY9IYfHRglgKQpruFU/T2RoUipNPJNZ4T0UM17E/E/r5vo8MLKI8TiefRQmDGoBJ3nAgEqCNUsNQVhSsyvEQyQR1ia1ignBnT95kbROa65Tc2/PqvWrIo4yOARH4AS4BzUwQ1ogCbA4BE8g1fwZj1ZL9a79TFrLVnFzD74A+vzB6XrlvQ=</latexit>

SLIDE 96

y x Quite mushroomy, a bit dramatic. ???

Logistic Regression

y = 1 1 + e−(~

w·~ x)

<latexit sha1_base64="xjF7J23uMAifTYmCo9gPQZs6Uv0=">ACFXicbVDLSgMxFM3UV62vqks3wSJU1DIjgm6EohuXFewDOmPJpHdqaOZBkqmWYX7Cjb/ixoUibgV3/o1pOwtPXDh5Jx7yb3HjTiTyjS/jdzc/MLiUn65sLK6tr5R3NxqyDAWFOo05KFouUQCZwHUFVMcWpEA4rscm7/cuQ3ByAkC4MbNYzA8UkvYB6jRGmpUzwc4nNse4LQxEoT6wBuk6OyPQCa3KfYpt1Q4fHrId1P06xZFbMfAsTJSQhlqneKX3Q1p7EOgKCdSti0zUk5ChGKUQ1qwYwkRoX3Sg7amAfFBOsn4qhTvaWLvVDoChQeq78nEuJLOfRd3ekTdSenvZH4n9eOlXfmJCyIYgUBnXzkxRyrEI8iwl0mgCo+1IRQwfSumN4RHZHSQRZ0CNb0ybOkcVyxzIp1fVKqXmRx5NEO2kVlZKFTVEVXqIbqiKJH9Ixe0ZvxZLwY78bHpDVnZDPb6A+Mzx9o4Z5g</latexit><latexit sha1_base64="xjF7J23uMAifTYmCo9gPQZs6Uv0=">ACFXicbVDLSgMxFM3UV62vqks3wSJU1DIjgm6EohuXFewDOmPJpHdqaOZBkqmWYX7Cjb/ixoUibgV3/o1pOwtPXDh5Jx7yb3HjTiTyjS/jdzc/MLiUn65sLK6tr5R3NxqyDAWFOo05KFouUQCZwHUFVMcWpEA4rscm7/cuQ3ByAkC4MbNYzA8UkvYB6jRGmpUzwc4nNse4LQxEoT6wBuk6OyPQCa3KfYpt1Q4fHrId1P06xZFbMfAsTJSQhlqneKX3Q1p7EOgKCdSti0zUk5ChGKUQ1qwYwkRoX3Sg7amAfFBOsn4qhTvaWLvVDoChQeq78nEuJLOfRd3ekTdSenvZH4n9eOlXfmJCyIYgUBnXzkxRyrEI8iwl0mgCo+1IRQwfSumN4RHZHSQRZ0CNb0ybOkcVyxzIp1fVKqXmRx5NEO2kVlZKFTVEVXqIbqiKJH9Ixe0ZvxZLwY78bHpDVnZDPb6A+Mzx9o4Z5g</latexit><latexit sha1_base64="xjF7J23uMAifTYmCo9gPQZs6Uv0=">ACFXicbVDLSgMxFM3UV62vqks3wSJU1DIjgm6EohuXFewDOmPJpHdqaOZBkqmWYX7Cjb/ixoUibgV3/o1pOwtPXDh5Jx7yb3HjTiTyjS/jdzc/MLiUn65sLK6tr5R3NxqyDAWFOo05KFouUQCZwHUFVMcWpEA4rscm7/cuQ3ByAkC4MbNYzA8UkvYB6jRGmpUzwc4nNse4LQxEoT6wBuk6OyPQCa3KfYpt1Q4fHrId1P06xZFbMfAsTJSQhlqneKX3Q1p7EOgKCdSti0zUk5ChGKUQ1qwYwkRoX3Sg7amAfFBOsn4qhTvaWLvVDoChQeq78nEuJLOfRd3ekTdSenvZH4n9eOlXfmJCyIYgUBnXzkxRyrEI8iwl0mgCo+1IRQwfSumN4RHZHSQRZ0CNb0ybOkcVyxzIp1fVKqXmRx5NEO2kVlZKFTVEVXqIbqiKJH9Ixe0ZvxZLwY78bHpDVnZDPb6A+Mzx9o4Z5g</latexit><latexit sha1_base64="xjF7J23uMAifTYmCo9gPQZs6Uv0=">ACFXicbVDLSgMxFM3UV62vqks3wSJU1DIjgm6EohuXFewDOmPJpHdqaOZBkqmWYX7Cjb/ixoUibgV3/o1pOwtPXDh5Jx7yb3HjTiTyjS/jdzc/MLiUn65sLK6tr5R3NxqyDAWFOo05KFouUQCZwHUFVMcWpEA4rscm7/cuQ3ByAkC4MbNYzA8UkvYB6jRGmpUzwc4nNse4LQxEoT6wBuk6OyPQCa3KfYpt1Q4fHrId1P06xZFbMfAsTJSQhlqneKX3Q1p7EOgKCdSti0zUk5ChGKUQ1qwYwkRoX3Sg7amAfFBOsn4qhTvaWLvVDoChQeq78nEuJLOfRd3ekTdSenvZH4n9eOlXfmJCyIYgUBnXzkxRyrEI8iwl0mgCo+1IRQwfSumN4RHZHSQRZ0CNb0ybOkcVyxzIp1fVKqXmRx5NEO2kVlZKFTVEVXqIbqiKJH9Ixe0ZvxZLwY78bHpDVnZDPb6A+Mzx9o4Z5g</latexit>

SLIDE 97

y x Quite mushroomy, a bit dramatic. ???

Logistic Regression

y = 1 1 + e−(~

w·~ x)

<latexit sha1_base64="xjF7J23uMAifTYmCo9gPQZs6Uv0=">ACFXicbVDLSgMxFM3UV62vqks3wSJU1DIjgm6EohuXFewDOmPJpHdqaOZBkqmWYX7Cjb/ixoUibgV3/o1pOwtPXDh5Jx7yb3HjTiTyjS/jdzc/MLiUn65sLK6tr5R3NxqyDAWFOo05KFouUQCZwHUFVMcWpEA4rscm7/cuQ3ByAkC4MbNYzA8UkvYB6jRGmpUzwc4nNse4LQxEoT6wBuk6OyPQCa3KfYpt1Q4fHrId1P06xZFbMfAsTJSQhlqneKX3Q1p7EOgKCdSti0zUk5ChGKUQ1qwYwkRoX3Sg7amAfFBOsn4qhTvaWLvVDoChQeq78nEuJLOfRd3ekTdSenvZH4n9eOlXfmJCyIYgUBnXzkxRyrEI8iwl0mgCo+1IRQwfSumN4RHZHSQRZ0CNb0ybOkcVyxzIp1fVKqXmRx5NEO2kVlZKFTVEVXqIbqiKJH9Ixe0ZvxZLwY78bHpDVnZDPb6A+Mzx9o4Z5g</latexit><latexit sha1_base64="xjF7J23uMAifTYmCo9gPQZs6Uv0=">ACFXicbVDLSgMxFM3UV62vqks3wSJU1DIjgm6EohuXFewDOmPJpHdqaOZBkqmWYX7Cjb/ixoUibgV3/o1pOwtPXDh5Jx7yb3HjTiTyjS/jdzc/MLiUn65sLK6tr5R3NxqyDAWFOo05KFouUQCZwHUFVMcWpEA4rscm7/cuQ3ByAkC4MbNYzA8UkvYB6jRGmpUzwc4nNse4LQxEoT6wBuk6OyPQCa3KfYpt1Q4fHrId1P06xZFbMfAsTJSQhlqneKX3Q1p7EOgKCdSti0zUk5ChGKUQ1qwYwkRoX3Sg7amAfFBOsn4qhTvaWLvVDoChQeq78nEuJLOfRd3ekTdSenvZH4n9eOlXfmJCyIYgUBnXzkxRyrEI8iwl0mgCo+1IRQwfSumN4RHZHSQRZ0CNb0ybOkcVyxzIp1fVKqXmRx5NEO2kVlZKFTVEVXqIbqiKJH9Ixe0ZvxZLwY78bHpDVnZDPb6A+Mzx9o4Z5g</latexit><latexit sha1_base64="xjF7J23uMAifTYmCo9gPQZs6Uv0=">ACFXicbVDLSgMxFM3UV62vqks3wSJU1DIjgm6EohuXFewDOmPJpHdqaOZBkqmWYX7Cjb/ixoUibgV3/o1pOwtPXDh5Jx7yb3HjTiTyjS/jdzc/MLiUn65sLK6tr5R3NxqyDAWFOo05KFouUQCZwHUFVMcWpEA4rscm7/cuQ3ByAkC4MbNYzA8UkvYB6jRGmpUzwc4nNse4LQxEoT6wBuk6OyPQCa3KfYpt1Q4fHrId1P06xZFbMfAsTJSQhlqneKX3Q1p7EOgKCdSti0zUk5ChGKUQ1qwYwkRoX3Sg7amAfFBOsn4qhTvaWLvVDoChQeq78nEuJLOfRd3ekTdSenvZH4n9eOlXfmJCyIYgUBnXzkxRyrEI8iwl0mgCo+1IRQwfSumN4RHZHSQRZ0CNb0ybOkcVyxzIp1fVKqXmRx5NEO2kVlZKFTVEVXqIbqiKJH9Ixe0ZvxZLwY78bHpDVnZDPb6A+Mzx9o4Z5g</latexit><latexit sha1_base64="xjF7J23uMAifTYmCo9gPQZs6Uv0=">ACFXicbVDLSgMxFM3UV62vqks3wSJU1DIjgm6EohuXFewDOmPJpHdqaOZBkqmWYX7Cjb/ixoUibgV3/o1pOwtPXDh5Jx7yb3HjTiTyjS/jdzc/MLiUn65sLK6tr5R3NxqyDAWFOo05KFouUQCZwHUFVMcWpEA4rscm7/cuQ3ByAkC4MbNYzA8UkvYB6jRGmpUzwc4nNse4LQxEoT6wBuk6OyPQCa3KfYpt1Q4fHrId1P06xZFbMfAsTJSQhlqneKX3Q1p7EOgKCdSti0zUk5ChGKUQ1qwYwkRoX3Sg7amAfFBOsn4qhTvaWLvVDoChQeq78nEuJLOfRd3ekTdSenvZH4n9eOlXfmJCyIYgUBnXzkxRyrEI8iwl0mgCo+1IRQwfSumN4RHZHSQRZ0CNb0ybOkcVyxzIp1fVKqXmRx5NEO2kVlZKFTVEVXqIbqiKJH9Ixe0ZvxZLwY78bHpDVnZDPb6A+Mzx9o4Z5g</latexit>

P(Y=1) = 0.38

SLIDE 98

98

SLIDE 99

Code-along!

99

from sklearn.linear_model import LogisticRegression