Recitation First recitation tomorrow 56:30 here Linear algebra - PowerPoint PPT Presentation

Recitation • First recitation tomorrow 5–6:30 here • Linear algebra Geoff Gordon—10-701 Machine Learning—Fall 2013 1

Probability P(a) = P(u) = P(~a) = Geoff Gordon—10-701 Machine Learning—Fall 2013 2

Conventions Geoff Gordon—10-701 Machine Learning—Fall 2013 3

Union, intersection Geoff Gordon—10-701 Machine Learning—Fall 2013 4

Conditioning Geoff Gordon—10-701 Machine Learning—Fall 2013 5

Law of total probability Geoff Gordon—10-701 Machine Learning—Fall 2013 6

Marginals Geoff Gordon—10-701 Machine Learning—Fall 2013 7

Finite vs. infinite |u| • http://www.amazon.com/Probability-Measure-Wiley-Series- Statistics/dp/1118122372 • http://en.wikipedia.org/wiki/Regular_conditional_probability • http://en.wikipedia.org/wiki/Borel%E2%80%93Kolmogorov_paradox Geoff Gordon—10-701 Machine Learning—Fall 2013 8

How I learned to stop worrying and love the density function… 1 a ≤ x ≤ b b − a 0 o/w 2 πσ exp( − 1 1 2 ( x − µ ) 2 /σ 2 ) √ ent: Geoff Gordon—10-701 Machine Learning—Fall 2013 9

Multivariate densities Geoff Gordon—10-701 Machine Learning—Fall 2013 10

Random variables Probability space ( σ -algebra) Geoff Gordon—10-701 Machine Learning—Fall 2013 11

Bayes rule • recall def of conditional: ‣ P(a|b) = P(a^b) / P(b) if P(b) != 0 Geoff Gordon—10-701 Machine Learning—Fall 2013 12

Bayes rule: sum version Geoff Gordon—10-701 Machine Learning—Fall 2013 13

Test for a rare disease • About 0.1% of all people are infected • Test detects all infections • Test is highly specific: 1% false positive • You test positive. What is the probability you have the disease? Geoff Gordon—10-701 Machine Learning—Fall 2013 14

Test for a rare disease • About 0.1% of all people are infected • Test detects all infections Bonus: what is probability an average med student • Test is highly specific: 1% false positive gets this question wrong? • You test positive. What is the probability you have the disease? Geoff Gordon—10-701 Machine Learning—Fall 2013 14

Follow-up test • Test 2: detects 90% of infections, 5% false positives ‣ P(+disease | +test1, +test2) = Geoff Gordon—10-701 Machine Learning—Fall 2013 15

Using Bayes rule $$$ Geoff Gordon—10-701 Machine Learning—Fall 2013 16

Using Bayes rule $$$ Geoff Gordon—10-701 Machine Learning—Fall 2013 16

Independence Geoff Gordon—10-701 Machine Learning—Fall 2013 17

Conditional independence Geoff Gordon—10-701 Machine Learning—Fall 2013 18

Conditionally Independent London taxi drivers: A survey has pointed out a positive and significant correlation between the number of accidents and wearing coats. They concluded that coats could hinder movements of drivers and be the cause of accidents. A new law was prepared to prohibit drivers from wearing coats when driving. Finally another study pointed out that people wear coats when it rains… slide credit: Barnabas humor credit: xkcd xkcd.com 31

Samples … Geoff Gordon—10-701 Machine Learning—Fall 2013 20

Recall: spam filtering Geoff Gordon—10-701 Machine Learning—Fall 2013 21

Bag of words Geoff Gordon—10-701 Machine Learning—Fall 2013 22

A ridiculously naive assumption • Assume: • Clearly false: • Given this assumption, use Bayes rule Geoff Gordon—10-701 Machine Learning—Fall 2013 23

Graphical model spam spam . . . x i x 1 x 2 x n i=1..n Geoff Gordon—10-701 Machine Learning—Fall 2013 24

Naive Bayes • P(spam | email ∧ award ∧ program ∧ for ∧ internet ∧ users ∧ lump ∧ sum ∧ of ∧ Five ∧ Million) Geoff Gordon—10-701 Machine Learning—Fall 2013 25

In log space z spam = ln(P(email | spam) P(award | spam) ... P(Million | spam) P(spam)) z ~spam = ln(P(email | ~spam) ... P(Million | ~spam) P(~spam)) Geoff Gordon—10-701 Machine Learning—Fall 2013 26

Collect terms z spam = ln(P(email | spam) P(award | spam) ... P(Million | spam) P(spam)) z ~spam = ln(P(email | ~spam) ... P(Million | ~spam) P(~spam)) z = z spam – z spam Geoff Gordon—10-701 Machine Learning—Fall 2013 27

Linear discriminant Geoff Gordon—10-701 Machine Learning—Fall 2013 28

Intuitions Geoff Gordon—10-701 Machine Learning—Fall 2013 29

How to get probabilities? Geoff Gordon—10-701 Machine Learning—Fall 2013 30

Improvements Geoff Gordon—10-701 Machine Learning—Fall 2013 31