Introduction to Machine Learning CMU-10701 2. MLE, MAP What - - PowerPoint PPT Presentation
Introduction to Machine Learning CMU-10701 2. MLE, MAP What happened last time? Barnabs Pczos & Aarti Singh 2014 Spring Administration Piazza: Please use it! Blackboard is ready Self assessment questions?
Barnabás Póczos & Aarti Singh 2014 Spring
2
3
Y and X don’t contain information about each other. Observing Y doesn’t help predicting X. Independent random variables:
Independent X,Y X Dependent X,Y X Y Y
4
5
Dependent: show size and reading skills Conditionally independent: show size and reading skills given age
Examples:
Conditionally independent: Knowing Z makes X and Y independent
6
Our first machine learning problem:
7
Data, D = P(Heads) = , P(Tails) = 1-
MLE: Choose that maximizes the probability of observed data
3/5
“Frequency of heads”
The estimated probability is:
Independent draws Identically distributed
8
MLE: Choose that maximizes the probability of observed data
9
I want to know the coin parameter 2[0,1] within = 0.1 error, with probability at least 1- = 0.95.
How many flips do I need?
24 120 60 12
10
Does the MLE estimation (relative frequancies) converge to the right value? How fast does it converge?
11
Does the empirical average converge to the true mean? How fast does it converge?
5 sample traces
12
How fast do they converge to the true mean?
13
From Hoeffding:
14
Convergence rate
Barnabás Póczos