Estimation Theory STAT 432 | UIUC | Fall 2019 | Dalpiaz Questions? - - PowerPoint PPT Presentation

▶
estimation theory
SMART_READER_LITE
LIVE PREVIEW

Estimation Theory STAT 432 | UIUC | Fall 2019 | Dalpiaz Questions? - - PowerPoint PPT Presentation

Feb 17, 2023 366 likes •853 views

Estimation Theory STAT 432 | UIUC | Fall 2019 | Dalpiaz Questions? Comments? Concerns? Administration CBTF Syllabus Exam PrairieLearn Quiz Quiz Policy Last Week This Week Do You Remember STAT 400? PROBS DISTRIBUTION S = t =

slide-1
SLIDE 1

Estimation Theory

STAT 432 | UIUC | Fall 2019 | Dalpiaz
slide-2
SLIDE 2

Questions?

Comments? Concerns?

slide-3
SLIDE 3

Administration

  • CBTF Syllabus Exam
  • PrairieLearn Quiz
  • Quiz Policy
slide-4
SLIDE 4

Last Week This Week

slide-5
SLIDE 5

Do You Remember STAT 400?

DISTRIBUTION = S t PROBS STATISTIC = f( DATA)

I

slide-6
SLIDE 6 p(x|n, p) = ( n x)px(1 − p)n−x, x = 0,1,…, n, n ∈ ℕ, 0 < p < 1

X ∼ bin(n, p)

FAMILY OF DISTRIBUTIONS
  • parameters
. ..

A

←

✓

SAMPLE PARAMETER SPACE SPACE

pmf

OUTPUTS A PROBABILITY
slide-7
SLIDE 7

X ∼ N (μ, σ2)

f(x|μ, σ2) = 1 σ 2π ⋅ exp [ −1 2 ( x − μ σ ) 2 ], − ∞ < x < ∞, − ∞ < μ < ∞, σ > 0 DISTRIBUTIONS
  • #
SAM }§ae€ PARAM SPACE

pdf

. OUTPUTS DENSITIES '

NII

PROBS
slide-8
SLIDE 8

X ∼ N (μ = 5, σ2 = 4) P[X = 4] = ? P[X < 4] = ?

= O

=pt¥¥

= phorm ( 4 , mean = 5 , so
  • 2)
slide-9
SLIDE 9

X ∼ N (μ = 5, σ2 = 4) P[2 < X < 4] = ? P[X > 3] = ?

diff( prior
  • (42,4)
, mean = 5 , so
  • Z))
= I
  • pnorn(3,mean=5,sd=Z)i¥

*¥¥E¥

slide-10
SLIDE 10

X ∼ N (μ = 5, σ2 = 4) P[X > c] = 0.75, c = ?

"

" """" "" " " "

slide-11
SLIDE 11 PROBABILITY IN R
  • d * ( x ,
. . . ) =

pdf/pmf

p * ( q ,
  • . . )
=

P[Xe×)=

cdf

q * ( p , . . . ) = c :

P[ x. c) =p

r * ( n ,
  • .
. ) → GENERATES RANDOM OBS * = NAMED DISTRIBUTIONS
slide-12
SLIDE 12

Expectations

X

  • N/a
, r:) Y
  • N (ux.fi)
  • X. Y
mo

E [X

  • Y)
= u×
  • my

←

STILL TRUE w/o end ✓ Are [2+3×-44] = 9 of ' +16 r} c- NEED WD
slide-13
SLIDE 13

Estimation

slide-14
SLIDE 14

What is Statistics?

slide-15
SLIDE 15

What is Statistics?

  • Technically: The study of statistics.
  • Practically: The science of collecting, organizing,
analyzing, interpreting, and presenting data.
slide-16
SLIDE 16

What is a statistic?

f-(DAIA)

slide-17
SLIDE 17

What is a statistic?

  • Technically: A function of (sample) data.
slide-18
SLIDE 18

Terminology

  • Population: The entire group of interest.
  • Parameter: A (usually unknown) numeric value associated with the population.
  • Sample: A subset of individuals taken from the population.
  • Statistic: A numeric value computed using the sample data.
slide-19
SLIDE 19 The Big Picture Sampo, no / HOPEFUL'T RANDO - u, popo
  • E" " -
O , 02 SAMPLE

O

x . .
  • tho)

t

⑤ = f ( x .,Xr. . .. . . X)
slide-20
SLIDE 20

Random Sample

  • A random sample is a sample where each
individual in the population is equally likely to be included.

Why do we care?

WE LIKE TO MAKE AN HD ASSUMPTION
slide-21
SLIDE 21

What is an estimator?

  • A statistic that attempts to provide a good guess
for an unknown population parameter.
slide-22
SLIDE 22

Parameters Estimators

cylon ! A Fw

EG)

(

IG , ,x ..

  • xn)
  • II. *

VARG] ( SD

censuses g2=

i€(

Xi
  • t )
"
  • X ,
, Xr ,
  • .
. . , Xn " to BETA ( N , B) MCE, MOM

P[ X - 4]

ECDF , MCE
slide-23
SLIDE 23

Statistics are Random Variables

  • An Estimator / A statistic: A function that tells us what calculation we will
perform after taking a sample from the population.
  • An Estimate / The value of a statistic: The numeric result of performing the
calculation on a particular sample of data.

X vs x

#

µ

POTENTIAL VALUE OF RANDOM VARIABLE . RANDOM VARIABLE
slide-24
SLIDE 24 Why so much focus on the mean? WANT TO MINIMIZE

↳ E

  • a))
= Efx '
  • 2aXtaJ=E[
x)
  • 2aE[x]
+ a ' WEE .

¥Eaction

÷ Efx

  • at]
=
  • LEG)
c- Za
  • O
⇒

la=ETx]/

  • PREDICT
THE MEAN to minimize SQUARED ERROR LOSS .
slide-25
SLIDE 25
slide-26
SLIDE 26
slide-27
SLIDE 27
slide-28
SLIDE 28 UNBIASED ←¥¥l# O = ECO] BIASED #§µE[
  • n]
slide-29
SLIDE 29

Definitions

bias [ ̂ θ] ≜ 피 [ ̂ θ] − θ

var [ ̂ θ] ≜ 피 [( ̂ θ − 피 [ ̂ θ]) 2 ] MSE [ ̂ θ] ≜ 피 [( ̂ θ − θ) 2 ] = var [ ̂ θ] + (Bias [ ̂ θ]) 2
slide-30
SLIDE 30 X1, X2, X3 ∼ N(μ, σ2) ¯ X = 1 n n ∑ i=1 Xi ̂ μ = 1 4 X1 + 1 5 X2 + 1 6 X3 MSEEX] us MSE I
  • EEE]=u
Bias = Eft]
  • a
= er
  • er
= O VAR = MSE = 043
  • E
= It Itf = SIT in Bias = u
  • Eon
  • In

a-

Van = + ¥ .

=¥÷m/nsEG

)
  • ⇐ a)
' + II. or
  • SMALL
WHEN U CLOSE TO 0
slide-31
SLIDE 31

Probability Models

slide-32
SLIDE 32 Professor Professorson received the following number of emails on each of the previous 40 days: 5 6 2 5 3 3 4 1 4 4 3 4 6 2 3 6 7 1 3 3 5 1 8 6 1 3 2 5 3 5 4 4 2 4 0 5 0 2 5 3
  • What is the probability that Professor Professorson receives three emails
  • n a particular day?
  • What is the probability that Professor Professorson receives nine emails
  • n a particular day?
  • 9/40

0/40

slide-33
SLIDE 33 Idea: Use a Poisson distribution to model the number of emails received per day. But which Poisson distribution?
slide-34
SLIDE 34

jpfx.SI#E

/

  • .am,

y pix =D

slide-35
SLIDE 35
slide-36
SLIDE 36
slide-37
SLIDE 37
slide-38
SLIDE 38
slide-39
SLIDE 39
slide-40
SLIDE 40 The previous “analysis” is flawed.
  • The previous 40 days is not a random sample.
  • Why do we need a random sample?
  • We were just guessing and checking.
  • How do we know if Poisson was a reasonable distribution?
  • How do we pick a good lambda given some data?
  • How do we know if our method for picking is good?
slide-41
SLIDE 41

Maximum Likelihood

slide-42
SLIDE 42

* a

"

'

slide-43
SLIDE 43 Assume X. , Xi ,
  • .
. Xu
  • f(x IO)
IF "D

①ne=trgm%¥g⇒IIf)

slide-44
SLIDE 44

Fitting a Probability Distribution

slide-45
SLIDE 45 data = c(4, 10, 9, 3, 2, 4, 3, 7, 3, 5) 5=5 WANT TO ESTIMATE THINGS LIKE p [× = 4) Empirical CDF F- Cx)=F[x. D= #*n ur Discrete P' [ X
  • x)
= #

f[x

  • 43=1/5

→ P' (x

  • o)
slide-46
SLIDE 46 → IS DATA NUMERIC ? CATEGORICAL ? LOOK AT THE DATA iii. ' 'Fi . i i
  • i
i ASSUME PROBABILITY DISTRIBUTION CONTINUOUS DISCRETE RANGE ? ?
  • O , 1,2 ,
. . . ? ESTIMATE PARAMETERS → MLE from
  • CHECK
RESULTS → Q
  • Q
plots
  • USE
RESULTS
  • 3
ESTIMATE QUANTITIES LIKE

P[X

> 3)
slide-47
SLIDE 47 X. , Xr . . . . X .
  • Pols ( X)

I

I

5=5

IF ⑤ is nice For O h (a) =P [ X= 2) = THEN hCG) is nee Fon ko)

icx=g=5÷

slide-48
SLIDE 48

Questions?

Comments? Concerns?