[PPT] - Factorization Theorem Lecture 02 Biostatistics 602 - Statistical PowerPoint Presentation

SLIDE 1

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

. .

Biostatistics 602 - Statistical Inference Lecture 02 Factorization Theorem

Hyun Min Kang January 15th, 2013

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 1 / 27

SLIDE 2

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Last Lecture - Key Questions

. . 1 What is the key difference between BIOSTAT601 and BIOSTAT602? . . 2 What is the difference between random variable and data? . . 3 What is a statistic? . 4 What is a sufficient statistic for

?

. . 5 How do we show that a statistic is sufficient for

?

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 2 / 27

SLIDE 3

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Last Lecture - Key Questions

. . 1 What is the key difference between BIOSTAT601 and BIOSTAT602? . . 2 What is the difference between random variable and data? . . 3 What is a statistic? . 4 What is a sufficient statistic for

?

. . 5 How do we show that a statistic is sufficient for

?

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 2 / 27

SLIDE 4

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Last Lecture - Key Questions

. . 1 What is the key difference between BIOSTAT601 and BIOSTAT602? . . 2 What is the difference between random variable and data? . . 3 What is a statistic? . . 4 What is a sufficient statistic for

?

. . 5 How do we show that a statistic is sufficient for

?

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 2 / 27

SLIDE 5

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Last Lecture - Key Questions

. . 1 What is the key difference between BIOSTAT601 and BIOSTAT602? . . 2 What is the difference between random variable and data? . . 3 What is a statistic? . . 4 What is a sufficient statistic for θ? . 5 How do we show that a statistic is sufficient for

?

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 2 / 27

SLIDE 6

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Last Lecture - Key Questions

. . 1 What is the key difference between BIOSTAT601 and BIOSTAT602? . . 2 What is the difference between random variable and data? . . 3 What is a statistic? . . 4 What is a sufficient statistic for θ? . . 5 How do we show that a statistic is sufficient for θ?

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 2 / 27

SLIDE 7

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Last Lecture

.

Definition 6.2.1

. . A statistic T(X) is a sufficient statistic for θ if the conditional distribution

f sample X given the value of T(X) does not depend on θ.

.

Example

. .

Suppose X1, · · · , Xn

i.i.d.

∼ Bernoulli(p), 0 < p < 1.

T(X1, · · · , Xn) = ∑n

i=1 Xi is a sufficient statistic for p.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 3 / 27

SLIDE 8

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Recap - A Theorem for Sufficient Statistics

.

Theorem 6.2.2

. .

Let fX(x|θ) is a joint pdf or pmf of X
and q(t|θ) is the pdf or pmf of T(X).
Then T(X) is a sufficient statistic for θ,
if, for every x ∈ X,
the ratio fX(x|θ)/q(T(x)|θ) is constant as a function of θ.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 4 / 27

SLIDE 9

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Recap - Example 6.2.3 - Binomial Sufficient Statistic

.

Proof

. . fX(x|p) = px1(1 − p)1−x1 · · · pxn(1 − p)1−xn = p

∑n

i=1 xi(1 − p)n−∑n i=1 xi

T X Binomial n p q t p n t pt p n

t

fX x p q T x p p

n i

xi

p n

n i

xi n

n i

xi p

n i

xi

p n

n i

xi n

n i

xi n T x

By theorem 6.2.2. T X is a sufficient statistic for p.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 5 / 27

SLIDE 10

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Recap - Example 6.2.3 - Binomial Sufficient Statistic

.

Proof

. . fX(x|p) = px1(1 − p)1−x1 · · · pxn(1 − p)1−xn = p

∑n

i=1 xi(1 − p)n−∑n i=1 xi

T(X) ∼ Binomial(n, p) q t p n t pt p n

t

fX x p q T x p p

n i

xi

p n

n i

xi n

n i

xi p

n i

xi

p n

n i

xi n

n i

xi n T x

By theorem 6.2.2. T X is a sufficient statistic for p.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 5 / 27

SLIDE 11

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Recap - Example 6.2.3 - Binomial Sufficient Statistic

.

Proof

. . fX(x|p) = px1(1 − p)1−x1 · · · pxn(1 − p)1−xn = p

∑n

i=1 xi(1 − p)n−∑n i=1 xi

T(X) ∼ Binomial(n, p) q(t|p) = (n t ) pt(1 − p)n−t fX x p q T x p p

n i

xi

p n

n i

xi n

n i

xi p

n i

xi

p n

n i

xi n

n i

xi n T x

By theorem 6.2.2. T X is a sufficient statistic for p.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 5 / 27

SLIDE 12

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Recap - Example 6.2.3 - Binomial Sufficient Statistic

.

Proof

. . fX(x|p) = px1(1 − p)1−x1 · · · pxn(1 − p)1−xn = p

∑n

i=1 xi(1 − p)n−∑n i=1 xi

T(X) ∼ Binomial(n, p) q(t|p) = (n t ) pt(1 − p)n−t fX(x|p) q(T(x)|p) = p

∑n

i=1 xi(1 − p)n−∑n i=1 xi

(

n ∑n

i=1 xi

) p

∑n

i=1 xi(1 − p)n−∑n i=1 xi

n

n i

xi n T x

By theorem 6.2.2. T X is a sufficient statistic for p.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 5 / 27

SLIDE 13

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Recap - Example 6.2.3 - Binomial Sufficient Statistic

.

Proof

. . fX(x|p) = px1(1 − p)1−x1 · · · pxn(1 − p)1−xn = p

∑n

i=1 xi(1 − p)n−∑n i=1 xi

T(X) ∼ Binomial(n, p) q(t|p) = (n t ) pt(1 − p)n−t fX(x|p) q(T(x)|p) = p

∑n

i=1 xi(1 − p)n−∑n i=1 xi

(

n ∑n

i=1 xi

) p

∑n

i=1 xi(1 − p)n−∑n i=1 xi

= 1 (

n ∑n

i=1 xi

) = 1 ( n

T(x)

) By theorem 6.2.2. T X is a sufficient statistic for p.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 5 / 27

SLIDE 14

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Recap - Example 6.2.3 - Binomial Sufficient Statistic

.

Proof

. . fX(x|p) = px1(1 − p)1−x1 · · · pxn(1 − p)1−xn = p

∑n

i=1 xi(1 − p)n−∑n i=1 xi

T(X) ∼ Binomial(n, p) q(t|p) = (n t ) pt(1 − p)n−t fX(x|p) q(T(x)|p) = p

∑n

i=1 xi(1 − p)n−∑n i=1 xi

(

n ∑n

i=1 xi

) p

∑n

i=1 xi(1 − p)n−∑n i=1 xi

= 1 (

n ∑n

i=1 xi

) = 1 ( n

T(x)

) By theorem 6.2.2. T(X) is a sufficient statistic for p.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 5 / 27

SLIDE 15

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem

.

Theorem 6.2.6 - Factorization Theorem

. .

Let fX(x|θ) denote the joint pdf or pmf of a sample X.
A statistic T X is a sufficient statistic for

, if and only if

There exists function g t

and h x such that,

for all sample points x,
and for all parameter points

,

fX x

g T x h x .

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 6 / 27

SLIDE 16

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem

.

Theorem 6.2.6 - Factorization Theorem

. .

Let fX(x|θ) denote the joint pdf or pmf of a sample X.
A statistic T(X) is a sufficient statistic for θ, if and only if
There exists function g t

and h x such that,

for all sample points x,
and for all parameter points

,

fX x

g T x h x .

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 6 / 27

SLIDE 17

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem

.

Theorem 6.2.6 - Factorization Theorem

. .

Let fX(x|θ) denote the joint pdf or pmf of a sample X.
A statistic T(X) is a sufficient statistic for θ, if and only if
There exists function g(t|θ) and h(x) such that,
for all sample points x,
and for all parameter points

,

fX x

g T x h x .

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 6 / 27

SLIDE 18

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem

.

Theorem 6.2.6 - Factorization Theorem

. .

Let fX(x|θ) denote the joint pdf or pmf of a sample X.
A statistic T(X) is a sufficient statistic for θ, if and only if
There exists function g(t|θ) and h(x) such that,
for all sample points x,
and for all parameter points

,

fX x

g T x h x .

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 6 / 27

SLIDE 19

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem

.

Theorem 6.2.6 - Factorization Theorem

. .

Let fX(x|θ) denote the joint pdf or pmf of a sample X.
A statistic T(X) is a sufficient statistic for θ, if and only if
There exists function g(t|θ) and h(x) such that,
for all sample points x,
and for all parameter points θ,
fX x

g T x h x .

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 6 / 27

SLIDE 20

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem

.

Theorem 6.2.6 - Factorization Theorem

. .

Let fX(x|θ) denote the joint pdf or pmf of a sample X.
A statistic T(X) is a sufficient statistic for θ, if and only if
There exists function g(t|θ) and h(x) such that,
for all sample points x,
and for all parameter points θ,
fX(x|θ) = g(T(x)|θ)h(x).

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 6 / 27

SLIDE 21

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem : Proof

The proof below is only for discrete distributions. .

nly if part

. .

Suppose that T(X) is a sufficient statistic
Choose g t

Pr T X t

and h x

Pr X x T X T x

Because T X is sufficient, h x does not depend on

. fX x Pr X x Pr X x T X T x Pr T X T x Pr X x T X T x Pr T X T x Pr X x T X T x g T x h x

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 7 / 27

SLIDE 22

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem : Proof

The proof below is only for discrete distributions. .

nly if part

. .

Suppose that T(X) is a sufficient statistic
Choose g(t|θ) = Pr(T(X) = t|θ)
and h x

Pr X x T X T x

Because T X is sufficient, h x does not depend on

. fX x Pr X x Pr X x T X T x Pr T X T x Pr X x T X T x Pr T X T x Pr X x T X T x g T x h x

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 7 / 27

SLIDE 23

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem : Proof

The proof below is only for discrete distributions. .

nly if part

. .

Suppose that T(X) is a sufficient statistic
Choose g(t|θ) = Pr(T(X) = t|θ)
and h(x) = Pr (X = x|T(X) = T(x))
Because T X is sufficient, h x does not depend on

. fX x Pr X x Pr X x T X T x Pr T X T x Pr X x T X T x Pr T X T x Pr X x T X T x g T x h x

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 7 / 27

SLIDE 24

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem : Proof

The proof below is only for discrete distributions. .

nly if part

. .

Suppose that T(X) is a sufficient statistic
Choose g(t|θ) = Pr(T(X) = t|θ)
and h(x) = Pr (X = x|T(X) = T(x))
Because T(X) is sufficient, h(x) does not depend on θ.

fX x Pr X x Pr X x T X T x Pr T X T x Pr X x T X T x Pr T X T x Pr X x T X T x g T x h x

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 7 / 27

SLIDE 25

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem : Proof

The proof below is only for discrete distributions. .

nly if part

. .

Suppose that T(X) is a sufficient statistic
Choose g(t|θ) = Pr(T(X) = t|θ)
and h(x) = Pr (X = x|T(X) = T(x))
Because T(X) is sufficient, h(x) does not depend on θ.

fX(x|θ) = Pr(X = x|θ) Pr X x T X T x Pr T X T x Pr X x T X T x Pr T X T x Pr X x T X T x g T x h x

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 7 / 27

SLIDE 26

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem : Proof

The proof below is only for discrete distributions. .

nly if part

. .

Suppose that T(X) is a sufficient statistic
Choose g(t|θ) = Pr(T(X) = t|θ)
and h(x) = Pr (X = x|T(X) = T(x))
Because T(X) is sufficient, h(x) does not depend on θ.

fX(x|θ) = Pr(X = x|θ) = Pr(X = x ∧ T(X) = T(x)|θ) Pr T X T x Pr X x T X T x Pr T X T x Pr X x T X T x g T x h x

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 7 / 27

SLIDE 27

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem : Proof

The proof below is only for discrete distributions. .

nly if part

. .

Suppose that T(X) is a sufficient statistic
Choose g(t|θ) = Pr(T(X) = t|θ)
and h(x) = Pr (X = x|T(X) = T(x))
Because T(X) is sufficient, h(x) does not depend on θ.

fX(x|θ) = Pr(X = x|θ) = Pr(X = x ∧ T(X) = T(x)|θ) = Pr(T(X) = T(x)|θ) Pr(X = x|T(X) = T(x), θ) Pr T X T x Pr X x T X T x g T x h x

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 7 / 27

SLIDE 28

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem : Proof

The proof below is only for discrete distributions. .

nly if part

. .

Suppose that T(X) is a sufficient statistic
Choose g(t|θ) = Pr(T(X) = t|θ)
and h(x) = Pr (X = x|T(X) = T(x))
Because T(X) is sufficient, h(x) does not depend on θ.

fX(x|θ) = Pr(X = x|θ) = Pr(X = x ∧ T(X) = T(x)|θ) = Pr(T(X) = T(x)|θ) Pr(X = x|T(X) = T(x), θ) = Pr(T(X) = T(x)|θ) Pr(X = x|T(X) = T(x)) g T x h x

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 7 / 27

SLIDE 29

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem : Proof

The proof below is only for discrete distributions. .

nly if part

. .

Suppose that T(X) is a sufficient statistic
Choose g(t|θ) = Pr(T(X) = t|θ)
and h(x) = Pr (X = x|T(X) = T(x))
Because T(X) is sufficient, h(x) does not depend on θ.

fX(x|θ) = Pr(X = x|θ) = Pr(X = x ∧ T(X) = T(x)|θ) = Pr(T(X) = T(x)|θ) Pr(X = x|T(X) = T(x), θ) = Pr(T(X) = T(x)|θ) Pr(X = x|T(X) = T(x)) = g(T(x)|θ)h(x)

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 7 / 27

SLIDE 30

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem : Proof (cont’d)

.

if part

. .

Assume that the factorization fX(x|θ) = g(T(x)|θ)h(x) exists.
Let q t

be the pmf of T X

Define At

y T y t . q t Pr T X t

y At

fX y

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 8 / 27

SLIDE 31

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem : Proof (cont’d)

.

if part

. .

Assume that the factorization fX(x|θ) = g(T(x)|θ)h(x) exists.
Let q(t|θ) be the pmf of T(X)
Define At

y T y t . q t Pr T X t

y At

fX y

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 8 / 27

SLIDE 32

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem : Proof (cont’d)

.

if part

. .

Assume that the factorization fX(x|θ) = g(T(x)|θ)h(x) exists.
Let q(t|θ) be the pmf of T(X)
Define At = {y : T(y) = t}.

q t Pr T X t

y At

fX y

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 8 / 27

SLIDE 33

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem : Proof (cont’d)

.

if part

. .

Assume that the factorization fX(x|θ) = g(T(x)|θ)h(x) exists.
Let q(t|θ) be the pmf of T(X)
Define At = {y : T(y) = t}.

q(t|θ) = Pr(T(X) = t|θ)

y At

fX y

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 8 / 27

SLIDE 34

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem : Proof (cont’d)

.

if part

. .

Assume that the factorization fX(x|θ) = g(T(x)|θ)h(x) exists.
Let q(t|θ) be the pmf of T(X)
Define At = {y : T(y) = t}.

q(t|θ) = Pr(T(X) = t|θ) = ∑

y∈At

fX(y|θ)

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 8 / 27

SLIDE 35

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem : Proof (cont’d)

.

if part (cont’d)

. . fX(x|θ) q(T(x)|θ) = g(T(x)|θ)h(x) q(T(x)|θ) = g(T(x)|θ)h(x) ∑

y∈AT(x) fX(y|θ)

g T x h x

y AT x g T y

h y g T x h x g T x

Ay

T x h y

h x

AT x h y

Thus, T X is a sufficient statistic for , if and only if fX x g T x h x .

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 9 / 27

SLIDE 36

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem : Proof (cont’d)

.

if part (cont’d)

. . fX(x|θ) q(T(x)|θ) = g(T(x)|θ)h(x) q(T(x)|θ) = g(T(x)|θ)h(x) ∑

y∈AT(x) fX(y|θ)

= g(T(x)|θ)h(x) ∑

y∈AT(x) g(T(y)|θ)h(y)

g T x h x g T x

Ay

T x h y

h x

AT x h y

Thus, T X is a sufficient statistic for , if and only if fX x g T x h x .

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 9 / 27

SLIDE 37

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem : Proof (cont’d)

.

if part (cont’d)

. . fX(x|θ) q(T(x)|θ) = g(T(x)|θ)h(x) q(T(x)|θ) = g(T(x)|θ)h(x) ∑

y∈AT(x) fX(y|θ)

= g(T(x)|θ)h(x) ∑

y∈AT(x) g(T(y)|θ)h(y) =

g(T(x)|θ)h(x) g(T(x)|θ) ∑

Ay∈T(x) h(y)

h x

AT x h y

Thus, T X is a sufficient statistic for , if and only if fX x g T x h x .

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 9 / 27

SLIDE 38

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem : Proof (cont’d)

.

if part (cont’d)

. . fX(x|θ) q(T(x)|θ) = g(T(x)|θ)h(x) q(T(x)|θ) = g(T(x)|θ)h(x) ∑

y∈AT(x) fX(y|θ)

= g(T(x)|θ)h(x) ∑

y∈AT(x) g(T(y)|θ)h(y) =

g(T(x)|θ)h(x) g(T(x)|θ) ∑

Ay∈T(x) h(y)

= h(x) ∑

AT(x) h(y)

Thus, T X is a sufficient statistic for , if and only if fX x g T x h x .

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 9 / 27

SLIDE 39

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization Theorem : Proof (cont’d)

.

if part (cont’d)

. . fX(x|θ) q(T(x)|θ) = g(T(x)|θ)h(x) q(T(x)|θ) = g(T(x)|θ)h(x) ∑

y∈AT(x) fX(y|θ)

= g(T(x)|θ)h(x) ∑

y∈AT(x) g(T(y)|θ)h(y) =

g(T(x)|θ)h(x) g(T(x)|θ) ∑

Ay∈T(x) h(y)

= h(x) ∑

AT(x) h(y)

Thus, T(X) is a sufficient statistic for θ, if and only if fX(x|θ) = g(T(x)|θ)h(x).

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 9 / 27

SLIDE 40

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.7 - Factorization of Normal Distribution

From Example 6.2.4, we know that fX(x|µ) = (2πσ2)−n/2 exp ( − ∑n

i=1(xi − x)2 + n(x − µ)2

2σ2 ) We can define h x , so that it does not depend on . h x

n

exp

n i

xi x Because T X X n , we have g t Pr T X t exp n t Then fX x h x g T x holds, and T X X is a sufficient statistic for by the factorization theorem.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 10 / 27

SLIDE 41

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.7 - Factorization of Normal Distribution

From Example 6.2.4, we know that fX(x|µ) = (2πσ2)−n/2 exp ( − ∑n

i=1(xi − x)2 + n(x − µ)2

2σ2 ) We can define h(x), so that it does not depend on µ. h(x) = (2πσ2)−n/2 exp ( − ∑n

i=1(xi − x)2

2σ2 ) Because T X X n , we have g t Pr T X t exp n t Then fX x h x g T x holds, and T X X is a sufficient statistic for by the factorization theorem.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 10 / 27

SLIDE 42

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.7 - Factorization of Normal Distribution

From Example 6.2.4, we know that fX(x|µ) = (2πσ2)−n/2 exp ( − ∑n

i=1(xi − x)2 + n(x − µ)2

2σ2 ) We can define h(x), so that it does not depend on µ. h(x) = (2πσ2)−n/2 exp ( − ∑n

i=1(xi − x)2

2σ2 ) Because T(X) = X ∼ N(µ, σ2/n), we have g(t|µ) = Pr(T(X) = t|µ) = exp ( −n(t − µ)2 2σ2 ) Then fX x h x g T x holds, and T X X is a sufficient statistic for by the factorization theorem.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 10 / 27

SLIDE 43

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.7 - Factorization of Normal Distribution

From Example 6.2.4, we know that fX(x|µ) = (2πσ2)−n/2 exp ( − ∑n

i=1(xi − x)2 + n(x − µ)2

2σ2 ) We can define h(x), so that it does not depend on µ. h(x) = (2πσ2)−n/2 exp ( − ∑n

i=1(xi − x)2

2σ2 ) Because T(X) = X ∼ N(µ, σ2/n), we have g(t|µ) = Pr(T(X) = t|µ) = exp ( −n(t − µ)2 2σ2 ) Then fX(x|µ) = h(x)g(T(x)|µ) holds, and T(X) = X is a sufficient statistic for µ by the factorization theorem.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 10 / 27

SLIDE 44

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.8 - Uniform Sufficient Statistic

.

Problem

. .

X1, · · · , Xn are iid observations uniformly drawn from {1, · · · , θ}.

fX(x|θ) = { 1

θ

x = 1, 2, · · · , θ

therwise
Find a sufficient statistic for θ using factorization theorem.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 11 / 27

SLIDE 45

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.8 - Uniform Sufficient Statistic

.

Joint pmf

. . The joint pmf of X1, · · · , Xn is fX(x|θ) = { θ−n x ∈ {1, 2, · · · , θ}n

therwise

.

Define h x

. . . . . . . . h x x

n

therwise

Note that h x is independent of .

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 12 / 27

SLIDE 46

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.8 - Uniform Sufficient Statistic

.

Joint pmf

. . The joint pmf of X1, · · · , Xn is fX(x|θ) = { θ−n x ∈ {1, 2, · · · , θ}n

therwise

.

Define h(x)

. . h(x) = { 1 x ∈ {1, 2, · · · }n

therwise

Note that h(x) is independent of θ.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 12 / 27

SLIDE 47

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.8 - Uniform Sufficient Statistic

.

Define T(X) and g(t|θ)

. . Define T(X) = maxi xi, then g(t|θ) = Pr(T(x) = t|θ) = Pr(max

i

xi = t|θ) = { θ−n t ≤ θ

therwise

.

Putting things together

. . . . . . . .

fX x

g T x h x holds.

Thus, by the factorization theorem, T X

maxi Xi is a sufficient statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 13 / 27

SLIDE 48

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.8 - Uniform Sufficient Statistic

.

Define T(X) and g(t|θ)

. . Define T(X) = maxi xi, then g(t|θ) = Pr(T(x) = t|θ) = Pr(max

i

xi = t|θ) = { θ−n t ≤ θ

therwise

.

Putting things together

. .

fX(x|θ) = g(T(x)|θ)h(x) holds.
Thus, by the factorization theorem, T X

maxi Xi is a sufficient statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 13 / 27

SLIDE 49

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.8 - Uniform Sufficient Statistic

.

Define T(X) and g(t|θ)

. . Define T(X) = maxi xi, then g(t|θ) = Pr(T(x) = t|θ) = Pr(max

i

xi = t|θ) = { θ−n t ≤ θ

therwise

.

Putting things together

. .

fX(x|θ) = g(T(x)|θ)h(x) holds.
Thus, by the factorization theorem, T(X) = maxi Xi is a sufficient

statistic for θ.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 13 / 27

SLIDE 50

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example of h(x) when θ = 5, n = 1

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 14 / 27

SLIDE 51

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example of g(x) when θ = 5, n = 1

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 15 / 27

SLIDE 52

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example of f(x) when θ = 5, n = 1

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 16 / 27

SLIDE 53

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Alternative Solution - Using Indicator Functions

IA(x) = 1 if x ∈ A, and IA(x) = 0 otherwise.
, and

fX x

n i

I xi

n n i

I xi

n i

I xi

n i

I xi I max

i

xi

n i

I xi I T x fX x

nI

T x

n i

I xi fX x can be factorized into g t

nI

t and h x

n i

I xi , and T x maxi xi is a sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 17 / 27

SLIDE 54

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Alternative Solution - Using Indicator Functions

IA(x) = 1 if x ∈ A, and IA(x) = 0 otherwise.
N = {1, 2, · · · }, and Nθ = {1, 2, · · · , θ}

fX x

n i

I xi

n n i

I xi

n i

I xi

n i

I xi I max

i

xi

n i

I xi I T x fX x

nI

T x

n i

I xi fX x can be factorized into g t

nI

t and h x

n i

I xi , and T x maxi xi is a sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 17 / 27

SLIDE 55

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Alternative Solution - Using Indicator Functions

IA(x) = 1 if x ∈ A, and IA(x) = 0 otherwise.
N = {1, 2, · · · }, and Nθ = {1, 2, · · · , θ}

fX(x|θ) =

n

∏

i=1

1 θINθ(xi) = θ−n

n

∏

i=1

INθ(xi)

n i

I xi

n i

I xi I max

i

xi

n i

I xi I T x fX x

nI

T x

n i

I xi fX x can be factorized into g t

nI

t and h x

n i

I xi , and T x maxi xi is a sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 17 / 27

SLIDE 56

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Alternative Solution - Using Indicator Functions

IA(x) = 1 if x ∈ A, and IA(x) = 0 otherwise.
N = {1, 2, · · · }, and Nθ = {1, 2, · · · , θ}

fX(x|θ) =

n

∏

i=1

1 θINθ(xi) = θ−n

n

∏

i=1

INθ(xi)

n

∏

i=1

INθ(xi) = ( n ∏

i=1

IN(xi) ) INθ [ max

i

xi ] = ( n ∏

i=1

IN(xi) ) INθ [T(x)] fX x

nI

T x

n i

I xi fX x can be factorized into g t

nI

t and h x

n i

I xi , and T x maxi xi is a sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 17 / 27

SLIDE 57

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Alternative Solution - Using Indicator Functions

IA(x) = 1 if x ∈ A, and IA(x) = 0 otherwise.
N = {1, 2, · · · }, and Nθ = {1, 2, · · · , θ}

fX(x|θ) =

n

∏

i=1

1 θINθ(xi) = θ−n

n

∏

i=1

INθ(xi)

n

∏

i=1

INθ(xi) = ( n ∏

i=1

IN(xi) ) INθ [ max

i

xi ] = ( n ∏

i=1

IN(xi) ) INθ [T(x)] fX(x|θ) = θ−nINθ [T(x)]

n

∏

i=1

IN(xi) fX x can be factorized into g t

nI

t and h x

n i

I xi , and T x maxi xi is a sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 17 / 27

SLIDE 58

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Alternative Solution - Using Indicator Functions

IA(x) = 1 if x ∈ A, and IA(x) = 0 otherwise.
N = {1, 2, · · · }, and Nθ = {1, 2, · · · , θ}

fX(x|θ) =

n

∏

i=1

1 θINθ(xi) = θ−n

n

∏

i=1

INθ(xi)

n

∏

i=1

INθ(xi) = ( n ∏

i=1

IN(xi) ) INθ [ max

i

xi ] = ( n ∏

i=1

IN(xi) ) INθ [T(x)] fX(x|θ) = θ−nINθ [T(x)]

n

∏

i=1

IN(xi) fX(x|θ) can be factorized into g(t|θ) = θ−nINθ(t) and h(x) = ∏n

i=1 IN(xi),

and T(x) = maxi xi is a sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 17 / 27

SLIDE 59

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.9 - Normal Sufficient Statistic

.

Problem

. .

X1, · · · , Xn

i.i.d.

∼ N(µ, σ2)

Both

and are unknown

The parameter is a vector :

.

The problem is to use the Factorization Theorem to find the sufficient

statistics for . .

How to solve it

. . . . . . . .

Propose T X

T X T X as sufficient statistic for and .

Use Factorization Theorem to decompose fX x

.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 18 / 27

SLIDE 60

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.9 - Normal Sufficient Statistic

.

Problem

. .

X1, · · · , Xn

i.i.d.

∼ N(µ, σ2)

Both µ and σ2 are unknown
The parameter is a vector :

.

The problem is to use the Factorization Theorem to find the sufficient

statistics for . .

How to solve it

. . . . . . . .

Propose T X

T X T X as sufficient statistic for and .

Use Factorization Theorem to decompose fX x

.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 18 / 27

SLIDE 61

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.9 - Normal Sufficient Statistic

.

Problem

. .

X1, · · · , Xn

i.i.d.

∼ N(µ, σ2)

Both µ and σ2 are unknown
The parameter is a vector : θ = (µ, σ2).
The problem is to use the Factorization Theorem to find the sufficient

statistics for . .

How to solve it

. . . . . . . .

Propose T X

T X T X as sufficient statistic for and .

Use Factorization Theorem to decompose fX x

.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 18 / 27

SLIDE 62

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.9 - Normal Sufficient Statistic

.

Problem

. .

X1, · · · , Xn

i.i.d.

∼ N(µ, σ2)

Both µ and σ2 are unknown
The parameter is a vector : θ = (µ, σ2).
The problem is to use the Factorization Theorem to find the sufficient

statistics for θ. .

How to solve it

. . . . . . . .

Propose T X

T X T X as sufficient statistic for and .

Use Factorization Theorem to decompose fX x

.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 18 / 27

SLIDE 63

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.9 - Normal Sufficient Statistic

.

Problem

. .

X1, · · · , Xn

i.i.d.

∼ N(µ, σ2)

Both µ and σ2 are unknown
The parameter is a vector : θ = (µ, σ2).
The problem is to use the Factorization Theorem to find the sufficient

statistics for θ. .

How to solve it

. .

Propose T(X) = (T1(X), T2(X)) as sufficient statistic for µ and σ2.
Use Factorization Theorem to decompose fX x

.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 18 / 27

SLIDE 64

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.9 - Normal Sufficient Statistic

.

Problem

. .

X1, · · · , Xn

i.i.d.

∼ N(µ, σ2)

Both µ and σ2 are unknown
The parameter is a vector : θ = (µ, σ2).
The problem is to use the Factorization Theorem to find the sufficient

statistics for θ. .

How to solve it

. .

Propose T(X) = (T1(X), T2(X)) as sufficient statistic for µ and σ2.
Use Factorization Theorem to decompose fX(x|µ, σ2).

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 18 / 27

SLIDE 65

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.9 - Normal Sufficient Statistic

.

Problem

. .

X1, · · · , Xn

i.i.d.

∼ N(µ, σ2)

Both µ and σ2 are unknown
The parameter is a vector : θ = (µ, σ2).
The problem is to use the Factorization Theorem to find the sufficient

statistics for θ. .

How to solve it

. .

Propose T(X) = (T1(X), T2(X)) as sufficient statistic for µ and σ2.
Use Factorization Theorem to decompose fX(x|µ, σ2).

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 18 / 27

SLIDE 66

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.9 - Solution

.

Decomposing fX(x|µ, σ2) - Similarly to Example 6.2.4

. . fX(x|µ, σ2) =

n

∏

i=1

1 √ 2πσ2 exp ( −(xi − µ)2 2σ2 )

n

exp

n i

xi

n

exp

n i

xi x x

n

exp

n i

xi x n x

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 19 / 27

SLIDE 67

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.9 - Solution

.

Decomposing fX(x|µ, σ2) - Similarly to Example 6.2.4

. . fX(x|µ, σ2) =

n

∏

i=1

1 √ 2πσ2 exp ( −(xi − µ)2 2σ2 ) = (2πσ2)−n/2 exp ( −

n

∑

i=1

(xi − µ)2 2σ2 )

n

exp

n i

xi x x

n

exp

n i

xi x n x

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 19 / 27

SLIDE 68

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.9 - Solution

.

Decomposing fX(x|µ, σ2) - Similarly to Example 6.2.4

. . fX(x|µ, σ2) =

n

∏

i=1

1 √ 2πσ2 exp ( −(xi − µ)2 2σ2 ) = (2πσ2)−n/2 exp ( −

n

∑

i=1

(xi − µ)2 2σ2 ) = (2πσ2)−n/2 exp ( −

n

∑

i=1

(xi − x + x − µ)2 2σ2 )

n

exp

n i

xi x n x

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 19 / 27

SLIDE 69

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.9 - Solution

.

Decomposing fX(x|µ, σ2) - Similarly to Example 6.2.4

. . fX(x|µ, σ2) =

n

∏

i=1

1 √ 2πσ2 exp ( −(xi − µ)2 2σ2 ) = (2πσ2)−n/2 exp ( −

n

∑

i=1

(xi − µ)2 2σ2 ) = (2πσ2)−n/2 exp ( −

n

∑

i=1

(xi − x + x − µ)2 2σ2 ) = (2πσ2)−n/2 exp ( − 1 2σ2

n

∑

i=1

(xi − x)2 − n 2σ2 (x − µ)2 )

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 19 / 27

SLIDE 70

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.9 - Solution

.

Propose a sufficient statistic

. . fX(x|µ, σ2) = (2πσ2)−n/2 exp ( − 1 2σ2

n

∑

i=1

(xi − x)2 − n 2σ2 (x − µ)2 ) T X T X T X T x x n

n i

xi T x

n i

xi x

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 20 / 27

SLIDE 71

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.9 - Solution

.

Propose a sufficient statistic

. . fX(x|µ, σ2) = (2πσ2)−n/2 exp ( − 1 2σ2

n

∑

i=1

(xi − x)2 − n 2σ2 (x − µ)2 ) T(X) = (T1(X), T2(X)) T x x n

n i

xi T x

n i

xi x

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 20 / 27

SLIDE 72

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.9 - Solution

.

Propose a sufficient statistic

. . fX(x|µ, σ2) = (2πσ2)−n/2 exp ( − 1 2σ2

n

∑

i=1

(xi − x)2 − n 2σ2 (x − µ)2 ) T(X) = (T1(X), T2(X)) T1(x) = x = 1 n

n

∑

i=1

xi T x

n i

xi x

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 20 / 27

SLIDE 73

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.9 - Solution

.

Propose a sufficient statistic

. . fX(x|µ, σ2) = (2πσ2)−n/2 exp ( − 1 2σ2

n

∑

i=1

(xi − x)2 − n 2σ2 (x − µ)2 ) T(X) = (T1(X), T2(X)) T1(x) = x = 1 n

n

∑

i=1

xi T2(x) =

n

∑

i=1

(xi − x)2

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 20 / 27

SLIDE 74

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.9 - Solution

.

Propose a sufficient statistic

. . fX(x|µ, σ2) = (2πσ2)−n/2 exp ( − 1 2σ2

n

∑

i=1

(xi − x)2 − n 2σ2 (x − µ)2 ) T(X) = (T1(X), T2(X)) T1(x) = x = 1 n

n

∑

i=1

xi T2(x) =

n

∑

i=1

(xi − x)2

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 20 / 27

SLIDE 75

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.9 - Solution

.

Factorize fX(x|µ, σ2)

. . fX(x|µ, σ2) = (2πσ2)−n/2 exp ( − 1 2σ2

n

∑

i=1

(xi − x)2 − n 2σ2 (x − µ)2 ) h x g t t

n

exp t n t fX x g T x T x h x Thus, T X T x T x x

n i

xi x is a sufficient statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 21 / 27

SLIDE 76

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.9 - Solution

.

Factorize fX(x|µ, σ2)

. . fX(x|µ, σ2) = (2πσ2)−n/2 exp ( − 1 2σ2

n

∑

i=1

(xi − x)2 − n 2σ2 (x − µ)2 ) h(x) = 1 g t t

n

exp t n t fX x g T x T x h x Thus, T X T x T x x

n i

xi x is a sufficient statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 21 / 27

SLIDE 77

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.9 - Solution

.

Factorize fX(x|µ, σ2)

. . fX(x|µ, σ2) = (2πσ2)−n/2 exp ( − 1 2σ2

n

∑

i=1

(xi − x)2 − n 2σ2 (x − µ)2 ) h(x) = 1 g(t1, t2|µ, σ2) = (2πσ2)−n/2 exp ( − 1 2σ2 t2 − n 2σ2 (t1 − µ)2 ) fX x g T x T x h x Thus, T X T x T x x

n i

xi x is a sufficient statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 21 / 27

SLIDE 78

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.9 - Solution

.

Factorize fX(x|µ, σ2)

. . fX(x|µ, σ2) = (2πσ2)−n/2 exp ( − 1 2σ2

n

∑

i=1

(xi − x)2 − n 2σ2 (x − µ)2 ) h(x) = 1 g(t1, t2|µ, σ2) = (2πσ2)−n/2 exp ( − 1 2σ2 t2 − n 2σ2 (t1 − µ)2 ) fX(x|µ, σ2) = g(T1(x), T2(x)|µ, σ2)h(x) Thus, T X T x T x x

n i

xi x is a sufficient statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 21 / 27

SLIDE 79

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Example 6.2.9 - Solution

.

Factorize fX(x|µ, σ2)

. . fX(x|µ, σ2) = (2πσ2)−n/2 exp ( − 1 2σ2

n

∑

i=1

(xi − x)2 − n 2σ2 (x − µ)2 ) h(x) = 1 g(t1, t2|µ, σ2) = (2πσ2)−n/2 exp ( − 1 2σ2 t2 − n 2σ2 (t1 − µ)2 ) fX(x|µ, σ2) = g(T1(x), T2(x)|µ, σ2)h(x) Thus, T(X) = (T1(x), T2(x)) = ( x, ∑n

i=1(xi − x)2)

is a sufficient statistic for θ = (µ, σ2).

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 21 / 27

SLIDE 80

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

One parameter, two-dimensional sufficient statistic

.

Problem

. . Assume X1, · · · , Xn

i.i.d.

∼ Uniform(θ, θ + 1), −∞ < θ < ∞. Find a

sufficient statistic for θ. .

Rewriting fX x

. . . . . . . . fX x if x

therwise

I x fX x

n i

I xi I x xn I min

i

xi max

i

xi

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 22 / 27

SLIDE 81

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

One parameter, two-dimensional sufficient statistic

.

Problem

. . Assume X1, · · · , Xn

i.i.d.

∼ Uniform(θ, θ + 1), −∞ < θ < ∞. Find a

sufficient statistic for θ. .

Rewriting fX(x|θ)

. . fX(x|θ) = { 1 if θ < x < θ + 1

therwise

= I(θ < x < θ + 1) fX x

n i

I xi I x xn I min

i

xi max

i

xi

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 22 / 27

SLIDE 82

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

One parameter, two-dimensional sufficient statistic

.

Problem

. . Assume X1, · · · , Xn

i.i.d.

∼ Uniform(θ, θ + 1), −∞ < θ < ∞. Find a

sufficient statistic for θ. .

Rewriting fX(x|θ)

. . fX(x|θ) = { 1 if θ < x < θ + 1

therwise

= I(θ < x < θ + 1) fX(x|θ) =

n

∏

i=1

I(θ < xi < θ + 1) I x xn I min

i

xi max

i

xi

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 22 / 27

SLIDE 83

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

One parameter, two-dimensional sufficient statistic

.

Problem

. . Assume X1, · · · , Xn

i.i.d.

∼ Uniform(θ, θ + 1), −∞ < θ < ∞. Find a

sufficient statistic for θ. .

Rewriting fX(x|θ)

. . fX(x|θ) = { 1 if θ < x < θ + 1

therwise

= I(θ < x < θ + 1) fX(x|θ) =

n

∏

i=1

I(θ < xi < θ + 1) = I(θ < x1 < θ + 1, · · · , θ < xn < θ + 1) I min

i

xi max

i

xi

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 22 / 27

SLIDE 84

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

One parameter, two-dimensional sufficient statistic

.

Problem

. . Assume X1, · · · , Xn

i.i.d.

∼ Uniform(θ, θ + 1), −∞ < θ < ∞. Find a

sufficient statistic for θ. .

Rewriting fX(x|θ)

. . fX(x|θ) = { 1 if θ < x < θ + 1

therwise

= I(θ < x < θ + 1) fX(x|θ) =

n

∏

i=1

I(θ < xi < θ + 1) = I(θ < x1 < θ + 1, · · · , θ < xn < θ + 1) = I ( min

i

xi > θ ∧ max

i

xi < θ + 1 )

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 22 / 27

SLIDE 85

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

One parameter, two-dimensional sufficient statistic

.

Factorization

. . h(x) = 1 T x min

i

xi T x max

i

xi g t t I t t fX x I min

i

xi max

i

g T x T x h x Thus, T x T x T x mini xi maxi xi is a sufficient statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 23 / 27

SLIDE 86

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

One parameter, two-dimensional sufficient statistic

.

Factorization

. . h(x) = 1 T1(x) = min

i

xi T2(x) = max

i

xi g t t I t t fX x I min

i

xi max

i

g T x T x h x Thus, T x T x T x mini xi maxi xi is a sufficient statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 23 / 27

SLIDE 87

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

One parameter, two-dimensional sufficient statistic

.

Factorization

. . h(x) = 1 T1(x) = min

i

xi T2(x) = max

i

xi g(t1, t2|θ) = I(t1 > θ ∧ t2 < θ + 1) fX x I min

i

xi max

i

g T x T x h x Thus, T x T x T x mini xi maxi xi is a sufficient statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 23 / 27

SLIDE 88

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

One parameter, two-dimensional sufficient statistic

.

Factorization

. . h(x) = 1 T1(x) = min

i

xi T2(x) = max

i

xi g(t1, t2|θ) = I(t1 > θ ∧ t2 < θ + 1) fX(x|θ) = I ( min

i

xi > θ ∧ max

i

< θ + 1 ) = g(T1(x), T2(x)|θ)h(x) Thus, T x T x T x mini xi maxi xi is a sufficient statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 23 / 27

SLIDE 89

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

One parameter, two-dimensional sufficient statistic

.

Factorization

. . h(x) = 1 T1(x) = min

i

xi T2(x) = max

i

xi g(t1, t2|θ) = I(t1 > θ ∧ t2 < θ + 1) fX(x|θ) = I ( min

i

xi > θ ∧ max

i

< θ + 1 ) = g(T1(x), T2(x)|θ)h(x) Thus, T(x) = (T1(x), T2(x)) = (mini xi, maxi xi) is a sufficient statistic for θ.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 23 / 27

SLIDE 90

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Sufficient Order Statistics

.

Problem

. .

X1, · · · , Xn

i.i.d.

∼ fX(x|θ).

fX(x|θ) = ∏n

i=1 fX(xi|θ)

Define order statistics x

x n as an ordered permutation of x

Is the order statistic a sufficient statistic for

? T x T x Tn x x x n

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 24 / 27

SLIDE 91

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Sufficient Order Statistics

.

Problem

. .

X1, · · · , Xn

i.i.d.

∼ fX(x|θ).

fX(x|θ) = ∏n

i=1 fX(xi|θ)

Define order statistics x(1) ≤ · · · ≤ x(n) as an ordered permutation of

x

Is the order statistic a sufficient statistic for

? T x T x Tn x x x n

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 24 / 27

SLIDE 92

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Sufficient Order Statistics

.

Problem

. .

X1, · · · , Xn

i.i.d.

∼ fX(x|θ).

fX(x|θ) = ∏n

i=1 fX(xi|θ)

Define order statistics x(1) ≤ · · · ≤ x(n) as an ordered permutation of

x

Is the order statistic a sufficient statistic for θ?

T(x) = (T1(x), · · · , Tn(x)) = (x(1), · · · , x(n))

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 24 / 27

SLIDE 93

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization of Order Statistics

h(x) = 1 g t tn

n i

fX ti fX x g T x Tn x h x (Note that T x Tn x is a permutation of x xn ) Therefore, T x x x n is a sufficient statistics for .

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 25 / 27

SLIDE 94

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization of Order Statistics

h(x) = 1 g(t1, · · · , tn|θ) =

n

∏

i=1

fX(ti|θ) fX x g T x Tn x h x (Note that T x Tn x is a permutation of x xn ) Therefore, T x x x n is a sufficient statistics for .

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 25 / 27

SLIDE 95

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Factorization of Order Statistics

h(x) = 1 g(t1, · · · , tn|θ) =

n

∏

i=1

fX(ti|θ) fX(x|θ) = g(T1(x), · · · , Tn(x)|θ)h(x) (Note that (T1(x), · · · , Tn(x)) is a permutation of (x1, · · · , xn)) Therefore, T(x) = (x(1), · · · , x(n)) is a sufficient statistics for θ.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 25 / 27

SLIDE 96

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Exercise 6.1

.

Problem

. . X is one observation from a N(0, σ2). Is |X| a sufficient statistic for σ2? .

Solution

. . . . . . . . fX x exp x Define h x T x x g t exp t Then fX x g T x h x holds, and T X X is a sufficient statistic by the Factorization Theorem.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 26 / 27

SLIDE 97

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Exercise 6.1

.

Problem

. . X is one observation from a N(0, σ2). Is |X| a sufficient statistic for σ2? .

Solution

. . fX(x|σ2) = 1 √ 2πσ2 exp ( − x2 2σ2 ) Define h(x) = 1 T x x g t exp t Then fX x g T x h x holds, and T X X is a sufficient statistic by the Factorization Theorem.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 26 / 27

SLIDE 98

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Exercise 6.1

.

Problem

. . X is one observation from a N(0, σ2). Is |X| a sufficient statistic for σ2? .

Solution

. . fX(x|σ2) = 1 √ 2πσ2 exp ( − x2 2σ2 ) Define h(x) = 1 T(x) = |x| g t exp t Then fX x g T x h x holds, and T X X is a sufficient statistic by the Factorization Theorem.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 26 / 27

SLIDE 99

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Exercise 6.1

.

Problem

. . X is one observation from a N(0, σ2). Is |X| a sufficient statistic for σ2? .

Solution

. . fX(x|σ2) = 1 √ 2πσ2 exp ( − x2 2σ2 ) Define h(x) = 1 T(x) = |x| g(t|θ) = 1 √ 2πσ2 exp ( − t2 2σ2 ) Then fX x g T x h x holds, and T X X is a sufficient statistic by the Factorization Theorem.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 26 / 27

SLIDE 100

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Exercise 6.1

.

Problem

. . X is one observation from a N(0, σ2). Is |X| a sufficient statistic for σ2? .

Solution

. . fX(x|σ2) = 1 √ 2πσ2 exp ( − x2 2σ2 ) Define h(x) = 1 T(x) = |x| g(t|θ) = 1 √ 2πσ2 exp ( − t2 2σ2 ) Then fX(x|θ) = g(T(x)|θ)h(x) holds, and T(X) = |X| is a sufficient statistic by the Factorization Theorem.

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 26 / 27

SLIDE 101

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Summary

.

Today : Factorization Theorem

. .

fX(x|θ) = g(T(x)|θ)h(x)
Necessary and sufficient condition of a sufficient statistic
Uniform sufficient statistic : maximum of observations
Normal distribution : multidimensional sufficient statistic
One parameter, two dimensional sufficient statistics

.

Next Lecture

. . . . . . . . • Minimal Sufficient Statistics

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 27 / 27

SLIDE 102

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . Factorization Theorem . Summary

Summary

.

Today : Factorization Theorem

. .

fX(x|θ) = g(T(x)|θ)h(x)
Necessary and sufficient condition of a sufficient statistic
Uniform sufficient statistic : maximum of observations
Normal distribution : multidimensional sufficient statistic
One parameter, two dimensional sufficient statistics

.

Next Lecture

. . • Minimal Sufficient Statistics

Hyun Min Kang Biostatistics 602 - Lecture 02 January 15th, 2013 27 / 27