[PPT] - Minimal Sufficient Statistics Lecture 03 Biostatistics 602 - PowerPoint Presentation

SLIDE 1

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

. .

Biostatistics 602 - Statistical Inference Lecture 03 Minimal Sufficient Statistics

Hyun Min Kang January 17th, 2013

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 1 / 25

SLIDE 2

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Last Lecture - Key Questions

. . 1 How do we show that a statistic is sufficient for θ? . . 2 What is a necessary and sufficient condition for a statistic to be

sufficient for ?

. . 3 What is an effective strategy to find sufficient statistics using the

Factorization Theorem?

. . 4 Is the dimension of a sufficient statistic the always same to the

dimension of the parameters?

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 2 / 25

SLIDE 3

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Last Lecture - Key Questions

. . 1 How do we show that a statistic is sufficient for θ? . . 2 What is a necessary and sufficient condition for a statistic to be

sufficient for θ?

. . 3 What is an effective strategy to find sufficient statistics using the

Factorization Theorem?

. . 4 Is the dimension of a sufficient statistic the always same to the

dimension of the parameters?

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 2 / 25

SLIDE 4

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Last Lecture - Key Questions

. . 1 How do we show that a statistic is sufficient for θ? . . 2 What is a necessary and sufficient condition for a statistic to be

sufficient for θ?

. . 3 What is an effective strategy to find sufficient statistics using the

Factorization Theorem?

. . 4 Is the dimension of a sufficient statistic the always same to the

dimension of the parameters?

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 2 / 25

SLIDE 5

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Last Lecture - Key Questions

. . 1 How do we show that a statistic is sufficient for θ? . . 2 What is a necessary and sufficient condition for a statistic to be

sufficient for θ?

. . 3 What is an effective strategy to find sufficient statistics using the

Factorization Theorem?

. . 4 Is the dimension of a sufficient statistic the always same to the

dimension of the parameters?

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 2 / 25

SLIDE 6

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Recap - Sufficient Statistic

.

Definition 6.2.1

. . A statistic T(X) is a sufficient statistic for θ if the conditional distribution

f sample X given the value of T(X) does not depend on θ.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 3 / 25

SLIDE 7

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Recap - A Theorem for Sufficient Statistics

.

Theorem 6.2.2

. .

Let fX(x|θ) is a joint pdf or pmf of X
and q(t|θ) is the pdf or pmf of T(X).
Then T(X) is a sufficient statistic for θ,
if, for every x ∈ X,
the ratio fX(x|θ)/q(T(x)|θ) is constant as a function of θ.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 4 / 25

SLIDE 8

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Recap - Factorization Theorem

.

Theorem 6.2.6 - Factorization Theorem

. .

Let fX(x|θ) denote the joint pdf or pmf of a sample X.
A statistic T(X) is a sufficient statistic for θ, if and only if
There exists function g(t|θ) and h(x) such that,
for all sample points x,
and for all parameter points θ,
fX(x|θ) = g(T(x)|θ)h(x).

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 5 / 25

SLIDE 9

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Minimal Sufficient Statistic

.

Sufficient statistics are not unique

. .

T(x) = x : The random sample itself is a trivial sufficient statistic for

any θ.

An ordered statistic X

X n is always a sufficient statistic for , if X Xn are iid.

For any sufficient statistic T X , its one-to-one function q T X

is also a sufficient statistic for . .

Question

. . . . . . . . Can we find a sufficient statistic that achieves the maximum data reduction?

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 6 / 25

SLIDE 10

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Minimal Sufficient Statistic

.

Sufficient statistics are not unique

. .

T(x) = x : The random sample itself is a trivial sufficient statistic for

any θ.

An ordered statistic (X(1), · · · , X(n)) is always a sufficient statistic for

θ, if X1, · · · , Xn are iid.

For any sufficient statistic T X , its one-to-one function q T X

is also a sufficient statistic for . .

Question

. . . . . . . . Can we find a sufficient statistic that achieves the maximum data reduction?

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 6 / 25

SLIDE 11

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Minimal Sufficient Statistic

.

Sufficient statistics are not unique

. .

T(x) = x : The random sample itself is a trivial sufficient statistic for

any θ.

An ordered statistic (X(1), · · · , X(n)) is always a sufficient statistic for

θ, if X1, · · · , Xn are iid.

For any sufficient statistic T(X), its one-to-one function q(T(X)) is

also a sufficient statistic for θ. .

Question

. . . . . . . . Can we find a sufficient statistic that achieves the maximum data reduction?

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 6 / 25

SLIDE 12

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Minimal Sufficient Statistic

.

Sufficient statistics are not unique

. .

T(x) = x : The random sample itself is a trivial sufficient statistic for

any θ.

An ordered statistic (X(1), · · · , X(n)) is always a sufficient statistic for

θ, if X1, · · · , Xn are iid.

For any sufficient statistic T(X), its one-to-one function q(T(X)) is

also a sufficient statistic for θ. .

Question

. . Can we find a sufficient statistic that achieves the maximum data reduction?

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 6 / 25

SLIDE 13

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Minimal Sufficient Statistic

.

Definition 6.2.11

. . A sufficient statistic T(X) is called a minimal sufficient statistic if, for any

ther sufficient statistic T′(X), T(X) is a function of T′(X).

.

Why is this called ”minimal” sufficient statistic?

. . . . . . . .

The sample space

consists of every possible sample - finest partition

Given T X ,

can be partitioned into At where t t t T X for some x

Maximum data reduction is achieved when

is minimal.

If size of

t t T x for some x is not less than , then can be called as a minimal sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 7 / 25

SLIDE 14

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Minimal Sufficient Statistic

.

Definition 6.2.11

. . A sufficient statistic T(X) is called a minimal sufficient statistic if, for any

ther sufficient statistic T′(X), T(X) is a function of T′(X).

.

Why is this called ”minimal” sufficient statistic?

. .

The sample space X consists of every possible sample - finest partition
Given T X ,

can be partitioned into At where t t t T X for some x

Maximum data reduction is achieved when

is minimal.

If size of

t t T x for some x is not less than , then can be called as a minimal sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 7 / 25

SLIDE 15

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Minimal Sufficient Statistic

.

Definition 6.2.11

. . A sufficient statistic T(X) is called a minimal sufficient statistic if, for any

ther sufficient statistic T′(X), T(X) is a function of T′(X).

.

Why is this called ”minimal” sufficient statistic?

. .

The sample space X consists of every possible sample - finest partition
Given T(X), X can be partitioned into At where

t ∈ T = {t : t = T(X) for some x ∈ X}

Maximum data reduction is achieved when

is minimal.

If size of

t t T x for some x is not less than , then can be called as a minimal sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 7 / 25

SLIDE 16

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Minimal Sufficient Statistic

.

Definition 6.2.11

. . A sufficient statistic T(X) is called a minimal sufficient statistic if, for any

ther sufficient statistic T′(X), T(X) is a function of T′(X).

.

Why is this called ”minimal” sufficient statistic?

. .

The sample space X consists of every possible sample - finest partition
Given T(X), X can be partitioned into At where

t ∈ T = {t : t = T(X) for some x ∈ X}

Maximum data reduction is achieved when |T | is minimal.
If size of

t t T x for some x is not less than , then can be called as a minimal sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 7 / 25

SLIDE 17

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Minimal Sufficient Statistic

.

Definition 6.2.11

. . A sufficient statistic T(X) is called a minimal sufficient statistic if, for any

ther sufficient statistic T′(X), T(X) is a function of T′(X).

.

Why is this called ”minimal” sufficient statistic?

. .

The sample space X consists of every possible sample - finest partition
Given T(X), X can be partitioned into At where

t ∈ T = {t : t = T(X) for some x ∈ X}

Maximum data reduction is achieved when |T | is minimal.
If size of T ′ = t : t = T′(x) for some x ∈ X is not less than |T |, then

|T | can be called as a minimal sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 7 / 25

SLIDE 18

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Theorem for Minimal Sufficient Statistics

.

Theorem 6.2.13

. .

fX(x) be pmf or pdf of a sample X.
Suppose that there exists a function T x such that,
For every two sample points x and y,
The ratio fX x

fX y is constant as a function of if and only if T x T y .

Then T X is a minimal sufficient statistic for

. .

In other words..

. . . . . . . .

fX x

fX y is constant as a function of = T x T y .

T x

T y = fX x fX y is constant as a function of

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 8 / 25

SLIDE 19

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Theorem for Minimal Sufficient Statistics

.

Theorem 6.2.13

. .

fX(x) be pmf or pdf of a sample X.
Suppose that there exists a function T(x) such that,
For every two sample points x and y,
The ratio fX x

fX y is constant as a function of if and only if T x T y .

Then T X is a minimal sufficient statistic for

. .

In other words..

. . . . . . . .

fX x

fX y is constant as a function of = T x T y .

T x

T y = fX x fX y is constant as a function of

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 8 / 25

SLIDE 20

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Theorem for Minimal Sufficient Statistics

.

Theorem 6.2.13

. .

fX(x) be pmf or pdf of a sample X.
Suppose that there exists a function T(x) such that,
For every two sample points x and y,
The ratio fX x

fX y is constant as a function of if and only if T x T y .

Then T X is a minimal sufficient statistic for

. .

In other words..

. . . . . . . .

fX x

fX y is constant as a function of = T x T y .

T x

T y = fX x fX y is constant as a function of

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 8 / 25

SLIDE 21

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Theorem for Minimal Sufficient Statistics

.

Theorem 6.2.13

. .

fX(x) be pmf or pdf of a sample X.
Suppose that there exists a function T(x) such that,
For every two sample points x and y,
The ratio fX(x|θ)/fX(y|θ) is constant as a function of θ if and only if

T(x) = T(y).

Then T X is a minimal sufficient statistic for

. .

In other words..

. . . . . . . .

fX x

fX y is constant as a function of = T x T y .

T x

T y = fX x fX y is constant as a function of

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 8 / 25

SLIDE 22

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Theorem for Minimal Sufficient Statistics

.

Theorem 6.2.13

. .

fX(x) be pmf or pdf of a sample X.
Suppose that there exists a function T(x) such that,
For every two sample points x and y,
The ratio fX(x|θ)/fX(y|θ) is constant as a function of θ if and only if

T(x) = T(y).

Then T(X) is a minimal sufficient statistic for θ.

.

In other words..

. . . . . . . .

fX x

fX y is constant as a function of = T x T y .

T x

T y = fX x fX y is constant as a function of

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 8 / 25

SLIDE 23

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Theorem for Minimal Sufficient Statistics

.

Theorem 6.2.13

. .

fX(x) be pmf or pdf of a sample X.
Suppose that there exists a function T(x) such that,
For every two sample points x and y,
The ratio fX(x|θ)/fX(y|θ) is constant as a function of θ if and only if

T(x) = T(y).

Then T(X) is a minimal sufficient statistic for θ.

.

In other words..

. .

fX(x|θ)/fX(y|θ) is constant as a function of θ =

⇒ T(x) = T(y).

T x

T y = fX x fX y is constant as a function of

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 8 / 25

SLIDE 24

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Theorem for Minimal Sufficient Statistics

.

Theorem 6.2.13

. .

fX(x) be pmf or pdf of a sample X.
Suppose that there exists a function T(x) such that,
For every two sample points x and y,
The ratio fX(x|θ)/fX(y|θ) is constant as a function of θ if and only if

T(x) = T(y).

Then T(X) is a minimal sufficient statistic for θ.

.

In other words..

. .

fX(x|θ)/fX(y|θ) is constant as a function of θ =

⇒ T(x) = T(y).

T(x) = T(y) =

⇒ fX(x|θ)/fX(y|θ) is constant as a function of θ

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 8 / 25

SLIDE 25

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Example from the first lecture

.

Problem

. .

X1, X2, X3

i.i.d.

∼ Bernoulli(p)

Q1: Is T1(X) = (X1 + X2, X3) a sufficient statistic for p?
Q2: Is T2(X) = X1 + X2 + X3 a minimal sufficient statistic for p?
Q3: Is T1(X) = (X1 + X2, X3) a minimal sufficient statistic for p?

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 9 / 25

SLIDE 26

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Is T1(X) = (X1 + X2, X3) a sufficient statistic?

fX(x|p) = px1+x2+x3(1 − p)3−x1−x2−x3 px

x

p

x x px

p

x

h x g t t p pt p

t pt

p

t

fX x p g x x x p h x By Factorization Theorem, T X X X X is a sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 10 / 25

SLIDE 27

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Is T1(X) = (X1 + X2, X3) a sufficient statistic?

fX(x|p) = px1+x2+x3(1 − p)3−x1−x2−x3 = px1+x2(1 − p)2−x1−x2px3(1 − p)1−x3 h x g t t p pt p

t pt

p

t

fX x p g x x x p h x By Factorization Theorem, T X X X X is a sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 10 / 25

SLIDE 28

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Is T1(X) = (X1 + X2, X3) a sufficient statistic?

fX(x|p) = px1+x2+x3(1 − p)3−x1−x2−x3 = px1+x2(1 − p)2−x1−x2px3(1 − p)1−x3 h(x) = 1 g t t p pt p

t pt

p

t

fX x p g x x x p h x By Factorization Theorem, T X X X X is a sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 10 / 25

SLIDE 29

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Is T1(X) = (X1 + X2, X3) a sufficient statistic?

fX(x|p) = px1+x2+x3(1 − p)3−x1−x2−x3 = px1+x2(1 − p)2−x1−x2px3(1 − p)1−x3 h(x) = 1 g(t1, t2|p) = pt1(1 − p)2−t1pt2(1 − p)1−t2 fX x p g x x x p h x By Factorization Theorem, T X X X X is a sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 10 / 25

SLIDE 30

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Is T1(X) = (X1 + X2, X3) a sufficient statistic?

fX(x|p) = px1+x2+x3(1 − p)3−x1−x2−x3 = px1+x2(1 − p)2−x1−x2px3(1 − p)1−x3 h(x) = 1 g(t1, t2|p) = pt1(1 − p)2−t1pt2(1 − p)1−t2 fX(x|p) = g(x1 + x2, x3|p)h(x) By Factorization Theorem, T X X X X is a sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 10 / 25

SLIDE 31

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Is T1(X) = (X1 + X2, X3) a sufficient statistic?

fX(x|p) = px1+x2+x3(1 − p)3−x1−x2−x3 = px1+x2(1 − p)2−x1−x2px3(1 − p)1−x3 h(x) = 1 g(t1, t2|p) = pt1(1 − p)2−t1pt2(1 − p)1−t2 fX(x|p) = g(x1 + x2, x3|p)h(x) By Factorization Theorem, T1(X) = (X1 + X2, X3) is a sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 10 / 25

SLIDE 32

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Is T2(X) = (X1 + X2 + X3) a minimal sufficient statistic?

fX x fX y p

xi

p

xi

p

yi

p

yi

p p

xi yi

If T

x T y , i.e. xi yi, then the ratio does not depend

n p.
The ratio above is constant as a function of p only if

xi yi, i.e. T x T y . Therefore, T X Xi is a minimal sufficient statistic for p by Theorem 6.2.13.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 11 / 25

SLIDE 33

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Is T2(X) = (X1 + X2 + X3) a minimal sufficient statistic?

fX(x|θ) fX(y|θ) = p

∑ xi(1 − p)3−∑ xi

p

∑ yi(1 − p)3−∑ yi

p p

xi yi

If T

x T y , i.e. xi yi, then the ratio does not depend

n p.
The ratio above is constant as a function of p only if

xi yi, i.e. T x T y . Therefore, T X Xi is a minimal sufficient statistic for p by Theorem 6.2.13.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 11 / 25

SLIDE 34

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Is T2(X) = (X1 + X2 + X3) a minimal sufficient statistic?

fX(x|θ) fX(y|θ) = p

∑ xi(1 − p)3−∑ xi

p

∑ yi(1 − p)3−∑ yi

= ( p 1 − p )∑ xi−∑ yi

If T

x T y , i.e. xi yi, then the ratio does not depend

n p.
The ratio above is constant as a function of p only if

xi yi, i.e. T x T y . Therefore, T X Xi is a minimal sufficient statistic for p by Theorem 6.2.13.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 11 / 25

SLIDE 35

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Is T2(X) = (X1 + X2 + X3) a minimal sufficient statistic?

fX(x|θ) fX(y|θ) = p

∑ xi(1 − p)3−∑ xi

p

∑ yi(1 − p)3−∑ yi

= ( p 1 − p )∑ xi−∑ yi

If T2(x) = T2(y), i.e. ∑ xi = ∑ yi, then the ratio does not depend
n p.
The ratio above is constant as a function of p only if

xi yi, i.e. T x T y . Therefore, T X Xi is a minimal sufficient statistic for p by Theorem 6.2.13.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 11 / 25

SLIDE 36

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Is T2(X) = (X1 + X2 + X3) a minimal sufficient statistic?

fX(x|θ) fX(y|θ) = p

∑ xi(1 − p)3−∑ xi

p

∑ yi(1 − p)3−∑ yi

= ( p 1 − p )∑ xi−∑ yi

If T2(x) = T2(y), i.e. ∑ xi = ∑ yi, then the ratio does not depend
n p.
The ratio above is constant as a function of p only if ∑ xi = ∑ yi,

i.e. T2(x) = T2(y). Therefore, T2(X) = ∑ Xi is a minimal sufficient statistic for p by Theorem 6.2.13.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 11 / 25

SLIDE 37

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Is T1(X) = (X1 + X2, X3) minimal sufficient?

Let A X X X , and B X X . fX x p px

x

p

x x px

p

x

pA x p

A x pB x

p

B x

pA x

B x

p

A x B x

fX x fX y pA x

B x

p

A x B x

pA y

B y

p

A x B y

p p

A x B x A y B y

The ratio above is constant as a function of p if (but not only if)

A x A y and B x B y

Because if A x

B x A y B y , even though A x A y and B x B y , the ratio above is still constant. Therefore, T X A X B X X X X is not a minimal sufficient statistic for p by Theorem 6.2.13.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 12 / 25

SLIDE 38

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Is T1(X) = (X1 + X2, X3) minimal sufficient?

Let A(X) = X1 + X2, and B(X) = X3. fX(x|p) = px1+x2(1 − p)2−x1−x2px3(1 − p)1−x3 pA x p

A x pB x

p

B x

pA x

B x

p

A x B x

fX x fX y pA x

B x

p

A x B x

pA y

B y

p

A x B y

p p

A x B x A y B y

The ratio above is constant as a function of p if (but not only if)

A x A y and B x B y

Because if A x

B x A y B y , even though A x A y and B x B y , the ratio above is still constant. Therefore, T X A X B X X X X is not a minimal sufficient statistic for p by Theorem 6.2.13.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 12 / 25

SLIDE 39

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Is T1(X) = (X1 + X2, X3) minimal sufficient?

Let A(X) = X1 + X2, and B(X) = X3. fX(x|p) = px1+x2(1 − p)2−x1−x2px3(1 − p)1−x3 = pA(x)(1 − p)2−A(x)pB(x)(1 − p)1−B(x) pA x

B x

p

A x B x

fX x fX y pA x

B x

p

A x B x

pA y

B y

p

A x B y

p p

A x B x A y B y

The ratio above is constant as a function of p if (but not only if)

A x A y and B x B y

Because if A x

B x A y B y , even though A x A y and B x B y , the ratio above is still constant. Therefore, T X A X B X X X X is not a minimal sufficient statistic for p by Theorem 6.2.13.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 12 / 25

SLIDE 40

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Is T1(X) = (X1 + X2, X3) minimal sufficient?

Let A(X) = X1 + X2, and B(X) = X3. fX(x|p) = px1+x2(1 − p)2−x1−x2px3(1 − p)1−x3 = pA(x)(1 − p)2−A(x)pB(x)(1 − p)1−B(x) = pA(x)+B(x)(1 − p)3−A(x)−B(x) fX x fX y pA x

B x

p

A x B x

pA y

B y

p

A x B y

p p

A x B x A y B y

The ratio above is constant as a function of p if (but not only if)

A x A y and B x B y

Because if A x

B x A y B y , even though A x A y and B x B y , the ratio above is still constant. Therefore, T X A X B X X X X is not a minimal sufficient statistic for p by Theorem 6.2.13.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 12 / 25

SLIDE 41

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Is T1(X) = (X1 + X2, X3) minimal sufficient?

Let A(X) = X1 + X2, and B(X) = X3. fX(x|p) = px1+x2(1 − p)2−x1−x2px3(1 − p)1−x3 = pA(x)(1 − p)2−A(x)pB(x)(1 − p)1−B(x) = pA(x)+B(x)(1 − p)3−A(x)−B(x) fX(x|θ) fX(y|θ) = pA(x)+B(x)(1 − p)3−A(x)−B(x) pA(y)+B(y)(1 − p)3−A(x)−B(y) p p

A x B x A y B y

The ratio above is constant as a function of p if (but not only if)

A x A y and B x B y

Because if A x

B x A y B y , even though A x A y and B x B y , the ratio above is still constant. Therefore, T X A X B X X X X is not a minimal sufficient statistic for p by Theorem 6.2.13.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 12 / 25

SLIDE 42

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Is T1(X) = (X1 + X2, X3) minimal sufficient?

Let A(X) = X1 + X2, and B(X) = X3. fX(x|p) = px1+x2(1 − p)2−x1−x2px3(1 − p)1−x3 = pA(x)(1 − p)2−A(x)pB(x)(1 − p)1−B(x) = pA(x)+B(x)(1 − p)3−A(x)−B(x) fX(x|θ) fX(y|θ) = pA(x)+B(x)(1 − p)3−A(x)−B(x) pA(y)+B(y)(1 − p)3−A(x)−B(y) = ( p 1 − p )A(x)+B(x)−A(y)−B(y)

The ratio above is constant as a function of p if (but not only if)

A x A y and B x B y

Because if A x

B x A y B y , even though A x A y and B x B y , the ratio above is still constant. Therefore, T X A X B X X X X is not a minimal sufficient statistic for p by Theorem 6.2.13.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 12 / 25

SLIDE 43

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Is T1(X) = (X1 + X2, X3) minimal sufficient?

Let A(X) = X1 + X2, and B(X) = X3. fX(x|p) = px1+x2(1 − p)2−x1−x2px3(1 − p)1−x3 = pA(x)(1 − p)2−A(x)pB(x)(1 − p)1−B(x) = pA(x)+B(x)(1 − p)3−A(x)−B(x) fX(x|θ) fX(y|θ) = pA(x)+B(x)(1 − p)3−A(x)−B(x) pA(y)+B(y)(1 − p)3−A(x)−B(y) = ( p 1 − p )A(x)+B(x)−A(y)−B(y)

The ratio above is constant as a function of p if (but not only if)

A(x) = A(y) and B(x) = B(y)

Because if A x

B x A y B y , even though A x A y and B x B y , the ratio above is still constant. Therefore, T X A X B X X X X is not a minimal sufficient statistic for p by Theorem 6.2.13.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 12 / 25

SLIDE 44

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Is T1(X) = (X1 + X2, X3) minimal sufficient?

Let A(X) = X1 + X2, and B(X) = X3. fX(x|p) = px1+x2(1 − p)2−x1−x2px3(1 − p)1−x3 = pA(x)(1 − p)2−A(x)pB(x)(1 − p)1−B(x) = pA(x)+B(x)(1 − p)3−A(x)−B(x) fX(x|θ) fX(y|θ) = pA(x)+B(x)(1 − p)3−A(x)−B(x) pA(y)+B(y)(1 − p)3−A(x)−B(y) = ( p 1 − p )A(x)+B(x)−A(y)−B(y)

The ratio above is constant as a function of p if (but not only if)

A(x) = A(y) and B(x) = B(y)

Because if A(x) + B(x) = A(y) + B(y), even though A(x) ̸= A(y)

and B(x) ̸= B(y), the ratio above is still constant. Therefore, T X A X B X X X X is not a minimal sufficient statistic for p by Theorem 6.2.13.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 12 / 25

SLIDE 45

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Is T1(X) = (X1 + X2, X3) minimal sufficient?

Let A(X) = X1 + X2, and B(X) = X3. fX(x|p) = px1+x2(1 − p)2−x1−x2px3(1 − p)1−x3 = pA(x)(1 − p)2−A(x)pB(x)(1 − p)1−B(x) = pA(x)+B(x)(1 − p)3−A(x)−B(x) fX(x|θ) fX(y|θ) = pA(x)+B(x)(1 − p)3−A(x)−B(x) pA(y)+B(y)(1 − p)3−A(x)−B(y) = ( p 1 − p )A(x)+B(x)−A(y)−B(y)

The ratio above is constant as a function of p if (but not only if)

A(x) = A(y) and B(x) = B(y)

Because if A(x) + B(x) = A(y) + B(y), even though A(x) ̸= A(y)

and B(x) ̸= B(y), the ratio above is still constant. Therefore, T1(X) = (A(X), B(X)) = (X1 + X2, X3) is not a minimal sufficient statistic for p by Theorem 6.2.13.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 12 / 25

SLIDE 46

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Partition of sample space

X1 X2 X3 T1(X) = (X1 + X2, X3) T2(X) = X1 + X2 + X3 (0,0) 1 (0,1) 1 1 (1,0) 1 1 1 (1,1) 2 1 1 1 1 (2,0) 1 1 1 (2,1) 3

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 13 / 25

SLIDE 47

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Background knowledges for proving if and only if

Assume that a, b, c, d, a1, · · · , an are constants.

. . 1 aθ2 + bθ + c = 0 for any θ ∈ R

a b c .

. . 2 k i

ai i c for any a ak .

. . 3 a

b c for all a b c .

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 14 / 25

SLIDE 48

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Background knowledges for proving if and only if

Assume that a, b, c, d, a1, · · · , an are constants.

. . 1 aθ2 + bθ + c = 0 for any θ ∈ R

⇔ a = b = c = 0.

. 2 k i

ai i c for any a ak .

. . 3 a

b c for all a b c .

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 14 / 25

SLIDE 49

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Background knowledges for proving if and only if

Assume that a, b, c, d, a1, · · · , an are constants.

. . 1 aθ2 + bθ + c = 0 for any θ ∈ R

⇔ a = b = c = 0.

. . 2 ∑k i=1 aiθi = c for any θ ∈ R

a ak .

. . 3 a

b c for all a b c .

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 14 / 25

SLIDE 50

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Background knowledges for proving if and only if

Assume that a, b, c, d, a1, · · · , an are constants.

. . 1 aθ2 + bθ + c = 0 for any θ ∈ R

⇔ a = b = c = 0.

. . 2 ∑k i=1 aiθi = c for any θ ∈ R

⇔ a1 = · · · = ak = 0.

. . 3 a

b c for all a b c .

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 14 / 25

SLIDE 51

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Background knowledges for proving if and only if

Assume that a, b, c, d, a1, · · · , an are constants.

. . 1 aθ2 + bθ + c = 0 for any θ ∈ R

⇔ a = b = c = 0.

. . 2 ∑k i=1 aiθi = c for any θ ∈ R

⇔ a1 = · · · = ak = 0.

. . 3 aθ1 + bθ2 + c = 0 for all (θ1, θ2) ∈ R2

a b c .

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 14 / 25

SLIDE 52

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Background knowledges for proving if and only if

Assume that a, b, c, d, a1, · · · , an are constants.

. . 1 aθ2 + bθ + c = 0 for any θ ∈ R

⇔ a = b = c = 0.

. . 2 ∑k i=1 aiθi = c for any θ ∈ R

⇔ a1 = · · · = ak = 0.

. . 3 aθ1 + bθ2 + c = 0 for all (θ1, θ2) ∈ R2

⇔ a = b = c = 0.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 14 / 25

SLIDE 53

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Background knowledges for proving if and only if

. . 4 The following equation is constant

1 + a1θ + a2θ2 + · · · + akθk

k

1 + b1θ + b2θ2 + · · · + bkθk

k

a b ak bk. Note that this does not hold without the constant 1, for example,

. 5 I a b I c d is constant a a function of

. a c, and b d.

. . 6 t is constant function of

. t .

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 15 / 25

SLIDE 54

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Background knowledges for proving if and only if

. . 4 The following equation is constant

1 + a1θ + a2θ2 + · · · + akθk

k

1 + b1θ + b2θ2 + · · · + bkθk

k

⇔ a1 = b1, · · · , ak = bk. Note that this does not hold without the constant 1, for example,

. 5 I a b I c d is constant a a function of

. a c, and b d.

. . 6 t is constant function of

. t .

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 15 / 25

SLIDE 55

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Background knowledges for proving if and only if

. . 4 The following equation is constant

1 + a1θ + a2θ2 + · · · + akθk

k

1 + b1θ + b2θ2 + · · · + bkθk

k

⇔ a1 = b1, · · · , ak = bk. Note that this does not hold without the constant 1, for example, θ + 2θ2 2θ + 4θ2 = 1 2

. 5 I a b I c d is constant a a function of

. a c, and b d.

. . 6 t is constant function of

. t .

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 15 / 25

SLIDE 56

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Background knowledges for proving if and only if

. . 4 The following equation is constant

1 + a1θ + a2θ2 + · · · + akθk

k

1 + b1θ + b2θ2 + · · · + bkθk

k

⇔ a1 = b1, · · · , ak = bk. Note that this does not hold without the constant 1, for example, θ + 2θ2 2θ + 4θ2 = 1 2

. . 5 I(a<θ<b) I(c<θ<d) is constant a a function of θ.

a c, and b d.

. . 6 t is constant function of

. t .

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 15 / 25

SLIDE 57

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Background knowledges for proving if and only if

. . 4 The following equation is constant

1 + a1θ + a2θ2 + · · · + akθk

k

1 + b1θ + b2θ2 + · · · + bkθk

k

⇔ a1 = b1, · · · , ak = bk. Note that this does not hold without the constant 1, for example, θ + 2θ2 2θ + 4θ2 = 1 2

. . 5 I(a<θ<b) I(c<θ<d) is constant a a function of θ. ⇔ a = c, and b = d. . . 6 t is constant function of

. t .

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 15 / 25

SLIDE 58

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Background knowledges for proving if and only if

. . 4 The following equation is constant

1 + a1θ + a2θ2 + · · · + akθk

k

1 + b1θ + b2θ2 + · · · + bkθk

k

⇔ a1 = b1, · · · , ak = bk. Note that this does not hold without the constant 1, for example, θ + 2θ2 2θ + 4θ2 = 1 2

. . 5 I(a<θ<b) I(c<θ<d) is constant a a function of θ. ⇔ a = c, and b = d. . . 6 θt is constant function of θ.

t .

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 15 / 25

SLIDE 59

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Background knowledges for proving if and only if

. . 4 The following equation is constant

1 + a1θ + a2θ2 + · · · + akθk

k

1 + b1θ + b2θ2 + · · · + bkθk

k

⇔ a1 = b1, · · · , ak = bk. Note that this does not hold without the constant 1, for example, θ + 2θ2 2θ + 4θ2 = 1 2

. . 5 I(a<θ<b) I(c<θ<d) is constant a a function of θ. ⇔ a = c, and b = d. . . 6 θt is constant function of θ. ⇔ t = 0.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 15 / 25

SLIDE 60

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Uniform Minimal Sufficient Statistic

.

Example 6.2.15

. .

X1, · · · , Xn

i.i.d.

∼ Uniform(θ, θ + 1), where −∞ < θ < ∞.

Find a minimal sufficient statistic for θ.

.

Joint pdf of X

. . . . . . . . fX x

n i

I xi

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 16 / 25

SLIDE 61

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Uniform Minimal Sufficient Statistic

.

Example 6.2.15

. .

X1, · · · , Xn

i.i.d.

∼ Uniform(θ, θ + 1), where −∞ < θ < ∞.

Find a minimal sufficient statistic for θ.

.

Joint pdf of X

. . fX(x|θ) =

n

∏

i=1

I(θ < xi < θ + 1)

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 16 / 25

SLIDE 62

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Uniform Minimal Sufficient Statistic

.

Examine fX(x|θ)/fX(y|θ)

. . fX(x|θ) fX(y|θ) = ∏n

i=1 I(θ < xi < θ + 1)

∏n

i=1 I(θ < yi < θ + 1)

I x xn I y yn I x x n I y y n I x n x I y n y The ratio above is constant if and only if x y and x n y n . Therefore, T X X X n is a minimal sufficient statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 17 / 25

SLIDE 63

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Uniform Minimal Sufficient Statistic

.

Examine fX(x|θ)/fX(y|θ)

. . fX(x|θ) fX(y|θ) = ∏n

i=1 I(θ < xi < θ + 1)

∏n

i=1 I(θ < yi < θ + 1)

= I(θ < x1 < θ + 1, · · · , θ < xn < θ + 1) I(θ < y1 < θ + 1, · · · , θ < yn < θ + 1) I x x n I y y n I x n x I y n y The ratio above is constant if and only if x y and x n y n . Therefore, T X X X n is a minimal sufficient statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 17 / 25

SLIDE 64

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Uniform Minimal Sufficient Statistic

.

Examine fX(x|θ)/fX(y|θ)

. . fX(x|θ) fX(y|θ) = ∏n

i=1 I(θ < xi < θ + 1)

∏n

i=1 I(θ < yi < θ + 1)

= I(θ < x1 < θ + 1, · · · , θ < xn < θ + 1) I(θ < y1 < θ + 1, · · · , θ < yn < θ + 1) = I(θ < x(1) ∧ x(n) < θ + 1) I(θ < y(1) ∧ y(n) < θ + 1) I x n x I y n y The ratio above is constant if and only if x y and x n y n . Therefore, T X X X n is a minimal sufficient statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 17 / 25

SLIDE 65

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Uniform Minimal Sufficient Statistic

.

Examine fX(x|θ)/fX(y|θ)

. . fX(x|θ) fX(y|θ) = ∏n

i=1 I(θ < xi < θ + 1)

∏n

i=1 I(θ < yi < θ + 1)

= I(θ < x1 < θ + 1, · · · , θ < xn < θ + 1) I(θ < y1 < θ + 1, · · · , θ < yn < θ + 1) = I(θ < x(1) ∧ x(n) < θ + 1) I(θ < y(1) ∧ y(n) < θ + 1) = I(x(n) − 1 < θ < x(1)) I(y(n) − 1 < θ < y(1)) The ratio above is constant if and only if x y and x n y n . Therefore, T X X X n is a minimal sufficient statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 17 / 25

SLIDE 66

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Uniform Minimal Sufficient Statistic

.

Examine fX(x|θ)/fX(y|θ)

. . fX(x|θ) fX(y|θ) = ∏n

i=1 I(θ < xi < θ + 1)

∏n

i=1 I(θ < yi < θ + 1)

= I(θ < x1 < θ + 1, · · · , θ < xn < θ + 1) I(θ < y1 < θ + 1, · · · , θ < yn < θ + 1) = I(θ < x(1) ∧ x(n) < θ + 1) I(θ < y(1) ∧ y(n) < θ + 1) = I(x(n) − 1 < θ < x(1)) I(y(n) − 1 < θ < y(1)) The ratio above is constant if and only if x(1) = y(1) and x(n) = y(n). Therefore, T(X) = (X(1), X(n)) is a minimal sufficient statistic for θ.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 17 / 25

SLIDE 67

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Normal Minimal Sufficient Statistics (Example 6.2.14)

fX(x|µ, σ2) fX(y|µ, σ2) = exp ( − ∑n

i=1(xi − µ)2

2σ2 ) / exp ( − ∑n

i=1(yi − µ)2

2σ2 ) exp

n i

xi xi

n i

yi yi exp

n i

xi

n i

yi

n i

xi

n i

yi The ratio above will not depend on if and only if

n i

xi

n i

yi

n i

xi

n i

yi Therefore, T X

n i

Xi

n i

Xi is a minimal sufficient statistic for by Theorem 6.2.13

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 18 / 25

SLIDE 68

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Normal Minimal Sufficient Statistics (Example 6.2.14)

fX(x|µ, σ2) fX(y|µ, σ2) = exp ( − ∑n

i=1(xi − µ)2

2σ2 ) / exp ( − ∑n

i=1(yi − µ)2

2σ2 ) = exp [ − 1 2σ2 ( n ∑

i=1

(x2

i − 2µxi + µ2) − n

∑

i=1

(y2

i − 2µyi + µ2)

)] exp

n i

xi

n i

yi

n i

xi

n i

yi The ratio above will not depend on if and only if

n i

xi

n i

yi

n i

xi

n i

yi Therefore, T X

n i

Xi

n i

Xi is a minimal sufficient statistic for by Theorem 6.2.13

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 18 / 25

SLIDE 69

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Normal Minimal Sufficient Statistics (Example 6.2.14)

fX(x|µ, σ2) fX(y|µ, σ2) = exp ( − ∑n

i=1(xi − µ)2

2σ2 ) / exp ( − ∑n

i=1(yi − µ)2

2σ2 ) = exp [ − 1 2σ2 ( n ∑

i=1

(x2

i − 2µxi + µ2) − n

∑

i=1

(y2

i − 2µyi + µ2)

)] = exp [ − 1 2σ2 ( n ∑

i=1

x2

i − n

∑

i=1

y2

i

) + µ σ2 ( n ∑

i=1

xi −

n

∑

i=1

yi )] The ratio above will not depend on if and only if

n i

xi

n i

yi

n i

xi

n i

yi Therefore, T X

n i

Xi

n i

Xi is a minimal sufficient statistic for by Theorem 6.2.13

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 18 / 25

SLIDE 70

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Normal Minimal Sufficient Statistics (Example 6.2.14)

fX(x|µ, σ2) fX(y|µ, σ2) = exp ( − ∑n

i=1(xi − µ)2

2σ2 ) / exp ( − ∑n

i=1(yi − µ)2

2σ2 ) = exp [ − 1 2σ2 ( n ∑

i=1

(x2

i − 2µxi + µ2) − n

∑

i=1

(y2

i − 2µyi + µ2)

)] = exp [ − 1 2σ2 ( n ∑

i=1

x2

i − n

∑

i=1

y2

i

) + µ σ2 ( n ∑

i=1

xi −

n

∑

i=1

yi )] The ratio above will not depend on (µ, σ2) if and only if { ∑n

i=1 x2 i

= ∑n

i=1 y2 i

∑n

i=1 xi

= ∑n

i=1 yi

Therefore, T X

n i

Xi

n i

Xi is a minimal sufficient statistic for by Theorem 6.2.13

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 18 / 25

SLIDE 71

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Normal Minimal Sufficient Statistics (Example 6.2.14)

fX(x|µ, σ2) fX(y|µ, σ2) = exp ( − ∑n

i=1(xi − µ)2

2σ2 ) / exp ( − ∑n

i=1(yi − µ)2

2σ2 ) = exp [ − 1 2σ2 ( n ∑

i=1

(x2

i − 2µxi + µ2) − n

∑

i=1

(y2

i − 2µyi + µ2)

)] = exp [ − 1 2σ2 ( n ∑

i=1

x2

i − n

∑

i=1

y2

i

) + µ σ2 ( n ∑

i=1

xi −

n

∑

i=1

yi )] The ratio above will not depend on (µ, σ2) if and only if { ∑n

i=1 x2 i

= ∑n

i=1 y2 i

∑n

i=1 xi

= ∑n

i=1 yi

Therefore, T(X) = (∑n

i=1 Xi, ∑n i=1 X2 i ) is a minimal sufficient statistic for

(µ, σ2) by Theorem 6.2.13

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 18 / 25

SLIDE 72

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Are Minimal Sufficient Statistics Unique?

A short answer is ”No”
For example, X sX

n i

Xi X n is also a minimal sufficient statistic for in normal distribution.

Important Facts

. . 1 If T X is a minimal sufficient statistic for

, then its one-to-one function is also a minimal sufficient statistic for .

. . 2 There is always a one-to-one function between any two minimal

sufficient statistics. In other words, the partition created by a minimal sufficient statistic is unique

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 19 / 25

SLIDE 73

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Are Minimal Sufficient Statistics Unique?

A short answer is ”No”
For example, X sX

n i

Xi X n is also a minimal sufficient statistic for in normal distribution.

Important Facts

. . 1 If T X is a minimal sufficient statistic for

, then its one-to-one function is also a minimal sufficient statistic for .

. . 2 There is always a one-to-one function between any two minimal

sufficient statistics. In other words, the partition created by a minimal sufficient statistic is unique

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 19 / 25

SLIDE 74

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Are Minimal Sufficient Statistics Unique?

A short answer is ”No”
For example,

( X, s2

X = ∑n i=1(Xi − X)2/(n − 1)

) is also a minimal sufficient statistic for (µ, σ2) in normal distribution.

Important Facts

. . 1 If T X is a minimal sufficient statistic for

, then its one-to-one function is also a minimal sufficient statistic for .

. . 2 There is always a one-to-one function between any two minimal

sufficient statistics. In other words, the partition created by a minimal sufficient statistic is unique

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 19 / 25

SLIDE 75

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Are Minimal Sufficient Statistics Unique?

A short answer is ”No”
For example,

( X, s2

X = ∑n i=1(Xi − X)2/(n − 1)

) is also a minimal sufficient statistic for (µ, σ2) in normal distribution.

Important Facts

. . 1 If T(X) is a minimal sufficient statistic for θ, then its one-to-one

function is also a minimal sufficient statistic for θ.

. . 2 There is always a one-to-one function between any two minimal

sufficient statistics. In other words, the partition created by a minimal sufficient statistic is unique

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 19 / 25

SLIDE 76

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Are Minimal Sufficient Statistics Unique?

A short answer is ”No”
For example,

( X, s2

X = ∑n i=1(Xi − X)2/(n − 1)

) is also a minimal sufficient statistic for (µ, σ2) in normal distribution.

Important Facts

. . 1 If T(X) is a minimal sufficient statistic for θ, then its one-to-one

function is also a minimal sufficient statistic for θ.

. . 2 There is always a one-to-one function between any two minimal

sufficient statistics. In other words, the partition created by a minimal sufficient statistic is unique

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 19 / 25

SLIDE 77

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Proving the important facts

.

Theorem for Fact 1

. . If T(X) is a minimal sufficient statistic for θ, then its one-to-one function is also a minimal sufficient statistic for θ. .

Strategies for Proof

. . . . . . . .

Let T

X q T X and q is a one-to-one function. Then there exist a q such that T X q T X

First is to prove that T

x is a sufficient statistic.

Next, prove that T

x is also a minimal sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 20 / 25

SLIDE 78

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Proving the important facts

.

Theorem for Fact 1

. . If T(X) is a minimal sufficient statistic for θ, then its one-to-one function is also a minimal sufficient statistic for θ. .

Strategies for Proof

. .

Let T∗(X) = q(T(X)) and q is a one-to-one function. Then there

exist a q−1 such that T(X) = q−1(T∗(X))

First is to prove that T∗(x) is a sufficient statistic.
Next, prove that T∗(x) is also a minimal sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 20 / 25

SLIDE 79

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Proof : T∗(x) is a sufficient statistic

Because T(X) is sufficient, by the Factorization Theorem, there exists h and g such that fX(x|θ) = g(T(x|θ))h(x) g q T x h x g q T x h x Therefore, by the Factorization Theorem, T is also a sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 21 / 25

SLIDE 80

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Proof : T∗(x) is a sufficient statistic

Because T(X) is sufficient, by the Factorization Theorem, there exists h and g such that fX(x|θ) = g(T(x|θ))h(x) = g(q−1(T∗(x|θ)))h(x) g q T x h x Therefore, by the Factorization Theorem, T is also a sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 21 / 25

SLIDE 81

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Proof : T∗(x) is a sufficient statistic

Because T(X) is sufficient, by the Factorization Theorem, there exists h and g such that fX(x|θ) = g(T(x|θ))h(x) = g(q−1(T∗(x|θ)))h(x) = (g ◦ q−1)(T∗(x|θ))h(x) Therefore, by the Factorization Theorem, T is also a sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 21 / 25

SLIDE 82

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Proof : T∗(x) is a sufficient statistic

Because T(X) is sufficient, by the Factorization Theorem, there exists h and g such that fX(x|θ) = g(T(x|θ))h(x) = g(q−1(T∗(x|θ)))h(x) = (g ◦ q−1)(T∗(x|θ))h(x) Therefore, by the Factorization Theorem, T∗ is also a sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 21 / 25

SLIDE 83

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Proof : T∗(x) is a minimal sufficient statistic

Because T(X) is minimal sufficient, by definition, for any sufficient statistic S(X), there exist a function w such that T(X) = w(S(x)). T x q T X q w S X q w S X Thus, T X is also a function of S X always, and by definition, T is also a minimal sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 22 / 25

SLIDE 84

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Proof : T∗(x) is a minimal sufficient statistic

Because T(X) is minimal sufficient, by definition, for any sufficient statistic S(X), there exist a function w such that T(X) = w(S(x)). T∗(x) = q(T(X)) q w S X q w S X Thus, T X is also a function of S X always, and by definition, T is also a minimal sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 22 / 25

SLIDE 85

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Proof : T∗(x) is a minimal sufficient statistic

Because T(X) is minimal sufficient, by definition, for any sufficient statistic S(X), there exist a function w such that T(X) = w(S(x)). T∗(x) = q(T(X)) = q(w(S(X))) q w S X Thus, T X is also a function of S X always, and by definition, T is also a minimal sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 22 / 25

SLIDE 86

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Proof : T∗(x) is a minimal sufficient statistic

Because T(X) is minimal sufficient, by definition, for any sufficient statistic S(X), there exist a function w such that T(X) = w(S(x)). T∗(x) = q(T(X)) = q(w(S(X))) = (q ◦ w)(S(X)) Thus, T X is also a function of S X always, and by definition, T is also a minimal sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 22 / 25

SLIDE 87

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Proof : T∗(x) is a minimal sufficient statistic

Because T(X) is minimal sufficient, by definition, for any sufficient statistic S(X), there exist a function w such that T(X) = w(S(x)). T∗(x) = q(T(X)) = q(w(S(X))) = (q ◦ w)(S(X)) Thus, T∗(X) is also a function of S(X) always, and by definition, T∗ is also a minimal sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 22 / 25

SLIDE 88

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Proving the important facts

.

Theorem for Fact 2

. . There is always a one-to-one function between any two minimal sufficient statistics. (In other words, the partition created by minimal sufficient statistics is unique) .

Examples

. . . . . . . . For normal statistics, let T X Xi Xi and T X X Xi X n . Then, there exists one-to-one functions such that Xi g X Xi X n Xi g X Xi X n X h Xi Xi Xi X n h Xi Xi

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 23 / 25

SLIDE 89

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Proving the important facts

.

Theorem for Fact 2

. . There is always a one-to-one function between any two minimal sufficient

statistics. (In other words, the partition created by minimal sufficient

statistics is unique) .

Examples

. . . . . . . . For normal statistics, let T X Xi Xi and T X X Xi X n . Then, there exists one-to-one functions such that Xi g X Xi X n Xi g X Xi X n X h Xi Xi Xi X n h Xi Xi

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 23 / 25

SLIDE 90

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Proving the important facts

.

Theorem for Fact 2

. . There is always a one-to-one function between any two minimal sufficient

statistics. (In other words, the partition created by minimal sufficient

statistics is unique) .

Examples

. . For normal statistics, let T1(X) = (∑ Xi, ∑ X2

i ) and

T2(X) = (X, ∑(Xi − X)2/(n − 1)). Then, there exists one-to-one functions such that Xi g X Xi X n Xi g X Xi X n X h Xi Xi Xi X n h Xi Xi

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 23 / 25

SLIDE 91

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Proving the important facts

.

Theorem for Fact 2

. . There is always a one-to-one function between any two minimal sufficient

statistics. (In other words, the partition created by minimal sufficient

statistics is unique) .

Examples

. . For normal statistics, let T1(X) = (∑ Xi, ∑ X2

i ) and

T2(X) = (X, ∑(Xi − X)2/(n − 1)). Then, there exists one-to-one functions such that ∑ Xi = g1 ( X, ∑(Xi − X)2/(n − 1) ) ∑ X2

i

= g2 ( X, ∑(Xi − X)2/(n − 1) ) X h Xi Xi Xi X n h Xi Xi

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 23 / 25

SLIDE 92

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Proving the important facts

.

Theorem for Fact 2

. . There is always a one-to-one function between any two minimal sufficient

statistics. (In other words, the partition created by minimal sufficient

statistics is unique) .

Examples

. . For normal statistics, let T1(X) = (∑ Xi, ∑ X2

i ) and

T2(X) = (X, ∑(Xi − X)2/(n − 1)). Then, there exists one-to-one functions such that ∑ Xi = g1 ( X, ∑(Xi − X)2/(n − 1) ) ∑ X2

i

= g2 ( X, ∑(Xi − X)2/(n − 1) ) X = h1(∑ Xi, ∑ X2

i )

∑(Xi − X)2(n − 1) = h2(∑ Xi, ∑ X2

i )

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 23 / 25

SLIDE 93

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Proof

Assume that both T(X) and T∗(X) are minimal sufficient. Then by the definition of minimal sufficient statistics, there exist q(·) and r(·) such that T X q T X T X r T X Therefore, q r holds and they are one-to-one functions.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 24 / 25

SLIDE 94

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Proof

Assume that both T(X) and T∗(X) are minimal sufficient. Then by the definition of minimal sufficient statistics, there exist q(·) and r(·) such that T(X) = q(T∗(X)) T∗(X) = r(T(X)) Therefore, q = r−1 holds and they are one-to-one functions.

Hyun Min Kang Biostatistics 602 - Lecture 03 January 17th, 2013 24 / 25

SLIDE 95

. . . . . .

. . . . Factorization . . . . . . . . . . . . . . . . . . . Minimal Sufficient Statistics . Summary

Summary

.

Today

. .

Recap of Factorization Theorem
Minimal Sufficient Statistics
Theorem 6.2.13
Two sufficient statistics from binomial distribution
Uniform Distribution
Normal Distribution
Minimal Sufficient Statistics are not unique