Basus Theorem Lecture 06 Biostatistics 602 - Statistical Inference - - PowerPoint PPT Presentation

▶
basu s theorem lecture 06 biostatistics 602 statistical
SMART_READER_LITE
LIVE PREVIEW

Basus Theorem Lecture 06 Biostatistics 602 - Statistical Inference - - PowerPoint PPT Presentation

Nov 09, 2022 201 likes •1.24k views

. . January 29th, 2013 Biostatistics 602 - Lecture 07 Hyun Min Kang January 29th, 2013 Hyun Min Kang Basus Theorem Lecture 06 Biostatistics 602 - Statistical Inference . Summary . . Basus Theorem Complete Statistics . . . . . .

slide-1
SLIDE 1

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

. .

Biostatistics 602 - Statistical Inference Lecture 06 Basu’s Theorem

Hyun Min Kang January 29th, 2013

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 1 / 21

slide-2
SLIDE 2

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Last Lecture

. . 1 What is a complete statistic? . . 2 Why it is called as ”complete statistic”? . . 3 Can the same statistic be both complete and incomplete statistics,

depending on the parameter space?

. . 4 What is the relationship between complete and sufficient statistics? . . 5 Is a minimal sufficient statistic always complete?

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 2 / 21

slide-3
SLIDE 3

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Last Lecture

. . 1 What is a complete statistic? . . 2 Why it is called as ”complete statistic”? . . 3 Can the same statistic be both complete and incomplete statistics,

depending on the parameter space?

. . 4 What is the relationship between complete and sufficient statistics? . . 5 Is a minimal sufficient statistic always complete?

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 2 / 21

slide-4
SLIDE 4

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Last Lecture

. . 1 What is a complete statistic? . . 2 Why it is called as ”complete statistic”? . . 3 Can the same statistic be both complete and incomplete statistics,

depending on the parameter space?

. . 4 What is the relationship between complete and sufficient statistics? . . 5 Is a minimal sufficient statistic always complete?

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 2 / 21

slide-5
SLIDE 5

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Last Lecture

. . 1 What is a complete statistic? . . 2 Why it is called as ”complete statistic”? . . 3 Can the same statistic be both complete and incomplete statistics,

depending on the parameter space?

. . 4 What is the relationship between complete and sufficient statistics? . . 5 Is a minimal sufficient statistic always complete?

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 2 / 21

slide-6
SLIDE 6

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Last Lecture

. . 1 What is a complete statistic? . . 2 Why it is called as ”complete statistic”? . . 3 Can the same statistic be both complete and incomplete statistics,

depending on the parameter space?

. . 4 What is the relationship between complete and sufficient statistics? . . 5 Is a minimal sufficient statistic always complete?

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 2 / 21

slide-7
SLIDE 7

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Complete Statistics

.

Definition

. .

  • Let T = {fT(t|θ), θ ∈ Ω} be a family of pdfs or pmfs for a statistic

T(X).

  • The family of probability distributions is called complete if
  • E g T

for all implies Pr g T for all .

  • In other words, g T

almost surely.

  • Equivalently, T X is called a complete statistic

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 3 / 21

slide-8
SLIDE 8

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Complete Statistics

.

Definition

. .

  • Let T = {fT(t|θ), θ ∈ Ω} be a family of pdfs or pmfs for a statistic

T(X).

  • The family of probability distributions is called complete if
  • E g T

for all implies Pr g T for all .

  • In other words, g T

almost surely.

  • Equivalently, T X is called a complete statistic

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 3 / 21

slide-9
SLIDE 9

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Complete Statistics

.

Definition

. .

  • Let T = {fT(t|θ), θ ∈ Ω} be a family of pdfs or pmfs for a statistic

T(X).

  • The family of probability distributions is called complete if
  • E[g(T)|θ] = 0 for all θ implies Pr[g(T) = 0|θ] = 1 for all θ.
  • In other words, g T

almost surely.

  • Equivalently, T X is called a complete statistic

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 3 / 21

slide-10
SLIDE 10

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Complete Statistics

.

Definition

. .

  • Let T = {fT(t|θ), θ ∈ Ω} be a family of pdfs or pmfs for a statistic

T(X).

  • The family of probability distributions is called complete if
  • E[g(T)|θ] = 0 for all θ implies Pr[g(T) = 0|θ] = 1 for all θ.
  • In other words, g(T) = 0 almost surely.
  • Equivalently, T X is called a complete statistic

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 3 / 21

slide-11
SLIDE 11

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Complete Statistics

.

Definition

. .

  • Let T = {fT(t|θ), θ ∈ Ω} be a family of pdfs or pmfs for a statistic

T(X).

  • The family of probability distributions is called complete if
  • E[g(T)|θ] = 0 for all θ implies Pr[g(T) = 0|θ] = 1 for all θ.
  • In other words, g(T) = 0 almost surely.
  • Equivalently, T(X) is called a complete statistic

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 3 / 21

slide-12
SLIDE 12

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Example - Poisson distribution

.

When parameter space is limited - NOT complete

. .

  • Suppose T =

{ fT : fT(t|λ) = λte−λ

t!

} for t ∈ {0, 1, 2, · · · }. Let λ ∈ Ω = {1, 2}. This family is NOT complete .

With full parameter space - complete

. . . . . . . .

  • X

Xn

i.i.d. Poisson

.

  • T X

n i

Xi is a complete statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 4 / 21

slide-13
SLIDE 13

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Example - Poisson distribution

.

When parameter space is limited - NOT complete

. .

  • Suppose T =

{ fT : fT(t|λ) = λte−λ

t!

} for t ∈ {0, 1, 2, · · · }. Let λ ∈ Ω = {1, 2}. This family is NOT complete .

With full parameter space - complete

. .

  • X1, · · · , Xn

i.i.d.

∼ Poisson(λ), λ > 0.

  • T(X) = ∑n

i=1 Xi is a complete statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 4 / 21

slide-14
SLIDE 14

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Example from Stigler (1972) Am. Stat.

.

Problem

. . Let X is a uniform random sample from {1, · · · , θ} where θ ∈ Ω = N. Is T X X a complete statistic? .

Solution

. . . . . . . . Consider a function g T such that E g T for all . Note that fX x I x I x . E g T E g X

x

g x

x

g x

x

g x

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 5 / 21

slide-15
SLIDE 15

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Example from Stigler (1972) Am. Stat.

.

Problem

. . Let X is a uniform random sample from {1, · · · , θ} where θ ∈ Ω = N. Is T(X) = X a complete statistic? .

Solution

. . . . . . . . Consider a function g T such that E g T for all . Note that fX x I x I x . E g T E g X

x

g x

x

g x

x

g x

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 5 / 21

slide-16
SLIDE 16

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Example from Stigler (1972) Am. Stat.

.

Problem

. . Let X is a uniform random sample from {1, · · · , θ} where θ ∈ Ω = N. Is T(X) = X a complete statistic? .

Solution

. . Consider a function g(T) such that E[g(T)|θ] = 0 for all θ ∈ N. Note that fX(x) = 1

θI(x ∈ {1, · · · , θ}) = 1 θINθ(x).

E g T E g X

x

g x

x

g x

x

g x

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 5 / 21

slide-17
SLIDE 17

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Example from Stigler (1972) Am. Stat.

.

Problem

. . Let X is a uniform random sample from {1, · · · , θ} where θ ∈ Ω = N. Is T(X) = X a complete statistic? .

Solution

. . Consider a function g(T) such that E[g(T)|θ] = 0 for all θ ∈ N. Note that fX(x) = 1

θI(x ∈ {1, · · · , θ}) = 1 θINθ(x).

E[g(T)|θ] = E[g(X)|θ] =

θ

∑

x=1

1 θg(x) = 1 θ

θ

∑

x=1

g(x) = 0

x

g x

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 5 / 21

slide-18
SLIDE 18

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Example from Stigler (1972) Am. Stat.

.

Problem

. . Let X is a uniform random sample from {1, · · · , θ} where θ ∈ Ω = N. Is T(X) = X a complete statistic? .

Solution

. . Consider a function g(T) such that E[g(T)|θ] = 0 for all θ ∈ N. Note that fX(x) = 1

θI(x ∈ {1, · · · , θ}) = 1 θINθ(x).

E[g(T)|θ] = E[g(X)|θ] =

θ

∑

x=1

1 θg(x) = 1 θ

θ

∑

x=1

g(x) = 0

θ

∑

x=1

g(x) =

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 5 / 21

slide-19
SLIDE 19

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Solution (cont’d)

for all θ ∈ N, which implies

  • if θ = 1, ∑θ

x=1 g(x) = g(1) = 0

  • if

,

x

g x g g g .

  • .

. .

  • if

k,

x

g x g g k g k . Therefore, g x for all x , and T X X is a complete statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 6 / 21

slide-20
SLIDE 20

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Solution (cont’d)

for all θ ∈ N, which implies

  • if θ = 1, ∑θ

x=1 g(x) = g(1) = 0

  • if θ = 2, ∑θ

x=1 g(x) = g(1) + g(2) = g(2) = 0.

  • .

. .

  • if

k,

x

g x g g k g k . Therefore, g x for all x , and T X X is a complete statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 6 / 21

slide-21
SLIDE 21

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Solution (cont’d)

for all θ ∈ N, which implies

  • if θ = 1, ∑θ

x=1 g(x) = g(1) = 0

  • if θ = 2, ∑θ

x=1 g(x) = g(1) + g(2) = g(2) = 0.

  • .

. .

  • if θ = k, ∑θ

x=1 g(x) = g(1) + · · · + g(k − 1) = g(k) = 0.

Therefore, g x for all x , and T X X is a complete statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 6 / 21

slide-22
SLIDE 22

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Solution (cont’d)

for all θ ∈ N, which implies

  • if θ = 1, ∑θ

x=1 g(x) = g(1) = 0

  • if θ = 2, ∑θ

x=1 g(x) = g(1) + g(2) = g(2) = 0.

  • .

. .

  • if θ = k, ∑θ

x=1 g(x) = g(1) + · · · + g(k − 1) = g(k) = 0.

Therefore, g(x) = 0 for all x ∈ N, and T(X) = X is a complete statistic for θ ∈ Ω = N.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 6 / 21

slide-23
SLIDE 23

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Is the previous example barely complete?

.

Modified Problem

. . Let X is a uniform random sample from {1, · · · , θ} where θ ∈ Ω = N − {n}. Is T X X a complete statistic? .

Solution

. . . . . . . . Define a nonzero g x as follows g x x n x n

  • therwise

E g T

x

g x n n Because does not include n, g x for all n , and T X X is not a complete statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 7 / 21

slide-24
SLIDE 24

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Is the previous example barely complete?

.

Modified Problem

. . Let X is a uniform random sample from {1, · · · , θ} where θ ∈ Ω = N − {n}. Is T(X) = X a complete statistic? .

Solution

. . . . . . . . Define a nonzero g x as follows g x x n x n

  • therwise

E g T

x

g x n n Because does not include n, g x for all n , and T X X is not a complete statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 7 / 21

slide-25
SLIDE 25

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Is the previous example barely complete?

.

Modified Problem

. . Let X is a uniform random sample from {1, · · · , θ} where θ ∈ Ω = N − {n}. Is T(X) = X a complete statistic? .

Solution

. . Define a nonzero g(x) as follows g(x) =    1 x = n −1 x = n + 1

  • therwise

E g T

x

g x n n Because does not include n, g x for all n , and T X X is not a complete statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 7 / 21

slide-26
SLIDE 26

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Is the previous example barely complete?

.

Modified Problem

. . Let X is a uniform random sample from {1, · · · , θ} where θ ∈ Ω = N − {n}. Is T(X) = X a complete statistic? .

Solution

. . Define a nonzero g(x) as follows g(x) =    1 x = n −1 x = n + 1

  • therwise

E[g(T)|θ] = 1 θ

θ

∑

x=1

g(x) = { 0 θ ̸= n

1 θ

θ = n Because does not include n, g x for all n , and T X X is not a complete statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 7 / 21

slide-27
SLIDE 27

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Is the previous example barely complete?

.

Modified Problem

. . Let X is a uniform random sample from {1, · · · , θ} where θ ∈ Ω = N − {n}. Is T(X) = X a complete statistic? .

Solution

. . Define a nonzero g(x) as follows g(x) =    1 x = n −1 x = n + 1

  • therwise

E[g(T)|θ] = 1 θ

θ

∑

x=1

g(x) = { 0 θ ̸= n

1 θ

θ = n Because Ω does not include n, g(x) = 0 for all θ ∈ Ω = N − {n}, and T(X) = X is not a complete statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 7 / 21

slide-28
SLIDE 28

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Last Lecture : Ancillary and Complete Statistics

.

Problem

. .

  • Let X1, · · · , Xn

i.i.d.

∼ Uniform(θ, θ + 1), θ ∈ R.

  • Is T(X) = (X(1), X(n)) a complete statistic?

.

A Simple Proof

. . . . . . . .

  • We know that R

X n X is an ancillary statistic, which do not depend on .

  • Define g T

X n X E R . Note that E R is constant to .

  • Then E g T

E R E R , so T is not a complete statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 8 / 21

slide-29
SLIDE 29

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Last Lecture : Ancillary and Complete Statistics

.

Problem

. .

  • Let X1, · · · , Xn

i.i.d.

∼ Uniform(θ, θ + 1), θ ∈ R.

  • Is T(X) = (X(1), X(n)) a complete statistic?

.

A Simple Proof

. .

  • We know that R = X(n) − X(1) is an ancillary statistic, which do not

depend on θ.

  • Define g T

X n X E R . Note that E R is constant to .

  • Then E g T

E R E R , so T is not a complete statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 8 / 21

slide-30
SLIDE 30

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Last Lecture : Ancillary and Complete Statistics

.

Problem

. .

  • Let X1, · · · , Xn

i.i.d.

∼ Uniform(θ, θ + 1), θ ∈ R.

  • Is T(X) = (X(1), X(n)) a complete statistic?

.

A Simple Proof

. .

  • We know that R = X(n) − X(1) is an ancillary statistic, which do not

depend on θ.

  • Define g(T) = X(n) − X(1) − E(R). Note that E(R) is constant to θ.
  • Then E g T

E R E R , so T is not a complete statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 8 / 21

slide-31
SLIDE 31

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Last Lecture : Ancillary and Complete Statistics

.

Problem

. .

  • Let X1, · · · , Xn

i.i.d.

∼ Uniform(θ, θ + 1), θ ∈ R.

  • Is T(X) = (X(1), X(n)) a complete statistic?

.

A Simple Proof

. .

  • We know that R = X(n) − X(1) is an ancillary statistic, which do not

depend on θ.

  • Define g(T) = X(n) − X(1) − E(R). Note that E(R) is constant to θ.
  • Then E[g(T)|θ] = E(R) − E(R) = 0, so T is not a complete statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 8 / 21

slide-32
SLIDE 32

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Useful Fact 1 : Ancillary and Complete Statistics

.

Fact

. . For a statistic T(X), If a non-constant function of T, say r(T) is ancillary, then T(X) cannot be complete .

Proof

. . . . . . . . Define g T r T E r T , which does not depend on the parameter because r T is ancillary. Then E g T for a non-zero function g T , and T X is not a complete statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 9 / 21

slide-33
SLIDE 33

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Useful Fact 1 : Ancillary and Complete Statistics

.

Fact

. . For a statistic T(X), If a non-constant function of T, say r(T) is ancillary, then T(X) cannot be complete .

Proof

. . Define g(T) = r(T) − E[r(T)], which does not depend on the parameter θ because r(T) is ancillary. Then E g T for a non-zero function g T , and T X is not a complete statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 9 / 21

slide-34
SLIDE 34

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Useful Fact 1 : Ancillary and Complete Statistics

.

Fact

. . For a statistic T(X), If a non-constant function of T, say r(T) is ancillary, then T(X) cannot be complete .

Proof

. . Define g(T) = r(T) − E[r(T)], which does not depend on the parameter θ because r(T) is ancillary. Then E[g(T)|θ] = 0 for a non-zero function g(T), and T(X) is not a complete statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 9 / 21

slide-35
SLIDE 35

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Useful Fact 2 : Arbitrary Function of Complete Statistics

.

Fact

. . If T(X) is a complete statistic, then a function of T, say T∗ = r(T) is also complete. .

Proof

. . . . . . . . E g T E g r T Assume that E g T for all , then E g r T holds for all

  • too. Because T X is a complete statistic, Pr g

r T . Therefore Pr g T , and T is a complete statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 10 / 21

slide-36
SLIDE 36

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Useful Fact 2 : Arbitrary Function of Complete Statistics

.

Fact

. . If T(X) is a complete statistic, then a function of T, say T∗ = r(T) is also complete. .

Proof

. . E[g(T∗)|θ] = E[g ◦ r(T)|θ] Assume that E g T for all , then E g r T holds for all

  • too. Because T X is a complete statistic, Pr g

r T . Therefore Pr g T , and T is a complete statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 10 / 21

slide-37
SLIDE 37

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Useful Fact 2 : Arbitrary Function of Complete Statistics

.

Fact

. . If T(X) is a complete statistic, then a function of T, say T∗ = r(T) is also complete. .

Proof

. . E[g(T∗)|θ] = E[g ◦ r(T)|θ] Assume that E[g(T∗)|θ] = 0 for all θ, then E g r T holds for all

  • too. Because T X is a complete statistic, Pr g

r T . Therefore Pr g T , and T is a complete statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 10 / 21

slide-38
SLIDE 38

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Useful Fact 2 : Arbitrary Function of Complete Statistics

.

Fact

. . If T(X) is a complete statistic, then a function of T, say T∗ = r(T) is also complete. .

Proof

. . E[g(T∗)|θ] = E[g ◦ r(T)|θ] Assume that E[g(T∗)|θ] = 0 for all θ, then E[g ◦ r(T)|θ] = 0 holds for all θ too. Because T X is a complete statistic, Pr g r T . Therefore Pr g T , and T is a complete statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 10 / 21

slide-39
SLIDE 39

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Useful Fact 2 : Arbitrary Function of Complete Statistics

.

Fact

. . If T(X) is a complete statistic, then a function of T, say T∗ = r(T) is also complete. .

Proof

. . E[g(T∗)|θ] = E[g ◦ r(T)|θ] Assume that E[g(T∗)|θ] = 0 for all θ, then E[g ◦ r(T)|θ] = 0 holds for all θ

  • too. Because T(X) is a complete statistic,

Pr g r T . Therefore Pr g T , and T is a complete statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 10 / 21

slide-40
SLIDE 40

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Useful Fact 2 : Arbitrary Function of Complete Statistics

.

Fact

. . If T(X) is a complete statistic, then a function of T, say T∗ = r(T) is also complete. .

Proof

. . E[g(T∗)|θ] = E[g ◦ r(T)|θ] Assume that E[g(T∗)|θ] = 0 for all θ, then E[g ◦ r(T)|θ] = 0 holds for all θ

  • too. Because T(X) is a complete statistic, Pr[g ◦ r(T) = 0] = 1, ∀θ ∈ Ω.

Therefore Pr[g(T∗) = 0] = 1, and T∗ is a complete statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 10 / 21

slide-41
SLIDE 41

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Theorem 6.2.28 - Lehman and Schefle (1950)

.

The textbook version

. . If a minimal sufficient statistic exists, then any complete statistic is also a minimal sufficient statistic. .

Paraphrased version

. . . . . . . . Any complete, and sufficient statistic is also a minimal sufficient statistic .

The converse is NOT true

. . . . . . . . A minimal sufficient statistic is not necessarily complete. (Recall the example in the last lecture).

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 11 / 21

slide-42
SLIDE 42

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Theorem 6.2.28 - Lehman and Schefle (1950)

.

The textbook version

. . If a minimal sufficient statistic exists, then any complete statistic is also a minimal sufficient statistic. .

Paraphrased version

. . Any complete, and sufficient statistic is also a minimal sufficient statistic .

The converse is NOT true

. . . . . . . . A minimal sufficient statistic is not necessarily complete. (Recall the example in the last lecture).

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 11 / 21

slide-43
SLIDE 43

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Theorem 6.2.28 - Lehman and Schefle (1950)

.

The textbook version

. . If a minimal sufficient statistic exists, then any complete statistic is also a minimal sufficient statistic. .

Paraphrased version

. . Any complete, and sufficient statistic is also a minimal sufficient statistic .

The converse is NOT true

. . A minimal sufficient statistic is not necessarily complete. (Recall the example in the last lecture).

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 11 / 21

slide-44
SLIDE 44

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Basu’s Theorem

.

Theorem 6.2.24

. . If T(X) is a complete sufficient statistic, then T(X) is independent of every ancillary statistic. .

Proof strategy - for discrete case

. . . . . . . . Suppose that S X is an ancillary statistic. We want to show that Pr S X s T X t Pr S X s t Alternatively, we can show that Pr T X t S X s Pr T X t Pr T X t S X s Pr T X t Pr S X s

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 12 / 21

slide-45
SLIDE 45

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Basu’s Theorem

.

Theorem 6.2.24

. . If T(X) is a complete sufficient statistic, then T(X) is independent of every ancillary statistic. .

Proof strategy - for discrete case

. . Suppose that S(X) is an ancillary statistic. We want to show that Pr(S(X) = s|T(X) = t) = Pr(S(X) = s), ∀t ∈ T Alternatively, we can show that Pr T X t S X s Pr T X t Pr T X t S X s Pr T X t Pr S X s

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 12 / 21

slide-46
SLIDE 46

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Basu’s Theorem

.

Theorem 6.2.24

. . If T(X) is a complete sufficient statistic, then T(X) is independent of every ancillary statistic. .

Proof strategy - for discrete case

. . Suppose that S(X) is an ancillary statistic. We want to show that Pr(S(X) = s|T(X) = t) = Pr(S(X) = s), ∀t ∈ T Alternatively, we can show that Pr(T(X) = t|S(X) = s) = Pr(T(X) = t) Pr T X t S X s Pr T X t Pr S X s

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 12 / 21

slide-47
SLIDE 47

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Basu’s Theorem

.

Theorem 6.2.24

. . If T(X) is a complete sufficient statistic, then T(X) is independent of every ancillary statistic. .

Proof strategy - for discrete case

. . Suppose that S(X) is an ancillary statistic. We want to show that Pr(S(X) = s|T(X) = t) = Pr(S(X) = s), ∀t ∈ T Alternatively, we can show that Pr(T(X) = t|S(X) = s) = Pr(T(X) = t) Pr(T(X) = t ∧ S(X) = s) = Pr(T(X) = t) Pr(S(X) = s)

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 12 / 21

slide-48
SLIDE 48

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Proof of Basu’s Theorem

  • As S(X) is ancillary, by definition, it does not depend on θ.
  • As T X is sufficient, by definition, fX X T X

is independent of .

  • Because S X is a function of X, Pr S X T X

is also independent

  • f

.

  • We need to show that

Pr S X s T X t Pr S X s t .

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 13 / 21

slide-49
SLIDE 49

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Proof of Basu’s Theorem

  • As S(X) is ancillary, by definition, it does not depend on θ.
  • As T(X) is sufficient, by definition, fX(X|T(X)) is independent of θ.
  • Because S X is a function of X, Pr S X T X

is also independent

  • f

.

  • We need to show that

Pr S X s T X t Pr S X s t .

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 13 / 21

slide-50
SLIDE 50

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Proof of Basu’s Theorem

  • As S(X) is ancillary, by definition, it does not depend on θ.
  • As T(X) is sufficient, by definition, fX(X|T(X)) is independent of θ.
  • Because S(X) is a function of X, Pr(S(X)|T(X)) is also independent
  • f θ.
  • We need to show that

Pr S X s T X t Pr S X s t .

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 13 / 21

slide-51
SLIDE 51

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Proof of Basu’s Theorem

  • As S(X) is ancillary, by definition, it does not depend on θ.
  • As T(X) is sufficient, by definition, fX(X|T(X)) is independent of θ.
  • Because S(X) is a function of X, Pr(S(X)|T(X)) is also independent
  • f θ.
  • We need to show that

Pr(S(X) = s|T(X) = t) = Pr(S(X) = s), ∀t ∈ T .

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 13 / 21

slide-52
SLIDE 52

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Proof of Basu’s Theorem (cont’d)

Pr(S(X) = s|θ) = ∑

t∈T

Pr(S(X) = s|T(X) = t) Pr(T(X) = t|θ) (1) Pr S X s Pr S X s

t

Pr T X t (2)

t

Pr S X s Pr T X t (3) Define g t Pr S X s T X t Pr S X s . Taking (1)-(3),

t

Pr S X s T X t Pr S X s Pr T X t

t

g t Pr T X t E g T X T X is complete, so g t almost surely for all possible t . Therefore, S X is independent of T X .

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 14 / 21

slide-53
SLIDE 53

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Proof of Basu’s Theorem (cont’d)

Pr(S(X) = s|θ) = ∑

t∈T

Pr(S(X) = s|T(X) = t) Pr(T(X) = t|θ) (1) Pr(S(X) = s|θ) = Pr(S(X) = s) ∑

t∈T

Pr(T(X) = t|θ) (2)

t

Pr S X s Pr T X t (3) Define g t Pr S X s T X t Pr S X s . Taking (1)-(3),

t

Pr S X s T X t Pr S X s Pr T X t

t

g t Pr T X t E g T X T X is complete, so g t almost surely for all possible t . Therefore, S X is independent of T X .

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 14 / 21

slide-54
SLIDE 54

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Proof of Basu’s Theorem (cont’d)

Pr(S(X) = s|θ) = ∑

t∈T

Pr(S(X) = s|T(X) = t) Pr(T(X) = t|θ) (1) Pr(S(X) = s|θ) = Pr(S(X) = s) ∑

t∈T

Pr(T(X) = t|θ) (2) = ∑

t∈T

Pr(S(X) = s) Pr(T(X) = t|θ) (3) Define g t Pr S X s T X t Pr S X s . Taking (1)-(3),

t

Pr S X s T X t Pr S X s Pr T X t

t

g t Pr T X t E g T X T X is complete, so g t almost surely for all possible t . Therefore, S X is independent of T X .

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 14 / 21

slide-55
SLIDE 55

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Proof of Basu’s Theorem (cont’d)

Pr(S(X) = s|θ) = ∑

t∈T

Pr(S(X) = s|T(X) = t) Pr(T(X) = t|θ) (1) Pr(S(X) = s|θ) = Pr(S(X) = s) ∑

t∈T

Pr(T(X) = t|θ) (2) = ∑

t∈T

Pr(S(X) = s) Pr(T(X) = t|θ) (3) Define g(t) = Pr(S(X) = s|T(X) = t) − Pr(S(X) = s). Taking (1)-(3),

t

Pr S X s T X t Pr S X s Pr T X t

t

g t Pr T X t E g T X T X is complete, so g t almost surely for all possible t . Therefore, S X is independent of T X .

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 14 / 21

slide-56
SLIDE 56

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Proof of Basu’s Theorem (cont’d)

Pr(S(X) = s|θ) = ∑

t∈T

Pr(S(X) = s|T(X) = t) Pr(T(X) = t|θ) (1) Pr(S(X) = s|θ) = Pr(S(X) = s) ∑

t∈T

Pr(T(X) = t|θ) (2) = ∑

t∈T

Pr(S(X) = s) Pr(T(X) = t|θ) (3) Define g(t) = Pr(S(X) = s|T(X) = t) − Pr(S(X) = s). Taking (1)-(3), ∑

t∈T

[Pr(S(X) = s|T(X) = t) − Pr(S(X) = s)] Pr(T(X) = t|θ) = 0

t

g t Pr T X t E g T X T X is complete, so g t almost surely for all possible t . Therefore, S X is independent of T X .

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 14 / 21

slide-57
SLIDE 57

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Proof of Basu’s Theorem (cont’d)

Pr(S(X) = s|θ) = ∑

t∈T

Pr(S(X) = s|T(X) = t) Pr(T(X) = t|θ) (1) Pr(S(X) = s|θ) = Pr(S(X) = s) ∑

t∈T

Pr(T(X) = t|θ) (2) = ∑

t∈T

Pr(S(X) = s) Pr(T(X) = t|θ) (3) Define g(t) = Pr(S(X) = s|T(X) = t) − Pr(S(X) = s). Taking (1)-(3), ∑

t∈T

[Pr(S(X) = s|T(X) = t) − Pr(S(X) = s)] Pr(T(X) = t|θ) = 0 ∑

t∈T

g(t) Pr(T(X) = t|θ) = E[g(T(X))|θ] = 0 T X is complete, so g t almost surely for all possible t . Therefore, S X is independent of T X .

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 14 / 21

slide-58
SLIDE 58

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Proof of Basu’s Theorem (cont’d)

Pr(S(X) = s|θ) = ∑

t∈T

Pr(S(X) = s|T(X) = t) Pr(T(X) = t|θ) (1) Pr(S(X) = s|θ) = Pr(S(X) = s) ∑

t∈T

Pr(T(X) = t|θ) (2) = ∑

t∈T

Pr(S(X) = s) Pr(T(X) = t|θ) (3) Define g(t) = Pr(S(X) = s|T(X) = t) − Pr(S(X) = s). Taking (1)-(3), ∑

t∈T

[Pr(S(X) = s|T(X) = t) − Pr(S(X) = s)] Pr(T(X) = t|θ) = 0 ∑

t∈T

g(t) Pr(T(X) = t|θ) = E[g(T(X))|θ] = 0 T(X) is complete, so g(t) = 0 almost surely for all possible t ∈ T . Therefore, S X is independent of T X .

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 14 / 21

slide-59
SLIDE 59

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Proof of Basu’s Theorem (cont’d)

Pr(S(X) = s|θ) = ∑

t∈T

Pr(S(X) = s|T(X) = t) Pr(T(X) = t|θ) (1) Pr(S(X) = s|θ) = Pr(S(X) = s) ∑

t∈T

Pr(T(X) = t|θ) (2) = ∑

t∈T

Pr(S(X) = s) Pr(T(X) = t|θ) (3) Define g(t) = Pr(S(X) = s|T(X) = t) − Pr(S(X) = s). Taking (1)-(3), ∑

t∈T

[Pr(S(X) = s|T(X) = t) − Pr(S(X) = s)] Pr(T(X) = t|θ) = 0 ∑

t∈T

g(t) Pr(T(X) = t|θ) = E[g(T(X))|θ] = 0 T(X) is complete, so g(t) = 0 almost surely for all possible t ∈ T . Therefore, S(X) is independent of T(X).

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 14 / 21

slide-60
SLIDE 60

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Application of Basu’s Theorem

.

Problem

. .

  • X1, · · · , Xn

i.i.d.

∼ Uniform(0, θ).

  • Calculate E

X X n

and E

X X X n

.

A strategy for the solution

. . . . . . . .

  • We know that X n is sufficient statistic.
  • We know that X n is complete, too.
  • We can easily show that X

X n is an ancillary statistic.

  • Then we can leverage Basu’s Theorem for the calculation.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 15 / 21

slide-61
SLIDE 61

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Application of Basu’s Theorem

.

Problem

. .

  • X1, · · · , Xn

i.i.d.

∼ Uniform(0, θ).

  • Calculate E

[ X(1)

X(n)

] and E [X(1)+X(2)

X(n)

] .

A strategy for the solution

. . . . . . . .

  • We know that X n is sufficient statistic.
  • We know that X n is complete, too.
  • We can easily show that X

X n is an ancillary statistic.

  • Then we can leverage Basu’s Theorem for the calculation.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 15 / 21

slide-62
SLIDE 62

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Application of Basu’s Theorem

.

Problem

. .

  • X1, · · · , Xn

i.i.d.

∼ Uniform(0, θ).

  • Calculate E

[ X(1)

X(n)

] and E [X(1)+X(2)

X(n)

] .

A strategy for the solution

. .

  • We know that X(n) is sufficient statistic.
  • We know that X n is complete, too.
  • We can easily show that X

X n is an ancillary statistic.

  • Then we can leverage Basu’s Theorem for the calculation.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 15 / 21

slide-63
SLIDE 63

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Application of Basu’s Theorem

.

Problem

. .

  • X1, · · · , Xn

i.i.d.

∼ Uniform(0, θ).

  • Calculate E

[ X(1)

X(n)

] and E [X(1)+X(2)

X(n)

] .

A strategy for the solution

. .

  • We know that X(n) is sufficient statistic.
  • We know that X(n) is complete, too.
  • We can easily show that X

X n is an ancillary statistic.

  • Then we can leverage Basu’s Theorem for the calculation.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 15 / 21

slide-64
SLIDE 64

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Application of Basu’s Theorem

.

Problem

. .

  • X1, · · · , Xn

i.i.d.

∼ Uniform(0, θ).

  • Calculate E

[ X(1)

X(n)

] and E [X(1)+X(2)

X(n)

] .

A strategy for the solution

. .

  • We know that X(n) is sufficient statistic.
  • We know that X(n) is complete, too.
  • We can easily show that X(1)/X(n) is an ancillary statistic.
  • Then we can leverage Basu’s Theorem for the calculation.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 15 / 21

slide-65
SLIDE 65

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Application of Basu’s Theorem

.

Problem

. .

  • X1, · · · , Xn

i.i.d.

∼ Uniform(0, θ).

  • Calculate E

[ X(1)

X(n)

] and E [X(1)+X(2)

X(n)

] .

A strategy for the solution

. .

  • We know that X(n) is sufficient statistic.
  • We know that X(n) is complete, too.
  • We can easily show that X(1)/X(n) is an ancillary statistic.
  • Then we can leverage Basu’s Theorem for the calculation.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 15 / 21

slide-66
SLIDE 66

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Showing that X(1)/X(n) is Ancillary

fX(x|θ) = 1 θI(0 < x < θ) Let y x , then dx dy , and Y Uniform . fY y I y X X n Y Y n Because the distribution of Y Yn does not depend on , X X n is an ancillary statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 16 / 21

slide-67
SLIDE 67

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Showing that X(1)/X(n) is Ancillary

fX(x|θ) = 1 θI(0 < x < θ) Let y = x/θ, then |dx/dy| = θ, and Y ∼ Uniform(0, 1). fY y I y X X n Y Y n Because the distribution of Y Yn does not depend on , X X n is an ancillary statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 16 / 21

slide-68
SLIDE 68

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Showing that X(1)/X(n) is Ancillary

fX(x|θ) = 1 θI(0 < x < θ) Let y = x/θ, then |dx/dy| = θ, and Y ∼ Uniform(0, 1). fY(y|θ) = I(0 < y < 1) X X n Y Y n Because the distribution of Y Yn does not depend on , X X n is an ancillary statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 16 / 21

slide-69
SLIDE 69

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Showing that X(1)/X(n) is Ancillary

fX(x|θ) = 1 θI(0 < x < θ) Let y = x/θ, then |dx/dy| = θ, and Y ∼ Uniform(0, 1). fY(y|θ) = I(0 < y < 1) X(1) X(n) = Y(1) Y(n) Because the distribution of Y Yn does not depend on , X X n is an ancillary statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 16 / 21

slide-70
SLIDE 70

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Showing that X(1)/X(n) is Ancillary

fX(x|θ) = 1 θI(0 < x < θ) Let y = x/θ, then |dx/dy| = θ, and Y ∼ Uniform(0, 1). fY(y|θ) = I(0 < y < 1) X(1) X(n) = Y(1) Y(n) Because the distribution of Y1, · · · , Yn does not depend on θ, X(1)/X(n) is an ancillary statistic for θ.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 16 / 21

slide-71
SLIDE 71

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Applying Basu’s Theorem

  • By Basu’s Theorem, X(1)/X(n) is independent of X(n).
  • If X and Y are independent, E XY

E X E Y . E X E X X n X n E X X n E X n E X X n E X E X n E Y E Y n E Y E Y n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 17 / 21

slide-72
SLIDE 72

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Applying Basu’s Theorem

  • By Basu’s Theorem, X(1)/X(n) is independent of X(n).
  • If X and Y are independent, E(XY) = E(X)E(Y).

E X E X X n X n E X X n E X n E X X n E X E X n E Y E Y n E Y E Y n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 17 / 21

slide-73
SLIDE 73

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Applying Basu’s Theorem

  • By Basu’s Theorem, X(1)/X(n) is independent of X(n).
  • If X and Y are independent, E(XY) = E(X)E(Y).

E[X(1)] = E [X(1) X(n) X(n) ] E X X n E X n E X X n E X E X n E Y E Y n E Y E Y n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 17 / 21

slide-74
SLIDE 74

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Applying Basu’s Theorem

  • By Basu’s Theorem, X(1)/X(n) is independent of X(n).
  • If X and Y are independent, E(XY) = E(X)E(Y).

E[X(1)] = E [X(1) X(n) X(n) ] = E [X(1) X(n) ] E [ X(n) ] E X X n E X E X n E Y E Y n E Y E Y n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 17 / 21

slide-75
SLIDE 75

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Applying Basu’s Theorem

  • By Basu’s Theorem, X(1)/X(n) is independent of X(n).
  • If X and Y are independent, E(XY) = E(X)E(Y).

E[X(1)] = E [X(1) X(n) X(n) ] = E [X(1) X(n) ] E [ X(n) ] E [X(1) X(n) ] E X E X n E Y E Y n E Y E Y n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 17 / 21

slide-76
SLIDE 76

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Applying Basu’s Theorem

  • By Basu’s Theorem, X(1)/X(n) is independent of X(n).
  • If X and Y are independent, E(XY) = E(X)E(Y).

E[X(1)] = E [X(1) X(n) X(n) ] = E [X(1) X(n) ] E [ X(n) ] E [X(1) X(n) ] = E[X(1)] E[X(n)] E Y E Y n E Y E Y n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 17 / 21

slide-77
SLIDE 77

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Applying Basu’s Theorem

  • By Basu’s Theorem, X(1)/X(n) is independent of X(n).
  • If X and Y are independent, E(XY) = E(X)E(Y).

E[X(1)] = E [X(1) X(n) X(n) ] = E [X(1) X(n) ] E [ X(n) ] E [X(1) X(n) ] = E[X(1)] E[X(n)] = E[θY(1)] E[θY(n)] E Y E Y n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 17 / 21

slide-78
SLIDE 78

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Applying Basu’s Theorem

  • By Basu’s Theorem, X(1)/X(n) is independent of X(n).
  • If X and Y are independent, E(XY) = E(X)E(Y).

E[X(1)] = E [X(1) X(n) X(n) ] = E [X(1) X(n) ] E [ X(n) ] E [X(1) X(n) ] = E[X(1)] E[X(n)] = E[θY(1)] E[θY(n)] = E[Y(1)] E[Y(n)]

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 17 / 21

slide-79
SLIDE 79

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Obtaining E[Y(1)]

Y ∼ Uniform(0, 1) fY y I y FY y yI y I y fY y n n fY y FY y

n

I y n y n I y Y Beta n E Y n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 18 / 21

slide-80
SLIDE 80

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Obtaining E[Y(1)]

Y ∼ Uniform(0, 1) fY(y) = I(0 < y < 1) FY y yI y I y fY y n n fY y FY y

n

I y n y n I y Y Beta n E Y n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 18 / 21

slide-81
SLIDE 81

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Obtaining E[Y(1)]

Y ∼ Uniform(0, 1) fY(y) = I(0 < y < 1) FY(y) = yI(0 < y < 1) + I(y ≥ 1) fY y n n fY y FY y

n

I y n y n I y Y Beta n E Y n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 18 / 21

slide-82
SLIDE 82

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Obtaining E[Y(1)]

Y ∼ Uniform(0, 1) fY(y) = I(0 < y < 1) FY(y) = yI(0 < y < 1) + I(y ≥ 1) fY(1)(y) = n! (n − 1)!fY(y) [1 − FY(y)]n−1 I(0 < y < 1) n y n I y Y Beta n E Y n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 18 / 21

slide-83
SLIDE 83

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Obtaining E[Y(1)]

Y ∼ Uniform(0, 1) fY(y) = I(0 < y < 1) FY(y) = yI(0 < y < 1) + I(y ≥ 1) fY(1)(y) = n! (n − 1)!fY(y) [1 − FY(y)]n−1 I(0 < y < 1) = n(1 − y)n−1I(0 < y < 1) Y Beta n E Y n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 18 / 21

slide-84
SLIDE 84

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Obtaining E[Y(1)]

Y ∼ Uniform(0, 1) fY(y) = I(0 < y < 1) FY(y) = yI(0 < y < 1) + I(y ≥ 1) fY(1)(y) = n! (n − 1)!fY(y) [1 − FY(y)]n−1 I(0 < y < 1) = n(1 − y)n−1I(0 < y < 1) Y(1) ∼ Beta(1, n) E Y n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 18 / 21

slide-85
SLIDE 85

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Obtaining E[Y(1)]

Y ∼ Uniform(0, 1) fY(y) = I(0 < y < 1) FY(y) = yI(0 < y < 1) + I(y ≥ 1) fY(1)(y) = n! (n − 1)!fY(y) [1 − FY(y)]n−1 I(0 < y < 1) = n(1 − y)n−1I(0 < y < 1) Y(1) ∼ Beta(1, n) E[Y(1)] = 1 n + 1

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 18 / 21

slide-86
SLIDE 86

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Obtaining E[Y(n)]

Y ∼ Uniform(0, 1) fY y I y FY y yI y I y fY n y n n fY y FY y

n

I y nyn I y Y n Beta n E Y n n n Therefore, E

X X n E Y E Y n n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 19 / 21

slide-87
SLIDE 87

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Obtaining E[Y(n)]

Y ∼ Uniform(0, 1) fY(y) = I(0 < y < 1) FY y yI y I y fY n y n n fY y FY y

n

I y nyn I y Y n Beta n E Y n n n Therefore, E

X X n E Y E Y n n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 19 / 21

slide-88
SLIDE 88

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Obtaining E[Y(n)]

Y ∼ Uniform(0, 1) fY(y) = I(0 < y < 1) FY(y) = yI(0 < y < 1) + I(y ≥ 1) fY n y n n fY y FY y

n

I y nyn I y Y n Beta n E Y n n n Therefore, E

X X n E Y E Y n n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 19 / 21

slide-89
SLIDE 89

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Obtaining E[Y(n)]

Y ∼ Uniform(0, 1) fY(y) = I(0 < y < 1) FY(y) = yI(0 < y < 1) + I(y ≥ 1) fY(n)(y) = n! (n − 1)!fY(y) [FY(y)]n−1 I(0 < y < 1) nyn I y Y n Beta n E Y n n n Therefore, E

X X n E Y E Y n n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 19 / 21

slide-90
SLIDE 90

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Obtaining E[Y(n)]

Y ∼ Uniform(0, 1) fY(y) = I(0 < y < 1) FY(y) = yI(0 < y < 1) + I(y ≥ 1) fY(n)(y) = n! (n − 1)!fY(y) [FY(y)]n−1 I(0 < y < 1) = nyn−1I(0 < y < 1) Y n Beta n E Y n n n Therefore, E

X X n E Y E Y n n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 19 / 21

slide-91
SLIDE 91

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Obtaining E[Y(n)]

Y ∼ Uniform(0, 1) fY(y) = I(0 < y < 1) FY(y) = yI(0 < y < 1) + I(y ≥ 1) fY(n)(y) = n! (n − 1)!fY(y) [FY(y)]n−1 I(0 < y < 1) = nyn−1I(0 < y < 1) Y(n) ∼ Beta(n, 1) E Y n n n Therefore, E

X X n E Y E Y n n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 19 / 21

slide-92
SLIDE 92

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Obtaining E[Y(n)]

Y ∼ Uniform(0, 1) fY(y) = I(0 < y < 1) FY(y) = yI(0 < y < 1) + I(y ≥ 1) fY(n)(y) = n! (n − 1)!fY(y) [FY(y)]n−1 I(0 < y < 1) = nyn−1I(0 < y < 1) Y(n) ∼ Beta(n, 1) E[Y(n)] = n n + 1 Therefore, E [ X(1)

X(n)

] =

E[Y(1)] E[Y(n)] = 1 n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 19 / 21

slide-93
SLIDE 93

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Obtaining E[Y(2)]

Y ∼ Uniform(0, 1) fY y I y FY y yI y I y fY y n n FY y

n

fY y FY y I y n n y y n I y Y Beta n E Y n Therefore, E

X X X n E Y Y E Y n E Y E Y E Y n n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 20 / 21

slide-94
SLIDE 94

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Obtaining E[Y(2)]

Y ∼ Uniform(0, 1) fY(y) = I(0 < y < 1) FY y yI y I y fY y n n FY y

n

fY y FY y I y n n y y n I y Y Beta n E Y n Therefore, E

X X X n E Y Y E Y n E Y E Y E Y n n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 20 / 21

slide-95
SLIDE 95

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Obtaining E[Y(2)]

Y ∼ Uniform(0, 1) fY(y) = I(0 < y < 1) FY(y) = yI(0 < y < 1) + I(y ≥ 1) fY y n n FY y

n

fY y FY y I y n n y y n I y Y Beta n E Y n Therefore, E

X X X n E Y Y E Y n E Y E Y E Y n n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 20 / 21

slide-96
SLIDE 96

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Obtaining E[Y(2)]

Y ∼ Uniform(0, 1) fY(y) = I(0 < y < 1) FY(y) = yI(0 < y < 1) + I(y ≥ 1) fY(2)(y) = n! (n − 2)! [1 − FY(y)]n−2 fY(y) [FY(y)] I(0 < y < 1) n n y y n I y Y Beta n E Y n Therefore, E

X X X n E Y Y E Y n E Y E Y E Y n n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 20 / 21

slide-97
SLIDE 97

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Obtaining E[Y(2)]

Y ∼ Uniform(0, 1) fY(y) = I(0 < y < 1) FY(y) = yI(0 < y < 1) + I(y ≥ 1) fY(2)(y) = n! (n − 2)! [1 − FY(y)]n−2 fY(y) [FY(y)] I(0 < y < 1) = n(n − 1)y(1 − y)n−2I(0 < y < 1) Y Beta n E Y n Therefore, E

X X X n E Y Y E Y n E Y E Y E Y n n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 20 / 21

slide-98
SLIDE 98

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Obtaining E[Y(2)]

Y ∼ Uniform(0, 1) fY(y) = I(0 < y < 1) FY(y) = yI(0 < y < 1) + I(y ≥ 1) fY(2)(y) = n! (n − 2)! [1 − FY(y)]n−2 fY(y) [FY(y)] I(0 < y < 1) = n(n − 1)y(1 − y)n−2I(0 < y < 1) Y(2) ∼ Beta(2, n − 1) E Y n Therefore, E

X X X n E Y Y E Y n E Y E Y E Y n n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 20 / 21

slide-99
SLIDE 99

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Obtaining E[Y(2)]

Y ∼ Uniform(0, 1) fY(y) = I(0 < y < 1) FY(y) = yI(0 < y < 1) + I(y ≥ 1) fY(2)(y) = n! (n − 2)! [1 − FY(y)]n−2 fY(y) [FY(y)] I(0 < y < 1) = n(n − 1)y(1 − y)n−2I(0 < y < 1) Y(2) ∼ Beta(2, n − 1) E[Y(2)] = 2 n + 1 Therefore, E

X X X n E Y Y E Y n E Y E Y E Y n n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 20 / 21

slide-100
SLIDE 100

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Obtaining E[Y(2)]

Y ∼ Uniform(0, 1) fY(y) = I(0 < y < 1) FY(y) = yI(0 < y < 1) + I(y ≥ 1) fY(2)(y) = n! (n − 2)! [1 − FY(y)]n−2 fY(y) [FY(y)] I(0 < y < 1) = n(n − 1)y(1 − y)n−2I(0 < y < 1) Y(2) ∼ Beta(2, n − 1) E[Y(2)] = 2 n + 1 Therefore, E [ X(1)+X(2)

X(n)

] =

E[Y(1)+Y(2)] E[Y(n)]

=

E[Y(1)]+E[Y(2)] E[Y(n)]

= 3

n

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 20 / 21

slide-101
SLIDE 101

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Summary

.

Today

. .

  • More on complete statistics
  • Basu’s Theorem

.

Next Lecture

. . . . . . . .

  • Exponential Family

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 21 / 21

slide-102
SLIDE 102

. . . . . .

. . . . . . . . . . Complete Statistics . . . . . . . . . Basu’s Theorem . Summary

Summary

.

Today

. .

  • More on complete statistics
  • Basu’s Theorem

.

Next Lecture

. .

  • Exponential Family

Hyun Min Kang Biostatistics 602 - Lecture 07 January 29th, 2013 21 / 21