Data Reduction - Summary Lecture 08 Biostatistics 602 - Statistical - - PowerPoint PPT Presentation

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

. .

Biostatistics 602 - Statistical Inference Lecture 08 Data Reduction - Summary

Hyun Min Kang February 5th, 2013

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 1 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Last Lecture

. . 1 What is an exponential family distribution? . . 2 Does a Bernoulli distribution belongs to an exponential family? . . 3 What is a curved exponential family? . . 4 What is an obvious sufficient statistic from an exponential family? . . 5 When can the sufficient statistic be complete?

Visit http://www.polleverywhere.com/survey/laGysmUTS to respond

• nline.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 2 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Last Lecture

. . 1 What is an exponential family distribution? . . 2 Does a Bernoulli distribution belongs to an exponential family? . . 3 What is a curved exponential family? . . 4 What is an obvious sufficient statistic from an exponential family? . . 5 When can the sufficient statistic be complete?

Visit http://www.polleverywhere.com/survey/laGysmUTS to respond

• nline.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 2 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Last Lecture

. . 1 What is an exponential family distribution? . . 2 Does a Bernoulli distribution belongs to an exponential family? . . 3 What is a curved exponential family? . . 4 What is an obvious sufficient statistic from an exponential family? . . 5 When can the sufficient statistic be complete?

Visit http://www.polleverywhere.com/survey/laGysmUTS to respond

• nline.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 2 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Last Lecture

. . 1 What is an exponential family distribution? . . 2 Does a Bernoulli distribution belongs to an exponential family? . . 3 What is a curved exponential family? . . 4 What is an obvious sufficient statistic from an exponential family? . . 5 When can the sufficient statistic be complete?

Visit http://www.polleverywhere.com/survey/laGysmUTS to respond

• nline.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 2 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Last Lecture

. . 1 What is an exponential family distribution? . . 2 Does a Bernoulli distribution belongs to an exponential family? . . 3 What is a curved exponential family? . . 4 What is an obvious sufficient statistic from an exponential family? . . 5 When can the sufficient statistic be complete?

Visit http://www.polleverywhere.com/survey/laGysmUTS to respond

• nline.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 2 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Theorem 6.2.25

Suppose X1, · · · , Xn is a random sample from pdf or pmf fX(x|θ) where fX(x|θ) = h(x)c(θ) exp  

k

∑

j=1

wj(θ)tj(x)   is a member of an exponential family. Then the statistic T(X) T(X) =  

n

∑

j=1

t1(Xj), · · · ,

n

∑

j=1

tk(Xj)   is complete as long as the parameter space Θ contains an open set in Rk

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 3 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Exponential Family Example

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ N(µ, σ2). Determine whether the following statistics are

whether (1) sufficient (2) complete, and (3) minimal sufficient. T1(X) = ( n ∑

i=1

Xi,

n

∑

i=1

X2

i

) , T2(X) = ( X, s2

X = n

∑

i=1

(Xi − X)2/(n − 1) ) .

How to solve it

. . . . . . . .

• Decompose fX x

in the form of an an exponential family.

• Apply Theorem 6.2.10 to obtain a sufficient statistic and see if it is

equivalent to or related to T X and T X .

• Apply Theorem 6.2.25 to show that it is complete.
• Apply Theorem 6.2.28 to show that it is minimal sufficient.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 4 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Exponential Family Example

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ N(µ, σ2). Determine whether the following statistics are

whether (1) sufficient (2) complete, and (3) minimal sufficient. T1(X) = ( n ∑

i=1

Xi,

n

∑

i=1

X2

i

) , T2(X) = ( X, s2

X = n

∑

i=1

(Xi − X)2/(n − 1) ) .

How to solve it

. .

• Decompose fX(x|µ, σ) in the form of an an exponential family.
• Apply Theorem 6.2.10 to obtain a sufficient statistic and see if it is

equivalent to or related to T X and T X .

• Apply Theorem 6.2.25 to show that it is complete.
• Apply Theorem 6.2.28 to show that it is minimal sufficient.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 4 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Exponential Family Example

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ N(µ, σ2). Determine whether the following statistics are

whether (1) sufficient (2) complete, and (3) minimal sufficient. T1(X) = ( n ∑

i=1

Xi,

n

∑

i=1

X2

i

) , T2(X) = ( X, s2

X = n

∑

i=1

(Xi − X)2/(n − 1) ) .

How to solve it

. .

• Decompose fX(x|µ, σ) in the form of an an exponential family.
• Apply Theorem 6.2.10 to obtain a sufficient statistic and see if it is

equivalent to or related to T1(X) and T2(X).

• Apply Theorem 6.2.25 to show that it is complete.
• Apply Theorem 6.2.28 to show that it is minimal sufficient.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 4 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Exponential Family Example

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ N(µ, σ2). Determine whether the following statistics are

whether (1) sufficient (2) complete, and (3) minimal sufficient. T1(X) = ( n ∑

i=1

Xi,

n

∑

i=1

X2

i

) , T2(X) = ( X, s2

X = n

∑

i=1

(Xi − X)2/(n − 1) ) .

How to solve it

. .

• Decompose fX(x|µ, σ) in the form of an an exponential family.
• Apply Theorem 6.2.10 to obtain a sufficient statistic and see if it is

equivalent to or related to T1(X) and T2(X).

• Apply Theorem 6.2.25 to show that it is complete.
• Apply Theorem 6.2.28 to show that it is minimal sufficient.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 4 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Exponential Family Example

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ N(µ, σ2). Determine whether the following statistics are

whether (1) sufficient (2) complete, and (3) minimal sufficient. T1(X) = ( n ∑

i=1

Xi,

n

∑

i=1

X2

i

) , T2(X) = ( X, s2

X = n

∑

i=1

(Xi − X)2/(n − 1) ) .

How to solve it

. .

• Decompose fX(x|µ, σ) in the form of an an exponential family.
• Apply Theorem 6.2.10 to obtain a sufficient statistic and see if it is

equivalent to or related to T1(X) and T2(X).

• Apply Theorem 6.2.25 to show that it is complete.
• Apply Theorem 6.2.28 to show that it is minimal sufficient.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 4 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Exponential Family Example

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ N(µ, σ2). Determine whether the following statistics are

whether (1) sufficient (2) complete, and (3) minimal sufficient. T1(X) = ( n ∑

i=1

Xi,

n

∑

i=1

X2

i

) , T2(X) = ( X, s2

X = n

∑

i=1

(Xi − X)2/(n − 1) ) .

How to solve it

. .

• Decompose fX(x|µ, σ) in the form of an an exponential family.
• Apply Theorem 6.2.10 to obtain a sufficient statistic and see if it is

equivalent to or related to T1(X) and T2(X).

• Apply Theorem 6.2.25 to show that it is complete.
• Apply Theorem 6.2.28 to show that it is minimal sufficient.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 4 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Applying Theorem 6.2.10

fX(x|µ, σ2) = 1 2πσ2 exp ( − µ2 2σ2 ) exp ( µ σ2 x − x2 2σ2 ) where h x c exp w w t x x t x x By Theorem 6.2.10,

n i

t Xi

n i

t Xi

n i

X

n i

Xi T X is a sufficient statistic

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 5 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Applying Theorem 6.2.10

fX(x|µ, σ2) = 1 2πσ2 exp ( − µ2 2σ2 ) exp ( µ σ2 x − x2 2σ2 ) where                  h(x) = 1 c(θ) =

1 2πσ2 exp

( − µ2

2σ2

) w1(θ) = µ/σ2 w2(θ) = − 1

2σ2

t1(x) = x t2(x) = x2 By Theorem 6.2.10,

n i

t Xi

n i

t Xi

n i

X

n i

Xi T X is a sufficient statistic

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 5 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Applying Theorem 6.2.10

fX(x|µ, σ2) = 1 2πσ2 exp ( − µ2 2σ2 ) exp ( µ σ2 x − x2 2σ2 ) where                  h(x) = 1 c(θ) =

1 2πσ2 exp

( − µ2

2σ2

) w1(θ) = µ/σ2 w2(θ) = − 1

2σ2

t1(x) = x t2(x) = x2 By Theorem 6.2.10, (∑n

i=1 t1(Xi), ∑n i=1 t2(Xi)) =

(∑n

i=1 X1, ∑n i=1 X2 i

) = T1(X) is a sufficient statistic

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 5 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Applying Theorem 6.2.25. and Theorem 6.2.28

A = {(w1(θ), w2(θ)) : θ ∈ R2} = { µ σ2 , − 1 2σ2 : µ ∈ R, σ > 0 } Contains a open subset in , so T X is also complete by Theorem 6.2.25. By Theorem 6.2.28, T X is also minimal sufficient.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 6 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Applying Theorem 6.2.25. and Theorem 6.2.28

A = {(w1(θ), w2(θ)) : θ ∈ R2} = { µ σ2 , − 1 2σ2 : µ ∈ R, σ > 0 } Contains a open subset in R2, so T1(X) is also complete by Theorem 6.2.25. By Theorem 6.2.28, T X is also minimal sufficient.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 6 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Applying Theorem 6.2.25. and Theorem 6.2.28

A = {(w1(θ), w2(θ)) : θ ∈ R2} = { µ σ2 , − 1 2σ2 : µ ∈ R, σ > 0 } Contains a open subset in R2, so T1(X) is also complete by Theorem 6.2.25. By Theorem 6.2.28, T1(X) is also minimal sufficient.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 6 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Connecting T2(X) to T1(X)

T1(X) = ( n ∑

i=1

Xi,

n

∑

i=1

X2

i

) T2(X) = ( X, s2

X

) X

n i

Xi n

g T X sX

n i

Xi X n

n i

Xi

n i

Xi n n

g T X

n i

Xi nX g T X

n i

Xi n sX nX g T X Therefore, T X is an one-to-one function of T X , and also is sufficient, complete, and minimal sufficient.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 7 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Connecting T2(X) to T1(X)

T1(X) = ( n ∑

i=1

Xi,

n

∑

i=1

X2

i

) T2(X) = ( X, s2

X

) { X =

∑n

i=1 Xi

n

= g1(T1(X)) s2

X = ∑n

i=1(Xi−X)2

n−1

=

∑n

i=1 X2 i +∑n i=1 X2 i /n

n−1

= g2(T1(X))

n i

Xi nX g T X

n i

Xi n sX nX g T X Therefore, T X is an one-to-one function of T X , and also is sufficient, complete, and minimal sufficient.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 7 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Connecting T2(X) to T1(X)

T1(X) = ( n ∑

i=1

Xi,

n

∑

i=1

X2

i

) T2(X) = ( X, s2

X

) { X =

∑n

i=1 Xi

n

= g1(T1(X)) s2

X = ∑n

i=1(Xi−X)2

n−1

=

∑n

i=1 X2 i +∑n i=1 X2 i /n

n−1

= g2(T1(X)) { ∑n

i=1 Xi = nX = g−1 1 (T2(X))

∑n

i=1 X2 i = (n − 1)s2 X − nX 2 = g−1 2 (T2(X))

Therefore, T X is an one-to-one function of T X , and also is sufficient, complete, and minimal sufficient.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 7 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Connecting T2(X) to T1(X)

T1(X) = ( n ∑

i=1

Xi,

n

∑

i=1

X2

i

) T2(X) = ( X, s2

X

) { X =

∑n

i=1 Xi

n

= g1(T1(X)) s2

X = ∑n

i=1(Xi−X)2

n−1

=

∑n

i=1 X2 i +∑n i=1 X2 i /n

n−1

= g2(T1(X)) { ∑n

i=1 Xi = nX = g−1 1 (T2(X))

∑n

i=1 X2 i = (n − 1)s2 X − nX 2 = g−1 2 (T2(X))

Therefore, T2(X) is an one-to-one function of T1(X), and also is sufficient, complete, and minimal sufficient.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 7 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Example of Curved Exponential Family

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ N(µ, µ2). Determine whether the following statistic is

whether (1) sufficient (2) complete, and (3) minimal sufficient. T(X) = ( n ∑

i=1

Xi,

n

∑

i=1

X2

i

) .

How to solve it

. . . . . . . .

• Decompose fX x

in the form of an an exponential family.

• Apply Theorem 6.2.10 to obtain a sufficient statistic and see if it is

equivalent to or related to T X

• Apply Theorem 6.2.25 to see if it is complete.
• Apply Theorem 6.2.28 to see if it is minimal sufficient.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 8 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Example of Curved Exponential Family

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ N(µ, µ2). Determine whether the following statistic is

whether (1) sufficient (2) complete, and (3) minimal sufficient. T(X) = ( n ∑

i=1

Xi,

n

∑

i=1

X2

i

) .

How to solve it

. .

• Decompose fX(x|µ) in the form of an an exponential family.
• Apply Theorem 6.2.10 to obtain a sufficient statistic and see if it is

equivalent to or related to T(X)

• Apply Theorem 6.2.25 to see if it is complete.
• Apply Theorem 6.2.28 to see if it is minimal sufficient.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 8 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Applying Theorem 6.2.10

fX(x|µ) = 1 2πµ2 exp ( −1 2 ) exp ( 1 µx − x2 2µ2 ) where h x c exp w w t x x t x x By Theorem 6.2.10,

n i

t Xi

n i

t Xi

n i

Xi

n i

Xi T X is a sufficient statistic for

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 9 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Applying Theorem 6.2.10

fX(x|µ) = 1 2πµ2 exp ( −1 2 ) exp ( 1 µx − x2 2µ2 ) where                  h(x) = 1 c(µ) =

1 2πµ2 exp

( − 1

2

) w1(µ) = 1/µ w2(µ) = − 1

2µ2

t1(x) = x t2(x) = x2 By Theorem 6.2.10,

n i

t Xi

n i

t Xi

n i

Xi

n i

Xi T X is a sufficient statistic for

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 9 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Applying Theorem 6.2.10

fX(x|µ) = 1 2πµ2 exp ( −1 2 ) exp ( 1 µx − x2 2µ2 ) where                  h(x) = 1 c(µ) =

1 2πµ2 exp

( − 1

2

) w1(µ) = 1/µ w2(µ) = − 1

2µ2

t1(x) = x t2(x) = x2 By Theorem 6.2.10, (∑n

i=1 t1(Xi), ∑n i=1 t2(Xi)) =

(∑n

i=1 Xi, ∑n i=1 X2 i

) = T(X) is a sufficient statistic for µ

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 9 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Applying Theorem 6.2.25.

A = {(w1(µ), w2(µ) : µ ∈ R} = { 1 µ2 , − 1 2µ2 : µ ∈ R } A does not contains a open subset in , so we cannot apply Theorem 6.2.25. We need to go back to the definition

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 10 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Applying Theorem 6.2.25.

A = {(w1(µ), w2(µ) : µ ∈ R} = { 1 µ2 , − 1 2µ2 : µ ∈ R } A does not contains a open subset in R2, so we cannot apply Theorem 6.2.25. We need to go back to the definition

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 10 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Is T(X) = (∑n

i=1 Xi, ∑n i=1 X2 i ) Complete?

E ( n ∑

i=1

Xi ) = nµ E

n i

Xi nE Xi n E Xi Var Xi n n Note that

n i

Xi n n . E

n i

Xi E

n i

Xi Var

n i

Xi n n n n

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 11 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Is T(X) = (∑n

i=1 Xi, ∑n i=1 X2 i ) Complete?

E ( n ∑

i=1

Xi ) = nµ E ( n ∑

i=1

X2

i

) = nE ( X2

i

) n E Xi Var Xi n n Note that

n i

Xi n n . E

n i

Xi E

n i

Xi Var

n i

Xi n n n n

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 11 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Is T(X) = (∑n

i=1 Xi, ∑n i=1 X2 i ) Complete?

E ( n ∑

i=1

Xi ) = nµ E ( n ∑

i=1

X2

i

) = nE ( X2

i

) = n [ E (Xi)2 + Var(Xi) ] n n Note that

n i

Xi n n . E

n i

Xi E

n i

Xi Var

n i

Xi n n n n

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 11 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Is T(X) = (∑n

i=1 Xi, ∑n i=1 X2 i ) Complete?

E ( n ∑

i=1

Xi ) = nµ E ( n ∑

i=1

X2

i

) = nE ( X2

i

) = n [ E (Xi)2 + Var(Xi) ] = n(µ2 + µ2) = 2nµ2 Note that ∑n

i=1 Xi ∼ N(nµ, nµ2).

E

n i

Xi E

n i

Xi Var

n i

Xi n n n n

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 11 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Is T(X) = (∑n

i=1 Xi, ∑n i=1 X2 i ) Complete?

E ( n ∑

i=1

Xi ) = nµ E ( n ∑

i=1

X2

i

) = nE ( X2

i

) = n [ E (Xi)2 + Var(Xi) ] = n(µ2 + µ2) = 2nµ2 Note that ∑n

i=1 Xi ∼ N(nµ, nµ2).

E   ( n ∑

i=1

Xi )2  = [ E ( n ∑

i=1

Xi )]2 + Var ( n ∑

i=1

Xi ) n n n n

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 11 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Is T(X) = (∑n

i=1 Xi, ∑n i=1 X2 i ) Complete?

E ( n ∑

i=1

Xi ) = nµ E ( n ∑

i=1

X2

i

) = nE ( X2

i

) = n [ E (Xi)2 + Var(Xi) ] = n(µ2 + µ2) = 2nµ2 Note that ∑n

i=1 Xi ∼ N(nµ, nµ2).

E   ( n ∑

i=1

Xi )2  = [ E ( n ∑

i=1

Xi )]2 + Var ( n ∑

i=1

Xi ) = (nµ)2 + nµ2 = n(n + 1)µ2

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 11 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Is T(X) = (∑n

i=1 Xi, ∑n i=1 X2 i ) Complete? (cont’d)

Define g(T(X)) = g ( n ∑

i=1

Xi,

n

∑

i=1

X2

i

) = ∑n

i=1 X2 i

2n − (∑n

i=1 Xi)2

n(n + 1) E g T E

n i

Xi n E

n i

Xi n n n n n n n n for all . Because there exist g T such that E T and Pr g T , T X is NOT complete.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 12 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Is T(X) = (∑n

i=1 Xi, ∑n i=1 X2 i ) Complete? (cont’d)

Define g(T(X)) = g ( n ∑

i=1

Xi,

n

∑

i=1

X2

i

) = ∑n

i=1 X2 i

2n − (∑n

i=1 Xi)2

n(n + 1) E[g(T)|µ] = E (∑n

i=1 X2 i

) 2n − E (∑n

i=1 Xi)2

n(n + 1) = 2nµ2 2n − n(n + 1)µ2 n(n + 1) = 0 for all µ ∈ R. Because there exist g T such that E T and Pr g T , T X is NOT complete.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 12 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Is T(X) = (∑n

i=1 Xi, ∑n i=1 X2 i ) Complete? (cont’d)

Define g(T(X)) = g ( n ∑

i=1

Xi,

n

∑

i=1

X2

i

) = ∑n

i=1 X2 i

2n − (∑n

i=1 Xi)2

n(n + 1) E[g(T)|µ] = E (∑n

i=1 X2 i

) 2n − E (∑n

i=1 Xi)2

n(n + 1) = 2nµ2 2n − n(n + 1)µ2 n(n + 1) = 0 for all µ ∈ R. Because there exist g(T) such that E[T|µ] = 0 and Pr(g(T) = 0) < 1, T(X) is NOT complete.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 12 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Is T(X) = (∑n

i=1 Xi, ∑n i=1 X2 i ) Minimal Sufficient?

fX(x|µ) fX(y|µ) = exp [∑n

i=1 y2 i − ∑n i=1 x2 i

2µ2 + ∑n

i=1 xi − ∑n i=1 yi

µ ] The ratio above is a constant to if and only if

n i

xi

n i

yi

n i

xi

n i

yi which is equivalent to T x T y . Therefore, T X is a minimal sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 13 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Is T(X) = (∑n

i=1 Xi, ∑n i=1 X2 i ) Minimal Sufficient?

fX(x|µ) fX(y|µ) = exp [∑n

i=1 y2 i − ∑n i=1 x2 i

2µ2 + ∑n

i=1 xi − ∑n i=1 yi

µ ] The ratio above is a constant to µ if and only if { ∑n

i=1 x2 i = ∑n i=1 y2 i

∑n

i=1 xi = ∑n i=1 yi

which is equivalent to T x T y . Therefore, T X is a minimal sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 13 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Is T(X) = (∑n

i=1 Xi, ∑n i=1 X2 i ) Minimal Sufficient?

fX(x|µ) fX(y|µ) = exp [∑n

i=1 y2 i − ∑n i=1 x2 i

2µ2 + ∑n

i=1 xi − ∑n i=1 yi

µ ] The ratio above is a constant to µ if and only if { ∑n

i=1 x2 i = ∑n i=1 y2 i

∑n

i=1 xi = ∑n i=1 yi

which is equivalent to T(x) = T(y). Therefore, T(X) is a minimal sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 13 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary of Sufficiency Principle

• Model : P = {fX(x|θ), θ ∈ Ω}
• Statistic : T

T X where X X Xn . .

Sufficient Statistic

. . . . . . . . Contains all info about Definition fX x T X does not depend on Theorem 6.2.2 fX x qT T X does not depend on Factorization Theorem fX x h x g T X Exponential Family

n i

t Xi

n i

tk Xi is sufficient

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 14 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary of Sufficiency Principle

• Model : P = {fX(x|θ), θ ∈ Ω}
• Statistic : T = T(X) where X = (X1, · · · , Xn).

.

Sufficient Statistic

. . . . . . . . Contains all info about Definition fX x T X does not depend on Theorem 6.2.2 fX x qT T X does not depend on Factorization Theorem fX x h x g T X Exponential Family

n i

t Xi

n i

tk Xi is sufficient

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 14 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary of Sufficiency Principle

• Model : P = {fX(x|θ), θ ∈ Ω}
• Statistic : T = T(X) where X = (X1, · · · , Xn).

.

Sufficient Statistic

. . Contains all info about θ Definition fX x T X does not depend on Theorem 6.2.2 fX x qT T X does not depend on Factorization Theorem fX x h x g T X Exponential Family

n i

t Xi

n i

tk Xi is sufficient

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 14 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary of Sufficiency Principle

• Model : P = {fX(x|θ), θ ∈ Ω}
• Statistic : T = T(X) where X = (X1, · · · , Xn).

.

Sufficient Statistic

. . Contains all info about θ Definition fX(x|T(X)) does not depend on θ Theorem 6.2.2 fX x qT T X does not depend on Factorization Theorem fX x h x g T X Exponential Family

n i

t Xi

n i

tk Xi is sufficient

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 14 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary of Sufficiency Principle

• Model : P = {fX(x|θ), θ ∈ Ω}
• Statistic : T = T(X) where X = (X1, · · · , Xn).

.

Sufficient Statistic

. . Contains all info about θ Definition fX(x|T(X)) does not depend on θ Theorem 6.2.2 fX x qT T X does not depend on Factorization Theorem fX x h x g T X Exponential Family

n i

t Xi

n i

tk Xi is sufficient

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 14 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary of Sufficiency Principle

• Model : P = {fX(x|θ), θ ∈ Ω}
• Statistic : T = T(X) where X = (X1, · · · , Xn).

.

Sufficient Statistic

. . Contains all info about θ Definition fX(x|T(X)) does not depend on θ Theorem 6.2.2 fX(x|θ)/qT(T(X)|θ) does not depend on θ Factorization Theorem fX x h x g T X Exponential Family

n i

t Xi

n i

tk Xi is sufficient

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 14 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary of Sufficiency Principle

• Model : P = {fX(x|θ), θ ∈ Ω}
• Statistic : T = T(X) where X = (X1, · · · , Xn).

.

Sufficient Statistic

. . Contains all info about θ Definition fX(x|T(X)) does not depend on θ Theorem 6.2.2 fX(x|θ)/qT(T(X)|θ) does not depend on θ Factorization Theorem fX x h x g T X Exponential Family

n i

t Xi

n i

tk Xi is sufficient

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 14 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary of Sufficiency Principle

• Model : P = {fX(x|θ), θ ∈ Ω}
• Statistic : T = T(X) where X = (X1, · · · , Xn).

.

Sufficient Statistic

. . Contains all info about θ Definition fX(x|T(X)) does not depend on θ Theorem 6.2.2 fX(x|θ)/qT(T(X)|θ) does not depend on θ Factorization Theorem fX(x|θ) = h(x)g(T(X)|θ) Exponential Family

n i

t Xi

n i

tk Xi is sufficient

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 14 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary of Sufficiency Principle

• Model : P = {fX(x|θ), θ ∈ Ω}
• Statistic : T = T(X) where X = (X1, · · · , Xn).

.

Sufficient Statistic

. . Contains all info about θ Definition fX(x|T(X)) does not depend on θ Theorem 6.2.2 fX(x|θ)/qT(T(X)|θ) does not depend on θ Factorization Theorem fX(x|θ) = h(x)g(T(X)|θ) Exponential Family

n i

t Xi

n i

tk Xi is sufficient

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 14 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary of Sufficiency Principle

• Model : P = {fX(x|θ), θ ∈ Ω}
• Statistic : T = T(X) where X = (X1, · · · , Xn).

.

Sufficient Statistic

. . Contains all info about θ Definition fX(x|T(X)) does not depend on θ Theorem 6.2.2 fX(x|θ)/qT(T(X)|θ) does not depend on θ Factorization Theorem fX(x|θ) = h(x)g(T(X)|θ) Exponential Family (∑n

i=1 t1(Xi), · · · , ∑n i=1 tk(Xi)) is sufficient

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 14 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary of Sufficiency Principle (cont’d)

.

Minimal Sufficient Statistic

. . Sufficient statistic that achieves the maximum data reduction Definition T is sufficient and it is a function of all other sufficient statistics. Theorem 6.2.13 fX x fX y is constant as a function of T x T y Exponential Family (Theorem 6.2.28) Complete and sufficient statistic is minimal sufficient

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 15 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary of Sufficiency Principle (cont’d)

.

Minimal Sufficient Statistic

. . Sufficient statistic that achieves the maximum data reduction Definition T is sufficient and it is a function of all other sufficient statistics. Theorem 6.2.13 fX x fX y is constant as a function of T x T y Exponential Family (Theorem 6.2.28) Complete and sufficient statistic is minimal sufficient

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 15 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary of Sufficiency Principle (cont’d)

.

Minimal Sufficient Statistic

. . Sufficient statistic that achieves the maximum data reduction Definition T is sufficient and it is a function of all other sufficient statistics. Theorem 6.2.13 fX x fX y is constant as a function of T x T y Exponential Family (Theorem 6.2.28) Complete and sufficient statistic is minimal sufficient

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 15 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary of Sufficiency Principle (cont’d)

.

Minimal Sufficient Statistic

. . Sufficient statistic that achieves the maximum data reduction Definition T is sufficient and it is a function of all other sufficient statistics. Theorem 6.2.13 fX x fX y is constant as a function of T x T y Exponential Family (Theorem 6.2.28) Complete and sufficient statistic is minimal sufficient

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 15 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary of Sufficiency Principle (cont’d)

.

Minimal Sufficient Statistic

. . Sufficient statistic that achieves the maximum data reduction Definition T is sufficient and it is a function of all other sufficient statistics. Theorem 6.2.13 fX(x|θ)/fX(y|θ) is constant as a function of θ ⇐ ⇒ T(x) = T(y) Exponential Family (Theorem 6.2.28) Complete and sufficient statistic is minimal sufficient

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 15 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary of Sufficiency Principle (cont’d)

.

Minimal Sufficient Statistic

. . Sufficient statistic that achieves the maximum data reduction Definition T is sufficient and it is a function of all other sufficient statistics. Theorem 6.2.13 fX(x|θ)/fX(y|θ) is constant as a function of θ ⇐ ⇒ T(x) = T(y) Exponential Family (Theorem 6.2.28) Complete and sufficient statistic is minimal sufficient

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 15 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary of Sufficiency Principle (cont’d)

.

Minimal Sufficient Statistic

. . Sufficient statistic that achieves the maximum data reduction Definition T is sufficient and it is a function of all other sufficient statistics. Theorem 6.2.13 fX(x|θ)/fX(y|θ) is constant as a function of θ ⇐ ⇒ T(x) = T(y) Exponential Family (Theorem 6.2.28) Complete and sufficient statistic is minimal sufficient

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 15 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary of Sufficiency Principle (cont’d)

.

Complete Statistic

. . This family have to contain ”many” distributions in order to be complete. The restriction E[g(T)|θ] = 0, ∀θ ∈ Ω is strong enough to rule out all non-zero functions Definition E g T implies Pr g T . Exponential Family The parameter space is an open subset of

k.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 16 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary of Sufficiency Principle (cont’d)

.

Complete Statistic

. . This family have to contain ”many” distributions in order to be complete. The restriction E[g(T)|θ] = 0, ∀θ ∈ Ω is strong enough to rule out all non-zero functions Definition E g T implies Pr g T . Exponential Family The parameter space is an open subset of

k.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 16 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary of Sufficiency Principle (cont’d)

.

Complete Statistic

. . This family have to contain ”many” distributions in order to be complete. The restriction E[g(T)|θ] = 0, ∀θ ∈ Ω is strong enough to rule out all non-zero functions Definition E[g(T)|θ] = 0 implies Pr(g(T) = 0|θ) = 1. Exponential Family The parameter space is an open subset of

k.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 16 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary of Sufficiency Principle (cont’d)

.

Complete Statistic

. . This family have to contain ”many” distributions in order to be complete. The restriction E[g(T)|θ] = 0, ∀θ ∈ Ω is strong enough to rule out all non-zero functions Definition E[g(T)|θ] = 0 implies Pr(g(T) = 0|θ) = 1. Exponential Family The parameter space is an open subset of

k.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 16 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary of Sufficiency Principle (cont’d)

.

Complete Statistic

. . This family have to contain ”many” distributions in order to be complete. The restriction E[g(T)|θ] = 0, ∀θ ∈ Ω is strong enough to rule out all non-zero functions Definition E[g(T)|θ] = 0 implies Pr(g(T) = 0|θ) = 1. Exponential Family The parameter space Ω is an open subset of Rk.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 16 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Example

.

Problem

. . The random variable X takes the values 0, 1, 2, according to one of the following distributions: Pr(X = 0) Pr(X = 1) Pr(X = 2) Distribution 1 p 3p 1 − 4p 0 < p < 1

4

Distribution 2 p p2 1 − p − p2 0 < p < 1

2

In each case, determine whether the family of distribution of X is complete.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 17 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution - Distribution 1

Suppose that there exist g(·) such that E[g(X)|p] = 0 for all 0 < p < 1

4.

fX x p pI x p I x p I x E g X p

x

g x fX x p g p g p g p p g g g g Therefore, g , g g must hold, and it is possible that g is a nonzero function that makes Pr g X . For example, g g g . Therefore the family of distribution of X is not complete.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 18 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution - Distribution 1

Suppose that there exist g(·) such that E[g(X)|p] = 0 for all 0 < p < 1

4.

fX(x|p) = pI(x=0)(3p)I(x=1)(1 − 4p)I(x=2) E g X p

x

g x fX x p g p g p g p p g g g g Therefore, g , g g must hold, and it is possible that g is a nonzero function that makes Pr g X . For example, g g g . Therefore the family of distribution of X is not complete.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 18 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution - Distribution 1

Suppose that there exist g(·) such that E[g(X)|p] = 0 for all 0 < p < 1

4.

fX(x|p) = pI(x=0)(3p)I(x=1)(1 − 4p)I(x=2) E[g(X)|p] = ∑

x∈{0,1,2}

g(x)fX(x|p) g p g p g p p g g g g Therefore, g , g g must hold, and it is possible that g is a nonzero function that makes Pr g X . For example, g g g . Therefore the family of distribution of X is not complete.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 18 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution - Distribution 1

Suppose that there exist g(·) such that E[g(X)|p] = 0 for all 0 < p < 1

4.

fX(x|p) = pI(x=0)(3p)I(x=1)(1 − 4p)I(x=2) E[g(X)|p] = ∑

x∈{0,1,2}

g(x)fX(x|p) = g(0) · p + g(1) · (3p) + g(2) · (1 − 4p) p g g g g Therefore, g , g g must hold, and it is possible that g is a nonzero function that makes Pr g X . For example, g g g . Therefore the family of distribution of X is not complete.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 18 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution - Distribution 1

Suppose that there exist g(·) such that E[g(X)|p] = 0 for all 0 < p < 1

4.

fX(x|p) = pI(x=0)(3p)I(x=1)(1 − 4p)I(x=2) E[g(X)|p] = ∑

x∈{0,1,2}

g(x)fX(x|p) = g(0) · p + g(1) · (3p) + g(2) · (1 − 4p) = p[g(0) + 3g(1) − 4g(2)] + g(2) = 0 Therefore, g , g g must hold, and it is possible that g is a nonzero function that makes Pr g X . For example, g g g . Therefore the family of distribution of X is not complete.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 18 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution - Distribution 1

Suppose that there exist g(·) such that E[g(X)|p] = 0 for all 0 < p < 1

4.

fX(x|p) = pI(x=0)(3p)I(x=1)(1 − 4p)I(x=2) E[g(X)|p] = ∑

x∈{0,1,2}

g(x)fX(x|p) = g(0) · p + g(1) · (3p) + g(2) · (1 − 4p) = p[g(0) + 3g(1) − 4g(2)] + g(2) = 0 Therefore, g(2) = 0, g(0) + 3g(1) = 0 must hold, and it is possible that g is a nonzero function that makes Pr[g(X) = 0] < 1. For example, g g g . Therefore the family of distribution of X is not complete.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 18 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution - Distribution 1

Suppose that there exist g(·) such that E[g(X)|p] = 0 for all 0 < p < 1

4.

fX(x|p) = pI(x=0)(3p)I(x=1)(1 − 4p)I(x=2) E[g(X)|p] = ∑

x∈{0,1,2}

g(x)fX(x|p) = g(0) · p + g(1) · (3p) + g(2) · (1 − 4p) = p[g(0) + 3g(1) − 4g(2)] + g(2) = 0 Therefore, g(2) = 0, g(0) + 3g(1) = 0 must hold, and it is possible that g is a nonzero function that makes Pr[g(X) = 0] < 1. For example, g(0) = 1, g(1) = −3, g(2) = 0. Therefore the family of distribution of X is not complete.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 18 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution - Distribution 1

Suppose that there exist g(·) such that E[g(X)|p] = 0 for all 0 < p < 1

4.

fX(x|p) = pI(x=0)(3p)I(x=1)(1 − 4p)I(x=2) E[g(X)|p] = ∑

x∈{0,1,2}

g(x)fX(x|p) = g(0) · p + g(1) · (3p) + g(2) · (1 − 4p) = p[g(0) + 3g(1) − 4g(2)] + g(2) = 0 Therefore, g(2) = 0, g(0) + 3g(1) = 0 must hold, and it is possible that g is a nonzero function that makes Pr[g(X) = 0] < 1. For example, g(0) = 1, g(1) = −3, g(2) = 0. Therefore the family of distribution of X is not complete.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 18 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution - Distribution 2

Suppose that there exist g(·) such that E[g(X)|p] = 0 for all 0 < p < 1

4.

fX(x|p) = pI(x=0)(p2)I(x=1)(1 − p − p2)I(x=2) E g X p

x

g x fX x p g p g p g p p p g g p g g g g g g must hold in order to E g X p for all p. Therefore the family of distribution of X is complete.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 19 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution - Distribution 2

Suppose that there exist g(·) such that E[g(X)|p] = 0 for all 0 < p < 1

4.

fX(x|p) = pI(x=0)(p2)I(x=1)(1 − p − p2)I(x=2) E[g(X)|p] = ∑

x∈{0,1,2}

g(x)fX(x|p) g p g p g p p p g g p g g g g g g must hold in order to E g X p for all p. Therefore the family of distribution of X is complete.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 19 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution - Distribution 2

Suppose that there exist g(·) such that E[g(X)|p] = 0 for all 0 < p < 1

4.

fX(x|p) = pI(x=0)(p2)I(x=1)(1 − p − p2)I(x=2) E[g(X)|p] = ∑

x∈{0,1,2}

g(x)fX(x|p) = g(0) · p + g(1) · p2 + g(2) · (1 − p − p2) p g g p g g g g g g must hold in order to E g X p for all p. Therefore the family of distribution of X is complete.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 19 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution - Distribution 2

Suppose that there exist g(·) such that E[g(X)|p] = 0 for all 0 < p < 1

4.

fX(x|p) = pI(x=0)(p2)I(x=1)(1 − p − p2)I(x=2) E[g(X)|p] = ∑

x∈{0,1,2}

g(x)fX(x|p) = g(0) · p + g(1) · p2 + g(2) · (1 − p − p2) = p2[g(1) − g(2)] + p[g(0) − g(2)] + g(2) = 0 g(0) = g(1) = g(2) = 0 must hold in order to E[g(X)|p] = 0 for all p. Therefore the family of distribution of X is complete.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 19 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Another Example

.

Problem

. . Let X1, · · · , Xn be iid samples from fX(x|µ, λ) = ( λ 2πx3 )1/2 exp [ −λ(x − µ)2 2µ2x ] where x > 0. Show that X = 1

n

∑n

i=1 Xi and T = n ∑n

i=1 1 X− 1 X

are sufficient and complete.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 20 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution

fX(x|θ) = ( λ 2πx3 )1/2 exp [ −λ(x − µ)2 2µ2x ] x exp x x x x x x exp x x x e exp x x h x c exp w t x w t x

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 21 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution

fX(x|θ) = ( λ 2πx3 )1/2 exp [ −λ(x − µ)2 2µ2x ] = ( λ 2πx3 )1/2 exp [ − λx2 2µ2x + 2λµx 2µ2x − λµ2 2µ2x ] x exp x x x e exp x x h x c exp w t x w t x

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 21 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution

fX(x|θ) = ( λ 2πx3 )1/2 exp [ −λ(x − µ)2 2µ2x ] = ( λ 2πx3 )1/2 exp [ − λx2 2µ2x + 2λµx 2µ2x − λµ2 2µ2x ] = ( 1 2πx3 )1/2 λ1/2 exp [ − λ 2µ2 x + λ µ − λ 2 · 1 x ] x e exp x x h x c exp w t x w t x

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 21 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution

fX(x|θ) = ( λ 2πx3 )1/2 exp [ −λ(x − µ)2 2µ2x ] = ( λ 2πx3 )1/2 exp [ − λx2 2µ2x + 2λµx 2µ2x − λµ2 2µ2x ] = ( 1 2πx3 )1/2 λ1/2 exp [ − λ 2µ2 x + λ µ − λ 2 · 1 x ] = ( 1 2πx3 )1/2 λ1/2eλ/µ exp [ − λ 2µ2 x − λ 2 · 1 x ] h x c exp w t x w t x

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 21 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution

fX(x|θ) = ( λ 2πx3 )1/2 exp [ −λ(x − µ)2 2µ2x ] = ( λ 2πx3 )1/2 exp [ − λx2 2µ2x + 2λµx 2µ2x − λµ2 2µ2x ] = ( 1 2πx3 )1/2 λ1/2 exp [ − λ 2µ2 x + λ µ − λ 2 · 1 x ] = ( 1 2πx3 )1/2 λ1/2eλ/µ exp [ − λ 2µ2 x − λ 2 · 1 x ] = h(x)c(θ) exp [w1(θ)t1(x) + w2(θ)t2(x)]

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 21 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution (cont’d)

where h(x) = 1 2πx3 c e w t x x w t x x Therefore T X T X T X

n i

Xi

n i

Xi is a complete sufficient statistic because contains an open set in .

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 22 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution (cont’d)

where h(x) = 1 2πx3 c(θ) = λ1/2eλ/µ w t x x w t x x Therefore T X T X T X

n i

Xi

n i

Xi is a complete sufficient statistic because contains an open set in .

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 22 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution (cont’d)

where h(x) = 1 2πx3 c(θ) = λ1/2eλ/µ w1(θ) = − λ 2µ2 t x x w t x x Therefore T X T X T X

n i

Xi

n i

Xi is a complete sufficient statistic because contains an open set in .

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 22 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution (cont’d)

where h(x) = 1 2πx3 c(θ) = λ1/2eλ/µ w1(θ) = − λ 2µ2 t1(x) = x w t x x Therefore T X T X T X

n i

Xi

n i

Xi is a complete sufficient statistic because contains an open set in .

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 22 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution (cont’d)

where h(x) = 1 2πx3 c(θ) = λ1/2eλ/µ w1(θ) = − λ 2µ2 t1(x) = x w2(θ) = −λ 2 t x x Therefore T X T X T X

n i

Xi

n i

Xi is a complete sufficient statistic because contains an open set in .

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 22 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution (cont’d)

where h(x) = 1 2πx3 c(θ) = λ1/2eλ/µ w1(θ) = − λ 2µ2 t1(x) = x w2(θ) = −λ 2 t2(x) = 1 x Therefore T X T X T X

n i

Xi

n i

Xi is a complete sufficient statistic because contains an open set in .

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 22 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution (cont’d)

where h(x) = 1 2πx3 c(θ) = λ1/2eλ/µ w1(θ) = − λ 2µ2 t1(x) = x w2(θ) = −λ 2 t2(x) = 1 x Therefore T(X) = (T1(X), T2(X)) = (∑n

i=1 Xi, ∑n i=1 1/Xi) is a complete

sufficient statistic because θ = (λ, µ) contains an open set in R2.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 22 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution (cont’d)

Now, we need to show that X = 1

n

∑n

i=1 Xi and T = n ∑n

i=1 1 X− 1 X

are

• ne-to-one function of T(X).

X n

n i

Xi nT X T n

n i X X

n T X

n T X

T X nX T X n T X Therefore, X T are one-to-one function of T X T X and are also a sufficient complete statistic.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 23 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Solution (cont’d)

Now, we need to show that X = 1

n

∑n

i=1 Xi and T = n ∑n

i=1 1 X− 1 X

are

• ne-to-one function of T(X).

X = 1 n

n

∑

i=1

Xi = 1 nT1(X) T = n ∑n

i=1 1 X − 1 X

= n T2(X) −

n T1(X)

T1(X) = nX T2(X) = n T + 1 X Therefore, (X, T) are one-to-one function of (T1(X), T2(X)) and are also a sufficient complete statistic.

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 23 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary

.

Today

. .

• More Examples of Exponential Family
• Review of Chapter 6

.

Next Lecture

. . . . . . . .

• Likelihood Function
• Point Estimation

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 24 / 24

. . . . . .

. . . . . . . . . . . . Exponential Family . . . . . . . . . . Review . Summary

Summary

.

Today

. .

• More Examples of Exponential Family
• Review of Chapter 6

.

Next Lecture

. .

• Likelihood Function
• Point Estimation

Hyun Min Kang Biostatistics 602 - Lecture 08 February 5th, 2013 24 / 24