Exponential Family Lecture 07 Biostatistics 602 - Statistical - PowerPoint PPT Presentation

. . January 31st, 2013 Biostatistics 602 - Lecture 07 Hyun Min Kang January 31st, 2013 Hyun Min Kang Exponential Family Lecture 07 Biostatistics 602 - Statistical Inference . . Summary . Exponential Family . . . . . . . . . . . . . . . . . . . . . . 1 / 20

. 2 What is the relationship between complete statistic and ancillary January 31st, 2013 Biostatistics 602 - Lecture 07 Hyun Min Kang 5 Any example where Basu’s Theorem is helpful? . . 4 What is the Basu’s Theorem? . . complete statistics? 3 What is the characteristic shared among non-constant functions of . statistics? . . . sufficient statistics? 1 What are differences between complete statistic and minimal . . Last Lecture Summary . Exponential Family . . . . . . . . . . . . . . . . . . . . . . 2 / 20

. 2 What is the relationship between complete statistic and ancillary January 31st, 2013 Biostatistics 602 - Lecture 07 Hyun Min Kang 5 Any example where Basu’s Theorem is helpful? . . 4 What is the Basu’s Theorem? . . complete statistics? 3 What is the characteristic shared among non-constant functions of . . statistics? . . . sufficient statistics? 1 What are differences between complete statistic and minimal . . Last Lecture Summary . Exponential Family . . . . . . . . . . . . . . . . . . . . . . 2 / 20

. 2 What is the relationship between complete statistic and ancillary January 31st, 2013 Biostatistics 602 - Lecture 07 Hyun Min Kang 5 Any example where Basu’s Theorem is helpful? . . 4 What is the Basu’s Theorem? . . complete statistics? 3 What is the characteristic shared among non-constant functions of . . statistics? . . . sufficient statistics? 1 What are differences between complete statistic and minimal . . Last Lecture Summary . Exponential Family . . . . . . . . . . . . . . . . . . . . . . 2 / 20

. 2 What is the relationship between complete statistic and ancillary January 31st, 2013 Biostatistics 602 - Lecture 07 Hyun Min Kang 5 Any example where Basu’s Theorem is helpful? . . 4 What is the Basu’s Theorem? . . complete statistics? 3 What is the characteristic shared among non-constant functions of . . statistics? . . . sufficient statistics? 1 What are differences between complete statistic and minimal . . Last Lecture Summary . Exponential Family . . . . . . . . . . . . . . . . . . . . . . 2 / 20

. 2 What is the relationship between complete statistic and ancillary January 31st, 2013 Biostatistics 602 - Lecture 07 Hyun Min Kang 5 Any example where Basu’s Theorem is helpful? . . 4 What is the Basu’s Theorem? . . complete statistics? 3 What is the characteristic shared among non-constant functions of . . statistics? . . . sufficient statistics? 1 What are differences between complete statistic and minimal . . Last Lecture Summary . Exponential Family . . . . . . . . . . . . . . . . . . . . . . 2 / 20

t j x • w j • and t j x and h x only involve data. . w j where • d d k . j k k are functions of alone. Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 j exp . Summary . . . . . . . . . . . . . . . . . . . . . . Exponential Family . Exponential Family h x c . Definition 3.4.1 . . The random variable X belongs to an exponential family of distributions, if its pdf/pmf can be written in the form f x 3 / 20

• w j • and t j x and h x only involve data. . where • d d k . k j . are functions of alone. Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 k 3 / 20 . The random variable X belongs to an exponential family of distributions, if . Summary Exponential Family . Definition 3.4.1 . . . . . . . . . . . . . . . . . . . . . . . its pdf/pmf can be written in the form Exponential Family   ∑ f ( x | θ ) = h ( x ) c ( θ ) exp w j ( θ ) t j ( x )   j =1

• w j • and t j x and h x only involve data. . its pdf/pmf can be written in the form January 31st, 2013 Biostatistics 602 - Lecture 07 Hyun Min Kang alone. are functions of k j where k . 3 / 20 The random variable X belongs to an exponential family of distributions, if Exponential Family . . . . . Definition 3.4.1 . . . . . . . . . . . . . . . . . . Exponential Family . Summary . .   ∑ f ( x | θ ) = h ( x ) c ( θ ) exp w j ( θ ) t j ( x )   j =1 • θ = ( θ 1 , · · · , θ d ) , d ≤ k .

• and t j x and h x only involve data. . . January 31st, 2013 Biostatistics 602 - Lecture 07 Hyun Min Kang where k . its pdf/pmf can be written in the form The random variable X belongs to an exponential family of distributions, if . 3 / 20 Definition 3.4.1 . . . . . . . . . . . . . . . . . . Exponential Family Summary . . . Exponential Family . . .   ∑ f ( x | θ ) = h ( x ) c ( θ ) exp w j ( θ ) t j ( x )   j =1 • θ = ( θ 1 , · · · , θ d ) , d ≤ k . • w j ( θ ) , j ∈ { 1 , · · · , k } are functions of θ alone.

. . January 31st, 2013 Biostatistics 602 - Lecture 07 Hyun Min Kang where k . its pdf/pmf can be written in the form The random variable X belongs to an exponential family of distributions, if . 3 / 20 Definition 3.4.1 . Exponential Family Summary . Exponential Family . . . . . . . . . . . . . . . . . . . . . .   ∑ f ( x | θ ) = h ( x ) c ( θ ) exp w j ( θ ) t j ( x )   j =1 • θ = ( θ 1 , · · · , θ d ) , d ≤ k . • w j ( θ ) , j ∈ { 1 , · · · , k } are functions of θ alone. • and t j ( x ) and h ( x ) only involve data.

. . January 31st, 2013 Biostatistics 602 - Lecture 07 Hyun Min Kang where k . its pdf/pmf can be written in the form The random variable X belongs to an exponential family of distributions, if . 3 / 20 Definition 3.4.1 . Exponential Family Summary . Exponential Family . . . . . . . . . . . . . . . . . . . . . .   ∑ f ( x | θ ) = h ( x ) c ( θ ) exp w j ( θ ) t j ( x )   j =1 • θ = ( θ 1 , · · · , θ d ) , d ≤ k . • w j ( θ ) , j ∈ { 1 , · · · , k } are functions of θ alone. • and t j ( x ) and h ( x ) only involve data.

f X x x e x e f X x x , c e x x exp log x exp x log Define h x . . , w log , and t x x , then h x c exp w t x Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 e . . . . . . . . . . . . . . . . . . . . . . . . . Exponential Family . Summary Example of Exponential Family Problem . . . . Proof . . . . . 4 / 20 Show that a Poisson( λ ) ( λ > 0 ) belongs to the exponential family

x e x e f X x , w exp log x exp x log Define h x x , c e . . x , then h x c exp w t x Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 log , and t x 4 / 20 . Problem . . . . . . . . . . . . . . . . . . . Exponential Family . Summary Example of Exponential Family . . . . Proof . . . . Show that a Poisson( λ ) ( λ > 0 ) belongs to the exponential family e − λ λ x f X ( x | λ ) = x !

x e f X x . log , and t x exp x log Define h x x , c e , w x , then . h x c exp w t x Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 . 4 / 20 . . . . . . . . . . . . . . . . . . . . . . . . Exponential Family . Summary Example of Exponential Family Problem . . . Proof Show that a Poisson( λ ) ( λ > 0 ) belongs to the exponential family e − λ λ x f X ( x | λ ) = x ! 1 x ! e − λ exp ( log λ x ) =

f X x x , then . Define h x x , c e , w log , and t x . . h x c exp w t x Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 . 4 / 20 Proof Problem . . . . . . . . . . . . . . . . . . . . . . Exponential Family . Summary Example of Exponential Family . . . . Show that a Poisson( λ ) ( λ > 0 ) belongs to the exponential family e − λ λ x f X ( x | λ ) = x ! 1 x ! e − λ exp ( log λ x ) = 1 x ! e − λ exp ( x log λ ) =

f X x . . January 31st, 2013 Biostatistics 602 - Lecture 07 Hyun Min Kang t x exp w h x c . . Proof . . . . . . . . . . . . . . . . . . . . . . . . . . Exponential Family 4 / 20 Example of Exponential Family Summary . Problem Show that a Poisson( λ ) ( λ > 0 ) belongs to the exponential family e − λ λ x f X ( x | λ ) = x ! 1 x ! e − λ exp ( log λ x ) = 1 x ! e − λ exp ( x log λ ) = Define h ( x ) = 1/ x ! , c ( λ ) = e − λ , w ( λ ) = log λ , and t ( x ) = x , then

. Problem January 31st, 2013 Biostatistics 602 - Lecture 07 Hyun Min Kang . . Proof . . . . . . . . . Example of Exponential Family 4 / 20 . . . . . . . . . . . . . . . . . . Summary Exponential Family . Show that a Poisson( λ ) ( λ > 0 ) belongs to the exponential family e − λ λ x f X ( x | λ ) = x ! 1 x ! e − λ exp ( log λ x ) = 1 x ! e − λ exp ( x log λ ) = Define h ( x ) = 1/ x ! , c ( λ ) = e − λ , w ( λ ) = log λ , and t ( x ) = x , then f X ( x | λ ) = h ( x ) c ( λ ) exp [ w ( λ ) t ( x )]

f X x t j x x , then , k , w , t x x , w , t x . , c h x c exp k j w j Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 exp Define h x . Normal Distribution Belongs to an Exponential Family . . . . . . . . . . . . . . . . . . . . . . Exponential Family . Summary 5 / 20 x exp x − ( x − µ ) 2 1 [ ] f X ( x | θ = ( µ, σ 2 )) √ = 2 σ 2 2 πσ 2 exp