DISTRIBUTION ECE565: ESTIMATION DETECTION and FILTERING YOUSEF - - PowerPoint PPT Presentation

PARAMETERS ESTIMATION OF POLYA DISTRIBUTION

ECE565: ESTIMATION DETECTION and FILTERING

YOUSEF QASSIM, ARASH ABBASI DECEMBER 7TH,2011

CONTENTS

 Introduction  FIM and CRLB  Maximum Likelihood (MLE)  Method of Moments (MoM)

INTRODUCTION

MULTIVARIATE POLYA DISTRIBUTION

 x is vector of counts of each category.

x=(n1,n2,...,nK)

 pk is the probability to draw from category k

p=(p1,p2,...,pK)

 Dirichlet distribution with parameter vector

BETA BINOMIAL DISTRIBUTION

 This project focuses on estimating the parameters of

beta binomial distribution, with the parameters α1 and α2 . The distribution is given by where n(xi) is the length of x,

    

m k i k k i k 1 1 m i=1 k k i i k k k k

Γ( α ) n(x )! Γ(n (x )+α )) p(x ,x ,...,x )= ( ), n (x )! Γ(n(x )+ α )) Γ(α )

BETA BINOMIAL DISTRIBUTION

 One dimensional version of the multivariate

Polya distribution

 A family of discrete probability distribution on

finite support arising, where probability of success is random

 probability of success is drawn randomly from

the Beta distribution with the parameters α1 and α2

BETA BINOMIAL DISTRIBUTION

 Polya urn, where two colored balls green and

blue placed with a probability p, and then a ball is drawn randomly

 Coins with different probability p(m), and then

toss n times

THE PROBLEM

 Compute FIM and CRLB  Estimate the parameters α1 and α2 using

maximum likelihood and method of moment

 Compare MSE of MLE and MoM with CRLB

values

FIM AND CRLB

FIM AND CRLB

 Find the log likelihood function

FIM AND CRLB

 Take expectation

FIM AND CRLB

 No close form solution  Compute them numerically

2 2

      

   

k i k ik k k 2 k k k k i k k k k j

d logp(x|α) = Ψ ( α )-Ψ (n + α )+Ψ (n +α )-Ψ (α ) dα d logp(x|α) = Ψ ( α )-Ψ (n + α ) (k j) dα α

SINGLE OBSERVATION VECTOR CASE

SINGLE OBSERVATION VECTOR CASE

MULTI OBSERVATION VECTORS CASE

MULTI OBSERVATION VECTORS CASE

MAXIMUM LIKELIHOOD (MLE)

MAXIMUM LIKELIHOOD (MLE)

 Log likelihood function

MAXIMUM LIKELIHOOD (MLE)

 To find the maximum differentiate and set to

zero

 No close form sloution

MAXIMUM LIKELIHOOD (MLE)

 Estimate the parameters using Minka’s fixed

point iteration

 Find MSE

   

ik k k new i k k i k k k k i

Ψ(n +α )-Ψ(α ) α =α Ψ(n + α )-Ψ( α )

SINGLE OBSERVATION VECTOR

SINGLE OBSERVATION VECTOR

MULTI OBSERVATION VECTORS CASE

MULTI OBSERVATION VECTORS CASE

METHOD OF MOMENTS (MOM)

METHOD OF MOMENTS (MOM)

 Use the moments, and sample moments to

estimate the parameters

METHOD OF MOMENTS (MOM)

 Equating the above equations  Find MSE

RESULTS

RESULTS