The Problem and its Modelling The Use for Markov-Chain Monte-Carlo Methods Computer Implementation and Future Work A Practical Implementation of the Gibbs Sampler for Mixture of Distributions: Application to the Determination of Specifications in Food Industry Julien Cornebise 1 Myriam Maumy 2 Philippe Girard 3 1 Ecole Sup´ erieure d’Informatique-Electronique-Automatique, and LSTA, Universit´ e Pierre et Marie Curie - Paris VI 2 IRMA Universit´ e Louis Pasteur - Strasbourg I 3 Quality Management Department, Nestl´ e ASMDA 2005, May 18 th , 2005 J. Cornebise, M. Maumy, P. Girard A Practical Implementation of the Gibbs Sampler . . .
The Problem and its Modelling The Use for Markov-Chain Monte-Carlo Methods Computer Implementation and Future Work The Problem and its Modelling 1 The Coffee Problem Mixture of Normal Laws Bayesian Inference The Use for Markov-Chain Monte-Carlo Methods 2 Choice of the Prior Gibbs Sampler Convergence Checking Label-Switching Model Selection Computer Implementation and Future Work 3 J. Cornebise, M. Maumy, P. Girard A Practical Implementation of the Gibbs Sampler . . .
The Problem and its Modelling The Use for Markov-Chain Monte-Carlo Methods Computer Implementation and Future Work The Problem and its Modelling 1 The Coffee Problem Mixture of Normal Laws Bayesian Inference The Use for Markov-Chain Monte-Carlo Methods 2 Choice of the Prior Gibbs Sampler Convergence Checking Label-Switching Model Selection Computer Implementation and Future Work 3 J. Cornebise, M. Maumy, P. Girard A Practical Implementation of the Gibbs Sampler . . .
The Problem and its Modelling The Use for Markov-Chain Monte-Carlo Methods Computer Implementation and Future Work The Problem and its Modelling 1 The Coffee Problem Mixture of Normal Laws Bayesian Inference The Use for Markov-Chain Monte-Carlo Methods 2 Choice of the Prior Gibbs Sampler Convergence Checking Label-Switching Model Selection Computer Implementation and Future Work 3 J. Cornebise, M. Maumy, P. Girard A Practical Implementation of the Gibbs Sampler . . .
The Problem and its Modelling The Coffee Problem The Use for Markov-Chain Monte-Carlo Methods Mixture of Normal Laws Computer Implementation and Future Work Bayesian Inference Outline The Problem and its Modelling 1 The Coffee Problem Mixture of Normal Laws Bayesian Inference The Use for Markov-Chain Monte-Carlo Methods 2 Choice of the Prior Gibbs Sampler Convergence Checking Label-Switching Model Selection Computer Implementation and Future Work 3 J. Cornebise, M. Maumy, P. Girard A Practical Implementation of the Gibbs Sampler . . .
The Problem and its Modelling The Coffee Problem The Use for Markov-Chain Monte-Carlo Methods Mixture of Normal Laws Computer Implementation and Future Work Bayesian Inference Pure coffee Adulterated coffee 1 Manufacturated with green 1 Addition of : coffee only Husk/Parchment Cereals 2 Low glucose rate Other plant extracts. . . 3 Low xylose rate 2 Glucose rate raises 3 Xylose rate raises J. Cornebise, M. Maumy, P. Girard A Practical Implementation of the Gibbs Sampler . . .
The Problem and its Modelling The Coffee Problem The Use for Markov-Chain Monte-Carlo Methods Mixture of Normal Laws Computer Implementation and Future Work Bayesian Inference Data and Quantities of Interest Provided : a set of 1002 coffee samples’ glucose and xylose rates Determine : 1 Number K of kinds of production : ( K − 1) different frauds, plus one for pure coffee 2 their parameters (mean, standard deviation) 3 their proportions 4 the specifications within which a soluble coffee can be considered as pure coffee J. Cornebise, M. Maumy, P. Girard A Practical Implementation of the Gibbs Sampler . . .
The Problem and its Modelling The Coffee Problem The Use for Markov-Chain Monte-Carlo Methods Mixture of Normal Laws Computer Implementation and Future Work Bayesian Inference Visualisation of the data J. Cornebise, M. Maumy, P. Girard A Practical Implementation of the Gibbs Sampler . . .
The Problem and its Modelling The Coffee Problem The Use for Markov-Chain Monte-Carlo Methods Mixture of Normal Laws Computer Implementation and Future Work Bayesian Inference Population k = 1 , . . . , K Normally distributed, parameters µ k , σ k ∀ 1 ≤ i ≤ T = 1002, observation x i comes from population k with probability π k , � K k =1 π k = 1. Density of the observations K � [ x i | µ , σ , π ] = π k N ( x i | µ k , σ k ) , 1 ≤ i ≤ T = 1002 k =1 µ = ( µ 1 , . . . , µ K ), σ = ( σ i , . . . , σ K ) , π = ( π 1 , . . . , π K ) [ ·|· ] denotes conditional probability density function (pdf) and parameters of the pdf (bayesian notation, Gelfand et al. , 1990). J. Cornebise, M. Maumy, P. Girard A Practical Implementation of the Gibbs Sampler . . .
The Problem and its Modelling The Coffee Problem The Use for Markov-Chain Monte-Carlo Methods Mixture of Normal Laws Computer Implementation and Future Work Bayesian Inference Simple example case, 2 populations [ x i | π, µ , σ ] = π N ( x i | µ 1 , σ 1 ) + (1 − π ) N ( x i | µ 2 , σ 2 ) Multiple different shapes : J. Cornebise, M. Maumy, P. Girard A Practical Implementation of the Gibbs Sampler . . .
The Problem and its Modelling The Coffee Problem The Use for Markov-Chain Monte-Carlo Methods Mixture of Normal Laws Computer Implementation and Future Work Bayesian Inference Augmented data : addition of z = ( z 1 , . . . , z T ) to the model, where z i indicates the population from wich observation x i comes from : ∀ i = 1 , . . . , T , z i ∈ { 1 , . . . , K } and [ z i = k ] = π k Thus : [ x i | µ , σ , π , z ] = N ( x i | µ z i , σ z i ) Other models exist, with many advantages, but lack the immediate physical interpretation (see for example Robert, in Droesbeke et al. (eds), 2002, or Marin et al. , in Dey and Rao (eds), to appear in 2005) J. Cornebise, M. Maumy, P. Girard A Practical Implementation of the Gibbs Sampler . . .
The Problem and its Modelling The Coffee Problem The Use for Markov-Chain Monte-Carlo Methods Mixture of Normal Laws Computer Implementation and Future Work Bayesian Inference Interested in estimating F ( µ , σ , π ), where : Function F can be The identity function, to estimate each parameter 99%-quantile of the “pure” population any other function Estimated through expectancy of posterior distribution [ µ , σ , π | x ] : Estimation � E [ F ( µ , σ , π ) | x ] F ( µ , σ , π ) = � = F ( µ , σ , π )[ µ , σ , π | x ] d ( µ , σ , π ) Θ where Θ is the space of the parameters, dimension 3 K − 1. J. Cornebise, M. Maumy, P. Girard A Practical Implementation of the Gibbs Sampler . . .
The Problem and its Modelling The Coffee Problem The Use for Markov-Chain Monte-Carlo Methods Mixture of Normal Laws Computer Implementation and Future Work Bayesian Inference The posterior density, key of the Bayesian inference, is simply obtained via : Bayes Formula, for posterior density [ µ , σ , π | x ] = [ x | µ , σ , π ] × [ µ , σ , π ] [ x ] where [ x | µ , σ , π ] comes from the model [ µ , σ , π ] is the prior distribution, carrying all information avalaible “a priori” (former experiences, experts’ knowledge, etc) [ x ] can be seen as a constant J. Cornebise, M. Maumy, P. Girard A Practical Implementation of the Gibbs Sampler . . .
Choice of the Prior The Problem and its Modelling Gibbs Sampler The Use for Markov-Chain Monte-Carlo Methods Convergence Checking Computer Implementation and Future Work Label-Switching Model Selection Outline The Problem and its Modelling 1 The Coffee Problem Mixture of Normal Laws Bayesian Inference The Use for Markov-Chain Monte-Carlo Methods 2 Choice of the Prior Gibbs Sampler Convergence Checking Label-Switching Model Selection Computer Implementation and Future Work 3 J. Cornebise, M. Maumy, P. Girard A Practical Implementation of the Gibbs Sampler . . .
Choice of the Prior The Problem and its Modelling Gibbs Sampler The Use for Markov-Chain Monte-Carlo Methods Convergence Checking Computer Implementation and Future Work Label-Switching Model Selection Analysis of mixture of distributions using MCMC methods has been the subject of many publications, for example : Diebolt and Robert, 1990, Richardson and Green, 1997, Stephens, 1997, Marin et al. , to appear in 2005. Gibbs sampler and connected questions also has been treated in much details, for example by : Gelfand et al. , 1990 Gelman and Rubin, 1992 Carlin and Chib, 1995 Kass and Raftery, 1995 Celeux et al. , 2000 Gelman et al. , 2003 . . . J. Cornebise, M. Maumy, P. Girard A Practical Implementation of the Gibbs Sampler . . .
Recommend
More recommend
