Multi-parameter models Applied Bayesian Statistics Dr. Earvin - - PowerPoint PPT Presentation

Multi-parameter models

Applied Bayesian Statistics

• Dr. Earvin Balderama

Department of Mathematics & Statistics Loyola University Chicago

September 26, 2017

1

Multi-parameter models Last edited September 24, 2017 by Earvin Balderama <ebalderama@luc.edu>

Multi-parameter models

Examples

Most analyses consist of several parameters. Example Consider the normal model, Yi ∼ Normal(µ, σ2). We want to study the joint posterior distribution f(µ, σ2 |y1, . . . , yn). Another Example Consider the simple linear regression model, Yi ∼ Normal(β0 + β1X1, σ2). We want to study the joint posterior distribution f(β0, β1, σ2 |y1, . . . , yn).

2

Multi-parameter models Last edited September 24, 2017 by Earvin Balderama <ebalderama@luc.edu>

Multi-parameter models

The normal model

The joint distribution (bivariate PDF) of (µ, σ2) is f

• µ, σ2 |y1, . . . , yn
• ∝ f
• y1, . . . , yn |µ, σ2

f

• µ, σ2

∝ f

• y1, . . . , yn |µ, σ2

f

• µ |σ2

f

• σ2

∝ 1 σ −n exp

• −

(yi − µ)2 2σ2

• exp
• −

(µ − θ)2 2τ 2

• σ2−a−1 exp
• − b

σ2

• 3

Multi-parameter models Last edited September 24, 2017 by Earvin Balderama <ebalderama@luc.edu>

Multi-parameter models

Summarizing the posterior

How can we compute high-dimensional posterior distributions? How to summarize them concisely?

1

Computing the posterior on a grid.

2

Coding MCMC sampling ourselves in R.

3

Using BUGS, JAGS, or Stan for MCMC sampling.

4

Multi-parameter models Last edited September 24, 2017 by Earvin Balderama <ebalderama@luc.edu>

Multi-parameter models

Computing the posterior on a grid

For models with only a few parameters, it may be simple enough to plot the posterior distribution on a grid.

1

For all combinations of r values of µ and r values of σ2, compute f

• µ, σ2 |y1, . . . , yn
• 2

If there are p parameters in the model, the number of grid points is r p.

5

Multi-parameter models Last edited September 24, 2017 by Earvin Balderama <ebalderama@luc.edu>

Multi-parameter models

Summarizing the results

Typically, we are interested in the marginal posterior distribution, f(µ |y1, . . . , yn) = ∞ f

• µ, σ2 |y1, . . . , yn
• dσ2,

which accounts for uncertainty in σ2. We can then form inference on this one parameter, summaries credible intervals hypothesis testing For most analyses the marginal posteriors will not be nice distributions, and a grid is impossible if there are many parameters. Instead we will use Monte Carlo methods to draw representative samples from the posterior.

6

Multi-parameter models Last edited September 24, 2017 by Earvin Balderama <ebalderama@luc.edu>

Multi-parameter models

Drawing from the joint posterior

Recall in the normal model, if either µ or σ2 is known, we can sample from the

• ther parameter, using its full conditional distribution.

µ |y1, . . . , yn, σ2 ∼ Normal

• n¯

y+mθ n+m , σ2 n+m

• σ2
• y1, . . . , yn, µ ∼ InverseGamma

n

2 + a, SSE 2

+ b

• But how do we draw from the joint distribution?

7

Multi-parameter models Last edited September 24, 2017 by Earvin Balderama <ebalderama@luc.edu>