SLIDE 1
STA 214: Probability & Statistical Models STA 214: Analysis of - - PowerPoint PPT Presentation
STA 214: Probability & Statistical Models STA 214: Analysis of - - PowerPoint PPT Presentation
STA 214: Probability & Statistical Models STA 214: Analysis of Statistical Models Probability/Statistical Models: Theory prob/math stats Simulation samples of y ( | ) p y Statistical analysis: Computation: Likelihood
SLIDE 2
SLIDE 3
STA 214: Simulation Methods
i i
x x x x
1 2 1 −
- ‘Complicated’ multivariate joint distribution P(x)
Goal: Generate sample values x1, x2, x3, …
- For input to an analysis (model simulators)
- For understanding P(x)
- For Monte Carlo integration: to approximate
expectations E[ g(x) ]
Independent samples: xi are independent:
- Direct methods, possibly weighted samples
Markov Chain methods (Markov Chain Monte Carlo - MCMC)
– Markovian stochastic dependence
SLIDE 4
STA 214: Simulation Methods
Simulating univariate and multivariate distributions
Direct methods, importance sampling: Many distributions MCMC theory and methods – sequential, ‘iterative’ methods
- Gibbs sampling – standard, everyday tools
- Metropolis methods
Markov chains on continuous, multivariate state-spaces Simulation for: Investigating, understanding models – first example: simple (first order) stochastic process model and ‘volatility’ Statistical computation: Estimation, prediction, especially computations for ‘complicated’ posterior distributions
SLIDE 5
STA 214: Initial Example Models
Simple models will begin course: AR models - AutoRegressive models Vehicles for
- developing distribution theory
- first examples of stochastic processes, Markov chains
- getting started with simulation, computing
- introduces a common but ‘complicated’ probability model (SV)
- example context for developing simulation methods
(direct and MCMC) for Bayesian analysis
) , ( ~ ,
1
v N x x
t t t t
ε ε φ + =
−
Course website –
Supporting material/notes on AR models, time series, inference
SLIDE 6
STA 214: Initial Example Models
Probability and statistical modelling questions:
- what ‘real data’ look like this model?
- what is the joint distribution of a set of x
variables under this model?
- how does this all depend on the values of
the model parameters?
- how can I simulate such a process?
- how can I simulate from p(x)?
- model fitting – inference on parameters
- prediction based on observed data and
model fit?
- more intricate problems (in which x is not
actually observed)
) , ( ~ ,
1
v N x x
t t t t
ε ε φ + =
−
- ,
, , , , , ,
1 2 1 n n
x x x x x
−
)' , , , , (
2 1 n s s s s
x x x x x
+ + +
=
- Vector random quantity:
Stochastic process: Model: Parameters:
) , ( v φ θ =
SLIDE 7
STA 214: Models & Distribution Theory
Gaussian Markov processes: AutoRegressive models
- Linear algebra and linear systems theory related to AR models
- Theory of AR models, stationary and nonstationary processes
+ Simulation and relation to Markov chain methods + Aspects of model fitting & inference + Stochastic volatility in finance:
example and vehicle for much theory, simulation ideas, methodology
Aspects of multivariate distribution theory
- Simulation! Ideas, theory, methods
- Multivariate normal – pervasive maths stats/structure
+ associated linear algebra and multivariate calculus
- Families of multivariate normal & Wishart distributions
- Many aspects of mixture models: data, probability structure
- Data and model decompositions, and tools of multivariate analysis
- eigentheory: Principal components, singular value/factors
- Other distributions, e.g. gamma, multinomial, Dirichlet, …
SLIDE 8
STA 214: Models & Distribution Theory
Examples of hidden Markov models
Linear and nonlinear filtering and related ideas Simulation methods in hidden Markov models Multivariate models: vectors y, x, ε and matrices F,V.Φ
Elements of (mainly Gaussian) multivariate “Graphical” models
Directed and undirected graphs, elements of graph theory Graphical models: multivariate distributions over graphs Relation to Markov models, multivariate statistical analysis, regression modelling
t t t t t t
x x x y p y ε φ + =
−1
), | ( ~
t t t t t t
x x V Fx N x y ε + Φ =
−1
), , ( ~ ) | (
SLIDE 9
STA 214: Hidden Markov & Graphical Model
t t
y y y y
1 2 1 −
- t
t
x x x x
1 2 1 −
- x process is hidden (latent) and Markov (in time)
Observed y process provides info on x
- parameters defining process characteristics
Complicated multivariate joint distribution of all y, x ‘Simple’ graphical model – sets of conditional distributions
) , ( v φ θ =
SLIDE 10
STA 214: In case you were wondering …
Stats+genomics project: NeuroOncogenomics
Graphical model:
Multilinear/Gaussian High-dimensional Sparse multivariate normal Dependencies ‘local’
SLIDE 11