STA 214: Probability & Statistical Models STA 214: Analysis of - PowerPoint PPT Presentation

STA 214: Probability & Statistical Models

STA 214: Analysis of Statistical Models Probability/Statistical Models: Theory – prob/math stats Simulation – samples of y θ ( | ) p y Statistical analysis: Computation: Likelihood functions, MLEs Theory/maths, Numerical optimisation, θ ∝ θ ( ) ( | ) L p y Simulation Bayesian inference: Posterior distributions θ ∝ θ θ ( | ) ( ) ( | ) p y p p y Posterior samples ‘Complicated’ pdfs y = Random samples Challenge of dimension y = Stochastic processes – time series θ = Parameters and latent variables

STA 214: Simulation Methods ‘Complicated’ multivariate joint distribution P(x) Goal: Generate sample values x 1 , x 2 , x 3 , … - For input to an analysis (model simulators) - For understanding P(x) - For Monte Carlo integration: to approximate expectations E[ g(x) ] Independent samples: x i are independent: - Direct methods, possibly weighted samples Markov Chain methods (Markov Chain Monte Carlo - MCMC) – Markovian stochastic dependence � � � x x x x − 1 2 1 i i

STA 214: Simulation Methods Simulating univariate and multivariate distributions Direct methods, importance sampling: Many distributions MCMC theory and methods – sequential, ‘iterative’ methods - Gibbs sampling – standard, everyday tools - Metropolis methods Markov chains on continuous, multivariate state-spaces Simulation for: Investigating, understanding models – first example: simple (first order) stochastic process model and ‘volatility’ Statistical computation: Estimation, prediction, especially computations for ‘complicated’ posterior distributions

STA 214: Initial Example Models Simple models will begin course: AR models - AutoRegressive models Vehicles for - developing distribution theory - first examples of stochastic processes, Markov chains - getting started with simulation, computing - introduces a common but ‘complicated’ probability model (SV) - example context for developing simulation methods (direct and MCMC) for Bayesian analysis = φ + ε ε , ~ ( 0 , ) x x N v − 1 t t t t Course website – Supporting material/notes on AR models, time series, inference

STA 214: Initial Example Models Model: Probability and statistical modelling questions: = φ + ε , x x − 1 t t t - what ‘real data’ look like this model? ε - what is the joint distribution of a set of x ~ ( 0 , ) N v t variables under this model? - how does this all depend on the values of the model parameters? Stochastic process: - how can I simulate such a process? � � � , , , , , , , x x x x x - how can I simulate from p(x)? − 0 1 2 1 n n - model fitting – inference on parameters - prediction based on observed data and model fit? Vector random quantity: - more intricate problems (in which x is not = � ( , , , , )' x x x x x + + + actually observed) 1 2 s s s s n θ = φ Parameters: ( , ) v

STA 214: Models & Distribution Theory Gaussian Markov processes: AutoRegressive models - Linear algebra and linear systems theory related to AR models - Theory of AR models, stationary and nonstationary processes + Simulation and relation to Markov chain methods + Aspects of model fitting & inference + Stochastic volatility in finance: example and vehicle for much theory, simulation ideas, methodology Aspects of multivariate distribution theory - Simulation! Ideas, theory, methods - Multivariate normal – pervasive maths stats/structure + associated linear algebra and multivariate calculus - Families of multivariate normal & Wishart distributions - Many aspects of mixture models: data, probability structure - Data and model decompositions, and tools of multivariate analysis - eigentheory: Principal components, singular value/factors - Other distributions, e.g. gamma, multinomial, Dirichlet, …

STA 214: Models & Distribution Theory Examples of hidden Markov models Linear and nonlinear filtering and related ideas Simulation methods in hidden Markov models = φ + ε ~ ( | ), y p y x x x − 1 t t t t t t Multivariate models: vectors y, x, ε and matrices F,V. Φ = Φ + ε ( | ) ~ ( , ), y x N Fx V x x − 1 t t t t t t Elements of (mainly Gaussian) multivariate “Graphical” models Directed and undirected graphs, elements of graph theory Graphical models: multivariate distributions over graphs Relation to Markov models, multivariate statistical analysis, regression modelling

STA 214: Hidden Markov & Graphical Model Complicated multivariate joint distribution of all y, x ‘Simple’ graphical model – sets of conditional distributions � � � y y y y − 1 2 1 t t � � � x x x x − 1 2 1 t t x process is hidden (latent) and Markov (in time) Observed y process provides info on x θ = φ - parameters defining process characteristics ( , ) v

STA 214: In case you were wondering … Graphical model: Multilinear/Gaussian High-dimensional Sparse multivariate normal Dependencies ‘local’ Stats+genomics project: NeuroOncogenomics

STA 214: Support, Computing, Logistics Course web site: Links and references Support texts Notes TAs: Jenhwa, & Kai – homeworks, troubleshooting, etc Some of your commitment: Homeworks – weekly (mainly) Latex/Tex, will provide templates Assessment: Homeworks - 50% Midterm: 15% Final exam: 35% Software: Matlab Other courses of interest: ISDS 200/300 level courses What have I missed?