LMS and GAMLSS Flexible Regression and Smoothing Mikis Stasinopoulos - - PowerPoint PPT Presentation
LMS and GAMLSS LMS and GAMLSS Flexible Regression and Smoothing Mikis Stasinopoulos 1 and Bob Rigby 1 1 STORM/FLSC, London Metropolitan University Royal Statistical Society, February 2016 1 LMS and GAMLSS Outline 1 The Lambda, Mu and Sigma
LMS and GAMLSS
Flexible Regression and Smoothing Mikis Stasinopoulos1 and Bob Rigby1
1STORM/FLSC, London Metropolitan University Royal Statistical Society, February 2016 1
LMS and GAMLSS Outline
1 The Lambda, Mu and Sigma method (LMS)
The problem The method Extensions of the method
2 The Generalised Additive Model for Location Scale and Shape
What is GAMLSS? Distributions and Additive terms R implementation
3 The lung function data 4 Conclusions
2
LMS and GAMLSS The Lambda, Mu and Sigma method (LMS)
The LMS method is a nice example of a model built to solve a problem. This is also typical of Tim’s work who developed the growth curve methodology to solve specific problems and then applied it to a variety of fields:
Nuitricion Medicine ...
3
LMS and GAMLSS The Lambda, Mu and Sigma method (LMS) The problem
BMI : the BMI of 7294 boys age : the age in years Source: van Buuren and Fredriks (2001)
4
LMS and GAMLSS The Lambda, Mu and Sigma method (LMS) The problem
5 10 15 20 15 20 25 30 35 age BMI
5
LMS and GAMLSS The Lambda, Mu and Sigma method (LMS) The problem
age Percent of Total
5 10 15 20 25 10 15 20 25 30 35
(1,2] (2,3]
10 15 20 25 30 35
(3,4] (4,5]
10 15 20 25 30 35
(5,6] (6,7] (7,8] (8,9] (9,10]
5 10 15 20 25
(10,11]
5 10 15 20 25
(11,12] (12,13] (13,14] (14,15] (15,16] (16,17]
10 15 20 25 30 35
(17,18] (18,19]
10 15 20 25 30 35 5 10 15 20 25
(19,20]
6
LMS and GAMLSS The Lambda, Mu and Sigma method (LMS) The problem
5 10 15 20 15 20 25 30 35 age BMI
Centile curves using BCT 7
LMS and GAMLSS The Lambda, Mu and Sigma method (LMS) The problem
5 10 15 20 15 20 25 30 35 age BMI
Centile curves using BCT 8
LMS and GAMLSS The Lambda, Mu and Sigma method (LMS) The problem
6 8 10 12 14 16 18 20 10 15 20 25 30 age bmi 9
LMS and GAMLSS The Lambda, Mu and Sigma method (LMS) The problem
Obesity Overw eight Normal Thinness Severe thinness
BMI (kg/m²) Age (completed months and years)
1 2
3 6 9 3 6 9 3 6 9 3 6 9 3 6 9 3 6 9 3 6 9 3 6 9 3 6 9 3 6 9 3 6 9 3 6 9 3 6 9 3 6 9 Months Years 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 10 12 14 16 18 20 22 24 26 28 30 32 10 12 14 16 18 20 22 24 26 28 30 322007 WHO Reference
BMI-for-age GIRLS
5 to 19 years (z-scores)
10
LMS and GAMLSS The Lambda, Mu and Sigma method (LMS) The problem
2007 WHO Reference
BMI-for-age BOYS
5 to 19 years (z-scores)
Obesity Overw eight Normal Thinness Severe thinness
BMI (kg/m²) Age (completed months and years)
1 2
3 6 9 3 6 9 3 6 9 3 6 9 3 6 9 3 6 9 3 6 9 3 6 9 3 6 9 3 6 9 3 6 9 3 6 9 3 6 9 3 6 9 Months Years 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 10 12 14 16 18 20 22 24 26 28 30 32 10 12 14 16 18 20 22 24 26 28 30 3211
LMS and GAMLSS The Lambda, Mu and Sigma method (LMS) The method
Let Y be a random variable with range Y > 0 defined through the transformed variable Z given by: Z = 1 σν Y µ λ − 1
if λ = 0 = 1 σ log Y µ
if λ = 0. where Z ∼ N(0, 1) (a truncated normal) µ = h1(age) σ = h2(age) λ = h3(age) where h() are smooth functions.
12
LMS and GAMLSS The Lambda, Mu and Sigma method (LMS) The method
1 Y ∼ D(µ, σ, λ) 2 All three parameters are smooth-function of one explanatory
variable age
3 The parameters can be interpreted as:
µ (approximate the median) a location parameter σ (approximate a coefficient of variation) a scale parameter λ (which from now on I will call ν) a skewness parameter Is there anything missing?
13
LMS and GAMLSS The Lambda, Mu and Sigma method (LMS) Extensions of the method
1 Y ∼ D(µ, σ, ν, τ) 2
g1(µ) = h1(age): modelling the location parameter g2(σ) = h2(age): modelling the scale parameter g3(ν) = h3(age): modelling the skewness parameter g4(τ) = h4(age): modelling the kutrosis parameter
3 the g() functions are link functions 14
LMS and GAMLSS The Lambda, Mu and Sigma method (LMS) Extensions of the method
Different assumptions about Z produce different LMS methods
1 if Z ∼ N(0, 1) then Y ∼ BCCG(µ, σ, ν) = LMS method 2 if Z ∼ tτ then Y ∼ BCT(µ, σ, ν, τ) = LMST method 3 if Z ∼ PE(0, 1, τ) then Y ∼ BCPE(µ, σ, ν, τ) = LMSP
method adopted by WHO
15
LMS and GAMLSS The Generalised Additive Model for Location Scale and Shape
Generalised additive model for location scale and shape Rigby and Stasinopoulos (2005) y ∼ D(µ, σ, ν, τ) gµ(µ) = Xµβµ + h1,µ(x1,µ) + ... + hk,µ(xk,µ) gσ(σ) = Xσβσ + h1,σ(x1,σ) + ... + hk,σ(xk,σ) gν(ν) = Xνβν + h1,ν(x1,ν) + ... + hk,ν(xk,ν) gτ(τ) = Xτβτ + h1,τ(x1,τ) + ... + hk,τ(xk,τ) where D(µ, σ, ν, τ) can be any distribution and where hj(xj) are smooth functions of the X’s.
16
LMS and GAMLSS The Generalised Additive Model for Location Scale and Shape
6 8 10 12 14 16 18 5 10 15 x y 17
LMS and GAMLSS The Generalised Additive Model for Location Scale and Shape What is GAMLSS?
GAMLSS: are semi-parametric regression type models. regression type: we have many explanatory variables X and
parametric: a parametric distribution assumption for the response variable, semi: the parameters of the distribution, as functions of explanatory variables, may involve non-parametric smoothing functions GAMLSS philosophy: try different models GAMLSS is a generalisation of GLM and GAM models.
18
LMS and GAMLSS The Generalised Additive Model for Location Scale and Shape Distributions and Additive terms
There are around 80 discrete
discrete , continuous continuous , and
mixed distributions,
mixed , implemented as gamlss.family in the
R including highly skew and kurtotic distributions
Shapes ,
creating a new distribution is relatively easy truncating
truncated an existing distribution
using a censored version of an existing distribution mixing
mixture different distributions to create a new finite
mixture distribution. discretise
discretise continuous distributions
log or logit any continuous distribution in (−∞, ∞)
19
LMS and GAMLSS The Generalised Additive Model for Location Scale and Shape Distributions and Additive terms
Additive terms R Name P-splines pb(), pbm(), cy() Varying coefficient pvc() Cubic splines cs() loess/ neural networks lo(), nn() Fractional/picewise polynomials fp(), fk() non-linear fit nl() Random effects random(), re() Ridge regression ri() Simon Wood’s GAM ga() decision threes tr() random walk and AR rw(), ar()
20
LMS and GAMLSS The Generalised Additive Model for Location Scale and Shape R implementation
GAMLSS is implemented in series of packages in R gamlss the original package gamlss.dist for distributions gamlss.data for distributions gamlss.demo for demos gamlss.nl for non-linear terms gamlss.tr for truncated distributions gamlss.cens for censored (left, right or interval) response variables gamlss.mx for finite mixtures and random effects gamlss.spatial for Gaussian Markov Random fields The GAMLSS packages can be downloaded from CRAN, the R library at http://www.r-project.org/
21
LMS and GAMLSS The lung function data
100 120 140 160 180 200 0.5 0.6 0.7 0.8 0.9 1.0 height FEV1/FVC 22
LMS and GAMLSS The lung function data
Y = FEV1/FVC : the Spirometric lung function an established index for diagnosing airway obstruction (3164 male) age : the height in cm Source: Stanojevic et al. 2009
23
LMS and GAMLSS The lung function data
four different models Hossain et al. (2015)
1 The LMS model 2 Beta Inflated (at 1) 3 Logit skew student t distribution (logitSST) inflated (at 1) 4 Generalised Tobit models 24
LMS and GAMLSS The lung function data
Y ∼ BCT(µ, σ, ν, τ) g1(µ) = h1(x) g2(σ) = h2(x) g3(ν) = h3(x) g4(τ) = h4(x) x = vξ.
25
LMS and GAMLSS The lung function data
Y ∼ BEINF1(µ, σ, ν) g1(µ) = h1(x) g2(σ) = h2(x) g3(ν) = h3(x) x = vξ.
26
LMS and GAMLSS The lung function data
Y ∼ logitSSTat1(µ, σ, ν.τ, ξ) µ = h1(x) log σ = h2(x) log ν = h3(x) log τ = h4(x) log
1 − p1
h5(x). x = vξ.
27
LMS and GAMLSS The lung function data
V ∼ D(µ, σ, ν, τ) be a flexible uncensored distribution on (0, ∞). Let Y ∼ Drc(µ, σ, ν, τ) be the corresponding right censored distribution on (0, 1], i,e censored above 1. Hence the probability (density) function of Y is given by fY (y) = fV (y) if 0 < y < 1 P(V ≥ 1) if y = 1
28
LMS and GAMLSS The lung function data
100 120 140 160 180 200 0.5 0.6 0.7 0.8 0.9 1.0 height FEV1/FVC
(a) LMS
100 120 140 160 180 200 0.5 0.6 0.7 0.8 0.9 1.0 height FEV1/FVC
(b) BEINF1
100 120 140 160 180 200 0.5 0.6 0.7 0.8 0.9 1.0 height FEV1/FVC
(c) Inf. logitSST
100 120 140 160 180 200 0.5 0.6 0.7 0.8 0.9 1.0 height FEV1/FVC
(d) Gen. Tobit 29
LMS and GAMLSS The lung function data
87 to 103 103 to 106 106 to 109 109 to 112 112 to 114 114 to 117 117 to 120 120 to 125 125 to 133 133 to 145 145 to 155 155 to 166 166 to 172 172 to 176 176 to 181 181 to 207 Z1 Z2 Z3 Z4
Z-Statistics (a) LMS
87 to 103 103 to 106 106 to 109 109 to 112 112 to 114 114 to 117 117 to 120 120 to 125 125 to 133 133 to 145 145 to 155 155 to 166 166 to 172 172 to 176 176 to 181 181 to 207 Z1 Z2 Z3 Z4
Z-Statistics (b) BEINF1
87 to 103 103 to 106 106 to 109 109 to 112 112 to 114 114 to 117 117 to 120 120 to 125 125 to 133 133 to 145 145 to 155 155 to 166 166 to 172 172 to 176 176 to 181 181 to 207 Z1 Z2 Z3 Z4
Z-Statistics (c) Inf.logitSST
87 to 103 103 to 106 106 to 109 109 to 112 112 to 114 114 to 117 117 to 120 120 to 125 125 to 133 133 to 145 145 to 155 155 to 166 166 to 172 172 to 176 176 to 181 181 to 207 Z1 Z2 Z3 Z4
Z-Statistics (d) Gen.Tobit 30
LMS and GAMLSS The lung function data
31
LMS and GAMLSS The lung function data
Flexible Regression and Smoothing (A practical guide to GAMLSS in R ) Distributions for Modelling Location Scale and Shape, (the GAMLSS collection) Generalised additive models for location, scale and shape, (Classical, Bayesian and boosting approaches)
32
LMS and GAMLSS Conclusions
GAMLSS is a very flexible statistical model It is a unified framework for univariate regression type of models Allows any distribution for the response variable Y Models all the parameters of the distribution of Y Allows a variety of penalised additive terms in the models for the distribution parameters The fitted algorithm is modular, where different components can be added easily it can easily introduced to students since it relies on known concepts It deals with overdispersion, skewness and kurtosis
33
LMS and GAMLSS Conclusions
present past Vlasios Voudouris Popi Akantziliotou Paul Eilers Nicoleta Mortan Gillian Heller Fiona McElduff Marco Enea Raydonal Ospina Majid Djennad Konstantinos Pateras Fernanda De Bastiani Luiz Nakamura Daniil Kiose Andreas Mayr Thomas Kneib Nadja Klein Abu Hossain
34
LMS and GAMLSS Conclusions
for more information see
35
LMS and GAMLSS Conclusions
Go back Distributions
2 4 6 8 10 0.00 0.05 0.10 0.15
Zero adjusted GA
y pdf
10 15 20 0.0 0.4 0.8
Zero adjusted Gamma c.d.f.
y F(y)
LMS and GAMLSS Conclusions
Go back Distributions
5 10 15 20 0.00 0.05 0.10 0.15
negative skewness
y f(y) 5 10 15 20 25 30 0.00 0.05 0.10 0.15
positive skewness
y f(y) 5 10 15 20 0.00 0.04 0.08
platy−kurtosis
y f(y) 5 10 15 20 0.00 0.10 0.20
lepto−kurtosis
y f(y)
37
LMS and GAMLSS Conclusions
Go back Distributions
−4 −2 2 4 0.0 0.4 0.8 1.2
(a)
y f(y) −4 −2 2 4 0.0 0.2 0.4 0.6 0.8
(b)
y f(y) −4 −2 2 4 0.0 0.2 0.4 0.6
(c)
y f(y) −4 −2 2 4 0.0 0.2 0.4 0.6 0.8 1.0
(d)
y f(y)
38
LMS and GAMLSS Conclusions
Go back Distributions 39
LMS and GAMLSS Conclusions
Go back Distributions 40
LMS and GAMLSS Conclusions
Go back Distributions 41
LMS and GAMLSS Conclusions
Go back to distributions
1 4 7 11 15 19 23
(b) Poisson
0.0 0.2 0.4 0.6
4 7 11 15 19 23
(c) negative binomial II
0.0 0.2 0.4 0.6
4 7 11 15 19 23
(c) Delaporte
0.0 0.2 0.4 0.6
4 7 11 15 19 23
(d) Sichel
0.0 0.2 0.4 0.6
LMS and GAMLSS Conclusions
Go back 0e+00 1e+07 2e+07 3e+07 4e+07 5e+07 6e+07 0e+00 1e+08 2e+08 3e+08 4e+08 Opening Box−Office Revenue Post Opening Revenue 4031 film openings in US 1988−1999 43
LMS and GAMLSS Conclusions
A small Quiz Question 1 which film in the 90’s has made the most money? Question 2 which film in the 90’s did badly in the first week but subsequently did very well? Question 3 which film in the 90’s did well in the first week but flopped after that?
Go back 44
LMS and GAMLSS Conclusions
Go back 0e+00 2e+07 4e+07 6e+07 0e+00 1e+08 2e+08 3e+08 4e+08 Opening Box−Office Revenue Post Opening Revenue 4031 film openings in US 1988−1999
LION KING, THE TITANIC HOME ALONE ANGELA'S ASHES INDEPENDENCE DAY BATMAN STAR WARS: EP.1−PHANTOM MENACE MISSION: IMPOSSIBLE LOST WORLD: JURASSIC PARK, THE
45
LMS and GAMLSS Conclusions
Linear Model, Gauss y = Xβ + ǫ where ǫ ∼ NO(0, σ2I) The model can be also written as: y ∼ NO(µ, σ2I) where µ = Xβ
Go back 70-80 46
LMS and GAMLSS Conclusions
Go back 70-80 Next page 5 10 15 20 25 30 35 40 100 200 300 400 y 47
LMS and GAMLSS Conclusions
Go back 70-80 Next page 5 10 15 20 25 30 35 40 100 200 300 400 y 48
LMS and GAMLSS Conclusions
estimation is achieved by Least Squares or Weighted Least Squares (WLS) the normal distribution is important for inference we only modelling the mean as linear function of the explanatory variables One of the top ten reasons to become statistician (according to Friedman, Friedman & Amoo, 2002, Journal of Statistics Education): “Statisticians are mean lovers” .
Go back 70-80 49
LMS and GAMLSS Conclusions
Generalised Linear Model, Nelder and Wedderburn(1972) g(µ) = Xβ where y ∼ ExpFamily(µ, φ) where g() is the link function The exponential family
1 normal 2 Gamma 3 inverse Gaussian 4 Poisson 5 binomial Go back 70-80 Next page 50
LMS and GAMLSS Conclusions
Go back 70-80 Next page 2 4 6 8 10 20 40 60 80 y 51
LMS and GAMLSS Conclusions
estimation is achieved by Iterative Reweighted Least Squares (IRLS) we can model discrete response variables we are still “mean lovers” .
Go back 70-80 52
LMS and GAMLSS Conclusions
Generalised additive model Hastie and Tibshirani (1990) y ∼ ExpFamily(µ, φ) g(µ) = Xβ + h1(x1) + ... + hk(xk) where hj(xj) are smooth functions of the X’s.
Go back Historical Next page 53
LMS and GAMLSS Conclusions
Go back Historical Next page
40 60 80 100 120 −0.4 0.0 0.2 0.4 0.6 Fl Partial for pb(Fl) 1900 1920 1940 1960 1980 −0.2 0.0 0.2 0.4 A Partial for pb(A) −0.5 −0.3 −0.1 Partial for B 1
54
LMS and GAMLSS Conclusions
the estimation is achieved by modified backfitting modified backfitting: is a combination of IRLS for the linear part and penalised IRLS for the smoothing part. the smoothing parameters λ’s are fixed we are still “mean lovers” .
Go back Historical 55