Lecture 1 Spatio-temporal data & Linear Models Colin Rundel - PowerPoint PPT Presentation

Lecture 1 Spatio-temporal data & Linear Models Colin Rundel 1/18/2017 1

Spatio-temporal data 2

Time Series Data - Discrete 3 S&P 500 Open (^GSPC) 2270 2265 2260 2255 Jan 03 Jan 05 Jan 09 Jan 11 Jan 13 Jan 17 2017 2017 2017 2017 2017 2017

Time Series Data - Continuous 4 FRN Measured PM25 20 15 PM25 ( µ g/m 3 ) 10 5 Jan 01 Jan 15 Feb 01 Feb 15

5 Spatial Data - Areal SID79 33.5 ° N 34 ° N 34.5 ° N 35 ° N 35.5 ° N 36 ° N 36.5 ° N 37 ° N 84 ° W 82 ° W 80 ° W 78 ° W 76 ° W

Spatial Data - Point referenced 6 Meuse River copper lead zinc 334000 334000 334000 333000 333000 333000 332000 332000 332000 331000 331000 331000 330000 330000 330000 329000 329000 329000 178500 179500 180500 181500 178500 179500 180500 181500 178500 179500 180500 181500

Point Pattern Data - Time 7 Old Faithful Eruption Duration 5 4 duration 3 2 500 1000 1500 time

Point Pattern Data - Space 8

Point Pattern Data - Space + Time 9

(Bayesian) Linear Models 10

Linear Models Pretty much everything we a going to see in this course will fall under the umbrella of linear or generalized linear models. which we can also express using matrix notation as Y X 11 Y i = β 0 + β 1 x i 1 + · · · + β p x ip + ϵ i ϵ i ∼ N ( 0 , σ 2 ) n × 1 = n × p β p × 1 + ϵ n × 1 n × 1 , σ 2 I n ϵ ∼ N ( 0 n × n )

Multivariate Normal Distribution . . . . ... . . . . . Y n . . . (positive semidefinite) can be written as Y 12 Y 1 For an n -dimension multivate normal distribution with covariance Σ n × 1 ∼ N ( µ n × 1 , Σ n × n ) where { Σ } ij = ρ ij σ i σ j         µ 1 ρ 11 σ 1 σ 1 · · · ρ 1 n σ 1 σ n          ∼ N  ,               µ n ρ n 1 σ n σ 1 · · · ρ nn σ n σ n

Multivariate Normal Distribution - Density and its log density is given by 1 2 1 2 n For the n dimensional multivate normal given on the last slide, its density is 13 1 given by ( ) ( 2 π ) − n / 2 det ( Σ ) − 1 / 2 exp 2 ( Y − µ ) ′ Σ − 1 − n × n ( Y − µ ) 1 × n n × 1 2 ( Y − µ ) ′ Σ − 1 − log 2 π − log det ( Σ ) − − n × n ( Y − µ ) 1 × n n × 1

A Simple Linear Regression Example Lets generate some simulated data where the underlying model is known and see how various regression preceedures function. 14 β 0 = 0 . 7 , β 1 = 1 . 5 , β 2 = − 2 . 2 , β 3 = 0 . 1 n = 100 , ϵ i ∼ N ( 0 , 1 )

Generating the data set.seed (01172017) beta = c (0.7,1.5,-2.2,0.1) X2 = rt (n,df=5) Y = X %*% beta + eps d = data.frame (Y,X[,-1]) 15 n = 100 eps = rnorm (n) X0 = rep (1, n) X1 = rt (n,df=5) X3 = rt (n,df=5) X = cbind (X0, X1, X2, X3)

1 X t Y Least squares fit n X t X ls With a bit of calculus and algebra we can derive 2 X i Y i 1 i arg min ls , is given by The least squares estimate, 16 Let ˆ Y be our estimate for Y based on our estimate of β , Y = ˆ β 0 + ˆ β 1 X 1 + ˆ β 2 X 2 + ˆ β 3 X 3 = X ˆ ˆ β

1 X t Y Least squares fit arg min X t X ls With a bit of calculus and algebra we can derive n 16 Let ˆ Y be our estimate for Y based on our estimate of β , Y = ˆ β 0 + ˆ β 1 X 1 + ˆ β 2 X 2 + ˆ β 3 X 3 = X ˆ ˆ β The least squares estimate, ˆ β ls , is given by ∑ ( Y i − X i · β ) 2 β i = 1

Least squares fit arg min With a bit of calculus and algebra we can derive n 16 Let ˆ Y be our estimate for Y based on our estimate of β , Y = ˆ β 0 + ˆ β 1 X 1 + ˆ β 2 X 2 + ˆ β 3 X 3 = X ˆ ˆ β The least squares estimate, ˆ β ls , is given by ∑ ( Y i − X i · β ) 2 β i = 1 ˆ β ls = ( X t X ) − 1 X t Y

Maximum Likelihood 17

Frequentist Fit ## 0.07996257 ## X3 ## X2 -2.16499958 1.65321096 ## X1 0.73726738 ## X0 [,1] (beta_hat = solve ( t (X) %*% X, t (X)) %*% Y) lm (Y ~ ., data=d)$coefficients 0.07996257 1.65321096 -2.16499958 0.73726738 ## X3 X2 X1 ## (Intercept) 18

Bayesian Model 19 Y 1 , . . . , Y 100 | β , σ 2 ∼ N ( X i · β , σ 2 ) β 0 , β 1 , β 2 , β 3 ∼ N ( 0 , σ 2 β = 100 ) τ 2 = 1 /σ 2 ∼ Gamma ( a = 1 , b = 1 )

1 Y i n 2 exp 4 2 exp 2 a b 1 exp Deriving the posterior 2 2 3 i 0 2 i 2 2 3 b a a 2 a b 2 2 1 2 0 where, Y 2 2 2 n i 1 X i 1 1 X i 2 3 X i 3 2 2 2 0 20 [ β 0 , β 1 , β 2 , β 3 , σ 2 | Y ] = [ Y | β , σ 2 ] [ β , σ 2 ] [ Y ] ∝ [ Y | β , σ 2 ][ β ][ σ 2 ]

4 2 exp 2 a b 1 exp 2 3 i 0 2 i Deriving the posterior 2 2 b a a 2 a b 2 2 2 20 3 2 1 0 where, [ β 0 , β 1 , β 2 , β 3 , σ 2 | Y ] = [ Y | β , σ 2 ] [ β , σ 2 ] [ Y ] ∝ [ Y | β , σ 2 ][ β ][ σ 2 ] ( ) ∑ n i = 1 ( Y i − β 0 − β 1 X i , 1 − β 1 X i , 2 − β 3 X i , 3 ) 2 2 πσ 2 ) − n / 2 exp [ Y | β , σ 2 ] = ( − 2 σ 2

2 a b 1 exp Deriving the posterior i 2 b a 2 a b a 20 where, [ β 0 , β 1 , β 2 , β 3 , σ 2 | Y ] = [ Y | β , σ 2 ] [ β , σ 2 ] [ Y ] ∝ [ Y | β , σ 2 ][ β ][ σ 2 ] ( ) ∑ n i = 1 ( Y i − β 0 − β 1 X i , 1 − β 1 X i , 2 − β 3 X i , 3 ) 2 2 πσ 2 ) − n / 2 exp [ Y | β , σ 2 ] = ( − 2 σ 2 ( ) ∑ 3 i = 0 β 2 β ) − 4 / 2 exp [ β 0 , β 1 , β 2 , β 3 | σ 2 β ] = ( 2 πσ 2 − 2 σ 2 β

Deriving the posterior i b b a 20 where, [ β 0 , β 1 , β 2 , β 3 , σ 2 | Y ] = [ Y | β , σ 2 ] [ β , σ 2 ] [ Y ] ∝ [ Y | β , σ 2 ][ β ][ σ 2 ] ( ) ∑ n i = 1 ( Y i − β 0 − β 1 X i , 1 − β 1 X i , 2 − β 3 X i , 3 ) 2 2 πσ 2 ) − n / 2 exp [ Y | β , σ 2 ] = ( − 2 σ 2 ( ) ∑ 3 i = 0 β 2 β ) − 4 / 2 exp [ β 0 , β 1 , β 2 , β 3 | σ 2 β ] = ( 2 πσ 2 − 2 σ 2 β ( ) [ σ 2 | a , b ] = Γ( a )( σ 2 ) − a − 1 exp − σ 2

Deriving the posterior (cont.) 3 b b a 21 [ β 0 , β 1 , β 2 , β 3 , σ 2 | Y ] ∝ ( ) ∑ n i = 1 ( Y i − β 0 − β 1 X i , 1 − β 1 X i , 2 − β 3 X i , 3 ) 2 2 πσ 2 ) − n / 2 exp ( − 2 σ 2 ( ) − β 2 0 + β 2 1 + β 2 2 + β 2 β ) − 4 / 2 exp ( 2 πσ 2 2 σ 2 β ( ) Γ( a )( σ 2 ) − a − 1 exp − σ 2

22 Deriving the Gibbs sampler ( σ 2 step)

23 Deriving the Gibbs sampler ( β i step)