Lecture 13 Gaussian Process Models - Part 2 3/06/2018 1 EDA and - PowerPoint PPT Presentation

Lecture 13 Gaussian Process Models - Part 2 3/06/2018 1

EDA and GPs 2

Variogram When fitting a Gaussian process model, it is often difficult to fit the covariance parameters (hard to identify). Today we will discuss some EDA approaches for getting a sense of the values for the scale, range and nugget parameters. From the spatial modeling literature the typical approach is to examine an empirical variogram , first we will define the theoretical variogram and its connection to the covariance. Variogram: 2𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 𝑊 𝑏𝑠(𝑍 (𝑢 𝑗 ) − 𝑍 (𝑢 𝑘 )) where 𝛿(𝑢 𝑗 , 𝑢 𝑘 ) is known as the semivariogram. 3

Variogram When fitting a Gaussian process model, it is often difficult to fit the covariance parameters (hard to identify). Today we will discuss some EDA approaches for getting a sense of the values for the scale, range and nugget parameters. From the spatial modeling literature the typical approach is to examine an empirical variogram , first we will define the theoretical variogram and its connection to the covariance. Variogram: 2𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 𝑊 𝑏𝑠(𝑍 (𝑢 𝑗 ) − 𝑍 (𝑢 𝑘 )) where 𝛿(𝑢 𝑗 , 𝑢 𝑘 ) is known as the semivariogram. 3

Variogram When fitting a Gaussian process model, it is often difficult to fit the covariance parameters (hard to identify). Today we will discuss some EDA approaches for getting a sense of the values for the scale, range and nugget parameters. From the spatial modeling literature the typical approach is to examine an empirical variogram , first we will define the theoretical variogram and its connection to the covariance. Variogram: 2𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 𝑊 𝑏𝑠(𝑍 (𝑢 𝑗 ) − 𝑍 (𝑢 𝑘 )) where 𝛿(𝑢 𝑗 , 𝑢 𝑘 ) is known as the semivariogram. 3

Some Properties of the theoretical Variogram / Semivariogram • are non-negative 𝛿(𝑢 𝑗 , 𝑢 𝑘 ) ≥ 0 • are equal to 0 at distance 0 𝛿(𝑢 𝑗 , 𝑢 𝑗 ) = 0 • are symmetric 𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 𝛿(𝑢 𝑘 , 𝑢 𝑗 ) • if there is no dependence then 2𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 𝑊 𝑏𝑠(𝑍 (𝑢 𝑗 )) + 𝑊 𝑏𝑠(𝑍 (𝑢 𝑘 )) for all 𝑗 ≠ 𝑘 • if the process is not stationary 2𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 𝑊 𝑏𝑠(𝑍 (𝑢 𝑗 )) + 𝑊 𝑏𝑠(𝑍 (𝑢 𝑘 )) − 2 𝐷𝑝𝑤(𝑍 (𝑢 𝑗 ), 𝑍 (𝑢 𝑘 )) • if the process is stationary 2𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 2𝑊 𝑏𝑠(𝑍 (𝑢 𝑗 )) − 2 𝐷𝑝𝑤(𝑍 (𝑢 𝑗 ), 𝑍 (𝑢 𝑘 )) 4

Some Properties of the theoretical Variogram / Semivariogram • are non-negative 𝛿(𝑢 𝑗 , 𝑢 𝑘 ) ≥ 0 • are equal to 0 at distance 0 𝛿(𝑢 𝑗 , 𝑢 𝑗 ) = 0 • are symmetric 𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 𝛿(𝑢 𝑘 , 𝑢 𝑗 ) • if there is no dependence then 2𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 𝑊 𝑏𝑠(𝑍 (𝑢 𝑗 )) + 𝑊 𝑏𝑠(𝑍 (𝑢 𝑘 )) for all 𝑗 ≠ 𝑘 • if the process is not stationary 2𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 𝑊 𝑏𝑠(𝑍 (𝑢 𝑗 )) + 𝑊 𝑏𝑠(𝑍 (𝑢 𝑘 )) − 2 𝐷𝑝𝑤(𝑍 (𝑢 𝑗 ), 𝑍 (𝑢 𝑘 )) • if the process is stationary 2𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 2𝑊 𝑏𝑠(𝑍 (𝑢 𝑗 )) − 2 𝐷𝑝𝑤(𝑍 (𝑢 𝑗 ), 𝑍 (𝑢 𝑘 )) 4

Some Properties of the theoretical Variogram / Semivariogram • are non-negative 𝛿(𝑢 𝑗 , 𝑢 𝑘 ) ≥ 0 • are equal to 0 at distance 0 𝛿(𝑢 𝑗 , 𝑢 𝑗 ) = 0 • are symmetric 𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 𝛿(𝑢 𝑘 , 𝑢 𝑗 ) • if there is no dependence then 2𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 𝑊 𝑏𝑠(𝑍 (𝑢 𝑗 )) + 𝑊 𝑏𝑠(𝑍 (𝑢 𝑘 )) for all 𝑗 ≠ 𝑘 • if the process is not stationary 2𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 𝑊 𝑏𝑠(𝑍 (𝑢 𝑗 )) + 𝑊 𝑏𝑠(𝑍 (𝑢 𝑘 )) − 2 𝐷𝑝𝑤(𝑍 (𝑢 𝑗 ), 𝑍 (𝑢 𝑘 )) • if the process is stationary 2𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 2𝑊 𝑏𝑠(𝑍 (𝑢 𝑗 )) − 2 𝐷𝑝𝑤(𝑍 (𝑢 𝑗 ), 𝑍 (𝑢 𝑘 )) 4

Some Properties of the theoretical Variogram / Semivariogram • are non-negative 𝛿(𝑢 𝑗 , 𝑢 𝑘 ) ≥ 0 • are equal to 0 at distance 0 𝛿(𝑢 𝑗 , 𝑢 𝑗 ) = 0 • are symmetric 𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 𝛿(𝑢 𝑘 , 𝑢 𝑗 ) • if there is no dependence then 2𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 𝑊 𝑏𝑠(𝑍 (𝑢 𝑗 )) + 𝑊 𝑏𝑠(𝑍 (𝑢 𝑘 )) for all 𝑗 ≠ 𝑘 • if the process is not stationary 2𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 𝑊 𝑏𝑠(𝑍 (𝑢 𝑗 )) + 𝑊 𝑏𝑠(𝑍 (𝑢 𝑘 )) − 2 𝐷𝑝𝑤(𝑍 (𝑢 𝑗 ), 𝑍 (𝑢 𝑘 )) • if the process is stationary 2𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 2𝑊 𝑏𝑠(𝑍 (𝑢 𝑗 )) − 2 𝐷𝑝𝑤(𝑍 (𝑢 𝑗 ), 𝑍 (𝑢 𝑘 )) 4

Some Properties of the theoretical Variogram / Semivariogram • are non-negative 𝛿(𝑢 𝑗 , 𝑢 𝑘 ) ≥ 0 • are equal to 0 at distance 0 𝛿(𝑢 𝑗 , 𝑢 𝑗 ) = 0 • are symmetric 𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 𝛿(𝑢 𝑘 , 𝑢 𝑗 ) • if there is no dependence then 2𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 𝑊 𝑏𝑠(𝑍 (𝑢 𝑗 )) + 𝑊 𝑏𝑠(𝑍 (𝑢 𝑘 )) for all 𝑗 ≠ 𝑘 • if the process is not stationary 2𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 𝑊 𝑏𝑠(𝑍 (𝑢 𝑗 )) + 𝑊 𝑏𝑠(𝑍 (𝑢 𝑘 )) − 2 𝐷𝑝𝑤(𝑍 (𝑢 𝑗 ), 𝑍 (𝑢 𝑘 )) • if the process is stationary 2𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 2𝑊 𝑏𝑠(𝑍 (𝑢 𝑗 )) − 2 𝐷𝑝𝑤(𝑍 (𝑢 𝑗 ), 𝑍 (𝑢 𝑘 )) 4

Some Properties of the theoretical Variogram / Semivariogram • are non-negative 𝛿(𝑢 𝑗 , 𝑢 𝑘 ) ≥ 0 • are equal to 0 at distance 0 𝛿(𝑢 𝑗 , 𝑢 𝑗 ) = 0 • are symmetric 𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 𝛿(𝑢 𝑘 , 𝑢 𝑗 ) • if there is no dependence then 2𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 𝑊 𝑏𝑠(𝑍 (𝑢 𝑗 )) + 𝑊 𝑏𝑠(𝑍 (𝑢 𝑘 )) for all 𝑗 ≠ 𝑘 • if the process is not stationary 2𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 𝑊 𝑏𝑠(𝑍 (𝑢 𝑗 )) + 𝑊 𝑏𝑠(𝑍 (𝑢 𝑘 )) − 2 𝐷𝑝𝑤(𝑍 (𝑢 𝑗 ), 𝑍 (𝑢 𝑘 )) • if the process is stationary 2𝛿(𝑢 𝑗 , 𝑢 𝑘 ) = 2𝑊 𝑏𝑠(𝑍 (𝑢 𝑗 )) − 2 𝐷𝑝𝑤(𝑍 (𝑢 𝑗 ), 𝑍 (𝑢 𝑘 )) 4

Empirical Semivariogram We will assume that our process of interest is stationary, in which case we Empirical Semivariogram: ̂ 𝛿(ℎ) = 1 2 𝑂(ℎ) ∑ |𝑢 𝑗 −𝑢 𝑘 |∈(ℎ−𝜗,ℎ+𝜗) (𝑍 (𝑢 𝑗 ) − 𝑍 (𝑢 𝑘 )) 2 Practically, for any data set with 𝑜 observations there are ( 𝑜 2 ) + 𝑜 possible data pairs to examine. Each individually is not very informative, so we aggregate into bins and calculate the empirical semivariogram for each bin. 5 will parameterize the semivariagram in terms of ℎ = |𝑢 𝑗 − 𝑢 𝑘 | .

Empirical Semivariogram We will assume that our process of interest is stationary, in which case we Empirical Semivariogram: ̂ 𝛿(ℎ) = 1 2 𝑂(ℎ) ∑ |𝑢 𝑗 −𝑢 𝑘 |∈(ℎ−𝜗,ℎ+𝜗) (𝑍 (𝑢 𝑗 ) − 𝑍 (𝑢 𝑘 )) 2 Practically, for any data set with 𝑜 observations there are ( 𝑜 2 ) + 𝑜 possible data pairs to examine. Each individually is not very informative, so we aggregate into bins and calculate the empirical semivariogram for each bin. 5 will parameterize the semivariagram in terms of ℎ = |𝑢 𝑗 − 𝑢 𝑘 | .

Connection to Covariance 6

Covariance vs Semivariogram - Exponential 7 exp cov exp semivar 1.00 l 1 1.7 0.75 2.3 3 3.7 0.50 y 4.3 5 0.25 5.7 6.3 7 0.00 0.0 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0 d

Covariance vs Semivariogram - Square Exponential 8 sq exp cov sq exp semivar 1.00 l 1 1.7 0.75 2.3 3 3.7 0.50 y 4.3 5 0.25 5.7 6.3 7 0.00 0.0 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0 d

9 From last time 1 0 y −1 −2 0.00 0.25 0.50 0.75 1.00 t

Empirical semivariogram - no bins / cloud 10 4 gamma 2 0 0.00 0.25 0.50 0.75 1.00 h

Empirical semivariogram (binned) 11 binwidth=0.05 binwidth=0.075 3 2 1 gamma 0 binwidth=0.1 binwidth=0.15 3 2 1 0 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 h

Empirical semivariogram (binned + n) 12 binwidth=0.05 binwidth=0.075 3 2 1 n 10 gamma 0 20 binwidth=0.1 binwidth=0.15 30 3 40 2 1 0 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 h

𝐷𝑝𝑤(ℎ) = 𝜏 2 exp ( − (ℎ 𝑚) 2 ) 𝛿(ℎ) = 𝜏 2 − 𝜏 2 exp ( − (ℎ 𝑚) 2 ) Theoretical vs empirical semivariogram After fitting the model last time we came up with a posterior median of = 1.89 − 1.89 exp ( − (5.86 ℎ) 2 ) 13 𝜏 2 = 1.89 and 𝑚 = 5.86 for a square exponential covariance.

Theoretical vs empirical semivariogram After fitting the model last time we came up with a posterior median of = 1.89 − 1.89 exp ( − (5.86 ℎ) 2 ) 13 𝜏 2 = 1.89 and 𝑚 = 5.86 for a square exponential covariance. 𝐷𝑝𝑤(ℎ) = 𝜏 2 exp ( − (ℎ 𝑚) 2 ) 𝛿(ℎ) = 𝜏 2 − 𝜏 2 exp ( − (ℎ 𝑚) 2 )