Masters Thesis Presentation Adaptive Sampling of Clouds with a Fleet - - PowerPoint PPT Presentation

▶

master s thesis presentation adaptive sampling of clouds

Masters Thesis Presentation Adaptive Sampling of Clouds with a Fleet - - PowerPoint PPT Presentation

Nov 03, 2022 379 likes •779 views

Masters Thesis Presentation Adaptive Sampling of Clouds with a Fleet of UAVs : Improving Gaussian Process Regression by Including Prior Knowledge Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Masters Thesis Presentation October 12, 2016 1 / 19

slide-1

SLIDE 1

Master’s Thesis Presentation Adaptive Sampling of Clouds with a Fleet of UAVs: Improving Gaussian Process Regression by Including Prior Knowledge

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 1 / 19

slide-2

SLIDE 2

Motivation: SkyScanner Project

Adaptive Sampling of Cumulus Clouds with a Fleet of UAVs: Clouds remain an uncertainty in current atmospherical models: Characterize the evolution of parameters (3D wind, liquid water content, etc.)

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 2 / 19

slide-3

SLIDE 3

Motivation: SkyScanner Project

Adaptive Sampling of Cumulus Clouds with a Fleet of UAVs: Clouds remain an uncertainty in current atmospherical models: Characterize the evolution of parameters (3D wind, liquid water content, etc.) − → dense spatial sampling

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 2 / 19

slide-4

SLIDE 4

Motivation: SkyScanner Project

Adaptive Sampling of Cumulus Clouds with a Fleet of UAVs: Clouds remain an uncertainty in current atmospherical models: Characterize the evolution of parameters (3D wind, liquid water content, etc.) − → dense spatial sampling Adaptive Sampling vs. Systematic Sampling:

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 2 / 19

slide-5

SLIDE 5

Motivation: SkyScanner Project

Adaptive Sampling of Cumulus Clouds with a Fleet of UAVs: Clouds remain an uncertainty in current atmospherical models: Characterize the evolution of parameters (3D wind, liquid water content, etc.) − → dense spatial sampling Adaptive Sampling vs. Systematic Sampling:

4D map of parameters, with only 1D manifolds available Information efficiency − → quantification of uncertainty Energy efficiency − → mapping and exploiting vertical wind.

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 2 / 19

slide-6

SLIDE 6

Motivation: SkyScanner Project

Adaptive Sampling of Cumulus Clouds with a Fleet of UAVs: Clouds remain an uncertainty in current atmospherical models: Characterize the evolution of parameters (3D wind, liquid water content, etc.) − → dense spatial sampling Adaptive Sampling vs. Systematic Sampling:

4D map of parameters, with only 1D manifolds available Information efficiency − → quantification of uncertainty Energy efficiency − → mapping and exploiting vertical wind.

→ Gaussian Process Regression

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 2 / 19

slide-7

SLIDE 7

Table of Contents

1

Motivation: SkyScanner Project

2

Introduction: Simulation and Architecture

3

Gaussian Process Regression

4

Spatial Statistics

5

Implementation

6

Summary and Outlook

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 3 / 19

slide-8

SLIDE 8

MesoNH Simulation and Sampling Architecture

Large Eddy Simulation(LES) of non-precipitating shallow cumulus clouds. Domain: 3540s × 4km × 4km × 4km (3TB of data),

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 4 / 19

slide-9

SLIDE 9

MesoNH Simulation and Sampling Architecture

Large Eddy Simulation(LES) of non-precipitating shallow cumulus clouds. Domain: 3540s × 4km × 4km × 4km (3TB of data), Grid: 3540x161x400x400 (t, z, x, y) and dt = 1s, dx = dy = 10m, dz = 10m...100m; dz = 10m for boundary and convective cloud layer. Variables: 3D wind, temperature, pressure, liquid water content(LWC), etc.

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 4 / 19

slide-10

SLIDE 10

MesoNH Simulation and Sampling Architecture

UAV Model Atmospheric Simulation Wind Sensors Model Trajectory Planner Wind GP Regression Models

Hyperparameter

ptimization

Wind prediction Sequence

f

Commands UAV Trajectory 3D Wind Ground Truth 3D Wind Ground Truth 3D Wind Samples @1 Hz @ 0.1 Hz

Large Eddy Simulation(LES) of non-precipitating shallow cumulus clouds. Domain: 3540s × 4km × 4km × 4km (3TB of data), Grid: 3540x161x400x400 (t, z, x, y) and dt = 1s, dx = dy = 10m, dz = 10m...100m; dz = 10m for boundary and convective cloud layer. Variables: 3D wind, temperature, pressure, liquid water content(LWC), etc.

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 4 / 19

slide-11

SLIDE 11

MesoNH Simulation and Sampling Architecture

UAV Model Atmospheric Simulation Wind Sensors Model Trajectory Planner Wind GP Regression Models

Hyperparameter

ptimization

Wind prediction Sequence

f

Commands UAV Trajectory 3D Wind Ground Truth 3D Wind Ground Truth 3D Wind Samples @1 Hz @ 0.1 Hz

Large Eddy Simulation(LES) of non-precipitating shallow cumulus clouds. Domain: 3540s × 4km × 4km × 4km (3TB of data), Grid: 3540x161x400x400 (t, z, x, y) and dt = 1s, dx = dy = 10m, dz = 10m...100m; dz = 10m for boundary and convective cloud layer. Variables: 3D wind, temperature, pressure, liquid water content(LWC), etc. → Wind predictions needed under real-time constraints

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 4 / 19

slide-12

SLIDE 12

Introduction to Gaussian Process Regression

Bayesian Machine Learning framework Generalization of the M-dim. Gaussian distribution to stochastic processes(functions), i.e. a Gaussian distribution over functions:

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 5 / 19

slide-13

SLIDE 13

Introduction to Gaussian Process Regression

Bayesian Machine Learning framework Generalization of the M-dim. Gaussian distribution to stochastic processes(functions), i.e. a Gaussian distribution over functions:

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 5 / 19

slide-14

SLIDE 14

Introduction to Gaussian Process Regression

Bayesian Machine Learning framework Generalization of the M-dim. Gaussian distribution to stochastic processes(functions), i.e. a Gaussian distribution over functions: Two key ingredients Mean function m(x): Center for the distribution of functions Covariance function, matrix k(x, x′), Σ: Defines smoothness and variability. Quantifies similarity. If x,x′ similar − → outputs similar

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 5 / 19

slide-15

SLIDE 15

Introduction to Gaussian Process Regression

Making predictions With training data: X, Y | new input vector x⋆| mean function m(x) | covariance matrices ΣX,X = [k(xi, xi)] , i, j = 1, ..., n | Σx⋆,X = [k(x⋆, xi)] , i = 1, ..., n |

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 6 / 19

slide-16

SLIDE 16

Introduction to Gaussian Process Regression

Making predictions With training data: X, Y | new input vector x⋆| mean function m(x) | covariance matrices ΣX,X = [k(xi, xi)] , i, j = 1, ..., n | Σx⋆,X = [k(x⋆, xi)] , i = 1, ..., n | p (y⋆|x⋆, X, Y) = N (y⋆, V[y⋆]) , (1) y⋆ = m(x⋆) + Σx⋆,XΣ−1

X,X(Y − m(X)),

(2) V[y⋆] = k(x⋆, x⋆) − Σx⋆,XΣ−1

X,XΣT x⋆,X

(3)

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 6 / 19

slide-17

SLIDE 17

Introduction to Gaussian Process Regression

Making predictions With training data: X, Y | new input vector x⋆| mean function m(x) | covariance matrices ΣX,X = [k(xi, xi)] , i, j = 1, ..., n | Σx⋆,X = [k(x⋆, xi)] , i = 1, ..., n | p (y⋆|x⋆, X, Y) = N (y⋆, V[y⋆]) , (1) y⋆ = m(x⋆) + Σx⋆,XΣ−1

X,X(Y − m(X)),

(2) V[y⋆] = k(x⋆, x⋆) − Σx⋆,XΣ−1

X,XΣT x⋆,X

(3) Advantages of GPR Inbuilt estimation of uncertainty adapted to test inputs Limitations Mean function and covariance function are parameterized − → Expensive optimization, usually Bayesian Marginal Log-Likelihood (several iterations of O(n3)) With no prior knowledge about process, “off-the-shelf”: − → m(x) = 0, k(x, x′) = σ2 exp

−0.5|x−x′|2

l2

Diego Selle (RIS @ LAAS-CNRS, RT-TUM)

Master’s Thesis Presentation October 12, 2016 6 / 19

slide-18

SLIDE 18

Introduction to Gaussian Process Regression

Types of prior knowledge to improve GPR:

1

Determining the mean function m(x)

2

Determining type and parameter distribution of covariance function k(x, x′)

3

If output multidimensional, then determine and exploit correlations between outputs

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 7 / 19

slide-19

SLIDE 19

Introduction to Gaussian Process Regression

Types of prior knowledge to improve GPR:

1

Determining the mean function m(x)

2

Determining type and parameter distribution of covariance function k(x, x′)

3

If output multidimensional, then determine and exploit correlations between outputs Approaches to determine prior knowledge Brute Force:

Cross-validate implementations that combine several mean-functions, covariance functions and output-correlation structures − → No real understanding about the process − → Computational complexity O(n3)

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 7 / 19

slide-20

SLIDE 20

Introduction to Gaussian Process Regression

Types of prior knowledge to improve GPR:

1

Determining the mean function m(x)

2

Determining type and parameter distribution of covariance function k(x, x′)

3

If output multidimensional, then determine and exploit correlations between outputs Approaches to determine prior knowledge Brute Force:

Cross-validate implementations that combine several mean-functions, covariance functions and output-correlation structures − → No real understanding about the process − → Computational complexity O(n3)

Spatial Statistics, Geostatistics:

Estimate statistics from data and do regular curve fitting on these statistics to infer the priors − → Computational complexity: statistics O(n), curve fitting O(m3), m << n

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 7 / 19

slide-21

SLIDE 21

Spatial Statistics: The Variogram

2γ(x, x′) is a measure of dissimilarity between x and x′

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 8 / 19

slide-22

SLIDE 22

Spatial Statistics: The Variogram

2γ(x, x′) is a measure of dissimilarity between x and x′ With assumptions Stationarity and Isotropy: 2γ(h), distance h = |x − x′| Estimated from the data and then fitted with a model

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 8 / 19

slide-23

SLIDE 23

Spatial Statistics: The Variogram

2γ(x, x′) is a measure of dissimilarity between x and x′ With assumptions Stationarity and Isotropy: 2γ(h), distance h = |x − x′| Estimated from the data and then fitted with a model Basis for spatial prediction in Geostatistics, i.e. Kriging Non-converging empirical variograms indicate problems with Stationarity

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 8 / 19

slide-24

SLIDE 24

Spatial Statistics: Estimating and fitting the Variogram

Estimating 2ˆ γ(h) ≡ 1 |N(h)|

N(h)

(Z(si) − Z(sj))2, h ∈ Rd, (4) N(h) ≡ {(si, sj) : si − sj = h; i, j = 1, ..., n} (5)

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 9 / 19

slide-25

SLIDE 25

Spatial Statistics: Estimating and fitting the Variogram

Estimating 2ˆ γ(h) ≡ 1 |N(h)|

N(h)

(Z(si) − Z(sj))2, h ∈ Rd, (4) N(h) ≡ {(si, sj) : si − sj = h; i, j = 1, ..., n} (5) y x h1 h1 h1 h1 h1 h1 h2 h2 h2

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 9 / 19

slide-26

SLIDE 26

Spatial Statistics: Estimating and fitting the Variogram

Estimating 2ˆ γ(h) ≡ 1 |N(h)|

N(h)

(Z(si) − Z(sj))2, h ∈ Rd, (4) N(h) ≡ {(si, sj) : si − sj = h; i, j = 1, ..., n} (5) y x h1 h1 h1 h1 h1 h1 h2 h2 h2 γ(|h|) |hx| h1 h2

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 9 / 19

slide-27

SLIDE 27

Spatial Statistics: Estimating and fitting the Variogram

Estimating 2ˆ γ(h) ≡ 1 |N(h)|

N(h)

(Z(si) − Z(sj))2, h ∈ Rd, (4) N(h) ≡ {(si, sj) : si − sj = h; i, j = 1, ..., n} (5) y x h1 h1 h1 h1 h1 h1 h2 h2 h2 γ(|h|) |hx| h1 h2 Fitting

k

j=1

|N(h(j))| ˆ γ(h(j)) γ(h(j); θ) − 1 2 (6)

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 9 / 19

slide-28

SLIDE 28

Spatial Statistics: The Variogram and Gaussian Process Regression

Converging variogram models and stationary covariance functions are related: γ(h) = k(0) − k(h), (7) k(h) = γ(∞) − γ(h), (8)

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 10 / 19

slide-29

SLIDE 29

Spatial Statistics: The Variogram and Gaussian Process Regression

Converging variogram models and stationary covariance functions are related: γ(h) = k(0) − k(h), (7) k(h) = γ(∞) − γ(h), (8) Examples: Exponential Variogram γ(h) = σ2(1 − exp( −|h|

l

)) Exponential Covariance Function k(h) = σ2 exp( −|h|

l

)

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 10 / 19

slide-30

SLIDE 30

Implementation: Vertical Wind Empirical Variograms

5 Clouds were segmented, and used to estimate variograms in t, z, x, y Values at big distances are very similar in x, y Variograms continue to grow over theoretical sill in x, y − → Non-stationarity, mean function?

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 11 / 19

slide-31

SLIDE 31

Implementation: New coordinates

Polar coordinates based on center of LWC more “natural” Vertical winds near the center are higher, near boundaries lower − → Radial mean function?

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 12 / 19

slide-32

SLIDE 32

Implementation: Estimating the Mean Function

Normalization of radius and vertical wind at center Over 300.000 radial trends to estimate the median

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 13 / 19

slide-33

SLIDE 33

Implementation: Detrended Empirical Variograms

Clouds were detrended with the mean function New variograms were computed in the four polar directions t, z, ϕ, r

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 14 / 19

slide-34

SLIDE 34

Implementation: Best Fit Detrended Variograms

Around 20-30 variograms with detrended vertical wind were fitted − → Parameters of covariance function Out of four possible models tested, Exponential Variogram best fit Similarity in sills suggests that range anisotropy is more accentuated − → γ(|r|), r 2 ≡ hTMh, M = diag(1/l2

xi)

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 15 / 19

slide-35

SLIDE 35

Implementation: Testing the new GPR

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 16 / 19

slide-36

SLIDE 36

Implementation: Testing the new GPR

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 17 / 19

slide-37

SLIDE 37

Summary and Outlook

Summary Prior on mean function Prior on covariance function Improved performance vs. “off-the-shelf” GPR Outlook Repeat line of analysis on other variables, e.g. liquid water content(LWC) Exploit correlations between LWC and vertical wind Integrate polar coordinates preprocessing to current adaptive sampling scheme

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 18 / 19

slide-38

SLIDE 38

Questions

Questions?

Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master’s Thesis Presentation October 12, 2016 19 / 19