Ensemble sensitivity and sampling error correction evaluated using a - - PowerPoint PPT Presentation

1 ISDA 2019

Tobias Necker (LMU)

• M. Weissmann (LMU, DWD), S. Geiss (LMU),
• T. Miyoshi (RIKEN), J. Ruiz (CIMA), G.-Y. Lien (TCBW)

and J. Anderson (NCAR)

Ensemble sensitivity and sampling error correction evaluated using a convective-scale 1000 member ensemble

2 ISDA 2019

Outline

Ø Motivation Ø 1000 member ensemble simulation Ø Sampling error (correction algorithms) Ø Potential impact of observed quantities Ø Conclusions

1000

3 ISDA 2019

1000

Motivation - 1000 member ensemble

Correlation of 3-h precipitation forecast inside the box to initial 850 hPa humidity

1000

Realistic correlations

4 ISDA 2019

Motivation - 1000 member ensemble

Correlation of 3-h precipitation forecast inside the box to initial 850 hPa humidity

1000 members

Sub- sampling

40 members

Realistic correlations

Goal: Potential impact of observations Goal: Quantifiaction of sampling errors Observing strategies

5 ISDA 2019

Convective-scale 1000 member ensemble

GFS Ens. BC GFS Ens. BC 3-h cycling GFS Ens. BC

CY

15 km forecast 0 3 6 9 12 15

FC

Down scaling

3 km forecast

t global

Domains: 14-h forecasts

Japanese "SCALE-RM" model

l Spin up: 1 week l Period of 5 days/10 FCs in Mai/June 2016 l Global GFS ensemble BC using NCEP 20

member analysis ensemble combined with 1000 random perturbations

l LETKF (15km; conventional observations) l Downscaling to from 15km (CY) to 3km (FC)

for convective-scale forecasts

l Forecast domain: 3 km grid spacing,

350x250 grid points with 30 levels

6 ISDA 2019

Sampling error correction

Ensemble sensitivity analysis / sample correlation: Sampling Error Correction (SEC):

• Designed to replace/reduce need of localization
• Offline Monte-Carlo technique -> look-up table
• rsec table depends on ensemble size,

sample correlation and assumed prior (normal) correlation distribution

• > Samling error corrected sensitivity:

S: sensitvity J : response function x : state variable r : sample correlation (-1,1) rsec: sampling error corrected correlation σ : sample standard deviation SEC: sampling error correction

Anderson (2012): Localization and Sampling Error Correction in EnKF DA

7 ISDA 2019

Example of temporal correlation

ESA setup:

l 3-h lead time l Response function J: Precipitation

coarse grained over 40x40 grid points

l Spatio-temporal correlations

1000

ECMWF analysis, 29.05.2016 – 0 UTC

J 1000 member ensemble mean precipitation Correlation of precipitation forecast to 2-m temperature field

Streamlines of 500 hPa wind

domain

8 ISDA 2019

Qualitative analysis

Sampling errors:

l 40 & 200 member

ensemble are sub-samples

• f 1000 member ensemble

l SEC systemtically reduces

sampling errors

l Confidence test (T95)

discards correlations 1000 40 200 40+SEC

Correlation of 3-h precipitation forecast to initial 2-m temperature, 1 forecast

40+T95

9 ISDA 2019

Quantitative analysis

Sampling error correction (SEC):

l Improves frequency distribution significantly l Reduces error for small correlation values l For 2-m T: no improvements for large correlation values (Note: small sample size!) l 1000 member PDF: better prior assumption could improve SEC performance

Frequency distribution Mean absolute error (MAE)

Correlation of 3-h precipitation forecast to initial 2-m temperature, 10 forecasts

10 ISDA 2019

Sampling error as function of ensemble size

Sampling errors:

l Doubling the ensemble size from 40 to 80 member decreases sampling error by 30% l SEC significantly reduces sampling errors for all investigated ensemble sizes l 40 member + SEC performs better then 80 member

Correlation of 3-h precipitation forecast to initial 2-m temperature, 10 forecasts

11 ISDA 2019

Sampling error correction for different variables

Spatio-temporal correlation using 40 member ensemble:

• SEC reduces RMSE by about 20 - 30%
• SEC significantly corrects magnitude BIAS caused by spurious correlations

Correlation of 3-h precipitation forecast to various model fields, 10 forecasts using 40 member

12 ISDA 2019

Spatial correlations for DA

Spatial correlations as function of horizontal distance:

l 40 member: SEC reduces sampling error up to 30% l For e.g. highly positivly correlated variables, no improvements due to insufficient prior

à Different or adaptive prior r(x,J) needed as in CER algorithm (Anderson, 2016) Absolute mean Absolute error Absolute error Absolute mean T 2m to T 2m T 500hPa to QV 500hPa Cross-Correlation Correlation

13 ISDA 2019

Correlation as function of ensemble size

Accumulated squared correlation (ASC) as proxy for potential impact:

l Saturation below 1000 member for all variables l Overestimation of potential impact for smaller ensembles

due to sampling errors

1000 member

500 hPa zonal wind 2-m temperature Precipitation

14 ISDA 2019

Sampling error and potential impact

Confidence test (T95) and SEC:

l SEC performes better then confidence test l 200 + SEC close to 1000 member (dotted line) l 1000 + T95 prove of confidence

1000 member

500 hPa zonal wind 2-m temperature Precipitation

15 ISDA 2019

Conclusions

Ø Temporal and spatial correlations have been evaluated for a convective-scale 1000 member ensemble simulation over Europe Ø Sampling error correction (ESA, spatio-temporal correlations):

• Significantly reduces sampling errors
• Simple prior assumption is suitable

Ø Sampling error correction (DA, spatial correlations):

• Promissing especially for convective-scale and vertical application
• Adaptiv prior required for better performance (as done by Anderson 2016)

Ø Confidence test (T95) ) -> suitable for qualitative analysis (ESA) Sampling error correction (SEC) -> qualitative & quantitative improvements Ø Accumulated squared correlation (ASC) used as proxy for potential impact ( Required ensemble size: 200 member + SEC close to 1000 member )

16 ISDA 2019

References

Torn, R. D., 2010: Ensemble-Based Sensitivity Analysis Applied to African Easterly Waves. Weather and Forecasting. Anderson, J. L. 2012: Localization and Sampling Error Correction in Ensemble Kalman Filter Data Assimilation. Mon. Wea. Rev. Anderson, J. L., 2016: Reducing Correlation Sampling Error in Ensemble Kalman Filter Data Assimilation. Mon. Wea. Rev. Necker, T. et al 2018: The importance of appropriate verification metrics for the assessment of observation impact in a convection-permitting modelling

• system. Q. J. R. Meteorol. Soc.