Generating Calibrated Ensembles of Physically Realistic, - - PowerPoint PPT Presentation
Generating Calibrated Ensembles of Physically Realistic, High-Resolution Precipitation Forecast Fields Based on GEFS Model Output Michael Scheuerer University of Colorado, Cooperative Institute for Research in Environmental Sciences and
Michael Scheuerer
University of Colorado, Cooperative Institute for Research in Environmental Sciences and NOAA/ESRL, Physical Sciences Division
1 / 30
GEFS ensemble forecast (lead time 12h - 24h) and climatology corrected analysis of 12h precipitation accumulations on 20 January 2013.
2 / 30
Quantiles and probabilities of threshold exceedance derived from raw ensemble forecasts directly are often unreliable due to biases, insufficient representation of uncertainty, etc. Statistical post-processing methods use forecast-observation pairs from the past to identify and correct those shortcomings.
3 / 30
To postprocess ensemble precipitation forecasts, we use the approach proposed by Scheuerer and Hamill (2015), modeling precipitation amounts by censored, shifted gamma distributions. This method also ac- counts for an increa- se of forecast uncer- tainty with the expec- ted amount of precipi- tation.
ensemble mean
0mm 5mm 10mm 15mm 20mm 25mm 30mm 0mm 20mm 40mm 60mm
µ σ σ
This methods yields reliable, probabilistic forecasts at each forecast lead time and each location.
4 / 30
By calculating certain quantiles, the calibrated forecast distributions can be turned back into an ensemble (of any desired size). Univariate post-processing, however, does not provide any information about serial dependence, i.e. we don’t know how to connect the ensemble forecasts at different lead times.
6−h precip. accumulation 0h 24h 48h 72h 96h 120h 144h 168h 192h 216h 240h 264h 288h 312h 336h 360h 0mm 10mm 20mm 30mm 40mm predictive marginal distributions
5 / 30
Hydrologists need to know not
whether or not that intense rainfall is expected at several locations simultaneously.
6 / 30
◮ select a number of past dates, e.g. same date in the previous 11 years ◮ determine the rank ordering of this historic ensemble ◮ order the samples of the predictive distribution in the same way ◮ this construction preserves the rank correlations as historic ensemble
lead time 6−h precip. accumulation
0h 24h 48h 72h 96h 120h 144h 168h 192h 216h 240h 264h 288h 312h 336h 360h 0mm 10mm 20mm 30mm 40mm
historical trajectories predictive marginal distributions
7 / 30
◮ select a number of past dates, e.g. same date in the previous 11 years ◮ determine the rank ordering of this historic ensemble ◮ order the samples of the predictive distribution in the same way ◮ this construction preserves the rank correlations as historic ensemble
lead time 6−h precip. accumulation
0h 24h 48h 72h 96h 120h 144h 168h 192h 216h 240h 264h 288h 312h 336h 360h 0mm 10mm 20mm 30mm 40mm
standard Schaake shuffle trajectories
8 / 30
Idea: Instead of selecting historic dates ad hoc, choose dates among a set
trajectories are similar to the calibrated predictive distributions:
6−h precip. accumulation
0h 24h 48h 72h 96h 120h 144h 168h 192h 216h 240h 264h 288h 312h 336h 360h 0mm 10mm 20mm 30mm 40mm
subset of 553 trajectories predictive marginal distributions
9 / 30
Idea: Instead of selecting historic dates ad hoc, choose dates among a set
trajectories are similar to the calibrated predictive distributions:
6−h precip. accumulation
0h 24h 48h 72h 96h 120h 144h 168h 192h 216h 240h 264h 288h 312h 336h 360h 0mm 10mm 20mm 30mm 40mm
subset of 11 trajectories predictive marginal distributions
10 / 30
Idea: Instead of selecting historic dates ad hoc, choose dates among a set
trajectories are similar to the calibrated predictive distributions:
6−h precip. accumulation
0h 24h 48h 72h 96h 120h 144h 168h 192h 216h 240h 264h 288h 312h 336h 360h 0mm 10mm 20mm 30mm 40mm
minimum divergence Schaake shuffle trajectories
11 / 30
Similar idea as Schaake shuffle:
◮ sample the calibrated, univariate predictive distributions (quantile) ◮ determine the rank ordering from the raw forecast ensemble ◮ order the samples of the predictive distribution in the same way
lead time 6−h precip. accumulation
0h 24h 48h 72h 96h 120h 144h 168h 192h 216h 240h 264h 288h 312h 336h 360h 0mm 10mm 20mm 30mm 40mm
GEFS forecasts predictive marginal distributions
12 / 30
Similar idea as Schaake shuffle:
◮ sample the calibrated, univariate predictive distributions (quantile) ◮ determine the rank ordering from the raw forecast ensemble ◮ order the samples of the predictive distribution in the same way
lead time 6−h precip. accumulation
0h 24h 48h 72h 96h 120h 144h 168h 192h 216h 240h 264h 288h 312h 336h 360h 0mm 10mm 20mm 30mm 40mm
ECC trajectories
13 / 30
We illustrate the strengths and limitations of these methods in a case study where we generate statistically calibrated, high-resolution precipitation forecast fields over the Russian River basin in California.
◮ 11 grid points of the 0.5° GEFS grid ◮ 4 consecutive 6-h accumulation
periods starting 1/19/2010, 00 UTC
◮ forecast lead times: 48 to 54-h,
54 to 60-h, 60 to 66-h, 66 to 72-h
◮ calibration/downscaling to climatology
corrected precipitation analyses (CCPA, resolution 2.5km)
14 / 30
GEFS ensemble mean
lead time 48 − 54h
lead time 54 − 60h
lead time 60 − 66h
lead time 66 − 72h
CSGD mean
lead time 48 − 54h lead time 54 − 60h lead time 60 − 66h lead time 66 − 72h
Analyzed field
Jan 19, 2010, 0Z − 6Z Jan 19, 2010, 6Z − 12Z Jan 19, 2010, 12Z − 18Z Jan 19, 2010, 18Z − 0Z
0 mm 10 mm 20 mm 30 mm 40 mm 50 mm 60 mm 70 mm
15 / 30
Historic field 5 lead time 48 − 54h lead time 54 − 60h lead time 60 − 66h lead time 66 − 72h Rank of historic field 5 StSS member 5
1 2 3 4 5 6 7 8 9 10 11 rank
0 mm 10 mm 20 mm 30 mm 40 mm 50 mm 60 mm 70 mm
Even the wettest historic field has large areas with zero precipitation. In the absence of reordering information, ranks were assigned at random, which results in unrealistic spatial structures.
16 / 30
Historic field 11 (upscaled)
X
lead time 48 − 54h
X
lead time 54 − 60h
X
lead time 60 − 66h
X
lead time 66 − 72h MDSS member 11 (GEFS scale)
X X X X
Adjustment function 0 mm 10 mm 20 mm 30 mm 40 mm 50 mm 60 mm 70 mm 0.1 1 4 10 adjustment factor
The MDSS algorithm selects a set of dates such that the corresponding upscaled analysis fields have marginal distributions similar to the coarse-scale calibrated forecast distributions.
17 / 30
Historic field 11 (upscaled)
X
lead time 48 − 54h
X
lead time 54 − 60h
X
lead time 60 − 66h
X
lead time 66 − 72h MDSS member 11 (GEFS scale)
X X X X
Adjustment function 0 mm 10 mm 20 mm 30 mm 40 mm 50 mm 60 mm 70 mm 0.1 1 4 10 adjustment factor
The rank order of these upscaled historic analysis fields is then imposed
coarse-scale MDSS forecast fields.
18 / 30
Historic field 11 (upscaled)
X
lead time 48 − 54h
X
lead time 54 − 60h
X
lead time 60 − 66h
X
lead time 66 − 72h MDSS member 11 (GEFS scale)
X X X X
Adjustment function 0 mm 10 mm 20 mm 30 mm 40 mm 50 mm 60 mm 70 mm 0.1 1 4 10 adjustment factor
A spatially smooth adjustment factor is then derived that maps the upscaled historic analysis fields to these MDSS forecast fields.
19 / 30
Adjustment function Historic field 11 MDSS member 11 0.1 1 4 10 adjustment factor 0 mm 10 mm 20 mm 30 mm 40 mm 50 mm 60 mm 70 mm
When this adjustment factor is applied to the original historic analysis field, the adjusted field retains the spatial structure of the historic field but has the desired coarse-scale precipitation amounts.
20 / 30
Interpolated GEFS member 2 lead time 48 − 54h lead time 54 − 60h lead time 60 − 66h lead time 66 − 72h Rank of int. GEFS member 2 ECC−Q member 2
1 2 3 4 5 6 7 8 9 10 11 rank
0 mm 10 mm 20 mm 30 mm 40 mm 50 mm 60 mm 70 mm
The ECC implementation described above imposes the rank order of the raw, interpolated GEFS ensemble on samples (quantiles) of the fine-scale scale calibrated forecast distributions.
21 / 30
The ECC-Q reordering implies a quantile mapping function at each fine-scale grid point. At the intersection of interpolated GEFS fields, the rank changes, and this translates into a spatial discontinuity of the mapping function and thus an abrupt change of the values of the ECC-Q ensemble.
interpolated GEFS forecasts calibrated forecast sample 0 mm 5 mm 10 mm 15 mm 0 mm 10 mm 20 mm 30 mm 1 2 3 4 5 6 7 8 9 10 11
− − − − − − − − − − − − − − − − − − − − − −
ECC−Q ensemble ECC−T ensemble interpolated GEFS forecasts calibrated forecast sample 0 mm 5 mm 10 mm 15 mm 0 mm 10 mm 20 mm 30 mm 1 2 3 4 5 6 7 8 9 10 11
− − − − − − − − − − − − − − − − − − − − − −
ECC−Q ensemble ECC−T ensemble
If we approximate this discontinuous mapping function by a penalized linear regression spline, the discontinuities can be avoided.
22 / 30
GEFS member 2
X
lead time 48 − 54h
X
lead time 54 − 60h
X
lead time 60 − 66h
X
lead time 66 − 72h ECC−Q member 2 ECC−T member 2 0 mm 10 mm 20 mm 30 mm 40 mm 50 mm 60 mm 70 mm
This new ECC-T implementation preserves the good statistical properties
physically more realistic fine-scale ensemble members.
23 / 30
In order to study low, intermediate, and high precipitation amounts separately, it is convenient to convert the precipitation fields into binary threshold exceedance fields:
0 mm 10 mm 20 mm 30 mm 40 mm 50 mm 60 mm 70 mm
Analyzed field, Jan 19, 2010, 12Z − 18Z Exceedance of 25 mm precipitation
This is done for both analyzed and calibrated forecast fields.
24 / 30
ECC−T member 1 ECC−T member 2 ECC−T member 3 ECC−T member 4 ECC−T member 5 ECC−T member 6 ECC−T member 7 ECC−T member 8 ECC−T member 9 ECC−T member 10 ECC−T member 11 Analyzed field
For reliable ensemble forecasts the rank of the fraction of threshold exceedance (FTE) of analyzed precipitation should be uniformly distributed among all FTE ranks.
25 / 30
StSS member 1 StSS member 2 StSS member 3 StSS member 4 StSS member 5 StSS member 6 StSS member 7 StSS member 8 StSS member 9 StSS member 10 StSS member 11 Analyzed field
Ensembles with inadequate spatial structure, on the contrary, might result in FTEs that are systematically too small, too large, or too similar too each other.
26 / 30
StSS 0.1 mm MDSS ECC−Q ECC−T 10 mm 25 mm 99.5% clim. 99.9% clim.
If we plot the rank of the FTEs of analyzed precipitation amounts in a histogram, we can observe different departures from uniformity.
27 / 30
Another way to study the representation of sub-grid scale ensemble properties is to consider the maximum precipitation over all 4 lead times and all fine-scale grid points associated with each coarse-scale grid point. Brier skill scores of the ensemble probabilities of threshold exceedance give an idea about the skill of the respective methods.
Brier skill scores, threshold: 0.1 mm
Nov Dec Jan Feb Mar Apr Mai −0.4 −0.2 0.0 0.2 0.4 0.6 Brier skill scores, threshold: 10 mm
Nov Dec Jan Feb Mar Apr Mai −0.4 −0.2 0.0 0.2 0.4 0.6
MDSS ECC−Q ECC−T Brier skill scores, threshold: 25 mm
Nov Dec Jan Feb Mar Apr Mai −0.4 −0.2 0.0 0.2 0.4 0.6
28 / 30
◮ Modeling dependence between lead times, spatial locations, and
different weather variables is crucial in many applications
◮ For precipitation amounts, discrete copula approaches are common ◮ The standard implementation of the Schaake shuffle needs to
randomize whenever the historic trajectories have zeros
◮ Our minimum divergence implementation of it avoids this problem ◮ The sharp gradients that occur when the ensemble copula coupling
technique is applied to high-resolution precipitation forecast fields can be avoided by regularizing the mapping function implied by ECC-Q
29 / 30
Clark, M., Gangopadhyay, S., Hay, L., Rajagopalan, B., and Wilby, R. The Schaake shuffle: A method for reconstructing space-time variability in forecasted precipitation and temperature fields.
Schefzik, R., Thorarinsdottir, T.L., and Gneiting, T. Uncertainty quantification in complex simulation models using ensemble copula coupling.
Scheuerer, M. and T. M. Hamill Statistical post-processing of ensemble precipitation forecasts by fitting censored, shifted Gamma distributions.
Scheuerer, M., T. M. Hamill, B. Whitin, M. He, and A. Henkel A method for preferential selection of dates in the Schaake shuffle approach to constructing spatio-temporal forecast fields of temperature and precipitation. Water Resour. Res., 53:3029–3046, 2017. Scheuerer, M. and T. M. Hamill Generating Calibrated Ensembles of Physically Realistic, High-Resolution Precipitation Forecast Fields Based on GEFS Model Output.
30 / 30