New Directions in 4D-Var: the ECMWF Perspective Massimo Bonavita and - - PowerPoint PPT Presentation

β–Ά
new directions in 4d var the ecmwf perspective
SMART_READER_LITE
LIVE PREVIEW

New Directions in 4D-Var: the ECMWF Perspective Massimo Bonavita and - - PowerPoint PPT Presentation

Sep 25, 2023 179 likes β€’455 views

New Directions in 4D-Var: the ECMWF Perspective Massimo Bonavita and many colleagues ECMWF Special Acknow.: Peter Lean and Elias Holm EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS October 29, 2014 1 Introduction Since its

slide-1
SLIDE 1

October 29, 2014 EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

1

New Directions in 4D-Var: the ECMWF Perspective

Massimo Bonavita and many colleagues

ECMWF Special Acknow.: Peter Lean and Elias Holm

slide-2
SLIDE 2

October 29, 2014

Introduction

  • Since its operational implementation (November 1997) incremental 4D-Var has been

the cornerstone of the ECMWF atmospheric DA system

  • Over the past ~10 years the main development in the ECMWF DA has been the

introduction of an error cycling component (EDA, Bonavita et al, 2012, 2016)

  • Over this period the ECMWF 4D-Var setup has been essentially remained the same,

except for resolution upgrades (both outer and inner loops)

  • However, this is about to change…

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

2

slide-3
SLIDE 3

October 29, 2014

Introduction

  • In the next few years a number of 4D-Var developments will significantly change the

way Data Assimilation is performed at ECMWF:

1) Outer Loop Atmosphere-Ocean Coupling (Laloyaux et al, 2016, 2018) 2) Bias-aware 4D-Var (a.k.a., weak constraint 4D-Var) 3) Parallel-in-time minimisation (no, not the Saddle Point algorithm) 4) 4D-Var analysis of surface variables 5) Model parameters’ estimation in 4D-Var 6) Cheaper, more effective EDA (Lang et al, 2018) 7) Continuous Data Assimilation (Lean et al, 2018)

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

3

slide-4
SLIDE 4

October 29, 2014

Observations

4

Courtesy of Gionata Biavati

ERA5 Reanalysis

10 millions 1 million 0.1 million

slide-5
SLIDE 5

October 29, 2014

Observations: timeliness

5

Timeline of number of obs received by the ECMWF acquisition and pre-proc. system (SAPP), 15/10/2018 Radiosonde obs. GEOS rad. Aircraft obs.

slide-6
SLIDE 6

October 29, 2014

Observations: nonlinearity

6

Forecast sensitivity (FSO) of major

  • bserving systems in ECMWF DA

from Alan Geer

  • -- ALL-SKY obs
  • -- Clear SKY MW
slide-7
SLIDE 7

October 29, 2014

  • For linear observation operators in an ensemble DA:

𝐼 𝐲𝑐

π‘‘π‘’π‘ π‘š = 𝐼 𝑁

𝐲𝑏

𝑗

β‰ˆ 𝐼 𝐲𝑐

𝑗

𝑗 = 1, … , π‘‚π‘“π‘œπ‘‘

Observation: nonlinearities

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

7

Radiosonde temperatures Point temperature measurement

𝐼 𝐲𝑐

𝑗

𝐼 𝐲𝑐

π‘‘π‘’π‘ π‘š

ATMS ch. 20 Tropospheric humidity AMSR-2 ch. 6 Cloud liquid water

slide-8
SLIDE 8

October 29, 2014

  • Increased resolution, fidelity

Model

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

8

From Sylvie Malardel

slide-9
SLIDE 9

October 29, 2014

  • Model nonlinearity: how much initial increments evolved by the linearised model and the

nonlinear model differ?

𝑑𝑒𝑒𝑓𝑀 𝑁 𝐲𝑒 + πœ€π²0 βˆ’ 𝑁 𝐲𝑒 + 𝐍 πœ€π²0 Vorticity 500 hPa (units: 10-5s-1)

Model nonlinearities

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

9

T+3h T+9h

slide-10
SLIDE 10

October 29, 2014

𝑇𝑒𝐸𝑓𝑀 𝑁 𝐲𝑒 + πœ€π²0 βˆ’ 𝑁 𝐲𝑒 + 𝐍 πœ€π²0

Vorticity

Model nonlinearities

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

10

  • Larger nonlinearities than in the

past, due to: 1. Increased resolution (40 km -> 9 km) 2. Increase mismatch of outer- inner loop resol. ( from 3 -> 5) 3. Less diffusive model

slide-11
SLIDE 11

October 29, 2014

  • Incremental 4D-Var deals with nonlinearities by a succession of quadratic optimizations around

progressively more accurate first guess trajectories => progressively smaller increments => more accurate local linearisation!

Nonlinear effects: the incremental approach

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

11

StDev of vorticity analysis incr. in successive minimizations

Taylor diagram for differences in obs-dep for nonlinear and linearised trajectories

slide-12
SLIDE 12

October 29, 2014

  • 1. Ever increasing number of observations
  • 2. Nearly continuous flow of available observations
  • 3. Increasing number and influence of nonlinear observations
  • 4. Increase of model resolution and fidelity => Increasing nonlinearity of error evolution in 4D-Var

assimilation window

  • 5. Need to do more re-linearisations, decrease mismatch between inner/outer loops

configurations, etc. => spend more time in DA computations

Observation and Model trends

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

12

slide-13
SLIDE 13

October 29, 2014

Current operational 4D-Var at ECMWF (Haseler, 2004)

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

13

00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23

12h 4DVar 09-21Z 6h 4DVar 21-03Z 12h 4DVar 21-09Z 6h 4DVar 09-15Z FG/BG FG/BG 00UTC 10-day FCST 12UTC 10-day FCST

  • 1. Analysis work is concentrated in two windows during the day (2-5 UTC and 14-17 UTC)
  • 2. Operational time constraints impose simplifications on the 6h 4D-Vars (Early Delivery Analyses)

used to provide initial conditions for the deterministic 10-day forecasts (and centre analysis for the ENS forecasts)

slide-14
SLIDE 14

October 29, 2014

Current Early Delivery system: Trade-off

14

cut-off time 04z Users expect all products are available 06:55z Collect more observations (later cut-off) Allow more time for computations (earlier cut-off)

?

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

slide-15
SLIDE 15

October 29, 2014

Continuous data assimilation

15

n-2 n-1 n n-3 Extra obs, start earlier 1 2 3 Current cutoff

  • Key point: Start running data assimilation before all of the observations have arrived
  • ~ 70% of 4D-Var is removed from the time critical path
  • Configurations which were previously unaffordable can now be considered
  • Opens the possibility of a fully continuous assimilation system.

1 2 3 Extra obs cutoff Effective cut-off time is 30 minutes later

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

slide-16
SLIDE 16

October 29, 2014

Continuous data assimilation: Trade-off

16

cut-off time 04z Users expect all products are available 06:55z Collect more observations (later cut-off) Allow more time for computations (earlier cut-off)

?

Continuous DA configuration allows both:

  • Later cut-off to collect more observations
  • A longer assimilation window
  • More time to perform DA computations

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

slide-17
SLIDE 17

October 29, 2014

Extra observations assimilated in Continuous DA configuration

17

Timeslot Number of obs

03z 21z 05z

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

slide-18
SLIDE 18

October 29, 2014

  • Of this new-found freedom to explore and use more complex 4D-Var configurations, only the

increase in # of outer loops has been exploited so far (IFS CY46R1 – Q1 2019)

  • This leads to 2-3% forecast skill improvement

Continuous DA

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

18

Impact of additional

  • bservations in Cont. DA

Additional observations + 4 outer loops

(good) (bad)

Vector wind RMS error

slide-19
SLIDE 19

October 29, 2014

  • Continuous DA offers significant advantages over current Early Delivery setup, but does not

solve all the problems

  • In particular, issues about homogeneous exploitation of 4D-Var convergence, time-

homogeneous exploitation of available computer resources, resilience of the assimilation cycle, etc.

  • We think a fully continuous DA configuration will help in these respects
  • We call this future setup Continuous Long Window DA

Continuous Long Window DA

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

19

slide-20
SLIDE 20

October 29, 2014

Continuous Long Window DA

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

20

00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 00

6h 4DVar 21-03Z

8h 4DVar 21-05Z 8h 4DVar 09-17Z

FG FG 00UTC 10-day FCST 12UTC 10-day FCST

  • 1. Single DA cycle
  • 2. Vastly improved resilience and product timeliness
  • 3. Time-uniform exploitation of computing resources
  • 4. Possibility of more complex/costly 4D-Var configurations within operational time constraints

10h 4DVar 21-07Z

FG

12h 4DVar 21-09Z

FG 6h 4DVar 09-15Z FG

10h 4DVar 09-19Z 12h 4DVar 09-21Z

FG BG/FG BG BG BG BG/FG BG BG BG

slide-21
SLIDE 21

October 29, 2014

Thanks for your attention!

Bonavita, M., P. Lean and E. HΓ³lm, 2018: Nonlinear effects in 4D-Var. Nonlin. Proc. Geo., https://doi.org/10.5194/npg-2018-20 Bonavita, M. and co-Authors, 2018: Non-linearity and non-Gaussianity in 4D-Var (and beyond). ECMWF Annual Seminar 2018. Available at https://www.ecmwf.int/en/learning/workshops/annual-seminar-2018 Lean, P., E. HΓ³lm and M. Bonavita, 2018: Continuous Data Assimilation, in preparation.

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

21

slide-22
SLIDE 22

October 29, 2014 EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

22

Additional Slides

slide-23
SLIDE 23

October 29, 2014

  • Many other aspects of 4D-Var algorithm can be improved within the Continuous DA setup:

1. Consistency between outer/inner loop models’ timesteps; 2. Increased consistency between outer/inner loop models’ resolutions; 3. Stricter convergence criteria in the minimisation of the linearised cost function;

Continuous DA

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

23

  • Nonlin. Model tstep=450s; Linearised Model tstep=900s
  • Nonlin. Model tstep=450s; Linearised Model tstep=450s

Difference between analysis increments computed by nonlinear and linearised model t+9h in the assimilation window (temperature at ~5 hPa)

slide-24
SLIDE 24

October 29, 2014

  • How important is the capacity to run outer loops for global analysis and forecast skill?
  • Relative difference of observation departures of 1 OL and 4 OL wrt 3OL control

Nonlinear effects: the incremental approach

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

24

O-A O-B O-A O-B

slide-25
SLIDE 25

October 29, 2014

  • Running incremental updates is key to control nonlinearity in 4D-Var
  • This is true in EnKF/EnKS/EnsVar world as well: IEnKF/IEnKS/IES, etc (e.g. Chen and Oliver,

2012; Sakov et al, 2012; Emerick and Reynolds, 2013; Bocquet and Sakov, 2014)

  • In EnKF/EnKS/EnsVar the analytic gradients are replaced by their ensemble approximations:

𝐂 𝐈𝐍 π‘ˆ β†’ 𝐂𝑦𝑧

π‘“π‘œπ‘‘

𝐈𝐍 𝐂 𝐈𝐍 π‘ˆ β†’ 𝐂𝑧𝑧

π‘“π‘œπ‘‘

  • This implies that the iterated versions of the EnKF/EnKS/EnsVar need to re-run the ensemble at

each iteration to compute the updated sensitivities

  • Another consequence is these ensemble re-runs cannot be computed before the observations

are available

Nonlinear effects: the incremental approach

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

25

slide-26
SLIDE 26

Hurricane Irma (Sept. 2017)

Evolution of the Jo cost function during the IFS 4D-Var minimization, 04-09-2017 12UTC

Radiosonde winds Near-core Dropsonde winds

VIIRS image from NOAA Suomi NPP satellite, 5/09/2017 17.06UTC