A Localized Markov Chain Particle Filter (LMCPF) for the Global - - PowerPoint PPT Presentation

โ–ถ
a localized markov chain particle filter lmcpf for the
SMART_READER_LITE
LIVE PREVIEW

A Localized Markov Chain Particle Filter (LMCPF) for the Global - - PowerPoint PPT Presentation

Mar 16, 2023 100 likes โ€ข456 views

A Localized Markov Chain Particle Filter (LMCPF) for the Global Weather Prediction Model ICON Anne Sophie Walter and Roland Potthast and Andreas Rhodin German Meteorological Service (DWD) Data Assimilation Unit 7th International Symposium on

slide-1
SLIDE 1

A Localized Markov Chain Particle Filter (LMCPF) for the Global Weather Prediction Model ICON

Anne Sophie Walter and Roland Potthast and Andreas Rhodin German Meteorological Service (DWD) Data Assimilation Unit 7th International Symposium on Data Assimilation 2019

slide-2
SLIDE 2

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 2

What is new?

  • Two localized particle filters working stably in an operational setup: LAPF and

LMCPF

  • Strong improvements of scores by including model error via a Gaussian

mixture approach for the prior distribution

  • LMCPF partly decouples spread and assimilation functionality (via model error)
  • First particle filter tests with full global operational resolution and setup,

including two-way European nest in global DA

  • Particle Filter shows behavior similar to LETKF, with some scores better than

LETKF, some worse (o-b and forecast!)

  • LMCPF also works similarly for COSMO convective scale

22.01.2019

slide-3
SLIDE 3

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 3

Content

  • 1. The ICON-Model and DA-System
  • 2. The Localized Markov Chain Particle Filter (LMCPF)
  • 3. Numerical Testing
  • 4. Summary

22.01.2019

slide-4
SLIDE 4

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 4

  • 1. The ICON-Model
  • Operational NWP model of DWD: ICON (ICOsahedral Nonhydrostatic)
  • Two-way NEST over Europe (~6.5 km)
  • Resolution: 13 km deterministic

40 km ensemble (40 member)

22.01.2019

slide-5
SLIDE 5

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 5

  • 1. DA System at DWD
  • Hybrid EnVar operational since

January 20, 2016 ๐‘Œ๐‘ = าง ๐‘ฆ๐‘ + ๐‘Œ๐‘๐‘‹

  • Following Hunt et al. (2007),

(see also Schraff et al., 2016)

  • EnVar-B-Matrix: 70% LETKF,

30% Climatology

22.01.2019

B-Matrix

slide-6
SLIDE 6

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 6

  • 2. The LMCPF
  • Our Particle Filters are based on the LETKF philosophy

โžข Localization: as LETKF in ensemble space โžข Adaptivity: tools for spread control

22.01.2019

PRIOR DATA Posterior Analysis Ensemble

โžข Able to handle with multi-modal distributions โžข Able to handle with Non-Gaussianity

slide-7
SLIDE 7

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 7

  • 2. The LMCPF

22.01.2019

LETKF Classical PF

DATA Prior Posterior DATA Prior Posterior

LMCPF

DATA Prior Posterior

slide-8
SLIDE 8

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 8

  • 2. The LMCPF
  • Employ Bayes formula to calculate new analysis distribution

๐‘ž๐‘™

๐‘ ๐‘ฆ : = ๐‘ž ๐‘ฆ ๐‘ง๐‘™ = ๐‘‘ ๐‘ž ๐‘ง๐‘™ ๐‘ฆ ๐‘ž๐‘™ ๐‘ ๐‘ฆ ,

๐‘ฆ โˆˆ โ„๐‘œ ๐‘‘ is a normalization factor: ืฌ

๐‘Œ ๐‘ž๐‘™ ๐‘

๐‘ฆ ๐‘’๐‘ฆ = 1

  • To carry out the analysis step at time ๐‘ข๐‘™ a posteriori weights ๐‘ฅ๐‘™,๐‘š

๐‘ are

calculated ๐‘ฅ๐‘™,๐‘š

(๐‘) = ๐‘‘ ๐‘“โˆ’1 2 ๐‘งโˆ’๐ผ๐‘ฆ ๐‘š

๐‘ˆ๐‘†โˆ’1(๐‘งโˆ’๐ผ๐‘ฆ ๐‘š )

๐‘‘ is chosen such that ฯƒ๐‘š=1

๐‘€

๐‘ฅ๐‘™,๐‘š

(๐‘) = ๐‘€

22.01.2019

First Step: The Classical Particle Filter

slide-9
SLIDE 9

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 9

  • 2. The LMCPF
  • Accumulated weights ๐‘ฅ๐‘๐‘‘ are defined:

๐‘ฅ๐‘๐‘‘0 = 0 ๐‘ฅ๐‘๐‘‘๐‘š = ๐‘ฅ๐‘๐‘‘๐‘šโˆ’1 + ๐‘ฅ๐‘š

๐‘,

๐‘š = 1, โ€ฆ , ๐‘€ where ๐‘€ denotes the ensemble size

  • Drawing ๐‘ 

๐‘š~๐‘‰ 0,1 , ๐‘š = 1, โ€ฆ , ๐‘€, set ๐‘†๐‘š = ๐‘š โˆ’ 1 + ๐‘  ๐‘š and define transform

matrix เทฒ ๐‘ฟ for the particles by: เทฒ ๐‘‹

๐‘—,๐‘š = เต1

๐‘—๐‘” ๐‘†๐‘š โˆˆ ๐‘ฅ๐‘๐‘‘๐‘šโˆ’1, ๐‘ฅ๐‘๐‘‘๐‘š , ๐‘๐‘ขโ„Ž๐‘“๐‘ ๐‘ฅ๐‘—๐‘ก๐‘“, ๐‘—, ๐‘š = 1, โ€ฆ , ๐‘€ with เทฑ ๐‘‹ โˆˆ โ„๐‘€๐‘ฆ๐‘€, (๐‘ก, ๐‘ข] denotes the interval of values ๐‘ก < ๐œƒ โ‰ค ๐‘ข.

22.01.2019

Second Step: Classical Resampling

slide-10
SLIDE 10

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 10

  • 2. The LMCPF
  • Based on the adaptive multiplicative inflation factor ๐‡ determined by the

LETKF ๐œ = ฮ• ๐’†๐‘โˆ’๐‘

๐‘ˆ

๐’†๐‘โˆ’๐‘ โˆ’ Tr(๐’) ๐‘ˆ๐‘ (๐‘ฐ๐‘ธ๐‘๐‘ฐ๐‘ˆ)

  • Weighting factor ๐œท has been chosen, due to the small ensemble size (๐‘€ = 40)

๐œ๐‘™ = ๐›ฝ เทค ๐œ๐‘™ + 1 โˆ’ ๐›ฝ ๐œ๐‘™โˆ’1

22.01.2019

Third Step: Spread Control

slide-11
SLIDE 11

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 11

  • 2. The LMCPF
  • Perturbation factor ๐œ is used to add spread to the system

๐œ = ๐‘‘0, ๐œ < ๐œ(0) ๐‘‘0 + ๐‘‘1 โˆ’ ๐‘‘0 โˆ— ๐œ โˆ’ ๐œ(0) ๐œ(1) โˆ’ ๐œ(0) , ๐œ(0) โ‰ค ๐œ โ‰ค ๐œ(1) ๐‘‘1, ๐œ > ๐œ(1)

where ๐‘‘0 = 0.02, ๐‘‘1 = 1.5, ๐œ(0) = 1.0 and ๐œ(1) = 1.4, with ๐œ = ๐‘‘1if ๐œ โ‰ฅ ๐œ(1) and ๐œ = ๐‘‘0 if ๐œ โ‰ค ๐œ(0)

22.01.2019

Third Step: Spread Control

slide-12
SLIDE 12

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 12

  • 2. The LMCPF

เทจ ๐ถ = ๐ฝ โˆ’ ๐ฟ๐ผ ๐ถ = ๐ฝ โˆ’ ๐ถ๐ผ๐‘ˆ ๐‘† + ๐ผ๐ถ๐ผ๐‘ˆ โˆ’1๐ผ ๐ถ = ๐ฝ โˆ’ ๐›ฟ๐‘Œ ๐‘Œ๐‘ˆ๐ผ๐‘ˆ ๐‘† + ๐›ฟ ๐ผ๐‘Œ ๐‘Œ๐‘ˆ๐ผ๐‘ˆ โˆ’1๐ผ ๐›ฟ๐‘Œ๐‘Œ๐‘ˆ = ๐‘Œ(๐ฝ โˆ’ ๐›ฟ ๐‘๐‘ˆ(๐‘† + ๐›ฟ๐‘๐‘๐‘ˆ)โˆ’1๐‘)๐›ฟ๐‘Œ๐‘ˆ = ๐‘Œ ๐ฝ โˆ’ ๐›ฟ ๐ฝ + ๐›ฟ๐‘๐‘ˆ๐‘†โˆ’1๐‘ โˆ’1๐‘๐‘ˆ๐‘†โˆ’1๐‘ ๐›ฟX๐‘ˆ = ๐‘Œ ๐ฝ + ๐›ฟ๐‘๐‘ˆ๐‘†โˆ’1๐‘ โˆ’1(๐ฝ + ๐›ฟ๐‘๐‘ˆ๐‘†โˆ’1๐‘ โˆ’ ๐›ฟ๐‘๐‘ˆ๐‘†โˆ’1๐‘) ๐›ฟ๐‘Œ๐‘ˆ = ๐‘Œ ๐ฝ + ๐›ฟ๐‘๐‘ˆ๐‘†โˆ’1๐‘ โˆ’1๐›ฟ๐‘Œ๐‘ˆ = ๐›ฟ๐‘Œ ๐ฝ + ๐›ฟ๐‘๐‘ˆ๐‘†โˆ’1๐‘ โˆ’1๐‘Œ๐‘ˆ

22.01.2019

Fourth Step: Determine posterior B โ‰ก เทจ ๐ถ

๐ฟ = ๐ถ๐ผ๐‘ˆ ๐‘† + ๐ผ๐ถ๐ผ๐‘ˆ โˆ’1 ๐ถ = ๐›ฟ๐‘Œ๐‘Œ๐‘ˆ ๐ผ๐‘Œ = ๐‘ ๐‘๐‘ˆ ๐‘† + ๐›ฟ๐‘๐‘๐‘ˆ โˆ’1 = ๐ฝ + ๐›ฟ๐‘๐‘ˆ๐‘†โˆ’1๐‘ โˆ’1๐‘๐‘ˆ๐‘†โˆ’1

slide-13
SLIDE 13

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 13

  • 2. The LMCPF

๐›พ(๐‘กโ„Ž๐‘—๐‘”๐‘ข,๐‘š) = ๐ฝ + ๐›ฟY๐‘ˆ๐‘†โˆ’1๐‘ โˆ’1๐‘๐‘ˆ๐‘†โˆ’1 ๐ท โˆ’ ๐‘“๐‘š with ๐ต = ๐‘๐‘ˆ๐‘†โˆ’1๐‘, ๐ท = ๐ตโˆ’1๐‘๐‘ˆ๐‘†โˆ’1(๐‘ง0 โˆ’ เดค ๐‘ง๐‘) ๐‘‹(๐‘กโ„Ž๐‘—๐‘”๐‘ข) โ‰” ๐›พ ๐‘กโ„Ž๐‘—๐‘”๐‘ข,1 , โ€ฆ , ๐›พ(๐‘กโ„Ž๐‘—๐‘”๐‘ข,๐‘€) โˆˆ โ„๐‘€ร—๐‘€ Important effects of ๐›พ(๐‘กโ„Ž๐‘—๐‘”๐‘ข,๐‘š):

  • it moves the particles towards the observations in ensemble space
  • by the use of model error, it constitutes a further degree of freedom which can be used for

tuning of a real system

22.01.2019

Fifth Step: Calculation of the Shift-Vector ๐›พ(๐‘กโ„Ž๐‘—๐‘”๐‘ข,๐‘š)

slide-14
SLIDE 14

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 14

  • 2. The LMCPF
  • Weights W are calculated by drawing from the posterior
.

๐‘‹ = เทฑ ๐‘‹ + ๐‘‹(๐‘กโ„Ž๐‘—๐‘”๐‘ข) โˆ— เทฑ ๐‘‹ + เทจ ๐ถ โˆ— ๐‘†๐‘œ๐‘’ โˆ— ๐œ

22.01.2019

Sixth Step: Gaussian Resampling

with ๐‘†๐‘œ๐‘’ normally distributed random numbers, ๐‘‹(๐‘กโ„Ž๐‘—๐‘”๐‘ข) and เทจ ๐ถ calculated with Gaussian radial basis function (rbf) Approximation for prior density and observation error

DATA Prior Posterior

โžขIt is an explicit calculation of the Bayes posterior based on radial basis function approximation of the prior.

Centers of posterior RBFs Shape of posterior RBFs

slide-15
SLIDE 15
  • 3. Numerical Testing
slide-16
SLIDE 16

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 16

  • Full ensemble: 40 members
  • Reduced resolution:
  • 26km deterministic
  • 52km ensemble
  • Period:

06.05.2016 โ€“ 31.05.2016

  • Full ensemble: 40 members
  • Routine resolution:
  • 13km deterministic
  • 40km ensemble

โžข two-way NEST:

  • 6.5km deterministic
  • 20km ensemble
  • Period: 01.09.2018 โ€“ 04.09.2018
  • Lot more satellite data: SEVIRI, IASI

& ATMS humidity channels Experimental setup (A) Experimental setup (B)

  • 3. Numerical Testing

T [K] at 03.05.2016 15 UTC ~1000 hPa

22.01.2019

slide-17
SLIDE 17
  • 3. Numerical Testing

Assimilation Cycle Experimental setup (A)

slide-18
SLIDE 18

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 18

  • 3. Numerical Testing โ€“ (A)

22.01.2019

RMSE LETKF LMCPF RMSE p-level [hPa] Global RMSE for obs-fg statistics (Radiosondes vs. Model) Period: 10.05.2016 โ€“ 31.05.2016 Relative humidity Temperature ~3% ~7% ~4% ~2% better

slide-19
SLIDE 19

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 19

  • 3. Numerical Testing โ€“ (A)

22.01.2019

RMSE LETKF LMCPF RMSE Channel number Global RMSE for obs-fg statistics Period: 10.05.2016 โ€“ 31.05.2016 Bending Angles (GPSRO) ๐‘ˆ๐ถ๐‘ ๐‘—๐‘•โ„Ž๐‘ข๐‘œ๐‘“๐‘ก๐‘ก (IASI) ~6% ~15% Height [m] ~17% ~3%

slide-20
SLIDE 20
  • 3. Numerical Testing

Assimilation Cycle Experimental setup (B)

slide-21
SLIDE 21

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 21

  • 3. Numerical Testing โ€“ (B)

22.01.2019

RMSE LETKF LMCPF RMSE p-level [hPa] Global RMSE for obs-fg statistics (Radiosondes vs. Model) Period: 01.09.2018 โ€“ 04.09.2018 Relative humidity Temperature ~3% ~2% ~3% better

slide-22
SLIDE 22

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 22

  • 3. Numerical Testing โ€“ (B)

22.01.2019

RMSE LETKF LMCPF RMSE Channel number Global RMSE for obs-fg statistics Period: 01.09.2018 โ€“ 04.09.2018 AIREP (WIND) IASI (๐‘ˆ๐ถ๐‘ ๐‘—๐‘•โ„Ž๐‘ข๐‘œ๐‘“๐‘ก๐‘ก) ~2% ~2% p-level [hPa] ~1% better ~6%

slide-23
SLIDE 23

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 23

  • 3. Numerical Testing

22.01.2019

mean min max LETKF LAPF LMCPF Global spread of T [K] ~ 500 hPa

slide-24
SLIDE 24
  • 3. Numerical Testing

Forecast Cycle

slide-25
SLIDE 25

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 25

  • 3. Numerical Testing โ€“ (A)

22.01.2019

Global verification of different lead times against Radiosondes Period: 10.05.2016 โ€“ 24.05.2016

slide-26
SLIDE 26

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 26

  • 3. Numerical Testing โ€“ (A)

22.01.2019

Global verification of different lead times against Radiosondes Period: 10.05.2016 โ€“ 24.05.2016

slide-27
SLIDE 27

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 27

  • 4. Summary
  • Two localized particle filters working stably in an operational setup: LAPF and

LMCPF

  • Strong improvements of scores by including model error via a Gaussian

mixture approach for the prior distribution

  • LMCPF partly decouples spread and assimilation functionality (via model error)
  • Particle Filter shows behavior similar to LETKF, with

some scores better than LETKF, some worse (o-b and forecast!)

  • First particle filter tests with full global operational resolution and setup,

including two-way European nest in global DA

  • LMCPF also works similarly for COSMO convective scale

22.01.2019

slide-28
SLIDE 28

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 28

References

  • Gerald Desroziers and Serguei Ivanov. โ€œDiagnosis and adaptive tuning of observation-error

parameters in a variational assimilationโ€. Quarterly Journal of the Royal Meteorological Society, 2001

  • Brian Hunt, Eric Kostelich and Istvan Szunyogh. โ€œEfficient data assimilation for spatiotemporal

chaos: A local ensemble transform Kalman filterโ€. Physica D: Nonlinear Phenomena, 2007

  • Gen Nakamura and Roland Potthast. โ€œInverse Modelingโ€. IOP Publishing, 2015
  • Roland Potthast, Anne Walter and Andreas Rhodin. โ€œA Localized Adaptive Particle Filter (LAPF)

within an operational NWP Frameworkโ€. MWR, 2019

  • Sebastian Reich and Colin Cotter. โ€œProbabilistic Forecasting and Bayesian Data Assimilationโ€.

Cambridge University Press, 2015

  • Christoph Schraff, Hjendrick Reich, Andreas Rhodin, Annika Schomburh, Klaus Stehphan, Africa

Periรกรฑez and Roland Potthast. โ€œKilometre-scale ensemble data assimilation for the cosmo model (kenda)โ€. Quarterly Journal of the Royal Meteorological Society, 2016

  • Peter Jan van Leeuwen, Yuan Cheng and Sebastian Reich. โ€œNonlinear Data Assimilationโ€. Frontiers

in Applied Dynamical Systems: Reviews and Tutorials. Springer, 2015

22.01.2019

slide-29
SLIDE 29

Thanks for the attention! Questions?

slide-30
SLIDE 30

Appendix

slide-31
SLIDE 31

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 31

Appendix

Global histogram of number of particles with weight โ‰ฅ 1 31.05.2016 21 UTC ~1000 hPa

22.01.2019

slide-32
SLIDE 32

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 32

Appendix

31.05.2016 21 UTC ~ 100 hPa 31.05.2016 21 UTC ~ 500 hPa 31.05.2016 21 UTC ~1000 hPa Global histograms of number of particles with weight โ‰ฅ 1

22.01.2019

slide-33
SLIDE 33

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 33

Global Model Verification Comparison

Created: 17.01.2019 EnVar ICON

22.01.2019

slide-34
SLIDE 34

anne.walter@dwd.de A Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 34

Global Model Verification Comparison - EPS

Created: 17.01.2019

22.01.2019