Using Spatial Gradient Information to Extract Small Scale - - PowerPoint PPT Presentation

Using Spatial Gradient Information to Extract Small Scale Information from Observations With Correlated Errors

Joël Bédard and Mark Buehner

Data Assimilation and Satellite Meteorology Research Section 7th International Symposium on Data Assimilation: 21 January 2019

Motivation

Even though satellite brightness temperature is the observation type with the biggest impact on atmospheric analyses and ensuing forecasts, they are still underused (e.g. ECCC does horizontal thinning at 150km)

• Brightness temperature errors can have significant spatial

correlations, due to:

– Correlated instrument noise and representativeness error – Correlation from background state used in the observation operator

• Assimilation algorithms assume uncorrelated observation errors

(especially spatial correlations), because:

– Inverting the full covariance matrix is prohibitively expensive – Difficult to estimate the real error correlations

• It is common practice to spatially thin obs. to reduce error correlation

between remaining obs., resulting in a loss of small-scale information

• Goal: Develop a practical assimilation approach to extract small-scale

information from observations with spatially correlated errors, while still assuming spatially uncorrelated obs. in the assimilation

2/15

3

16 January, 2019

Experimental framework

Assimilation experiments are performed in a simplified context to assess the impact of observations with spatially correlated errors on analyses

• Unidimensional periodic domain with 100 grid points
• Single dimensionless variable
• Observations are available at every grid point (no interpolation, H = I)
• Background and observation error variances are set to 1.0
• Tests performed with several correlation functions (GAUS, SOAR, FOAR)

and correlation length scales are defined w.r.t. the e-folding scale:

– Background (prior) error correlation length scale (Lx): 3 grid points –

• Obs. error correlation length scale (Ly): 0.0, 1.5, 3.0, and 4.5 grid points
• The true analysis error covariances (A) computed as:

A = (I – K)B(I – K)T + KRKT where K = B (Rest + B)-1

– B and R are the true background and obs. error covariance matrices – Rest is the estimated obs. error covariance matrix used (typically diagonal)

• All assimilation tests using diagonal observation error covariance matrix

employ optimally inflated variances 3/15

4

16 January, 2019

Correlated observation error

• Positively correlated errors have more energy at the large scales
• The spectrum is similar to red noise

4/15

5

16 January, 2019

Correlated observation error

• Simply neglecting the error correlations can degrade the resulting

analysis by overfitting large scales and underfitting the small scales

5/15

6

16 January, 2019

• Inflation allows correctly fitting the scales at which background error

has the most energy (large scales), but further neglects small scales

Correlated observation error

6/15

7

16 January, 2019

Correlated observation error

• Thinning reduces spatial error correlation between the remaining obs.

(spectrum more flat), but information on the small scales is lost

7/15

8

16 January, 2019

• With more thinning, the impact of neglecting correlations is reduced,

at the expense of losing skill for an increasing number of wavenumbers at the small scales

• Similar results (but less significant) are obtained for the different

correlation functions (GAUS, SOAR, FOAR) and observation error correlation length scales

Impact of thinning on analysis error

8/15

9

16 January, 2019

• Optimal inflation ≈ 1

when correlation ≈ 0.2 (Liu and Rabier, 2002)

• With optimally inflated

diagonal R, the analysis is degraded slightly with thinning (it is best to keep most or all the obs.)

• The analysis error is

sensitive to the obs. error correlation function (GAUS, SOAR, FOAR) and length scale (Ly) used (not shown)

Thinning impact with inflated diagonal R

9/15

10

16 January, 2019

• Thinning and inflation both can prevent overfitting large scales,

but at the expense of loosing information at the small scales

• Similar to other studies that use spatial transformations to reduce

error correlations (Ruggiereo et al., 2016), spatial differences of neighboring observations (difference-obs.) can be used to increase the weight on small scales (Inspired from Anderson's presentation at the Australia data assimilation workshop in 2016)

• Combining thinned obs. (as they are currently used in operational

NWP) with spatial difference-obs. (both with their own tuned diagonal R) may allow introducing complementary information at small scales

Difference-observations

Yn-1 Y1 Y3 Y2 Y4 Yn

Original observations

Y2 - Y1 Y3 – Y2 Y4 – Y3 Yn – Yn-1

Difference observations … … 10/15

11

16 January, 2019

Difference-observations

• Spatial differential operator = multiplying spectrum by wavenumber
• More weight is given to "intermediate" scales and the information at

wavenumber 0 is lost

11/15

12

16 January, 2019

Difference-observations

• The link between neighboring difference-obs. introduces negative

correlations not accounted for in the assimilation (not shown)

• By computing differences, the information at wavenumber 0 is lost
• Using difference-obs. with diagonal R improves "intermediate" scales

12/15

Difference-obs Original obs

GAUS Ly > Lx

13

16 January, 2019

Combining original and difference-obs

• The analysis benefits from both the large scales of the original
• bservations and the small scales from difference-obs.
• Ignoring difference-obs. correlations still limits the skill at fine scales

when observation error correlation length scale is large (Ly > Lx)

13/15

14

16 January, 2019

Combining original and difference-obs

• The analysis error is reduced and it is now less sensitive to thinning
• Optimal inflation has to be computed for both thinned and diff-obs.
• The analysis error is also less sensitive to the observation error

correlation function (GAUS, SOAR, FOAR) and length scale (Ly) used (not shown)

14/15

Take Home Messages

The necessity of using a diagonal R matrix sacrifices small scales

• Both inflation and thinning can be used to give correct weight to large scales,

but underweights the small scales (when obs. error is correlated)

• The analysis error is sensitive to the observation error correlation

function (GAUS, SOAR, FOAR) and length scale (Ly) used

• With optimally inflated diagonal R, the analysis is degraded with thinning

and it is best to keep all the obs.

Supplement current methods with complementary information

• Difference observations: lose large scale information, but improves analysis

at smaller ("intermediate") scales while still using a diagonal R matrix

Combining thinned observations with difference-obs.

• Two parameters (inflation factors) need to be tuned experimentally
• Reduces overall analysis error as compared with thinned obs. alone
• Provides good fit to observations at both large and finer scales
• Analysis error is less sensitive to the amount of thinning and also to the

structure (Gaussian, SOAR, FOAR) and length scale of obs. error correlations 15/15

Additional Material

Joël Bédard and Mark Buehner

Data Assimilation and Satellite Meteorology Research Section 7th International Symposium on Data Assimilation: 21 January 2019

17

16 January, 2019

Combining original and difference-obs

A1

• The link between neighboring difference-obs. introduces negative

correlations not accounted for in the assimilation (not shown)

18

16 January, 2019

Thinning impact on combined obs.

A2

• With optimally inflated

diagonal R, the analysis is not significantly degraded with thinning of the original observations

• Optimal inflation has to be

computed for both thinned and diff-obs.

• The analysis error is less

sensitive to the obs. error correlation function (GAUS, SOAR, FOAR) and length scale (Ly) used (not shown)