Outline Introduction Sigma-point Kalman filters Introduction - - PDF document

8 February 2008 1

8 February 2008 1

Advanced Data Assimilation

in strongly nonlinear systems

Youmin Tang and Jaison Thomas Ambadan

Environmental Science & Engineering Program Natural Resources & Environmental Science Data assimilation workshop at Banff, Feb 3-8

8 February 2008 2

Outline

• Introduction
• Sigma-point Kalman filters
• Simulations with Lorenz model
• A Reduced SPKF
• Summary & Conclusion.

Simulations with Lorenz model Summary & Conclusion A Reduced SPKF Sigma-point Kalman filters Introduction

8 February 2008 2

8 February 2008 3

Introduction

Data assimilation – a dynamical state space estimate problem

(2) y equation (1) equation

k 1

) , ( ); , (

k k k k k

v x h n Observatio u x f x State = =

−

where is state vector at time k, f state transition function, and process noise with known distribution; is observations at time instant k, h

• bservation function, and observation noise with

known distribution.

k

x

k

u

k

y

k

v

Simulations with Lorenz model Summary & Conclusion A Reduced SPKF Sigma-point Kalman filters Introduction 8 February 2008 4

Statistical estimation

KF & EKF

− − −

− = − + =

k k

x k k x k k k k k k

P H K I P v x h E y K x x ) ( ))), , ˆ ( ( ( ˆ ˆ

1 1 1 1 1

1

− − − − − − −

+ = =

−

k T k x k x k k k

Q L P L P u x f E x

k k

)], , ˆ ( [ ˆ

1 − − −

+ = ] [

T x k k T k x k

H P H R H P K

k k

Analysis step The forecast step

] ) )( [( ], ) )( [(

T k k k k k T k k k k k

v v v v E R u u u u E Q − − = − − =

Simulations with Lorenz model Summary & Conclusion A Reduced SPKF Sigma-point Kalman filters Introduction

8 February 2008 3

8 February 2008 5

Statistical estimation

EnKF--- Popular approach

))), , ˆ ( ( ( ˆ ˆ

k k k k k k

v x h E y K x x

− −

− + = Linear assumption for measurement function.

] ) ˆ )( ˆ [( )], , ˆ ( [ ˆ

1 1 T k k x k k k

x x x x E P u x f E x

k

− − = =

− − − − − −

1 − − −

+ = ] [

T x k k T k x k

H P H R H P K

k k

Analysis step The forecast step

Simulations with Lorenz model Summary & Conclusion A Reduced SPKF Sigma-point Kalman filters Introduction 8 February 2008 6

Statistical estimation

EnKF--- classic approach

))), , ˆ ( ( ( ˆ ˆ

k k k k k k

v x h E y K x x

− −

− + = No linear assumption for measurement function.

] ))) , ˆ ( ( )))( , ˆ ( ( [( ] ))) , ˆ ( ( )( ˆ [( )], , ˆ ( [ ˆ

1 1 T k k k k y T k k k y x k k k

v x h E y v x h E y E P v x h E y x x E P u x f E x

k k k

− − − − − − −

− − = − − = = Analysis step The forecast step

Simulations with Lorenz model Summary & Conclusion A Reduced SPKF Sigma-point Kalman filters Introduction

1 −

=

k k k

y y x k

P P K

Ref: (Gelb, 1974)

T k y k x x

K P K P P

k k k

− =

−

8 February 2008 4

8 February 2008 7

Sigma-point Kalman filter

• The SPKF makes use of a reformulated Kalman gain K

and “chooses” the ensemble deterministically in such a way that it can capture the statistical moments of the nonlinear model accurately; in other words, the forecast error covariance equation is computed using deterministically chosen samples, called “sigma-points”.

• The SPKF algorithm has been successfully implemented in

many areas like robotics, artificial intelligence, natural language processing, and global positioning systems navigation.

Ref: (Julier et al. 1995; Nørgad Magnus et al. 2000; Ito and Xiong 2000; Lefebvre et al. 2002;Wan and Van Der Merve 2000; Haykin 2001; Van der Merwe 2004, Van der Merwe and Wan4 April 2001,M;).

Simulations with Lorenz model Summary & Conclusion A Reduced SPKF Sigma-point Kalman filters Introduction 8 February 2008 8

SP-Unscented Kalman filter

Unscented transformation (Julier et al. 1995; Julier 1998;

Wan and Van Der Merve 2000; Julier 2002). Consider the propagation pf a L-dimensional random variable x through an arbitrary nonlinear function:

x

P x x x g covariance and mean has ); ( = ϕ

i x i i x i i i i

P k L x P k L x x w S ) ) ( ( , ) ) ( ( ; ); , { + − = + + = = = χ χ χ χ

parameter scaling a is 2 1 2 1 k L i k L w i k L k w

i

,..., ) ( = + = = + =

the dimension of the state-space L increases, the radius of the sphere that bounds all the sigma-points increases as well. Simulations with Lorenz model Summary & Conclusion A Reduced SPKF Sigma-point Kalman filters Introduction

); (

i i

g χ ψ =

8 February 2008 5

8 February 2008 9

SP-UKF Algorithm

)), ˆ ( ( ˆ ˆ

− −

− + =

k k k k k

x h y K x x

T k i k k i k L i i x L i i k i k i k i k

x x w P w x f

k

) ˆ )( ˆ ( ˆ ), (

2 2 − − = − = −

− − = = =

∑ ∑

ψ ψ ψ χ ψ

Analysis step Forecast step

Simulations with Lorenz model Summary & Conclusion A Reduced SPKF Sigma-point Kalman filters Introduction

1 −

=

k k k

y y x k

P P K

T k y k x x

K P K P P

k k k

− =

−

T k i k k L i i k i k y x T k i k k i k L i i y L i i k i k i k i k

y Y x w P y Y y Y w P Y w y h Y

k k

) ˆ )( ˆ ( ) ˆ )( ˆ ( ˆ ), (

2 2 2 − − = − − = = −

− − = − − = = =

∑ ∑ ∑

ψ χ Measurement step

8 February 2008 10

SP-Central Difference KF

• In SP-CDKF the analytical derivatives in EKF

are replaced by numerically evaluated central divided differences, based on Sterling’s polynomial interpolation.

(Ito and Xiong 2000; Nørga°d Magnus et al.

2000; Lefebvre et al. 2002).

Simulations with Lorenz model Summary & Conclusion A Reduced SPKF Sigma-point Kalman filters Introduction

8 February 2008 6

8 February 2008 11

Lorenz model

Experimental conditions are the same as those used by Miller (Miller, 1994) and Evensen (Evensen 1997)

True value: integrate the model over 4000 time steps using prescribed parameters and initial conditions. Observation: true value plus normal distribute noise;

Simulations with Lorenz model Summary & Conclusion A Reduced SPKF Sigma-point Kalman filters Introduction 8 February 2008 12

Lorenz model: state estimation

Observation and initial conditions: their true values plus normal distributed noise . The assimilation interval is 25 time steps.

) , ( 2 N

Simulations with Lorenz model Summary & Conclusion A Reduced SPKF Sigma-point Kalman filters Introduction

8 February 2008 7

8 February 2008 13 2

1 ) (

true k k

x x N Error − =

Simulations with Lorenz model Summary & Conclusion A Reduced SPKF Sigma-point Kalman filters Introduction 8 February 2008 14

EnKF with 19 members

Simulations with Lorenz model Summary & Conclusion A Reduced SPKF Sigma-point Kalman filters Introduction

8 February 2008 8

8 February 2008 15

Stronger noise (10 times)

Simulations with Lorenz model Summary & Conclusion A Reduced SPKF Sigma-point Kalman filters Introduction 8 February 2008 16

Strong noise + Less observations

Simulations with Lorenz model Summary & Conclusion A Reduced SPKF Sigma-point Kalman filters Introduction

8 February 2008 9

8 February 2008 17

Parameter estimation

• If the measurement function is nonlinear, it

has to be linearized in the EnKF.

k k k k k k k

r x f y q + Λ = + Λ = Λ

− −

) , (

1 1

Simulations with Lorenz model Summary & Conclusion Constraints & issues Sigma-point Kalman filters Introduction 8 February 2008 18

Parameter estimation

Initial parameters: true parameters plus normal distributed noise of covariance 100. Two parameters are estimated:

β ρ and

Kivman, G.A. 2003: Sequential parameter estimation for stochastic system, Nonlinear. Process. in Geophysics, 10, 253-259.

Simulations with Lorenz model Summary & Conclusion A Reduced SPKF Sigma-point Kalman filters Introduction

8 February 2008 10

8 February 2008 19

Joint estimation (state +parameters)

Simulations with Lorenz model Summary & Conclusion A Reduced SPKF Sigma-point Kalman filters Introduction 8 February 2008 20

A “big” drawback of SPKF

• For an L-dimensional system, the number of sigma-points

required to estimate the true statistics is 2L+1

• 2L+1 sigma-point integration is impossible, when the

dimension system is of the order of tens of millions as it happens often in GCMs

Simulations with Lorenz model Summary & Conclusion A Reduced SPKF Sigma-point Kalman filters Introduction

8 February 2008 11

8 February 2008 21

A Possible solution – reducing sigma-points

Simulations with Lorenz model Summary & Conclusion A Reduced SPKF Sigma-point Kalman filters Introduction i x i i x i i i i

P k L x P k L x x w S ) ) ( ( , ) ) ( ( ; ); , { + − = + + = = = χ χ χ χ

8 February 2008 22

A Reduced Sigma-point KF - (Lorenz ’96 model)

Simulations with Lorenz model Summary & Conclusion A Reduced SPKF Sigma-point Kalman filters Introduction with reduced Sigma-points (100) with Full Sigma-points (241 )

8 February 2008 12

8 February 2008 23 Simulations with Lorenz model Summary & Conclusion A Reduced SPKF Sigma-point Kalman filters Introduction with reduced Sigma-points (20)

Reduced sigma-points

8 February 2008 24

Conclusions

• The SPKF is a technique for a derivative-less optimal

estimation using a deterministic sampling approach that ensures accurate estimation of error statistics.

• The SPKF is capable of estimating model state and

parameters with better accuracy than EKF and EnKF for strong nonlinear systems.

• The SPKF is practically difficult for high dimensional
• systems. A possible solution is to reduce the number of

sigma-points by “selecting a particular set of sigma- points”.

Simulations with Lorenz model Summary & Conclusion A Reduced SPKF Sigma-point Kalman filters Introduction