Monitoring and data filtering III. The Kalman Filter and its - - PowerPoint PPT Presentation

Dias 1

Monitoring and data filtering

• III. The Kalman Filter and its relation with the
• ther methods

Advanced Quantitative Herd Management Dan Jensen, IPH

Dias 2

Before this part of the course

Compare key figures (k) with expected results

κ = θ + es + eo

Results from 2 herds

780 790 800 810 820 830 840 850 860 870 880 2 4 6 8 10 12 Quarter Gain (g) Expected Herd A Herd B

Dias 3

The methods we looked at – In part I of the course (1/3)

Key figures regarded as a time series of observations, treated as a whole κ = θ + es + eo κt = θ + est + eot = θ + vt I - Shewart Control Chart

• Plot our Time Series of Observations : raw observations
• Plot our Target Value (θ’)
• Plot of Control Limits (UCL & LCL)

Not so good ... for our type

• f data ...

Dias 4

The methods we looked at – In part I of the course (2/3)

II - Shewart Control Chart

• Plot a Moving Average of observations
• Plot our Target Value (θ’)
• Plot of Control Limits (UCL & LCL)

III - Shewart Control Chart

• Plot an EWMA of our observations
• Plot our Target Value (θ’)
• Plot of Control Limits (UCL & LCL)

Still not so good ... for our type of data Here: milk yield

Dias 5

The methods we looked at – In part I of the course (3/3)

IV – Autocorrelated data

• Model the data
• Calculate a prediction for next obs.
• Plot the prediction errors

V – EWMA for autocorrelated data

• Use EWMA as one-step-ahead

predictor

• Plot the prediction errors

Here we use 0 as Center Line and see how error terms are distributed

Dias 6

The methods we looked at – In part II of the course (1/4)

A Simple DLM Time series Yt = (y1, ... , yn) Observation equation: yt = θ θ θ θt + vt, vt » N(0, Vt)

Like before: vt = es + eo The symbol θ θ θ θ t is the underlying true value at time t.

System equation: θ θ θ θ t = θ θ θ θ t-1 + wt, wt » N(0, Wt) The true value is not any longer assumed to be constant.

Before: κt = θ + est + eot = θ + vt

Dias 7

The methods we looked at – In part II of the course (2/4)

The Kalman Filter (µt-1 | Dt-1) » N(mt-1, Ct-1) (µt | Dt ) » N(mt, Ct)

1

) 1 (

−

− + =

t t t

z z λ λκ

1

) 1 (

−

− + =

t t t t t

m A Y A m

DLM and EWMA As the model adapts to the data, At converges and becomes A The calculation of the mean of the underlying level for the simple DLM looks very similar to the calculation of the EWMA

DLM EWMA

The updating equations of the Kalman Filter are used for stepwise calculation of mt and Ct

Dias 8

The methods we looked at – In part II of the course (3/4)

The different models

• Simple DLM
• DLM with a trend

General form of the DLM Observation Equation: Yt = F’t θt + νt , νt ~ N(0,Vt) System Equation: θt = Gt θt-1 + ωt, ωt ~ N(0,Wt) θt = (θ1, … , θm)’ is a vector of parameters describing the system at time t. Can include: Level, Trend, Seasonality, Periodicity, ... According to the data observed, the use of DLM allows many modeling possibilities + External information Structure

Dias 9

The methods we looked at – In part II of the course (4/4)

Monitoring methods used with DLM Monitor the forecast errors (et = Yt – ft) as we did with autocorrelated data in part I NO need for ’expected value’ V-mask (cusum) alarm limits Can look at the different components of the models (trends, seasons)

Dias 10

The multi process kalman filter (MPKF) (1/2)

Multi Process Models Class I ex: Classifying sows activities in Farrowing house A single out of a range of possible DLMs is viewed as appropriate at all time for describing the entire time series Parameters : V and W We try to find out which model is it at time t?

Dias 11

The multi process kalman filter (MPKF) (2/2)

Multi Process Models Class II ex: Monitoring Somatic Cell Counts / Boar visits No single DLM as adequate for all time: the possibility that different models are appropriate at different times is explicitly recognised and modelled through different defining parameters Parameters : V and W and Π (values for Π define fixed prior probabilities) We try to find out normal evolution, outlier, level shift We need to go back f.x. two, three steps to check which model is the right one

Dias 12

Sum up of sum up

We increased the complexity of our models and monitoring methods to try to adapt to the complexity of monitoring living beings Methods here based on a whole time series of observations

• > need for automatic registration
• > potential on-line monitoring

From part I to part II, we made our model dynamics, so the ’true underlying level’ is no more constant