[PPT] - robfilter : An R-Package for Robust Time Series Filters Karen PowerPoint Presentation

SLIDE 1

robfilter: An R-Package for Robust Time Series Filters

Karen Schettlinger, Roland Fried, Ursula Gather

Fakult¨ at Statistik

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund

SLIDE 2

i1

q

Motivation i1

q

Multivariate physiological time series

Time 15:00 15:10 15:20 15:30 15:40 15:50 16:00 50 100 150 200 250 HR PLS SpO2 ART_S ART_M ART_D

Filter signals to improve ICU monitoring systems

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 1

SLIDE 3

i1

q

Contents of robfilter Version 2.0 i1

q

7 time series filters robreg.filter Simple regression filters hybrid.filter Median and repeated median hybrid filters dw.filter Two-step location-/regression-based filters wrm.filter Weighted repeated median filters robust.filter Regression filters with additional rules (outlier & level shift detection) adore.filter Adaptive repeated median filters madore.filter Multivariate adaptive repeated median filters 1 univariate smoothing method wrm.smooth Weighted repeated median smoothing

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 2

SLIDE 4

i1

q

robreg.filter – Illustration i1

q

Robust regression in a moving time window:

5

10 15 20 25 30 2 3 4 5 6 7 time time series

with delay:
5

10 15 20 25 30 2 3 4 5 6 7 time time series

nline:

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 3

SLIDE 5

i1

q

robreg.filter – Illustration i1

q

Robust regression in a moving time window:

5

10 15 20 25 30 2 3 4 5 6 7 time time series

with delay:
5

10 15 20 25 30 2 3 4 5 6 7 time time series

nline:

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 3

SLIDE 6

i1

q

robreg.filter – Illustration i1

q

Robust regression in a moving time window:

5

10 15 20 25 30 2 3 4 5 6 7 time time series

with delay:
5

10 15 20 25 30 2 3 4 5 6 7 time time series

nline:

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 3

SLIDE 7

i1

q

robreg.filter – Illustration i1

q

Robust regression in a moving time window:

5

10 15 20 25 30 2 3 4 5 6 7 time time series

with delay:
5

10 15 20 25 30 2 3 4 5 6 7 time time series

nline:

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 3

SLIDE 8

i1

q

robreg.filter – Illustration i1

q

Robust regression in a moving time window:

5

10 15 20 25 30 2 3 4 5 6 7 time time series

with delay:
5

10 15 20 25 30 2 3 4 5 6 7 time time series

nline:

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 3

SLIDE 9

i1

q

robreg.filter – Illustration i1

q

Robust regression in a moving time window:

5

10 15 20 25 30 2 3 4 5 6 7 time time series

with delay:
5

10 15 20 25 30 2 3 4 5 6 7 time time series

nline:

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 3

SLIDE 10

i1

q

robreg.filter – Illustration i1

q

Robust regression in a moving time window:

5

10 15 20 25 30 2 3 4 5 6 7 time time series

with delay:
5

10 15 20 25 30 2 3 4 5 6 7 time time series

nline:

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 3

SLIDE 11

i1

q

robreg.filter – Illustration i1

q

Robust regression in a moving time window:

5

10 15 20 25 30 2 3 4 5 6 7 time time series

with delay:
5

10 15 20 25 30 2 3 4 5 6 7 time time series

nline:

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 3

SLIDE 12

i1

q

robreg.filter – Illustration i1

q

Robust regression in a moving time window:

5

10 15 20 25 30 2 3 4 5 6 7 time time series

with delay:
5

10 15 20 25 30 2 3 4 5 6 7 time time series

nline:

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 3

SLIDE 13

i1

q

robreg.filter – Illustration i1

q

Robust regression in a moving time window:

5

10 15 20 25 30 2 3 4 5 6 7 time time series

with delay:
5

10 15 20 25 30 2 3 4 5 6 7 time time series

nline:

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 3

SLIDE 14

i1

q

robreg.filter – Illustration i1

q

Robust regression in a moving time window:

5

10 15 20 25 30 2 3 4 5 6 7 time time series

with delay:
5

10 15 20 25 30 2 3 4 5 6 7 time time series

nline:

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 3

SLIDE 15

i1

q

robreg.filter – Illustration i1

q

Robust regression in a moving time window:

5

10 15 20 25 30 2 3 4 5 6 7 time time series

with delay:
5

10 15 20 25 30 2 3 4 5 6 7 time time series

nline:

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 3

SLIDE 16

i1

q

robreg.filter – Illustration i1

q

Robust regression in a moving time window:

5

10 15 20 25 30 2 3 4 5 6 7 time time series

with delay:
5

10 15 20 25 30 2 3 4 5 6 7 time time series

nline:

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 3

SLIDE 17

i1

q

robreg.filter – Illustration i1

q

Robust regression in a moving time window:

5

10 15 20 25 30 2 3 4 5 6 7 time time series

with delay:
5

10 15 20 25 30 2 3 4 5 6 7 time time series

nline:

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 3

SLIDE 18

i1

q

robreg.filter – Illustration i1

q

Robust regression in a moving time window:

5

10 15 20 25 30 2 3 4 5 6 7 time time series

with delay:
5

10 15 20 25 30 2 3 4 5 6 7 time time series

nline:

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 3

SLIDE 19

i1

q

robreg.filter – Illustration i1

q

Robust regression in a moving time window:

5

10 15 20 25 30 2 3 4 5 6 7 time time series

with delay:
5

10 15 20 25 30 2 3 4 5 6 7 time time series

nline:

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 3

SLIDE 20

i1

q

robreg.filter – Illustration i1

q

Robust regression in a moving time window:

5

10 15 20 25 30 2 3 4 5 6 7 time time series

with delay:
5

10 15 20 25 30 2 3 4 5 6 7 time time series

nline:

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 3

SLIDE 21

i1

q

robreg.filter – Illustration i1

q

Robust regression in a moving time window:

5

10 15 20 25 30 2 3 4 5 6 7 time time series

with delay:
5

10 15 20 25 30 2 3 4 5 6 7 time time series

nline:

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 3

SLIDE 22

i1

q

robreg.filter – Illustration i1

q

Robust regression in a moving time window:

5

10 15 20 25 30 2 3 4 5 6 7 time time series

with delay:
5

10 15 20 25 30 2 3 4 5 6 7 time time series

nline:

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 3

SLIDE 23

i1

q

robreg.filter – Illustration i1

q

Robust regression in a moving time window:

5

10 15 20 25 30 2 3 4 5 6 7 time time series

with delay:
5

10 15 20 25 30 2 3 4 5 6 7 time time series

nline:

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 3

SLIDE 24

i1

q

robreg.filter – Illustration i1

q

Robust regression in a moving time window:

5

10 15 20 25 30 2 3 4 5 6 7 time time series

with delay:

5

10 15 20 25 30 2 3 4 5 6 7 time time series

nline:

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 3

SLIDE 25

i1

q

robreg.filter – Parameter Options i1

q

Robust regression techniques

robreg.filter(...,method=" ") MED Median RM Repeated Median

(Siegel, 1982)

LMS Least Median of Squares

(Hampel, 1975; Rousseeuw, 1984)

LTS Least Trimmed Squares

(Rousseeuw, 1983)

LQD Least Quartile Difference

(Croux, Rousseeuw, H¨

ssjer, 1994)

DR Deepest Regression

(Rousseeuw and Hubert, 1999)

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 4

SLIDE 26

i1

q

robreg.filter – Parameter Options i1

q

Robust regression techniques

robreg.filter(...,method=" ") MED Median RM Repeated Median

(Siegel, 1982)

LMS Least Median of Squares

(Hampel, 1975; Rousseeuw, 1984)

LTS Least Trimmed Squares

(Rousseeuw, 1983)

LQD Least Quartile Difference

(Croux, Rousseeuw, H¨

ssjer, 1994)

DR Deepest Regression

(Rousseeuw and Hubert, 1999)

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 4

SLIDE 27

i1

q

robreg.filter – Parameter Options i1

q

Robust regression techniques

robreg.filter(...,method=" ") MED Median RM Repeated Median

(Siegel, 1982)

LMS Least Median of Squares

(Hampel, 1975; Rousseeuw, 1984)

LTS Least Trimmed Squares

(Rousseeuw, 1983)

LQD Least Quartile Difference

(Croux, Rousseeuw, H¨

ssjer, 1994)

DR Deepest Regression

(Rousseeuw and Hubert, 1999)

Window width ∈ N

(width)

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 4

SLIDE 28

i1

q

robreg.filter – Parameter Options i1

q

Robust regression techniques

robreg.filter(...,method=" ") MED Median RM Repeated Median

(Siegel, 1982)

LMS Least Median of Squares

(Hampel, 1975; Rousseeuw, 1984)

LTS Least Trimmed Squares

(Rousseeuw, 1983)

LQD Least Quartile Difference

(Croux, Rousseeuw, H¨

ssjer, 1994)

DR Deepest Regression

(Rousseeuw and Hubert, 1999)

Window width ∈ N

(width)

Minimum number of non-missing observations within one window

(minNonNAs)

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 4

SLIDE 29

i1

q

robreg.filter – Parameter Options i1

q

Robust regression techniques

robreg.filter(...,method=" ") MED Median RM Repeated Median

(Siegel, 1982)

LMS Least Median of Squares

(Hampel, 1975; Rousseeuw, 1984)

LTS Least Trimmed Squares

(Rousseeuw, 1983)

LQD Least Quartile Difference

(Croux, Rousseeuw, H¨

ssjer, 1994)

DR Deepest Regression

(Rousseeuw and Hubert, 1999)

Window width ∈ N

(width)

Minimum number of non-missing observations within one window

(minNonNAs)

Online estimation

(online = TRUE / FALSE)

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 4

SLIDE 30

i1

q

robreg.filter – Parameter Options i1

q

Robust regression techniques

robreg.filter(...,method=" ") Aliases MED Median med.filter RM Repeated Median rm.filter LMS Least Median of Squares lms.filter LTS Least Trimmed Squares lts.filter LQD Least Quartile Difference lqd.filter DR Deepest Regression dr.filter

Window width ∈ N

(width)

Minimum number of non-missing observations within one window

(minNonNAs)

Online estimation

(online = TRUE / FALSE)

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 4

SLIDE 31

i1

q

robreg.filter – Example i1

q

10 20 30 40 50 60 2 4 6 8 10 time MED LTS LMS 10 20 30 40 50 60 2 4 6 8 10 time LQD RM DR

Filter Output with Delay

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 5

SLIDE 32

i1

q

robreg.filter – Example i1

q

10 20 30 40 50 60 2 4 6 8 10 time MED LTS LMS 10 20 30 40 50 60 2 4 6 8 10 time LQD RM DR

Online Filter Output

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 5

SLIDE 33

i1

q

robreg.filter – Option extrapolate i1

q

rm.filter(ts, width=11, extrapolate=FALSE, ... )

10 20 30 40 50 60 2 4 6 8 10

RM−Filter Output with Delay

time 10 20 30 40 50 60 2 4 6 8 10

Online RM−Filter Output

time

nline=FALSE
nline=TRUE

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 6

SLIDE 34

i1

q

robreg.filter – Option extrapolate i1

q

rm.filter(ts, width=11, extrapolate=TRUE , ... )

10 20 30 40 50 60 2 4 6 8 10

RM−Filter Output with Delay

time 10 20 30 40 50 60 2 4 6 8 10

Online RM−Filter Output

time

nline=FALSE
nline=TRUE

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 6

SLIDE 35

i1

q

Repeated Median Regression i1

q

hi

(Siegel, 1982)

Sample in one time window yt = (yt+i)′

βRM

t

= med

i

med

j=i

yt+i − yt+j i − j

µRM

t

= med

i

yt+i +

βRM

t

· i

with i =

−n + 1, . . . , −1, 0 (online) −m, . . . , m and n = 2m + 1 (with delay)

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 7

SLIDE 36

i1

q

hybrid.filter i1

q

Application of ’subfilters’ to a moving time window yt = (yt−m, . . . , yt−1, yt, yt+1, . . . , yt+m)′ with m = width−1

2

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 8

SLIDE 37

i1

q

hybrid.filter i1

q

Application of ’subfilters’ to a moving time window yt = (yt−m, . . . , yt−1, yt, yt+1, . . . , yt+m)′ with m = width−1

2

Φj(yt), j = 1, . . . , k, subfilters Filter output ˆ µt = med{Φ1(yt), Φ2(yt), . . . , Φk(yt)}

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 8

SLIDE 38

i1

q

hybrid.filter i1

q

Application of ’subfilters’ to a moving time window yt = (yt−m, . . . , yt−1, yt, yt+1, . . . , yt+m)′ with m = width−1

2

Φj(yt), j = 1, . . . , k, subfilters Filter output ˆ µt = med{Φ1(yt), Φ2(yt), . . . , Φk(yt)} Parameters options: method, width, minNonNAs, and extrapolate Online estimation not possible!!!

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 8

SLIDE 39

i1

q

hybrid.filter – Estimation in One Window i1

q

2

4 6 t−m ... t−1 t t+1 ... t+m UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 9

SLIDE 40

i1

q

hybrid.filter – Estimation in One Window i1

q

2

4 6

FMH

t−m ... t−1 t t+1 ... t+m

mean

yt UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 9

SLIDE 41

i1

q

hybrid.filter – Estimation in One Window i1

q

2

4 6

FMH

t−m ... t−1 t t+1 ... t+m

mean

yt

2

4 6

PFMH

t−m ... t−1 t t+1 ... t+m

LS

yt UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 9

SLIDE 42

i1

q

hybrid.filter – Estimation in One Window i1

q

2

4 6

FMH

t−m ... t−1 t t+1 ... t+m

mean

yt

2

4 6

PFMH

t−m ... t−1 t t+1 ... t+m

LS

yt

2

4 6

CFMH

t−m ... t−1 t t+1 ... t+m

mean

LS yt UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 9

SLIDE 43

i1

q

hybrid.filter – Estimation in One Window i1

q

2

4 6

MH

t−m ... t−1 t t+1 ... t+m

median

yt

2

4 6

PRMH

t−m ... t−1 t t+1 ... t+m

RM

yt

2

4 6

CRMH

t−m ... t−1 t t+1 ... t+m

median

RM yt UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 9

SLIDE 44

i1

q

hybrid.filter – Estimation in One Window i1

q

2

4 6

MMH

t−m ... t−1 t t+1 ... t+m

median
2

4 6

PRMMH

t−m ... t−1 t t+1 ... t+m

median

RM

2

4 6

CRMMH

t−m ... t−1 t t+1 ... t+m

median

RM UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 9

SLIDE 45

i1

q

dw.filter – Parameter Options i1

q

Double window filtering techniques with

uter.width

Outer window width (∈ N) inner.width Inner window width (∈ N, < outer.width) method Filter method(s) scale Scale estimation method d Trimming factor minNonNAs Minimum number of non-missing observations within one window

nline

TRUE / FALSE for online / delayed estimation extrapolate TRUE / FALSE for extrapolation to the edges

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 10

SLIDE 46

i1

q

dw.filter – Estimation in One Window i1

q

●
●
● ●
●
−2

2 4 6 8 10 t−15 t−10 t−5 t t+5 t+10 t+15

inner

uter

method window window DWMTM MED MEAN DWTRM RM LS DWMRM RM RM RM RM DWRM slope location

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 11

SLIDE 47

i1

q

dw.filter – Estimation in One Window i1

q

●
●
● ●
●
−2

2 4 6 8 10 t−15 t−10 t−5 t t+5 t+10 t+15 inner window initial RM fit

inner

uter

method window window DWMTM MED MEAN DWTRM RM LS DWMRM RM RM RM RM DWRM slope location

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 11

SLIDE 48

i1

q

dw.filter – Estimation in One Window i1

q

●
●
● ●
●
−2

2 4 6 8 10 t−15 t−10 t−5 t t+5 t+10 t+15 inner window initial RM fit

inner

uter

method window window DWMTM MED MEAN DWTRM RM LS DWMRM RM RM RM RM DWRM slope location

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 11

SLIDE 49

i1

q

dw.filter – Estimation in One Window i1

q

●
●
● ●
●
−2

2 4 6 8 10 t−15 t−10 t−5 t t+5 t+10 t+15 inner window initial RM fit trimming boundaries +dσ ^ −dσ ^

inner

uter

method window window DWMTM MED MEAN DWTRM RM LS DWMRM RM RM RM RM DWRM slope location scale ˆ σ median absolute MAD deviation QN Qn scale SN Sn scale

(Rousseeuw, Croux, 1993)

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 11

SLIDE 50

i1

q

dw.filter – Estimation in One Window i1

q

●
●
● ●
●
−2

2 4 6 8 10 t−15 t−10 t−5 t t+5 t+10 t+15 inner window initial RM fit trimming boundaries +dσ ^ −dσ ^

●
●
● ●
●
inner
uter

method window window DWMTM MED MEAN DWTRM RM LS DWMRM RM RM RM RM DWRM slope location scale ˆ σ median absolute MAD deviation QN Qn scale SN Sn scale

(Rousseeuw, Croux, 1993)

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 11

SLIDE 51

i1

q

dw.filter – Estimation in One Window i1

q

●
●
● ●
●
−2

2 4 6 8 10 t−15 t−10 t−5 t t+5 t+10 t+15 inner window

●
●
● ●
●
+dσ

^ −dσ ^ initial RM fit trimming boundaries final LS fit to the trimmed data final estimate

inner

uter

method window window DWMTM MED MEAN DWTRM RM LS DWMRM RM RM RM RM DWRM slope location scale ˆ σ Median Absolute MAD Deviation QN Qn scale SN Sn scale

(Rousseeuw, Croux, 1993)

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 11

SLIDE 52

i1

q

robust.filter – Parameter Options i1

q

width Window width ∈ N trend "MED" Median "RM" Repeated Median "LMS" Least Median of Squares "LTS" Least Trimmed Squares

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 12

SLIDE 53

i1

q

robust.filter – Parameter Options i1

q

width Window width ∈ N trend "MED" Median "RM" Repeated Median "LMS" Least Median of Squares "LTS" Least Trimmed Squares scale "MAD" Median Absolute Deviation "QN" Qn scale "SN" Sn scale "LSH" Length of the Shortest Half

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 12

SLIDE 54

i1

q

robust.filter – Parameter Options i1

q

width Window width ∈ N trend "MED" Median "RM" Repeated Median "LMS" Least Median of Squares "LTS" Least Trimmed Squares scale "MAD" Median Absolute Deviation "QN" Qn scale "SN" Sn scale "LSH" Length of the Shortest Half Options online and extrapolate Further arguments for level shift & outlier detection and outlier treatment

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 12

SLIDE 55

i1

q

robust.filter – Illustration i1

q

50 100 150 200 250 5 10 time time series true signal

fit <- robust.filter(series, width=31)

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 13

SLIDE 56

i1

q

robust.filter – Illustration i1

q

50 100 150 200 250 5 10 time time series true signal robust.filter−signal

> fit $level [1] -2.391516 -2.335128 -2.278741 -2.222354 -2.165966 ... $slope [1] 0.056387 0.056387 0.056387 0.056387 0.056387 ... $sigma [1] 0.798726 0.798726 0.798726 0.798726 0.798726 ... 245

bservations omitted

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 13

SLIDE 57

i1

q

robust.filter – Illustration i1

q

50 100 150 200 250 5 10 time time series true signal robust.filter−signal

Level shifts?

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 13

SLIDE 58

i1

q

robust.filter – Illustration i1

q

50 100 150 200 250 5 10 time time series true signal robust.filter−signal positive level shift detected at t=96

Level shifts?

> which(fit$level.shift!=0) [1] 96 > fit$level.shift[which(fit$level.shift!=0)] [1] 1

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 13

SLIDE 59

i1

q

robust.filter – Illustration i1

q

50 100 150 200 250 5 10 time time series true signal robust.filter−signal positive level shift detected at t=96

Outliers?

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 13

SLIDE 60

i1

q

robust.filter – Illustration i1

q

50 100 150 200 250 5 10 time positive level shift detected at t=96

●
●
●
●●
time series

true signal robust.filter−signal negative outliers positive outliers

Outliers?

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 13

SLIDE 61

i1

q

robust.filter – Illustration i1

q

50 100 150 200 250 5 10 time positive level shift detected at t=96

●
●
●
●●
time series

true signal robust.filter−signal negative outliers positive outliers

Outliers? 23 correctly detected 9 falsely detected 2 not detected

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 13

SLIDE 62

i1

q

Weighted Repeated Median Regression i1

q

hi

(Fried, Einbeck, Gather, 2007)

Sample in one time window yt = (yt+i)′

βRM

t

= med

i

wi✸

med

j=i

wj✸ yt+i − yt+j i − j

µRM

t

= med

i

wi✸(yt+i +

βRM

t

· i)

with i =

−n + 1, . . . , −1, 0 (online) −m, . . . , m and n = 2m + 1 (with delay)

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 14

SLIDE 63

i1

q

Weighted Repeated Median Regression i1

q

hi

(Fried, Einbeck, Gather, 2007)

Sample in one time window yt = (yt+i)′

βRM

t

= med

i

wi✸

med

j=i

wj✸ yt+i − yt+j i − j

µRM

t

= med

i

wi✸(yt+i +

βRM

t

· i)

with i =

−n + 1, . . . , −1, 0 (online) −m, . . . , m and n = 2m + 1 (with delay) w✸a denotes the set of w replications of a: 3✸a = {a, a, a}.

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 14

SLIDE 64

i1

q

Weighted Repeated Median Regression i1

q

hi

(Fried, Einbeck, Gather, 2007)

Generally: Sample (x, y) = (xi, yi)′, i = 1, . . . , n

βRM

= med

i

wi✸

med

j=i

wj✸ yi − yj xi − xj

µRM

= med

i

wi✸(yi

+ βRM · xi)

UseR! The R User Conference 2008, August 12-14, Technische Universit¨

at Dortmund 14

SLIDE 65

i1

q

Weighted Repeated Median Regression i1

q

hi

(Fried, Einbeck, Gather, 2007)

Generally: Sample (x, y) = (xi, yi)′, i = 1, . . . , n

βRM

= med

i

wi✸

med

j=i

wj✸ yi − yj xi − xj

µRM

= med

i

wi✸(yi

+ βRM · xi)

wrm.smooth:
Non-equidistant regressor variables
Smoothing with kernel weight functions

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 14

SLIDE 66

i1

q

wrm.filter – Parameter Options i1

q

y input time series (ts-object or vector) width window width (∈ N) del delay of the extracted signal: del=0 means ’online’, default is del=floor(width/2) (delayed) weight weight function: 0: equal weighting 1: triangular weights 2: Epanechnikov weights

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 15

SLIDE 67

i1

q

wrm.filter – Weight Functions i1

q

Window width = 21

5 10 15 20 0.6 0.8 1.0 1.2 1.4 Uniform Weights for Online Estimation weights 5 10 15 20 0.5 1.0 1.5 Triangular Weights for Online Estimation weights 5 10 15 20 0.1 0.2 0.3 0.4 0.5 0.6 0.7 Epanechnikov Weights for Online Estimation weights

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 16

SLIDE 68

i1

q

wrm.filter – Weight Functions i1

q

Window width = 21

5 10 15 20 0.6 0.8 1.0 1.2 1.4 Uniform Weights with a Delay of del= 1 weights 5 10 15 20 0.5 1.0 1.5 Triangular Weights with a Delay of del= 1 weights 5 10 15 20 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Epanechnikov Weights with a Delay of del= 1 weights

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 16

SLIDE 69

i1

q

wrm.filter – Weight Functions i1

q

Window width = 21

5 10 15 20 0.6 0.8 1.0 1.2 1.4 Uniform Weights with a Delay of del= 2 weights 5 10 15 20 0.5 1.0 1.5 Triangular Weights with a Delay of del= 2 weights 5 10 15 20 0.2 0.4 0.6 0.8 Epanechnikov Weights with a Delay of del= 2 weights

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 16

SLIDE 70

i1

q

wrm.filter – Weight Functions i1

q

Window width = 21

5 10 15 20 0.6 0.8 1.0 1.2 1.4 Uniform Weights with a Delay of del= 3 weights 5 10 15 20 0.5 1.0 1.5 Triangular Weights with a Delay of del= 3 weights 5 10 15 20 0.2 0.4 0.6 0.8 Epanechnikov Weights with a Delay of del= 3 weights

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 16

SLIDE 71

i1

q

wrm.filter – Weight Functions i1

q

Window width = 21

5 10 15 20 0.6 0.8 1.0 1.2 1.4 Uniform Weights with a Delay of del= 4 weights 5 10 15 20 0.5 1.0 1.5 Triangular Weights with a Delay of del= 4 weights 5 10 15 20 0.2 0.4 0.6 0.8 Epanechnikov Weights with a Delay of del= 4 weights

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 16

SLIDE 72

i1

q

wrm.filter – Weight Functions i1

q

Window width = 21

5 10 15 20 0.6 0.8 1.0 1.2 1.4 Uniform Weights with a Delay of del= 5 weights 5 10 15 20 0.5 1.0 1.5 Triangular Weights with a Delay of del= 5 weights 5 10 15 20 0.2 0.4 0.6 0.8 1.0 Epanechnikov Weights with a Delay of del= 5 weights

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 16

SLIDE 73

i1

q

wrm.filter – Weight Functions i1

q

Window width = 21

5 10 15 20 0.6 0.8 1.0 1.2 1.4 Uniform Weights with a Delay of del= 6 weights 5 10 15 20 0.5 1.0 1.5 Triangular Weights with a Delay of del= 6 weights 5 10 15 20 0.2 0.4 0.6 0.8 1.0 Epanechnikov Weights with a Delay of del= 6 weights

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 16

SLIDE 74

i1

q

wrm.filter – Weight Functions i1

q

Window width = 21

5 10 15 20 0.6 0.8 1.0 1.2 1.4 Uniform Weights with a Delay of del= 7 weights 5 10 15 20 0.5 1.0 1.5 Triangular Weights with a Delay of del= 7 weights 5 10 15 20 0.2 0.4 0.6 0.8 1.0 Epanechnikov Weights with a Delay of del= 7 weights

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 16

SLIDE 75

i1

q

wrm.filter – Weight Functions i1

q

Window width = 21

5 10 15 20 0.6 0.8 1.0 1.2 1.4 Uniform Weights with a Delay of del= 8 weights 5 10 15 20 0.5 1.0 1.5 Triangular Weights with a Delay of del= 8 weights 5 10 15 20 0.2 0.4 0.6 0.8 1.0 1.2 Epanechnikov Weights with a Delay of del= 8 weights

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 16

SLIDE 76

i1

q

wrm.filter – Weight Functions i1

q

Window width = 21

5 10 15 20 0.6 0.8 1.0 1.2 1.4 Uniform Weights with a Delay of del= 9 weights 5 10 15 20 0.5 1.0 1.5 Triangular Weights with a Delay of del= 9 weights 5 10 15 20 0.2 0.4 0.6 0.8 1.0 1.2 Epanechnikov Weights with a Delay of del= 9 weights

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 16

SLIDE 77

i1

q

wrm.filter – Weight Functions i1

q

Window width = 21

5 10 15 20 0.6 0.8 1.0 1.2 1.4 Uniform Weights with a Delay of del= 10 weights 5 10 15 20 0.5 1.0 1.5 Triangular Weights with a Delay of del= 10 weights 5 10 15 20 0.4 0.6 0.8 1.0 1.2 1.4 Epanechnikov Weights with a Delay of del= 10 weights

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 16

SLIDE 78

i1

q

Influence of the Window Width i1

q

20

40 60 80 100 −2 2 4 6 time signal n = 10 n = 40

n small n large + small bias + adapts quickly to changes + short computation time + small variance + smooth + robust Data adaptive choice of window width

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 17

SLIDE 79

i1

q

adore.filter i1

q

hi

(Schettlinger, Fried, Gather, 2008)

adaptive online repeated median filtering: Window width adaptation by a test using the ’balance’ of the residual signs n

i=1 sign(rt,i) = 0

RM approximation in current time window

❄

Test: Selection of positive & negative residuals balanced?

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 18

SLIDE 80

i1

q

adore.filter i1

q

hi

(Schettlinger, Fried, Gather, 2008)

adaptive online repeated median filtering: Window width adaptation by a test using the ’balance’ of the residual signs n

i=1 sign(rt,i) = 0

RM approximation in current time window

❄

Test: Selection of positive & negative residuals balanced?

✟ ✟ ✟ ✟ ✟ ✟ ✟ ✟ ✟ ✟ ✟ ✙

No Reduce window width

✲

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 18

SLIDE 81

i1

q

adore.filter i1

q

hi

(Schettlinger, Fried, Gather, 2008)

adaptive online repeated median filtering: Window width adaptation by a test using the ’balance’ of the residual signs n

i=1 sign(rt,i) = 0

RM approximation in current time window

❄

Test: Selection of positive & negative residuals balanced?

✟ ✟ ✟ ✟ ✟ ✟ ✟ ✟ ✟ ✟ ✟ ✙

No Reduce window width

✲ ❍❍❍❍❍❍❍❍❍❍ ❍ ❥

Yes Save current signal estimate

❄

Update window t → t + 1 and n → n + 1

✛

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 18

SLIDE 82

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 11

n(t) = 11

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 83

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 12

n(t) = 12

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 84

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 13

n(t) = 13

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 85

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 14

n(t) = 14

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 86

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 15

n(t) = 15

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 87

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 16

n(t) = 16

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 88

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 17

n(t) = 17

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 89

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 18

n(t) = 18

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 90

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 19

n(t) = 19

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 91

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 19

n(t) = 18

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 92

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 19

n(t) = 17

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 93

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 19

n(t) = 16

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 94

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 19

n(t) = 15

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 95

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 19

n(t) = 14

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 96

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 19

n(t) = 13

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 97

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 19

n(t) = 12

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 98

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 19

n(t) = 11

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 99

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 20

n(t) = 12

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 100

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 20

n(t) = 11

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 101

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 21

n(t) = 12

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 102

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 22

n(t) = 13

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 103

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 23

n(t) = 14

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 104

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 24

n(t) = 15

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 105

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 25

n(t) = 16

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 106

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 26

n(t) = 17

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 107

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 27

n(t) = 18

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 108

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 28

n(t) = 19

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 109

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 29

n(t) = 20

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 110

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5 10 15 20 25 30 5 10 15 20 time

t = 30

n(t) = 21

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 111

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5

10 15 20 25 30 5 10 15 20 time

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 112

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100 rtr=0: No restriction on the estimated signal level

5

10 15 20 25 30 5 10 15 20 time

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 113

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100 rtr=0: No restriction on the estimated signal level

5

10 15 20 25 30 5 10 15 20 time

level estimates

utside range

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 114

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100 rtr=0: No restriction on the estimated signal level

5

10 15 20 25 30 5 10 15 20 time

restrict level estimates to observational range within each window

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 115

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100 rtr=1: ’restrict to range’ of observations in the current window

5

10 15 20 25 30 5 10 15 20

time

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 116

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100

5

10 15 20 25 30 5 10 15 20 time

utlier

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 117

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100 rtr=1: ’restrict to range’ of observations in the window

5

10 15 20 25 30 5 10 15 20 time

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 118

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100 rtr=1: ’restrict to range’ of observations in the window

5 10 15 20 25 30 5 10 15 20 time

t = 18

n(t) = 18

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 119

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100 rtr=1: ’restrict to range’ of observations in the window

5 10 15 20 25 30 5 10 15 20 time

bservational range

within window

t = 18 n(t) = 18

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 120

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100 rtr=2: ’restrict to range’ of p.test most recent observations

5 10 15 20 25 30 5 10 15 20 time

bservational range at

5 most recent times

t = 18 n(t) = 18

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 121

i1

q

adore.filter – Illustration i1

q

min.width=11 p.test=5 max.width=100 rtr=2: ’restrict to range’ of p.test most recent observations

5

10 15 20 25 30 5 10 15 20 time

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 19

SLIDE 122

i1

q

madore.filter i1

q

hi

(Borowski, Schettlinger, Gather 2008)

Time window of length nt / k-variate sample with nt observations (yt+i) =       y1, t−nt+1+lag · · · y1, t+lag y2, t−nt+1+lag · · · y2, t+lag . . . . . . yk, t−nt+1+lag · · · yk, t+lag       with i = t − nt + 1 +lag, . . . , t+lag

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 20

SLIDE 123

i1

q

madore.filter i1

q

hi

(Borowski, Schettlinger, Gather 2008)

Time window of length nt / k-variate sample with nt observations (yt+i) =       y1, t−nt+1+lag · · · y1, t+lag y2, t−nt+1+lag · · · y2, t+lag . . . . . . yk, t−nt+1+lag · · · yk, t+lag       with i = t − nt + 1 + lag, . . . , t + lag and lag = 0 for online estimation

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 20

SLIDE 124

i1

q

madore.filter i1

q

hi

(Borowski, Schettlinger, Gather 2008)

Within a time window of length nt / a k-variate sample (yt+i):

1. Use adaptive univariate RM to find

µt and βt and to determine overall window width nt (→ adore.filter)

2. Estimate local covariance matrix of multivariate residuals rt+i
4. Trim observations yt+i where the Mahalanobis distance of the

corresponding residuals rt+i is larger than dnt e.g. = χ2

k;α

5. Use multivariate Least Squares on the trimmed observations

to find the level estimate µt (→ dw.filter)

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 20

SLIDE 125

i1

q

(m)adore.filter – Application i1

q

adore.filter(..., min.width=61, p.test=10, rtr=0) component-wise application

Time 15:00 15:10 15:20 15:30 15:40 15:50 16:00 50 100 150 200 250 HR PLS SpO2 ART_S ART_M ART_D

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 21

SLIDE 126

i1

q

(m)adore.filter – Application i1

q

adore.filter(..., min.width=61, p.test=10, rtr=1) component-wise application

Time 15:00 15:10 15:20 15:30 15:40 15:50 16:00 50 100 150 200 250 HR PLS SpO2 ART_S ART_M ART_D

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 21

SLIDE 127

i1

q

(m)adore.filter – Application i1

q

adore.filter(..., min.width=61, p.test=10, rtr=2) component-wise application

Time 15:00 15:10 15:20 15:30 15:40 15:50 16:00 50 100 150 200 250 HR PLS SpO2 ART_S ART_M ART_D

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 21

SLIDE 128

i1

q

(m)adore.filter – Application i1

q

madore.filter(..., min.width=61, p.test=10, rtr=2) dgt

Time 15:00 15:10 15:20 15:30 15:40 15:50 16:00 50 100 150 200 250 HR PLS SpO2 ART_S ART_M ART_D

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 21

SLIDE 129

i1

q

(m)adore.filter – Application i1

q

madore.filter(..., min.width=61, p.test=10, rtr=2) applied to blocks of highly correlated variables

Time 15:00 15:10 15:20 15:30 15:40 15:50 16:00 50 100 150 200 250 HR PLS SpO2 ART_S ART_M ART_D

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 21

SLIDE 130

i1

q

Filter Output i1

q

An object of class ...filter / a list of the elements:

level filtered signal level slope corresponding slope within each time window Each of these elements is a data.frame with column names specified by the applied method(s).

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 22

SLIDE 131

i1

q

Filter Output i1

q

An object of class ...filter / a list of the elements:

level filtered signal level slope corresponding slope within each time window Each of these elements is a data.frame with column names specified by the applied method(s).

Input parameters
scale output within each time window possible for

hybrid.filter, dw.filter, robust.filter, and adore.filter

Filter specific output

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 22

SLIDE 132

i1

q

Summary i1

q

with delay

nline

missing values adaptive widths multivariate robreg.filter ✦ ✦ ✦ ✪ ✪ hybrid.filter ✦ ✪ ✦ ✪ ✪ dw.filter ✦ ✦ ✦ ✪ ✪ robust.filter ✦ ✦ ✪ ✦ ✪ wrm.filter ✦ ✦ ✦ ✪ ✪ adore.filter ✪ ✦ ✦ ✦ ✪ madore.filter ✦ ✦ ✦ ✦ ✦

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 23

SLIDE 133

i1

q

Background References i1

q Croux, C., Rousseeuw, P.J. and H¨

ssjer, O. (1994) Generalized S-Estimators. J. Am. Stat. Assoc.

89 1271-1281. Hampel, F.R. (1975) Beyond Location Parameters: Robust Concepts and Methods. Bulletin ISI 46, 375-382. Rousseeuw, P.J. (1983) Multivariate Estimation with High Breakdown Point. In: W. Grossmann, G. Pflug, I. Vincze, W. Wertz (eds.) Proceedings of the 4th Pannonian Symposium on Mathematical Statistics and Probability, Vol. B, D. Reidel Publishing Company, Dordrecht (The Netherlands). Rousseeuw, P.J. (1984) Least Median of Squares Regression. J. Am. Stat. Assoc. 79, 871-880. Rousseeuw, P.J. and Hubert, M. (1999) Regression Depth. J. Am. Stat. Assoc. 94, 388-402. Rousseeuw, P.J., Croux, C. (1993) Alternatives to the Median Absolute Deviation. J. Am. Stat.

Assoc. 88, 1273-1283.

Siegel, A.F. (1982) Robust Regression Using Repeated Medians. Biometrika 69, 242-244.

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 24

SLIDE 134

i1

q

robfilter-References i1

q

robreg.filter:

Davies, P.L., Fried, R., Gather, U. (2004) Robust Signal Extraction for On-line Monitoring Data. J.

Stat. Plann. Inference 122 (1-2), 65-78.

Gather, U., Schettlinger, K., Fried, R. (2006) Online Signal Extraction by Robust Linear

Regression. Computation. Stat. 21 (1), 33-52.

hybrid.filter:

Fried, R., Bernholt, T., Gather, U. (2006) Repeated Median and Hybrid Filters. Comput. Stat. Data An. 50, 2313-2338.

dw.filter:

Bernholt, T., Fried, R., Gather, U., Wegener, I. (2006) Modified Repeated Median Filters. Stat.

Comput. 16, 177-192.

robust.filter:

Fried, R. (2004) Robust Filtering of Time Series with Trends. J. Nonparametr. Stat. 16, 313-328. Gather, U., Fried, R. (2004) Methods and Algorithms for Robust Filtering. COMPSTAT 2004: Proceedings in Computational Statistics, J. Antoch (eds.), Physika-Verlag, Heidelberg, 159-170. Fried, R., Gather, U. (2007) On Rank Tests for Shift Detection in Time Series. Comput. Stat. Data

An. 52, 221-233.

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 25

SLIDE 135

i1

q

robfilter-References i1

q

wrm.filter and wrm.smooth:

Fried, R., Einbeck, J., Gather, U. (2007) Weighted Repeated Median Smoothing and Filtering. J.

Am. Stat. Assoc. 102, 1300-1308.

adore.filter:

Schettlinger, K., Fried, R., Gather, U. (2008) Real Time Signal Processing by Adaptive Repeated Median Filters. Submitted.

madore.filter:

Borowski, M., Schettlinger, K., Gather, U. (2008) Multivariate Real Time Signal Processing by a Robust Adaptive Regression Filter. Submitted. Lanius, V., Gather, U. (2007) Robust Online Signal Extraction from Multivariate Time Series. Technical Report 38/2007, SFB 475, Technische Universit¨ at Dortmund.

Overview

Schettlinger, K., Fried, R., Gather, U. (2006) Robust Filters for Intensive Care Monitoring: Beyond the Running Median, Biomedizinische Technik 51(2), 49-56.

UseR! The R User Conference 2008, August 12-14, Technische Universit¨ at Dortmund 26