One Step Studentized M -estimator M -Estimator Marek Omelka - - PowerPoint PPT Presentation

▶
one step studentized m estimator
SMART_READER_LITE
LIVE PREVIEW

One Step Studentized M -estimator M -Estimator Marek Omelka - - PowerPoint PPT Presentation

Aug 29, 2023 316 likes •892 views

EYSM 2005 Marek Omelka One Step Studentized One Step Studentized M -estimator M -Estimator Marek Omelka Department of Probability and Mathematical Statistics, Charles University in Prague omelka@karlin.mff.cuni.cz EYSM 2005

slide-1
SLIDE 1

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

One Step Studentized M-estimator

Marek Omelka

Department of Probability and Mathematical Statistics, Charles University in Prague

  • melka@karlin.mff.cuni.cz

EYSM 2005 August 22–26, 2005, Debrecen

slide-2
SLIDE 2

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Outline

  • 1. Motivation
  • 2. Consistency of the one step estimator
  • 3. Hadamard differentiability and asymptotic normality
  • 4. Bootstrapping
  • 5. Numerical results
slide-3
SLIDE 3

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Motivation

Let X1, . . . , Xn be independent random variables, identically dis- tributed according to a distribution function F. In the robust setting, we usually suppose that F ∈ Hε(F0), where Hε(F0) = {F : F(x) = (1 − ε)F0(x−µ

σ ) + εH(x)}.

Our aim is to estimate the location parameter µ.

slide-4
SLIDE 4

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Motivation

Let X1, . . . , Xn be independent random variables, identically dis- tributed according to a distribution function F. In the robust setting, we usually suppose that F ∈ Hε(F0), where Hε(F0) = {F : F(x) = (1 − ε)F0(x−µ

σ ) + εH(x)}.

Our aim is to estimate the location parameter µ. The studentized M-estimator is defined as Mn = arg min

t∈R n

  • i=1

ρ(Xi−t

Sn ),

where Sn is an appropriate estimate of the (nuisance) scale parameter. In the following, we will suppose that the function ρ is symmetric.

slide-5
SLIDE 5

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Motivation

We usually find the M-estimator as the root of the equation

n

  • i=1

ψ

  • Xi−Mn

Sn

  • = 0,

(1) where ψ = ρ′.

slide-6
SLIDE 6

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Motivation

We usually find the M-estimator as the root of the equation

n

  • i=1

ψ

  • Xi−Mn

Sn

  • = 0,

(1) where ψ = ρ′. As a quick approximation to the solution of (1) the so called one step estimator is used, which is defined as M (1)

n

= M (0)

n + Sn 1 n

n

i=1 ψ

  • Xi−M(0)

n

Sn

  • 1

n

n

i=1 ψ′

  • Xi−M(0)

n

Sn

, (2) where M (0)

n

is an initial (scale equivariant) estimate of location.

slide-7
SLIDE 7

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Motivation

Reasons for one-step estimator M (1)

n

  • easy to compute
  • simulation studies show good properties (e.g. Andrews et al. (1972))
  • in the case of symmetric distribution function F – the same asymp-

totic efficiency as Mn

  • at least asymptotically it solves the problem of multiple roots of the

defining equation for M-estimator

  • it usually has a lower bias than Mn (e.g. Rousseeuw and Croux

(1994))

slide-8
SLIDE 8

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Motivation

One step with proper length in the right direction is often enough.

slide-9
SLIDE 9

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Consistency

To study asymptotic properties of the one-step estimator it is conve- nient to look at the estimator as the statistical functional. T : D(F) → R

slide-10
SLIDE 10

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Consistency

To study asymptotic properties of the one-step estimator it is conve- nient to look at the estimator as the statistical functional. Some examples of statistical functionals :

  • Median – med(F) = F −1(1

2)

  • MAD – MAD(F) = inf{t : F(F −1(1

2) + t) − F(F −1(1 2) − t) > 1 2}

  • studentized M-estimator – EF ψ
  • X1−M(F)

S(F)

  • = 0
slide-11
SLIDE 11

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Consistency

To study asymptotic properties of the one-step estimator it is conve- nient to look at the estimator as the statistical functional. Some examples of statistical functionals :

  • Median – med(F) = F −1(1

2)

  • MAD – MAD(F) = inf{t : F(F −1(1

2) + t) − F(F −1(1 2) − t) > 1 2}

  • studentized M-estimator – EF ψ
  • X1−M(F)

S(F)

  • = 0

The statistical functional for the one-step estimator is T 1(F) = T 0(F) + EF ψ

  • X1−T 0(F)

S(F)

  • EF ψ′
  • X1−T 0(F)

S(F)

.

  • Note. If F is symmetric, then usually T 0(F) = M(F) and so also T 1(F) = M(F).
slide-12
SLIDE 12

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Consistency

By consistency of the one-step estimator we will understand that T 1(Fn) − − − →

n→∞ T 1(F)

[F] − a.s..

slide-13
SLIDE 13

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Consistency

By consistency of the one-step estimator we will understand that T 1(Fn) − − − →

n→∞ T 1(F)

[F] − a.s.. As

T 1(Fn) = T 0(Fn) + S(Fn)

1 n

n

i=1 ψ

  • Xi−T 0(Fn)

S(Fn)

  • 1

n

n

i=1 ψ′

  • Xi−T 0(Fn)

S(Fn)

,

for consistency of T 1(Fn) we need at least consistency of the estimators T 0(Fn) and S(Fn), and also continuity of the functions λF(s, t) = EF ψ(X1−t

s ),

λ′

F(s, t) = EF ψ′(X1−t s )

at the point (S(F), T 0(F)).

slide-14
SLIDE 14

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Consistency - artificial example

A very popular ψ-function is the Huber function defined as

ψk(x) =    −k, x < −k x, |x| ≤ k k, x > k

−4 −2 2 4 −1.5 −1.0 −0.5 0.0 0.5 1.0 1.5

The problem of this function is its nondifferentiability in the points ±k.

slide-15
SLIDE 15

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Consistency - artificial example

A very popular ψ-function is the Huber function defined as

ψk(x) =    −k, x < −k x, |x| ≤ k k, x > k

−4 −2 2 4 −1.5 −1.0 −0.5 0.0 0.5 1.0 1.5

The problem of this function is its nondifferentiability in the points ±k. Let the distribution function be F(x) = 0.7 Φ(x) + 0.3 δ2, where Φ is a d.f.

  • f N(0, 1) and δ2 is Dirac measure at the point 2. Let the initial estimate

be the median – that is T 0(F) = Φ−1

1 2(1−0.3)

  • ( .

= 0.566), and for simplicity

we will fix the scale as S(F) = 1. Then for k = 2 − T 0(F) the function

λ′

F(t) = EF ψ′ k(X1 − t) is not continuous at the point T 0(F), which causes

inconsistency of one-step estimator.

slide-16
SLIDE 16

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Consistency - artificial example

The function λ′

F(t) = EF ψ′ k(X1 − t)

0.0 0.2 0.4 0.6 0.8 1.0 0.5 0.6 0.7 0.8 0.9 1.0 t

slide-17
SLIDE 17

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Consistency - artificial example

The function λ′

F(t) = EF ψ′ k(X1 − t)

0.0 0.2 0.4 0.6 0.8 1.0 0.5 0.6 0.7 0.8 0.9 1.0 t

A simple calculation shows that the one-step estimator should oscillate between the values 0.69 and 0.75.

slide-18
SLIDE 18

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Consistency - artificial example

0.5 0.6 0.7 0.8 2 4 6 8 10 12

One step estimator

n = 1 000

slide-19
SLIDE 19

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Consistency - artificial example

0.66 0.68 0.70 0.72 0.74 0.76 0.78 0.80 5 10 15 20

One step estimator

n = 10 000

slide-20
SLIDE 20

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Consistency - artificial example

0.66 0.68 0.70 0.72 0.74 0.76 0.78 0.80 5 10 15 20

One step estimator

n = 10 000

slide-21
SLIDE 21

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Asymptotic normality

It is quite well known that if the d.f. F is symmetric, then the one-step estimator T 1(Fn) has under some mild conditions the same asymptotic distribution as (fully iterated) estimator M(Fn).

Under a little bit more restrictive conditions we can even prove that √n(T 1(Fn) − M(Fn)) = OP(n−1/2).

slide-22
SLIDE 22

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Asymptotic normality

It is quite well known that if the d.f. F is symmetric, then the one-step estimator T 1(Fn) has under some mild conditions the same asymptotic distribution as (fully iterated) estimator M(Fn).

Under a little bit more restrictive conditions we can even prove that √n(T 1(Fn) − M(Fn)) = OP(n−1/2).

To prove the asymptotic normality and some other asymptotic results we will show that the functional T 1(G) = T 0(G) + S(G) EG ψ

  • X1−T 0(G)

S(G)

  • EG ψ′
  • X1−T 0(G)

S(G)

. is Hadamard differentiable at the point F.

slide-23
SLIDE 23

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Asymptotic normality

Def.. A map φ : D → E (between normed spaces) is called differ- entiable at θ if there is a continuous linear map φ′

θ : D → E such

that sup

h∈K

  • φ(θ + th) − φ(θ)

t − φ′

θ(h)

  • → 0,

as t → 0. for every compact K ⊂ D. More about Hadamard differentiability can be found e.g. in Fernholz (1983) and van der Vaart and Wellner (1996).

slide-24
SLIDE 24

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Asymptotic normality

To prove the Hadamard differentiability of the mapping φ : G → S(G) EG ψ

  • X1−T 0(G)

S(G)

  • EG ψ′
  • X1−T 0(G)

S(G)

,

slide-25
SLIDE 25

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Asymptotic normality

To prove the Hadamard differentiability of the mapping φ : G → S(G) EG ψ

  • X1−T 0(G)

S(G)

  • EG ψ′
  • X1−T 0(G)

S(G)

, it is useful to decompose it as φ = φ3 ◦ φ2 ◦ φ1, where φ1 : G →

  • S(G), T 0(G), EG ψ(X1−t

s ), EG ψ′(X1−t s )

  • φ2 : (s, t, f1(·, ·), f2(·, ·)) → (s, f1(s, t), f2(s, t))

φ3 : (A, B, C) → A B C .

slide-26
SLIDE 26

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Asymptotic normality

Let T 0(F) and S(F) be Hadamard differentiable with the derivatives

√nT 0

H(Fn − F) =

1 √n

n

  • i=1

ϕT 0(Xi), √nS0

H(Fn − F) =

1 √n

n

  • i=1

ϕS(Xi),

slide-27
SLIDE 27

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Asymptotic normality

Let T 0(F) and S(F) be Hadamard differentiable with the derivatives

√nT 0

H(Fn − F) =

1 √n

n

  • i=1

ϕT 0(Xi), √nS0

H(Fn − F) =

1 √n

n

  • i=1

ϕS(Xi),

then we can prove the expansion

√n(T 1(Fn) − T 1(F)) =

1 √nγ1 n

  • i=1
  • ψ
  • Xi−T 0(F)

S(F)

  • − γ
  • −

γ √nγ2

1

n

  • i=1
  • ψ′

Xi−T 0(F) S(F)

  • − γ1
  • + γγ2

γ2

1

1 √n

n

  • i=1

ϕT 0(Xi) +

  • γ

γ1 + γγ2x γ2

1 − γ1x

γ1

1 √n

n

  • i=1

ϕS(Xi) + oP(1),

slide-28
SLIDE 28

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Asymptotic normality

Let T 0(F) and S(F) be Hadamard differentiable with the derivatives

√nT 0

H(Fn − F) =

1 √n

n

  • i=1

ϕT 0(Xi), √nS0

H(Fn − F) =

1 √n

n

  • i=1

ϕS(Xi),

then we can prove the expansion

√n(T 1(Fn) − T 1(F)) =

1 √nγ1 n

  • i=1
  • ψ
  • Xi−T 0(F)

S(F)

  • − γ
  • −

γ √nγ2

1

n

  • i=1
  • ψ′

Xi−T 0(F) S(F)

  • − γ1
  • + γγ2

γ2

1

1 √n

n

  • i=1

ϕT 0(Xi) +

  • γ

γ1 + γγ2x γ2

1 − γ1x

γ1

1 √n

n

  • i=1

ϕS(Xi) + oP(1),

where

γ = E ψ

  • X1−T 0(F)

S(F)

  • γ1x

=

1 S E ψ′ X1−T 0(F) S(F) X1−T 0(F) S(F)

  • γ1

=

1 S E ψ′ X1−T 0(F) S(F)

  • γ2x

=

1 S2 E ψ′′ X1−T 0(F) S(F) X1−T 0(F) S(F)

  • γ2

=

1 S2 E ψ′′ X1−T 0(F) S(F)

slide-29
SLIDE 29

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Asymptotic normality

Let T 0(F) and S(F) be Hadamard differentiable with the derivatives

√nT 0

H(Fn − F) =

1 √n

n

  • i=1

ϕT 0(Xi), √nS0

H(Fn − F) =

1 √n

n

  • i=1

ϕS(Xi),

then we can prove the expansion

√n(T 1(Fn) − T 1(F)) =

1 √nγ1 n

  • i=1
  • ψ
  • Xi−T 0(F)

S(F)

  • − γ
  • −

γ √nγ2

1

n

  • i=1
  • ψ′

Xi−T 0(F) S(F)

  • − γ1
  • + γγ2

γ2

1

1 √n

n

  • i=1

ϕT 0(Xi) +

  • γ

γ1 + γγ2x γ2

1 − γ1x

γ1

1 √n

n

  • i=1

ϕS(Xi) + oP(1),

where

γ = E ψ

  • X1−T 0(F)

S(F)

  • = 0

γ1x =

1 S E ψ′ X1−T 0(F) S(F) X1−T 0(F) S(F)

  • = 0

γ1 =

1 S E ψ′ X1−T 0(F) S(F)

  • γ2x

=

1 S2 E ψ′′ X1−T 0(F) S(F) X1−T 0(F) S(F)

  • γ2

=

1 S2 E ψ′′ X1−T 0(F) S(F)

  • = 0
slide-30
SLIDE 30

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Asymptotic normality

Let T 0(F) and S(F) be Hadamard differentiable with the derivatives

√nT 0

H(Fn − F) =

1 √n

n

  • i=1

ϕT 0(Xi), √nS0

H(Fn − F) =

1 √n

n

  • i=1

ϕS(Xi),

then we can prove the expansion

√n(T 1(Fn) − T 1(F)) =

1 √nγ1 n

  • i=1
  • ψ
  • Xi−T 0(F)

S(F)

  • − γ
  • −

γ √nγ2

1

n

  • i=1
  • ψ′

Xi−T 0(F) S(F)

  • − γ1
  • + γγ2

γ2

1

1 √n

n

  • i=1

ϕT 0(Xi) +

  • γ

γ1 + γγ2x γ2

1 − γ1x

γ1

1 √n

n

  • i=1

ϕS(Xi) + oP(1),

where

γ = E ψ

  • X1−T 0(F)

S(F)

  • = 0

γ1x =

1 S E ψ′ X1−T 0(F) S(F) X1−T 0(F) S(F)

  • = 0

γ1 =

1 S E ψ′ X1−T 0(F) S(F)

  • γ2x

=

1 S2 E ψ′′ X1−T 0(F) S(F) X1−T 0(F) S(F)

  • γ2

=

1 S2 E ψ′′ X1−T 0(F) S(F)

  • = 0
slide-31
SLIDE 31

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Bootstrap

According to the Lemma 3.9.11 of van der Vaart and Wellner (1996) Hadamard differentiability of the functional T 1(G) at the point F implies also weak consistency of the bootstrap distribution, that is

sup

x∈R

  • P ∗ √n[T 1(F ∗

n) − T 1(Fn)] ≤ x

  • − P

√n[T 1(Fn) − T 1(F)] ≤ x

  • P

− − − →

n→∞ 0,

where F ∗

n denotes the empirical distribution function of a bootstrap

sample.

slide-32
SLIDE 32

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Bootstrap

According to the Lemma 3.9.11 of van der Vaart and Wellner (1996) Hadamard differentiability of the functional T 1(G) at the point F implies also weak consistency of the bootstrap distribution, that is

sup

x∈R

  • P ∗ √n[T 1(F ∗

n) − T 1(Fn)] ≤ x

  • − P

√n[T 1(Fn) − T 1(F)] ≤ x

  • P

− − − →

n→∞ 0,

where F ∗

n denotes the empirical distribution function of a bootstrap

sample. For strong consistency of the bootstrap distribution, we need a stronger differentiability property of the statistical functional. In fact, we need √n

  • T 1(Fn + n−1/2hn) − T 1(Fn)
  • −

− − →

n→∞ T 1 H(h)

for almost every sequence Fn and every converging sequence hn → h.

slide-33
SLIDE 33

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Bootstrap

  • Theorem. If T 0(F) and S(F) satisfied stronger differentiability

condition (and some other regularity conditions are satisfied) then the bootstrap distribution is strongly consistent consistent estima- tor of the sampling distribution.

slide-34
SLIDE 34

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Bootstrap

  • Theorem. If T 0(F) and S(F) satisfied stronger differentiability

condition (and some other regularity conditions are satisfied) then the bootstrap distribution is strongly consistent consistent estima- tor of the sampling distribution. Bootstrap distribution can be also used to estimate the variance of √n(T 1(Fn) − T 1(F)). But to justify bootstrap we have to show that the sequence n(T 1(F ∗

n) − T 1(Fn))2 is uniformly integrable.

slide-35
SLIDE 35

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Bootstrap

  • Theorem. If T 0(F) and S(F) satisfied stronger differentiability

condition (and some other regularity conditions are satisfied) then the bootstrap distribution is strongly consistent consistent estima- tor of the sampling distribution. Bootstrap distribution can be also used to estimate the variance of √n(T 1(Fn) − T 1(F)). But to justify bootstrap we have to show that the sequence n(T 1(F ∗

n) − T 1(Fn))2 is uniformly integrable.

  • Theorem. If n(T 0(F ∗

n)−T 0(Fn))2 and n(S(F ∗ n)−S(Fn))2 are uni-

formly integrable (and some other regularity conditions are satis- fied) then the bootstrap variance is strongly consistent estimator

  • f the asymptotic variance.
slide-36
SLIDE 36

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Bootstrap

Some further comments

  • First numerical results in accordance with Singh (1981) show that

for small and medium samples bootstrap of robust estimators is very unrobust.

slide-37
SLIDE 37

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Bootstrap

Some further comments

  • First numerical results in accordance with Singh (1981) show that

for small and medium samples bootstrap of robust estimators is very unrobust.

slide-38
SLIDE 38

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Bootstrap

Some further comments

  • First numerical results in accordance with Singh (1981) show that

for small and medium samples bootstrap of robust estimators is very

  • unrobust. That is why we ‘winsorize’ bootstrap.
slide-39
SLIDE 39

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Bootstrap

Some further comments

  • First numerical results in accordance with Singh (1981) show that

for small and medium samples bootstrap of robust estimators is very

  • unrobust. That is why we ‘winsorize’ bootstrap.
  • As both median and MAD are unsmooth functionals, some kind of

smooth bootstrap may be helpful. But I have found no method of constructing smooth bootstrap which would work satisfactory in all situations.

slide-40
SLIDE 40

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Numerical study

Let the initial estimate be the median (mn) and the scale estimate (Sn) be the MAD (multiplied by 1.4826). As the psi-function we will have the ‘smoothed-Huber function’ – ψ1.5

ψ(x) =    |x|, x < 0.8 p4(x), 0.8 < |x| ≤ 1 p4(1), x > |1|,

−4 −2 2 4 −1.5 −1.0 −0.5 0.0 0.5 1.0 1.5

where p4(u) = 38.4 − 175.0u + 300.0u2 − 225.0u3 + 62.5u4 and ψk(x) =

k 0.9ψ(0.9x k ).

slide-41
SLIDE 41

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Numerical study

Let the initial estimate be the median (mn) and the scale estimate (Sn) be the MAD (multiplied by 1.4826). As the psi-function we will have the ‘smoothed-Huber function’ – ψ1.5

ψ(x) =    |x|, x < 0.8 p4(x), 0.8 < |x| ≤ 1 p4(1), x > |1|,

−4 −2 2 4 −1.5 −1.0 −0.5 0.0 0.5 1.0 1.5

where p4(u) = 38.4 − 175.0u + 300.0u2 − 225.0u3 + 62.5u4 and ψk(x) =

k 0.9ψ(0.9x k ).

The winsorized bootstrap is defined as (c = 2.25)

Y ∗

i =

   mn − cSn, X∗

i − mn

< −cSn X∗

i ,

|X∗

i − mn|

≤ cSn mn + cSn, X∗

i − mn

≥ cSn .

slide-42
SLIDE 42

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Numerical study

Let the initial estimate be the median (mn) and the scale estimate (Sn) be the MAD (multiplied by 1.4826). As the psi-function we will have the ‘smoothed-Huber function’ – ψ1.5

ψ(x) =    |x|, x < 0.8 p4(x), 0.8 < |x| ≤ 1 p4(1), x > |1|,

−4 −2 2 4 −1.5 −1.0 −0.5 0.0 0.5 1.0 1.5

where p4(u) = 38.4 − 175.0u + 300.0u2 − 225.0u3 + 62.5u4 and ψk(x) =

k 0.9ψ(0.9x k ).

The winsorized bootstrap is defined as (c = 2.25)

Y ∗

i =

   mn − cSn, X∗

i − mn

< −cSn X∗

i ,

|X∗

i − mn|

≤ cSn mn + cSn, X∗

i − mn

≥ cSn .

We use percentile method for construction of confidence intervals.

slide-43
SLIDE 43

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Numerical study - confidence intervals

We compare confidence intervals from the classical and robust boot- strap with

  • NA – normal approximation based on the whole expansion
  • NA-simple – normal approximation under the assumption of symme-

try (Huber modification)

√n(T 1(Fn) − T 1(F)) =

1 √nγ1 n

  • i=1
  • ψ
  • Xi−T 0(F)

S(F)

  • − γ
  • −

γ √nγ2

1

n

  • i=1
  • ψ′

Xi−T 0(F) S(F)

  • − γ1
  • + γγ2

γ2

1

1 √n

n

  • i=1

ϕT 0(Xi) +

  • γ

γ1 + γγ2x γ2

1 − γ1x

γ1

1 √n

n

  • i=1

ϕS(Xi) + oP(1),

slide-44
SLIDE 44

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Numerical study - confidence intervals

We compare confidence intervals from the classical and robust boot- strap with

  • NA – normal approximation based on the whole expansion
  • NA-simple – normal approximation under the assumption of symme-

try (Huber modification)

√n(T 1(Fn) − T 1(F)) =

1 √nγ1 n

  • i=1
  • ψ
  • Xi−T 0(F)

S(F)

  • − γ
  • −

γ √nγ2

1

n

  • i=1
  • ψ′

Xi−T 0(F) S(F)

  • − γ1
  • + γγ2

γ2

1

1 √n

n

  • i=1

ϕT 0(Xi) +

  • γ

γ1 + γγ2x γ2

1 − γ1x

γ1

1 √n

n

  • i=1

ϕS(Xi) + oP(1),

slide-45
SLIDE 45

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Numerical study - confidence intervals

0.86 0.87 0.88 0.89 0.90 0.91 0.92 0.3 0.4 0.5 0.6 0.7 0.8

Normal distribution

Coverage Mean length NA_s

20 30 50 100

NA

20 30 50 100

Boot

20 30 50 100

RoB

20 30 50 100

X1 . . . N(0, 1)

slide-46
SLIDE 46

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Numerical study - confidence intervals

0.86 0.88 0.90 0.92 0.94 0.4 0.6 0.8 1.0 1.2 1.4

Symmetric contamination

Coverage Mean length NA_s

20 30 50 100

NA

20 30 50 100

Boot

20 30 50 100

RoB

20 30 50 100

X1 . . . 0.8 N(0, 1) + 0.2 N(0, 16)

slide-47
SLIDE 47

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Numerical study - confidence intervals

0.80 0.82 0.84 0.86 0.88 0.90 0.92 0.6 0.8 1.0 1.2 1.4 1.6

Asymmetric contamination

Coverage Mean length NA_s

20 30 50 100

NA

20 30 50 100

Boot

20 30 50 100

RoB

20 30 50 100

X1 . . . N(0, 1) + Exp(0, 2)

slide-48
SLIDE 48

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Numerical study - variance estimation

Methods:

  • 1. Boot – Bootstrap
  • 2. RoB – Winsorized Bootstrap
  • 3. NA – variance estimation based on the whole asymptotic expansion
  • 4. NA-s – variance estimation based on the whole asymptotic expansion

based on the symmetric expansion

slide-49
SLIDE 49

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Numerical study - variance estimation

Methods:

  • 1. Boot – Bootstrap
  • 2. RoB – Winsorized Bootstrap
  • 3. NA – variance estimation based on the whole asymptotic expansion
  • 4. NA-s – variance estimation based on the whole asymptotic expansion

based on the symmetric expansion Let σ2 be the true variance. We will use the following loss function L(ˆ σ) = E

  • log

ˆ

σ σ

2 .

slide-50
SLIDE 50

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Numerical study - variance estimation

20 40 60 80 100 0.00 0.05 0.10 0.15 0.20 0.25

Symmetric contamination

Sample size Loss NA_s NA Boot RoB

X1 . . . 0.8 N(0, 1) + 0.2 N(0, 16)

slide-51
SLIDE 51

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Numerical study - variance estimation

20 40 60 80 100 0.00 0.05 0.10 0.15

Asymmetric contamination

Sample size Loss NA_s NA Boot RoB

X1 . . . N(0, 1) + Exp(0, 2)

slide-52
SLIDE 52

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Conclusion

  • Simple normal approximation works well for nearly symmetric pop-

ulations, it is very stable, but it tends to underestimate the true variance.

slide-53
SLIDE 53

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Conclusion

  • Simple normal approximation works well for nearly symmetric pop-

ulations, it is very stable, but it tends to underestimate the true variance.

  • Normal approximation based on the whole expansions gives the right

coverage but it is not very stable and the confidence intervals are rather long

slide-54
SLIDE 54

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Conclusion

  • Simple normal approximation works well for nearly symmetric pop-

ulations, it is very stable, but it tends to underestimate the true variance.

  • Normal approximation based on the whole expansions gives the right

coverage but it is not very stable and the confidence intervals are rather long

  • Bootstrap can fail in the presence of outliers.
slide-55
SLIDE 55

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Conclusion

  • Simple normal approximation works well for nearly symmetric pop-

ulations, it is very stable, but it tends to underestimate the true variance.

  • Normal approximation based on the whole expansions gives the right

coverage but it is not very stable and the confidence intervals are rather long

  • Bootstrap can fail in the presence of outliers.
  • The performance of Winsorized bootstrap was satisfactory in all sit-

uations.

slide-56
SLIDE 56

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

Conclusion

  • Simple normal approximation works well for nearly symmetric pop-

ulations, it is very stable, but it tends to underestimate the true variance.

  • Normal approximation based on the whole expansions gives the right

coverage but it is not very stable and the confidence intervals are rather long

  • Bootstrap can fail in the presence of outliers.
  • The performance of Winsorized bootstrap was satisfactory in all sit-

uations. Thank you for your attention!

slide-57
SLIDE 57

EYSM 2005 Marek Omelka One Step Studentized M-Estimator

◭◭ ◮◮ ◭ ◮

Back Close

References

Andrews, D. F., Bickel, P., Hampel, F. R., Huber, P. J., Rogers, W. H., and Tukey, J. W. (1972). Robust Estimates of Location. Princeton University Press. Fernholz, L., T. (1983). Von Mises Calculus for Statistical Functional, volume 19 of Lecture Notes in Statistics. Springer, New York. Rousseeuw, P. J. and Croux, C. (1994). The bias of k-step M-estimators. Statistics and Probability Letters, 20:411–420. Singh, K. (1981). Breakdown theory for bootstrap quantiles. Ann. Statist., 26:1719–1732. van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes With Applications to Statistics. Springer, New York.