Tracking Predictable Drifting Parameters Paulo Serra of a Time - - PowerPoint PPT Presentation

▶
tracking predictable drifting parameters
SMART_READER_LITE
LIVE PREVIEW

Tracking Predictable Drifting Parameters Paulo Serra of a Time - - PowerPoint PPT Presentation

Oct 27, 2022 119 likes •351 views

Tracking Predictable Drifting Parameters of a Time Series Tracking Predictable Drifting Parameters Paulo Serra of a Time Series The Model Joint work with Eduard Belitser Results Examples of Parameter Variation Paulo Serra

slide-1
SLIDE 1

Tracking Predictable Drifting Parameters

  • f a Time

Series Paulo Serra The Model Results Examples of Parameter Variation Construction

  • f Gain

Functions Application: Quantile Tracking References

Tracking Predictable Drifting Parameters

  • f a Time Series

Joint work with Eduard Belitser Paulo Serra

Department of Mathematics and Computer Science Eindhoven University of Technology

12th September, 2013

1 / 22

slide-2
SLIDE 2

Tracking Predictable Drifting Parameters

  • f a Time

Series Paulo Serra The Model Results Examples of Parameter Variation Construction

  • f Gain

Functions Application: Quantile Tracking References

Model Description

Assume we observed Xn = (X0, X1, . . . , Xn) following: X0 ∼ P0, Xk|Xk−1 ∼ Pk(·|Xk−1), k ∈ N.

  • Xk takes values in X ⊆ Rl, l ∈ N (i.e. P(Xk ∈ X) = 1).
  • The distribution of Xn, n ∈ N0, is given by

P(n) = P(n)(xn) =

n

  • k=0

Pk(xk|xk−1), xk ∈ X k+1, where P0(x0|x−1) should be understood as P0(x0).

  • At time n ∈ N0, the underlying growing statistical model

is P(n) = n

k=0 Pk(xk|xk−1) : Pk(·|xk−1) ∈ Pk

  • .

2 / 22

slide-3
SLIDE 3

Tracking Predictable Drifting Parameters

  • f a Time

Series Paulo Serra The Model Results Examples of Parameter Variation Construction

  • f Gain

Functions Application: Quantile Tracking References

Objective

  • Consider a filtration {Fk}∞

k=−1 such that Fk ⊆ σ(Xk).

  • Consider a sequence of appropriately measurable operators

Ak, which map measures Pk(·|xk−1) ∈ Pk Ak(Pk(·|xk−1)) = θk(xk−1), xk−1 ∈ X k−1, with θk(xk−1) predictable with respect to Fk. Objective We would like to track θk = θk(Xk−1).

3 / 22

slide-4
SLIDE 4

Tracking Predictable Drifting Parameters

  • f a Time

Series Paulo Serra The Model Results Examples of Parameter Variation Construction

  • f Gain

Functions Application: Quantile Tracking References

Definition of the Algorithm

Assumption (A0) The drifting parameter satisfies P(θk(Xk−1) ∈ Θ) for some Θ such that supθ∈Θ θ2 ≤ CΘ. The following algorithm constitutes our tracking sequence. Tracking Algorithm Define ˆ θk+1 = ˆ θk + γkGk(ˆ θk, Xk), k ∈ N0 where 0 ≤ γk ≤ Γ and arbitrary F−1-measurable ˆ θ0 ∈ Θ ⊂ Rd. The functions Gk(ˆ θk, Xk) are called gain vectors.

4 / 22

slide-5
SLIDE 5

Tracking Predictable Drifting Parameters

  • f a Time

Series Paulo Serra The Model Results Examples of Parameter Variation Construction

  • f Gain

Functions Application: Quantile Tracking References

Assumptions on the Gain Function

Assumption (A1) For all k ∈ N0, constants λ1, λ2 and θk = Ak

  • Pk(·|xk−1)
  • ,

gk(ˆ θk, θk) = gk(ˆ θk, θk|Xk−1) = E

  • Gk(ˆ

θk, Xk)|Fk−1

  • ,

exists; for a Fk−1-measurable symmetric PD matrix Mk, a.s. gk(ˆ θk, θk|Xk−1)= −Mk(ˆ θk − θk), 0 < λ1 ≤ E[λ(1)(Mk)|Fk−2] ≤ λ(d)(Mk) ≤ λ2 < ∞. Assumption (A2) There exists a constant Cg > 0 such that EGk(ˆ θk, Xk) − gk(ˆ θk, θk|Xk−1)2 ≤ Cg, k ∈ N0.

5 / 22

slide-6
SLIDE 6

Tracking Predictable Drifting Parameters

  • f a Time

Series Paulo Serra The Model Results Examples of Parameter Variation Construction

  • f Gain

Functions Application: Quantile Tracking References

Main Results

L1 risk bound

Theorem (Bound on L1 risk) Let (A0) – (A2) hold and δk = δk(Xk−1) = ˆ θk − θk, k ∈ N0. Then for any k0, k ∈ N0 and sequence {γk, k ∈ N0} (satisfying the conditions of the previous lemma) such that γiλ2 ≤ 1, i ∈ {k0, . . . , k}, the following relation holds: Eδk+1 ≤C1 exp

  • − λ1

2

k

  • i=k0

γi

  • +C2
  • k
  • i=k0

γ2

i

1/2 + +C3 max

k0≤i≤k Eθi+1 − θk0,

k0 ≤ k, where C1 = √ 2( ¯ CΘ + CΘ)1/2, C2 = C1/2

g

(1 + λ2/λ1), C3 = (1 + λ2/λ1).

6 / 22

slide-7
SLIDE 7

Tracking Predictable Drifting Parameters

  • f a Time

Series Paulo Serra The Model Results Examples of Parameter Variation Construction

  • f Gain

Functions Application: Quantile Tracking References

Stronger Assumptions on the Gain Function

Assumption (A1) gk(ˆ θk, θk|Xk−1) = −Mk(ˆ θk − θk). 0 < λ1 ≤ E[λ(1)(Mk)|Fk−2] ≤ λ(d)(Mk) ≤ λ2 < ∞, (a.s.) ↓ 0 < λ1 ≤ λ(1)(Mk) ≤ λ(d)(Mk) ≤ λ2 < ∞, (a.s.) Assumption (A2) EGk(ˆ θk, Xk) − gk(ˆ θk, θk|Xk−1)2 ≤ Cg, k ∈ N0. ↓ Gk(ˆ θk, Xk)2 ≤ Cg, k ∈ N0, (a.s.)

7 / 22

slide-8
SLIDE 8

Tracking Predictable Drifting Parameters

  • f a Time

Series Paulo Serra The Model Results Examples of Parameter Variation Construction

  • f Gain

Functions Application: Quantile Tracking References

Main Results

Lp risk bound

Theorem (Lp risk bound) Suppose that the conditions of the previous theorem are

  • fulfilled. If, in addition (to assumption (A1)), λ(1)(Mi) ≥ λ1

and Gi(ˆ θi, Xi) ≤ Cg (instead of (A2)) a.s. for all i = k0 . . . , k, then for any p ≥ 1 Eδk+1p

p ≤C′ 1 exp

  • −pλ1

k

  • i=k0

γi

  • +C′

2

  • k
  • i=k0

γ2

i

p/2 +C′

3 max k0≤i≤k Eθi+1 − θk0p p,

k0 ≤ k, for C′

1 = 3p−1Kp pEδk0p p, C′ 2 = 3p−12pdBpCp g

  • 1 + K2

pλ2/λ1

p, C′

3 = 3p−1

1 + K2

pλ2/λ1

p, and Kp and Bp are constants.

8 / 22

slide-9
SLIDE 9

Tracking Predictable Drifting Parameters

  • f a Time

Series Paulo Serra The Model Results Examples of Parameter Variation Construction

  • f Gain

Functions Application: Quantile Tracking References

Lipschitz Signal: θn

k = ϑ(k/n),

ϑ(·) ∈ Lβ

Eδk+1 exp

  • − λ1

2

k

  • i=k0

γi

  • +
  • k
  • i=k0

γ2

i

1/2 + max

k0≤i≤kEθi+1−θk0,

Eδk+1p

p exp

  • −pλ1

k

  • i=k0

γi

  • +
  • k
  • i=k0

γ2

i

p/2 + max

k0≤i≤kEθi+1−θk0p p,

Xn

0 ∼ Pθn

0 ,

Xn

k |Xn k−1 ∼ Pθn

k (·|Xn

k−1),

k ≤ n ∈ N,

  • Assume that θn

k = ϑ(k/n), with ϑ(·) ∈ Lβ, k = 1, . . . , n.

  • For 0 < β ≤ 1, γk ≡ Cγ(log n)(2β−1)/(2β+1)n−2β/(2β+1),

k0 = Kn = (log n)2/(2β+1)n2β/(2β+1) we get sup

ϑ∈Lβ k≥Kn

En−

β 2β+1 δk

(log n)

2β 2β+1

≤ C and sup

ϑ∈Lβ k≥Kn

E n−

β 2β+1 δkp

(log n)

2β 2β+1

p ≤ C.

9 / 22

slide-10
SLIDE 10

Tracking Predictable Drifting Parameters

  • f a Time

Series Paulo Serra The Model Results Examples of Parameter Variation Construction

  • f Gain

Functions Application: Quantile Tracking References

Example 1

Signal + noise setting

The model is: Xk = θk + ξk, k ∈ N0, where {θk}k∈N0 is a predictable process (θk = θk(Xk−1)), {ξk}k∈N0 is a martingale difference noise with respect to the filtration {Fk}k∈N−1. We can simply take the following gain function Gk(ˆ θk, Xk) = −(ˆ θk − Xk), k ∈ N0, since gk(ˆ θk, θk|Xk−1) = E[Gk(ˆ θk, Xk)|Xk−1] = −(ˆ θk−θk), k ∈ N0.

10 / 22

slide-11
SLIDE 11

Tracking Predictable Drifting Parameters

  • f a Time

Series Paulo Serra The Model Results Examples of Parameter Variation Construction

  • f Gain

Functions Application: Quantile Tracking References

General Gain Construction

  • Assume each measure in Pk = {Pθ(x|Xk−1), θ ∈ Θ ⊂ Rd}

has a density pθ(x|Xk−1), θ ∈ Θ, with respect to some σ-finite dominating measure.

  • Assume also that there is a common support X for these

densities, and that for any x ∈ X, xk−1 ∈ X k−1, and θ ∈ Θ, the partial derivatives ∂pθ(x|xk−1)/∂θi, i = 1, . . . , d, exist and are finite.

  • Let ∇θ log pθ(x|xk−1) be a gradient.

11 / 22

slide-12
SLIDE 12

Tracking Predictable Drifting Parameters

  • f a Time

Series Paulo Serra The Model Results Examples of Parameter Variation Construction

  • f Gain

Functions Application: Quantile Tracking References

General Gain Construction

We can use the gains Gk(ϑ, xk) = ∇ϑ log pϑ(xk|xk−1). If expectation and differentiation can be interchanged, then gk(ϑ, θ|Xk−1)=Eθ

  • ∇ϑ log pϑ(Xk|Xk−1)
  • Xk−1
  • = ∇ϑEθ
  • log pϑ(Xk|Xk−1)
  • Xk−1
  • = −∇ϑKL
  • Pθ(·|Xk−1), Pϑ(·|Xk−1)
  • .

12 / 22

slide-13
SLIDE 13

Tracking Predictable Drifting Parameters

  • f a Time

Series Paulo Serra The Model Results Examples of Parameter Variation Construction

  • f Gain

Functions Application: Quantile Tracking References

Motivation

  • In quantile regression we estimate a conditional quantile

(rather than conditional expectation) of one random variable given another.

  • More robust than regression: no moment assumptions.
  • Estimating multiple quantiles simultaneously gives more

comprehensive picture of the distribution.

  • Relevant in econometrics, social sciences and ecology.
  • Relation between response variable and the measured

predictors might be complex and not be captured by the conditional expectation.

13 / 22

slide-14
SLIDE 14

Tracking Predictable Drifting Parameters

  • f a Time

Series Paulo Serra The Model Results Examples of Parameter Variation Construction

  • f Gain

Functions Application: Quantile Tracking References

Notion of Depth

  • Multidimensional analogues of the median, center points
  • f a distribution, can be defined using a depth function.
  • Given P with support in Rd, the depth of x ∈ Rd with

respect to P, DF(x, P), measures the centrality of x in P.

  • Depth functions should give a P-based, center-outward
  • rdering of the points x ∈ Rd via contours of the function

x → DF(x, P).

  • Points of maxima of the depth function DF(x, P) will be

the “most central” points of the distribution P.

14 / 22

slide-15
SLIDE 15

Tracking Predictable Drifting Parameters

  • f a Time

Series Paulo Serra The Model Results Examples of Parameter Variation Construction

  • f Gain

Functions Application: Quantile Tracking References

Properties of the Half-Space Depth

  • Tukey’s notion of depth, the half-space depth, is defined as

DF(x, P) = inf

  • P
  • H
  • : H is a closed half-space, x ∈ H
  • .
  • In one dimension, points of maximum half-space depth

are, by definition, medians.

  • The half-space depth has attractive properties, namely:
  • it is invariant under affine transformations;
  • for distributions with a natural notion of center, it attains

its maximum at this center;

  • it decays monotonically relative to its deepest point;
  • it vanishes at infinity.

15 / 22

slide-16
SLIDE 16

Tracking Predictable Drifting Parameters

  • f a Time

Series Paulo Serra The Model Results Examples of Parameter Variation Construction

  • f Gain

Functions Application: Quantile Tracking References

Half-Space Symmetrical Distributions

Quantiles and spatial medians

  • We work with distributions for which there is a proper

notion of center.

  • Absolutely continuous distributions P which are half-space

symmetric about θ: θ ∈ H ⇒ P(H) ≥ 1/2, H is a closed half-space.

  • In one dimension, for α ∈ [0, 1/2),

θ(α) = inf{x ∈ R : DF(x, P) ≥ α}, θ(1 − α) = sup{x ∈ R : DF(x, P) ≥ α},

  • In d dimensions, d ≥ 1,

θ(1/2) = {x ∈ Rd : DF(x, P) ≥ 1/2}.

16 / 22

slide-17
SLIDE 17

Tracking Predictable Drifting Parameters

  • f a Time

Series Paulo Serra The Model Results Examples of Parameter Variation Construction

  • f Gain

Functions Application: Quantile Tracking References

Model

Assume we observe Xn = (X0, X1, . . . , Xn) according to: X0 ∼ P0, Xk|Xk−1 ∼ Pk(·|Xk−1), k ∈ N, where these conditional measures have support X ⊂ Rd. Objective Given αk ∈ (0, 1), k ∈ N, we would like to track θk(Xk−1, αk). If d = 1 we track θk(Xk−1, αk), the conditional quantile of level αk of Pk(·|Xk−1). If d ≥ 2 we fix αk = 1/2, k ∈ N, and track θk(Xk−1, 1/2), the conditional spatial median of Pk(·|Xk−1).

17 / 22

slide-18
SLIDE 18

Tracking Predictable Drifting Parameters

  • f a Time

Series Paulo Serra The Model Results Examples of Parameter Variation Construction

  • f Gain

Functions Application: Quantile Tracking References

Assumptions

Let H(x, w) be the half-space containing x + w, delimited by the hyper-plane which contains x and is perpendicular to w. Assumption (B1) For b, B, δ > 0, any ǫ ∈ (0, δ], any unit vectors v, w ∈ Rd: b ≤ Pk

  • H(θk − ǫv, w)
  • xk−1
  • − αk

vT w ǫ ≤ B, xk ∈ X k, where k ∈ N, θk = θk(xk−1, αk) as before. Assumption (B2) The support X is a compact set, such that for all x ∈ X, x ≤ CX and the conditional spatial quantiles θk take values in some convex subset Θ ⊆ X with θ ≤ CΘ, θ ∈ Θ.

18 / 22

slide-19
SLIDE 19

Tracking Predictable Drifting Parameters

  • f a Time

Series Paulo Serra The Model Results Examples of Parameter Variation Construction

  • f Gain

Functions Application: Quantile Tracking References

Definition of the Gains and Algorithms

Proposed Gains

  • For d = 1 u ∈ X, v ∈ R and α ∈ (0, 1)

R(u, v, α) = α − I

  • u ≤ v
  • ,

k ∈ N.

  • For d ≥ 2, u ∈ X ⊂ Rd, v ∈ Rd and w ∈ Rd a unit vector

S(u, v, w) = w

  • I
  • u ∈ H(v, w)
  • − 1/2
  • ,

k ∈ N. Let B = {e1, . . . , ed} be an orthonormal basis for Rd. Call D such that P(D = ei) = 1/d, i = 1, . . . , d, a random direction.

19 / 22

slide-20
SLIDE 20

Tracking Predictable Drifting Parameters

  • f a Time

Series Paulo Serra The Model Results Examples of Parameter Variation Construction

  • f Gain

Functions Application: Quantile Tracking References

Definition of the Gains and Algorithms

Proposed Gains

  • For d = 1 u ∈ X, v ∈ R and α ∈ (0, 1)

R(u, v, α) = α − I

  • u ≤ v
  • ,

k ∈ N.

  • For d ≥ 2, u ∈ X ⊂ Rd, v ∈ Rd and w ∈ Rd a unit vector

S(u, v, w) = w

  • I
  • u ∈ H(v, w)
  • − 1/2
  • ,

k ∈ N. Proposed Algorithms

  • For d = 1, αk ∈ (0, 1), k ∈ N,

ˆ θk+1 = ˆ θk + γkR(Xk, ˆ θk, αk), ˆ θ1 ∈ Θ, k ∈ N,

  • For d ≥ 2 and Dk a sequence of i.i.d. random directions

ˆ θk+1 = ˆ θk + γkS(Xk, ˆ θk, Dk), ˆ θ1 ∈ Θ, k ∈ N,

20 / 22

slide-21
SLIDE 21

Tracking Predictable Drifting Parameters

  • f a Time

Series Paulo Serra The Model Results Examples of Parameter Variation Construction

  • f Gain

Functions Application: Quantile Tracking References

Preliminary Results

Verifying assumption (A1) and assumption (A2)

Lemma (Representation d = 1) A.s. R(Xk, ˆ θk, αk) ≤ 1. If (B1) and (B2) hold, then E

  • R(Xk, ˆ

θk, αk)

  • Fk−1
  • = −Mk(ˆ

θk − θk), k ∈ N, with Fk−1 = σ(Dk−1, Xk−1), for Fk−1-measurable random variables Mk such that a.s. 0 < λ1 ≤ Mk ≤ λ2 < ∞. Lemma (Representation d ≥ 2) A.s. S(Xk, ˆ θk, Dk) ≤ 1/2. If (B1) and (B2) hold, then E

  • S(Xk, ˆ

θk, Dk)

  • Fk−1
  • = −Mk(ˆ

θk − θk), k ∈ N, with Fk−1 = σ(Dk−1, Xk−1), for Fk−1-measurable matrices Mk such that a.s. 0 < λ1 ≤ λ(1)(Mk) ≤ λ(d)(Mk) ≤ λ2 < ∞.

21 / 22

slide-22
SLIDE 22

Tracking Predictable Drifting Parameters

  • f a Time

Series Paulo Serra The Model Results Examples of Parameter Variation Construction

  • f Gain

Functions Application: Quantile Tracking References

References

References

  • G. Bassett and R. Koenker, Asymptotic theory of least absolute error regression, JASA,

73(363):618–622, 1978.

  • E. Belitser and P. Serra, On properties of the algorithm for pursuing a drifting quantile,

Automation and Remote Control, 74(4):613–627, 2013.

  • E. Belitser and P. Serra, Online Tracking of a Predictable Drifting Parameter of a Time Series,

arXiv:1306.0325 [math.ST], 2013.

  • E. Belitser and P. Serra, Tracking of a Conditional Spatial Median, pre-print, 2013.
  • R. Koenker, Quantile Regression, Cambridge University Press, 2005.
  • H. J. Kushner and G. Yin, Stochastic Approximation and Recursive Algorithms and Applications,

Berlin and New York: Springer-Verlag, 2003.

  • H. Robbins and S. Monro, A stochastic approximation method, The Annals of Mathematical

Statistics, 22(3):400–407, 1951.

  • J. W. Tukey, Mathematics and the picturing of data, Proceedings of the international congress of

mathematicians, volume 2, pages 523–531, 1975.

  • M. T. Wasan, Stochastic Approximation, Cambridge University Press, 1969.
  • Y. Zuo and R. Serfling, General notions of statistical depth function, Annals of Statistics, pages

461–482, 2000. 22 / 22