Ultra-high dimensional statistics and statistical learning on some - - PowerPoint PPT Presentation

▶
ultra high dimensional statistics and statistical
SMART_READER_LITE
LIVE PREVIEW

Ultra-high dimensional statistics and statistical learning on some - - PowerPoint PPT Presentation

Dec 21, 2023 316 likes •1.19k views

Examples Linear model Mathematical ingredients LOL algo Ultra-high dimensional statistics and statistical learning on some applications Dominique Picard Universit e Paris-Diderot Laboratoire Probabilit es et Mod` eles Al eatoires

slide-1
SLIDE 1

Examples Linear model Mathematical ingredients LOL algo

Ultra-high dimensional statistics and statistical learning on some applications

Dominique Picard

Universit´ e Paris-Diderot Laboratoire Probabilit´ es et Mod` eles Al´ eatoires

M2MO : 20 ans !

slide-2
SLIDE 2

Examples Linear model Mathematical ingredients LOL algo

Plan

Examples Linear model Mathematical ingredients LOL algo

slide-3
SLIDE 3

Examples Linear model Mathematical ingredients LOL algo

Example 1 : prediction of electrical consumption

5 10 15 20 25 30 35 40 45 50 7 7.2 7.4 7.6 7.8 8 8.2 8.4 8.6 x 10

4

(354) 20071220 with 12 coeff (Group 9) 0.0010%

  • riginal

Model

Figure: Signal- Prediction

  • M. Mougeot, K. Tribouley, Laurence Maillard, V. Lefieux, D.P.
slide-4
SLIDE 4

Examples Linear model Mathematical ingredients LOL algo

Examples of days (worst)

(2009 07 14)

5 10 15 20 25 30 35 40 45 50 3.2 3.4 3.6 3.8 4 4.2 4.4 4.6 x 10

4

20090714 (2387) mape 0.06769

  • riginal

Model

slide-5
SLIDE 5

Examples Linear model Mathematical ingredients LOL algo

Example 3 : genomic

slide-6
SLIDE 6

Examples Linear model Mathematical ingredients LOL algo

Example 4 : Estimate a probability density on the sphere

slide-7
SLIDE 7

Examples Linear model Mathematical ingredients LOL algo

Example 5 : CMB

slide-8
SLIDE 8

Examples Linear model Mathematical ingredients LOL algo

C M B : mask

slide-9
SLIDE 9

Examples Linear model Mathematical ingredients LOL algo

High frequency signal FBUND

500 1000 1500 2000 2500 122.6 122.65 122.7 122.75 122.8 122.85 122.9 122.95 123 123.05 123.1 FBund 20091207 Trading time

  • E. Bacry
slide-10
SLIDE 10

Examples Linear model Mathematical ingredients LOL algo

High frequency signal FBUND

500 1000 1500 2000 2500 122.9 123 123.1 123.2 123.3 123.4 123.5 123.6 123.7 123.8 123.9 FBund 20091208 Trading time

  • E. Bacry
slide-11
SLIDE 11

Examples Linear model Mathematical ingredients LOL algo

High frequency signal FBOBL

500 1000 1500 2000 2500 115.95 116 116.05 116.1 116.15 116.2 116.25 116.3 FBobl 20091207 Trading time

  • E. Bacry
slide-12
SLIDE 12

Examples Linear model Mathematical ingredients LOL algo

High frequency signal FBOBL

500 1000 1500 2000 2500 116.2 116.3 116.4 116.5 116.6 116.7 116.8 116.9 117 FBobl 20091208 Trading time

  • E. Bacry
slide-13
SLIDE 13

Examples Linear model Mathematical ingredients LOL algo

Model

slide-14
SLIDE 14

Examples Linear model Mathematical ingredients LOL algo

Linear Model

Observation : Y = (Y1, . . . , Yn)t Y = Φα + ǫ α ∈ I Rp is the unknown parameter (to be estimated)

  • ǫ = (ǫ1, . . . , ǫn)t is a (non observed) vector of random errors.

It is assumed to be variables i.i.d. N(0, σ2)

  • Φ is a known matrix n × p.

High dimension : p >> n

slide-15
SLIDE 15

Examples Linear model Mathematical ingredients LOL algo

Example : genomic

Y =      1 . . . 1      Φ =

slide-16
SLIDE 16

Examples Linear model Mathematical ingredients LOL algo

  • Large random matrices : Φ is composed of n × p random

variables i.i.d. N(0, 1).

slide-17
SLIDE 17

Examples Linear model Mathematical ingredients LOL algo

Signal denoising

Y =

500 1000 1500 2000 2500 122.6 122.65 122.7 122.75 122.8 122.85 122.9 122.95 123 123.05 123.1 FBund 20091207 Trading time

What is Φ in this case ?

slide-18
SLIDE 18

Examples Linear model Mathematical ingredients LOL algo

  • Statistical learning, regression estimation

Yi = f (Xi) + ǫi + ui, i = 1 . . . n

  • ǫ′

is are i.i.d. N(0, 1).

  • ui’s possibly random, not necessarily random nor iid but

’small’.

  • X ′

i ’s random i.i.d. taking values in a compact set of Rd.

  • f is the parameter to be estimated.
slide-19
SLIDE 19

Examples Linear model Mathematical ingredients LOL algo

To embed this problem in a linear model, we consider a dictionary D of size p, of real functions defined on I

  • Rd. We assume that f

can be ’reasonably’ well approached by the dictionary functions {g ∈ D} : i.e. there exists αg tel que f =

  • g∈D

αgg + h where h is ’small’. Then the model writes Yi =

  • g∈D

αgg(Xi) + h(Xi) + ǫi, i = 1, . . . , n Y = Φα + u + ǫ if we put ui = h(Xi) pour i = 1, . . . , n et Φ being the matrix with general terms Φiℓ = gℓ(Xi)

slide-20
SLIDE 20

Examples Linear model Mathematical ingredients LOL algo

Associated problems

Y = Φα + u + ǫ n observations : Y (n × 1), Φ (n × p) ◮ Estimation : determine ˆ α ◮ Selection : Find the significant coefficients ˆ α∗ = ˆ α1|ˆ

α|>T

◮ Predict : ˆ Y = Φˆ α

slide-21
SLIDE 21

Examples Linear model Mathematical ingredients LOL algo

Conditions generally required to solve the problem

  • ’sparsity’ of the vector α
  • good approximation of the ’true function’ by the dictionary
  • ’Coherence’ conditions on the matrix Φ
slide-22
SLIDE 22

Examples Linear model Mathematical ingredients LOL algo

Approximation

ui = h(Xi) = f (Xi) −

  • g∈D

αgg(Xi) Asking the ui’s to be small means that f est well approximated by a linear combination of the dictionnary

slide-23
SLIDE 23

Examples Linear model Mathematical ingredients LOL algo

Sparsity conditions : what does it means to be sparse ?

slide-24
SLIDE 24

Examples Linear model Mathematical ingredients LOL algo

Sparsity conditions

  • {αℓ}ℓ≤p S sparse
slide-25
SLIDE 25

Examples Linear model Mathematical ingredients LOL algo

Sparsity conditions

  • {αℓ}ℓ≤p S sparse
  • Strict sparsity

# {ℓ ∈ {1, . . . , p}, |αℓ| = 0} ≤ S

slide-26
SLIDE 26

Examples Linear model Mathematical ingredients LOL algo

Sparsity conditions

  • {αℓ}ℓ≤p S sparse
  • Strict sparsity

# {ℓ ∈ {1, . . . , p}, |αℓ| = 0} ≤ S

  • more generally
  • ℓ

|αℓ|q ≤ M, 0 < q < 1

slide-27
SLIDE 27

Examples Linear model Mathematical ingredients LOL algo

The dictionary problem

Of course sparsity is linked with the dictionary.

  • Fourier Basis
  • Wavelet basis
  • Needlets
  • Combination of ’bases’
slide-28
SLIDE 28

Examples Linear model Mathematical ingredients LOL algo

Fourier basis

5 10 15 20 25 30 35 40 45 50 −1 −0.5 0.5 1 1.5 Dictionary func 1 5 10 15 20 25 30 35 40 45 50 −0.25 −0.2 −0.15 −0.1 −0.05 0.05 0.1 0.15 0.2 0.25 Dictionary func 4 5 10 15 20 25 30 35 40 45 50 −0.25 −0.2 −0.15 −0.1 −0.05 0.05 0.1 0.15 0.2 0.25 Dictionary func 30

slide-29
SLIDE 29

Examples Linear model Mathematical ingredients LOL algo

Haar wavelets

50 −0.2 0.2 Dictionary func 48 50 −0.5 0.5 Dictionary func 49 50 −0.5 0.5 Dictionary func 50 50 −0.5 0.5 Dictionary func 51 50 −0.5 0.5 Dictionary func 52 50 −0.5 0.5 Dictionary func 53 50 −0.5 0.5 Dictionary func 54 50 −0.5 0.5 Dictionary func 55 50 −0.5 0.5 Dictionary func 56 50 −0.5 0.5 Dictionary func 57 50 −0.5 0.5 Dictionary func 58 50 −0.5 0.5 Dictionary func 59 −0.5 0.5 Dictionary func 60 −0.5 0.5 Dictionary func 61 −0.5 0.5 Dictionary func 62

slide-30
SLIDE 30

Examples Linear model Mathematical ingredients LOL algo

Functions defined on the sphere

slide-31
SLIDE 31

Examples Linear model Mathematical ingredients LOL algo

Spherical Harmonics on the sphere

0.5 1 1.5 2 2.5 3 3.5 1 2 3 4 5 6 7 −0.8 −0.6 −0.4 −0.2 0.2 0.4 0.6 THETA PHI

slide-32
SLIDE 32

Examples Linear model Mathematical ingredients LOL algo

Needlets on the sphere (Petrushev-co-authors)

0.5 1 1.5 2 2.5 3 3.5 2 4 6 8 −0.5 0.5 1 1.5 THETA PHI

slide-33
SLIDE 33

Examples Linear model Mathematical ingredients LOL algo

Needlets associated to Jacobi polynomials on [0,1] (Petrushev-co-authors)

−20 20 40 60 −2 2 4 6 −1 1 2 3 −2 2 4 6

slide-34
SLIDE 34

Examples Linear model Mathematical ingredients LOL algo

Sparsity conditions and functional approximation spaces

In the wavelet, needlet cases, Besov spaces are especially adapted to reflect sparsity conditions. More complex : How to translate in terms of spaces sparsity conditions for combinations of bases ?

Petrushev, Narkowitch, Ward, Xu, Kyriasis ; Coulon, Kerkyacharian, Petrushev

slide-35
SLIDE 35

Examples Linear model Mathematical ingredients LOL algo

Conditions generally required to solve the problem

  • ’Sparsity’ sur le vecteur α
  • good approximation of the ’true function’ by the dictionary
  • ’Coherence’ conditions on the matrix Φ
slide-36
SLIDE 36

Examples Linear model Mathematical ingredients LOL algo

RIP- Coherence

The raws of Φ are supposed to be normalized For C ⊂ {1, . . . p}, denote ΦC the matrix Φ restricted to the raws which are in C and the associated Gram-matrix M(C) := 1 nΦt

CΦC

RIP(m0, ν) assumes that M(C) is almost diagonal for any C as soon as #(C) ≤ m0, in the following sense : There exist 0 ≤ ν < 1 and m0 ≥ 1 such that : ∀x ∈ I Rm, x2

l2(m)(1 − ν) ≤ xtM(C)x ≤ x2 l2(m)(1 + ν),

slide-37
SLIDE 37

Examples Linear model Mathematical ingredients LOL algo

Coherence.

  • Introduce the p × p Gram matrix :

M := 1 nΦtΦ. and the Coherence τn = sup

ℓ=m

|Mℓm| = sup

ℓ=m

|1 n

n

  • i=1

ΦiℓΦim| Coherence = ⇒ RIP(⌊ν/τn⌋, ν)

slide-38
SLIDE 38

Examples Linear model Mathematical ingredients LOL algo

Coherence/Illustration

Φ : i.i.d.N(0, 1)

1.5 2 2.5 3 3.5 4 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 log10(n) coh p10000 p1000 p100

slide-39
SLIDE 39

Examples Linear model Mathematical ingredients LOL algo

Coherence/ Illustration

Φ : i.i.d.N(0, 1), U(-1,1), , B{−1, 1}

1.5 2 2.5 3 3.5 4 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 log10(n) coh G U B p100 p1000 p5000

slide-40
SLIDE 40

Examples Linear model Mathematical ingredients LOL algo

Coherence τn versus

  • log p/n

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

slide-41
SLIDE 41

Examples Linear model Mathematical ingredients LOL algo

Dictionaries of bases

−0.1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.2 0.4 0.6 0.8 1

Figure: Empirical Distribution of the correlations (absolute value) for each dictionary. D1-blue dashed line, D2-red dashed line (D1, D2 confounded), D3-magenta dotted line, D4-green line, D5-cyan dashed line.

slide-42
SLIDE 42

Examples Linear model Mathematical ingredients LOL algo

Statistical methods

slide-43
SLIDE 43

Examples Linear model Mathematical ingredients LOL algo

Low-dimensional linear model

  • more observations than variables n >> p
  • and weak collinearity between co variables,

ΦTΦ invertible       y1 y2 yn       =       Φ11 .. Φ1p Φn1 .. Φnp       ∗     α1 α2 αp     + ǫ ”Thin matrix” → Unique Solution : ˆ α = (ΦT Φ)−1ΦTY

slide-44
SLIDE 44

Examples Linear model Mathematical ingredients LOL algo

High dimensional linear model

  • More variables than observations n << p
  • or Strong Collinearity (p < n)ΦTΦ,

NOT invertible     y1 y2 yn     =     Φ11 . . . . . . Φ1p Φn1 . . . Φnp     ∗         α1 α2 . . . . . . αp         + ǫ ”Fat matrix” → Pseudo inverse : infinity of ˆ α solutions. → Need for more assumptions on α to solve the problem

slide-45
SLIDE 45

Examples Linear model Mathematical ingredients LOL algo

Penalizations

Penalization are introduced in the regression framework (constraints on α)

  • Ridge : E(α, λ) = ||Y − Φα||2 + λΣjα2

j

  • Lasso : E(α, λ) = ||Y − Φα||2 + λΣj|αj|
  • Scad : E(α, λ) = ||Y − Φα||2 + λΣjwjg(αj)

Solutions based on : → Convex Optimization for L1, non convex Opti. for Scad Fan & Lv (2008, 2010), Candes & Tao (2007) ... → Non sparse solutions for ridge regression

slide-46
SLIDE 46

Examples Linear model Mathematical ingredients LOL algo

Alternative :Learning Out of Leaders

  • Very simple Procedure with low complexity
  • Powerful in ultra high dimension, p < exp(c′n)
  • Powerful for small coherence and sparsity
  • Competitive with more sophisticated penalization procedures,

SIS-Lasso, lasso-Reg,... (cf Simulations)

  • Competitive even in case of dependency (cf Simulations)
  • M. Mougeot, K. Tribouley, D.P.
slide-47
SLIDE 47

Examples Linear model Mathematical ingredients LOL algo

2-steps thresholding : LOL

slide-48
SLIDE 48

Examples Linear model Mathematical ingredients LOL algo

LOL Procedure

Y = Φα + u + ǫ Y (n × 1), Φ (n × p) S non zero coefficients α step compute size

  • 1. SELECTION

Find b Leaders Φb (n, b) b < n << p

  • 2. REGRESSION
  • n Leaders

˜ α = (ΦT

b Φb)−1ΦT b Y

(1, b)

  • 3. THRESHOLD

the coefficients ˆ α (1, ˆ S)

slide-49
SLIDE 49

Examples Linear model Mathematical ingredients LOL algo

LOL - step1

SELECTION Find the Leaders Φ → Φb (n, b) based on a ”correlation” search and thresholding : Kℓ = |1 n

n

  • i=1

ΦiℓYi| ∀ℓ, 1 ≤ ℓ ≤ p

  • Find the set B = {ℓ, Kℓ ≥ λ1}.
  • Threshold,

λ1 = T1

  • log p

n ,

T1 : constant (σ, ν, M, c0)

  • Data driven choice of λ1 for practical applications (LOLA)
slide-50
SLIDE 50

Examples Linear model Mathematical ingredients LOL algo

LOL, step 1 : Select the Leaders

Some details...

1 n < Y , Φ.ℓ >

= 1

n < Σk=1...pΦ.kαk + u + ǫ, Φ.ℓ >

slide-51
SLIDE 51

Examples Linear model Mathematical ingredients LOL algo

LOL, step 1 : Select the Leaders

Some details...

1 n < Y , Φ.ℓ >

= 1

n < Σk=1...pΦ.kαk + u + ǫ, Φ.ℓ > 1 n < Y , Φ.ℓ >= 1 nαℓ < Φ.ℓ, Φ.ℓ >

αℓ + 1

nΣk=ℓ αk < Φ.k, Φ.ℓ > 1 nΣk=ℓαk < Φ.k, Φ.ℓ >

(αk = 0) + 1

n < u + ǫ, Φ.ℓ > 1 n < u + ǫ, Φ.ℓ > (small)

Function of Coherence τn and Sparsity S.

slide-52
SLIDE 52

Examples Linear model Mathematical ingredients LOL algo

LOL, step 1 : Select the Leaders

B = {ℓ, Kl ≥ λ1}, with λ1 = T1

  • log p

n

Φ i.i.d. N(0, 1), S = 10

100 200 300 400 500 600 700 800 900 1000 0.5 1 1.5 2 2.5 3 3.5 4

λ1

n=250, S=10, b=2.0,

n = 250, p = 1000 → ρ = S

n = 0.025, δ = 1 − n p = 0.75

slide-53
SLIDE 53

Examples Linear model Mathematical ingredients LOL algo

LOL - step 2

Y = Φα + u + ǫ, Y (n × 1), Φ (n × p), S non zero coefficients α SELECTION Find b Leaders Φb REGRESSION

  • n Leaders

˜ α = (Φt

bΦb)−1Φt bY

THRESHOLD the coefficients ˆ α

slide-54
SLIDE 54

Examples Linear model Mathematical ingredients LOL algo

LOL, step 2 : Regress on leaders

  • Consider the pseudo-regression model :

Yi =

  • ℓ∈B

Φiℓαℓ + ei . The extracted matrix ΦB is : (ΦB)ℓ, i = Φiℓ for any ℓ ∈ B and i ∈ {1, . . . , n}. (1)

  • ˆ

α(B) = Arg min

α=(αℓ)ℓ∈B

n

  • i=1

(Yi −

  • ℓ∈B

Φiℓαℓ)2

  • = (Φt

BΦB)−1Φt B Y .

  • α(B) = (

αℓ(B), ℓ ∈ B) is the minimum least square error in the pseudo- model :

slide-55
SLIDE 55

Examples Linear model Mathematical ingredients LOL algo

LOL - step 3

Y = Φα + u + ǫ, Y (n × 1), Φ (n × p), S non zero coefficients α step compute size SELECTION Find b Leaders Φb (1, b) REGRESSION

  • n Leaders

˜ α = (Φt

bΦb)−1ΦdY

(1, b) THRESHOLD the coefficients ˆ α (1, ˆ S)

slide-56
SLIDE 56

Examples Linear model Mathematical ingredients LOL algo

LOL, step 3 : Threshold

  • Threshold (again like step 1) the estimated coefficients to
  • btain the final predictor

ˆ α∗

ℓ =

αℓ I{| αℓ| ≥λ2}

  • Threshold λ2 = T2
  • log p

n

,

  • For some constant T2 > 0, T2(σ, ν, M, c0)
  • Data driven choice of λ1, λ2 for pratical applications (LOLA)
slide-57
SLIDE 57

Examples Linear model Mathematical ingredients LOL algo

Step2 to remind : ”The leaders are”

B = {ℓ, Kl ≥ λ1}, with λ1 = T1

  • log p

n

Φ i.i.d. N(0, 1), α∗ ∼ N(2, 1), S = 10

100 200 300 400 500 600 700 800 900 1000 0.5 1 1.5 2 2.5 3 3.5 4

λ1

n=250, S=10, b=2.0,

n = 250, p = 1000 → ρ = S

n = 0.025, δ = 1 − n p = 0.75

slide-58
SLIDE 58

Examples Linear model Mathematical ingredients LOL algo

LOL(3) : Threshold

Ex1 : Φ i.i.d. N(0, 1), α∗ = N(2, 1),S = 10,

(Leaders b = 170)

20 40 60 80 100 120 140 160 180 −3 −2 −1 1 2 3 4 5 n=250, S=10, p=1000,b=2,SNR=5

n = 250,p = 1000 → ρ = 0.04, δ = 0.75

slide-59
SLIDE 59

Examples Linear model Mathematical ingredients LOL algo

LOL(3) : Threshold

Ex1 : Φ i.i.d. N(0, 1), α∗ = N(2, 1),S = 10,

(Leaders b = 170)

20 40 60 80 100 120 140 160 180 −3 −2 −1 1 2 3 4 5

λ2 λ2

n=250, S=10, p=1000,b=2,SNR=5

slide-60
SLIDE 60

Examples Linear model Mathematical ingredients LOL algo

LOL(3) : Threshold

Ex2 : Φ i.i.d. N(0, 1), α∗ = N(2, 1), S = 20, ρ = 0.08, δ = 0.75

20 40 60 80 100 120 140 160 180 −4 −3 −2 −1 1 2 3 4

λ2 λ2

n=250, S=20, p=1000,b=2,SNR=5

slide-61
SLIDE 61

Examples Linear model Mathematical ingredients LOL algo

Loss Function

d(ˆ α∗, α)2 = 1

n

n

i=1

  • Yi − EYi

2 = 1

n

n

i=1

p

ℓ=1(ˆ

α∗

ℓ − αℓ)Φiℓ + ui

2 where Yi = Φi•α + ui + ǫi (model)

  • α ∈ Rp true coefficient,
  • Φi• : i−th observation of the model, (i−th line of Φ)
  • Yi = Φi•ˆ

α∗ (prediction)

  • ˆ

α∗ prediction computed by LOL. observation.

slide-62
SLIDE 62

Examples Linear model Mathematical ingredients LOL algo

LOL assumptions

  • Sparsity :

for q ∈ (0, 1], Bq(M) := {α ∈ Rp, αlq(p) ≤ M}

slide-63
SLIDE 63

Examples Linear model Mathematical ingredients LOL algo

LOL assumptions

  • Sparsity :

for q ∈ (0, 1], Bq(M) := {α ∈ Rp, αlq(p) ≤ M} Particularly, for q = 0 : B0(S, M) := {α ∈ Rp,

p

  • j=1

I{|αj = 0} ≤ S, αl1(p) ≤ M}.

slide-64
SLIDE 64

Examples Linear model Mathematical ingredients LOL algo

LOL assumptions

  • Sparsity :

for q ∈ (0, 1], Bq(M) := {α ∈ Rp, αlq(p) ≤ M} Particularly, for q = 0 : B0(S, M) := {α ∈ Rp,

p

  • j=1

I{|αj = 0} ≤ S, αl1(p) ≤ M}.

  • Dimension : p ≤ exp(c′n),

(c′ constant)

slide-65
SLIDE 65

Examples Linear model Mathematical ingredients LOL algo

LOL assumptions

  • Sparsity :

for q ∈ (0, 1], Bq(M) := {α ∈ Rp, αlq(p) ≤ M} Particularly, for q = 0 : B0(S, M) := {α ∈ Rp,

p

  • j=1

I{|αj = 0} ≤ S, αl1(p) ≤ M}.

  • Dimension : p ≤ exp(c′n),

(c′ constant)

  • Coherence and u :

(c0,c constants) τn ≤ c

  • log p

n and sup

i=1,...,n

|ui| ≤ c0

  • 1

n.

  • let M > 0, fix ν in ]0, 1[,
slide-66
SLIDE 66

Examples Linear model Mathematical ingredients LOL algo

LOL Thresholds

Choose :

  • Threshold λ1 such that

λ1 = T1

  • log p

n

  • Threshold λ2 such that

λ2 = T2

  • log p

n T1, T2 (σ, M, ν, c0)

slide-67
SLIDE 67

Examples Linear model Mathematical ingredients LOL algo

Convergence (1)

sup

α∈ Bq(M)

P (d(ˆ α∗, α) > η) ≤          4e−γnη2 for η2 ≥ D

  • log p

n

1−q/2 1 for η2 ≤ D

  • log p

n

1−q/2 where D and γ are positive constants (ν, c, c′, c0, T3, T4)

slide-68
SLIDE 68

Examples Linear model Mathematical ingredients LOL algo

Convergence (1)

sup

α∈ Bq(M)

P (d(ˆ α∗, α) > η) ≤          4e−γnη2 for η2 ≥ D

  • log p

n

1−q/2 1 for η2 ≤ D

  • log p

n

1−q/2 where D and γ are positive constants (ν, c, c′, c0, T3, T4) Particularly, for q = 0 and for any S < ν/τn : sup

α∈ B0(S,M)

P (d(ˆ α∗, α) > η) ≤    4e−γnη2 for η2 ≥ D S log p

n

1 for η2 ≤ D S log p

n

slide-69
SLIDE 69

Examples Linear model Mathematical ingredients LOL algo

Convergence (2)

Under the same assumptions, for r ≥ 1 arbitrary, we have : sup

Bq(M)

Ed(ˆ α∗, α)r ≤ D′ log p n (1−q/2)r/2 for some positive constant D′ depending on ν, c, c′, c0, T3, T4, And for q = 0 and for any S < ν/τn sup

B0(S,M)

Ed(ˆ α∗, α)r ≤ D′ S log p n r/2 .

slide-70
SLIDE 70

Examples Linear model Mathematical ingredients LOL algo

Financial signals

Are High frequency financial signals sparse ?

slide-71
SLIDE 71

Examples Linear model Mathematical ingredients LOL algo

FBUND sparse reconstruction

500 1000 1500 2000 2500 122.6 122.65 122.7 122.75 122.8 122.85 122.9 122.95 123 123.05 123.1 FBund 20091207, S=11 Trading time

  • M. Mougeot
slide-72
SLIDE 72

Examples Linear model Mathematical ingredients LOL algo

FBUND sparse reconstruction

500 1000 1500 2000 2500 122.9 123 123.1 123.2 123.3 123.4 123.5 123.6 123.7 123.8 123.9 FBund 20091208, S=9 Trading time

  • M. Mougeot
slide-73
SLIDE 73

Examples Linear model Mathematical ingredients LOL algo

FBUND sparse reconstruction

500 1000 1500 2000 2500 123.2 123.25 123.3 123.35 123.4 123.45 123.5 123.55 123.6 123.65 FBund 20091209, S=14 Trading time

  • M. Mougeot
slide-74
SLIDE 74

Examples Linear model Mathematical ingredients LOL algo

FBUND sparse reconstruction

500 1000 1500 2000 2500 122.9 123 123.1 123.2 123.3 123.4 123.5 FBund 20091210, S=12 Trading time

  • M. Mougeot
slide-75
SLIDE 75

Examples Linear model Mathematical ingredients LOL algo

FBUND sparse reconstruction

500 1000 1500 2000 2500 122.5 122.55 122.6 122.65 122.7 122.75 122.8 122.85 122.9 122.95 123 FBund 20091211, S=7 Trading time

  • M. Mougeot
slide-76
SLIDE 76

Examples Linear model Mathematical ingredients LOL algo

Sparsity

signals : n = 2048, p = 2558 (1023 sin, 1023 cos, reste haar)

2 4 6 8 10 12 14 16 18 20 22 2 4 6 8 10 12 14 16 18 20 Adaptive Sparsity Distribution FBund (137)

Figure:

slide-77
SLIDE 77

Examples Linear model Mathematical ingredients LOL algo

Sparsity

500 1000 1500 2000 2500 3000 −10 10 20 30 40 50 60 70 80 90 Support FBund (137) C S H

Figure: 66 non zero positions

  • M. Mougeot
slide-78
SLIDE 78

Examples Linear model Mathematical ingredients LOL algo

FBOBL sparse reconstruction

500 1000 1500 2000 2500 115.95 116 116.05 116.1 116.15 116.2 116.25 116.3 FBobl 20091207, S=5 Trading time

  • M. Mougeot
slide-79
SLIDE 79

Examples Linear model Mathematical ingredients LOL algo

FBOBL sparse reconstruction

500 1000 1500 2000 2500 116.2 116.3 116.4 116.5 116.6 116.7 116.8 116.9 117 FBobl 20091208, S=7 Trading time

  • M. Mougeot
slide-80
SLIDE 80

Examples Linear model Mathematical ingredients LOL algo

FBOBL sparse reconstruction

500 1000 1500 2000 2500 116.6 116.65 116.7 116.75 116.8 116.85 116.9 FBobl 20091209, S=11 Trading time

  • M. Mougeot
slide-81
SLIDE 81

Examples Linear model Mathematical ingredients LOL algo

FBOBL sparse reconstruction

500 1000 1500 2000 2500 116.65 116.7 116.75 116.8 116.85 116.9 116.95 FBobl 20091210, S=15 Trading time

  • M. Mougeot
slide-82
SLIDE 82

Examples Linear model Mathematical ingredients LOL algo

FBOBL sparse reconstruction

500 1000 1500 2000 2500 116.3 116.35 116.4 116.45 116.5 116.55 116.6 116.65 FBobl 20091211, S=8 Trading time

  • M. Mougeot
slide-83
SLIDE 83

Examples Linear model Mathematical ingredients LOL algo

Sparsity

signals : n = 2048, p = 2558 (1023 sin, 1023 cos, reste haar)

5 10 15 20 25 30 35 5 10 15 20 25 30 Adaptive Sparsity Distribution FBobl (148)

Figure:

slide-84
SLIDE 84

Examples Linear model Mathematical ingredients LOL algo

Sparsity

500 1000 1500 2000 2500 3000 −10 10 20 30 40 50 60 70 80 90 Support FBobl (148) C S H

Figure: 66 non zero positions

  • M. Mougeot
slide-85
SLIDE 85

Examples Linear model Mathematical ingredients LOL algo

Adaptive Compressing procedure LOLA

The procedure LOL described above depends on two tuning parameters s and t. LOLA procedure, which is LOL completed by A, where A, described in the following lines is the adaptive tuning of the thresholds LOLA(Φ, Y ) = LOL(Φ, Y ,ˆ t,ˆ s) with the following choices of the tuning parameters ˆ t = A((Φ1tY , . . . , ΦptY )) and ˆ s = A(( α1, . . . , αp)) where A denotes the following algorithm

slide-86
SLIDE 86

Examples Linear model Mathematical ingredients LOL algo

Adaptive Compressing procedure LOLA

u ←A(Z) Input : variables Z = (Z1, . . . , Zm) Output : level u The inputs of A are the m variables Z = (Z1, . . . , Zm) and the

  • utput is the level u. Let |Z|(1) ≤ |Z|(2) ≤ . . . ≤ |Z|(m) be the
  • rdered sample and let us consider the deviance defined as follows

dev(J) =

J

  • j=1
  • |Z|(j) − |Z|

(J−)2

+

m

  • j=J+1
  • |Z|(j) − |Z|

(J+)2

where |Z|

(J−) and |Z| (J+) are the empirical means of the |Z|(j)’s

for respectively j = 1, . . . , J and j = J + 1, . . . , m. We choose as threshold level u = |Z|(

J)

for ˆ J = Arg min

j=1,...m dev(J).