New Algebraic estimation techniques in signal processing Mamadou - - PowerPoint PPT Presentation

New Algebraic estimation techniques in signal processing

Mamadou Mboup

UFR de Math´ ematiques et Informatique Universit´ e Ren´ e Descartes-Paris 5 Projet ALIEN, INRIA-Futurs Email: mboup@math-info.univ-paris5.fr

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

Outline

1

General overview

2

Mathematical background

3

Applications Polynomial phase signal Signal analysis and representation

Introduction Derivative estimation

Signal Denoising - Change points detection

4

Concluding remarks

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

A simple example - Notations and algorithm

y(t) = n(t)

• ae− t2

σ2 + γ + n0(t)

• x(t)

: σ2 unknown parameter Differential equation : σ2 ˙ x(t) = −tx(t) − tγ → σ2sx = x′ + σ2x(0) − γ

s2

Elimination of structured perturbations: Π(·) = d3

ds3 s2(·)

⇒ (s3x(3) + 9s2x′′ + 18sx′ + 6x)σ2 = s2x(4) + 6sx(3) + 6x′′ Linear estimator: (s3y(3) + 9s2y′′ + 18sy′ + 6y) σ2 = s2y(4) + 6sy(3) + 6y′′ (Strictly) proper estimator: multiply both sides by s−ν, ν > 2:

• y(3)

sν−3 + 9 y′′ sν−2 + 18 y′ sν−1 + 6 y sν−1

σ2 = y(4)

sν−2 + 6 y(3) sν−1 + 6y′′ sν

The estimator is strictly proper for ν 4.

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

A simple example - Notations and algorithm

y(t) = n(t)

• ae− t2

σ2 + γ + n0(t)

• x(t)

: σ2 unknown parameter Differential equation : σ2 ˙ x(t) = −tx(t) − tγ → σ2sx = x′ + σ2x(0) − γ

s2

Elimination of structured perturbations: Π(·) = d3

ds3 s2(·)

⇒ (s3x(3) + 9s2x′′ + 18sx′ + 6x)σ2 = s2x(4) + 6sx(3) + 6x′′ Linear estimator: (s3y(3) + 9s2y′′ + 18sy′ + 6y) σ2 = s2y(4) + 6sy(3) + 6y′′ (Strictly) proper estimator: multiply both sides by s−ν, ν > 2:

• y(3)

sν−3 + 9 y′′ sν−2 + 18 y′ sν−1 + 6 y sν−1

σ2 = y(4)

sν−2 + 6 y(3) sν−1 + 6y′′ sν

The estimator is strictly proper for ν 4.

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

A simple example - Notations and algorithm

y(t) = n(t)

• ae− t2

σ2 + γ + n0(t)

• x(t)

: σ2 unknown parameter Differential equation : σ2 ˙ x(t) = −tx(t) − tγ → σ2sx = x′ + σ2x(0) − γ

s2

Elimination of structured perturbations: Π(·) = d3

ds3 s2(·)

⇒ (s3x(3) + 9s2x′′ + 18sx′ + 6x)σ2 = s2x(4) + 6sx(3) + 6x′′ Linear estimator: (s3y(3) + 9s2y′′ + 18sy′ + 6y) σ2 = s2y(4) + 6sy(3) + 6y′′ (Strictly) proper estimator: multiply both sides by s−ν, ν > 2:

• y(3)

sν−3 + 9 y′′ sν−2 + 18 y′ sν−1 + 6 y sν−1

σ2 = y(4)

sν−2 + 6 y(3) sν−1 + 6y′′ sν

The estimator is strictly proper for ν 4.

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

. . . A simple example - Notations and algorithm

Numerical estimate (time domain)

• σ2(t) =

t {[ν]2τ 2-6[ν]1τ(t-τ)+6(t-τ)2}τ 2(t-τ)ν-3y(τ)dτ t {6(t-τ)3-[ν]3τ 3+9[ν]2τ 2(t-τ)-18τ(t-τ)2}(t-τ)ν-4y(τ)dτ where [ν]i = i

k=1(ν − k)

y(k) sα → t tα−1 · · · t1 (−1)kτ ky(τ)dτdt1 · · · dtα−1 = (−1)k (α − 1)! t (t − τ)α−1τ ky(τ)dτ

The estimation time t may be very small ⇒ fast estimation. The noise effect is attenuated by the iterated integrals

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

. . . A simple example - Notations and algorithm

y(t) = −2π2(t − d)e−2π2(t−λ)2 + n(t), λ : unknown parameter application:

PPM in UWB

Differential equation: t ˙ y − y + 2π2t2y = ( ˙ y + 4π2ty)λ − (2π2y)λ2 2π2y(3) − sy′′ − 3y′ = ([sy]′ − 4π2y′′)λ − (2π2y)λ2 P λ

• λ2
• = Q

Integral equation: y(t) = −2π2(t−λ) t y(τ)dτ + c0t + c1

• +c2

d3 ds3 {s3y − 2π2(s2y′ − sy)} = {2π2 d3 ds3 (s2y)}λ λ = q p

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

Simulation: SNR = 23 dB

λ = 0.11

λ

2

^ ^ λ sqrt( )

• Diff. eq.:
• Int. eq.

−2.0 −1.5 −1.0 −0.5 0.0 0.5 1.0 1.5 2.0 −3 −2 −1 1 2 3 Noise correlation function 10 20 30 40 50 60 70 80 90 100 110 −0.4 −0.2 0.0 0.2 0.4 0.6 0.8 1.0 Estimations of lambda vs time −0.5 0.0 0.5 1.0 1.5 2.0 −10 −5 5 10 15 Zooming 0.0 0.5 1.0 1.5 2.0 2.5 3.0 0.06 0.07 0.08 0.09 0.10 0.11 0.12 0.13 0.14 0.15 0.16

• Diff. Eq.:

The noise need not be white, Gaussian, etc ... !

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

Mathematical background: Differential algebra - operational calculus

Differential algebra, Commutative ring/field, equipped with a derivation a differential ring/field, provides a powerful and elegant mean to exhibit hidden linear structures. In control theory: nonlinear inversion, flatness,... In signal processing: nonlinear inversion (equalizability). Significant breakthrough is expected

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

Some definitions

Differential field extension L/K: Two differential fields K, L

K ⊂ L restriction to K of the derivation of L is the derivation of K.

L/K is termed differentially algebraic iff, ∀ x ∈ L, ∃P, polyn over K | P(x, dx

ds, . . . , dnx dsn ) = 0.

Otherwise, L/K is said to be differentially transcendental. Mikusi´ nski’s field of operators: C = ({f : [0, +∞) → C, f continuous}, +, ⋆) commutative ring without zero divisors. Denote by M, the quotient field of C; its elements{f} are called operators. Equip M with d

f ds = {−tf} (algebraic derivative)

differential field of operators

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

Identifiability

Let k0 be a differential field of constants and Θ = (θ1, . . . , θr) a set of unknown parameters. Let k = k0(Θ). Example:

λ1θ2

1θ2

λ2+λ3θ3

2 ∈ k0(Θ), λi ∈ k0

Let K/k(s) be a finitely generated differentially algebraic extension A signal is an element of K. Consider a finite collection of signals: x = (x1, . . . , xκ) The parameters Θ are linearly identifiable with respect to x if, and only if, PΘ = Q

Pi,j, Qj ∈ spank0(s)[ d

ds ](1, x), i, j = 1, . . . , r and

det(P) = 0.

weakly linearly identifiable with respect to x if, and only if,

Θ′ = (θ′

1, . . . , θ′ q′) are linearly identifiable, wrt x and

each θ′

i (resp. θi) is algebraic over k0(Θ) (resp. k0(Θ′))

projectively linearly identifiable wrt x if, and only if,

θ1 θǫ , . . . , θǫ−1 θǫ , θǫ+1 θǫ , . . . , θr θǫ are linearly identifiable,

for some θǫ = 0.

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

Perturbations

A perturbation noise n is either structured: ⇐ ⇒ ∃ Π ∈ k0(s)[ d

ds], Π = 0 | Πn = 0.

Example : n = γ

sν is annihilated by Π = νsν−1 + sν d ds

non structured: rapid oscillating (high frequency) signal, attenuated by the iterated integrals Non standard analysis description [M. Fliess]

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

Outline

1

General overview

2

Mathematical background

3

Applications Polynomial phase signal Signal analysis and representation

Introduction Derivative estimation

Signal Denoising - Change points detection

4

Concluding remarks

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

Polynomial phase signal

Noisy chirp signal : y(t) = A sin ϕ(t) + n(t), where ϕ(t) = ϕ0 + ϕ1t + ϕ2t2, n(t) is the noise. Set x(t) = y(t) − n(t) : noise-free signal. Dϕ⋆ : ... x(t) + ˙ ϕ(t)2 ˙ x(t) + 3 ˙ ϕ(t) ¨ ϕ(t)x(t) = 0 which reads in the operational domain as

• (2ϕ1ϕ2 + ϕ2

1s + s3) + 4ϕ2(ϕ2 + ϕ1s) d

ds + 4ϕ2

2s d2

ds2

• x

= (¨ x(0) + x(0)ϕ2

1) + ˙

x(0)s + x(0)s2 x(t) is a differentially rational signal

• Dϕ⋆ :

˙ ϕ(t)¨ x(t) − ¨ ϕ(t) ˙ x(t) + ˙ ϕ(t)3x(t) = 0 Different estimator for different Dϕ⋆

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

Linear estimator for the chirp

Setting θ′

1 = ¨

x(0) + ϕ2

1x(0) = 2ϕ2A cos ϕ0

θ′

2 = ˙

x(0) = ϕ1A cos ϕ0 θ′

3 = x(0) = A sin ϕ0

θ′

4 = −ϕ2 1

θ′

5 = 2ϕ1ϕ2

θ′

6 = −4ϕ2 2

we get θ′

1+θ′ 2s+θ′ 3s2+θ′ 4sx+θ′ 5

• 2s d

ds − 1

• x+θ′

6

• s d2

ds2 − d ds

• x = s3x

Θ′ = (θ′

1, . . . , θ′ 6) is linearly identifiable with respect to x

Θ = (A, ϕ0, ϕ1, ϕ2) is weakly linearly identifiable with respect to x.

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

Simulation

y(t) = 2.291 cos (0.897 + 0.876t − 3.892t2) + n(t)

−9 −7 −5 −3 −1 1 3 5 7 9 0.0 0.4 0.8 1.2 4.0 3.6 3.2 2.8 2.4 2.0 1.6

Noisy chirp signal, SNR = 10dB

0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8

0.0 0.0

1.8 1.6 1.4 1.2 1.0 0.8 0.6 0.4 0.2 10.00 7.86 5.71 3.57 1.43 −0.71 −2.86 −5.00 10 8 6 4 2 −2 −4 −6 −8 −10

ϕ ϕ

1 2

ϕ1 and ϕ2 vs. estimation time

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

Outline

1

General overview

2

Mathematical background

3

Applications Polynomial phase signal Signal analysis and representation

Introduction Derivative estimation

Signal Denoising - Change points detection

4

Concluding remarks

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

Introduction

Fourier analysis: smooth and stationary signals Time-frequency analysis (Gabor, W. Ville, Y. Meyer)

the atoms are localized both in time and frequency Heisenberg-Gabor uncertainty principle Signals with different high energy structures are problematic (S. Mallat)

Algebraic approach: time atoms (e.g. polynomials) + derivative estimation.

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

Derivative estimation

(Time) analytique signal : To the convergent series x(t) =

k≥0 ak tk k!, ak ∈ C −

→ x =

k≥0 ak sk+1 ∈ M.

Let xN(t) = N

k=0 ak tk k!, then

dN+1 dtN+1 xN(t) = 0 ⇒ sN+1xN−sNx(0)−sN−1 ˙ x(0)−. . .−x(N)(0) = 0. Replace xN by x. The estimates of the derivatives

• x(i)(0)

N

i=0

are linearly identifiable from s−ν dm dsm

• x(N)(0) + x(N−1)(0)s + . . . + x(0)sN

= s−ν dm dsm

• sN+1x
• , m = 0, . . . , N

ν N

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

Estimating the derivative of a transient signal

x(t) = ℜ(f(t)) + n(t), where f(t) = K0 √S0 g t − t0 S0

• exp (iω0t) + K1

√S1 g t − t1 S1

• exp (iω0t)

+ K2 √S1 g t − t0 S1

• exp (iω1t) + K3

√S0 g t − t1 S0

• exp (iω1t)

with K0 = 0.4326, K1 = −1.6656, K2 = 0.1253, K3 = 0.2877, ω0 = 2π73, ω1 = 2π123, S0 = 1.5, S1 = 3, t0 = 2, t1 = 4; g(t) = 0.54 + 0.46 cos(2πt) : Hamming window x(t) is a noisy transient signal with different high energy structures in the time-frequency plane: difficult to analyze (S. Mallat).

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

Derivative of a transient signal

2.4 2.45 2.5 2.55 2.6 2.65 2.7 x 10

4

−200 −150 −100 −50 50 100 150 200 Time (Te)

d dt(x − n) (blue), its estimation (green), estimation error (red)

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

Outline

1

General overview

2

Mathematical background

3

Applications Polynomial phase signal Signal analysis and representation

Introduction Derivative estimation

Signal Denoising - Change points detection

4

Concluding remarks

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

Piecewise polynomial signals

Signal model: x(t) =

K

• i=1

χ[ti−1,ti](t) pi(t − ti−1) + n(t), deg pi = N − 1 = 3

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 −14 −12 −10 −8 −6 −4 −2 2 4

Noise-free signal

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 −15 −13 −11 −9 −7 −5 −3 −1 1 3 5

Noisy signal, SNR = 21.9 dB

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

Change point detection: Index of discontinuity

Let y(t) = x(t) − n(t) : noise free signal and I∆(τ) =]τ, τ + ∆[ with at most one discontinuity at tr. deg pi(t) = N − 1, i = 1, . . . K ⇒ dN dtN y(t)

• t∈I∆(τ)

=

N

• k=1

σN−kδ(t − tr)(k−1) In M: Dr(s, ∆)

△

= s−ν dN dsN {sNy} = s−ν dN dsN N

• k=1

σN−ksk−1e−trs

• Dr = 0 ⇐

⇒ σk = 0, k = 0, . . . , N − 1 ⇐ ⇒ tr / ∈ I∆(τ) Replace y by x and go back to the time domain

• Dr(τ, ∆): change point index

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

Change point index

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

Change point detection: Derivative estimation

For each window I∆(τ) = [τ, τ + ∆], ˙ x(τ) is estimated from the Taylor expansion of order 1: [ ˙ x(τ, ∆)] = (ν−1)(ν−2)

∆

1 (1 − t)ν−4[1 − (ν − 2)t]x(∆(t + τ))dt

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

Signal denoising and reconstruction

As before, we consider Taylor expansion of order 1 and possibly,

• ne discontinuity in I∆(τ) = [τ, τ + ∆]. The signal estimator

(order 0 derivative) is: [x∆(τ)] = ν−1

∆

1 (1−t)ν−3((ν −1)[νt−2∆]+ν −2)x(∆(t+τ))dt

Denoised signal, SNR=27.74 dB Reconstructed signal, SNR=39,58 dB

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

Concluding remarks

Algebraic estimation theory in signal processing: another standpoint Deterministic setting The physical continuous-time nature of the signals is exploited No distinction between stationary and non stationary signals The noises need not be zero-mean, white, Gaussian Fast estimation, simple implementation and robustness to a wide variety of perturbations and a significant step towards reconciling signal processing and control !

Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.

. . . A simple example - Notations and algorithm

Pulse Position Modulation w(t) = t τ 2 e−2π2 t2

τ2 : Shaping pulse u(t)

A/D

− − − − − − → |0101|..|01..

SC

− − − − − − → b0, b1, ..

CC

− − − − − − → x(t) = P

k w(t − kT − f(bk))

? ? yChannel ˆ u(t)

D/A

← − − − − − − |0101|..|01..

SC−1

← − − − − − − b λk = f(bk)

Estim

← − − − − − − y(t) = x(t) + n(t) Return Algebraic estimation Control Theory, Estimation and Signal Processing, M. Fliess anniversary, IHP mars 30-31, 2006.