Adaptive Sinusoidal Models A tutorial George Kafentzis Ph.D. - - PowerPoint PPT Presentation

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptive Sinusoidal Models

A tutorial George Kafentzis Ph.D. student University of Crete University of Rennes I November 2012

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

1

Parametric Techniques

2

QHM

3

iQHM

4

adaptive QHM

5

extended aQHM

6

References

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Outline

1

Parametric Techniques

2

QHM

3

iQHM

4

adaptive QHM

5

extended aQHM

6

References

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Introduction

Sinusoidal modeling of speech

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Introduction

Sinusoidal modeling of speech Dates back to the early 80ies

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Introduction

Sinusoidal modeling of speech Dates back to the early 80ies

(Almeida et al, Quatieri et al, Xerra et al)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Introduction

Sinusoidal modeling of speech Dates back to the early 80ies

(Almeida et al, Quatieri et al, Xerra et al)

Modeling of speech as a sum of sinusoids

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Introduction

Sinusoidal modeling of speech Dates back to the early 80ies

(Almeida et al, Quatieri et al, Xerra et al)

Modeling of speech as a sum of sinusoids Parameters: amplitude, frequency, and phase

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Introduction

Sinusoidal modeling of speech Dates back to the early 80ies

(Almeida et al, Quatieri et al, Xerra et al)

Modeling of speech as a sum of sinusoids Parameters: amplitude, frequency, and phase Essential assumption: local stationarity!

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Introduction

Sinusoidal modeling of speech Dates back to the early 80ies

(Almeida et al, Quatieri et al, Xerra et al)

Modeling of speech as a sum of sinusoids Parameters: amplitude, frequency, and phase Essential assumption: local stationarity!

Speech is considered stationary in short time intervals

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Introduction

Sinusoidal modeling of speech Dates back to the early 80ies

(Almeida et al, Quatieri et al, Xerra et al)

Modeling of speech as a sum of sinusoids Parameters: amplitude, frequency, and phase Essential assumption: local stationarity!

Speech is considered stationary in short time intervals (it is not, but it is a convenient assumption :-) )

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Sinusoidal modeling - Quatieri, McAulay, 1986

Each frame is modeled as a sum of sinusoids: s(t) =

K

• k=−K

akej(2πfkt)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Sinusoidal modeling - Quatieri, McAulay, 1986

Each frame is modeled as a sum of sinusoids: s(t) =

K

• k=−K

akej(2πfkt) ak: complex amplitudes of the kth sinusoid

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Sinusoidal modeling - Quatieri, McAulay, 1986

Each frame is modeled as a sum of sinusoids: s(t) =

K

• k=−K

akej(2πfkt) ak: complex amplitudes of the kth sinusoid fk: frequency of the kth sinusoid

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Sinusoidal modeling - Quatieri, McAulay, 1986

Each frame is modeled as a sum of sinusoids: s(t) =

K

• k=−K

akej(2πfkt) ak: complex amplitudes of the kth sinusoid fk: frequency of the kth sinusoid

Estimation of the sinusoidal parameters: FFT + peak picking

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Sinusoidal modeling - Quatieri, McAulay, 1986

Each frame is modeled as a sum of sinusoids: s(t) =

K

• k=−K

akej(2πfkt) ak: complex amplitudes of the kth sinusoid fk: frequency of the kth sinusoid

Estimation of the sinusoidal parameters: FFT + peak picking Various improvements (e.g. quadratic interpolation)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Sinusoidal modeling - Quatieri, McAulay, 1986

Each frame is modeled as a sum of sinusoids: s(t) =

K

• k=−K

akej(2πfkt) ak: complex amplitudes of the kth sinusoid fk: frequency of the kth sinusoid

Estimation of the sinusoidal parameters: FFT + peak picking Various improvements (e.g. quadratic interpolation)

Highlight: no distinction between voiced and unvoiced frames

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Sinusoidal model - Quatieri, McAulay, 1986

Signal reconstruction: Overlap Add method: ˆ s(t) = K

k=−K ˆ

akej2πˆ

fkt

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Sinusoidal model - Quatieri, McAulay, 1986

Signal reconstruction: Overlap Add method: ˆ s(t) = K

k=−K ˆ

akej2πˆ

fkt

Frame by frame parameter interpolation (PI):

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Sinusoidal model - Quatieri, McAulay, 1986

Signal reconstruction: Overlap Add method: ˆ s(t) = K

k=−K ˆ

akej2πˆ

fkt

Frame by frame parameter interpolation (PI):

Linear amplitude interpolation

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Sinusoidal model - Quatieri, McAulay, 1986

Signal reconstruction: Overlap Add method: ˆ s(t) = K

k=−K ˆ

akej2πˆ

fkt

Frame by frame parameter interpolation (PI):

Linear amplitude interpolation Cubic phase interpolation

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Sinusoidal model - Quatieri, McAulay, 1986

Pros:

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Sinusoidal model - Quatieri, McAulay, 1986

Pros:

Fast

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Sinusoidal model - Quatieri, McAulay, 1986

Pros:

Fast Good signal reconstruction

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Sinusoidal model - Quatieri, McAulay, 1986

Pros:

Fast Good signal reconstruction

Cons:

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Sinusoidal model - Quatieri, McAulay, 1986

Pros:

Fast Good signal reconstruction

Cons:

Local stationarity assumption holds

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Sinusoidal model - Quatieri, McAulay, 1986

Pros:

Fast Good signal reconstruction

Cons:

Local stationarity assumption holds Not good modifications

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Sinusoidal model - Quatieri, McAulay, 1986

Pros:

Fast Good signal reconstruction

Cons:

Local stationarity assumption holds Not good modifications Requires large windows

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Harmonic + Noise model - Stylianou, 1993-1996

Separate signal into periodic (deterministic) and aperiodic (stochastic) components:

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Harmonic + Noise model - Stylianou, 1993-1996

Separate signal into periodic (deterministic) and aperiodic (stochastic) components: s(t) = sd(t) + ss(t)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Harmonic + Noise model - Stylianou, 1993-1996

Separate signal into periodic (deterministic) and aperiodic (stochastic) components: s(t) = sd(t) + ss(t) Highlight: In voiced frames, fk = kf0 : deterministic − → harmonic sd(t) =

K

• k=−K

akej2πkˆ

f0tw(t)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Harmonic + Noise model - Stylianou, 1993-1996

Separate signal into periodic (deterministic) and aperiodic (stochastic) components: s(t) = sd(t) + ss(t) Highlight: In voiced frames, fk = kf0 : deterministic − → harmonic sd(t) =

K

• k=−K

akej2πkˆ

f0tw(t)

Least Squares method for finding amplitudes and phases (f0 is considered as known)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Harmonic + Noise model - Stylianou, 1993-1996

Separate signal into periodic (deterministic) and aperiodic (stochastic) components: s(t) = sd(t) + ss(t) Highlight: In voiced frames, fk = kf0 : deterministic − → harmonic sd(t) =

K

• k=−K

akej2πkˆ

f0tw(t)

Least Squares method for finding amplitudes and phases (f0 is considered as known) Window length in HNM < Window length in SM

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Harmonic + Noise model - Stylianou, 1993-1996

Separate signal into periodic (deterministic) and aperiodic (stochastic) components: s(t) = sd(t) + ss(t) Highlight: In voiced frames, fk = kf0 : deterministic − → harmonic sd(t) =

K

• k=−K

akej2πkˆ

f0tw(t)

Least Squares method for finding amplitudes and phases (f0 is considered as known) Window length in HNM < Window length in SM Reconstruction:

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Harmonic + Noise model - Stylianou, 1993-1996

Separate signal into periodic (deterministic) and aperiodic (stochastic) components: s(t) = sd(t) + ss(t) Highlight: In voiced frames, fk = kf0 : deterministic − → harmonic sd(t) =

K

• k=−K

akej2πkˆ

f0tw(t)

Least Squares method for finding amplitudes and phases (f0 is considered as known) Window length in HNM < Window length in SM Reconstruction:

OLA or PI for harmonic part

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Harmonic + Noise model - Stylianou, 1993-1996

Separate signal into periodic (deterministic) and aperiodic (stochastic) components: s(t) = sd(t) + ss(t) Highlight: In voiced frames, fk = kf0 : deterministic − → harmonic sd(t) =

K

• k=−K

akej2πkˆ

f0tw(t)

Least Squares method for finding amplitudes and phases (f0 is considered as known) Window length in HNM < Window length in SM Reconstruction:

OLA or PI for harmonic part OLA for stochastic part

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Harmonic + Noise model - Stylianou, 1993-1996

Pros:

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Harmonic + Noise model - Stylianou, 1993-1996

Pros:

Pitch synchronous analysis

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Harmonic + Noise model - Stylianou, 1993-1996

Pros:

Pitch synchronous analysis Smaller window lengths

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Harmonic + Noise model - Stylianou, 1993-1996

Pros:

Pitch synchronous analysis Smaller window lengths Convenient for modifications

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Harmonic + Noise model - Stylianou, 1993-1996

Pros:

Pitch synchronous analysis Smaller window lengths Convenient for modifications

Cons:

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Harmonic + Noise model - Stylianou, 1993-1996

Pros:

Pitch synchronous analysis Smaller window lengths Convenient for modifications

Cons:

Local stationarity assumption holds

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Harmonic + Noise model - Stylianou, 1993-1996

Pros:

Pitch synchronous analysis Smaller window lengths Convenient for modifications

Cons:

Local stationarity assumption holds Depends on good f0 estimation

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Harmonic + Noise model - Stylianou, 1993-1996

Pros:

Pitch synchronous analysis Smaller window lengths Convenient for modifications

Cons:

Local stationarity assumption holds Depends on good f0 estimation Speech is not purely harmonic (fk ≈ kf0)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Harmonic + Noise model - Stylianou, 1993-1996

Pros:

Pitch synchronous analysis Smaller window lengths Convenient for modifications

Cons:

Local stationarity assumption holds Depends on good f0 estimation Speech is not purely harmonic (fk ≈ kf0)

...this last observation is the motivation for the following model...

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Outline

1

Parametric Techniques

2

QHM

3

iQHM

4

adaptive QHM

5

extended aQHM

6

References

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Quasi-Harmonic model - Pantazis, Stylianou, Rosec, 2007-2010

sd(t) =

K

• k=−K

(ak + tbk)ej2πˆ

fktw(t)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Quasi-Harmonic model - Pantazis, Stylianou, Rosec, 2007-2010

sd(t) =

K

• k=−K

(ak + tbk)ej2πˆ

fktw(t)

ak, bk are complex numbers

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Quasi-Harmonic model - Pantazis, Stylianou, Rosec, 2007-2010

sd(t) =

K

• k=−K

(ak + tbk)ej2πˆ

fktw(t)

ak, bk are complex numbers usually fk = kf0, where f0 is considered as known

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Quasi-Harmonic model - Pantazis, Stylianou, Rosec, 2007-2010

sd(t) =

K

• k=−K

(ak + tbk)ej2πˆ

fktw(t)

ak, bk are complex numbers usually fk = kf0, where f0 is considered as known w(t) is the analysis window

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Quasi-Harmonic model - Pantazis, Stylianou, Rosec, 2007-2010

sd(t) =

K

• k=−K

(ak + tbk)ej2πˆ

fktw(t)

ak, bk are complex numbers usually fk = kf0, where f0 is considered as known w(t) is the analysis window Again: Least Squares method for finding amplitudes

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Quasi-Harmonic model - Pantazis, Stylianou, Rosec, 2007-2010

sd(t) =

K

• k=−K

(ak + tbk)ej2πˆ

fktw(t)

ak, bk are complex numbers usually fk = kf0, where f0 is considered as known w(t) is the analysis window Again: Least Squares method for finding amplitudes Window length ≈ 3 pitch periods

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Quasi-Harmonic model - Pantazis, Stylianou, Rosec, 2007-2010

sd(t) =

K

• k=−K

(ak + tbk)ej2πˆ

fktw(t)

ak, bk are complex numbers usually fk = kf0, where f0 is considered as known w(t) is the analysis window Again: Least Squares method for finding amplitudes Window length ≈ 3 pitch periods Reconstruction: ˆ sd(t) =

K

• k=−K

(ˆ ak + tˆ bk)ej2πˆ

fkt

(1)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Quasi-Harmonic model

HM versus QHM in frequency estimation - pure tone @ 100 Hz

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Quasi-Harmonic model

HM versus QHM in frequency estimation - pure tone @ 100 Hz given frequency for both models: 90 Hz

50 100 150 200 250 0.1 0.2 0.3 0.4 0.5 0.6 0.7 Frequency (Hz) Magnitude Original Harmonic Quasi−Harmonic

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Quasi-Harmonic model

Time domain properties:

• Inst. amplitude:

Mk(t) = |ak + tbk| =

• (aR

k + tbR k )2 + (aI k + tbI k)2

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Quasi-Harmonic model

Time domain properties:

• Inst. amplitude:

Mk(t) = |ak + tbk| =

• (aR

k + tbR k )2 + (aI k + tbI k)2

• Inst. phase: Φk(t) = 2πˆ

fkt + tan−1 aI

k + tbI k

aR

k + tbR k

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Quasi-Harmonic model

Time domain properties:

• Inst. amplitude:

Mk(t) = |ak + tbk| =

• (aR

k + tbR k )2 + (aI k + tbI k)2

• Inst. phase: Φk(t) = 2πˆ

fkt + tan−1 aI

k + tbI k

aR

k + tbR k

• Inst. frequency: Fk(t) = 1

2πΦ′(t) = ˆ fk + 1 2π aR

k bI k − aI kbR k

M2

k(t)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Quasi-Harmonic model

Time domain properties:

• Inst. amplitude:

Mk(t) = |ak + tbk| =

• (aR

k + tbR k )2 + (aI k + tbI k)2

• Inst. phase: Φk(t) = 2πˆ

fkt + tan−1 aI

k + tbI k

aR

k + tbR k

• Inst. frequency: Fk(t) = 1

2πΦ′(t) = ˆ fk + 1 2π aR

k bI k − aI kbR k

M2

k(t)

where xR, xI denote the real and imaginary part of x

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Quasi-Harmonic model

HM vs QHM on frequency tracks - pure tone @ 100 Hz:

−20 −15 −10 −5 5 10 15 20 85 90 95 100 105 Time (ms) Frequency (Hz) True Freq. Estimated Freq. QHM Inst. Freq.

Highlight: frequency correction mechanism

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Quasi-Harmonic model

HM vs QHM on frequency tracks - pure tone @ 100 Hz:

−20 −15 −10 −5 5 10 15 20 85 90 95 100 105 Time (ms) Frequency (Hz) True Freq. Estimated Freq. QHM Inst. Freq.

Highlight: frequency correction mechanism

Let’s discuss a bit on that...

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Frequency mismatch correction

Frequency domain view: Fourier Transform of the model:

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Frequency mismatch correction

Frequency domain view: Fourier Transform of the model: Reminder: xk(t) = akej2πfktw(t) + tbkej2πfktw(t)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Frequency mismatch correction

Frequency domain view: Fourier Transform of the model: Reminder: xk(t) = akej2πfktw(t) + tbkej2πfktw(t)

Xk(f ) = akW (f − ˆ fk) + j bk 2π W ′(f − ˆ fk) (2)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Frequency mismatch correction

Frequency domain view: Fourier Transform of the model: Reminder: xk(t) = akej2πfktw(t) + tbkej2πfktw(t)

Xk(f ) = akW (f − ˆ fk) + j bk 2π W ′(f − ˆ fk) (2) where W (f ) = FT{w(t)} and W ′(f ) = dW (f )/df .

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Frequency mismatch correction

Frequency domain view: Fourier Transform of the model: Reminder: xk(t) = akej2πfktw(t) + tbkej2πfktw(t)

Xk(f ) = akW (f − ˆ fk) + j bk 2π W ′(f − ˆ fk) (2) where W (f ) = FT{w(t)} and W ′(f ) = dW (f )/df .

Projecting bk to ak: bk = ρ1,kak + ρ2,kjak (3)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Frequency mismatch correction

Frequency domain view: Fourier Transform of the model: Reminder: xk(t) = akej2πfktw(t) + tbkej2πfktw(t)

Xk(f ) = akW (f − ˆ fk) + j bk 2π W ′(f − ˆ fk) (2) where W (f ) = FT{w(t)} and W ′(f ) = dW (f )/df .

Projecting bk to ak: bk = ρ1,kak + ρ2,kjak (3) Then, Xk(f ) = ak

• W (f − ˆ

fk) − ρ2,k 2π W ′(f − ˆ fk) +j ρ1,k 2π W ′(f − ˆ fk)

• (4)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Frequency mismatch correction

Taylor series expansion of W (f − ˆ fk − ρ2,k

2π ):

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Frequency mismatch correction

Taylor series expansion of W (f − ˆ fk − ρ2,k

2π ):

W (f − ˆ fk − ρ2,k 2π ) = W (f − ˆ fk) − ρ2,k 2π W ′(f − ˆ fk)+ O(ρ2

2,kW ′′(f − ˆ

fk)) (5)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Frequency mismatch correction

Taylor series expansion of W (f − ˆ fk − ρ2,k

2π ):

W (f − ˆ fk − ρ2,k 2π ) = W (f − ˆ fk) − ρ2,k 2π W ′(f − ˆ fk)+ O(ρ2

2,kW ′′(f − ˆ

fk)) (5) If the value of term W ′′(f ) at fk is small, then for small values

• f ρ2,k, it is:

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Frequency mismatch correction

Taylor series expansion of W (f − ˆ fk − ρ2,k

2π ):

W (f − ˆ fk − ρ2,k 2π ) = W (f − ˆ fk) − ρ2,k 2π W ′(f − ˆ fk)+ O(ρ2

2,kW ′′(f − ˆ

fk)) (5) If the value of term W ′′(f ) at fk is small, then for small values

• f ρ2,k, it is:

W (f − ˆ fk − ρ2,k 2π ) ≈ W (f − ˆ fk) − ρ2,k 2π W ′(f − ˆ fk) (6)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Frequency mismatch correction

Taylor series expansion of W (f − ˆ fk − ρ2,k

2π ):

W (f − ˆ fk − ρ2,k 2π ) = W (f − ˆ fk) − ρ2,k 2π W ′(f − ˆ fk)+ O(ρ2

2,kW ′′(f − ˆ

fk)) (5) If the value of term W ′′(f ) at fk is small, then for small values

• f ρ2,k, it is:

W (f − ˆ fk − ρ2,k 2π ) ≈ W (f − ˆ fk) − ρ2,k 2π W ′(f − ˆ fk) (6)

So, Xk(f ) ≈ ak

• W (f − ˆ

fk − ρ2,k 2π ) + j ρ1,k 2π W ′(f − ˆ fk)

• (7)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Frequency mismatch correction

which goes back in time domain as...

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Frequency mismatch correction

which goes back in time domain as... xk(t) ≈ ak

• ej(2πˆ

fk+ρ2,k)t + ρ1,ktej2πˆ fkt

w(t) (8)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Frequency mismatch correction

which goes back in time domain as... xk(t) ≈ ak

• ej(2πˆ

fk+ρ2,k)t + ρ1,ktej2πˆ fkt

w(t) (8) Then, ρ2,k/2π can be an estimator of the frequency error ηk: ˆ ηk = ρ2,k/2π = 1 2π aR

k bI k − aI kbR k

|ak|2 (9)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Frequency mismatch correction

which goes back in time domain as... xk(t) ≈ ak

• ej(2πˆ

fk+ρ2,k)t + ρ1,ktej2πˆ fkt

w(t) (8) Then, ρ2,k/2π can be an estimator of the frequency error ηk: ˆ ηk = ρ2,k/2π = 1 2π aR

k bI k − aI kbR k

|ak|2 (9) In other words, QHM suggests a frequency correction to the input frequencies ˆ fk (or a frequency estimator). This suggestion is however conditional on the magnitude of ρ2,k and the value of term W ′′(f ) at fk

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Frequency mismatch correction

which goes back in time domain as... xk(t) ≈ ak

• ej(2πˆ

fk+ρ2,k)t + ρ1,ktej2πˆ fkt

w(t) (8) Then, ρ2,k/2π can be an estimator of the frequency error ηk: ˆ ηk = ρ2,k/2π = 1 2π aR

k bI k − aI kbR k

|ak|2 (9) In other words, QHM suggests a frequency correction to the input frequencies ˆ fk (or a frequency estimator). This suggestion is however conditional on the magnitude of ρ2,k and the value of term W ′′(f ) at fk Also, the correction term depends on the window mainlobe width

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Outline

1

Parametric Techniques

2

QHM

3

iQHM

4

adaptive QHM

5

extended aQHM

6

References

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

iterative QHM - Pantazis, Stylianou, Rosec, 2007-2010

This frequency updating mechanism provides frequencies which can be used in the model iteratively and result in better parameter estimation (ak, bk)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

iterative QHM - Pantazis, Stylianou, Rosec, 2007-2010

This frequency updating mechanism provides frequencies which can be used in the model iteratively and result in better parameter estimation (ak, bk) This iterative parameter estimation is referred to as the iterative QHM

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

iterative QHM

HM versus iQHM in frequency estimation - speech signal:

50 100 150 200 250 300 350 400 450 −0.2 −0.1 0.1 0.2 Time (samples) Speech 500 1000 1500 2000 2500 3000 3500 4000 −40 −20 20 Frequency (Hz) Magnitude Speech spectrum kf0 k(f0+∆ f0)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

iterative QHM

Noise robustness:

20 40 60 80 10

−10

10

−5

10 SNR (dB) MSE(a1) 20 40 60 80 10

−10

10

−5

10 SNR (dB) MSE(a2) 20 40 60 80 10

−10

10

−5

10 SNR (dB) MSE(a3) 20 40 60 80 10

−10

10

−5

10 SNR (dB) MSE(a4) CRLB no iter 3 iter CRLB no iter 3 iter CRLB no iter 3 iter CRLB no iter 3 iter

(a) MSE for amplitudes

20 40 60 80 10

−10

10

−5

10 SNR (dB) MSE(f1) 20 40 60 80 10

−10

10

−5

10 SNR (dB) MSE(f2) 20 40 60 80 10

−10

10

−5

10 SNR (dB) MSE(f3) 20 40 60 80 10

−10

10

−5

10 SNR (dB) MSE(f4) CRLB no iter 3 iter CRLB no iter 3 iter CRLB no iter 3 iter CRLB no iter 3 iter

(b) MSE for frequencies Figure: Noise Robustness

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

iterative QHM

Pros:

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

iterative QHM

Pros:

Linear amplitude evolution

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

iterative QHM

Pros:

Linear amplitude evolution Frequency mismatch correction: ηk = fk − ˆ fk

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

iterative QHM

Pros:

Linear amplitude evolution Frequency mismatch correction: ηk = fk − ˆ fk

Cons:

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

iterative QHM

Pros:

Linear amplitude evolution Frequency mismatch correction: ηk = fk − ˆ fk

Cons:

Needs larger analysis window

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

iterative QHM

Pros:

Linear amplitude evolution Frequency mismatch correction: ηk = fk − ˆ fk

Cons:

Needs larger analysis window And what about...

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

iterative QHM

Pros:

Linear amplitude evolution Frequency mismatch correction: ηk = fk − ˆ fk

Cons:

Needs larger analysis window And what about...

local stationarity??

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

iterative QHM

Pros:

Linear amplitude evolution Frequency mismatch correction: ηk = fk − ˆ fk

Cons:

Needs larger analysis window And what about...

local stationarity?? Speech is non stationary even in very short time intervals

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

iterative QHM

Pros:

Linear amplitude evolution Frequency mismatch correction: ηk = fk − ˆ fk

Cons:

Needs larger analysis window And what about...

local stationarity?? Speech is non stationary even in very short time intervals iQHM still holds the local stationary assumption

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Outline

1

Parametric Techniques

2

QHM

3

iQHM

4

adaptive QHM

5

extended aQHM

6

References

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptive Quasi-Harmonic Model

Adaptive models tackle the problem of local non stationarity[5] - How?

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptive Quasi-Harmonic Model

Adaptive models tackle the problem of local non stationarity[5] - How?

By projecting the signal onto non-stationary basis functions ejφk(t)!

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptive Quasi-Harmonic Model

Adaptive models tackle the problem of local non stationarity[5] - How?

By projecting the signal onto non-stationary basis functions ejφk(t)!

Model: sd(t) =

K

• k=−K

(ak + tbk)

• ejφk(t)

w(t)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptive Quasi-Harmonic Model

Adaptive models tackle the problem of local non stationarity[5] - How?

By projecting the signal onto non-stationary basis functions ejφk(t)!

Model: sd(t) =

K

• k=−K

(ak + tbk)

• ejφk(t)

w(t) complex amplitudes ak, bk: estimated via Least Squares

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptive Quasi-Harmonic Model

Adaptive models tackle the problem of local non stationarity[5] - How?

By projecting the signal onto non-stationary basis functions ejφk(t)!

Model: sd(t) =

K

• k=−K

(ak + tbk)

• ejφk(t)

w(t) complex amplitudes ak, bk: estimated via Least Squares

ˆ a ˆ b

• = (E HW HWE)−1E HW HWs

(10)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptive Quasi-Harmonic Model

Adaptive models tackle the problem of local non stationarity[5] - How?

By projecting the signal onto non-stationary basis functions ejφk(t)!

Model: sd(t) =

K

• k=−K

(ak + tbk)

• ejφk(t)

w(t) complex amplitudes ak, bk: estimated via Least Squares

ˆ a ˆ b

• = (E HW HWE)−1E HW HWs

(10) where E = [E0|E1]:

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptive Quasi-Harmonic Model

Adaptive models tackle the problem of local non stationarity[5] - How?

By projecting the signal onto non-stationary basis functions ejφk(t)!

Model: sd(t) =

K

• k=−K

(ak + tbk)

• ejφk(t)

w(t) complex amplitudes ak, bk: estimated via Least Squares

ˆ a ˆ b

• = (E HW HWE)−1E HW HWs

(10) where E = [E0|E1]: (E0)n,k =)ejφk(tn)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptive Quasi-Harmonic Model

Adaptive models tackle the problem of local non stationarity[5] - How?

By projecting the signal onto non-stationary basis functions ejφk(t)!

Model: sd(t) =

K

• k=−K

(ak + tbk)

• ejφk(t)

w(t) complex amplitudes ak, bk: estimated via Least Squares

ˆ a ˆ b

• = (E HW HWE)−1E HW HWs

(10) where E = [E0|E1]: (E0)n,k =)ejφk(tn) (E1)n,k = tnejφk(tn) = tn(E0)n,k

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptive Quasi-Harmonic Model

Definition of phase: ˘ φk(t) = ˆ φk(tl−1) + t

tl−1

2πˆ fk(u)du

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptive Quasi-Harmonic Model

Definition of phase: ˘ φk(t) = ˆ φk(tl−1) + t

tl−1

2πˆ fk(u)du where ˆ fk(t) is the estimated instantaneous frequency

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptive Quasi-Harmonic Model

Definition of phase: ˘ φk(t) = ˆ φk(tl−1) + t

tl−1

2πˆ fk(u)du where ˆ fk(t) is the estimated instantaneous frequency Estimation error due to non stationarity is reduced [5]

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptive Quasi-Harmonic Model

Definition of phase: ˘ φk(t) = ˆ φk(tl−1) + t

tl−1

2πˆ fk(u)du where ˆ fk(t) is the estimated instantaneous frequency Estimation error due to non stationarity is reduced [5] Signal is reconstructed using its instantaneous components ˆ Ak(t), ˆ fk(t), ˆ φk(t):

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptive Quasi-Harmonic Model

Definition of phase: ˘ φk(t) = ˆ φk(tl−1) + t

tl−1

2πˆ fk(u)du where ˆ fk(t) is the estimated instantaneous frequency Estimation error due to non stationarity is reduced [5] Signal is reconstructed using its instantaneous components ˆ Ak(t), ˆ fk(t), ˆ φk(t):

ˆ x(t) =

K

• k=−K

ˆ Ak(t)ej ˆ

φk(t)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptive Quasi-Harmonic Model

Definition of phase: ˘ φk(t) = ˆ φk(tl−1) + t

tl−1

2πˆ fk(u)du where ˆ fk(t) is the estimated instantaneous frequency Estimation error due to non stationarity is reduced [5] Signal is reconstructed using its instantaneous components ˆ Ak(t), ˆ fk(t), ˆ φk(t):

ˆ x(t) =

K

• k=−K

ˆ Ak(t)ej ˆ

φk(t)

Adaptation algorithm can be found in [5]

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptive Quasi-Harmonic model

QHM vs aQHM:

Original frequency QHM’s analysis frequency aQHM’s analysis frequency tl

t f

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptation algorithm for aQHM

Let ˆ Ak(tl), ˆ fk(tl), ˆ φk(tl) denote the inst. amplitude, frequency, and phase at time instant tl of the kth component, with l = 1, · · · , L, where L is the number of frames:

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptation algorithm for aQHM

Let ˆ Ak(tl), ˆ fk(tl), ˆ φk(tl) denote the inst. amplitude, frequency, and phase at time instant tl of the kth component, with l = 1, · · · , L, where L is the number of frames: Initialization: (QHM) 1) ˆ f 0

k (tl) = ˆ

f 0

k (tl−1) + ρ0 2,k/2π

2) ˆ A0

k(tl) = |a0 k|, ˆ

φ0

k(tl) = ∠a0 k

3) ˆ f 0

k (tl+1) = ˆ

f 0

k (tl)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptation algorithm for aQHM

Let ˆ Ak(tl), ˆ fk(tl), ˆ φk(tl) denote the inst. amplitude, frequency, and phase at time instant tl of the kth component, with l = 1, · · · , L, where L is the number of frames: Initialization: (QHM) 1) ˆ f 0

k (tl) = ˆ

f 0

k (tl−1) + ρ0 2,k/2π

2) ˆ A0

k(tl) = |a0 k|, ˆ

φ0

k(tl) = ∠a0 k

3) ˆ f 0

k (tl+1) = ˆ

f 0

k (tl)

FOR adaptation i = 1, 2, · · · FOR frame l = 1, 2, · · · , L END

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptation algorithm for aQHM

Let ˆ Ak(tl), ˆ fk(tl), ˆ φk(tl) denote the inst. amplitude, frequency, and phase at time instant tl of the kth component, with l = 1, · · · , L, where L is the number of frames: Initialization: (QHM) 1) ˆ f 0

k (tl) = ˆ

f 0

k (tl−1) + ρ0 2,k/2π

2) ˆ A0

k(tl) = |a0 k|, ˆ

φ0

k(tl) = ∠a0 k

3) ˆ f 0

k (tl+1) = ˆ

f 0

k (tl)

FOR adaptation i = 1, 2, · · · FOR frame l = 1, 2, · · · , L

1

Compute al

k, bl k using ˆ

φi−1

k

(t) and (10)

END

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptation algorithm for aQHM

Let ˆ Ak(tl), ˆ fk(tl), ˆ φk(tl) denote the inst. amplitude, frequency, and phase at time instant tl of the kth component, with l = 1, · · · , L, where L is the number of frames: Initialization: (QHM) 1) ˆ f 0

k (tl) = ˆ

f 0

k (tl−1) + ρ0 2,k/2π

2) ˆ A0

k(tl) = |a0 k|, ˆ

φ0

k(tl) = ∠a0 k

3) ˆ f 0

k (tl+1) = ˆ

f 0

k (tl)

FOR adaptation i = 1, 2, · · · FOR frame l = 1, 2, · · · , L

1

Compute al

k, bl k using ˆ

φi−1

k

(t) and (10)

2

Update ˆ f i

k (tl) using (9)

END

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptation algorithm for aQHM

Let ˆ Ak(tl), ˆ fk(tl), ˆ φk(tl) denote the inst. amplitude, frequency, and phase at time instant tl of the kth component, with l = 1, · · · , L, where L is the number of frames: Initialization: (QHM) 1) ˆ f 0

k (tl) = ˆ

f 0

k (tl−1) + ρ0 2,k/2π

2) ˆ A0

k(tl) = |a0 k|, ˆ

φ0

k(tl) = ∠a0 k

3) ˆ f 0

k (tl+1) = ˆ

f 0

k (tl)

FOR adaptation i = 1, 2, · · · FOR frame l = 1, 2, · · · , L

1

Compute al

k, bl k using ˆ

φi−1

k

(t) and (10)

2

Update ˆ f i

k (tl) using (9)

3

Compute ˆ Ai

k(tl) and ˆ

φi

k(tl)

END

END

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptation algorithm for aQHM

Let ˆ Ak(tl), ˆ fk(tl), ˆ φk(tl) denote the inst. amplitude, frequency, and phase at time instant tl of the kth component, with l = 1, · · · , L, where L is the number of frames: Initialization: (QHM) 1) ˆ f 0

k (tl) = ˆ

f 0

k (tl−1) + ρ0 2,k/2π

2) ˆ A0

k(tl) = |a0 k|, ˆ

φ0

k(tl) = ∠a0 k

3) ˆ f 0

k (tl+1) = ˆ

f 0

k (tl)

FOR adaptation i = 1, 2, · · · FOR frame l = 1, 2, · · · , L

1

Compute al

k, bl k using ˆ

φi−1

k

(t) and (10)

2

Update ˆ f i

k (tl) using (9)

3

Compute ˆ Ai

k(tl) and ˆ

φi

k(tl)

END

4

Interpolation of the parameters {ˆ Ai

k(t), ˆ

f i

k (t), ˆ

φi

k(t)}

END

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptive Quasi-Harmonic model

Synthetic signal:

0.05 0.1 1 1.5 2 2.5 3 Time (s) (a) AM 1 0.05 0.1 600 650 700 750 800 Time (s) (b) FM 1 (Hz) 0.05 0.1 1 1.5 2 2.5 3 Time (s) (c) AM 2 0.05 0.1 900 950 1000 1050 1100 Time (s) (d) FM 2 (Hz) True Estimated True Estimated True Estimated True Estimated

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptive Quasi-Harmonic model

Real Signal:

0.05 0.1 0.15 0.2 0.25 0.3 0.35 −0.5 0.5 Time (s) (a) 0.05 0.1 0.15 0.2 0.25 0.3 0.35 −0.1 −0.05 0.05 0.1 Time (s) (b) 0.05 0.1 0.15 0.2 0.25 0.3 0.35 −0.1 −0.05 0.05 0.1 Time (s) (c) 0.05 0.1 0.15 0.2 0.25 0.3 0.35 −0.1 −0.05 0.05 0.1 Time (s) (d) Original QHM Recon. Error aQHM Recon. Error SM Recon. Error

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

aQHM

Pros:

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

aQHM

Pros:

Phase adaptation, non-parametric approach

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

aQHM

Pros:

Phase adaptation, non-parametric approach Local nonstationarity is partially solved

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

aQHM

Pros:

Phase adaptation, non-parametric approach Local nonstationarity is partially solved High signal-to-reconstruction error ratio (SRER): SRER = 20 log10 σˆ

x(t)

σˆ

x(t)−x(t)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

aQHM

Pros:

Phase adaptation, non-parametric approach Local nonstationarity is partially solved High signal-to-reconstruction error ratio (SRER): SRER = 20 log10 σˆ

x(t)

σˆ

x(t)−x(t)

Cons:

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

aQHM

Pros:

Phase adaptation, non-parametric approach Local nonstationarity is partially solved High signal-to-reconstruction error ratio (SRER): SRER = 20 log10 σˆ

x(t)

σˆ

x(t)−x(t)

Cons:

Needs larger analysis window (as iQHM)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

aQHM

Pros:

Phase adaptation, non-parametric approach Local nonstationarity is partially solved High signal-to-reconstruction error ratio (SRER): SRER = 20 log10 σˆ

x(t)

σˆ

x(t)−x(t)

Cons:

Needs larger analysis window (as iQHM) Amplitudes are not adapted to the signal

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Outline

1

Parametric Techniques

2

QHM

3

iQHM

4

adaptive QHM

5

extended aQHM

6

References

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Extended Adaptive Quasi-Harmonic model - Kafentzis, Pantazis, Stylianou, Rosec, 2011

Adaptation is allowed for amplitude as well as phase

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Extended Adaptive Quasi-Harmonic model - Kafentzis, Pantazis, Stylianou, Rosec, 2011

Adaptation is allowed for amplitude as well as phase

Projection of the signal onto non-stationary basis functions αk(t)ejφk(t) [6]!

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Extended Adaptive Quasi-Harmonic model - Kafentzis, Pantazis, Stylianou, Rosec, 2011

Adaptation is allowed for amplitude as well as phase

Projection of the signal onto non-stationary basis functions αk(t)ejφk(t) [6]!

Model: sd(t) =

K

• k=−K

(ak + tbk)

• αk(t)ejφk(t)

w(t)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Extended Adaptive Quasi-Harmonic model - Kafentzis, Pantazis, Stylianou, Rosec, 2011

Adaptation is allowed for amplitude as well as phase

Projection of the signal onto non-stationary basis functions αk(t)ejφk(t) [6]!

Model: sd(t) =

K

• k=−K

(ak + tbk)

• αk(t)ejφk(t)

w(t) complex amplitudes ak, bk: estimated via Least Squares

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Extended Adaptive Quasi-Harmonic model - Kafentzis, Pantazis, Stylianou, Rosec, 2011

Adaptation is allowed for amplitude as well as phase

Projection of the signal onto non-stationary basis functions αk(t)ejφk(t) [6]!

Model: sd(t) =

K

• k=−K

(ak + tbk)

• αk(t)ejφk(t)

w(t) complex amplitudes ak, bk: estimated via Least Squares

ˆ a ˆ b

• = (E HW HWE)−1E HW HWs

(11)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Extended Adaptive Quasi-Harmonic model - Kafentzis, Pantazis, Stylianou, Rosec, 2011

Adaptation is allowed for amplitude as well as phase

Projection of the signal onto non-stationary basis functions αk(t)ejφk(t) [6]!

Model: sd(t) =

K

• k=−K

(ak + tbk)

• αk(t)ejφk(t)

w(t) complex amplitudes ak, bk: estimated via Least Squares

ˆ a ˆ b

• = (E HW HWE)−1E HW HWs

(11) where E = [E0|E1]:

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Extended Adaptive Quasi-Harmonic model - Kafentzis, Pantazis, Stylianou, Rosec, 2011

Adaptation is allowed for amplitude as well as phase

Projection of the signal onto non-stationary basis functions αk(t)ejφk(t) [6]!

Model: sd(t) =

K

• k=−K

(ak + tbk)

• αk(t)ejφk(t)

w(t) complex amplitudes ak, bk: estimated via Least Squares

ˆ a ˆ b

• = (E HW HWE)−1E HW HWs

(11) where E = [E0|E1]: (E0)n,k = αk(tn)ejφk(tn)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Extended Adaptive Quasi-Harmonic model - Kafentzis, Pantazis, Stylianou, Rosec, 2011

Adaptation is allowed for amplitude as well as phase

Projection of the signal onto non-stationary basis functions αk(t)ejφk(t) [6]!

Model: sd(t) =

K

• k=−K

(ak + tbk)

• αk(t)ejφk(t)

w(t) complex amplitudes ak, bk: estimated via Least Squares

ˆ a ˆ b

• = (E HW HWE)−1E HW HWs

(11) where E = [E0|E1]: (E0)n,k = αk(tn)ejφk(tn) (E1)n,k = tnαk(tn)ejφk(tn) = tn(E0)n,k

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Extended Adaptive Quasi-Harmonic model

Synthetic signal:

0.05 0.1 1 1.5 2 2.5 3 Time (s) (a) AM 1 0.05 0.1 600 650 700 750 800 Time (s) (b) FM 1 (Hz) 0.05 0.1 1 1.5 2 2.5 3 Time (s) (c) AM 2 0.05 0.1 900 950 1000 1050 1100 Time (s) (d) FM 2 (Hz) True Estimated True Estimated True Estimated True Estimated

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Extended Adaptive Quasi-Harmonic model

Synthetic signal: Robustness in noise is demonstrated:

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Extended Adaptive Quasi-Harmonic model

Synthetic signal: Robustness in noise is demonstrated: MAE{ˆ θ} = 1 M

M

• i=1

|ˆ θ(i) − θ| where θ(i) is the estimated parameter at the ith simulation, and M is the number of Monte Carlo simulations.

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Extended Adaptive Quasi-Harmonic model

Synthetic signal: Robustness in noise is demonstrated: MAE{ˆ θ} = 1 M

M

• i=1

|ˆ θ(i) − θ| where θ(i) is the estimated parameter at the ith simulation, and M is the number of Monte Carlo simulations.

MAE scores and SRER SNR Model a1(t) a2(t) F1(t) F2(t) SRER(dB) ∞ aQHM 0.2380 0.1842 7.6105 9.1731 22.6 eaQHM 0.0889 0.0949 5.9217 7.0505 42.0 10 dB aQHM 0.2317 0.1860 8.6071 9.0302 10.7 eaQHM 0.1490 0.1476 8.0513 8.1022 10.9

Table: MAE scores and SRER for aQHM and eaQHM for 104 Monte

Carlo simulations.

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Extended Adaptive Quasi-Harmonic model

Real Signal:

0.1 0.2 0.3 0.4 −0.5 0.5 Time (s) Original Signal 0.1 0.2 0.3 0.4 −0.5 0.5 Time (s) aQHM Rec. Signal 0.1 0.2 0.3 0.4 −0.5 0.5 Time (s) eaQHM Rec. Signal 0.1 0.2 0.3 0.4 −0.01 0.01 Time (s) aQHM Rec. Error 0.1 0.2 0.3 0.4 −0.01 0.01 Time (s) eaQHM Rec. Error

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Adaptation algorithm for eaQHM

Let ˆ Ak(tl), ˆ fk(tl), ˆ φk(tl) denote the inst. amplitude, frequency, and phase at time instant tl of the kth component, with l = 1, · · · , L, where L is the number of frames: Initialization: (QHM) 1) ˆ f 0

k (tl) = ˆ

f 0

k (tl−1) + ρ0 2,k/2π

2) ˆ A0

k(tl) = |a0 k|, ˆ

φ0

k(tl) = ∠a0 k

3) ˆ f 0

k (tl+1) = ˆ

f 0

k (tl)

FOR adaptation i = 1, 2, · · · FOR frame l = 1, 2, · · · , L

1

Compute al

k, bl k using ˆ

φi−1

k

(t) and (11)

2

Update ˆ f i

k (tl) using (9)

3

Compute ˆ Ai

k(tl) and ˆ

φi

k(tl)

END

4

Interpolation of the parameters {ˆ Ai

k(t), ˆ

f i

k (t), ˆ

φi

k(t)}

END

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Extended Adaptive Quasi-Harmonic model

Analysis-Synthesis System Seperate speech into two parts: deterministic and stochastic

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Extended Adaptive Quasi-Harmonic model

Analysis-Synthesis System Seperate speech into two parts: deterministic and stochastic Deterministic part: a sum of non-stationary sinusoids (ea/aQHM)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Extended Adaptive Quasi-Harmonic model

Analysis-Synthesis System Seperate speech into two parts: deterministic and stochastic Deterministic part: a sum of non-stationary sinusoids (ea/aQHM) Stochastic part: time and frequency modulated (energy-based envelope and AR modeling)

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Extended Adaptive Quasi-Harmonic model

Analysis-Synthesis System Seperate speech into two parts: deterministic and stochastic Deterministic part: a sum of non-stationary sinusoids (ea/aQHM) Stochastic part: time and frequency modulated (energy-based envelope and AR modeling) Very high quality of speech signal reconstruction

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Outline

1

Parametric Techniques

2

QHM

3

iQHM

4

adaptive QHM

5

extended aQHM

6

References

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

References I

• R. J. McAulay and T. F. Quatieri,

Speech Analysis/Synthesis based on a Sinusoidal Representation IEEE Trans. on Acoust., Speech and Sig. Proc., vol. 34, pp. 744754, 1986.

• X. Serra,

A system for sound analysis/transformation/synthsis based on a determistic plus stochastic decomposition Ph.D. dissertation, Stanford University, 1989.

• Y. Stylianou,

Harmonic plus noise models for speech, combined with statistical methods, for speech and speaker modification Ph.D. dissertation, E.N.S.T - Paris, 1996.

• M. Macon,

Speech synthesis based on sinusoidal modeling Ph.D. dissertation, Georgia Institute of Technology, 1996.

• Y. Pantazis, O. Rosec, and Y. Stylianou,

Adaptive AMFM signal decomposition with application to speech analysis, IEEE Trans. on Audio, Speech, and Lang. Proc., vol. 19, pp. 290300, 2011.

• G. P. Kafentzis, Y. Pantazis, O. Rosec, and Y. Stylianou,

An Extension of the Adaptive Quasi-Harmonic Model in Proc. IEEE ICASSP, Kyoto, March 2012.

• Y. Pantazis, O. Rosec, and Y. Stylianou,

On the Properties of a Time-Varying Quasi-Harmonic Model of Speech in Interspeech, Brisbane, Sep 2008.

Outline Parametric Techniques QHM iQHM adaptive QHM extended aQHM References

Time for Questions!

Thank you for your attention! Any questions? :-)