Adaptive Filters Adaptation Control Gerhard Schmidt - - PowerPoint PPT Presentation

Slide 1

Gerhard Schmidt

Christian-Albrechts-Universität zu Kiel Faculty of Engineering Electrical Engineering and Information Technology Digital Signal Processing and System Theory

Adaptive Filters – Adaptation Control

Slide 2 Slide 2 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Today:

Contents of the Lecture

Adaptation Control:

 Introduction and Motivation  Prediction of the System Distance  Optimum Control Parameters  Estimation Schemes  Examples

Slide 3 Slide 3 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Basics

Application Example – Echo Cancellation

e(n) +

x(n) d(n) s(n) b(n) b d(n)

Echo cancel- lation filter y(n) e(n) +

x(n) b d(n)

y(n) + +

s(n) b(n)

Application example: Model:

d(n)

Objective:

Remove those components in the microphone signal that originate from the remote communication partner!

h(n) b h(n)

x(n) b h(n)

Slide 4 Slide 4 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Basic Approach

Application Example – Echo Cancellation

Cancelling acoustic echoes by means of an adaptive filter with coefficients, operating at a sample rate kHz. For the adaptation of the filter the NLMS algorithm should be used.

Model:

The loudspeaker-enclosure-microphone (LEM) system is modelled as a linear (only slowly changing) system with finite memory.

Approach: Advantages and disadvantages:

In contrast to former approaches (loss controls) simultaneous speech activity in both communication directions is possible now. The NLMS algorithm is a robust and computationally efficient approach. Compared to former solutions more memory and a larger computational load are required. Stability can not be guaranteed. + + _ _

Slide 5 Slide 5 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

NLMS-Algorithm

Application Example – Echo Cancellation

Computation of the error signal (output signal of the echo cancellation filter): Recursive computation of the norm of the excitation signal vector Adaptation of the filter vector:

Slide 6 Slide 6 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Convergence Examples – Part 1

Application Example – Echo Cancellation

Convergence without background noise and without local speech signals

Excitation signal Local signal Error signal Microphone and error power Time in seconds Microphone signal

Slide 7 Slide 7 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Convergence Examples – Part 2

Application Example – Echo Cancellation

Excitation signal Local signal Error signal Microphone and error power Time in seconds Microphone signal

Convergence with background noise but without local speech signals

Slide 8 Slide 8 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Convergence Examples – Part 3

Application Example – Echo Cancellation

Excitation signal Local signal Error signal Microphone and error power Time in seconds Microphone signal

Convergence without background noise but with local speech signals (step size = 1)

Slide 9 Slide 9 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Convergence Examples – Part 4

Application Example – Echo Cancellation

Excitation signal Local signal Error signal Microphone and error power Time in seconds Microphone signal

Convergence without background noise but with local speech signals (step size = 0.1)

Slide 10 Slide 10 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Literature

Adaptation Control

 E. Hänsler / G. Schmidt: Acoustic Echo and Noise Control – Chapter 7

(Algorithms for Adaptive Filters), Wiley, 2004

 E. Hänsler / G. Schmidt: Acoustic Echo and Noise Control – Chapter 13

(Control of Echo Cancellation Systems), Wiley, 2004

Basic texts: Further details:

 S. Haykin: Adaptive Filter Theory – Chapter 6 (Normalized Least-Mean-Square Adaptive

Filters), Prentice Hall, 2002

 C. Breining, A. Mader: Intelligent Control Strategies for Hands-Free Telephones, in

• E. Hänsler, G. Schmidt, Topics on Acoustic Echo and Noise Control – Chapter 8,

Springer, 2006

Slide 11 Slide 11 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Control Approaches – Part 1

Adaptation Control

Scalar control approach: Vector control approach:

Regularization Step size

Slide 12 Slide 12 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Control Approaches – Part 2

Adaptation Control

Example for a sparse impulse response For such systems a vector based control scheme can be advantageous.

Impulse response of the system to be identified Example for a vector step size Coefficient index i

Slide 13 Slide 13 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

How do we go on …

Adaptation Control

Problem (echo cancellation performance during „double talk“) Analysis of the average system distance (taking local signals into account) Derivation of an optimal step size (using non-measurable signals) Estimation of the non-measurable signal components (leads to an implementable control scheme) Solution (robust echo cancellation due to step-size control)

Slide 14 Slide 14 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Average System Distance – Part 1

Adaptation Control

 Adaptation using the NLMS algorithm (only step-size controlled) :

Assumptions:

 White noise as excitation and (stationary) distortion:  Statistical independence between filter vector

and excitation vector.

Definition of the average system distance:

e(n) +

x(n) b d(n)

y(n) +

s(n) b(n)

d(n)

b h(n)

+ n(n) h

 Time-invariant system:

Slide 15 Slide 15 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Average System Distance – Part 2

Adaptation Control

… Derivation during the lecture …

Slide 16 Slide 16 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Average System Distance – Part 3

Adaptation Control

Contraction parameter Expansion parameter

Generic approach (control scheme with step size and regularization): Result:

Slide 17 Slide 17 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Contraction and Expansion Parameters

Adaptation Control

Contraction parameter :

 Range:  Desired: as small as possible  Determines the speed of convergence without distortions

Expansion parameter :

 Range:  Desired: as small as possible  Determines the robustness against distortions

Opposite to each

• ther – a common

solution (optimization) Has to be found!

Slide 18 Slide 18 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Influence of the Control Parameters

Adaptation Control

Values for the contraction and expansion parameters for the conditions:

 

Contraction parameter Step size Step size Step size Regularization Regularization Regularization Step size Expansion parameter Regularization

Slide 19 Slide 19 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

True and Prediction System Distance

Adaptation Control

Boundary conditions of the simulation:

 Excitation: white noise  Distortion: white noise  SNR: 30 dB

System distance Distortion Excitation Iterations

(Simulation) (Simulation) (Theory) (Theory)

Slide 20 Slide 20 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Maximum Convergence Speed – Part 1

Adaptation Control

For the special case without any distortions and with optimal control parameters for that case we get Meaning that the average system distance can be reduced per adaptation step by a factor of . As a result adaptive filters with a lower amount of coefficients converge faster than long adaptive filters.

Slide 21 Slide 21 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Maximum Convergence Speed – Part 2

Adaptation Control

If we want to know how long it takes to improve the filter convergence by 10 dB, we can make the following ansatz: As on the previous slide we assumed an undisturbed adaptation process. By applying the natural logarithm we obtain By using the following approximations for and we get This means: At maximum speed of convergence it takes about 2N iterations until the average system distance is reduced by 10 dB.

Slide 22 Slide 22 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

The „10 dB per 2N“ Rule

Adaptation Control

Boundary conditions of the simulation:

 Excitation: white noise  Distortion: white noise  SNR: 30 dB  Step size: 1  Different filter lengths

(500 and 1000)

Average system distance Average system distance Iterations

Slide 23 Slide 23 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Prediction of the Steady-State Convergence – Part 1

Adaptation Control

Recursion of the average system distance: For and appropriately chosen control parameters we obtain:

Slide 24 Slide 24 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Prediction of the Steady-State Convergence – Part 2

Adaptation Control

By inserting the results from the previous slide we obtain: For the adaptation without regularization we get: Inserting these values leads to:

Slide 25 Slide 25 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Optimal Step Size – Motivation

Adaptation Control

Remarks:

With a large step size

• ne can achieve a fast

initial convergence, but

• nly a poor steady-state

performance. With a small step size a good steady-state performance can be

• btained, but only a

slow initial convergence.

Solution:

Utilization of a time- variant step-size.

Estimated speed

• f convergence

Average system distance (only step-size control) Estimated steady- state performance Iterations

Slide 26 Slide 26 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Optimal Step Size – Derivation

Adaptation Control

… Derivation during the lecture …

Slide 27 Slide 27 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Optimal Step Size – Example

Adaptation Control

Computation of the step size:

with

Boundary conditions of the simulation:

 Excitation: white noise  Distortion: white noise  SNR: 30 dB  Filter length: 1000 coefficients

Iterations Time-variant step size Average system distance

Step size = 1 Step size = 0.5 Step size = 0.25 Time-variant step size

Slide 28 Slide 28 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Estimation of the Optimal Step Size

Adaptation Control

Approximation for the optimal step size: For white excitation we get: Ansatz:

Short-term power of the error signal Short-term power of the excitation signal Estimated system distance

Slide 29 Slide 29 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Estimation Procedures for the Optimal Step Size – Part 1

Adaptation Control

First order IIR smoothing with different time constants for rising and falling signal edges: Basic structure:

Different time constants are used to achieve smoothing on one hand but also being able to follow sudden signal increments quickly on the other hand.

Slide 30 Slide 30 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Estimation Procedures for the Optimal Step Size – Part 2

Adaptation Control

Boundary conditions of the simulation:

 Excitation: speech  SNR: about 20 dB   Sample rate: 8 kHz

¯r = 0:007; ¯f = 0:002

Microphone signal Estimated short-term power Time in seconds

Slide 31 Slide 31 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Estimation Procedures for the Optimal Step Size – Part 3

Adaptation Control

Estimating the system distance: Problem:

The coefficients are not known.

Solution:

We extend the system by an artificial delay of samples. For that part of the impulse response we have With these so-called delay coefficients we can extrapolate the system distance: for

Slide 32 Slide 32 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Estimation Procedures for the Optimal Step Size – Part 4

Adaptation Control

Structure of the system distance estimation:

Slide 33 Slide 33 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Estimation Procedures for the Optimal Step Size – Part 5

Adaptation Control

„Error spreading property“

• f the NLMS algorithm:

System error vector (magnitudes) after 0 iterations System error vector (magnitudes) after 500 iterations System error vector (magnitudes) after 2000 iterations System error vector (magnitudes) after 4000 iterations Coefficient index

Slide 34 Slide 34 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Estimation Procedures for the Optimal Step Size – Part 6

Adaptation Control

Boundary conditions of the simulation:

 Excitation: speech  Distortion: speech  SNR during single talk:

30 dB

 Filter length:

1000 coefficients

Excitation signal Local speech signal Measured and estimated system distance Step size Iterations

Measured system distance Estimated system distance

Slide 35 Slide 35 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Convergence Examples – Part 5

Application Example – Echo Cancellation

For Comparison:

Fixed step size 1.0 Fixed step size 0.1

Excitation signal Local signal Error signal Microphone and error power Time in seconds Microphone signal

Slide 36 Slide 36 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Convergence Examples – Part 6

Application Example – Echo Cancellation

Controlled step size Fixed step size (0.1) Fixed step size (1.0) Microphone signal

Time in seconds Short-term microphone and error power

Slide 37 Slide 37 Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Summary and Outlook

Adaptive Filters – Adaptation Control

This week:

 Introduction and Motivation  Prediction of the System Distance  Optimum Control Parameters  Estimation Schemes  Examples

Next week:

 Reducing the Computational Complexity of Adaptive Filters