Lecture 21: Wiener Filter Mark Hasegawa-Johnson All content CC-SA - - PowerPoint PPT Presentation

Review Derivation Uncorrelated Noise and Signal Expectation Summary

Lecture 21: Wiener Filter

Mark Hasegawa-Johnson All content CC-SA 4.0 unless otherwise specified. ECE 401: Signal and Image Analysis, Fall 2020

Review Derivation Uncorrelated Noise and Signal Expectation Summary

1

Review: Wiener Filter

2

An Alternate Derivation of the Wiener Filter

3

Wiener Filter for Uncorrelated Noise and Signal

4

How can you compute Expected Value?

5

Summary

Review Derivation Uncorrelated Noise and Signal Expectation Summary

Outline

1

Review: Wiener Filter

2

An Alternate Derivation of the Wiener Filter

3

Wiener Filter for Uncorrelated Noise and Signal

4

How can you compute Expected Value?

5

Summary

Review Derivation Uncorrelated Noise and Signal Expectation Summary

Wiener’s Theorem and Parseval’s Theorem

Wiener’s theorem says that the power spectrum is the DTFT

• f autocorrelation:

rxx[n] = 1 2π π

−π

Rxx(ω)ejωndω Parseval’s theorem says that energy in the time domain is the average of the energy spectrum:

∞

• n=−∞

x2[n] = 1 2π π

−π

|X(ω)|2dω

Review Derivation Uncorrelated Noise and Signal Expectation Summary

Filtered Noise

If y[n] = h[n] ∗ x[n], x[n] is any signal, then ryy[n] = rxx[n] ∗ h[n] ∗ h[−n] Ryy(ω) = Rxx(ω)|H(ω)|2

Review Derivation Uncorrelated Noise and Signal Expectation Summary

The Wiener Filter

Y (ω) = E [Rsx(ω)] E [Rxx(ω)]X(ω) = E [S(ω)X ∗(ω)] E [X(ω)X ∗(ω)]X(ω) The numerator, Rsx(ω), makes sure that y[n] is predicted from x[n] as well as possible (same correlation, E [ryx[n]] = E [rsx[n]]). The denominator, Rxx(ω), divides out the noise power, so that y[n] has the same expected power as s[n].

Review Derivation Uncorrelated Noise and Signal Expectation Summary

Power Spectrum and Cross-Power Spectrum

Remember that the power spectrum is defined to be the Fourier transform of the autocorrelation: Rxx(ω) = lim

N→∞

1 N |X(ω)|2 rxx[n] = lim

N→∞

1 N x[n] ∗ x[−n] In the same way, we can define the cross-power spectrum to be the Fourier transform of the cross-correlation: Rsx(ω) = lim

N→∞

1 N S(ω)X ∗(ω) rsx[n] = lim

N→∞

1 N s[n] ∗ x[−n]

Review Derivation Uncorrelated Noise and Signal Expectation Summary

Outline

1

Review: Wiener Filter

2

An Alternate Derivation of the Wiener Filter

3

Wiener Filter for Uncorrelated Noise and Signal

4

How can you compute Expected Value?

5

Summary

Review Derivation Uncorrelated Noise and Signal Expectation Summary

An Alternate Derivation of the Wiener Filter

The goal is to design a filter h[n] so that y[n] = x[n] ∗ h[n] in order to make y[n] as much like s[n] as possible. In other words, let’s minimize the mean-squared error: E =

∞

• n=−∞

E

• (s[n] − y[n])2

Review Derivation Uncorrelated Noise and Signal Expectation Summary

Use Parseval’s Theorem!

In order to turn the convolutions into multiplications, let’s use Parseval’s theorem! E =

∞

• n=−∞

E

• (s[n] − y[n])2

= 1 2π π

−π

E

• |S(ω) − Y (ω)|2

dω = 1 2π π

−π

E

• |S(ω) − H(ω)X(ω)|2

dω E = 1 2π π

−π

(E [S(ω)S∗(ω)] − H(ω)E [X(ω)S∗(ω)] − E [S(ω)X ∗(ω)] H∗(ω) + H(ω)E [X(ω)X ∗(ω)] H∗(ω)) dω Now let’s try to find the minimum, by setting dE dH(ω) =0

Review Derivation Uncorrelated Noise and Signal Expectation Summary

Differentiate and Solve!

Differentiating by H(ω) (and pretending that H∗(ω) stays constant), we get dE dH(ω) = −E [X(ω)S∗(ω)] dω + E [X(ω)X ∗(ω)] H∗(ω)dω So we can set

dE dH(ω) = 0 if we choose

H∗(ω) = E [X(ω)S∗(ω)] E [|X(ω)|2]

• r, equivalently,

H(ω) = E [S(ω)X ∗(ω)] E [|X(ω)|2] = E [Rsx(ω)] E [Rxx(ω)]

Review Derivation Uncorrelated Noise and Signal Expectation Summary

Outline

1

Review: Wiener Filter

2

An Alternate Derivation of the Wiener Filter

3

Wiener Filter for Uncorrelated Noise and Signal

4

How can you compute Expected Value?

5

Summary

Review Derivation Uncorrelated Noise and Signal Expectation Summary

What is X made of?

So here’s the Wiener filter: H(ω) = E [S(ω)X ∗(ω)] E [|X(ω)|2] But now let’s break it down a little. What’s X? That’s right, it’s S + V — signal plus noise. H(ω) = E [S(ω)(S∗(ω) + V ∗(ω))] E [|X(ω)|2] = E

• |S(ω)|2

+ E [S(ω)V ∗(ω)] E [|X(ω)|2] = E [Rss(ω)] + E [Rsv(ω)] E [Rxx(ω)]

Review Derivation Uncorrelated Noise and Signal Expectation Summary

What if S and V are uncorrelated?

In most real-world situations, the signal and noise are uncorrelated, so we can write E [S(ω)V ∗(ω)] = E [S(ω)] E [V ∗(ω)] = 0

Review Derivation Uncorrelated Noise and Signal Expectation Summary

What if S and V are uncorrelated?

Similarly, if S and V are uncorrelated, E

• |X(ω)|2

= E

• |S(ω) + V (ω)|2

= E

• |S(ω)|2

+ E [S(ω)V ∗(ω)] + E [S∗(ω)V (ω)] + E

• |V (ω)|2

= E

• |S(ω)|2

+ E

• |V (ω)|2

Review Derivation Uncorrelated Noise and Signal Expectation Summary

Wiener Filter in the General Case In the general case, the Wiener Filter is H(ω) = E [Rsx(ω)] E [Rxx(ω)] = E [Rss(ω)] + E [Rsv(ω)] E [Rss(ω)] − E [Rsv(ω)] − E [Rvs(ω)] + E [Rvv(ω)] Wiener Filter for Uncorrelated Noise If noise and signal are uncorrelated, H(ω) = E [Rss(ω)] E [Rxx(ω)] = E [Rss(ω)] E [Rss(ω)] + E [Rvv(ω)]

Review Derivation Uncorrelated Noise and Signal Expectation Summary

Wiener Filter in the General Case

H(ω) = E [Rsx(ω)] E [Rxx(ω)] In the general case, the numerator captures the correlation between the noisy signal, x[n], and the desired clean signal s[n]. The idea is to give y[n] the same correlation. We can’t make y[n] equal s[n] exactly, but we can give it the same statistical properties as s[n]: specifically, make it correlate with x[n] the same way.

Review Derivation Uncorrelated Noise and Signal Expectation Summary

Wiener Filter for Correlated Noise

H(ω) = E [Rss(ω)] E [Rxx(ω)] If s[n] and v[n] are uncorrelated, then the correlation between the clean and noisy signals is exactly equal to the autocorrelation of the clean signal: E [rsx[n]] = E [rss[n]] So in that case, the Wiener filter is just exactly the desired, clean power spectrum, E [Rss(ω)], divided by the given, noisy power spectrum E [Rxx(ω)],

Review Derivation Uncorrelated Noise and Signal Expectation Summary

Outline

1

Review: Wiener Filter

2

An Alternate Derivation of the Wiener Filter

3

Wiener Filter for Uncorrelated Noise and Signal

4

How can you compute Expected Value?

5

Summary

Review Derivation Uncorrelated Noise and Signal Expectation Summary

How can you compute expected value?

Finally: we need to somehow estimate the expected power spectra, E [Rss(ω)] and E [Rxx(ω)]. How can we do that? Generative model: if you know where the signal came from, you might have a pencil-and-paper model of its statistics, from which you can estimate Rss(ω). Multiple experiments: If you have the luxury of running the experiment 1000 times, that’s actually the best way to do it. Welch’s method: chop the signal into a large number of small frames, computing |X(ω)|2 from each small frame, and then average. As long as the signal statistics don’t change

• ver time, this method works well.

Review Derivation Uncorrelated Noise and Signal Expectation Summary

Pros and Cons of Welch’s Method Con: Because each |X(ω)|2 is being computed from a shorter window, you get less spectral resolution. Pro: Actually, less spectral resolution is usually a good thing. Micro-variations in the spectrum are probably noise, and should probably be smoothed away.

Public domain image, 2016, Bob K, https://commons.wikimedia.org/wiki/File: Comparison_of_periodogram_and_Welch_methods_of_spectral_density_estimation.png

Review Derivation Uncorrelated Noise and Signal Expectation Summary

Outline

1

Review: Wiener Filter

2

An Alternate Derivation of the Wiener Filter

3

Wiener Filter for Uncorrelated Noise and Signal

4

How can you compute Expected Value?

5

Summary

Review Derivation Uncorrelated Noise and Signal Expectation Summary

Summary

Wiener Filter in the General Case: H(ω) = E [Rsx(ω)] E [Rxx(ω)] Wiener Filter for Uncorrelated Noise: H(ω) = E [Rss(ω)] E [Rxx(ω)] Welch’s Method: chop the signal into frames, compute |X(ω)|2 for each frame, and then average them.