EI331 Signals and Systems Lecture 11 Bo Jiang John Hopcroft Center - - PowerPoint PPT Presentation

EI331 Signals and Systems

Lecture 11 Bo Jiang

John Hopcroft Center for Computer Science Shanghai Jiao Tong University

April 2, 2019

1/34

Contents

• 1. Mean-square Convergence of Fourier Series
• 2. Pointwise Convergence of Fourier Series
• 3. Fourier Series of Periodic Impulse Train
• 4. Filtering

2/34

Best Mean-square Approximation

Given orthonormal system {e1, . . . , en}, best mean-square approximation of x by element in span{e1, . . . , en}, i.e. by element of form y =

n

• k=1

akek is y =

n

• k=1

x, ekek, with minimum mean-square error x − y2 = x2 −

n

• k=1

|x, ek|2.

3/34

Geometry of Best Mean-square Approximation

Orthogonal projection onto subspace span{e1, . . . , en} e1 e2 O x y x, e1e1 x, e2e2 x − y span{e1, e2} Pythagorean theorem x2 =

n

• k=1

|x, ek|2 + x − y2

4/34

Convergence in Normed Vector Space

In normed vector space (V, · ), sequence {xn} converges to x ∈ V if lim

n→∞ xn − x = 0

We say x is the limit of {xn} and write x = lim

n→∞ xn,

• r

xn → x, as n → ∞. Sequence {xn} ⊂ V is Cauchy sequence iff xn − xm → 0, as n, m → ∞.

• Theorem. Every convergent sequence is Cauchy.
• Proof. If xn → x, triangle inequality yields

xn − xm ≤ xn − x + xm − x → 0, as n, m → ∞.

5/34

Banach Space

Normed vector space (V, · ) is complete if every Cauchy sequence converges. Complete normed vector space is called Banach space. Examples of Banach spaces

• (Rn, · p), (Cn, · p)
• DT signal space ℓp = {x ∈ CZ : xp < ∞} with ℓp norm
• CT signal space Lp = {x ∈ CR : xp < ∞} with Lp norm
• Space L2(T) of CT signals with period T and finite

average power, x2 = 1

T

• T |x(t)|2dt < ∞
• Space C(T) of continuous T-periodic CT signals with L∞

norm x∞ = sup

t∈[0,T]

|x(t)| = sup

t∈R

|x(t)|. xn → x in L∞ norm means uniform convergence

6/34

Banach Space

Examples of incomplete spaces

• (Q, | · |). Its completion is (R, | · |)
• Space C(T) of continuous T-periodic CT signals with L2
• norm. Its completion is L2(T)

◮ periodic odd function with period T = 1 xn =      nt, 0 ≤ t ≤ 1

n

1,

1 n ≤ 1 2 − 1 n

1 − nt,

1 2 − 1 n ≤ t ≤ 1 2

◮ xn ∈ C(1) ◮ xn converges to periodic square wave in L2 norm ◮ but periodic square function not in C(1)

Same vector space with different norms becomes different normed vector spaces, e.g (C(T), · ∞) vs. (C(T), · 2)

7/34

Incompleteness of C(T) with L2 Norm

xn t xn(t)

1 1 −1

1 n

x t x(t)

1 1 −1

xn − x2

2 =

• 1

n

− 1

n

+

• 1

2+ 1 n 1 2− 1 n

• |xn(t) − x(t)|2dt ≤ 4

n → 0 x ∈ L2(1) but x / ∈ C(1). Also xn − x∞ = 1, ∀n

8/34

L2 Convergence vs. Pointwise Convergence

For n ≥ 0 and 0 ≤ k < 2n, x2n+k =

• 1,

k 2n ≤ t < k+1 2n

0,

• therwise

n = 0, k = 0

t

1 1 n = 1, k = 0

t

1 1 n = 1, k = 1

t

1 1 n = 2, k = 0

t

1 1 n = 2, k = 1

t

1 1 n = 2, k = 2

t

1 1 n = 2, k = 3

t

1 1

• x2n+k → 0 in L2 norm

x2n+k − 02 = 1 2n/2 → 0

• not convergent at any t0 ∈ [0, 1)

◮ ∀n, ∃k1 s.t x2n+k1(t0) = 1 ◮ ∀n, ∃k0 s.t x2n+k0(t0) = 0

9/34

Hilbert Space

Inner product space (V, ·, ·) is also normed vector space with induced norm x =

• x, x.

(V, ·, ·) is complete if (V, · ) with induced norm is complete. Complete inner produce space is called Hilbert space. Examples of Hilbert spaces

• Rn with x, y = n

k=1 xkyk; Cn with x, y = n k=1 xk¯

yk

• Space ℓ2 with x, y = ∞

k=−∞ xk¯

yk

• Space L2 with x, y =
• R x(t)y(t)dt
• Space L2(T) with x, y = 1

T

• T x(t)y(t)dt

Example of incomplete space

• Space C(T) with x, y = 1

T

• T x(t)y(t)dt

L2(T) is completion of C(T).

10/34

Complete Orthonormal Sequence

Orthonormal sequence {ek : k ∈ N} in Hilbert space H is complete if every x ∈ H has expansion x =

∞

• k=1

x, ekek i.e. lim

n→∞ x − n

• k=1

x, ekek = 0 Complete orthonormal sequence also called orthonormal basis of H.

• Example. {δk : k ∈ Z} is orthonormal basis of ℓ2

11/34

Parseval’s Identity

If {ek : k ∈ N} orthonormal basis of H, x2 =

∞

• k=1

|x, ek|2

• Proof. By Pythagorean theorem

x2 =

n

• k=1

|x, ek|2 + x −

n

• k=1

x, ekek2 Let n → ∞ and last term goes to zero by definition of

• rthonormal basis.

12/34

Parseval’s Identity

H is isomorphic to ℓ2, x ↔ {x, ek : k ∈ N} and x, y =

∞

• k=1

x, eky, ek Proof. x, y =

• k

x, ekek,

• m

y, emem =

• k

x, ek

• m

y, emek, em =

• k

x, ek

• m

y, emδ[k − m] =

• k

x, eky, ek

• NB. When x = y, we recover x2 = ∞

k=1 |x, ek|2

13/34

Mean-square Convergence of Fourier Series

• Theorem. {ejk 2π

T t : k ∈ Z} is orthonormal basis of L2(T).

For any x ∈ L2(T), Fourier series converges in mean-square, i.e. in L2 norm lim

N→∞ x − SN(x) = 0

• NB. Convergence in mean-square does not imply pointwise

convergence Parseval’s identity

∞

• k=−∞

|ˆ x[k]|2 = x2 = 1 T

• T

|x(t)|2dt Interpretation: Energy conservation

• |ˆ

x[k]|2 is average power of k-th harmonic component

• x2 is average power of x

14/34

Fourier Series of L2 Signals

Correspondence between periodic functions and doubly infinite sequences; time domain vs. frequency domain x

FS

← − − → ˆ x

• r

x(t)

FS

← − − → ˆ x[k] Fourier series is isomorphism between Hilbert space of L2 signals and Hilbert space of ℓ2 Fourier coefficients FS : L2(T) → ℓ2 x → ˆ x Will see discrete-time Fourier transform goes from ˆ x to x Parseval’s theorem x, yL2(T) = ˆ x,ˆ yℓ2

15/34

Contents

• 1. Mean-square Convergence of Fourier Series
• 2. Pointwise Convergence of Fourier Series
• 3. Fourier Series of Periodic Impulse Train
• 4. Filtering

16/34

Integral Representation of Partial Sum

WLOG, focus on T = 2π and ω0 = 1 SN(x)(t) =

N

• k=−N

ˆ x[k]ejkt =

N

• k=−N

1 2π

• 2π

x(τ)e−jkτdτ

• ejkt

= 1 2π

• 2π

x(τ)

N

• k=−N

ejk(t−τ)dτ = 1 2π

• 2π

x(τ)DN(t − τ)dτ = (x ∗ DN 2π )(t) where DN is Dirichlet kernel DN(t) =

N

• k=−N

ejkt = sin((N + 1

2)t)

sin(t/2)

17/34

Dirichlet Kernel

Plot of DN

2π for various N

t N = 4 t N = 8 t N = 16 If DN

2π → δ (more precisely, periodic impulse train on slide 22),

then would have SN(x) = x ∗ DN

2π → x ∗ δ = x at least for

continuous x Unfortunately, exists continuous x whose Fourier partial sum SN(x), as N → ∞, fails to converge at all at some t, let alone converges to x(t) at such t

18/34

Pointwise Convergence

• Theorem. If x is differentiable, then lim

N→∞ SN(x)(t) = x(t), ∀t.

• Proof. Fix t. Define

F(τ) =    x(t − τ) − x(t) sin(τ/2) , τ = 0 −2x′(t), τ = 0 which is continuous on [−π, π]. Note

1 2π

π

−π DN(τ)dτ = 1.

SN(x)(t) − x(t) = 1 2π π

−π

x(t − τ)DN(τ)dτ − 1 2π π

−π

x(t)DN(τ)dτ = 1 2π π

−π

[x(t − τ) − x(t)]DN(τ)dτ = 1 2π π

−π

F(τ) sin

• (N + 1

2)τ

• dτ

19/34

Pointwise Convergence

• Theorem. If x is differentiable, then lim

N→∞ SN(x)(t) = x(t).

Proof (cont’d). Define y(τ) = e−j τ

2 F(τ).

SN(x)(t) − x(t) = 1 2π π

−π

F(τ) sin

• (N + 1

2)τ

• dτ

= −Im 1 2π π

−π

y(τ)e−jNτdτ = −Imˆ y[N] Being continuous on [−π, π], y(τ) ∈ L2(2π). Bessel’s inequality implies ˆ y[N] → 0, so SN(x)(t) − x(t) = −Imˆ y[N] → 0, as N → ∞

20/34

Dirichlet Test

• Theorem. If x satisfies following Dirichlet conditions
• 1. x is bounded, i.e. x∞ < ∞,
• 2. x is piecewise monotone, i.e. ∃ finite partition of a period

s.t. x is monotone on each segment, then lim

N→∞ SN(x)(t) = x(t+) + x(t−)

2 Example. t x(t)

1 1 −1

lim

N→∞ SN(x)(t) =

     1, t ∈ (n, n + 1

2),

−1, t ∈ (n − 1

2, n),

0, t = n

2,

, n ∈ Z

21/34

Contents

• 1. Mean-square Convergence of Fourier Series
• 2. Pointwise Convergence of Fourier Series
• 3. Fourier Series of Periodic Impulse Train
• 4. Filtering

22/34

Periodic Impulse Train

x(t) =

∞

• k=−∞

δ(t − kT) t x(t)

1 −4T −3T −2T −1T 0T 1T 2T 3T 4T

For “good” enough function φ, e.g. infinitely differentiable function with compact support, (φ ∗ x)(t) =

∞

• k=−∞

φ(t − kT)

• R

φ(t)x(t)dt =

∞

• k=−∞
• R

φ(t)δ(t − kT)dt =

∞

• k=−∞

φ(kT)

23/34

Periodic Impulse Train

x(t) =

∞

• k=−∞

δ(t − kT) t x(t)

1 −4T −3T −2T −1T 0T 1T 2T 3T 4T

Fourier coefficients ˆ x[k] = 1 T T/2

−T/2

δ(t)e−jk 2π

T tdt = 1

T Fourier series x(t) =

∞

• k=−∞

1 T ejk 2π

T t = lim

N→∞

1 T DN 2π T t

• where DN is Dirichlet kernel

24/34

Periodic Impulse Train

x(t) =

∞

• k=−∞

δ(t − kT) =

∞

• k=−∞

1 T ejk 2π

T t

t x(t)

1 −4T −3T −2T −1T 0T 1T 2T 3T 4T

k ˆ x[k]

1 T

−4 −3 −2 −1 1 2 3 4

25/34

Periodic Impulse Train

• R

φ(t)x(t)dt = lim

N→∞

1 T

• R

φ(t)DN

• −2π

T t

• dt

(DN is even) = lim

N→∞

1 2π

• R

φ

• − T

2πt

• DN(t)dt

= lim

N→∞ ∞

• k=−∞

1 2π (2k+1)π

(2k−1)π

φ

• − T

2πt

• DN(t)dt

=

∞

• k=−∞

lim

N→∞

1 2π π

−π

φ T 2π(2πk − t)

• DN(t − 2kπ)dt

=

∞

• k=−∞

lim

N→∞

1 2π π

−π

φ T 2π(2πk − t)

• DN(t)dt

=

∞

• k=−∞

lim

N→∞ SN(φ)(kT) = ∞

• k=−∞

φ(kT)

(DN is 2π-periodic)

26/34

Relation with Periodic Square Wave

t r(t)

1

− T

2 T 2

−T1 T1 −T T −2T 2T

t r′(t)

1 −1

r′(t) = x(t + T1) − x(t − T1)

• r′[k] = (ejkω0T1 − e−jkω0T1)ˆ

x[k] = 2j sin(kω0T1) T ˆ r[k] = 1 jkω0

• r′[k] = sin(kω0T1)

kπ , k = 0

27/34

Contents

• 1. Mean-square Convergence of Fourier Series
• 2. Pointwise Convergence of Fourier Series
• 3. Fourier Series of Periodic Impulse Train
• 4. Filtering

28/34

Frequency Response

Recall response of CT LTI system to exponential input est T

• k

akeskt

• =
• k

akH(sk)eskt where H(s) is system function H(s) =

• R

h(t)e−stdt When restricted to s = jω, H(jω) as function of ω is called frequency response of the system H(jω) =

• R

h(t)e−jωtdt

29/34

Filtering

Periodic input x in Fourier series representation x(t) =

∞

• k=−∞

ˆ x[k]ejkω0t Output of LTI system with frequency response H(jω) y(t) = T

• ∞
• k=−∞

ˆ x[k]ejkω0t

• =

∞

• k=−∞

H(jkω0)ˆ x[k]ejkω0t periodic with same periodic, Fourier coefficients related by ˆ y[k] = H(jkω0)ˆ x[k] Filtering changes relative amplitudes of frequency components

• r eliminates some frequency components entirely

30/34

Filtering

ω |H(jω)| ω arg H(jω) − π

2 π 2

• Example. Differentiator T(x) = x′

has frequency response H(jω) = jω For periodic input x(t) =

∞

• k=−∞

ˆ x[k]ejkω0t

• utput

y(t) =

∞

• k=−∞

jkω0ˆ x[k]ejkω0t Fourier coefficients related by ˆ y[k] = jkω0ˆ x[k]

31/34

Filtering

LTI systems as filters

• cannot create new frequency components
• can only scale magnitudes or shift phases of existing

components Example of nonlinear filter: Clipping y(t) = max{x(t), c} Focus on LTI filters in this course Frequency-shaping filters change shape of spectrum

• e.g. equalizer

Frequency-selective filters pass some frequencies essentially undistorted and significantly attenuate or eliminate others

32/34

Ideal Frequency-selective Filters

Ideal lowpass filter H(jω) =

• 1,

|ω| ≤ ωc 0,

• therwise

ωc: cutoff frequency ω H(jω) 1 −ωc ωc

passband stopband stopband

33/34

Ideal Frequency-selective Filters

Ideal highpass filter H(jω) =

• 1,

|ω| ≥ ωc 0,

• therwise

ωc: cutoff frequency ω H(jω) 1 −ωc ωc

stopband passband passband

34/34

Ideal Frequency-selective Filters

Ideal bandpass filter H(jω) =

• 1,

ωc1 ≤ |ω| ≤ ωc2 0,

• therwise

ωc1: lower cutoff frequency ωc2: upper cutoff frequency ω H(jω) 1 −ωc2 −ωc1 ωc1 ωc2