AIMS CDT - Signal Processing Michaelmas Term 2020 Xiaowen Dong - - PowerPoint PPT Presentation

AIMS CDT - Signal Processing

Michaelmas Term 2020

Xiaowen Dong

Department of Engineering Science

• Lectures
• Monday-Thursday 10:00-12:00
• slides: http://www.robots.ox.ac.uk/~xdong/teaching.html
• Lab sessions
• Tuesday & Wednesday 14:00-17:00
• notes: http://www.robots.ox.ac.uk/~xdong/teaching.html
• lab demonstrators
• Henry Kenlay (henry.kenlay@worc.ox.ac.uk)
• Alan Chau (siu.chau@spc.ox.ac.uk)
• Yin-Cong Zhi (yin-cong.zhi@st-annes.ox.ac.uk)
• A light-weight assignment
• to be submitted to xdong@robots.ox.ac.uk by Monday Oct 26th 18:00
• Questions & Comments
• Xiaowen Dong (xdong@robots.ox.ac.uk)

/76

Outline

2

• Part I Classical signal processing
• Day 1 Introduction: Basic concepts and tools (thanks to Steve Roberts)
• linear systems, convolution, time-frequency analysis
• filtering, analogue & digital filters
• Day 2 Representation of signals
• time-frequency representation
• Fourier & wavelet transforms, dictionary learning
• Part II Graph signal processing (GSP)
• Day 3 Introduction to GSP
• graph signals, graph Fourier transform
• filtering and representation of graph signals
• Day 4 Graph neural networks (Guest lecture by Dorina Thanou)
• graph convolution, localisation
• spatial-domain vs spectral-domain designs

/76

Outline

3

/76

Textbooks (Part I)

4

• Lynn. An introduction to the analysis and

processing of signals. Macmillan, 1989.

• Oppenheim and Schafer. Digital signal
• processing. Prentice Hall, 1975.
• Proakis and Manolakis. Digital signal processing:

Principles, algorithms and applications. Prentice Hall, 2007

• Orfanidis. Introduction to signal processing.

Prentice Hall, 1996. Available online at http://eceweb1.rutgers.edu/~orfanidi/intro2sp/

/76

Textbooks (Part I)

5

• Vetterli et al. Foundations of signal processing.

Cambridge University Press, 2014. Available

• nline at http://www.fourierandwavelets.org
• Kovačević et al. Fourier and wavelet signal
• processing. Available online at

http://www.fourierandwavelets.org

• MATLAB Signal Processing Toolbox:
• https://www.mathworks.com/help/signal/
• SciPy Signal Processing Toolbox:
• https://docs.scipy.org/doc/scipy/reference/tutorial/signal.html
• https://scipy-cookbook.readthedocs.io/items/idx_signal_processing.html

Toolboxes

/76 6

Introduction

Basic Concepts and Tools

/76

Historical notes

7 Morse code (1830s) electronic communication (today) semaphore telegraph (1792) smoke signal tower (1570)

/76

Modern signal processing applications

8 speech processing image denoising EEG signal classification seismology

/76

Signal types

9

time continuous discrete amplitude discrete continuous analogue signal digital signal sampling quantisation

t T t T t t

/76

Analogue vs. Digital signal processing

• Many signals of practical interest are analogue: e.g., speech, seismic,

radar, and sonar signals

• Analogue signal processing systems are based on analogue equipment:

e.g., channel vocoder

10

• Dramatic advance of digital computing moves the

trend towards digital systems

/76

Linear systems

• Principle of superposition

11 system

ax1(t) + bx2(t) ay1(t) + by2(t)

Fout ⊆ Fin

sub- system sub- system

x

sub- system sub- system

x

system

x ≡ ≡

• Frequency preservation:
• Can be broken down into simpler sub-systems

t t system

/76

Time-invariant systems

• Time-invariance

12 system system

x(t − t0) y(t − t0) x(t) y(t)

t t t t

t0 t0

/76

Linear time-invariant (LTI) systems

• Linear time-invariant (LTI) systems are both linear and time-invariant

13

y(t) = x(t) − x(t − 1) y(t) = x(t + 1) − x(t) y(t) = x(t) − x(t − 1) y(t) = x(t) − x(t − 1)

y(t) = 1 x(t)

• Causality: “present” only depends on “present” and “past”
• Stability: a system is bounded-input bounded-output (BIBO) stable if

y(t) = x(2t)

y(t) = [x(t)]2

|y(t)| ≤ My < ∞

|x(t)| ≤ Mx < ∞

/76

Linear processes

• Input-output characteristics can be defined by
• impulse response in the time domain
• transfer function in the frequency domain
• There is an invertible mapping between time- and frequency-domain

representations

14

input LTI system

• utput

amplification, (un)-mixing, filtering, etc.

/76

Time-domain analysis - Convolution

• Convolution allows the evaluation of the output signal from an LTI

system, given its impulse response and input signal

15 system t t

• Evaluate system output for
• input: succession of impulse functions (which generate weighted impulse responses)
• utput: sum of the effect of each impulse function

g(t) δ(t)

system t

τ

1 2δ(t − τ) 1 2g(t − τ)

t

τ

/76

Time-domain analysis - Convolution

16

20 40 60 80 100 0.02 0.04 0.06 0.08 0.1 components 20 40 60 80 100 0.05 0.1 0.15 0.2 0.25 total 20 40 60 80 100 0.05 0.1 0.15 0.2 0.25 2 4 6 0.1 0.2 0.3 0.4

20 40 60 input signal impulse response

/76

Time-domain analysis - Convolution

17 system t t

... ...

y(t) = X

τ

{x(τ)dτ}g(t − τ)

dτ→0

− → Z ∞ x(τ)g(t − τ)dτ

• this gives the convolution integral

τ

x(t)

τ

{x(τ)dτ}g(t − τ)

{x(τ)dτ}δ(t)

• the system response is the convolution of the input and the impulse response

x(τ)dτ

dτ

• the system is completely characterised by impulse response in time-domain

/76

Time-domain analysis - Convolution

• Convolution is commutative

18

y(t) = Z ∞ x(τ)g(t − τ)dτ = Z ∞ x(t − τ)g(τ)dτ f(t) = Z ∞

−∞

x(τ)g(t − τ)dτ Rxy(τ) = Z ∞

−∞

x(t)y(t − τ)dt

integral over lags at a fixed time integral over time for a fixed lag

• Convolution vs. Correlation

/76

Frequency-domain analysis

• Consider the following LTI system

19

y(t) = Z ∞

−∞

esτg(t − τ)dτ = Z ∞

−∞

g(t)e−stdt · est

x(t) = est

g(t)

y(t) G(s)

est

• is an eigenfunction of an LTI system with eigenvalue , an integral

that involves the impulse response and complex constant G(s)

• knowledge of for all completely characterises the system

G(s)

s s

g(t)

/76

The Laplace transform

• Laplace transform of
• Transfer function

20

X(s)

G(s)

Y (s) = G(s)X(s)

x(t)

G(s)

G(s) = A(s − z1) . . . (s − zm) (s − p1)(s − p2) . . . (s − pn) X(s) = Z ∞

−∞

x(t)e−stdt

• Can be expressed as a pole-zero representation of the form

pole at infinity ( ) if zero at infinity ( ) if n > m

n < m

G(∞) = ∞ G(∞) = 0

/76

The Laplace transform and LTI system

21

s-plane

σ

jω

σ > −a

−a

region of convergence (ROC):

a > 0

where

• causal system: if the ROC extends

rightward from the rightmost pole and

• stable system: ROC includes the

imaginary axis

• causal and stable system: all poles must

be in the left-half of the s-plane

n ≥ m

G(s) = Z ∞

−∞

g(t)e−stdt = Z ∞

−∞

g(t)e−σte−jωtdt < ∞

g(t) = e−atu(t)

G(s) = 1 s + a

/76

The Fourier transform

• Laplace transform reduces to Fourier transform with
• Transfer function reduces to frequency response

22

s = jω

X(jω) = Z ∞

−∞

x(t)e−jωtdt

G(jω)

G(jω) X(jω)

Y (jω) = G(jω)X(jω)

y(t) = 1 2π Z ∞

−∞

Y (jω)ejωtdω

• Inverse Fourier transform

/76

The Fourier transform

• Fourier series for periodic signal

23 image credit: https://community.sw.siemens.com/s/article/what-is-the-fourier-transform

• When the period approaches infinity, the spectrum becomes continuous

leading to Fourier transform for aperiodic signal (previous slide)

/76

Laplace vs. Fourier transform

24

Laplace transform Fourier transform

exists when integral converges (may exist even if FT doesn’t) exists when integral converges transfer function complex s-plane imaginary axis of complex s-plane

X(s) = Z ∞

−∞

x(t)e−stdt X(jω) = Z ∞

−∞

x(t)e−jωtdt

frequency response

/76

Time-domain vs. Frequency-domain

• Theorem
• Proof Consider , by convolution:

25

x(t) = A cos ωt

y(t) = Z ∞ Acosω(t − τ)g(τ)dτ = A 2 Z ∞ ejω(t−τ)g(τ)dτ + A 2 Z ∞ e−jω(t−τ)g(τ)dτ = A 2 ejωt Z ∞

−∞

g(τ)e−jωτdτ + A 2 e−jωt Z ∞

−∞

g(τ)ejωτdτ = A 2 {ejωtG(jω) + e−jωtG(−jω)}

If is the impulse response of an LTI system, then its Fourier transform, , is the frequency response of the system

g(t) G(jω)

/76

g(t)

Time-domain vs. Frequency-domain

26

Let , i.e.,

G(jω) = Cejφ C = |G(jω)|, φ = arg{G(jω)}

then y(t) = AC

2 {ej(ωt+φ) + e−j(ωt+φ)} = CA cos(ωt + φ)

that is, an input sinusoid has its amplitude scaled by and phase changed by , where is the Fourier transform of the impulse response .

|G(jω)|

arg{G(jω)}

G(jω)

/76

Time-domain vs. Frequency-domain

• Theorem
• Proof

27

Convolution in the time domain is equivalent to multiplication in the frequency domain, i.e.,

y(t) = g(t) ∗ x(t) ≡ F−1{Y (jω) = G(jω)X(jω)} y(t) = g(t) ∗ x(t) ≡ L−1{Y (s) = G(s)X(s)} L{f(t) ∗ g(t)} = Z

t

Z

τ

f(t − τ)g(τ)dτe−stdt = Z

τ

g(τ)e−sτdτL{f(t)} = L{g(t)}L{f(t)}

By letting we prove the result for the Fourier transform.

s = jω

/76

Time-domain vs. Frequency-domain

• We can move losslessly between time and frequency domains, choosing

whichever is the easier to work with

• Convolution theorem provides the mathematical underpinning that helps

guarantee stability and properties of linear systems such as filters

28

• stable system: ROC extends rightward

from the rightmost pole and n ≥ m s-plane

σ

jω

−a

• low-pass system: frequency response can

be analysed by drawing vectors from poles and zeros to imaginary axis

jω2

jω3

jω1

/76

Filtering

• Filtering as input-output relationship

29

input LTI system

• utput

g(t)

y(t)

X(s)

G(s) G(jω) X(jω)

x(t)

impulse response transfer function frequency response

/76

Filtering

• Filters are frequency-selective linear systems
• Low-pass: extract short-term average or to eliminate high-frequency fluctuations
• High-pass: follow small-amplitude high-frequency perturbations in presence of

much larger slowly-varying component

• Band-pass: select a required modulated carrier frequency out of many
• Band-stop: eliminate single-frequency interference (also known as notch filtering)

30

low-pass high-pass band-pass notch ω G( ) ω | |

|G(jω)|

/76

Design of analogue filters

• A filter may be described by its impulse response or by its frequency

response (or transfer function)

31

• Design procedure
• consider some desired response as a ratio of two polynomials in even

powers of

• design the filter by assigning the “stable” poles to (remember the condition
• n location of poles for stability!)

|G(s)|2 s

G(s)

|G(s)|2 = G(s)G∗(s)

• Squared magnitude of the transfer function

G(s)G(−s)

s = jω

• Filter design takes into account
• the desired magnitude response
• the desired phase response

/76

Analogue vs. Digital filters

32

• Analogue filters
• constructed from analogue circuit

components (e.g., resistors, inductors, capacitors)

• Digital filters
• “hardware” form: set of digital (logic)

electronic circuits

• “software” form: program a general-purpose

digital computer

/76

Digital filtering

• Can be easily (re-)programmed to implement a number of different filters
• Accuracy only depends on round-off error in the arithmetic
• hence is predictable and performance known a priori
• can meet very tight specifications on frequency response
• Widespread use of mini- and micro-computers increased digital signals

stored and processed

• Robust against noise and change in external environment (e.g., power

supply issues, temperature variations)

33

/76

Digital filtering

• Digital filtering can be done in
• time domain: convolution with the impulse response
• frequency domain: multiplication by the desired filter characteristics

34

impulse response pulse transfer function

x[k] g[k] y[k] = x[k] ∗ g[k] X(z) G(z)

Y (z) = X(z)G(z)

/76

The sampling process

35 t

continuous

radians sec cycle sec (Hz) radians sample cycle sample

sampling

sample sec sec sample T t

discrete

xa(t) = Acos(2πfat + φ) = Acos(ωat + φ)

= Acos(2πfdn + φ) = Acos(wdn + φ)

xd(n) = Acos(2πfanT + φ) = Acos(2π fa fs n + φ)

−1 2fs = − 1 2T ≤ fa ≤ 1 2T = 1 2fs

Nyquist frequency

/76

Aliasing

36

f(t) = cos(5π 2 t T )

1 2 3 4 5 6 7 8 9 10 −1 −0.5 0.5 1 1 2 3 4 5 6 7 8 9 10 −1 −0.5 0.5 1

aliasing frequencies: Hz

k = 1, 2, ...

( )

T T T T T T T T T T T T T T T T T T T T T T

π π fa = 0.25 ± k T

/76

Aliasing

• Sampling in time results in repeated spectrum in frequency

37 (from lecture notes by David Murray)

/76

Digital filtering and reconstruction

38

A/D converter D/A converter input anti-aliasing (anal. LPF) suppress freq. above 1

2fs

digital filter passband well below 1

2fs

samples DAC LPF

step inter- polation (staircase approx.)

• utput

smoothing (anal. LPF) (recovery filter)

/76

Digital filtering as regression

• Noise reduction: Polynomial fit using least-squares

39

• 2 -1 0 1 2
• 2 -1 0 1 2

parabolic fit centre point, k=0

/76

Parabolic fit

40

k = {−2, −1, 0, 1, 2}

p[k] = s0 + ks1 + k2s2

for approximation error:

E(s0, s1, s2) =

2

X

k=−2

(x[k] − [s0 + ks1 + k2s2])2

coefficients of the fit

∂E ∂s0 = 0 ∂E ∂s1 = 0 ∂E ∂s2 = 0

5s0 + 10s2 =

k=2

X

k=−2

x[k] 10s1 =

k=2

X

k=−2

kx[k] 10s0 + 34s2 =

k=2

X

k=−2

k2x[k]

s0 = 1 35(−3x[−2] + 12x[−1] + 17x[0] + 12x[1] − 3x[2])

s1 = 1 10(−2x[−2] − x[−1] + x[1] + 2x[2]) s2 = 1 14(2x[−2] − x[−1] − 2x[0] − x[1] + 2x[2])

/76

Parabolic fit

41

• 2 -1 0 1 2

parabolic fit centre point, k=0

• the parabola coefficient is the filtering output
• it provides a smoothed approximation of each set of five data points

s0

p[k] |k=0 = s0 + ks1 + k2s2 |k=0= s0 = 1 35(−3x[−2] + 12x[−1] + 17x[0] + 12x[1] − 3x[2])

/76

Parabolic fit

42

10 20 30 40 50 60 70 80 90 100 −0.5 0.5 1 1.5

100 200 300 400 500 600 700 0.2 0.4 0.6 0.8 1 f |H(z)| fsamp / 2

Fig 2.5: Noisy data (thin line) and 5-point parabolic filtered (thick line). Fig 2.6: Frequency response of the 5-point parabolic filter.

|G(jω)|

2π/T π/T

ω

/76

Parabolic fit

• The parabolic filter we just considered is
• a low-pass filter (LPF)
• a non-recursive filter:

43

y[k] =

N

X

i=0

aix[k − i]

delay by 2T y[k] = 1 35(−3x[k + 2] + 12x[k + 1] + 17x[k] + 12x[k − 1] − 3x[k − 2]) y[k] = 1 35(−3x[k] + 12x[k − 1] + 17x[k − 2] + 12x[k − 3] − 3x[k − 4])

• a non-causal filter:

/76

Impulse response of digital filters

44

y[k] =

N

X

i=0

aix[k − i]

Finite-Impulse Response (FIR): Infinite-Impulse Response (IIR):

y[k] =

N

X

i=0

aix[k − i] y[k] =

N

X

i=0

aix[k − i] +

M

X

i=1

biy[k − i]

• The equation represents a discrete convolution of

the input data with the filter coefficients

x[k] = ( 1, if k = 0 0,

• therwise

Then y[k] =

X

i

aix[k − i] = akx[0] = ak

Proof Let

• Theorem The coefficients constitute the impulse response of the filter.

recursive!

/76

The z-transform

• The z-transform is important in digital filtering
• it describes frequency-domain properties of discrete (sampled) data
• it is similar to the Laplace transform in analogue filtering

45 1T t 2T kT 0T

...

z = eT s

• Consider the Laplace transform of a discrete function as a succession of

impulses

z may be thought of as a shift operator

= eσT · ejωT

f(0) f(1) f(2)

f(k)

Fd(s) = f(0) + f(1)e−T s + f(2)e−2T s + . . . + f(k)e−kT s +...

F(z) = f(0) + f(1)z−1 + f(2)z−2 + . . . f(k)z−k +...

/76

The z-transform

• For many functions, the infinite series can be represented in “closed-form”

as the ratio of two polynomials in

46

f[k] = ( 0, if k < 0 1, if k ≥ 0

F(z) = 1 + z−1 + z−2 + . . . + z−k + . . . = 1 1 − z−1

z−1

step function decaying exponential

(|z−1| < 1)

f(t) = e−αt − → f[k] = e−αkT

F(z) = 1 + e−αT z−1 + e−α2T z−2 + . . . + e−αkT z−k + . . . = 1 1 − e−αT z−1

sinusoid

F(z) = 1 2( 1 1 − ejωT z−1 + 1 1 − e−jωT z−1 ) = 1 − cos ωTz−1 1 − 2 cos ωTz−1 + z−2

f(t) = cos ωt − →

f[k] = cos kωT = ejkωT + e−jkωT 2

/76

Pulse transfer function (PTF)

• PTF is z-transform of output divided by z-transform of input

47 1 n 2 3 N

... a1 a2

a3 aN

a0

1 n 2 3

...

x[0] x[1] x[2] x[3] X(z) = x[0] + x[1]z−1 + x[2]z−2 + . . . + x[k]z−k + . . .

x[n]

• Consider an input
• Consider an FIR filter with the impulse response

G(z) = a0 + a1z−1 + a2z−2 + . . . + aiz−i + . . . + aNz−N

/76

Pulse transfer function (PTF)

48

G(z)X(z)

in which the coefficient for is

z−k

this is again a discrete convolution that gives the output , and therefore : similar to the transfer function! the product

y[k]

Y (z) = G(z)X(z)

X(s)

G(s)

transfer function pulse transfer function

Y (s) = G(s)X(s) Y (z) = G(z)X(z) X(z) G(z)

G(z)X(z) = (a0 + a1z−1 + . . . + aiz−i + . . . + aNz−N)(x[0] + x[1]z−1 + . . . + x[k]z−k + . . .)

a0x[k] + a1x[k − 1] + . . . + aix[k − i] + . . . + aNx[k − N]

/76

Pulse transfer function (PTF)

• PTF is the z-transform of impulse response
• For non-recursive filters
• For recursive filters

49

Y (z) =

N

X

i=0

aiz−iX(z) +

M

X

i=1

biz−iY (z) y[k] =

N

X

i=0

aix[k − i]

y[k] =

N

X

i=0

aix[k − i] +

M

X

i=1

biy[k − i]

G(z) = Y (z) X(z) = PN

i=0 aiz−i

1 − PM

i=1 biz−i

G(z) =

N

X

i=0

aiz−i

/76

The z-transform and LTI system

50

z-plane

region of convergence (ROC):

where

• causal system: if the ROC extends outward

from the outmost pole

• stable system: ROC includes the unit circle
• causal and stable system: all poles must

be inside the unit circle

low frequency high frequency

G(z) =

∞

X

−∞

x[n]z−n < ∞ x[n] = anu[n]

X(z) = 1 1 − az−1

|z| > |a|

z = ejωT

a

0 < a < 1

/76

Mapping from s-plane to z-plane

51

z = esT s = σ + jω z = eσT ejωT

(from lecture slides by Mark Cannon)

imaginary axis ( )

σ = 0

unit circle ( )

|z| = 1

left-half plane ( )

σ < 0

|z| < 1

inside unit circle ( )

σ > 0

|z| > 1

right-half plane ( )

• utside unit circle ( )

poles in left-half plane for stability poles inside unit circle for stability

/76

Example

• What’s the condition for the following filter to be stable?

52

y[k] = x[k − 1] + αy[k − 1]

Y (z)(1 − αz−1) = z−1X(z) G(z) = Y (z) X(z) = z−1 1 − αz−1 = 1 z − α

hence for the filter to be stable we need .

|α| < 1

/76

Frequency response of a digital filter

• Theorem
• Proof Consider the general form of a digital filter

53

The frequency response of a digital filter can be obtained by evaluating the PTF on the unit circle ( ) y[k] =

∞

X

i=0

aix[k − i] consider an input sampled at

z = ejωT

cos(ωt + θ)

t = 0, T, . . . , kT

therefore

x[k] = cos(ωkT + θ) = 1 2{ej(ωkT +θ) + e−j(ωkT +θ)}

/76

Frequency response of a digital filter

54

y[k] = 1 2

∞

X

i=0

aiej{ω[k−i]T +θ} + 1 2

∞

X

i=0

aie−j{ω[k−i]T +θ} = 1 2ej(ωkT +θ)

∞

X

i=0

aie−jωiT + 1 2e−j(ωkT +θ)

∞

X

i=0

aiejωiT

N.B.

∞

X

i=0

aie−jωiT =

∞

X

i=0

ai(ejωT )−i =

∞

X

i=0

aiz−i|z=ejωT = G(z)|z=ejωT let

G(z)|z=ejωT = Aejφ

then

∞

X

i=0

aiejωiT = Ae−jφ hence

y[k] = 1 2ej(ωkT +θ)Aejφ + 1 2e−j(ωkT +θ)Ae−jφ y[k] = A cos(ωkT + θ + φ)

• r

x[k] = cos(ωkT + θ)

while thus and represent the gain and phase of the frequency response, i.e., the frequency response is .

A

φ

G(z)|z=ejωT

/76

Example

55

G(z) = 1 z − 0.8

G(z)|z=ejωT = 1 ejωT − 0.8

z-plane

0.5 1 1.5 2 2.5 3 3.5 1 2 3 4 5

ωT

|G(ejωT )|

z = ejω1T

z = ejω2T

z = ejω3T

ωT = π

ωT = 0

z = 1

z = −1

/76

Example

• Consider the 5-point parabolic filter

56

y[k] = 1 35(−3x[k] + 12x[k − 1] + 17x[k − 2] + 12x[k − 3] − 3x[k − 4]) Y (z) = 1 35(−3 + 12z−1 + 17z−2 + 12z−3 − 3z−4)X(z) G(z)|z=ejωT = 1 35(−3 + 12e−jωT + 17e−2jωT + 12e−3jωT − 3e−4jωT ) = 1 35e−2jωT (17 + 24cosωT − 6cos2ωT)

/76

Example

57

therefore

|G(ejωT )| = 1 35|17 + 24 cos ωT − 6 cos 2ωT|

ωT = 0 → |G(ejωT )| = 1

∠G(ejωT ) = −2ωT (linear-phase - all frequencies delayed by 2T)

ωT ≈ 0.48π (i.e., f/fs = 0.24) → |G(ejωT )| = 0.707

3dB cut-off

G(z)|z=ejωT = 1 35e−2jωT (17 + 24cosωT − 6cos2ωT)

/76

Design of digital filters

• Three basic steps
• specification of desired frequency response
• approximation of the specification using a causal discrete-time system
• realisation of the system using finite-precision arithmetic

58

passband transition stopband

• Different design techniques for FIR and IIR filters

/76

Continuous vs. Discrete system

59

linear differential equation (impulse response)

continuous discrete

linear difference equation (impulse response) Laplace transform (transfer function) z-transform (pulse transfer function) Frequency response (imaginary axis -> Fourier transform) Frequency response (unit circle -> discrete- time Fourier transform) analogue filter digital filter convolution integral convolution sum

/76

Discrete Fourier transform

• We have introduced Fourier series (FS) for continuous periodic signals
• In digital signal processing and filtering we need to deal with discrete-time

signals - what about the discrete-time counterparts of FS and FT?

60

X(jω) = Z ∞

−∞

x(t)e−jωtdt y(t) = 1 2π Z ∞

−∞

Y (jω)ejωtdω and Fourier transform (FT) for continuous aperiodic signals

/76

Discrete Fourier transform

• Discrete Fourier transform (DFT) is the equivalent of continuous Fourier

transform

• DFT is a transform for discrete sequence
• the spectrum is only evaluated at discrete values in frequency

61

• Consider the Fourier transform of a discrete function as a sampled version
• f an underlying continuous function

t 1T 2T (N-1)T 0T

...

f[0] f[1] f[N − 1] f[2] f(t)

F(jω) = Z (N−1)T f(t)e−jωtdt = f[0]e−j0 + f[1]e−jωT + . . . + f[N − 1]e−jω(N−1)T =

N−1

X

k=0

f[k]e−jωkT

/76

Discrete Fourier transform

• The sequence has a continuous and periodic spectrum

62

π

• this is called the discrete-time

Fourier transform (DTFT)

• we can only evaluate it at

discrete values in frequency (points on the unit circle) f[k] F(j(ω + 2π T )) =

N−1

X

k=0

f[k]e−j(ω+ 2π

T )kT = F(jω)

ω1T

evaluation of

F(jω1)

/76

Discrete Fourier transform

• Continuous Fourier transform could be evaluated over one period if the

waveform was periodic: this leads to the Fourier series

63

where

harmonically related complex exponentials evaluation at n-th harmonic

f0 = 1/T0

fundamental frequency

f(t) =

∞

X

n=−∞

Cnej2πnf0t Cn = 1 T0 Z T0/2

−T0/2

f(t)e−j2πnf0tdt

(from lecture notes by David Murray)

/76

Discrete Fourier transform

• Deriving DFT: 1) treat the finite sequence as if it were periodic

64

1 2 3 4 5 6 7 8 9 10 11 0.2 0.4 0.6 0.8 1 (a) 5 10 15 20 25 30 0.2 0.4 0.6 0.8 1 (b)

/76

Discrete Fourier transform

• Deriving DFT: 2) evaluate DTFT at the harmonics in the discrete case

65

π

• we only have a finite number
• f such harmonics

2π N

n = 0, 1, ..., N − 1

ej2πnf0k = ej2π n

N k

2π × (N − 1) N 2π × 2 N

for

• equi-spaced on the unit

circle and the n-th harmonic has the following form

F(jω) =

N−1

X

k=0

f[k]e−jωkT

ωT =2π n

N

− − − − − − → F[n] =

N−1

X

k=0

f[k]e−j2π n

N k

/76

Discrete Fourier transform

• This gives the discrete Fourier transform (DFT) of the sequence

66

n = 0, 1, ..., N − 1

for compare: Cn = 1

T0 Z T0/2

−T0/2

f(t)e−j2π n

T0 tdt

F[n] =

N−1

X

k=0

f[k]e−j2π n

N k

for

n ∈ (−∞, ..., ∞)

f[k]

• the periodic coefficients form the discrete-time Fourier series (DTFS) for the

periodic sequence

• Fundamental assumption of periodicity behind the DFT
• the input sequence is treated as if it were periodic
• the coefficients are themselves periodic

f[k]

F[n]

• the N-point input and N DFT coefficients above correspond to one period of the

underlying DTFS transform pair

/76

Discrete Fourier transform

• Interpretation: A finite sequence of length N can be represented by

samples of its z-transform at N equi-spaced points on the unit circle

67

π

F(z)|z=ej2π n

N

=

∞

X

k=−∞

f[k]e−j2π n

N k

=

N−1

X

k=0

f[k]e−j2π n

N k

what happens if we sample L>N or L<N points?

/76

Discrete Fourier transform

• From L sample points we recover a periodic sequence:
• if L>N, this is equivalent to padding zeros to original sequence

68

π

t N L t N

• does NOT provide new information but “better display” of spectrum

˜ f[k] =

∞

X

r=−∞

f[k + rL]

/76

Discrete Fourier transform

• From L sample points we recover a periodic sequence:
• if L<N, one period of is shorter than original

69

π

t N L t N L

• cannot recover original sequence due to time-domain aliasing

t N L

˜ f[k] =

∞

X

r=−∞

f[k + rL]

˜ f[k]

f[k]

/76

Discrete Fourier transform

70

n = 0, 1, ..., N − 1

for

F[n] =

N−1

X

k=0

f[k]e−j2π n

N k

W = e−j 2π

N

W N = W 2N = ... = 1 where and

/76

Example

• Consider the following signal

71

f(t) = 5 |{z} dc + 2 cos(2πt − 90o) | {z } 1Hz + 3 cos 4πt | {z } 2Hz

1 2 3 4 5 6 7 8 9 10 −4 −2 2 4 6 8 10

sample at Hz

fs = 4

f[k] = 5 + 2cos(π 2 k − 90o) + 3cosπk

t = kT = k/4 sec

f[0] = 8, f[1] = 4, f[2] = 8, f[3] = 0 (N = 4)

1s 2s 0s

/76

Example

• Therefore

72

F[n] =

3

X f[k]e−j π

2 nk =

3

X

k=0

f[k](−j)nk

1 2 3 5 10 15 20 f (Hz) |F[n]|

/76

Inverse discrete Fourier transform

73

F[n] =

N−1

X

k=0

f[k]e−j2π n

N k

f[k] = 1 N

N−1

X

n=0

F[n]e+j 2π

N nk

DFT: IDFT:

• for real signal, coefficients form complex conjugate pairs: F[N − n] = F ∗(n)

F[n] F[N − n] f[k]

fn[k] = 2 N |F[n]| cos{(2π n NT )kT + arg(F[n])}

sampled sinewave at Hz and of magnitude

n NT 2 N |F[n]|

• contribution of and together to :

/76

Interpretation of example

74

f(t) = 5 |{z} dc + 2 cos(2πt − 90o) | {z } 1Hz + 3 cos 4πt | {z } 2Hz

/76

The Fourier transform - Four different forms

75

[Proakis] pp. 270

Fourier Series Discrete-Time Fourier Series Discrete-Time Fourier Transform Fourier Transform (DFT)

/76

Summary

• LTI systems are of central importance to modern signal processing
• Time- and frequency-domain representations of the system are equivalent;

such equivalence is established by the convolution theorem

• Frequency-selective filters are one of the most important signal processing

tools

• The DFT, which represents a finite sequence with finite number of

coefficients, plays a central role in digital signal processing and filtering

76