[PPT] - Signal Representations DS-GA 1013 / MATH-GA 2824 Mathematical Tools PowerPoint Presentation

SLIDE 1

Signal Representations

DS-GA 1013 / MATH-GA 2824 Mathematical Tools for Data Science

https://cims.nyu.edu/~cfgranda/pages/MTDS_spring19/index.html Carlos Fernandez-Granda

SLIDE 2

Motivation

Limitation of frequency representation: no time resolution Limitation of Wiener filtering: cannot adapt to noisy signal

SLIDE 3

Windowing Short-time Fourier transform Multiresolution analysis Denoising via thresholding

SLIDE 4

Speech signal

0.9 1.0 1.1 1.2 1.3

Time (s)

7500 5000 2500 2500 5000 7500

SLIDE 5

Beyond Fourier

Problem: How to capture local fluctuations

SLIDE 6

Beyond Fourier

Problem: How to capture local fluctuations First segment signal, then compute DFT

SLIDE 7

Beyond Fourier

Problem: How to capture local fluctuations First segment signal, then compute DFT Naive segmentation: multiplication by rectangular window

SLIDE 8

Rectangular window

Rectangular window π ∈ CN with width 2w:

π [j] :=
1

if |j| ≤ w,

therwise.

Is this a good choice?

SLIDE 9

Signal

100 75 50 25 25 50 75 100

Time

1.00 0.75 0.50 0.25 0.00 0.25 0.50 0.75 1.00

SLIDE 10

Window

100 75 50 25 25 50 75 100

Time

0.0 0.2 0.4 0.6 0.8 1.0

SLIDE 11

Windowed signal

100 75 50 25 25 50 75 100

Time

1.00 0.75 0.50 0.25 0.00 0.25 0.50 0.75 1.00

SLIDE 12

DFT of windowed signal

100 75 50 25 25 50 75 100

Frequency

10 10 20 30 40

SLIDE 13

DFT of signal

100 75 50 25 25 50 75 100

Frequency

25 50 75 100 125 150 175 200

SLIDE 14

Multiplication in time is convolution in frequency

Let y := x1 ◦ x2 for x1, x2 ∈ CN The DFT of y equals ˆ y = 1 N ˆ x1 ∗ ˆ x2, ˆ x1 and ˆ x2 are the DFTs of x1 and x2 respectively

SLIDE 15

Proof

ˆ y [k] :=

N

j=1
x1(j)

x2(j) exp

−i2πkj

N

SLIDE 16

Proof

ˆ y [k] :=

N

j=1
x1(j)

x2(j) exp

−i2πkj

N

=

N

j=1

1 N

n

l=1

ˆ x1(l) exp i2πlj N

x2(j) exp
−i2πkj

N

SLIDE 17

Proof

ˆ y [k] :=

N

j=1
x1(j)

x2(j) exp

−i2πkj

N

=

N

j=1

1 N

n

l=1

ˆ x1(l) exp i2πlj N

x2(j) exp
−i2πkj

N

= 1

N

l=1

ˆ x1(l)

N

j=1
x2(j) exp
−i2π(k − l)j

N

SLIDE 18

Proof

ˆ y [k] :=

N

j=1
x1(j)

x2(j) exp

−i2πkj

N

=

N

j=1

1 N

n

l=1

ˆ x1(l) exp i2πlj N

x2(j) exp
−i2πkj

N

= 1

N

l=1

ˆ x1(l)

N

j=1
x2(j) exp
−i2π(k − l)j

N

= 1

N

l=1

ˆ x1(l)ˆ x ↓l

2 [k]

SLIDE 19

DFT of rectangular window

π [j] :=
1

if |j| ≤ w,

therwise,

SLIDE 20

DFT of rectangular window

ˆ π (0) =

N/2

j=−N/2+1
π [j]

=

w

j=−w

1 = 2w + 1

SLIDE 21

DFT of rectangular window

ˆ π (k) =

N/2

j=−N/2+1
x [j] exp
−i2πkj

N

SLIDE 22

DFT of rectangular window

ˆ π (k) =

N/2

j=−N/2+1
x [j] exp
−i2πkj

N

=

w

j=−w

exp

−i2πk

N j

SLIDE 23

DFT of rectangular window

ˆ π (k) =

N/2

j=−N/2+1
x [j] exp
−i2πkj

N

=

w

j=−w

exp

−i2πk

N j = exp i2πkw

N

− exp
− i2πk(w+1)

N

1 − exp
− i2πk

N

SLIDE 24

DFT of rectangular window

ˆ π (k) =

N/2

j=−N/2+1
x [j] exp
−i2πkj

N

=

w

j=−w

exp

−i2πk

N j = exp i2πkw

N

− exp
− i2πk(w+1)

N

1 − exp
− i2πk

N

=

exp

− i2πk

2N

2i sin
2πk(w+1/2)

N

exp
− i2πk

2N

2i sin

πk

N

SLIDE 25

DFT of rectangular window

ˆ π (k) =

N/2

j=−N/2+1
x [j] exp
−i2πkj

N

=

w

j=−w

exp

−i2πk

N j = exp i2πkw

N

− exp
− i2πk(w+1)

N

1 − exp
− i2πk

N

=

exp

− i2πk

2N

2i sin
2πk(w+1/2)

N

exp
− i2πk

2N

2i sin

πk

N

=

sin

2πk(w+1/2)

N

sin

πk

N

SLIDE 26

DFT of rectangular window

100 75 50 25 25 50 75 100

Frequency

20 20 40 60 80

SLIDE 27

DFT of signal

100 75 50 25 25 50 75 100

Frequency

25 50 75 100 125 150 175 200

SLIDE 28

DFT of windowed signal

100 75 50 25 25 50 75 100

Frequency

10 10 20 30 40

SLIDE 29

Hann window

100 75 50 25 25 50 75 100

Time

0.0 0.2 0.4 0.6 0.8 1.0

SLIDE 30

Hann window

The Hann window h ∈ CN of width 2w equals

h [j] :=

1

2

1 + cos
πj

w

if |j| ≤ w,
therwise

SLIDE 31

Shifting in the frequency domain

For any vector x ∈ CN, if y ∈ CN is defined as

y [j] :=

x[j] exp i2πmj N

,

then the DFT of y equals ˆ y = ˆ x↓ m, where ˆ x is the DFT of x

SLIDE 32

Proof

ˆ y [k] =

N

j=1
y [j] exp
−i2πkj

N

SLIDE 33

Proof

ˆ y [k] =

N

j=1
y [j] exp
−i2πkj

N

=

N

j=1
x[j] exp

i2πmj N

exp
−i2πkj

N

SLIDE 34

Proof

ˆ y [k] =

N

j=1
y [j] exp
−i2πkj

N

=

N

j=1
x[j] exp

i2πmj N

exp
−i2πkj

N

=

N

j=1
x[j] exp
−i2π(k − m)j

N

SLIDE 35

DFT of the Hann window

h [j] = 1

2

1 + cos

πj w

π[j]

SLIDE 36

DFT of the Hann window

h [j] = 1

2

1 + cos

πj w

π[j]

= 1 2

1 + 1

2 exp i2πNj 2wN

+ 1

2 exp

−i2πNj

2wN

π[j]

SLIDE 37

DFT of the Hann window

h [j] = 1

2

1 + cos

πj w

π[j]

= 1 2

1 + 1

2 exp i2πNj 2wN

+ 1

2 exp

−i2πNj

2wN

π[j]

ˆ h = 1 2 ˆ π + 1 4 ˆ π↓ −N/2w + 1 4 ˆ π↓ N/2w

SLIDE 38

DFT of the Hann window

30 20 10 10 20 30

Frequency

10 10 20 30 40

SLIDE 39

Signal

100 75 50 25 25 50 75 100

Time

1.00 0.75 0.50 0.25 0.00 0.25 0.50 0.75 1.00

SLIDE 40

Hann window

100 75 50 25 25 50 75 100

Time

0.0 0.2 0.4 0.6 0.8 1.0

SLIDE 41

Windowed signal

100 75 50 25 25 50 75 100

Time

1.00 0.75 0.50 0.25 0.00 0.25 0.50 0.75 1.00

SLIDE 42

DFT of signal

100 75 50 25 25 50 75 100

Frequency

25 50 75 100 125 150 175 200

SLIDE 43

DFT of Hann window

100 75 50 25 25 50 75 100

Frequency

10 20 30 40

SLIDE 44

DFT of windowed signal

100 75 50 25 25 50 75 100

Frequency

5 10 15 20

SLIDE 45

Time-frequency resolution

Time resolution governed by width of window Can we just make the window arbitrarily narrow?

SLIDE 46

Compressing in time dilates in frequency and vice versa

x ∈ L2 [−T/2, T/2] is nonzero in a band of width 2w around zero Let y be such that y(t) = x(αt), for all t ∈ [−T/2, T/2] , for some positive real number α such that w/α < T The Fourier series coefficients of y equal ˆ y [k] = 1 α

x, φk/α

SLIDE 47

Proof

ˆ y [k] = T/2

t=−T/2

y(t) exp

−i2πkt

T

dt

SLIDE 48

Proof

ˆ y [k] = T/2

t=−T/2

y(t) exp

−i2πkt

T

dt

= w/α

t=−w/α

x(αt) exp

−i2πkt

T

dt

SLIDE 49

Proof

ˆ y [k] = T/2

t=−T/2

y(t) exp

−i2πkt

T

dt

= w/α

t=−w/α

x(αt) exp

−i2πkt

T

dt

= 1 α w

τ=−w

x(τ) exp

−i2πkτ

αT

dτ

SLIDE 50

Proof

ˆ y [k] = T/2

t=−T/2

y(t) exp

−i2πkt

T

dt

= w/α

t=−w/α

x(αt) exp

−i2πkt

T

dt

= 1 α w

τ=−w

x(τ) exp

−i2πkτ

αT

dτ

= 1 α T/2

τ=−T/2

x(τ) exp

−i2πkτ

αT

dτ

SLIDE 51

w = 90

Time Frequency

100 75 50 25 25 50 75 100

Time

0.0 0.2 0.4 0.6 0.8 1.0 100 75 50 25 25 50 75 100

Frequency

20 40 60 80

SLIDE 52

w = 30

Time Frequency

100 75 50 25 25 50 75 100

Time

0.0 0.2 0.4 0.6 0.8 1.0 100 75 50 25 25 50 75 100

Frequency

5 10 15 20 25 30

SLIDE 53

w = 5

Time Frequency

100 75 50 25 25 50 75 100

Time

0.0 0.2 0.4 0.6 0.8 1.0 100 75 50 25 25 50 75 100

Frequency

1 2 3 4 5

SLIDE 54

Time-frequency resolution

Fundamental trade-off Uncertainty principle: cannot resolve in time and frequency simultaneously

SLIDE 55

Windowing Short-time Fourier transform Multiresolution analysis Denoising via thresholding

SLIDE 56

Short-time Fourier transform

1. Segment in overlapping intervals of length ℓ
2. Multiply by window vector
3. Compute DFT of length ℓ

SLIDE 57

Short-time Fourier transform

The short-time Fourier transform of x ∈ CN is STFT[ℓ]( x)[k, s] :=

x,

ψ↓ s(1−αov)ℓ

k

,

0 ≤ k ≤ ℓ − 1, 0 ≤ s ≤ N (1 − αov)ℓ,

ψk[j] :=
w[ℓ](j) exp
i2πkj

ℓ

if 1 ≤ j ≤ ℓ
therwise

Overlap between adjacent segments equals αovℓ

SLIDE 58

k = 3 s = 2 (N := 500, ℓ := 128, αov := 0.5)

Basis vector

100 200 300 400 500 Time 1.0 0.5 0.0 0.5 1.0

STFT coefficient

1 2 3 4 5 6 7 8 9

Time

10 20 30 40 50 60

Frequency

0.0 0.2 0.4 0.6 0.8 1.0

SLIDE 59

k = 3 s = 3 (N := 500, ℓ := 128, αov := 0.5)

Basis vector

100 200 300 400 500 Time 1.0 0.5 0.0 0.5 1.0

STFT coefficient

1 2 3 4 5 6 7 8 9

Time

10 20 30 40 50 60

Frequency

0.0 0.2 0.4 0.6 0.8 1.0

SLIDE 60

k = 8 s = 2 (N := 500, ℓ := 128, αov := 0.5)

Basis vector

100 200 300 400 500 Time 1.0 0.5 0.0 0.5 1.0

STFT coefficient

1 2 3 4 5 6 7 8 9

Time

10 20 30 40 50 60

Frequency

0.0 0.2 0.4 0.6 0.8 1.0

SLIDE 61

k = 23 s = 6 (N := 500, ℓ := 128, αov := 0.5)

Basis vector

100 200 300 400 500 Time 1.0 0.5 0.0 0.5 1.0

STFT coefficient

1 2 3 4 5 6 7 8 9

Time

10 20 30 40 50 60

Frequency

0.0 0.2 0.4 0.6 0.8 1.0

SLIDE 62

Matrix representation of STFT

         

F[ℓ]

· · · · · ·

F[ℓ]

· · · · · ·

F[ℓ]

· · · · · · · · · · · · · · · · · · · · · · · · · · ·                     diag

w[ℓ]
· · ·

· · · diag

w[ℓ]
· · ·

· · · diag

w[ℓ]
· · ·

· · · · · · · · · · · · · · · · · ·          

x,

SLIDE 63

Computing the STFT

Number of segments: nseg := N/(1 − αov)ℓ Complexity of multiplication by window: Complexity of applying DFT: Total complexity:

SLIDE 64

Computing the STFT

Number of segments: nseg := N/(1 − αov)ℓ Complexity of multiplication by window: nsegℓ Complexity of applying DFT: Total complexity:

SLIDE 65

Computing the STFT

Number of segments: nseg := N/(1 − αov)ℓ Complexity of multiplication by window: nsegℓ Complexity of applying DFT: nsegℓ log ℓ (FFT) Total complexity:

SLIDE 66

Computing the STFT

Number of segments: nseg := N/(1 − αov)ℓ Complexity of multiplication by window: nsegℓ Complexity of applying DFT: nsegℓ log ℓ (FFT) Total complexity: O(N log ℓ) (overlap is a fixed fraction)

SLIDE 67

Inverting the STFT

Apply inverse DFT to each segment Combine segments Same complexity

SLIDE 68

Speech signal (window length = 62.5 ms)

1 2 3 4 5 6

Time (s)

10000 5000 5000 10000

SLIDE 69

Speech signal (window length = 62.5 ms)

1 2 3 4 5 6

Time (s)

1000 2000 3000 4000

Frequency (Hz)

10

3

10

2

10

1

100 101 102 103

SLIDE 70

Windowing Short-time Fourier transform Multiresolution analysis Denoising via thresholding

SLIDE 71

Image

SLIDE 72

Vertical line (column 135)

100 200 300 400 500 0.2 0.4 0.6 0.8 1.0

SLIDE 73

Multiresolution analysis

Scale / resolution at which information is encoded is not uniform Goal: Decompose signals into components at different resolutions

SLIDE 74

Multiresolution decomposition

Let N := 2K for some K, a multiresolution decomposition of CN is a sequence of nested subspaces VK ⊂ VK−1 ⊂ . . . ⊂ V0 satisfying:

◮ V0 = CN ◮ Vk is invariant to translations of scale 2k for 0 ≤ k ≤ K. If

x ∈ Vk then

x ↓ 2k ∈ Vk

for all l ∈ Z

◮ For any

x ∈ Vj that is nonzero only between 1 and N/2, the dilated vector x↔2 belongs to Vj+1

SLIDE 75

Dilation

Let x ∈ CN be such that x[j] = 0 for all j ≥ N/M, where M is a positive integer The dilation of x by a factor of M is

x↔M[j] =

x j M

SLIDE 76

How to build a multiresolution decomposition

◮ Set the coarsest subspace to be spanned by a low-frequency vector

ϕ, called a scaling vector or father wavelet VK := span ( ϕ) .

SLIDE 77

How to build a multiresolution decomposition

◮ Decompose the finer subspaces into the direct sum

Vk := Wk ⊕ Vk+1, 0 ≤ k ≤ K − 1, where Wk captures the finest resolution available at level k

◮ Set Wk to be spanned by shifts of a vector

µ dilated to have the appropriate resolution: Vk := Wk ⊕ Vk+1, 0 ≤ k ≤ K − 1, WK−1 :=

N−1 2k+1

m=0

span

µ ↓ m2k+1

↔2k

.

The vector µ is called a mother wavelet

SLIDE 78

Challenge

How to choose mother and father wavelets? If chosen appropriately, basis vectors can be orthonormal

SLIDE 79

Haar wavelet basis

The Haar father wavelet ϕ is a constant vector, such that

ϕ[j] :=

1 √ N , 1 ≤ j ≤ N The mother wavelet µ satisfies

µ[j] :=

       − 1

√ 2,

j = 1,

1 √ 2,

j = 2, 0, j > 2 Other options: Meyer, Daubechies, coiflets, symmlets, etc.

SLIDE 80

Haar wavelets

Scale Basis functions 20

SLIDE 81

Haar wavelets

Scale Basis functions 20

SLIDE 82

Haar wavelets

Scale Basis functions 20

SLIDE 83

Haar wavelets

Scale Basis functions 20 21

SLIDE 84

Haar wavelets

Scale Basis functions 20 21

SLIDE 85

Haar wavelets

Scale Basis functions 20 21

SLIDE 86

Haar wavelets

Scale Basis functions 20 21 22

SLIDE 87

Haar wavelets

Scale Basis functions 20 21 22

SLIDE 88

Haar wavelets

Scale Basis functions 20 21 22 23

SLIDE 89

Haar wavelets

Scale Basis functions 20 21 22 23

SLIDE 90

Multiresolution decomposition

VK

SLIDE 91

Multiresolution decomposition

VK W2

SLIDE 92

Multiresolution decomposition

VK W2 W1

SLIDE 93

Multiresolution decomposition

VK W2 W1 W0 PVk x is an approximation of x at scale 2k

SLIDE 94

Vertical line (column 135)