[PPT] - Some Essentials of Data Analysis with Wavelets Slid Slides for the PowerPoint Presentation

SLIDE 1

Some Essentials of Data Analysis with Wavelets

Slid f h l l f h i d l i Th Slides for the wavelet lectures of the course in data analysis at The Swedish National Graduate School of Space Technology

Niklas Grip, Department of Mathematics, Luleå University of Technology

Last update: 2009-11-12

SLIDE 2

f(x) a0(x) 1 1 x

SLIDE 3 0 6 0.8 1 (x) 0 6 0.8 1 (x)

0.4
0.2

0.2 0.4 0.6

0.4
0.2

0.2 0.4 0.6

0.5

0.5 1 1.5

1
0.8
0.6

x

0.5

0.5 1 1.5

1
0.8
0.6

x

Old approximation Old approximation New approximation

f(x) a1(x)

1

1/2 1 x

x

SLIDE 4 0 6 0.8 1 (x) 0 6 0.8 1 (x)

0.4
0.2

0.2 0.4 0.6

0.4
0.2

0.2 0.4 0.6

0.5

0.5 1 1.5

1
0.8
0.6

x

0.5

0.5 1 1.5

1
0.8
0.6

x

f(x) a2(x) Old approximation New approximation 1/4 2/4 3/4 1 1/2 1 x x

SLIDE 5 0 6 0.8 1 (x) 0 6 0.8 1 (x)

0.4
0.2

0.2 0.4 0.6

0.4
0.2

0.2 0.4 0.6

0.5

0.5 1 1.5

1
0.8
0.6

x

0.5

0.5 1 1.5

1
0.8
0.6

x

f(x) a3(x) Old approximation New approximation 1/8 2/8 3/8 4/8 5/8 6/8 7/8 1 1/4 2/4 3/4 1 x x

SLIDE 6 0 6 0.8 1 (x) 0 6 0.8 1 (x)

0.4
0.2

0.2 0.4 0.6

0.4
0.2

0.2 0.4 0.6

0.5

0.5 1 1.5

1
0.8
0.6

x

0.5

0.5 1 1.5

1
0.8
0.6

x

f(x) a4(x) Old approximation New approximation 2/16 4/16 6/16 8/16 10/16 12/16 14/16 1 x 1/8 2/8 3/8 4/8 5/8 6/8 7/8 1 x x x

SLIDE 7 0 6 0.8 1 (x) 0 6 0.8 1 (x)

0.4
0.2

0.2 0.4 0.6

0.4
0.2

0.2 0.4 0.6

0.5

0.5 1 1.5

1
0.8
0.6

x

0.5

0.5 1 1.5

1
0.8
0.6

x

f(x) a5(x) Old approximation New approximation 4/32 8/32 12/32 16/32 20/32 24/32 28/32 1 2/16 4/16 6/16 8/16 10/16 12/16 14/16 1 x x

SLIDE 8

f(x) a6(x) 8/64 16/64 24/64 32/64 40/64 48/64 56/64 1 x

SLIDE 9

{ }

/2

The Haar basis is of the type ( ), ( ) 2 (2 ) .

n n k

x k x x k j y y

=
Wavelet bases

{ }

,

The Haar basis is of the type ( ), ( ) 2 (2 ) . Such a basis is called a with and .

n k

x k x x k j y y j y wavelet basis scaling function mother wavelet j y Consequence : The scaling function gives a large scale approximation and the wavelets adds finer details (illustrated in next slide).

SLIDE 10

Orthonormal bases Both the Haar basis and the usual Fourier basis is a set of building blocks { } with the following properties

k

e

2

Any function ( ) can be decomposed into a sum . There is

k k k

f L f c e · Î = ·

å

 a simple formula for computing the coefficients: p p g ( ) ( ) ,

k k k

c f x e x dx f e

¥

= =

ò

Inner product

1 if The building blocks are : , if

k n

rthonormal

e e k n

¥

ì = ï ï · = í ï ¹ ï î Any such set of building blocks is called an î

rt

. honormal basis

SLIDE 11

Good properites of the Haar wavelet basis :

Orthonormal (just like the Fourier basis).

Well localized Better suited for good approximation of small local ·  Good properites of the Haar wavelet basis : g pp details in a signal with a small number of terms (contrary to the Fourier basis). Discontinuities 1) Many terms needed for goo ·  Usually less desirable properties of the Haar wavelet basis : d approximation Discontinuities 1) Many terms needed for goo ·  d approximation (=small "edges" in last slide ) of continuous signals. 2) Bad frequency localization (drawback in ) q y ( in time-frequency analysis (explained soon)).

SLIDE 12

A h MRA adds smoothness N t l ti ti Are there any way to contruct a well localized and

rthonormal wavelet basis?

, or even " (say, k times differentiable), Natural question : continuous arbitrarily smooth"

Yes. T

Answer : he construction is a generalization of the telescope sums in last lecture. Described in any wavelet book under the name lti l ti l i (MRA). multiresolution analysis (MRA)

SLIDE 13

Extra bonus :

The fast wavelet transform

It follows from the MRA theory that there is a special algorithm for quick computation of the wavelet coefficients. The computation time is proportional to the signal length ( ) N and thus faster than the fast Fourier transform ( log ). N N

SLIDE 14

Pyramid algorithm / filter banks / Mallat’s algorithm

SLIDE 15

SLIDE 16

Example 1: Daubechies n scaling functions, n=1-12 p g ,

0.5 1 1.5 n=1 0.5 1 1.5 n=2 0.5 1 1.5 n=3

Nonzero only in

Daubechies scaling functions

5 10

0.5

5 10

0.5

5 10

0.5

0.5 1 1.5 n=4 0.5 1 1.5 n=5 0.5 1 1.5 n=6

the interval [0,n-1].

For any k and

5 10

0.5

5 10

0.5

5 10

0.5

0.5 1 1.5 n=7 0.5 1 1.5 n=8 0.5 1 1.5 n=9

large enough n, the Daubechies n wavelet and li f i

5 10

0.5

5 10

0.5

5 10

0.5

0.5 1 1.5 n=10 0.5 1 1.5 n=11 0.5 1 1.5 n=12

scaling function is k times differentiable.

5 10

0.5

5 10

0.5

5 10

0.5

SLIDE 17

Corresponding Daubechies wavelets. Corresponding Daubechies wavelets.

1 n=1 1 n=2 1 n=3

Daubechies wavelets

10 20

1

10 20

1

10 20

1

1 n=4 1 n=5 1 n=6 10 20

1

10 20

1

10 20

1

1 n=7 1 n=8 1 n=9 10 20

1

10 20

1

10 20

1

1 n=10 1 n=11 1 n=12 10 20

1

10 20

1

10 20

1

SLIDE 18

Example 2: Spline wavelets of degree 2 p p g

2 Translated scaling functions

Exponential decay

( l th

Spline wavelets

−3 −2 −1 1 2 3 −2 2 Translated wavelets

(slower than Daubechies, but still fast). S li l t f

−3 −2 −1 1 2 3 −2 2 Translated and dilated (with factor 2) wavelets

Spline wavelets of

degree n is n times differentiable.

nth degree

−3 −2 −1 1 2 3 −2 2 Translated and dilated (with factor 4) wavelets

nth degree

polynomial in intervals [k,k+1] (scaling function)

−3 −2 −1 1 2 3 −2

(scaling function) and [k/2,(k+1)/2] (wavelet).

SLIDE 19

SLIDE 20

Some threshold techniques q

SLIDE 21 Caruso wax roll example

Source: http://www.fmah.com

1) Original, 2) Single pass denosied, 3) Removed noise, 4), second pass denoised seeking decorrelation between the noise model and the original file

SLIDE 22

L H L H H L L H H L

SLIDE 23

256 256 i l 256 l Whi i dd d 256x256 pixels, 256 greyscale White noise added Restored, daub4, reduced to 1,8 % of the original file size

SLIDE 24 FBI fingerprint example

SLIDE 25

Image size: 847x683 pixels x 24 bit colours =1 66 MB

Original image

1.66 MB

SLIDE 26

Compression:

JPEG
65.8 times

JPEG-compressed image

SLIDE 27

Compression:

JPEG2000
130 times

JPEG2000-compressed image

SLIDE 28

Original: Denoised:

Movie example

Removed noise:

Source: http://www.fmah.com

SLIDE 29

Digital subscriber lines Digital subscriber lines

SLIDE 30

Multicarrier transmission examples:

ADSL: Out now. About 2-8.5 megabits per second (Mbps) VDSL: (Originally) planned for 2001.

ADSL vs. VDSL

VDSL: (Originally) planned for 2001. From 5 Mbps in 1500 m long wires up to about 60 Mbps in 400 m long wires. p g (50 Mbps is enough for, for example, 8 digital TV channels or 2-4 high definition TV channels ) TV channels or 2 4 high definition TV channels.)

SLIDE 31

Maximum delay restrictions Maximum delay restrictions

SLIDE 32

Each symbol is built up of basis functions N (t)= ( )

N k k l k l

s c f t

å

The transmitted information

Choice of basis functions

, , 1 ,

( ) ( ) must be well localized in time (because too

k k l k l l k l

f f

=

å

,

long symbols introduce unacceptable delays).

k l

Wavelets can be used, but for this particular application, the short time Fourier transform pp , has some advantages and is used in VDSL.

SLIDE 33

Railway bridge strains Railway bridge strains

SLIDE 34

10 Channel A1 (m/m), 7 level decomposition with haar wavelet. 20 Channel A2 (m/m), 7 level decomposition with haar wavelet. Approximation Detaljer Approximation Detaljer 5 10 15 20 25 30 35 40 45

10
5

5 Tid (timmar) 5 10 15 20 25 30 35 40 45

30
20
10

10 Tid (timmar) Channel A4 (m/m) 7 level decomposition with haar wavelet Channel A5 (m/m) 7 level decomposition with haar wavelet j j

30
20
10

10 20 Channel A4 (m/m), 7 level decomposition with haar wavelet. 15

10
5

5 Channel A5 (m/m), 7 level decomposition with haar wavelet. Approximation Detaljer Approximation Detaljer 5 10 15 20 25 30 35 40 45 Tid (timmar) 5 10 15 20 25 30 35 40 45

15

Tid (timmar) 10 20 30 Channel A6 (m/m), 7 level decomposition with haar wavelet. 10 20 30 Channel A7 (m/m), 7 level decomposition with haar wavelet. Approximation Detaljer Approximation Detaljer 5 10 15 20 25 30 35 40 45

10

Tid (timmar) 5 10 15 20 25 30 35 40 45

20
10

Tid (timmar) 10 15 Channel A8 (m/m), 7 level decomposition with haar wavelet. 5 Channel R1 (m/m), 7 level decomposition with haar wavelet. Approximation Detaljer Approximation Detaljer 5 10 15 20 25 30 35 40 45

5

5 Tid (timmar) 5 10 15 20 25 30 35 40 45

5

Tid (timmar)

SLIDE 35

10 Channel A1 (m/m), 7 level decomposition with db2 wavelet. 10 20 Channel A2 (m/m), 7 level decomposition with db2 wavelet. Approximation Detaljer Approximation Detaljer 5 10 15 20 25 30 35 40 45

10
5

5 Tid (timmar) 5 10 15 20 25 30 35 40 45

30
20
10

10 Tid (timmar) Channel A4 (m/m), 7 level decomposition with db2 wavelet. Channel A5 (m/m), 7 level decomposition with db2 wavelet.

30
20
10

10 20 

15
10
5

5  Approximation Detaljer Approximation Detaljer 5 10 15 20 25 30 35 40 45 Tid (timmar) 5 10 15 20 25 30 35 40 45 Tid (timmar) 10 20 30 Channel A6 (m/m), 7 level decomposition with db2 wavelet. 10 20 30 Channel A7 (m/m), 7 level decomposition with db2 wavelet. Approximation Detaljer Approximation Detaljer 5 10 15 20 25 30 35 40 45

10

Tid (timmar) 5 10 15 20 25 30 35 40 45

20
10

Tid (timmar) 10 15 Channel A8 (m/m), 7 level decomposition with db2 wavelet. 5 Channel R1 (m/m), 7 level decomposition with db2 wavelet. Approximation Detaljer Approximation Detaljer 5 10 15 20 25 30 35 40 45

5

5 Tid (timmar) 5 10 15 20 25 30 35 40 45

5

Tid (timmar)

SLIDE 36

10 Channel A1 (m/m), 7 level decomposition with coif3 wavelet. 10 20 Channel A2 (m/m), 7 level decomposition with coif3 wavelet. Approximation Detaljer Approximation Detaljer 5 10 15 20 25 30 35 40 45

10
5

5 Tid (timmar) 5 10 15 20 25 30 35 40 45

30
20
10

10 Tid (timmar) Channel A4 (m/m), 7 level decomposition with coif3 wavelet. Channel A5 (m/m), 7 level decomposition with coif3 wavelet.

30
20
10

10 20 ( ), p

15
10
5

5 ( ), p Approximation Detaljer Approximation Detaljer 5 10 15 20 25 30 35 40 45 Tid (timmar) 5 10 15 20 25 30 35 40 45 Tid (timmar) 10 20 30 Channel A6 (m/m), 7 level decomposition with coif3 wavelet. 10 20 30 Channel A7 (m/m), 7 level decomposition with coif3 wavelet. Approximation Detaljer Approximation Detaljer 5 10 15 20 25 30 35 40 45

10

Tid (timmar) 5 10 15 20 25 30 35 40 45

20
10

Tid (timmar) 10 15 Channel A8 (m/m), 7 level decomposition with coif3 wavelet. 5 Channel R1 (m/m), 7 level decomposition with coif3 wavelet. Approximation Detaljer Approximation Detaljer 5 10 15 20 25 30 35 40 45

5

5 Tid (timmar) 5 10 15 20 25 30 35 40 45

5

Tid (timmar)

SLIDE 37

Wavelets were developed independently in the fields of Wavelets were developed independently in the fields of mathematics, quantum physics, electrical engineering and siesmic geology. Some application areas are

Some applications

Data compressision
Astronomy
Acoustics
Nuclear engineering
Turbulence
Earthquake-prediction
Radar
Nuclear engineering
Sub-band coding
Signal and image processing
Neurophysiology
Human vision
Mathematical analysis
Partial differential equations

N i l l i p y gy

Music
Magnetic resonance imaging
Speech discrimination

O i

Numerical analysis
Statistics
Econometrics
Communication theory
Optics
Fractals

Communication theory

Computer graphics

SLIDE 38