Time-Frequency analysis for multi-channel and/or multi-trial signals - - PowerPoint PPT Presentation

▶
time frequency analysis for multi channel and or multi
SMART_READER_LITE
LIVE PREVIEW

Time-Frequency analysis for multi-channel and/or multi-trial signals - - PowerPoint PPT Presentation

Feb 08, 2023 229 likes •712 views

Multi-channel/multi-trial time-frequency analysis Time-Frequency analysis for multi-channel and/or multi-trial signals B. Torr esani Aix-Marseille Univ. Laboratoire dAnalyse, Topologie et Probabilit es ESI, December 2012 B. Torr

slide-1
SLIDE 1

Multi-channel/multi-trial time-frequency analysis

Time-Frequency analysis for multi-channel and/or multi-trial signals

  • B. Torr´

esani

Aix-Marseille Univ. Laboratoire d’Analyse, Topologie et Probabilit´ es

ESI, December 2012

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 1 / 45

slide-2
SLIDE 2

Multi-channel/multi-trial time-frequency analysis

Outline

1

Introduction Multi-channel signals, multi-trial signals Time-frequency analysis

2

Multi-channel signals and time-frequency The need for structures A regression model

3

Introducing time dependencies : a detection model Time dependencies via Markov chain A case study : alpha waves based characterization of multiple sclerosis

4

Conclusions

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 2 / 45

slide-3
SLIDE 3

Multi-channel/multi-trial time-frequency analysis

Motivation : Multi-sensor biosignals, such as EEG, MEG,... contain information that shows up differently in various channels, and may be difficult to extract from single channel. In this context, one often looks for features that are localized in some joint space-time-frequency domain. To detect weak signals, experiments are often repeated several times : multi-trial signals Problem : tackle inter-trial variability... which may sometimes be modelled as time-frequency jitter and amplitude variability...

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 3 / 45

slide-4
SLIDE 4

Multi-channel/multi-trial time-frequency analysis

Outline

1

Introduction Multi-channel signals, multi-trial signals Time-frequency analysis

2

Multi-channel signals and time-frequency The need for structures A regression model

3

Introducing time dependencies : a detection model Time dependencies via Markov chain A case study : alpha waves based characterization of multiple sclerosis

4

Conclusions

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 4 / 45

slide-5
SLIDE 5

Multi-channel/multi-trial time-frequency analysis

Time-frequency analysis :

Time-frequency transforms are inherently single channel techniques. Can be trivially extended to multi-channel signals, by individually transforming each channel ; multi-channel cooperation is enforced by post-processing. Synthesis-based frameworks allow one to enforce multi-channel information sharing already in the first stage. The multi-trial situation is much more complex... need of time-frequency registration techniques prior to trial averaging.

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 5 / 45

slide-6
SLIDE 6

Multi-channel/multi-trial time-frequency analysis Notations : Gabor atoms

Modulated and translated copies of a reference window gkn[t] = e2iπnν0(t−kb0)/Lg[t −kb0] , k ∈ ZK , n ∈ ZN where ν0 and b0 are divisors of L, K = L/b0 et N = L/ν0. Given f ∈ CL, the family of coefficients

Vgf[k,n] = f,gkn =

L−1

∑

t=0

f[t]g[t −kb0]e−2iπnν0(t−kb0)/L form a short time Fourier transform (if b0 = ν0 = 1) or a Gabor transform of f.

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 6 / 45

slide-7
SLIDE 7

Multi-channel/multi-trial time-frequency analysis

Examples of Gabor atoms :

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 7 / 45

slide-8
SLIDE 8

Multi-channel/multi-trial time-frequency analysis Notations : MDCT atoms

In CL, let M ∈ Z+ be a un divisor of L.

ZL is segmented into K =L/N intervals of length N

For all k = 0,...K −1, let wk ∈ CL be such that

wk[t] = 0 for t < (k −1/2)N and t > (k +3/2)N. wk[kN +τ] = wk+1[kN −τ] for all τ = 1−N/2,...N/2−1 wk[kN +τ]2 +wk+1[kN +τ]2 = 1 for all τ = 1−N/2,...N/2−1

Denote by ukn ∈ CL the vectors defined by ukn[t] =

  • 2

N wk[t]cos

  • π
  • n + 1

2

  • (t −kN)
  • The collection {ukn} is an orthonormal basis of CL.
  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 8 / 45

slide-9
SLIDE 9

Multi-channel/multi-trial time-frequency analysis

Examples of MDCT atoms : Being a basis has a price : the time-frequency localization of MDCT atoms is more difficult to control.

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 9 / 45

slide-10
SLIDE 10

Multi-channel/multi-trial time-frequency analysis Frames and bases

Given a time-frequency basis Ψ = {ψtf} : the transform x ∈ CL → {x,ψtf} is unitary. Any x has a unique expansion x = ∑

tf

αtfψtf .

Given a time-frequency frame Ψ = {ψtf} (which is not a basis). Any x has infinitely many expansions of the form x = ∑

tf

αtfψtf ,

finding the most relevant one requires extra information,... and is application dependent.

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 10 / 45

slide-11
SLIDE 11

Multi-channel/multi-trial time-frequency analysis

Outline

1

Introduction Multi-channel signals, multi-trial signals Time-frequency analysis

2

Multi-channel signals and time-frequency The need for structures A regression model

3

Introducing time dependencies : a detection model Time dependencies via Markov chain A case study : alpha waves based characterization of multiple sclerosis

4

Conclusions

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 11 / 45

slide-12
SLIDE 12

Multi-channel/multi-trial time-frequency analysis

Multichannel signals : x = {xc, c = 1,...Nc} signals from different channels are often dependent the dependence structure is often complex, and not necessarily known in advance

Example (Propagation from two sources)

Signals, originating from two inner “sources”, propagating to the boundary of some region where they are measured. Quasi-static approximation : time-locked signals

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 12 / 45

slide-13
SLIDE 13

Multi-channel/multi-trial time-frequency analysis

Example : EEG signals

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 13 / 45

slide-14
SLIDE 14

Multi-channel/multi-trial time-frequency analysis Multichannel time-frequency expansions

In the framework of quasi-static type approximations (no time delay) : use the same time-frequency dictionary for all channels :

Ψ = {ψtf}. Transform + post-processing : example

Compute time-frequency transform coefficients

α = {αc

tf} ,

αc

tf = xc,ψtf

Describe the data cube α via space-time-frequency modes, using factor decomposition (PARAFAC, Kruskal,...)

α = ∑

k

Ck ⊗Tk ⊗Fk + res. ,

αc

tf = ∑ k

Cc

kTktFkf + res.

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 14 / 45

slide-15
SLIDE 15

Multi-channel/multi-trial time-frequency analysis

Outline

1

Introduction Multi-channel signals, multi-trial signals Time-frequency analysis

2

Multi-channel signals and time-frequency The need for structures A regression model

3

Introducing time dependencies : a detection model Time dependencies via Markov chain A case study : alpha waves based characterization of multiple sclerosis

4

Conclusions

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 15 / 45

slide-16
SLIDE 16

Multi-channel/multi-trial time-frequency analysis Synthesis models

estimate multichannel time-frequency expansions of the form xc = ∑

t,f

αc

tfψtf +noise

Elementary models : channel, time and frequency are independent variables ; e.g. bridge regression type approaches : for 1 ≤ p ≤ 2, solve min

α

 1

2 ∑

c

  • xc −∑

t,f

αc

tfψtf

  • 2

+ µ

p αp

p

 

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 16 / 45

slide-17
SLIDE 17

Multi-channel/multi-trial time-frequency analysis

More complex models : introduce correlation structures, via either function space models (involving sophisticated mixed norms) or probabilistic models.

Gaussian and Gaussian mixture models

Gaussian prior model : p(α) ∼ N (0,Σ) Gaussian mixture prior model : for example p(α) ∼ ∑K

k=1 pkN (0,Σk)

Generalizations... In most cases, Σ and/or Σk are large matrices : difficult to estimate... and to exploit.

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 17 / 45

slide-18
SLIDE 18

Multi-channel/multi-trial time-frequency analysis Gaussian mixture prior

min

  1

2σ 2

  • x −∑

t,f

αtfψtf

  • 2

−ln(p(α))  

with p a Gaussian or Gaussian mixture prior. Gaussian prior : the (explicit) solution requires the inversion of a large matrix involving the inverse covariance matrix Σ−1 and the Gram matrix of the frame. Gaussian mixture priors : MM numerical strategies require at each iteration the inversion of matrices of the same size. Typical size : Nc ≈ 20 channels, time-frequency blocks of dimension MN ≈ 1000... yields matrices of size ≈ 20000×20000.

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 18 / 45

slide-19
SLIDE 19

Multi-channel/multi-trial time-frequency analysis

Extra structure has to be assumed for the covariance model : coefficient cube α with independent slices

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 19 / 45

slide-20
SLIDE 20

Multi-channel/multi-trial time-frequency analysis

Extra structure has to be assumed for the covariance model : coefficient cube α with independent slices If the time-frequency frame and the covariance structure are compatible, corresponding estimation algorithms can be designed.

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 19 / 45

slide-21
SLIDE 21

Multi-channel/multi-trial time-frequency analysis Definition (Translation invariant TF frame)

A time-frequency frame Ψ is invariant by (circular) translations if the columns of the corresponding matrix Ψ satisfy

ψλ[k] = ψm,n[k] = ψ0,n[k −m] , m = 0,...M −1, n = 0,...N −1 .

The corresponding Gram matrix Ψ∗Ψ is block circulant. G = Ψ∗Ψ =

    

G0 G1 G2

...

GN−1 GN−1 G0 G1

...

GN−2 . . . . . . . . . ... . . . G1 G2

...

GN−1 G0

    ,

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 20 / 45

slide-22
SLIDE 22

Multi-channel/multi-trial time-frequency analysis Examples

Gabor frames (time locked version) MDCT bases Translation invariant wavelet frames Arbitrary subband frames can be made translation invariant ...

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 21 / 45

slide-23
SLIDE 23

Multi-channel/multi-trial time-frequency analysis Theorem (MM approach convergence)

Consider the Gaussian mixture prior model. Set A = ∑k pkΣ−1

k

,

C(α) = lnp(α), and let ε be a positive integer.

1

The iteration αn −

→ αn+1 defined by 1 σ 2 Ψ∗Ψ+2(A+εI)

  • αn+1 =

1 σ 2 Ψ∗x−∇C(αn)−2(A+εI)αn

  • converges to a local minimum of the objective function.

2

If Ψ is translation invariant, and the coefficient cube α has independent fixed-time slices, the matrix M below is block circulant M = 1

σ 2 Ψ∗Ψ+2(A+εI)

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 22 / 45

slide-24
SLIDE 24

Multi-channel/multi-trial time-frequency analysis

Kronecker product : A⊗B =

    

a1,1B

...

a1,N′

aB

a2,1B

...

a2,N′

aB

. . . ... . . . aNa,1B

...

aNa,N′

aB

    . Proposition (De Mazancourt)

Block-circulant matrices M can be diagonalized using the block Fourier transform F (Kronecker product of the standard Fourier transform and the identity), yielding M = F∗PF with P invertible block-diagonal. Hence, the size of the matrices to be inverted is reduced. If further dimension reduction is needed : frequency-channel matrices can be seeked in the form of Kronecker products :

Σ(cf) = Σ(c) ⊗Σ(f) , Σ−1

(cf) = Σ−1 (c) ⊗Σ−1 (f) .

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 23 / 45

slide-25
SLIDE 25

Multi-channel/multi-trial time-frequency analysis

Related problem : estimation of the model parameters : Covariance matrices Σk (or Kronecker factors), Membership probabilities pk. Current solution : (ad hoc) re-estimation at each iteration of the

  • algorithm. No convergence proof for the combined approach.
  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 24 / 45

slide-26
SLIDE 26

Multi-channel/multi-trial time-frequency analysis

Related problem : estimation of the model parameters : Covariance matrices Σk (or Kronecker factors), Membership probabilities pk. Current solution : (ad hoc) re-estimation at each iteration of the

  • algorithm. No convergence proof for the combined approach.

Numerical simulation : Preliminary : single sensor, Gaussian mixture (N = 2) with known covariance matrices.

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 24 / 45

slide-27
SLIDE 27

Multi-channel/multi-trial time-frequency analysis

Simulation Frequency covariance matrices (state 2 : alpha waves)

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 25 / 45

slide-28
SLIDE 28

Multi-channel/multi-trial time-frequency analysis

Simulation Original, noisy and reconstructed signals

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 26 / 45

slide-29
SLIDE 29

Multi-channel/multi-trial time-frequency analysis

Outline

1

Introduction Multi-channel signals, multi-trial signals Time-frequency analysis

2

Multi-channel signals and time-frequency The need for structures A regression model

3

Introducing time dependencies : a detection model Time dependencies via Markov chain A case study : alpha waves based characterization of multiple sclerosis

4

Conclusions

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 27 / 45

slide-30
SLIDE 30

Multi-channel/multi-trial time-frequency analysis

So far : fixed-time coefficient vectors were assumed independent.

Multichannel harmonic hidden Markov model

A hidden state t → Xt ∈ {1,2,...Ns} controls the distribution of corresponding coefficients α. Fixed time coefficients α·

t· are modeled as before as a

Gaussian random vector N (0,Σs), whose covariance depends on the state Xs. Conditional to the hidden states, fixed time coefficient vectors

α·

t· are statistically independent.

The dynamics of hidden states is governed by a Markov chain : transition Xt = s to Xt+1 = s′ with fixed probabilities.

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 28 / 45

slide-31
SLIDE 31

Multi-channel/multi-trial time-frequency analysis Problems to solve

Estimate the model parameters :

Covariance matrices : Σfc or Σf ⊗Σc Characteristics of the chain : transition probabilities P{Xt+1 = s′|Xt = s}, initial probabilities P{X0 = s}.

Estimate the hidden states sequences

Answers

MDCT or Wilson basis : standard procedure

Computation of TF coefficients Parameter estimation : Baum Welch algorithm (provable convergence even for Kronecker covariance matrices) Hidden states estimation : Viterbi algorithm (low complexity)

For Gabor frames : ad hoc procedures... not really satisfactory

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 29 / 45

slide-32
SLIDE 32

Multi-channel/multi-trial time-frequency analysis

Details : Forward and backward variables as

t = P{Xt = s|α0:t}×Lt

with Lt the likelihood of the observations until time t, bs

t = P

  • y(t+1):(Nt−1)|Xt = s
  • .

are computed recursively using the forward-backward equations. as

t+1 = fs(αt+1) Ns

∑

s′=1

πs′sas′

t ,

bs

t = Ns

∑

s′=1

πss′fs′(αt+1)bs′

t+1 .

with π the transition matrix of the chain, and fs the pdf of fixed-time coefficient vectors αt in state s.

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 30 / 45

slide-33
SLIDE 33

Multi-channel/multi-trial time-frequency analysis

Details : Initial probabilities and transition matrix re-estimation

  • νs = as

0bs

L

  • πs,s′ = πs,s′

1 L ∑Nt−2 t=0 as t bs′ t+1fs′(αt+1) 1 L ∑Nt−2 t=0 as t bs t

,

with

L = LNt−1 =

Ns

∑

s=1

as

t bs t

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 31 / 45

slide-34
SLIDE 34

Multi-channel/multi-trial time-frequency analysis

Estimation of Σ(c)

s

given Σ(f)

s

: define Ms

t (c,c′) = (Σ(f) s )−1αc t ,αc′ t

and set

  • Σ(c)

s

= 1

Nf

∑Nt−1

t=0 P{Xt = s}Ms t

∑Nt−1

t=0 P{Xt = s}

Normalization : set

  • Σ(c)

s

= Σ(c)

s /

  • Σ(c)

s

  • F

,

Estimation of Σ(f)

s

given

Σ(c)

s

: define Ps

t (f,f ′) = (Σ(c))−1αtf,αtf ′

and set

  • Σ(f)

s

= 1

Nc

∑Nt−1

t=0 P{Xt = s}Ps t

∑Nt−1

t=0 P{Xt = s}

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 32 / 45

slide-35
SLIDE 35

Multi-channel/multi-trial time-frequency analysis Application to rest EEG

Rest EEG basically features Alpha waves : short duration time-localized oscillations (frequencies around 10 Hz) which appear in specific situations ; topographically localized in specific sensors located in posterior regions of the head. Alpha wave occurrence may be considered a departure from a stationary background signal. This motivates the use of hidden Markov models as described above.

Remark (Time-frequency resolution)

alpha waves are actually close to the Heisenberg limit. One needs frequency resolution of approximately 4Hz, and time resolution of approximately 250 msec....

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 33 / 45

slide-36
SLIDE 36

Multi-channel/multi-trial time-frequency analysis

Alpha waves in rest EEG

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 34 / 45

slide-37
SLIDE 37

Multi-channel/multi-trial time-frequency analysis

Application to rest EEG : real data MDCT coefficients of a 30 sec. long EEG recording (rest EEG)

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 35 / 45

slide-38
SLIDE 38

Multi-channel/multi-trial time-frequency analysis

Application to rest EEG Frequency covariance matrices estimates for the two classes Channel covariance matrices estimates for the two classes

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 36 / 45

slide-39
SLIDE 39

Multi-channel/multi-trial time-frequency analysis

Hidden states estimation : simulated data

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 37 / 45

slide-40
SLIDE 40

Multi-channel/multi-trial time-frequency analysis

Outline

1

Introduction Multi-channel signals, multi-trial signals Time-frequency analysis

2

Multi-channel signals and time-frequency The need for structures A regression model

3

Introducing time dependencies : a detection model Time dependencies via Markov chain A case study : alpha waves based characterization of multiple sclerosis

4

Conclusions

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 38 / 45

slide-41
SLIDE 41

Multi-channel/multi-trial time-frequency analysis Multiple sclerosis

Multiple sclerosis has been reported to affect the left-right synchronization in the alpha band. This assumption can be tested using the model.

Dataset

EEG data originating from the CODYSEP dataset, designed to study the impact of multiple sclerosis in inter-hemispherical transfer. The dataset consists in 31 patients and 20 controls ; 17 channels EEG signals were collected at a 256 Hz sampling rate. EEG data essentially contain alpha waves bursts.

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 39 / 45

slide-42
SLIDE 42

Multi-channel/multi-trial time-frequency analysis

2 minutes of recording

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 40 / 45

slide-43
SLIDE 43

Multi-channel/multi-trial time-frequency analysis

4 seconds of recording

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 41 / 45

slide-44
SLIDE 44

Multi-channel/multi-trial time-frequency analysis Testing protocole :

1

For both classes (patient and control)

Select relevant left and right subsets of the set of sensors For each subset :

Estimate corresponding model parameters Estimate left and right hidden states sequence X (L) and X (R)

Compute the Hamming distance between left and right hidden states sequences : dH = X (L) −X (R)1.

2

Compare estimated Hamming distances of controls and patients : boxplots, p-values,...

3

Compare with the results obtained using inter-coherence : left-right cross-correlation after band pass filtering.

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 42 / 45

slide-45
SLIDE 45

Multi-channel/multi-trial time-frequency analysis

Results Left : boxplots of Hamming distances dH between hidden states Right : boxplots of inter-coherences dC between signals Mann-Whitney test : P-value≈ 0.0384 : confirms quantitatively the hypothesis of two distinct distributions.

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 43 / 45

slide-46
SLIDE 46

Multi-channel/multi-trial time-frequency analysis Conclusions

When going multi-channel, one has to fight the curse of dimensionality. Factorized models can help in this respect Two approaches were presented, tackling two different

  • problems. Next question : how to keep the best of the two ?

Multi-trial : matching pursuit type approach (Consensus matching pursuit)

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 44 / 45

slide-47
SLIDE 47

Multi-channel/multi-trial time-frequency analysis Thanks

Joint works with E. Villaron and S. Anthoine Pleasant collaborations and discussions with the LATP signal processing group NuHaG and partners for organizing this nice event ... The audience for your attention !

  • B. Torr´

esani (LATP , Aix-Marseille Univ.) ESI, December 2012 45 / 45