Exploring the Brains Activity: a signal and image modeling challenge - - PowerPoint PPT Presentation

Introduction

Exploring the Brain’s Activity: a signal and image modeling challenge

Maureen Clerc

Inria

EUSIPCO, Nice, September 2015

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 1 / 45

Introduction

Introduction

Functional areas of the brain schematic organization variability of cortical foldings subject-dependent localization through exploration How to localize brain activity: invasively: brain stimulation non-invasively: functional brain imaging Example: presurgical evaluation of epilepsy Epileptogenic regions Eloquent functional regions

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 1 / 45

Introduction

Introduction

1924: Hans Berger measures electrical potential variations on the scalp. birth of Electro-Encephalography (EEG) several types of oscillations detected (alpha 10 Hz, beta 15 Hz)

• rigin of the signal unclear at the time

scalp topographies ressemble dipolar field patterns dipolar topographies modern

• n a

sphere EEG

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 2 / 45

Introduction

From electric to magnetic fields

A dipole generates electric + magnetic fields 1963: MCG Magnetocardiography, 1972: MEG Magneto-Encephalography,

• D. Cohen, MIT.

But: very weak signal, only measured in shielded environment Relies on Superconductive QUantum Interference Device. car at 50m 10−9 lung particles screwdriver at 5m 10−10 human heart 10−11 skeletal muscles, fetal heart transistor at 2m 10−12 human eye, human brain (alpha) 10−13 human brain (evoked response) (order of magnitude, 10−14 SQUID system noise level in Tesla) 10−15

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 3 / 45

Introduction

MEG instrumentation

MEG center, Piti´ e-Salpˆ etri` ere hospital, Paris Advantage of MEG over EEG: spatially more focal

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 4 / 45

Introduction

Example: functional imaging within the visual cortex

Primary visual cortex = V1 (Brodmann area 17) Receptive field = the visual field neurons respond to Adjacent neurons have overlapping receptive fields. Retinotopy = map between V1 and visual field.

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 5 / 45

Introduction

Functional brain imaging: functional MRI

fMRI mesures metabolism (local O2 consumption)

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 6 / 45

Introduction

Functional brain imaging: Magneto-Encephalography

Sensor measurements Flickering stimulus at 7.5 Hz

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 7 / 45

Introduction

Functional brain imaging: Magneto-Encephalography

Sensor measurements Flickering stimulus at 7.5 Hz At 15 Hz on MEG sensors, retinotopic organization: topography depends

• n stimulus position

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 7 / 45

Introduction

Functional brain imaging: Magneto-Encephalography

stimulus positions Source activity reconstruction [Cottereau, Gramfort et al, HBM 2008] MEG and EEG naturally provide information about timing of brain activity.

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 8 / 45

Introduction

Origin of brain activity measured in EEG and MEG

[Baillet et al., IEEE Signal Processing Mag, 2001]

Pyramidal neurons Current perpendicular Neurons in a post-synaptic currents to cortical surface macrocolumn co-activate

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 9 / 45

Introduction

Source models: distributed or isolated

Distributed current source defined on a surface S with current density q(r): Jp(r) = q(r) n δS (orthogonal to S) Isolated current dipole defined at a position p with current (moment) q: Jp = q δp Also linear combinations Jp =

n

• i=1

qi δpi

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 10 / 45

Inverse source reconstruction

Outline

1

Introduction

2

Inverse Source Reconstruction

Forward Problem Inverse Problem Anatomically Constrained Regularization

3

Neuroelectrical signal analysis

4

Current challenges

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 11 / 45

Inverse source reconstruction

Specificities of MEG and EEG

• rigin of activity: depolarization / repolarization of

neural membranes postsynaptic potentials represented by dipoles in grey matter dipole orientation perpendicular to cortex Low freq (< 1000 Hz): quasistatic approx to Maxwell’s eqs: Electric and magnetic fields become decoupled. Electrostatic equation: Biot-Savart equation: ∇ · (σ∇V ) = ∇ · Jp B(r) = µ0

4π

• (Jp(r′) − σ∇V (r′)) ×

r−r′ r−r′3 dr′

= B0(r) − µ0

4π

• σ∇V (r′) ×

r−r′ r−r′3 dr′

B0(r): “primary magnetic field” coming from the sources

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 12 / 45

Inverse source reconstruction

Influence of orientation (spherical geometry)

[courtesy of S.Baillet] Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 13 / 45

Inverse source reconstruction

Influence of depth (realistic geometry)

[courtesy of S.Baillet] Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 14 / 45

Inverse source reconstruction

Modeling the conductivity σ within the head

simplest model: overlapping spheres

no meshing required analytical methods × crude approximation of head conduction, especially for EEG

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 15 / 45

Inverse source reconstruction

Modeling the conductivity σ within the head

simplest model: overlapping spheres

no meshing required analytical methods × crude approximation of head conduction, especially for EEG

surface-based-model: piecewise constant conductivity

• nly surfaces need to be meshed

Boundary Element Method (BEM) × only isotropic conductivities

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 15 / 45

Inverse source reconstruction

Modeling the conductivity σ within the head

simplest model: overlapping spheres

no meshing required analytical methods × crude approximation of head conduction, especially for EEG

surface-based-model: piecewise constant conductivity

• nly surfaces need to be meshed

Boundary Element Method (BEM) × only isotropic conductivities

most sophisticated model: volume-based conductivity

detailed conductivity model, (anisotropic: tensor at each voxel) Finite Element Method (FEM), × huge meshes, difficult to handle

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 15 / 45

Inverse source reconstruction

Head Geometrical Modeling from T1/T2 MRIs

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 16 / 45

Inverse source reconstruction

Forward problem: computing the gain matrix

Jp sum of n isolated dipoles with fixed position and time-varying moment qi(t) = si(t) ni (n unitary) M(t) =    G1(p1, n1) . . . Gm(p1, n1)    × s1(t) + · · · +    G1(pn, nn) . . . Gm(pn, nn)    × sn(t) = G    s1(t) . . . sn(t)    In gain matrix G: columns = sources / lines = sensors. G =    G1(p1, n1) . . . G1(pn, nn) . . . ... . . . Gm(p1, n1) . . . Gm(pn, nn)    [Kybic et al 2005, Gramfort et al 2010, OpenMEEG software]

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 17 / 45

Inverse source reconstruction

Source reconstruction

Two types of source models considered: isolated distributed unknowns ≪ measurements unknowns ≫ measurements sensitivity to model order indeterminacy regularization necessary Uniqueness of reconstruction: proven for each model. Ill-posedness, due to instability.

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 18 / 45

Inverse source reconstruction

Distributed sources: minimum norm estimators

Measurements on m EEG / MEG sensors. Linear relationship between sources and sensor data:    M1(t) . . . Mm(t)    =    G1(p1, n1) . . . G1(pn, nn) . . . ... . . . Gm(p1, n1) . . . Gm(pn, nn)       s1(t) . . . sn(t)    + N m × n m × n n × n M G gain matrix S M = G S + N n ≫ m → regularization necessary Find sources S minimizing E(S) + λR(S) where E(S) = (M − G S)TΣ−1(M − G S) and R(S) source regularization term.

[Adde Clerc Keriven 2005]

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 19 / 45

Inverse source reconstruction

Different source regularization terms

Find S minimizing E(S) + λR(S) where R(S) source regularization terms. Consider the three different regularizations: L2 norm (MN): R(S) = S2

2 =

• cortex

|S|2 L2 norm of gradient (HEAT): R(S) = ∇S2

2 =

• cortex

|∇S|2 L1 norm of gradient (TV), edge-preserving: R(S) = ∇S2

1 =

• cortex

|∇S| 2

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 20 / 45

Inverse source reconstruction

Influence of regularization

simulated EEG, 10% noise

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 21 / 45

Inverse source reconstruction

Anatomically constrained regularization

Regularization applied to “parcels” coming from anatomical information. How to segregate source space into parcels ?

Functional specialization linked to regional differences in structural connectivity [Sporns 2004] Connectivity-based parcellation of cortical source space White matter fiber anisotropy measured by diffusion MRI diffusion MRI processed to yield probabilistic tractograms

Sources in gray matter = seeds for tractogram Connectivity Profile CPi of source i High dimensional vector (number of voxels) Correlation clustering of the CPs [Anwander 2007]

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 22 / 45

Inverse source reconstruction

Tractography-based parcellation

Three connectivity profiles (CPi) Correlation clustering of CPs: Sources i and j clustered together if CPi and CPj similarly correlated to the CPs of all sources.

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 23 / 45

Inverse source reconstruction

Tractography-based parcellation

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 24 / 45

Inverse source reconstruction

Anatomically constrained regularization

Find source distribution S such that S = argminSM − GS2 + λS2 + µRPS2

2

Comparison between: MNE µ = 0 MNE-PC RP = parcellation-constrained laplacian MNE-PSS MNE in reduced-dimensional source space (one scalar per parcel)

cf B.Belaoucha on Thursday, session MISP-L1 Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 25 / 45

Inverse source reconstruction

Epileptic spike propagation

1st step: clustering of spikes in time-domain (Inserm La Timone) → several classes 2nd step: source reconstruction for each class average spike

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 26 / 45

Inverse source reconstruction

Epileptic spike propagation

1st step: clustering of spikes in time-domain (Inserm La Timone) → several classes 2nd step: source reconstruction for each class average spike single spike

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 26 / 45

Neuroelectrical Signal Analysis

Outline

1

Introduction

2

Inverse Source Reconstruction

3

Neuroelectrical Signal Analysis

4

Current challenges

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 27 / 45

Neuroelectrical Signal Analysis

Latency (jitter) compensation

Low SNR → use of multitrial data Woody’s method (1967) xm = d(· − δm) + εm xm trial, epoch (or channel) d signal of interest δm latency εm noise Drawback: only one constant signal component

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 28 / 45

Neuroelectrical Signal Analysis

Linear Decompositions

xm =

• k

akmdk + εm Principal Components Analysis (PCA): Maximizes variance Orthogonality between components Independent Components Analysis (ICA): Statistical independence between components Dictionary Learning: Sparsity of components Drawback: no explicit modeling

• f temporal variability

[Jung et al 2000]

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 29 / 45

Neuroelectrical Signal Analysis

Time-Frequency analysis

Representations in short-time Fourier

• r wavelet bases

Time-frequency: meaningful Complex coeffs: phase / amplitude separation Drawback: smearing of activity due to latency jitter

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 30 / 45

Neuroelectrical Signal Analysis

Average map vs. vote map

Large dictionary with 3 dimensions: time - frequency - number of periods ξ

Example - true signal has ξ = 11. Average map Vote map Global max at ξ = 13. Global max at ξ = 11: Consensus Atom

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 31 / 45

Neuroelectrical Signal Analysis

Consensus Matching Pursuit (CMP)

xm =

• k

akmdkm + εm At iteration k for each trial m find dkm local maximum of t-f-ξ map closest to Consensus Atom Drawback: atoms not physiologically realistic

[B´ enar, Papadopoulo, Torr´ esani, Clerc, 2009]

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 32 / 45

Neuroelectrical Signal Analysis

Towards physiologically realistic atoms

Discrete set of deformations: φp including combinations of: translations dilations ... Goal: learn the templates dk and which of their deformations are active → Adaptive Waveform Learning

min

akpm, dk M

• m=1

 

• xm −

K

• k=1

P

• p=−P

akpmφp(dk)

• 2

2

+ λ

K

• k=1

P

• p=−P

|akpm|  

[Hitziger et al, Int Conf Learning Repr 2013]

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 33 / 45

Neuroelectrical Signal Analysis

Adaptive Waveform Learning

D = argmin

D

min

{am} M

• m=1

1

2xm − Dam2 2 + λam1

• Dictionary D :

templates with all their deformations Coefficients am: atom selection

update

Schematic DL Algorithm: Alternate Minimization INPUT: -signal matrix X and

• initial dictionary D or -initial coefficients A

1000 2000 3000 2000 4000 6000 8000 A (coefficients) time [s] amplitude 0.1 0.2 0.3 0.4 0.5 −0.2 −0.15 −0.1 −0.05 0.05 D (waveforms) time [s] amplitude (normalized)

loop... OUTPUT: learned dictionary D and coefficients A ...until convergence

update

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 34 / 45

Neuroelectrical Signal Analysis

Adaptive Waveform Learning (AWL): epoched

Manual epoching of 169 isolated spikes in a Local Field Potential recording Hierarchical AWL

Results: Oscillatory component Energy profile of coefficients

−0.5

waveform 1

waveform latency distribution

waveforms coefficients

1 0.5 1 1.5 −0.5

waveform 2 waveforms

1 2 −0.5

waveform 3 waveforms

1 2 3 −0.5

waveform 4 waveforms

1 2 3 4 −1 1 2 −0.5

time [s]

−1 1 2

time [s]

−1 1 2

time [s]

−1 1 2

time [s]

−1 1 2

waveform 5 time [s] waveforms epochs

50 100 150 1 2 3 4 5

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 35 / 45

Neuroelectrical Signal Analysis

Adaptive Waveform Learning (AWL): epoched

Comparison with PCA, ICA

−0.4 −0.2 0.2

waveform 1 PCA waveform 2 waveform 3 waveform 4 waveform 5 waveforms abs coefficients

1 2 3 4 5 0.5 1 1.5 −0.4 −0.2 0.2

ICA169 waveforms

1 2 3 4 5 0.5 1 1.5 −0.4 −0.2 0.2

ICA10 waveforms

1 2 3 4 5 0.5 1 1.5 −0.4 −0.2 0.2

ICA5 waveforms

1 2 3 4 5 0.5 1 1.5 −1 1 2 −0.4 −0.2 0.2

time [s] AWL

−1 1 2

time [s]

−1 1 2

time [s]

−1 1 2

time [s]

−1 1 2

time [s] epochs waveforms

50 100 150 1 2 3 4 5 0.5 1 1.5

PCA/ICA: sinusoidal oscillatory component, separation not proper PCA: first component overly dominant

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 36 / 45

Neuroelectrical Signal Analysis

Adaptive Waveform Learning (AWL): continuous

Hierarchical (start k = 1)

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 37 / 45

Neuroelectrical Signal Analysis

Adaptive Waveform Learning (AWL): continuous

Processing full Local Field Potential recording Learning 5 waveforms

Spike shapes vary in duration and amplitude Coefficients are clustered Inter-spike interval correlates with energy

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 38 / 45

Neuroelectrical Signal Analysis

Adaptive Waveform Learning (AWL): continuous

Neurovascular coupling

CBF around 1 Hz (respiration) Spiking rate matches CBF component Spikes synchronize, phase-locked to CBF Reversed causality: haemodynamics drives spikes

670 675 680 685 690 695 700 705 710 715 720 −800 −600 −400 −200 200 µV time [s] 670 675 680 685 690 695 700 705 710 715 720 −0.5 0.5 aribtrary units (a. u.) LFP CBF bandpass filtered (0.8−1.1 Hz) frequency [Hz] time [s] 500 1000 1500 2000 2500 3000 0.2 0.4 0.6 0.8 1 1.2 50 100 150 200 LSR of detected spikes frequency [Hz] time [s] 500 1000 1500 2000 2500 3000 0.2 0.4 0.6 0.8 1 1.2 50 100 150 200

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 39 / 45

Current challenges

Outline

1

Introduction

2

Inverse Source Reconstruction

3

Neuroelectrical signal analysis

4

Current challenges

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 40 / 45

Current challenges

Neurological data analysis progress

Multi- → Single-trial

Automatic event detection Statistical significance

Group → Individual

Clinical: diagnosis, prognosis, biomarkers Cognitive: individual traits, correlations

Static → Adaptive

Time drifting Co-adaptive systems

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 41 / 45

Current challenges

Current challenges in Variability

Variability in: time lag / time duration / amplitude / phase Heterogeneous dimensions: sensors × time × trials × subjects Adaptive Waveform Learning

templates and their variations progressivity in learning specialized to specific study/dataset (a priori info)

Advanced Statistical Models

capturing source of variability Information Geometry space of covariance matrices

New opportunity: availability of large multi-subject datasets Human Connectome Brain Computer Interface / Decoding Challenges

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 42 / 45

Current challenges

Current challenges in Multimodality

Concurrent recordings via several modalities Understanding physiological origin of signals Coupling different generative models Multiscale integration

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 43 / 45

Current challenges

Current challenges in Adaptation

Related to variability and to online processing Online analysis: tracking of variability

Baseline adaptation in monitoring Classifier adaptation for brain computer interfaces

Adapting experimental protocol

Maximizing yield of an experiment of limited duration Hypothesis testing (cognitive and clinical neuroscience) adaptive design optimization

Adapting stimulation parameters

Implant tuning Open-loop → Closed-loop

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 44 / 45

Current challenges

Acknowledgements

Colleagues Th´ eo Alex Papadopoulo Gramfort Christian Bruno B´ enar Torr´ esani Jean-Michel Juliette Badier Leblond Jan Rachid Kybic Deriche PhD Students Anne-Charlotte Sebastian Philippe Hitziger Emmanuel Sylvain Olivi Vallagh´ e Christos Kai Papageorgakis Dang Brahim Balaoucha

Maureen Clerc (Inria) Exploring the Brain’s Activity EUSIPCO, Nice, September 2015 45 / 45