Nilearn: Machine learning for brain imaging in Python Ga el - - PowerPoint PPT Presentation

▶

Jan 29, 2024 458 likes •805 views

Nilearn: Machine learning for brain imaging in Python Ga el Varoquaux INRIA/Parietal 1 Magnetic Resonance Imaging of the brain 2 Machine learning and brain imaging 3 NiLearn G Varoquaux 2 1 Magnetic Resonance Imaging of the brain G

SLIDE 1

Nilearn:

Machine learning for brain imaging in Python

Ga¨ el Varoquaux

INRIA/Parietal

SLIDE 2

1 Magnetic Resonance Imaging of the brain 2 Machine learning and brain imaging 3 NiLearn

G Varoquaux 2

SLIDE 3

1 Magnetic Resonance Imaging

f the brain

G Varoquaux 3

SLIDE 4

1 anatomical MRI Lesions? Bleeding? Shape, cortical thickness

G Varoquaux 4

SLIDE 5

1 functional MRI (fMRI)

t

Time-resolved recordings of brain activity

G Varoquaux 5

SLIDE 6

1 Mapping cognitive processes with fMRI Stimulus Activation maps

G Varoquaux 6

SLIDE 7

2 Machine learning and brain imaging

G Varoquaux 7

SLIDE 8

Medical applications

G Varoquaux 8

SLIDE 9

2 Some prediction problem Diagnosis Finding the nature or cause of a disease condition Pronosis Predicting the future evolution of the condition ⇒ Therapeutic indications Early biomarkers Measures enabling the detection of disease before standard symptoms ⇒ Population screening Quantitative biomarkers Metric to follow disease progression ⇒ Drug development

G Varoquaux 9

SLIDE 10

2 More than prediction accuracy Cannot replace the physician: Patient history Therapeutic strategies subject to logistics ... ⇒ No black-box Segmentation, denoising task as much as prediction

G Varoquaux 10

SLIDE 11

2 More than prediction accuracy Cannot replace the physician: Patient history Therapeutic strategies subject to logistics ... ⇒ No black-box Segmentation, denoising task as much as prediction

G Varoquaux 10

SLIDE 12

Understanding brain function Cognitive neuroimaging: from neural activity to thoughts

G Varoquaux 11

SLIDE 13

2 Machine learning for cognitive neuroImaging

[Varoquaux & Thirion, 2014]

Learn a bilateral link between brain activity and cognitive function

G Varoquaux 12

SLIDE 14

2 Machine learning for cognitive neuroImaging

[Varoquaux & Thirion, 2014]

Predicting neural response: encoding models

G Varoquaux 12

SLIDE 15

2 Machine learning for cognitive neuroImaging

[Varoquaux & Thirion, 2014]

“Brain reading”: decoding

G Varoquaux 12

SLIDE 16

3 NiLearn

Machine learning for Neuro-Imaging in Python http://nilearn.github.io

ni

G Varoquaux 13

SLIDE 17

3 Going beyond the IEEE publication How to we reach our target audience (neuroscientists)? For neuroscience research How do we disseminate our ideas? For applied-math research How do we facilitate new ideas? For our own lab

G Varoquaux 14

SLIDE 18

3 Going beyond the IEEE publication How to we reach our target audience (neuroscientists)? For neuroscience research How do we disseminate our ideas? For applied-math research How do we facilitate new ideas? For our own lab

G Varoquaux 14

SLIDE 19

3 6 years ago Visual image reconstruction from human brain activity [Miyawaki, et al. (2008)] “brain reading”

G Varoquaux 15

SLIDE 20

3 6 years ago ... back to the future Visual image reconstruction from human brain activity [Miyawaki, et al. (2008)] “if it’s not open and verifiable by others, it’s not science, or engineering...” Stodden, 2010

G Varoquaux 15

SLIDE 21

3 6 years ago ... back to the future Visual image reconstruction from human brain activity [Miyawaki, et al. (2008)] Make it work, make it right, make it boring

SLIDE 22

3 6 years ago ... back to the future Visual image reconstruction from human brain activity [Miyawaki, et al. (2008)] Make it work, make it right, make it boring

http://nilearn.github.io/auto examples/ plot miyawaki reconstruction.html

Code, data, ... just worksTM

http://nilearn.github.io

ni

G Varoquaux 15

SLIDE 23

3 Nilearn: making learning for neuroimaging routine

Project scope CDS-funded Machine learning for neuroimaging: make using scikit-learn on neuroimaging easy The target user base is small Examples in the docs Run out of the box, downloading open data Produce a clear figure

Data from Miyawaki 2008

Routine, simple, reproduction of papers

G Varoquaux 16

SLIDE 24

3 Challenges we have to solve Getting the data Struggle for open data Massaging the data for machine-learning Very simple signal processing Documentation Users do not know what they need Output + visualization of results Putting it in application terms

G Varoquaux 17

SLIDE 25

3 Nilearn in practice Getting the data

f i l e s = d a t a s e t s . f e t c h h a x b y ()

Caching of the downloads Resume of partial downloads

G Varoquaux 18

SLIDE 26

3 Nilearn in practice Getting the data

f i l e s = d a t a s e t s . f e t c h h a x b y ()

Massaging the data for machine-learning

masker = N i f t i M a s k e r ( mask img =’mask.nii’, s t a n d a r d i z e = True ) data = masker . f i t t r a n s f o r m (’fmri.nii’)

Filenames to data matrix (memory-efficient I/O) Common preprocessing steps included

G Varoquaux 18

SLIDE 27

3 Nilearn in practice Getting the data

f i l e s = d a t a s e t s . f e t c h h a x b y ()

Massaging the data for machine-learning

masker = N i f t i M a s k e r ( mask img =’mask.nii’, s t a n d a r d i z e = True ) data = masker . f i t t r a n s f o r m (’fmri.nii’)

Learning with scikit-learn

e s t i m a t o r . f i t ( data , l a b e l s )

That’s easy!

G Varoquaux 18

SLIDE 28

3 Nilearn in practice Getting the data

f i l e s = d a t a s e t s . f e t c h h a x b y ()

Massaging the data for machine-learning

masker = N i f t i M a s k e r ( mask img =’mask.nii’, s t a n d a r d i z e = True ) data = masker . f i t t r a n s f o r m (’fmri.nii’)

Learning with scikit-learn

e s t i m a t o r . f i t ( data , l a b e l s )

Output

p l o t s t a t m a p ( masker . i n v e r s e t r a n s f o r m ( e s t i m a t o r . w e i g h t s ))

G Varoquaux 18

SLIDE 29

3 Nilearn in practice Getting the data

f i l e s = d a t a s e t s . f e t c h h a x b y ()

Massaging the data for machine-learning

masker = N i f t i M a s k e r ( mask img =’mask.nii’, s t a n d a r d i z e = True ) data = masker . f i t t r a n s f o r m (’fmri.nii’)

Learning with scikit-learn

e s t i m a t o r . f i t ( data , l a b e l s )

Output

p l o t s t a t m a p ( masker . i n v e r s e t r a n s f o r m ( e s t i m a t o r . w e i g h t s ))

Demo

Brain reading @ home G Varoquaux 18

SLIDE 30

3 There is more Domain-specific brain-reading algorithm Image-penalties on linear models Unsupervised dictionary-learning Brain regions from uncontrolled mental activity Graph learning “Connectome”: who talks to who

G Varoquaux 19

SLIDE 31

3 NeuroSynth + Neurovault: web brain reading

G Varoquaux 20

SLIDE 32

ni

Nilearn: Machine learning for brain imaging Medical and cognitive science applications Learning problems, but not

nly about prediction error

Nilearn:

Machine learning for brain imaging in Python

Ga¨ el Varoquaux

1 Magnetic Resonance Imaging of the brain 2 Machine learning and brain imaging 3 NiLearn

1 Magnetic Resonance Imaging

1 anatomical MRI Lesions? Bleeding? Shape, cortical thickness

1 functional MRI (fMRI)

t

Time-resolved recordings of brain activity

1 Mapping cognitive processes with fMRI Stimulus Activation maps

2 Machine learning and brain imaging

Medical applications

2 More than prediction accuracy Cannot replace the physician: Patient history Therapeutic strategies subject to logistics ... ⇒ No black-box Segmentation, denoising task as much as prediction

2 More than prediction accuracy Cannot replace the physician: Patient history Therapeutic strategies subject to logistics ... ⇒ No black-box Segmentation, denoising task as much as prediction

Understanding brain function Cognitive neuroimaging: from neural activity to thoughts

2 Machine learning for cognitive neuroImaging

Learn a bilateral link between brain activity and cognitive function

2 Machine learning for cognitive neuroImaging

Predicting neural response: encoding models

2 Machine learning for cognitive neuroImaging

“Brain reading”: decoding

3 NiLearn

Machine learning for Neuro-Imaging in Python http://nilearn.github.io

ni

3 Going beyond the IEEE publication How to we reach our target audience (neuroscientists)? For neuroscience research How do we disseminate our ideas? For applied-math research How do we facilitate new ideas? For our own lab

3 Going beyond the IEEE publication How to we reach our target audience (neuroscientists)? For neuroscience research How do we disseminate our ideas? For applied-math research How do we facilitate new ideas? For our own lab

3 6 years ago Visual image reconstruction from human brain activity [Miyawaki, et al. (2008)] “brain reading”

3 6 years ago ... back to the future Visual image reconstruction from human brain activity [Miyawaki, et al. (2008)] “if it’s not open and verifiable by others, it’s not science, or engineering...” Stodden, 2010

3 6 years ago ... back to the future Visual image reconstruction from human brain activity [Miyawaki, et al. (2008)] Make it work, make it right, make it boring

3 6 years ago ... back to the future Visual image reconstruction from human brain activity [Miyawaki, et al. (2008)] Make it work, make it right, make it boring

Code, data, ... just worksTM

ni

3 Nilearn: making learning for neuroimaging routine

Project scope CDS-funded Machine learning for neuroimaging: make using scikit-learn on neuroimaging easy The target user base is small Examples in the docs Run out of the box, downloading open data Produce a clear figure

Routine, simple, reproduction of papers

3 Challenges we have to solve Getting the data Struggle for open data Massaging the data for machine-learning Very simple signal processing Documentation Users do not know what they need Output + visualization of results Putting it in application terms

3 Nilearn in practice Getting the data

f i l e s = d a t a s e t s . f e t c h h a x b y ()

Caching of the downloads Resume of partial downloads

3 Nilearn in practice Getting the data

f i l e s = d a t a s e t s . f e t c h h a x b y ()

Massaging the data for machine-learning

masker = N i f t i M a s k e r ( mask img =’mask.nii’, s t a n d a r d i z e = True ) data = masker . f i t t r a n s f o r m (’fmri.nii’)

Filenames to data matrix (memory-efficient I/O) Common preprocessing steps included

3 Nilearn in practice Getting the data

f i l e s = d a t a s e t s . f e t c h h a x b y ()

Massaging the data for machine-learning

masker = N i f t i M a s k e r ( mask img =’mask.nii’, s t a n d a r d i z e = True ) data = masker . f i t t r a n s f o r m (’fmri.nii’)

Learning with scikit-learn

e s t i m a t o r . f i t ( data , l a b e l s )

That’s easy!

3 Nilearn in practice Getting the data

f i l e s = d a t a s e t s . f e t c h h a x b y ()

Massaging the data for machine-learning

masker = N i f t i M a s k e r ( mask img =’mask.nii’, s t a n d a r d i z e = True ) data = masker . f i t t r a n s f o r m (’fmri.nii’)

Learning with scikit-learn

e s t i m a t o r . f i t ( data , l a b e l s )

Output

p l o t s t a t m a p ( masker . i n v e r s e t r a n s f o r m ( e s t i m a t o r . w e i g h t s ))

3 Nilearn in practice Getting the data

f i l e s = d a t a s e t s . f e t c h h a x b y ()

Massaging the data for machine-learning

masker = N i f t i M a s k e r ( mask img =’mask.nii’, s t a n d a r d i z e = True ) data = masker . f i t t r a n s f o r m (’fmri.nii’)

Learning with scikit-learn

e s t i m a t o r . f i t ( data , l a b e l s )

Output

p l o t s t a t m a p ( masker . i n v e r s e t r a n s f o r m ( e s t i m a t o r . w e i g h t s ))

Demo

3 There is more Domain-specific brain-reading algorithm Image-penalties on linear models Unsupervised dictionary-learning Brain regions from uncontrolled mental activity Graph learning “Connectome”: who talks to who

3 NeuroSynth + Neurovault: web brain reading

ni

Nilearn: Machine learning for brain imaging Medical and cognitive science applications Learning problems, but not

Reaching domain scientists First challenge: get the user to do simple tasks Useful for methods research lowers the bar to test methods on new data

@GaelVaroquaux