High-dimensional geometry of cortical population activity Marius - - PowerPoint PPT Presentation

High-dimensional geometry of cortical population activity

Marius Pachitariu University College London

• Part I: introduction to “the brave new world of large-scale neuroscience”
• Part II: large-scale data preprocessing with Suite2p
• Part III: large-scale data analysis
• Visual stimulus responses
• Ongoing spontaneous activity
• Behaviorally-related activity

The brave new world of large-scale neuroscience*

However, we are accelerating!!!

*Gao and Ganguli, Current Opinion in Neurobiology 2015 Stevenson & Kording, 2011 2017 prediction: ∼200 neurons

“Standard”, ∼200 cell recordings

https://www.youtube.com/watch?v=xr-flH2Ow2Y

10x real time Conventional resonant 2p scope 12 planes @ 2.5 Hz / plane GCaMP6s in excitatory neurons (Ai94, EMX-Cre) Layers 2/3 and 4

“Zoomed out”, multiplane imaging, ∼10,000 cell recordings https://www.youtube.com/watch?v=xr-flH2Ow2Y

2016, year of the mesoscopes

Sofroniew et al, 2016, eLife Chen et al, 2016, eLife Stirman et al, 2016, Nature Methods Nadella et al, 2016, Nature Methods

2016, year of the mesoscopes 2017, year of the high-density probes

70 μm Neuropixels probe 960 sites 1 cm 384 channels digitized

Right now, we can record

1,000 neurons with electrodes 10,000 neurons with two-photon But why do we need to record so many neurons? Is it really necessary?

How do we make sense of this kind of data?

• dimensionality reduction

data B x

10,000 neurons stimuli

number of dimensions

• measure tuning to

stimuli

• relate to behavior,

decision-making, perception

• etc.
• then do statistics

Lo Low dim dimensio ional? l?

• good for us

we only need to record a subset of all neurons

• bad for the

e brain no room for complex computations, wasted neurons

High gh dim dimensio ional?

• bad for us

we need to record a LOT of neurons

• good for the

e brain complex computations, like object recognition in deep networks

Is cortical activity:

MOUNTCASTLE

• All neurons in a column encode the

same quantity, redundantly BARLOW

• Cortex recodes into a high dimensional sparse code

Classical theories of visual cortex

Low dimensional input in few neurons Low dimensional dense code in many neurons High dimensional sparse code

Expansion Nonlinear transformation Thalamocortical inputs Cortical membrane potentials Cortical spiking Barlow, Possible principles underlying the transformations of sensory messages, 1961 Bosking et al, 1997, J Neurosci

Gao and Ganguli, Curr. Op. Neuro. 2015

Is cortical activity low or high dimensional?

Gao,…,Ganguli, CoSyNe 2014

• we cannot really know yet
• not enough recorded neurons, stimuli

Our study

• we recorded ∼ 10,000 neurons
• we showed ∼ 3,000 stimuli
• long periods of spontaneous activity (2 hours)

Is cortical activity low or high dimensional?

data

10,000 neurons 2,800 stimuli

data

100 neurons 100 trials

Multiplane imaging in visual cortex of awake mice

10x real time Conventional resonant 2p scope 12 planes @ 2.5 Hz / plane GCaMP6s in excitatory neurons (Ai94, EMX-Cre) Layers 2/3 and 4

https://www.youtube.com/watch?v=xr-flH2Ow2Y

Suite2p pipeline

Cell detection model 𝑠𝑙 = ෍

𝑗

Λ𝑙𝑗 𝒈𝑗 + 𝛽𝑙 ෍

𝑘

𝐶𝑙𝑘𝒐𝑘 + 𝜃

• 𝑠𝑙 is the timecourse of pixel 𝑙
• Λ𝑙𝑗 is the weight of the ROI 𝑗 onto pixel 𝑙
• 𝒈𝑗 is the timecourse of ROI 𝑗
• B𝑙𝑘 is the weight of the background component 𝑘 onto pixel 𝑙
• 𝒐𝑘 is the timecourse of background component 𝑘
• 𝜃 is some noise, which we’re going to model as Gaussian

12,392 neurons

Processed in 2 hours

• n a GPU by Suite2p

Registration

Th The effect of

• f the

the ba background sign signal l on

• n fluo

fluorescence at t the the som soma

Mo Model elli ling the the ba background sig signal is is rea eall lly im impo portant!!!

Graphical user interface for quality control

Comparing with the other major pipeline (Pnevmatikakis et al) …we find more cells!

The activity of boutons (pre-synaptic terminals)

The activity of dendrites and spines (post-synaptic terminals)

Spike deconvolution

ሻ 𝑫 𝐭 = 𝐆 − 𝐭 ∗ 𝐥 2 + 𝜇 ⋅ 𝑀(𝐭 𝐆 is the fluorescence of one cell 𝐥 is the calcium response kernel 𝐭 is the actual spike train 𝜇𝑀 𝒕 is a regularization penalty

We have >10,000 cells, now what?

example neuron more examples mean (12,392 neurons)

• 200
• 100

100 200

degrees from preferred direction

0.2 0.4 0.6

responses (test data) Gaussian fit sd = 11.4 deg

Neural tuning to drifting gratings

Responses to visual stimuli

100 of 3,000 stimuli 300 of 10,000 neurons 9 of 3,000 stimuli (presented twice – over 2 hours)

data

Dimensionality estimation data B x

Neurons Stimuli Stimuli Neurons

≈

Dimensions

Linear model

32 directions

more diverse stimuli = more dimensions

Model (linear)

data B x

1 4 8 12 16 0.5 1

explained variance fraction number of dimensions

32 nat scenes

number of dimensions

1 4 8 12 16 0.5 1

Dimensionality of thousands of stimuli

repeat 1 repeat 2

Model (linear)

data B x

16 128 1024

number of dimensions

0.5 1

explained variance fraction

• compute signal variance
• fit model to each repeat
• unexplained variance

= signal variance of residuals of model fit

upper bound ~1,000

Nonli linear dimensionality reduction

Hypothetical scenario Dimensionality is

• 2 --- linear
• 1 --- nonlinear

10 20 30 40

neuron 1

10 20 30 40 50

neuron 2

Defining nonlinear dimensionality

Linear dimension: 2 Nonlinear dimension: 1 Ambient dimension: 3

Nonli linear dimensionality reduction

Model (nonlinear)

data = f(Bx)

f

Model (linear)

data B x

linear nonlinear

~4x fewer dimensions in nonlinear model

16

• rientations

32 directions 32 nat scenes linear nonlinear

16 128 1024 Number of dimensions Nonlinear model 4 95%

2, 2,800 na natural l im images, , rep epeated tw twic ice, , ~4x fewer dimensions in nonlinear model

16 128 1024 Number of dimensions Linear model Explained variance (%) 4 95% 50 100

16

• rientations

linear nonlinear

Number of dimensions

45 90 135

• rientation (deg)

threshold basis function reconstruction 45 90 135

• rientation (deg)

response

recorded neuron #3943 rectified fit

How can the nonlinear dimensionality be so much lower? basis functions B

45 90 135

• rientation (deg)

16 128 1024 Number of dimensions Nonlinear model 4 95%

Dimensionality is higher than predicted by filtering images

16 128 1024 Number of dimensions Linear model Explained variance (%) 4 95% 50 100 Gabor filters Gabor filters

16 128 1024 Number of dimensions Nonlinear model 4

Did we present enough stim timuli?

16 128 1024 Number of dimensions Linear model Explained variance (%) 4 95% 50 100 1,000 2,000 Number of stimuli Dimensions to explain 95% variance 300 600 900 3,000 more stimuli more stimuli

• Nope. No sign of saturation.

16 128 1024 Number of dimensions Nonlinear model 4

Did we record enough ne neurons?

16 128 1024 Number of dimensions Linear model Explained variance (%) 4 95% 50 100 Number of neurons Dimensions to explain 95% variance more neurons more neurons 2,000 4,000 300 600 900 6,000

• Nope. No sign of saturation.

The sensory cortex

What about spontaneous activity?

1,500 of 13,451 neurons during spontaneous activity

30 minutes 1,500 neurons

Top principal component data B x

Neurons Time Time Neurons

≈

Dimensions

Linear model Top principal component

Same neurons, reordered by first principal component

PC1 1,500 neurons

The first principal component is… the the pu pupil il

Pupil area 1,500 neurons PC1

Non-negative matrix factorization (is kind of like clustering) data B x

Neurons Time Time Neurons

≈

Dimensions

Non-negative constraints B>0 X>0

1,500 neurons

Same neurons, organized into clusters

Pairwise spontaneous correlations are consistent

Correlation matrices (10 out of 12,384 neurons) 1st half of data 2nd half of data

• similarity of matrices: 90.34% common variance

Ho How many dim dimensions s of f sp spontaneous act ctivity y acc ccount for th the correlation matrix?

Decomposing the correlation matrix Correlation matrix B B𝑈

Neurons Neurons Neurons

≈

Dimensions Dimensions Neurons

Z-score (data) B 𝑦1

Neurons Time Time Neurons

≈

Dimensions Two different time periods

𝑦2

Time

B

Neurons Dimensions

Spontaneous activity: dimensionality

variance explained

number of dimensions

data B x

10,000 neurons 20,000 timepoints

explained variance

• f the correlation matrix

Have we recorded enough neurons? Yes!

increasing number of neurons

Does spontaneous activity resemble stimulus responses? Can we visualize them together?

Kenet et al, Nature, 2003 Ringach, Curr Op Neurobiol 2009 Berkes et al, Science 2011 (and many more)

Visualizing together spontaneous and stimulus components

• neural component 𝝃 = vector in ℝ𝑂 space
• 1. trial-averaged responses to a stimulus
• 2. principal component of spontaneous activity

data

neurons time

𝝃

component activity

20 minutes

Stimuli Stimuli

Signal dimensions Spont dimensions

Summary of scientific results

Stimulus-driven activity in visual cortex is high dimensional: >1,000 linear dimensions, >300 nonlinear dimensions, no sign of hitting a limit. Dimensionality is higher than predicted from image filtering. Consistent with the efficient coding hypothesis. Spontaneous activity in visual cortex is relatively low dimensional: ~50 dimensions, does not go up with increasing number of neurons Encodes behavioral state and reflects brain-wide activity Does not resemble stimulus-driven activity. Is uninterrupted by sensory activity

Acknowledgements

Carsen Stringer Nick Steinmetz Kenneth Harris Matteo Carandini