Point process latent variable models of larval zebrafish behavior - PowerPoint PPT Presentation

Point process latent variable models of larval zebrafish behavior Anuj Sharma Robert E. Johnson Florian Engert Scott W. Linderman Columbia University * Harvard University Harvard University Columbia University * Anuj is currently a research engineer at Imagen Technologies

Why larval zebrafish behavior? To understand the computations of the nervous system, we need to understand its behavioral outputs.

Real recording of a freely behaving larval zebrafish

Key questions Q1 : How should we characterize types of swim bouts?

Key questions Q1 : How should we characterize types of swim bouts? Q2 : What dynamics govern how swim bouts are sequenced together over time?

Key questions Q1 : How should we characterize types of swim bouts? Q2 : What dynamics govern how swim bouts are sequenced together over time? Q3 : How are these dynamics modulated by internal states like hunger?

Modeling larval zebrafish behavior as a marked point process time i 1 i 2 i N − 1

Point process latent variable models Full Generative Model observed i n interval observed mark y n (eye and tail movement) ... n = 1 n = N latent observed LSTM state dependency neural net dep. clique

Point process latent variable models Full Generative Model latent mark h n embedding observed i n interval observed mark y n (eye and tail movement) ... n = 1 n = N latent neural net dep. latent observed LSTM state dependency neural net dep. clique

Point process latent variable models Full Generative Model discrete z n latent state latent mark h n embedding observed i n interval observed mark y n (eye and tail movement) ... n = 1 n = N latent observed LSTM state dependency neural net dep. clique

Point process latent variable models Full Generative Model continuous x n latent state discrete z n latent state latent mark h n embedding observed i n interval observed mark y n (eye and tail movement) ... n = 1 n = N latent observed LSTM state dependency neural net dep. clique

Point process latent variable models Full Generative Model parameters θ continuous x n latent state discrete z n latent state latent mark h n embedding observed i n interval observed mark y n (eye and tail movement) ... n = 1 n = N latent neural net dep. latent observed LSTM state dependency neural net dep. clique

Point process latent variable models Collapsed Generative Model Full Generative Model parameters θ continuous x n latent state discrete z n latent state latent mark h n embedding observed i n interval observed mark y n (eye and tail movement) ... ... n = 1 n = N n = 1 n = N latent neural net dep. latent observed LSTM state dependency neural net dep. clique

Point process latent variable models Bidirectional LSTM Collapsed Generative Model Full Generative Model Recognition Network parameters θ continuous x n latent state discrete z n latent state latent mark h n embedding observed i n interval observed mark y n (eye and tail movement) ... ... ... n = 1 n = N n = 1 n = N n = 1 n = N latent observed LSTM state dependency neural net dep. clique

PPLVMs help answer key questions A1: Bouts cluster A1’: Held-out into discrete likelihood offers a types in low-d quantitative latent space. metric for comparing representations.

PPLVMs help answer key questions A1: Bouts cluster A1’: Held-out into discrete likelihood offers a types in low-d quantitative latent space. metric for comparing representations. A2: Bout types follow characteristic Explore transition patterns between J-turn hunting and Pursuit exploring. Hunt-end Pursuit Hunt-end J-turn Explore

PPLVMs help answer key questions A1: Bouts cluster A1’: Held-out into discrete likelihood offers a types in low-d quantitative latent space. metric for comparing representations. A2: Bout types A3: These follow transition characteristic Explore patterns change transition over time as a patterns between function of J-turn hunting and hunger. Pursuit exploring. Hunt-end Pursuit Hunt-end J-turn Explore

Come to our poster! Extend our model to include • Environmental dependencies (prey locations, sizes, dynamics) • Whole brain neural activity dynamics Apply PPLVMs to other domains: • Healthcare • Social media • Consumer behavior Ahrens et al (Nature Methods, 2013) Acknowledgements: Misha Ahrens (video), John Cunningham, Kristian Herrera (animations), Liam Paninski, Haim Sopolinsky (video), SWL: Simons Foundation SCGB-418011; FE: National Institutes of Health’s Brain Initiative U19NS104653, R24NS086601 and R43OD024879, Simons Foundation SCGB-542973 and 325207