Global convergence of neuron birth-death dynamics ICML 2019 Long - PowerPoint PPT Presentation

Nov 27, 2022 •171 likes •252 views

Global convergence of neuron birth-death dynamics ICML 2019 Long Beach, CA 13 June 2019 Grant M. Rotskofg, Samy Jelassi, Joan Bruna, and Eric Vanden-Eijnden Center for Data Science, New York University Courant Institute of Mathematical

Global convergence of neuron birth-death dynamics ICML 2019 Long Beach, CA 13 June 2019 Grant M. Rotskofg, Samy Jelassi, Joan Bruna, and Eric Vanden-Eijnden Center for Data Science, New York University Courant Institute of Mathematical Sciences, New York University
Synopsis of main results →! Modifjcation of gradient descent (GD) with unbalanced optimal transport ● Parameter birth-death process ● Proof of global convergence ● Rate of convergence scales as →! Based on mean-fjeld perspective on neural networks ● Analysis of deterministic partial difgerential equation (PDE) ● PDE leads to practical, effjcient algorithm →! Experiments show faster convergence relative to GD ● Illustrative examples show efgect of transport / exploration ● Easy to implement with no additional gradient computations ICML 13 Jun 2019
Single hidden layer neural network d -dim input PDE for parameter distribution Distinct from kernel learning / NTK, dynamics leads to feature selection ICML 13 Jun 2019
Non-local mass transport (particle birth-death) Parameters are Total population killed / cloned is fjxed ICML 13 Jun 2019
Theorem [R,J,B,V-E]: global convergence Compare with [Chizat & Bach, 2018] without any restriction on homogeneity for the units ICML 13 Jun 2019
Dramatic efgect of birth/death dynamics ICML 13 Jun 2019
Come see our poster! Thu Jun 13th 06:30 -- 09:00 PM @ Pacifjc Ballroom #93 Grant Rotskofg Samy Jelassi Joan Bruna Eric Vanden-Eijnden arXiv:1902.01843 ICML 13 Jun 2019

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Birth and Death Processes Today: Birth processes Birth and Death Processes Death

Birth and Death Processes Today: Birth processes Birth and Death Processes Death processes Biarth and death processes Bo Friis Nielsen 1 Limiting behaviour of birth and death processes Next week 1 DTU Informatics Finite state

309 views • 7 slides

Birth-Death Processes Birth-Death Processes: Transient Solution Poisson Process: State

Birth-Death Processes Birth-Death Processes: Transient Solution Poisson Process: State Distribution Poisson Process: Inter-arrival Times Dr Conor McArdle EE414 - Birth-Death Processes 1/17 Birth-Death Processes Birth-Death Processes are

282 views • 17 slides

for information such as the birth place, birth date and parents' names of the deceased. What

Indexes and Sources of Info on Deaths, Obituaries, Burials Death records are primary source records because they are completed at, or close to, the time of the death by someone who was present at the death. Death records are especially helpful,

66 views • 5 slides

Using Single Neuron Dynamics to Predict Synchronous Global Network Activities Robert Kim

Using Single Neuron Dynamics to Predict Synchronous Global Network Activities Robert Kim Neurosciences Graduate Program University of California, San Diego December 8, 2017 Neurodynamics Final Project December 8, 2017 1 / 16 Outline

761 views • 29 slides

Theoretical neuroscience: From single neuron to network dynamics Nicolas Brunel Outline

Theoretical neuroscience: From single neuron to network dynamics Nicolas Brunel Outline Single neuron stochastic dynamics Network dynamics Learning and memory Single neurons in vivo seem highly stochastic Single neuron stochastic

475 views • 19 slides

Birth and Fetal Death Reporting Enhanced (BFDR-E) and Vital Records Death Reporting (VRDR)

Birth and Fetal Death Reporting Enhanced (BFDR-E) and Vital Records Death Reporting (VRDR) Alaina Elliott Lori Reed-Fourquet Consultant IT Specialist E-HealthSign Centers for Disease Control and Prevention National Center for Health

294 views • 6 slides

Calculating bounds on expected return and first passage times in finite-state imprecise

Calculating bounds on expected return and first passage times in finite-state imprecise birth-death chains Stavros Lopatatzidis, Jasper De Bock and Gert de Cooman Birth-death chains Birth-death chain special type of Markov chain X : =

404 views • 15 slides

Death before Birth Understanding, informing and supporting the choices made by people who have

Death before Birth Understanding, informing and supporting the choices made by people who have experienced miscarriage, termination, and stillbirth Overall project aim: To examine the law surrounding the disposal of the remains of pregnancy

362 views • 21 slides

TxEVER Birth and Death for Local Registrars CDCAT Fall Conference Derek Johnson, Field

TxEVER Birth and Death for Local Registrars CDCAT Fall Conference Derek Johnson, Field Services, VSS 0 2 Aug 2 0 1 8 Agenda This conference call presentation will cover the following: 1 . Birth Local Registration a. Local

527 views • 28 slides

Shift techniques for Quasi-Birth-and-Death processes: canonical factorizations and matrix

Shift techniques for Quasi-Birth-and-Death processes: canonical factorizations and matrix equations Beatrice Meini Universit` a di Pisa Joint work with D.A. Bini and G. Latouche MAM9 - Budapest QBD processes Let A A 0 0

680 views • 24 slides

Cause-of-Death Mortality: What Can Be Learned From Population Dynamics? A Study of a

Cause-of-Death Mortality: What Can Be Learned From Population Dynamics? A Study of a Heterogeneous Portfolio Dynamic Hlose Labit Hardy PhD Student, University of Lausanne, Switzerland joint work with S. Arnold, A. Boumezoued and N. El

211 views • 18 slides

Myra Dioquino Ne Neuron Ne Neuron ons ons ~30x out ~30x in 20 000 20,000x out o t ~30x

Myra Dioquino Ne Neuron Ne Neuron ons ons ~30x out ~30x in 20 000 20,000x out o t ~30x out Describes the equilibrium potential of the cell for a particular ion R Universal gas constant (8.314472 J K ) T

455 views • 9 slides

REQUIEM BY ALFRED SCHNITTKE Composer's name and birth/death dates: Alfred Schnittke (Russian,

REQUIEM BY ALFRED SCHNITTKE Composer's name and birth/death dates: Alfred Schnittke (Russian, German/Jewish, 1938-1996) Full title of composition: Requiem (from the stage music to Don Carlos by Friedich von Schiller) Placement in the

205 views • 3 slides

Spatial Birth-Death Model for Wireless Networks Abishek Sankararaman and Franois Baccelli UT

Spatial Birth-Death Model for Wireless Networks Abishek Sankararaman and Franois Baccelli UT Austin Outline Motivation and Background. Math Model - Interacting Particle System. Summary of Results. Further

1.07k views • 74 slides

Stochastic dynamics of spiking neuron models and implications for network dynamics Nicolas Brunel

Stochastic dynamics of spiking neuron models and implications for network dynamics Nicolas Brunel The question What is the input-output relationship of single neurons? The question What is the input-output relationship of single neurons?

489 views • 32 slides

Population Dynamics Deductive modelling: based on physical laws Inductive modelling: based on

Population Dynamics Deductive modelling: based on physical laws Inductive modelling: based on observation + intuition Single species: Birth (in migration) Rate, Death (out

706 views • 33 slides

Gene Network inference for high-dimensional problems A. Mohammadi and E. Wit March 28, 2013 G

G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES Gene Network inference for high-dimensional problems A. Mohammadi and E. Wit March 28, 2013 G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES M OTIVATION Flow cytometry data with 11 proteins

781 views • 18 slides

APPLY LEARNING FROM KANSAS MATERNAL MORTALITY REVIEW PROCESS Randall Morgan MD Taylor Bertschy

APPLY LEARNING FROM KANSAS MATERNAL MORTALITY REVIEW PROCESS Randall Morgan MD Taylor Bertschy DO Kansas Maternal Mortality and Severe Maternal Morbidity Outcome measures 2010 2011 2012 2013 2014 Trend Severe maternal morbidity per 10,000

370 views • 11 slides

Network determination based on birth-death MCMC inference A. Mohammadi and E. Wit February 4,

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES Network determination based on birth-death MCMC inference A. Mohammadi and E. Wit February 4, 2013 N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC

602 views • 28 slides

Omnithermal Perfect Simulation Stephen Connor stephen.connor@york.ac.uk 61st World Statistics

Omnithermal Perfect Simulation Stephen Connor stephen.connor@york.ac.uk 61st World Statistics Congress Marrakech, July 2017 Introduction Random Cluster Model Area Interaction Process M / G / c Queue Outline Introduction 1 Random Cluster

531 views • 31 slides

Markovian-modulated Models and Their Application Potential Magorzata OReilly

Application examples Classic Markov Chains Markovian-modulated models Markovian-modulated Models and Their Application Potential Magorzata OReilly http://youtu.be/BMaeGBh_Lnc University of Tasmania School of Physical Sciences

588 views • 25 slides

Ciberseguridad Probability, Random Processes and Inference Dr. Ponciano Jorge Escamilla Ambrosio

INSTITUTO POLITCNICO NACIONAL CENTRO DE INVESTIGACION EN COMPUTACION Laboratorio de Ciberseguridad Probability, Random Processes and Inference Dr. Ponciano Jorge Escamilla Ambrosio pescamilla@cic.ipn.mx http://www.cic.ipn.mx/~pescamilla/

1.7k views • 138 slides

Bayesian model selection in graphs by using BDgraph package A. Mohammadi and E. Wit March 26,

G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES Bayesian model selection in graphs by using BDgraph package A. Mohammadi and E. Wit March 26, 2013 G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES M OTIVATION Flow cytometry data with 11

373 views • 20 slides

Combinatorics (2.6) The Birthday Problem (2.7) Prof. Tesler Math 186 Winter 2020 Prof. Tesler

Combinatorics (2.6) The Birthday Problem (2.7) Prof. Tesler Math 186 Winter 2020 Prof. Tesler Combinatorics & Birthday Problem Math 186 / Winter 2020 1 / 29 Multiplication rule Combinatorics is a branch of Mathematics that deals with

327 views • 29 slides