Course Introduction Neural Information Processing: Introduction - PowerPoint PPT Presentation

Course Introduction Neural Information Processing: Introduction Welcome and administration Matthias Hennig and Mark van Rossum Course outline and context A short neuroscience summary School of Informatics, University of Edinburgh January 2018 1 / 17 2 / 17 Administration Notes You need a good grounding in maths, specifically in probability and statistics vectors and matrices Assessment is an exam (75%) and coursework (25%) Assignments: We do not expect any background in neurobiology. Assignment 1: 28 March 2018, 4pm (tbc) We will work on the board a lot - some material will be covered in Assignment 2: 3 April 2018, 4pm (tbc) handouts, but make sure to take notes. A1 will be an exercise, A2 will be on class papers. Interrupt and ask questions in class if something is unclear, or you feel more explanation is useful. Treat everything we say as ’examinable’, except where we explicitly say otherwise. Any questions/issues - please email us. 3 / 17 4 / 17

Course aims Reading materials Theoretical Neuroscience by P Dayan and L F Abbott (MIT Press 2001) This course will explore Neuronal Dynamics by Wulfram Gerstner, Werner M. Kistler, how the brain computes, Richard Naud and Liam Paninski (http://neuronaldynamics.epfl.ch/) how neuroscience can inspire technology, Information Theory, Inference and Learning Algorithms by David how computer science can help address important questions in MacKay (http://www.inference.phy.cam.ac.uk/itila/book.html) neuroscience. Introduction to the Theory of Neural Computation, by John Hertz et al. and literature cited in the lecture notes/slides 5 / 17 6 / 17 Relationships between courses Course outline Computational methods to get better insight in neural coding and 1 computation: Neural code is complex: distributed and high dimensional NC Wider introduction, more biological, but less abstract than NIP Data collection is improving (van Rossum, first term) Biologically inspired algorithms and hardware. 2 CCN Cognition and coding, high level understanding (Series) Topics covered: PMR Pure ML perspective (Guttmann) Neural coding: encoding and decoding. Statistical models: modelling neural activity and neuro-inspired machine learning. Unconventional computing: dynamics and attractors. 7 / 17 8 / 17

Linsker (1988) Real Neurons The fundamental unit of all nervous system tissue is the neuron R. Linsker, IEEE Computer Magazine, March 1988 Axonal arborization Might there be organizing principles Axon from another cell that explain some essential aspects of how a perceptual system 1 Synapse develops and functions, Dendrite Axon that we can attempt to infer without waiting for far more detailed 2 experimental information, Nucleus that can lead to profitable experimental programs, testable 3 Synapses predictions, and applications to synthetic perception as well as to Cell body or Soma neuroscientific understanding. [Figure: Russell and Norvig, 1995] 9 / 17 10 / 17 Each neuron can form synapses with anywhere between 10 and 10 5 other neurons Signals are propagated at the synapse through the release of A neuron consists of chemical transmitters which raise or lower the electrical potential a soma , the cell body, which contains the cell nucleus of the cell dendrites : input fibres which branch out from the cell body When the potential reaches a threshold value , an action an axon : a single long (output) fibre which branches out over a potential is sent down the axon distance that can be up to 1m long This eventually reaches the synapses and they release synapse : a connecting junction between the axon and other cells transmitters that affect subsequent neurons Synapses can be inhibitory (lower the post-synaptic potential) or excitatory (raise the post-synaptic potential) Synapses can also exhibit long term changes of strength (plasticity) in response to the pattern of stimulation 11 / 17 12 / 17

Assumptions Recent developments: Neurobiology technique Spikes are assumed to be the fundamental information carrier We will ignore non-linear interactions between inputs Spikes can be modelled as rate-modulated random processes We will ignore biophysical details [Stevenson and Kording, 2011] Recordings from many neurons at once (Moore’s law) 13 / 17 14 / 17 Recent developments: Computing Hardware Recent developments: Machine Learning [Le et al., 2012] Neural network algorithms, developed 30 years ago, were [Furber et al., 2014] considered superseeded. Single CPU speed limit reached But now, using GPUs and big data, they are top performers in Renewed call for parallel hardware and algorithms, including vision, audition and natural language. brain-inspired ones (slow, noisy, enery-efficient). 15 / 17 16 / 17

References I Furber, S. B., Galluppi, F ., Temple, S., and Plana, L. A. (2014). The spinnaker project. Proc IEEE , 102(5):652–665. Le, Q. V., Ranzato, M., Monga, R., Devin, M., Chen, K., Corrado, G. S., Dean, J., and Ng, A. Y. (2012). Building high-level features using large scale unsupervised learning. In ICM . ICML 2012: 29th International Conference on Machine Learning, Edinburgh, Scotland, June, 2012. Stevenson, I. H. and Kording, K. P . (2011). How advances in neural recording affect data analysis. Nat Neurosci , 14(2):139–142. 17 / 17