neural networks

Neural Networks Part 1 Yingyu Liang yliang@cs.wisc.edu Computer - PowerPoint PPT Presentation

Sep 13, 2023 •356 likes •651 views

Neural Networks Part 1 Yingyu Liang yliang@cs.wisc.edu Computer Sciences Department University of Wisconsin, Madison slide 1 [Based on slides from Jerry Zhu] Motivation I: learning features Example task Experience/Data: images with

  1. Neural Networks Part 1 Yingyu Liang yliang@cs.wisc.edu Computer Sciences Department University of Wisconsin, Madison slide 1 [Based on slides from Jerry Zhu]

  2. Motivation I: learning features • Example task Experience/Data: images with labels indoor Indoor outdoor slide 2

  3. Motivation I: learning features • Featured designed for the example task Extract features Indoor 0 slide 3

  4. Motivation I: learning features • More complicated tasks: hard to design • Would like to learn features slide 4

  5. Motivation II: neuroscience • Inspirations from human brains • Networks of simple and homogenous units slide 5

  6. Motivation II: neuroscience • Human brain: 100, 000, 000, 000 neurons • Each neuron receives input from 1,000 others • Impulses arrive simultaneously • Added together* ▪ an impulse can either increase or decrease the possibility of nerve pulse firing • If sufficiently strong, a nerve pulse is generated • The pulse forms the input to other neurons. • The interface of two neurons is called a synapse http://www.bris.ac.uk/synaptic/public/brainbasic.html slide 7

  7. Successful applications • Computer vision: object location Slides from Kaimin He, MSRA slide 8

  8. Successful applications • NLP: Question & Answer Figures from the paper “Ask Me Anything: Dynamic Memory Networks for Natural Language Processing ”, by Ankit Kumar, Ozan Irsoy, Peter Ondruska, Mohit Iyyer, James Bradbury, Ishaan Gulrajani, Richard Socher slide 9

  9. Successful applications • Game: AlphaGo slide 10

  10. Outline • A single neuron ▪ Linear perceptron ▪ Non-linear perceptron ▪ Learning of a single perceptron ▪ The power of a single perceptron • Neural network: a network of neurons ▪ Layers, hidden units ▪ Learning of neural network: backpropagation ▪ The power of neural network ▪ Issues • Everything revolves around gradient descent slide 11

  11. Linear perceptron • 1 … slide 13

  12. Learning in linear perceptron • slide 14

  13. Learning in linear perceptron slide 15

  14. Visualization of gradient descent slide 16

  15. The (limited) power of linear perceptron • 1 slide 17

  16. Non-linear perceptron • 1 … slide 18

  17. Non-linear perceptron • 1 … Now we see the reason for bias terms slide 19

  18. Non-linear perceptron for AND • slide 20

  19. Example Question • Will you go to the festival? • Go only if at least two conditions are favorable ? Weather Company Proximity All inputs are binary; 1 is favorable slide 21

  20. Example Question • Will you go to the festival? • Go only if Weather is favorable and at least one of the other two conditions is favorable ? Weather Company Proximity All inputs are binary; 1 is favorable slide 22

  21. Sigmod activation function: Our second non-linear perceptron • slide 23

  22. Sigmod activation function: Our second non-linear perceptron • slide 24

  23. Sigmod activation function: Our second non-linear perceptron • slide 25

  24. Learning in non-linear perceptron slide 26

  25. The (limited) power of non-linear perceptron • Even with a non-linear sigmoid function, the decision boundary a perceptron can produce is still linear • AND, OR, NOT revisited • How about XOR? slide 27

  26. The (limited) power of non-linear perceptron • Even with a non-linear sigmoid function, the decision boundary a perceptron can produce is still linear • AND, OR, NOT revisited • How about XOR? • This contributed to the first AI winter slide 28

  27. Brief history of neural networks slide 29

  28. (Multi-layer) neural network • Given sigmoid perceptrons • Can you produce output like • which had non-linear decision boundarys 0 1 0 1 0 slide 30

Recommend

Learning Neural Networks Learning Neural Networks Neural Networks can represent complex Neural

Learning Neural Networks Learning Neural Networks Neural Networks can represent complex Neural

Learning Neural Networks Learning Neural Networks Neural Networks can represent complex Neural Networks can represent complex decision boundaries decision boundaries Variable size. Any boolean function can be Variable size. Any boolean

358 views • 14 slides
Neural Networks and Handwriting Recognition Background Neural Networks Neural Network Steven

Neural Networks and Handwriting Recognition Background Neural Networks Neural Network Steven

Neural Networks and Handwriting Recognition Steven Sloss Math 164 Neural Networks and Handwriting Recognition Background Neural Networks Neural Network Steven Sloss Structure Training Neural Networks Math 164 Motivation Problem

889 views • 41 slides
Neural Networks Neural networks arise from attempts to model Neural Networks human/animal

Neural Networks Neural networks arise from attempts to model Neural Networks human/animal

Feed-forward Networks Network Training Error Backpropagation Deep Learning Feed-forward Networks Network Training Error Backpropagation Deep Learning Neural Networks Neural networks arise from attempts to model Neural Networks

380 views • 9 slides
Sequential Data with Neural Networks Recurrent Neural Networks Sequential input / output Greg

Sequential Data with Neural Networks Recurrent Neural Networks Sequential input / output Greg

Recurrent Neural Networks Long Short-Term Memory Temporal Convolutional Networks Examples Recurrent Neural Networks Long Short-Term Memory Temporal Convolutional Networks Examples Sequential Data with Neural Networks Recurrent Neural

303 views • 4 slides
Neural Information Retrieval Wassila Lalouani 1 Plan Neural network architectures Neural

Neural Information Retrieval Wassila Lalouani 1 Plan Neural network architectures Neural

Neural Information Retrieval Wassila Lalouani 1 Plan Neural network architectures Neural IR tasks Neural IR architecture Feature Representations Neural IR query auto completion Neural IR query suggestion Neural IR document

1.48k views • 18 slides
CHAPTER II I CHAPTER I Recurrent Neural Networks Recurrent Neural Networks CHAPTER II : I :

CHAPTER II I CHAPTER I Recurrent Neural Networks Recurrent Neural Networks CHAPTER II : I :

Ugur HALICI - METU EEE - ANKARA 11/18/2004 CHAPTER II I CHAPTER I Recurrent Neural Networks Recurrent Neural Networks CHAPTER II : I : Recurrent Neural Networks CHAPTER I Recurrent Neural Networks Introduction In this chapter first the

404 views • 27 slides
CHAPTER II III I CHAPTER Neural Networks as Neural Networks as Associative Memory

CHAPTER II III I CHAPTER Neural Networks as Neural Networks as Associative Memory

Ugur HALICI - METU EEE - ANKARA 11/18/2004 CHAPTER II III I CHAPTER Neural Networks as Neural Networks as Associative Memory Associative Memory CHAPTER III : III : Neural Networks as Associative Memory CHAPTER Neural Networks as

513 views • 22 slides
Convolutional Neural Networks Convolutional neural networks One of the major kinds of ANNs in use

Convolutional Neural Networks Convolutional neural networks One of the major kinds of ANNs in use

Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional Neural Networks Convolutional neural networks One of the major kinds of ANNs in use UMaine

412 views • 9 slides
Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks

Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks

Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they have

852 views • 33 slides
Neural Networks and their Application to Go Neural Networks Learning Blackjack Theory Training

Neural Networks and their Application to Go Neural Networks Learning Blackjack Theory Training

Neural Networks and their Application to Go A. Bausch Neural Networks and their Application to Go Neural Networks Learning Blackjack Theory Training neural networks Problems AlphaGo Anne-Marie Bausch The Game of Go Policy Network

280 views • 24 slides
Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks

Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks

Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they have

1.17k views • 91 slides
Neural Networks Neural Net Basics Dan Klein, John DeNero UC Berkeley Slides adapted from Greg

Neural Networks Neural Net Basics Dan Klein, John DeNero UC Berkeley Slides adapted from Greg

Neural Networks Neural Net Basics Dan Klein, John DeNero UC Berkeley Slides adapted from Greg Durrett Neural Networks Neural Networks Linear classification: argmax y w > f ( x, y ) possible because Linear Neural we transformed

316 views • 4 slides
Relaxation and Hopfield Networks Neural Networks Neural Networks - Hopfield 1 Bibliography

Relaxation and Hopfield Networks Neural Networks Neural Networks - Hopfield 1 Bibliography

Relaxation and Hopfield Networks Neural Networks Neural Networks - Hopfield 1 Bibliography Hopfield, J. J., "Neural networks and physical systems with emergent collective computational abilities," Proceedings of the National Academy

367 views • 19 slides
Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they

1.63k views • 119 slides
Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova

Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova

Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova November 21, 2016 Neural Networks 2/20 Neural Networks 3/20 Neural Networks Neural computing requires a number of neurons , to be connected

813 views • 21 slides
Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they

1.57k views • 124 slides
NEURON + Python Michael Hines HBP CodeJam Workshop #7 Manchester 2016 NINDS I r n e t t

NEURON + Python Michael Hines HBP CodeJam Workshop #7 Manchester 2016 NINDS I r n e t t

NEURON + Python Michael Hines HBP CodeJam Workshop #7 Manchester 2016 NINDS I r n e t t e e r r p p r r e e t Python t e n HOC r I Neuron specific syntax Compiled I r n e t t e e r r p p r r e e t

428 views • 26 slides
Pattern Recognition Two main challenges Representation Matching Jain CSE 802, Spring

Pattern Recognition Two main challenges Representation Matching Jain CSE 802, Spring

Chapter 6: Multilayer Neural Networks (Sections 6.1-6.3) Introduction Feedforward Operation and Classification Backpropagation Algorithm Pattern Recognition Two main challenges Representation Matching Jain CSE 802, Spring

889 views • 53 slides
Lecture 10: Training Neural Networks (Part 1) Justin Johnson Lecture 1 - 1 October 7, 2019

Lecture 10: Training Neural Networks (Part 1) Justin Johnson Lecture 1 - 1 October 7, 2019

Lecture 10: Training Neural Networks (Part 1) Justin Johnson Lecture 1 - 1 October 7, 2019 Reminder: A3 Due Monday, October 14 (1 week from today!) Remember to run the validation script! Justin Johnson Lecture 10 - 2 October 7, 2019

1.45k views • 101 slides
Neural Networks Philipp Koehn 14 April 2020 Philipp Koehn Artificial Intelligence: Neural

Neural Networks Philipp Koehn 14 April 2020 Philipp Koehn Artificial Intelligence: Neural

Neural Networks Philipp Koehn 14 April 2020 Philipp Koehn Artificial Intelligence: Neural Networks 14 April 2020 Supervised Learning 1 Examples described by attribute values (Boolean, discrete, continuous, etc.) E.g., situations where

715 views • 47 slides
THE NEURAL SIMULATION TOOL NEST 1st HPAC Platform Training December 11, 2018 Jochen M. Eppler

THE NEURAL SIMULATION TOOL NEST 1st HPAC Platform Training December 11, 2018 Jochen M. Eppler

THE NEURAL SIMULATION TOOL NEST 1st HPAC Platform Training December 11, 2018 Jochen M. Eppler (j.eppler@fz-juelich.de) SimLab Neuroscience Member of the Helmholtz Association OUTLINE Introduction Neuronal simulations Technological

611 views • 47 slides
Logistic Regression & Neural Networks CMSC 723 / LING 723 / INST 725 Marine Carpuat Slides

Logistic Regression & Neural Networks CMSC 723 / LING 723 / INST 725 Marine Carpuat Slides

Logistic Regression & Neural Networks CMSC 723 / LING 723 / INST 725 Marine Carpuat Slides credit: Graham Neubig, Jacob Eisenstein Logistic Regression Perceptron & Probabilities What if we want a probability p(y|x)? The

554 views • 37 slides
Fast Homomorphic Evaluation of Deep Discretized Neural Networks Florian Bourse Michele Minelli

Fast Homomorphic Evaluation of Deep Discretized Neural Networks Florian Bourse Michele Minelli

Fast Homomorphic Evaluation of Deep Discretized Neural Networks Florian Bourse Michele Minelli Matthias Minihold Pascal Paillier ENS, CNRS, PSL Research University, INRIA (Work done while visiting CryptoExperts) CRYPTO 2018 UCSB, Santa

938 views • 83 slides
Deep Neural Network Mathematical Mysteries for High Dimensional Learning Stphane Mallat

Deep Neural Network Mathematical Mysteries for High Dimensional Learning Stphane Mallat

Deep Neural Network Mathematical Mysteries for High Dimensional Learning Stphane Mallat cole Normale Suprieure www.di.ens.fr/data High Dimensional Learning High-dimensional x = ( x (1) , ..., x ( d )) R d : Classification:

1.34k views • 87 slides

More recommend