neural networks prediction i e the forward pass
play

Neural Networks: Prediction (i.e. the forward pass) Machine - PowerPoint PPT Presentation

Nov 14, 2022 •289 likes •470 views

Neural Networks: Prediction (i.e. the forward pass) Machine Learning Based on slides and material from Geoffrey Hinton, Richard Socher, Dan Roth, 1 Yoav Goldberg, Shai Shalev-Shwartz and Shai Ben-David, and others Neural Networks What is a

  1. Neural Networks: Prediction (i.e. the forward pass) Machine Learning Based on slides and material from Geoffrey Hinton, Richard Socher, Dan Roth, 1 Yoav Goldberg, Shai Shalev-Shwartz and Shai Ben-David, and others

  2. Neural Networks • What is a neural network? • Predicting with a neural network • Training neural networks • Practical concerns 2

  3. This lecture • What is a neural network? • Predicting with a neural network • Training neural networks • Practical concerns 3

  4. Let us consider an example network output We will use this example network as to introduce the general principle of how to make predictions with a neural network. 4

  5. Let us consider an example network output Naming conventions for this example Inputs: x • Hidden: z • 𝑧 Output: y • 1 𝑨 ! 𝑨 " 1 𝑦 ! 𝑦 " 5

  6. Let us consider an example network output Naming conventions for this example Inputs: x • Hidden: z • 𝑧 Output: y • 1 𝑨 ! 𝑨 " Bias feature, always 1 1 𝑦 ! 𝑦 " 6

  7. Let us consider an example network output Naming conventions for this example Inputs: x • Hidden: z • 𝑧 Output: y • Sigmoid activations 1 𝑨 ! 𝑨 " Bias feature, always 1 1 𝑦 ! 𝑦 " 7

  8. Let us consider an example network output Naming conventions for this example Inputs: x • Linear activation Hidden: z • 𝑧 Output: y • Sigmoid activations 1 𝑨 ! 𝑨 " Bias feature, always 1 1 𝑦 ! 𝑦 " 8

  9. Let us consider an example network output Naming Convention for Weights -.)/0- - 1.20) 𝑧 𝑥 ()*+,-* #$% 𝑥 "! #$% 𝑥 &! #$% 𝑥 !! 1 𝑨 ! 𝑨 " ' 𝑥 &! ' 𝑥 "" 1 𝑦 ! 𝑦 " 9

  10. Let us consider an example network output Naming Convention for Weights -.)/0- - 1.20) 𝑧 𝑥 ()*+,-* #$% 𝑥 "! #$% 𝑥 &! #$% 𝑥 !! #$% 𝑥 &! 1 𝑨 ! 𝑨 " From neuron #0 ' 𝑥 &! to neuron #1 in ' 𝑥 "" output layer 1 𝑦 ! 𝑦 " 10

  11. How to predict: The forward pass Given an input x , how is the output predicted output 𝑧 #$% 𝑥 "! #$% 𝑥 &! #$% 𝑥 !! 1 𝑨 ! 𝑨 " ' 𝑥 &! ' 𝑥 "" 1 𝑦 ! 𝑦 " 11

  12. The forward pass Given an input x , how is the output predicted output 𝑧 #$% 𝑥 "! #$% 𝑥 &! # + 𝑥 !! # 𝑦 ! + 𝑥 $! # 𝑦 $ ) #$% 𝑥 !! z ! = 𝜏(𝑥 "! 1 𝑨 ! 𝑨 " ' 𝑥 &! ' 𝑥 "" 1 𝑦 ! 𝑦 " 12

  13. The forward pass Given an input x , how is the output predicted output 𝑧 # + 𝑥 !$ # 𝑦 ! + 𝑥 $$ # 𝑦 $ ) 𝑨 $ = 𝜏(𝑥 "$ #$% 𝑥 "! #$% 𝑥 &! # + 𝑥 !! # 𝑦 ! + 𝑥 $! # 𝑦 $ ) #$% 𝑥 !! z ! = 𝜏(𝑥 "! 1 𝑨 ! 𝑨 " ' 𝑥 &! ' 𝑥 "" 1 𝑦 ! 𝑦 " 13

  14. The forward pass Given an input x , how is the output predicted output % + 𝑥 !! % 𝑨 ! + 𝑥 $! % 𝑨 $ output y = 𝑥 "! 𝑧 # + 𝑥 !$ # 𝑦 ! + 𝑥 $$ # 𝑦 $ ) 𝑨 $ = 𝜏(𝑥 "$ #$% 𝑥 "! #$% 𝑥 &! # + 𝑥 !! # 𝑦 ! + 𝑥 $! # 𝑦 $ ) #$% 𝑥 !! z ! = 𝜏(𝑥 "! 1 𝑨 ! 𝑨 " ' 𝑥 &! ' 𝑥 "" 1 𝑦 ! 𝑦 " 14

  15. The forward pass Given an input x , how is the output predicted output % + 𝑥 !! % 𝑨 ! + 𝑥 $! % 𝑨 $ output y = 𝑥 "! 𝑧 # + 𝑥 !$ # 𝑦 ! + 𝑥 $$ # 𝑦 $ ) 𝑨 $ = 𝜏(𝑥 "$ #$% 𝑥 &! # + 𝑥 !! # 𝑦 ! + 𝑥 $! # 𝑦 $ ) #$% 𝑥 !! z ! = 𝜏(𝑥 "! 1 𝑨 ! 𝑨 " ' 𝑥 &! ' 𝑥 "" 1 𝑦 ! 𝑦 " 15

  16. The forward pass output Given an input x , how is the output predicted % + 𝑥 !! % 𝑨 ! + 𝑥 $! % 𝑨 $ output y = 𝑥 "! 𝑧 # + 𝑥 !$ # 𝑦 ! + 𝑥 $$ # 𝑦 $ ) 𝑨 $ = 𝜏(𝑥 "$ #$% 𝑥 "! #$% 𝑥 &! # + 𝑥 !! # 𝑦 ! + 𝑥 $! # 𝑦 $ ) #$% 𝑥 !! z ! = 𝜏(𝑥 "! 1 𝑨 ! 𝑨 " In general, before visiting (i.e. computing) the value of a node, visit all nodes that serve as ' 𝑥 &! inputs to it. ' 𝑥 "" 1 𝑦 ! 𝑦 " 16

  17. The forward pass output Given an input x , how is the output predicted % + 𝑥 !! % 𝑨 ! + 𝑥 $! % 𝑨 $ output y = 𝑥 "! 𝑧 # + 𝑥 !$ # 𝑦 ! + 𝑥 $$ # 𝑦 $ ) 𝑨 $ = 𝜏(𝑥 "$ #$% 𝑥 "! #$% 𝑥 &! # + 𝑥 !! # 𝑦 ! + 𝑥 $! # 𝑦 $ ) #$% 𝑥 !! z ! = 𝜏(𝑥 "! 1 𝑨 ! 𝑨 " In general, before visiting (i.e. computing) the value of a node, visit all nodes that serve as inputs to it. ' 𝑥 &! ' 𝑥 "" Questions? 1 𝑦 ! 𝑦 " 17

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Neural Networks for Time Series Prediction 15-486/782: Artificial Neural Networks Fall 2006

Neural Networks for Time Series Prediction 15-486/782: Artificial Neural Networks Fall 2006

Neural Networks for Time Series Prediction 15-486/782: Artificial Neural Networks Fall 2006 (based on earlier slides by Dave Touretzky and Kornel Laskowski) What is a Time Series? A sequence of vectors (or scalars) which depend on time t . In

975 views • 42 slides
Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks

Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks

Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they have

1.17k views • 91 slides
Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks

Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks

Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they have

852 views • 33 slides
CS4501: Introduction to Computer Vision Neural Networks (NNs) Artificial Neural Networks (ANNs)

CS4501: Introduction to Computer Vision Neural Networks (NNs) Artificial Neural Networks (ANNs)

CS4501: Introduction to Computer Vision Neural Networks (NNs) Artificial Neural Networks (ANNs) Multi-layer Perceptrons (MLPs Previous Neural Networks The Perceptron Model The Multi-layer Perceptron (MLP) Forward-pass in an MLP

1.54k views • 40 slides
Neural Networks for Time Series Prediction 15-496/782: Artificial Neural Networks Kornel

Neural Networks for Time Series Prediction 15-496/782: Artificial Neural Networks Kornel

Neural Networks for Time Series Prediction 15-496/782: Artificial Neural Networks Kornel Laskowski Spring 2004 (based on slides by D. Touretzky, Spring 2002) What is a Time Series? A sequence of vectors (or scalars) which depend on time t . In

719 views • 42 slides
Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they

1.63k views • 119 slides
Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they

1.57k views • 124 slides
Outline Evolution of neurocomputing Artificial neural networks Feed forward

Outline Evolution of neurocomputing Artificial neural networks Feed forward

Introduction to Neural Networks and Deep Learning Ling Guan References: S. Haykin, Neural Networks , 2 nd Edition, New Jersey: Prentice Hall, 1. 2004. C. M. Bishop, Neural Networks for Pattern Re cognition, New York: 2. Oxford University

451 views • 17 slides
Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking

Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking

Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In

987 views • 40 slides
Prediction of genetic Values using Neural Networks Paulino Perez 1 Daniel Gianola 2 Jose Crossa 1 1

Prediction of genetic Values using Neural Networks Paulino Perez 1 Daniel Gianola 2 Jose Crossa 1 1

Prediction of genetic Values using Neural Networks Paulino Perez 1 Daniel Gianola 2 Jose Crossa 1 1 CIMMyT-Mexico 2 University of Wisconsin, Madison. September, 2014 SLU,Sweden Prediction of genetic Values using Neural Networks 1/26 Contents

597 views • 26 slides
Vehicle Velocity Prediction Using Artificial Neural Networks and Effect of Real-World Signals on

Vehicle Velocity Prediction Using Artificial Neural Networks and Effect of Real-World Signals on

21-23 20 Vehicle Velocity Prediction Using Artificial Neural Networks and Effect of Real-World Signals on Prediction Window Tushar D. Gaikwad , Zachary D. Asher, Aaron Rabinowitz, Farhang Motallebiaraghi, Thomas Bradley, Alvis Fong, Rick Meyer

1.19k views • 27 slides
Vehicle Velocity Prediction Using Artificial Neural Networks and Effect of Real-World Signals on

Vehicle Velocity Prediction Using Artificial Neural Networks and Effect of Real-World Signals on

Vehicle Velocity Prediction Using Artificial Neural Networks and Effect of Real-World Signals on Prediction Window by, Tushar D. Gaikwad Committee Dr. Zachary Asher, Chair Dr. Richard Meyer Dr. Alvis Fong Western Michigan University 1

906 views • 65 slides
Closed Loop Neural-Symbolic Learning via Integrating Neural Perception, Grammar Parsing, and

Closed Loop Neural-Symbolic Learning via Integrating Neural Perception, Grammar Parsing, and

Closed Loop Neural-Symbolic Learning via Integrating Neural Perception, Grammar Parsing, and Symbolic Reasoning Motivation NS-RL Neural Symbolic Input Prediction Error Network Reasoning Ground Truth Forward pass

1.13k views • 27 slides
Neural Networks CS 6355: Structured Prediction Based on slides and material from Geoffrey Hinton,

Neural Networks CS 6355: Structured Prediction Based on slides and material from Geoffrey Hinton,

Neural Networks CS 6355: Structured Prediction Based on slides and material from Geoffrey Hinton, Richard Socher, Dan Roth, Yoav Goldberg, Shai Shalev-Shwartz and Shai Ben-David, and others This lecture What is a neural network? Training

1.28k views • 93 slides
Neural Networks Neural networks arise from attempts to model Neural Networks human/animal

Neural Networks Neural networks arise from attempts to model Neural Networks human/animal

Feed-forward Networks Network Training Error Backpropagation Deep Learning Feed-forward Networks Network Training Error Backpropagation Deep Learning Neural Networks Neural networks arise from attempts to model Neural Networks

380 views • 9 slides
Neural Networks: Computation + Gradient Descent LING572 Advanced Statistical Methods in NLP

Neural Networks: Computation + Gradient Descent LING572 Advanced Statistical Methods in NLP

Neural Networks: Computation + Gradient Descent LING572 Advanced Statistical Methods in NLP February 27 2020 1 Todays Outline Computation: the forward pass Functional form / matrix notation Parameters and Hyperparameters

832 views • 40 slides
Forward vs. backward proof Consider the following Natural Deduction rule: Rule can be read in two

Forward vs. backward proof Consider the following Natural Deduction rule: Rule can be read in two

Forward vs. backward proof Consider the following Natural Deduction rule: Rule can be read in two ways: Backwards: to prove suffjces to prove Forwards: if holds then holds With apply -style proofs we are reasoning

387 views • 10 slides
FORWARD PASS w b u v exp y x * + w b u v exp y x * + CHAIN RULE w b u v exp

FORWARD PASS w b u v exp y x * + w b u v exp y x * + CHAIN RULE w b u v exp

FORWARD PASS w b u v exp y x * + w b u v exp y x * + CHAIN RULE w b u v exp y x * + w b u v exp y x * + BACKWARD PASS w b u v exp y x * +

297 views • 5 slides
Synthesizing Loops For Program Inversion Cong Hou, Daniel Quinlan, David Jefferson, Richard

Synthesizing Loops For Program Inversion Cong Hou, Daniel Quinlan, David Jefferson, Richard

For RC 2012 Synthesizing Loops For Program Inversion Cong Hou, Daniel Quinlan, David Jefferson, Richard Fujimoto, Richard Vuduc Program Inversion Given a program P , and its inverse P - , then we have P ; P - = no-op Examples: ++a / --a

427 views • 39 slides
Topic III.1: Swap Randomization Discrete Topics in Data Mining Universitt des Saarlandes,

Topic III.1: Swap Randomization Discrete Topics in Data Mining Universitt des Saarlandes,

Topic III.1: Swap Randomization Discrete Topics in Data Mining Universitt des Saarlandes, Saarbrcken Winter Semester 2012/13 T III.1- 1 Topic III.1: Swap Randomization 1. Motivation & Basic Idea 2. Markov Chains and Sampling 2.1.

555 views • 34 slides
2/10/2020 Today: Floating Point Background: Fractional binary numbers IEEE floating point

2/10/2020 Today: Floating Point Background: Fractional binary numbers IEEE floating point

2/10/2020 Today: Floating Point Background: Fractional binary numbers IEEE floating point standard: Definition Floating Point Example and properties Rounding, addition, multiplication CSci 2021: Machine Architecture and

411 views • 7 slides
The Hardware/Software Interface CSE351 Spring2013 Floating-Point Numbers University of

The Hardware/Software Interface CSE351 Spring2013 Floating-Point Numbers University of

University of Washington The Hardware/Software Interface CSE351 Spring2013 Floating-Point Numbers University of Washington Data & addressing Roadmap Integers & floats Machine code & C C: Java: x86 assembly Car c = new Car();

734 views • 45 slides
CISC 322 Software/Game Architecture Module 7: Project Scheduling (PERT/CPM) Ahmed E. Hassan

CISC 322 Software/Game Architecture Module 7: Project Scheduling (PERT/CPM) Ahmed E. Hassan

CISC 322 Software/Game Architecture Module 7: Project Scheduling (PERT/CPM) Ahmed E. Hassan Project A project is a temporary endeavour undertaken to create a "unique" product or service A project is composed of a

739 views • 53 slides
Dynamic Memory Allocation What it is How Python does it: garbage collection How C does

Dynamic Memory Allocation What it is How Python does it: garbage collection How C does

As you arrive: Plus in-class time working on these 1. Start up your computer and plug it in concepts AND 2. Log into Angel and go to CSSE 120 practicing previous 3. Do the Attendance Widget the PIN is on the board concepts, continued

329 views • 16 slides

More recommend