neural networks deep learning
play

Neural Networks - Deep Learning Artificial Intelligence @ Allegheny - PowerPoint PPT Presentation

Sep 24, 2023 •213 likes •709 views

Neural Networks - Deep Learning Artificial Intelligence @ Allegheny College Janyl Jumadinova March 4-6, 2020 Credit: Google Workshop Janyl Jumadinova March 4-6, 2020 1 / 35 Neural Networks - Deep Learning Neural Networks Janyl Jumadinova

  1. Neural Networks - Deep Learning Artificial Intelligence @ Allegheny College Janyl Jumadinova March 4-6, 2020 Credit: Google Workshop Janyl Jumadinova March 4-6, 2020 1 / 35 Neural Networks - Deep Learning

  2. Neural Networks Janyl Jumadinova March 4-6, 2020 2 / 35 Neural Networks - Deep Learning

  3. Neural Networks Janyl Jumadinova March 4-6, 2020 3 / 35 Neural Networks - Deep Learning

  4. Neural Networks Neural computing requires a number of neurons , to be connected together into a neural network . Neurons are arranged in layers. Janyl Jumadinova March 4-6, 2020 4 / 35 Neural Networks - Deep Learning

  5. Neural Networks Neural computing requires a number of neurons , to be connected together into a neural network . Neurons are arranged in layers. Two main hyperparameters that control the architecture or topology of the network: 1) the number of layers, and 2) the number of nodes in each hidden layer. Janyl Jumadinova March 4-6, 2020 4 / 35 Neural Networks - Deep Learning

  6. Activation Functions The activation function is generally non-linear. Linear functions are limited because the output is simply proportional to the input. Janyl Jumadinova March 4-6, 2020 5 / 35 Neural Networks - Deep Learning

  7. Activation Functions Janyl Jumadinova March 4-6, 2020 6 / 35 Neural Networks - Deep Learning

  8. Network structures Two phases in each iteration: 1 Calculating the predicted output y , known as feed-forward 2 Updating the weights and biases, known as backpropagation Janyl Jumadinova March 4-6, 2020 7 / 35 Neural Networks - Deep Learning

  9. Feed-forward example Feed-forward networks: Single-layer perceptrons Multi-layer perceptrons Janyl Jumadinova March 4-6, 2020 8 / 35 Neural Networks - Deep Learning

  10. Feed-forward example Feed-forward networks: Single-layer perceptrons Multi-layer perceptrons Janyl Jumadinova March 4-6, 2020 8 / 35 Neural Networks - Deep Learning

  11. Single-layer Perceptrons Output units all operate separately – no shared weights. Adjusting weights moves the location, orientation, and steepness of cliff. Janyl Jumadinova March 4-6, 2020 9 / 35 Neural Networks - Deep Learning

  12. Multi-layer Perceptrons Layers are usually fully connected. Numbers of hidden units typically chosen by hand. Janyl Jumadinova March 4-6, 2020 10 / 35 Neural Networks - Deep Learning

  13. Neural Networks Janyl Jumadinova March 4-6, 2020 11 / 35 Neural Networks - Deep Learning

  14. Neural Networks Janyl Jumadinova March 4-6, 2020 12 / 35 Neural Networks - Deep Learning

  15. Neural Networks A fully connected NN layer Janyl Jumadinova March 4-6, 2020 13 / 35 Neural Networks - Deep Learning

  16. Implementation as Matrix Multiplication Janyl Jumadinova March 4-6, 2020 14 / 35 Neural Networks - Deep Learning

  17. Non-Linear Data Distributions Janyl Jumadinova March 4-6, 2020 15 / 35 Neural Networks - Deep Learning

  18. Janyl Jumadinova March 4-6, 2020 16 / 35 Neural Networks - Deep Learning

  19. Deep Learning Most current machine learning works well because of human-designed representations and input features. Machine learning becomes just optimizing weights to best make a final prediction. Janyl Jumadinova March 4-6, 2020 17 / 35 Neural Networks - Deep Learning

  20. Deep Learning Most current machine learning works well because of human-designed representations and input features. Machine learning becomes just optimizing weights to best make a final prediction. Deep learning algorithms attempt to learn multiple levels of representation of increasing complexity/abstraction. Janyl Jumadinova March 4-6, 2020 17 / 35 Neural Networks - Deep Learning

  21. Deep Learning Each neuron implements a relatively simple mathematical function. y = g ( w · x + b ) Janyl Jumadinova March 4-6, 2020 18 / 35 Neural Networks - Deep Learning

  22. Deep Learning Each neuron implements a relatively simple mathematical function. y = g ( w · x + b ) The composition of 10 6 − 10 9 such functions is powerful. Janyl Jumadinova March 4-6, 2020 18 / 35 Neural Networks - Deep Learning

  23. Deep Learning Book: http://www.deeplearningbook.org/ Chapter 5 “A core idea in deep learning is that we assume that the data was generated by the composition of factors or features, potentially at multiple levels in a hierarchy.” Janyl Jumadinova March 4-6, 2020 19 / 35 Neural Networks - Deep Learning

  24. Results get better (to a degree) with: more data bigger models more computation Janyl Jumadinova March 4-6, 2020 20 / 35 Neural Networks - Deep Learning

  25. Results get better (to a degree) with: more data bigger models more computation Better algorithms, new insights and improved methods help, too! Janyl Jumadinova March 4-6, 2020 20 / 35 Neural Networks - Deep Learning

  26. TensorFlow Janyl Jumadinova March 4-6, 2020 21 / 35 Neural Networks - Deep Learning

  27. Adoption of Deep Learning Tools on GitHub Janyl Jumadinova March 4-6, 2020 22 / 35 Neural Networks - Deep Learning

  28. TensorFlow Epoch : a training iteration (one pass through the dataset). Batch : Portion of the dataset (number of samples after dataset has been divided). Regularization : a set of techniques that helps learning models to converge ( http://www.godeep.ml/regularization-using-tensorflow/ ). Janyl Jumadinova March 4-6, 2020 23 / 35 Neural Networks - Deep Learning

  29. TensorFlow Operates over tensors : n-dimensional arrays Janyl Jumadinova March 4-6, 2020 24 / 35 Neural Networks - Deep Learning

  30. TensorFlow Operates over tensors : n-dimensional arrays Using a flow graph : data flow computation framework Janyl Jumadinova March 4-6, 2020 24 / 35 Neural Networks - Deep Learning

  31. TensorFlow Operates over tensors : n-dimensional arrays Using a flow graph : data flow computation framework Janyl Jumadinova March 4-6, 2020 24 / 35 Neural Networks - Deep Learning

  32. TensorFlow 5.7 ← Scalar Number, Float, etc. Janyl Jumadinova March 4-6, 2020 25 / 35 Neural Networks - Deep Learning

  33. TensorFlow Janyl Jumadinova March 4-6, 2020 26 / 35 Neural Networks - Deep Learning

  34. TensorFlow Janyl Jumadinova March 4-6, 2020 27 / 35 Neural Networks - Deep Learning

  35. TensorFlow Tensors have a Shape that is described with a vector Janyl Jumadinova March 4-6, 2020 28 / 35 Neural Networks - Deep Learning

  36. TensorFlow Tensors have a Shape that is described with a vector [1000 , 256 , 256 , 3] 10000 Images Each Image has 256 Rows Each Row has 256 Pixels Each Pixel has 3 values (RGB) Janyl Jumadinova March 4-6, 2020 28 / 35 Neural Networks - Deep Learning

  37. TensorFlow Computation is a dataflow graph Janyl Jumadinova March 4-6, 2020 29 / 35 Neural Networks - Deep Learning

  38. TensorFlow Computation is a dataflow graph with tensors Janyl Jumadinova March 4-6, 2020 30 / 35 Neural Networks - Deep Learning

  39. TensorFlow Computation is a dataflow graph with state Janyl Jumadinova March 4-6, 2020 31 / 35 Neural Networks - Deep Learning

  40. Core TensorFlow data structures and concepts Graph : A TensorFlow computation, represented as a dataflow graph: - collection of ops that may be executed together as a group. Janyl Jumadinova March 4-6, 2020 32 / 35 Neural Networks - Deep Learning

  41. Core TensorFlow data structures and concepts Graph : A TensorFlow computation, represented as a dataflow graph: - collection of ops that may be executed together as a group. Operation : a graph node that performs computation on tensors Janyl Jumadinova March 4-6, 2020 32 / 35 Neural Networks - Deep Learning

  42. Core TensorFlow data structures and concepts Graph : A TensorFlow computation, represented as a dataflow graph: - collection of ops that may be executed together as a group. Operation : a graph node that performs computation on tensors Tensor : a handle to one of the outputs of an Operation: - provides a means of computing the value in a TensorFlow Session. Janyl Jumadinova March 4-6, 2020 32 / 35 Neural Networks - Deep Learning

  43. TensorFlow Constants Janyl Jumadinova March 4-6, 2020 33 / 35 Neural Networks - Deep Learning

  44. TensorFlow Constants Placeholders : must be fed with data on execution. Janyl Jumadinova March 4-6, 2020 33 / 35 Neural Networks - Deep Learning

  45. TensorFlow Constants Placeholders : must be fed with data on execution. Variables : a modifiable tensor that lives in TensorFlow’s graph of interacting operations. Janyl Jumadinova March 4-6, 2020 33 / 35 Neural Networks - Deep Learning

  46. TensorFlow Constants Placeholders : must be fed with data on execution. Variables : a modifiable tensor that lives in TensorFlow’s graph of interacting operations. Session : encapsulates the environment in which Operation objects are executed, and Tensor objects are evaluated. Janyl Jumadinova March 4-6, 2020 33 / 35 Neural Networks - Deep Learning

  47. TensorFlow Janyl Jumadinova March 4-6, 2020 34 / 35 Neural Networks - Deep Learning

  48. TensorFlow https://playground.tensorflow.org Janyl Jumadinova March 4-6, 2020 35 / 35 Neural Networks - Deep Learning

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan Zelener Machine Learning Reading Group January 7 th 2016 The Graduate Center, CUNY Objectives Explain some of the trends of deep learning and

680 views • 44 slides
Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and Courville [chapt. 6,7,8]; AIMA [sect. 21.1-21.3]; Sutton and Barto, Reinforcement Learning: an

527 views • 35 slides
Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova

Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova

Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova November 21, 2016 Neural Networks 2/20 Neural Networks 3/20 Neural Networks Neural computing requires a number of neurons , to be connected

813 views • 21 slides
CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided

CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided

CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided Backpropagation, Deep Dream, Deep Art, Fooling Convolutional Neural Networks Mitesh M. Khapra Department of Computer Science and Engineering Indian Institute

1.07k views • 72 slides
Outline Evolution of neurocomputing Artificial neural networks Feed forward

Outline Evolution of neurocomputing Artificial neural networks Feed forward

Introduction to Neural Networks and Deep Learning Ling Guan References: S. Haykin, Neural Networks , 2 nd Edition, New Jersey: Prentice Hall, 1. 2004. C. M. Bishop, Neural Networks for Pattern Re cognition, New York: 2. Oxford University

451 views • 17 slides
Layer-wise Relevance Propagation in Neural Neural Networks Deep Learning Shortcomings Networks

Layer-wise Relevance Propagation in Neural Neural Networks Deep Learning Shortcomings Networks

LRP Ariyan Zarei Motivation Having More interpretable Layer-wise Relevance Propagation in Neural Neural Networks Deep Learning Shortcomings Networks to have more interpretable Papers and Demo Introduction Machine Learning models

373 views • 35 slides
Recurrent Neural Networks Deep neural networks have enabled major advances in machine learning

Recurrent Neural Networks Deep neural networks have enabled major advances in machine learning

Recurrent Neural Networks Deep neural networks have enabled major advances in machine learning and AI y t-1 y t y t+1 Computer vision h t-1 h t h t+1 Language translation Speech recognition h t-1 h t h t+1 Question answering x t-1 x t x t+1

454 views • 43 slides
PATTERN RECOGNITION AND MACHINE LEARNING Slide Set 5: Neural Networks and Deep Learning November

PATTERN RECOGNITION AND MACHINE LEARNING Slide Set 5: Neural Networks and Deep Learning November

PATTERN RECOGNITION AND MACHINE LEARNING Slide Set 5: Neural Networks and Deep Learning November 2019 Heikki Huttunen heikki.huttunen@tuni.fi Signal Processing Tampere University default Traditional Neural Networks Neural networks have

571 views • 21 slides
6. Convolutional Neural Networks CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from

6. Convolutional Neural Networks CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from

6. Convolutional Neural Networks CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from Zsolt Kira Quiz coming up Next Monday (2/5) 30 minutes Topics: Optimization Basic neural networks Neural Network

619 views • 41 slides
Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations

Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations

Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations deeplearning.cce2020@gmail.com Deep Networks Intuition Neural networks with multiple hidden layers - Deep networks [Hinton, 2006] Deep Networks Intuition

1.33k views • 28 slides
Neural Networks: What can a network represent Deep Learning, Fall 2020 1 Recap : Neural

Neural Networks: What can a network represent Deep Learning, Fall 2020 1 Recap : Neural

Neural Networks: What can a network represent Deep Learning, Fall 2020 1 Recap : Neural networks have taken over AI Tasks that are made possible by NNs, aka deep learning Tasks that were once assumed to be purely in the human domain

1.92k views • 150 slides
Deep Neural Networks CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu Deep learning slides credit:

Deep Neural Networks CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu Deep learning slides credit:

Deep Neural Networks CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu Deep learning slides credit: Vlad Morariu Training (Deep) Neural Networks Computational graphs Improvements to gradient descent Stochastic gradient descent Momentum

814 views • 21 slides
(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning

(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning

(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning objectives What are neural networks? What are deep neural networks? How do we train neural networks? What variants of neural network

331 views • 31 slides
Neural Networks: What can a network represent Deep Learning, Spring 2018 1 Recap : Neural

Neural Networks: What can a network represent Deep Learning, Spring 2018 1 Recap : Neural

Neural Networks: What can a network represent Deep Learning, Spring 2018 1 Recap : Neural networks have taken over AI Tasks that are made possible by NNs, aka deep learning 2 Recap : NNets and the brain In their basic form, NNets

2.14k views • 145 slides
Deep Neural Networks and Molecular Dynamics Roberto Car Princeton University MAX Conference

Deep Neural Networks and Molecular Dynamics Roberto Car Princeton University MAX Conference

Deep Neural Networks and Molecular Dynamics Roberto Car Princeton University MAX Conference ICTP, Trieste, Jan 29-31 2018 Deep Neural Networks When properly trained, deep NN are capable of learning the complex functional dependence of

442 views • 17 slides
Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Learning in Neural Networks w 1,3 w 3,5 1 3 5 w w 1,4 3,6 w w 2,3 4,5 2 4 6 w w 2,4 4,6 Class #18: Back-Propagation; } A neural network can learn a classification function by Tuning Hyper-Parameters adjusting its weights to

259 views • 4 slides
Modelization of membrane potentials and information transmission in large systems of neurons

Modelization of membrane potentials and information transmission in large systems of neurons

introduction membrane potential as a (jump) diffusion process Poisson spike trains information transmission in large systems of neurons statistical inference, Modelization of membrane potentials and information transmission in large systems of

507 views • 28 slides
SnappyData: Apache Spark Meets Embedded In-Memory Database Masaki Yamakawa UL Systems, Inc.

SnappyData: Apache Spark Meets Embedded In-Memory Database Masaki Yamakawa UL Systems, Inc.

SnappyData: Apache Spark Meets Embedded In-Memory Database Masaki Yamakawa UL Systems, Inc. About me Masaki Yamakawa UL Systems, Inc. Managing Consultant { Sector Financial Skills [ Distributed system ,

674 views • 51 slides
The CARES Act & Return to Work Overview & Major Provisions for Individuals $2 Trillion

The CARES Act & Return to Work Overview & Major Provisions for Individuals $2 Trillion

The CARES Act & Return to Work Overview & Major Provisions for Individuals $2 Trillion Package Through the Federal Reserve potential for additional $4 trillion in support for the economy Referred to as the Phase 3 Bill

447 views • 8 slides
Public Believes Ebola is Likely to Spread from Symptomatic Patients by Multiple Routes % saying

Public Believes Ebola is Likely to Spread from Symptomatic Patients by Multiple Routes % saying

Public Believes Ebola is Likely to Spread from Symptomatic Patients by Multiple Routes % saying Ebola is likely to spread from a symptomatic person to other people (81%) 49% 32% % saying how likely a person is to get Ebola if they Come into

71 views • 3 slides
Board AND COMPUTER, PLEASE LET ADAM KNOW WHAT PHONE NUMBER YOU USED TO DIAL IN SO YOU CAN BE

Board AND COMPUTER, PLEASE LET ADAM KNOW WHAT PHONE NUMBER YOU USED TO DIAL IN SO YOU CAN BE

ENTER YOUR NAME WHEN YOU ENTER THE CONFERENCE ROOM. ALL VIRTUAL ATTENDEES ARE MUTED BY THE MEETING HOST FROM THE BEGINNING. IF YOU ARE A BOARD MEMBER JOINING BY BOTH PHONE Board AND COMPUTER, PLEASE LET ADAM KNOW WHAT PHONE NUMBER

1.11k views • 85 slides
Connectomics! Neurons in the brain illustration Image credit: Benedict Campbell. Welcome

Connectomics! Neurons in the brain illustration Image credit: Benedict Campbell. Welcome

Connectomics! Neurons in the brain illustration Image credit: Benedict Campbell. Welcome Images Neural Structure Reconstruction from Fluorescence Imaging of Neural Activity Karan Singh Pankaj Gupta Objectives Communicate research

887 views • 25 slides
INF5470 Fall 2012 Lecture 10: Analog Storage Content Overview Volatile Short Term Storage

INF5470 Fall 2012 Lecture 10: Analog Storage Content Overview Volatile Short Term Storage

INF5470 Fall 2012 Lecture 10: Analog Storage Content Overview Volatile Short Term Storage Indirect Multi-Level (AD/DA Digital) Storage Direct Multi-Level Storage Non-Volatile Storage Weekly Questions Lecture 10: Analog Storage 2

326 views • 18 slides
INTRODUCTION TO RIVANNA 20 March 2019 Rivanna in More Detail Compute Nodes Head Nodes ssh

INTRODUCTION TO RIVANNA 20 March 2019 Rivanna in More Detail Compute Nodes Head Nodes ssh

INTRODUCTION TO RIVANNA 20 March 2019 Rivanna in More Detail Compute Nodes Head Nodes ssh client Ethernet Infiniband Home Other Scratch Directory Storage (Lustre) Allocations Rivanna is allocated: At the most basic level, an

1.17k views • 104 slides

More recommend