neural networks convolutional neural networks last class
play

Neural Networks + Convolutional Neural Networks Last Class Global - PowerPoint PPT Presentation

Mar 06, 2024 •434 likes •789 views

CS4501: Introduction to Computer Vision Neural Networks + Convolutional Neural Networks Last Class Global Features The perceptron model Neural Networks multilayer perceptron model (MLP) Backpropagation Todays Class

  1. CS4501: Introduction to Computer Vision Neural Networks + Convolutional Neural Networks

  2. Last Class • Global Features • The perceptron model • Neural Networks – multilayer perceptron model (MLP) • Backpropagation

  3. Today’s Class • Neural Networks – multilayer perceptron model (MLP) • Backpropagation • Convolutional Neural Networks

  4. Perceptron Model Frank Rosenblatt (1957) - Cornell University Activation function " ; + ; + < - " < ! " = $1, if * + , " , + 1 > 0 * + = ,./ 0, otherwise " = + > " > More: https://en.wikipedia.org/wiki/Perceptron

  5. Perceptron Model Frank Rosenblatt (1957) - Cornell University !? " ; + ; + < - " < ! " = $1, if * + , " , + 1 > 0 * + = ,./ 0, otherwise " = + > " > More: https://en.wikipedia.org/wiki/Perceptron

  6. Perceptron Model Frank Rosenblatt (1957) - Cornell University Activation function " ; + ; + < - " < ! " = $1, if * + , " , + 1 > 0 * + = ,./ 0, otherwise " = + > " > More: https://en.wikipedia.org/wiki/Perceptron

  7. Activation Functions Sigmoid(x) Step(x) Tanh(x) ReLU(x) = max(0, x)

  8. Pytorch - Perceptron

  9. Two-layer Multi-layer Perceptron (MLP) ”hidden" layer & Loss / Criterion ! " ' " & ! # ' # ( ) " ( " & ! $ ' $ & ! % ' % &

  10. Forward pass / ! + = 567896:(* + ) * + = & - "+. ' + + 3 " +01 / & = " = & - #+ ! + + 3 # +01 ( " = 567896:(= + ) ! " ' " & ! # ' # ( ) " ( " & ! $ ' $ & ! % ' % >9?? = >(( " , ( ) " ) &

  11. Backward pass GradInputs *+ = * /012304(, - ) *+ *+ = ( * @ + D 7 ) *+ & ? "-7 ' - *, - *, - *! 7 *' 7 *' 7 *, - -AB *+ = ( * + D # ) *+ @ & ? #- ! - *! 7 *! 7 *< " & -AB 89 8 89 = 8: ; /012304(< - ) ) ; 8: ; 8= ! " ' " & ! # ' # ( ) " ( " & ! $ ' $ & ! % *+ = *! 7 *+ ' % 89 8 ) ; = ) ; +(( " , ( ) " ) *? #- *? #- *! 7 8= 8= & *+ = *' 7 *+ *? "-E *? "-E *' 7 GradParams

  12. Pytorch – Two-layer MLP + Regression

  13. Pytorch – Two-layer MLP + LogSoftmax # of Hidden Units

  14. Pytorch – Two-layer MLP + LogSoftmax LogSoftmax + Negative Likelihood Loss

  15. Bookmark Opportunity!

  16. SGD training code (Project 4)

  17. Convolutional (Neural) Networks

  18. Convolutional Layer

  19. Convolutional Layer

  20. Convolutional Layer Weights

  21. Convolutional Layer Weights 4

  22. Convolutional Layer Weights 1 4

  23. Convolutional Layer (with 4 filters) weights: 4x1x9x9 Output: 4x224x224 Input: 1x224x224 if zero padding, and stride = 1

  24. Convolutional Layer (with 4 filters) weights: 4x1x9x9 Output: 4x112x112 Input: 1x224x224 if zero padding, but stride = 2

  25. Convolutional Layer in Torch kW Input Output nOutputPlane x kH nInputPlane nOutputPlane (equals the number of convolutional filters for this layer) nInputPlane (e.g. 3 for RGB inputs)

  26. Convolutional Layer in Keras Convolution2D(nOutputPlane, kW, kH, input_shape = (3, 224, 224), subsample = 2, border_mode = valid) kW Input Output nOutputPlane x kH nInputPlane nOutputPlane (equals the number of convolutional filters for this layer) nInputPlane (e.g. 3 for RGB inputs)

  27. Convolutional Layer in pytorch kernel_size Input Output out_channels x kernel_size in_channels out_channels (equals the number of convolutional filters for this layer) in_channels (e.g. 3 for RGB inputs)

  28. Convolutional Network: LeNet

  29. LeNet in Pytorch

  30. SpatialMaxPooling Layer take the max in this neighborhood 8 8 8 8 8

  31. Convolutional Layers as Matrix Multiplication https://petewarden.com/2015/04/20/why-gemm-is-at-the-heart-of-deep-learning/

  32. Convolutional Layers as Matrix Multiplication https://petewarden.com/2015/04/20/why-gemm-is-at-the-heart-of-deep-learning/

  33. Convolutional Layers as Matrix Multiplication Pros? Cons? https://petewarden.com/2015/04/20/why-gemm-is-at-the-heart-of-deep-learning/

  34. Questions? 34

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks Neuron Non-linearity Softmax layer DNN training Loss function and regularization SGD and backprop Learning rate Overfitting

1.8k views • 86 slides
Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural

Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural

Transfer Learning with Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural Networks A breakthough Convolutional Neural Networks VGG-16 example Layers of Convolutional filters Bottleneck

625 views • 23 slides
Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN)

Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN)

Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN) A.k.a. CNN or ConvNet Adit Deshpande, A Beginner's Guide To Understanding Convolutional Neural Networks. Digital Images Input array: an images

1.42k views • 72 slides
Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly

Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly

Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly used for what sort of task? a) Recognizing images b) Transcribing speech c) Translating documents d) Playing games Neural network review

431 views • 13 slides
Convolutional Neural Networks 08, 10 &amp; 17 Nov, 2016 J. Ezequiel Soto S. Image Processing

Convolutional Neural Networks 08, 10 &amp; 17 Nov, 2016 J. Ezequiel Soto S. Image Processing

Convolutional Neural Networks 08, 10 &amp; 17 Nov, 2016 J. Ezequiel Soto S. Image Processing 2016 Prof. Luiz Velho Convolutional Neural Networks 1 Summary &amp; References 08/11 ImageNet Classification with Deep Convolutional Neural Networks

648 views • 26 slides
Convolutional Neural Networks (CNNs) Recurrent Neural Networks (RNNs) L1 Scalar Processor L0

Convolutional Neural Networks (CNNs) Recurrent Neural Networks (RNNs) L1 Scalar Processor L0

Convolutional Neural Networks (CNNs) Recurrent Neural Networks (RNNs) L1 Scalar Processor L0 L0 M*V Processor N N N N N N Project Brainwave Pretrained DNN Model Scalable DNN Hardware Neural Processing Unit Microservice Extract

525 views • 30 slides
MICROBOONE Taritree Wongjirad DPF 2017 Tufts/MIT Outline Convolutional neural networks

MICROBOONE Taritree Wongjirad DPF 2017 Tufts/MIT Outline Convolutional neural networks

Run 3493 Event 41075, Oct. 23rd, 2015 CONVOLUTIONAL NEURAL NETWORKS IN MICROBOONE Taritree Wongjirad DPF 2017 Tufts/MIT Outline Convolutional neural networks (CNNs) are a type of deep, feed-forward neural networks that have been

831 views • 43 slides
Convolutional Neural Networks Rachel Hu and Zhi Zhang Amazon AI d2l.ai Outline GPUs

Convolutional Neural Networks Rachel Hu and Zhi Zhang Amazon AI d2l.ai Outline GPUs

Convolutional Neural Networks Rachel Hu and Zhi Zhang Amazon AI d2l.ai Outline GPUs Convolutions Pooling, Padding and Stride Convolutional Neural Networks (LeNet) Deep ConvNets (AlexNet) Networks using Blocks (VGG)

1.91k views • 89 slides
CS4501: Introduction to Computer Vision Deeper Convolutional Neural Network Architectures Last

CS4501: Introduction to Computer Vision Deeper Convolutional Neural Network Architectures Last

CS4501: Introduction to Computer Vision Deeper Convolutional Neural Network Architectures Last Class Neural Networks multilayer perceptron model (MLP) Backpropagation Convolutional Neural Networks Todays Class More on

772 views • 41 slides
Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) CMSC 678 UMBC Recap

Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) CMSC 678 UMBC Recap

Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) CMSC 678 UMBC Recap from last time Feed-Forward Neural Network: Multilayer Perceptron + 0 ) = (

1.04k views • 93 slides
Convolutional Neural Networks (Part III) 08, 10 &amp; 17 Nov, 2016 J. Ezequiel Soto S. Image

Convolutional Neural Networks (Part III) 08, 10 &amp; 17 Nov, 2016 J. Ezequiel Soto S. Image

Convolutional Neural Networks (Part III) 08, 10 &amp; 17 Nov, 2016 J. Ezequiel Soto S. Image Processing 2016 Prof. Luiz Velho Convolutional Neural Networks 1 Summary &amp; References 08/11 ImageNet Classification with Deep Convolutional

444 views • 27 slides
Lecture 11: Neural Networks (Part 3) March 2nd, 2020 Lecturer: Steven Wu Scribe: Steven Wu 1

Lecture 11: Neural Networks (Part 3) March 2nd, 2020 Lecturer: Steven Wu Scribe: Steven Wu 1

CSCI 5525 Machine Learning Fall 2019 Lecture 11: Neural Networks (Part 3) March 2nd, 2020 Lecturer: Steven Wu Scribe: Steven Wu 1 Convolutional Neural Networks We will now study a special type of neural networks convolutional neural

662 views • 6 slides
Neural Networks Hugo Larochelle ( @hugo_larochelle ) Google Brain 2 NEURAL NETWORKS What

Neural Networks Hugo Larochelle ( @hugo_larochelle ) Google Brain 2 NEURAL NETWORKS What

Neural Networks Hugo Larochelle ( @hugo_larochelle ) Google Brain 2 NEURAL NETWORKS What well cover ... f ( x ) computer vision architectures - convolutional networks - data augmentation 1 - residual networks ... ...

1.29k views • 75 slides
MIT 9.520/6.860, Fall 2018 Class 11: Neural networks tips, tricks &amp; software Andrzej

MIT 9.520/6.860, Fall 2018 Class 11: Neural networks tips, tricks &amp; software Andrzej

MIT 9.520/6.860, Fall 2018 Class 11: Neural networks tips, tricks &amp; software Andrzej Banburski Last time - Convolutional neural networks source: github.com/vdumoulin/conv arithmetic Large-scale Datasets General Purpose GPUs AlexNet

1.46k views • 111 slides
Convolutional Neural Networks in Speech Lecture 20 CS 753 Instructor: Preethi Jyothi

Convolutional Neural Networks in Speech Lecture 20 CS 753 Instructor: Preethi Jyothi

Convolutional Neural Networks in Speech Lecture 20 CS 753 Instructor: Preethi Jyothi Convolutional Neural Networks (CNNs) Fully connected (dense) layers have no awareness of spatial information Key concept behind convolutional layers is

983 views • 41 slides
Convolutional Neural Networks Convolutional neural networks One of the major kinds of ANNs in use

Convolutional Neural Networks Convolutional neural networks One of the major kinds of ANNs in use

Convolutional Neural Networks&lt;br/&gt;&lt;br/&gt; 5/4/19, 4(03 PM Convolutional Neural Networks&lt;br/&gt;&lt;br/&gt; 5/4/19, 4(03 PM Convolutional Neural Networks Convolutional neural networks One of the major kinds of ANNs in use UMaine

412 views • 9 slides
Obfuscation and Weak Multilinear Maps Mark Zhandry Princeton

Obfuscation and Weak Multilinear Maps Mark Zhandry Princeton

Obfuscation and Weak Multilinear Maps Mark Zhandry Princeton University Joint work with Saikrishna Badrinarayanan , Sanjam Garg, Eric Miles, Pratyay Mukherjee, Amit Sahai,

762 views • 49 slides
Intelligent Low-level Signal Detection and Zero- Suppession in Raw LArTPC Waveforms through Deep

Intelligent Low-level Signal Detection and Zero- Suppession in Raw LArTPC Waveforms through Deep

Intelligent Low-level Signal Detection and Zero- Suppession in Raw LArTPC Waveforms through Deep Learning Techniques Lorenzo Uboldi Supervisor Michael Wang Final report 24 September 2019 DUNE Deep Underground Neutrino Experiment is a LArTPC

492 views • 19 slides
Probabilistic Computation Lecture 14 BPP, ZPP 1 Zoo NEXP EXP NPSPACE PSPACE 2P NP P

Probabilistic Computation Lecture 14 BPP, ZPP 1 Zoo NEXP EXP NPSPACE PSPACE 2P NP P

Probabilistic Computation Lecture 14 BPP, ZPP 1 Zoo NEXP EXP NPSPACE PSPACE 2P NP P NL L 2 Zoo NEXP EXP NPSPACE PSPACE 2P NP RP P NL L 2 Zoo NEXP EXP NPSPACE PSPACE 2P NP BPP RP P NL L 2 Zoo NEXP EXP

1.69k views • 94 slides
On the Connection Between Multiple-Unicast Network Coding and Single-Source Single-Sink Network

On the Connection Between Multiple-Unicast Network Coding and Single-Source Single-Sink Network

On the Connection Between Multiple-Unicast Network Coding and Single-Source Single-Sink Network Error Correction Jrg Kliewer NJIT Joint work with Wentao Huang and Michael Langberg Network Error Correction Problem: Adversary has control of

1.04k views • 66 slides
Public Key Algorithms hash: irreversible transformation(message) secret key: reversible

Public Key Algorithms hash: irreversible transformation(message) secret key: reversible

1 Public Key Algorithms hash: irreversible transformation(message) secret key: reversible transformation(block) encryption digital signatures authentication RSA yes yes yes El Gamal no yes no Zero-knowledge proofs no no yes

423 views • 21 slides
Efficient Padding Oracle Attacks On Cryptographic Hardware or The Million Message Attack in 15

Efficient Padding Oracle Attacks On Cryptographic Hardware or The Million Message Attack in 15

Efficient Padding Oracle Attacks On Cryptographic Hardware or The Million Message Attack in 15 000 Messages Graham Steel joint work with R. Bardou, R. Focardi, Y. Kawamoto, L. Simionato, J.-K. Tsay CRYPTO August 2012 BLUF (Bottom Line Up

602 views • 19 slides
Lectu ture 8 Recap Prof. Leal-Taix and Prof. Niessner 1 Wh What d do w we k know so so

Lectu ture 8 Recap Prof. Leal-Taix and Prof. Niessner 1 Wh What d do w we k know so so

Lectu ture 8 Recap Prof. Leal-Taix and Prof. Niessner 1 Wh What d do w we k know so so fa far? r? Width Depth Prof. Leal-Taix and Prof. Niessner 2 Wh What d do w we k know so so fa far? r? Activation Functions

917 views • 90 slides
Machine Learning Lecture 07: Convolutional Neural Networks Nevin L. Zhang lzhang@cse.ust.hk

Machine Learning Lecture 07: Convolutional Neural Networks Nevin L. Zhang lzhang@cse.ust.hk

Machine Learning Lecture 07: Convolutional Neural Networks Nevin L. Zhang lzhang@cse.ust.hk Department of Computer Science and Engineering The Hong Kong University of Science and Technology This set of notes is based on various sources on the

1.07k views • 58 slides

More recommend