convolutional neural networks
play

Convolutional Neural Networks Convolutional neural networks One of - PowerPoint PPT Presentation

Mar 18, 2024 •312 likes •412 views

Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional Neural Networks Convolutional neural networks One of the major kinds of ANNs in use UMaine

  1. Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional Neural Networks Convolutional neural networks One of the major kinds of ANNs in use UMaine COS 470/570 – Introduction to AI One of the reasons deep learning is so successful: addresses computational tractability addresses vanishing/exploding gradient problem Spring 2019 Start – 80s First truly successful modern version: LeNet (LeCun, 1989) LeNet-5 (LeCun, 1998): 7-layer CNN for reading numbers on checks Created: 2019-04-29 Mon 21:45 1 2 . 1 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 1 of 17 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 2 of 17

  2. Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Human visual system The problem Goal: High-accuracy image recognition Standard supervised learning with deep (fully-connected) networks: Images require connections from each pixel → each neuron E.g., 1028 × 768 image ⇒ about 789,504 weights per neuron Slow to train Vanishing/exploding gradient problem Also no spatial locality exploited Can we take inspiration from biological vision systems? Image credit: user Clock, CC BY-SA 3.0 , via Wikimedia Commons 3 . 1 4 . 1 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 3 of 17 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 4 of 17

  3. Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional layers Shared weights Instead of fully-connected layer, think of using a layer whose neurons each have a receptive fi eld : So—how to compute the layer? For a receptive fi eld : n × n weights n × n If input layer is , hidden layer is m × m m − n + 1 × m − n + 1 For hidden layer neuron at , activation is: x , y n − 1 n − 1 ∑ ∑ σ ( b + w i , j a x + i , y + j ) i =0 j =0 Overlapping receptive fi elds Slide the kernel across, down the image by some stride Neurons then learn local features, only have a few weight each weights = kernel or fi lter b Have multiple feature-detecting layers per convolutional layer Hidden layer = feature map Problem: Update weights based on entire hidden layer’s computed loss Features should be location-independent function ⇒ Weights for nodes should be shared, learned together 5 . 1 6 . 1 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 5 of 17 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 6 of 17

  4. Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Why convolutional layer? Pooling layers Learns local spatial features of input Convolutional layers are coupled with pooling layers Location-independent (location-invariant) Each node of pooling layer connected to some region of i × j Example (from Nielsen, M: Neural Networks and Deep Learning ): feature map Typically feature map/layer ⇒ learn di ff erent kinds of features > 1 7 . 1 8 . 1 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 7 of 17 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 8 of 17

  5. Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Pooling layers Using the features Pool based on some function—max, average, etc. Pooled layers’ output ⇒ fully-connected layer – e.g., for MNIST: (Aphex34 [ CC BY-SA 4.0 ], via Wikimedia Commons) Purpose(s): (From Nielson)) Reduce # weights needed Learn con fi guration of features Blur/average/smooth feature map Could have multiple fully-connected layers, too Determining if a feature is in a particular region 9 . 1 10 . 1 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 9 of 17 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 10 of 17

  6. Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Multiple convolutional layers Multiple convolutional + pooling layers: Learning in CNNs Backpropagation learning, gradient descent Equations for fully-connected nets have to be modi fi ed, though Theano, TensorFlow, PyTorch – all have support for training CNNs (Aphex34 [ CC BY-SA 4.0 ], via Wikimedia Commons) Deeper layers ⇒ more complex features 11 . 1 12 . 1 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 11 of 17 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 12 of 17

  7. Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Feature detection in CNNs Multiple convolutional layers LeNet-5: From ConvNet: 7 layers Recognize numbers on checks Recall the DQN we talked about used CNNs Many additional variants of CNNs now ResNet: 152 layers, general image recognition, lots of additions to LeNet’s basic architecture 13 . 1 14 . 1 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 13 of 17 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 14 of 17

  8. Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Progress in image recognition competition 15 . 1 (Aphex34 [ CC BY-SA 4.0 ], via Wikimedia Commons) file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 15 of 17 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 16 of 17

  9. Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Your turn 1. Build a CNN Get into groups, one of whom has a laptop with Keras on it Create a simple CNN for MNIST 2. Explain a CNN Get into groups with at least 2 laptops Part of group: Look up an “inception” layer in (e.g.) GoogleNet Other part: Look up ResNet Explain them to each other after a few minutes 16 . 1 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 17 of 17

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural

Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural

Transfer Learning with Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural Networks A breakthough Convolutional Neural Networks VGG-16 example Layers of Convolutional filters Bottleneck

625 views • 23 slides
Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN)

Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN)

Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN) A.k.a. CNN or ConvNet Adit Deshpande, A Beginner's Guide To Understanding Convolutional Neural Networks. Digital Images Input array: an images

1.42k views • 72 slides
Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly

Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly

Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly used for what sort of task? a) Recognizing images b) Transcribing speech c) Translating documents d) Playing games Neural network review

431 views • 13 slides
CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks Neuron Non-linearity Softmax layer DNN training Loss function and regularization SGD and backprop Learning rate Overfitting

1.8k views • 86 slides
Convolutional Neural Networks 08, 10 &amp; 17 Nov, 2016 J. Ezequiel Soto S. Image Processing

Convolutional Neural Networks 08, 10 &amp; 17 Nov, 2016 J. Ezequiel Soto S. Image Processing

Convolutional Neural Networks 08, 10 &amp; 17 Nov, 2016 J. Ezequiel Soto S. Image Processing 2016 Prof. Luiz Velho Convolutional Neural Networks 1 Summary &amp; References 08/11 ImageNet Classification with Deep Convolutional Neural Networks

648 views • 26 slides
MICROBOONE Taritree Wongjirad DPF 2017 Tufts/MIT Outline Convolutional neural networks

MICROBOONE Taritree Wongjirad DPF 2017 Tufts/MIT Outline Convolutional neural networks

Run 3493 Event 41075, Oct. 23rd, 2015 CONVOLUTIONAL NEURAL NETWORKS IN MICROBOONE Taritree Wongjirad DPF 2017 Tufts/MIT Outline Convolutional neural networks (CNNs) are a type of deep, feed-forward neural networks that have been

831 views • 43 slides
Convolutional Neural Networks (Part III) 08, 10 &amp; 17 Nov, 2016 J. Ezequiel Soto S. Image

Convolutional Neural Networks (Part III) 08, 10 &amp; 17 Nov, 2016 J. Ezequiel Soto S. Image

Convolutional Neural Networks (Part III) 08, 10 &amp; 17 Nov, 2016 J. Ezequiel Soto S. Image Processing 2016 Prof. Luiz Velho Convolutional Neural Networks 1 Summary &amp; References 08/11 ImageNet Classification with Deep Convolutional

444 views • 27 slides
Convolutional Neural Networks Rachel Hu and Zhi Zhang Amazon AI d2l.ai Outline GPUs

Convolutional Neural Networks Rachel Hu and Zhi Zhang Amazon AI d2l.ai Outline GPUs

Convolutional Neural Networks Rachel Hu and Zhi Zhang Amazon AI d2l.ai Outline GPUs Convolutions Pooling, Padding and Stride Convolutional Neural Networks (LeNet) Deep ConvNets (AlexNet) Networks using Blocks (VGG)

1.91k views • 89 slides
Convolutional Neural Networks in Speech Lecture 20 CS 753 Instructor: Preethi Jyothi

Convolutional Neural Networks in Speech Lecture 20 CS 753 Instructor: Preethi Jyothi

Convolutional Neural Networks in Speech Lecture 20 CS 753 Instructor: Preethi Jyothi Convolutional Neural Networks (CNNs) Fully connected (dense) layers have no awareness of spatial information Key concept behind convolutional layers is

983 views • 41 slides
Convolutional Neural Networks (CNNs) Recurrent Neural Networks (RNNs) L1 Scalar Processor L0

Convolutional Neural Networks (CNNs) Recurrent Neural Networks (RNNs) L1 Scalar Processor L0

Convolutional Neural Networks (CNNs) Recurrent Neural Networks (RNNs) L1 Scalar Processor L0 L0 M*V Processor N N N N N N Project Brainwave Pretrained DNN Model Scalable DNN Hardware Neural Processing Unit Microservice Extract

525 views • 30 slides
Dependency-based Convolutional Neural Networks for Sentence Embedding What is Hawaii

Dependency-based Convolutional Neural Networks for Sentence Embedding What is Hawaii

Dependency-based Convolutional Neural Networks for Sentence Embedding What is Hawaii s state flower ? ROOT Mingbo Ma Liang Huang Bing Xiang Bowen Zhou CUNY IBM T. J. Watson ACL 2015 Beijing Convolutional Neural Network

742 views • 50 slides
Semantic Segmentation of the sekleton in bone scintigraphy images with convolutional neural

Semantic Segmentation of the sekleton in bone scintigraphy images with convolutional neural

Semantic Segmentation of the sekleton in bone scintigraphy images with convolutional neural networks Problem Description Convolutional Neural Networks Convolutional Layer Max Pooling Transforming Classification networks to segmentation

157 views • 12 slides
and Inference for Convolutional Neural Networks 1 2 FFT IFFT 3 4 Mathieu et al.: Fast

and Inference for Convolutional Neural Networks 1 2 FFT IFFT 3 4 Mathieu et al.: Fast

Band-limited Training and Inference for Convolutional Neural Networks 1 2 FFT IFFT 3 4 Mathieu et al.: Fast Training of Convolutional Networks through FFTs Fast Convolutional Nets With fbfft: A GPU Performance Evaluation Data: x

648 views • 46 slides
Neural Networks + Convolutional Neural Networks Last Class Global Features The perceptron

Neural Networks + Convolutional Neural Networks Last Class Global Features The perceptron

CS4501: Introduction to Computer Vision Neural Networks + Convolutional Neural Networks Last Class Global Features The perceptron model Neural Networks multilayer perceptron model (MLP) Backpropagation Todays Class

789 views • 34 slides
Neural Network Part 3: Convolutional Neural Networks CS 760@UW-Madison Goals for the lecture

Neural Network Part 3: Convolutional Neural Networks CS 760@UW-Madison Goals for the lecture

Neural Network Part 3: Convolutional Neural Networks CS 760@UW-Madison Goals for the lecture you should understand the following concepts convolutional neural networks (CNN) convolution and its advantage pooling and its advantage 2

750 views • 50 slides
CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided

CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided

CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided Backpropagation, Deep Dream, Deep Art, Fooling Convolutional Neural Networks Mitesh M. Khapra Department of Computer Science and Engineering Indian Institute

1.07k views • 72 slides
Effective Approaches to Attention-based Neural Machine Translation Minh-Thang Luong , Hieu Pham,

Effective Approaches to Attention-based Neural Machine Translation Minh-Thang Luong , Hieu Pham,

Effective Approaches to Attention-based Neural Machine Translation Minh-Thang Luong , Hieu Pham, Christopher D. Manning Lan Li (present) Outline Abstract Introduction Related Work Models &amp; Comparison Experiment Takeaways Abstract

2.52k views • 146 slides
Simulating neural computation and information processing with Brian Marcel Stimberg Institut de

Simulating neural computation and information processing with Brian Marcel Stimberg Institut de

Simulating neural computation and information processing with Brian Marcel Stimberg Institut de la Vision/Sorbonne Universit marcel.stimberg@inserm.fr Course material Updated material will be uploaded here: g i t h u b . c o m /

394 views • 14 slides
CSCE 478/878 Lecture 4: Artificial Neural Networks Stephen D. Scott (Adapted from Tom

CSCE 478/878 Lecture 4: Artificial Neural Networks Stephen D. Scott (Adapted from Tom

CSCE 478/878 Lecture 4: Artificial Neural Networks Stephen D. Scott (Adapted from Tom Mitchells slides) September 24 , 200 8 1 Outline Threshold units: Perceptron, Winnow Gradient descent/exponentiated gradient Multilayer

438 views • 43 slides
CMP784 DEEP LEARNING Lecture #03 Multi-layer Perceptrons Aykut Erdem // Hacettepe University

CMP784 DEEP LEARNING Lecture #03 Multi-layer Perceptrons Aykut Erdem // Hacettepe University

Image: Jose-Luis Olivares CMP784 DEEP LEARNING Lecture #03 Multi-layer Perceptrons Aykut Erdem // Hacettepe University // Spring 2018 Breaking news! Practical 1 is out! Learning neural word embeddings Due Friday, Mar. 16,

1.08k views • 64 slides
Safe and Robust Deep Learning Gagandeep Singh PhD Student Department of Computer Science 1

Safe and Robust Deep Learning Gagandeep Singh PhD Student Department of Computer Science 1

Safe and Robust Deep Learning Gagandeep Singh PhD Student Department of Computer Science 1 SafeAI @ ETH Zurich (safeai.ethz.ch) Joint work with Markus Timon Matthew Mislav Petar Martin Maximilian Dana Vechev Pschel Gehr Mirman

580 views • 36 slides
SLID IDE : In In Defen ense e of Smart Algorithms over er Ha Hardware e Accel eler

SLID IDE : In In Defen ense e of Smart Algorithms over er Ha Hardware e Accel eler

SLID IDE : In In Defen ense e of Smart Algorithms over er Ha Hardware e Accel eler eration for Large-Scale e Deep eep Lea earning System ems Beidi Chen Collaborators: Tharun Medini * , James Farwell , Sameh Gobriel , Charlie

824 views • 39 slides
CS480/680 Lecture 9: June 5, 2019 Perceptrons, Neural Networks [D] Chapt. 4, [HTF] Chapt. 11,

CS480/680 Lecture 9: June 5, 2019 Perceptrons, Neural Networks [D] Chapt. 4, [HTF] Chapt. 11,

CS480/680 Lecture 9: June 5, 2019 Perceptrons, Neural Networks [D] Chapt. 4, [HTF] Chapt. 11, [B] Sec. 4.1.7, 5.1, [M] Sec. 8.5.4, [RN] Sec. 18.7 University of Waterloo CS480/680 Spring 2019 Pascal Poupart 1 Outline Neural networks

885 views • 29 slides
1/16/2014 HIV Disease and Distal Sensory Polyneuropathy (DSP) David M. Kietrys, PT, PhD, OCS

1/16/2014 HIV Disease and Distal Sensory Polyneuropathy (DSP) David M. Kietrys, PT, PhD, OCS

1/16/2014 HIV Disease and Distal Sensory Polyneuropathy (DSP) David M. Kietrys, PT, PhD, OCS Associate Professor, Rutgers, The State University of New Jersey Mary Lou Galantino, PT, PhD Professor, The Richard Stockton College of New Jersey

306 views • 3 slides

More recommend