introduction to deep learning
play

Introduction to Deep Learning A. G. Schwing & S. Fidler - PowerPoint PPT Presentation

Dec 29, 2023 •217 likes •965 views

Introduction to Deep Learning A. G. Schwing & S. Fidler University of Toronto, 2015 A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 1 / 39 Outline Universality of Neural Networks 1 Learning Neural

  1. Introduction to Deep Learning A. G. Schwing & S. Fidler University of Toronto, 2015 A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 1 / 39

  2. Outline Universality of Neural Networks 1 Learning Neural Networks 2 Deep Learning 3 Applications 4 References 5 A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 2 / 39

  3. What are neural networks? Let’s ask • Biological Input Hidden Output layer layer layer Input #1 Input #2 Output Input #3 • Computational Input #4 A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 3 / 39

  4. What are neural networks? ... Neural networks (NNs) are computational models inspired by biological neural networks [...] and are used to estimate or approximate functions ... [Wikipedia] A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 4 / 39

  5. What are neural networks? Origins: Traced back to threshold logic [W. McCulloch and W. Pitts, 1943] Perceptron [F . Rosenblatt, 1958] A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 5 / 39

  6. What are neural networks? Use cases Classification Playing video games Captcha Neural Turing Machine (e.g., learn how to sort) Alex Graves http://www.technologyreview.com/view/532156/googles-secretive-deepmind-startup-unveils-a-neural-turing-machine/ A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 6 / 39

  7. What are neural networks? Example: input x parameters w 1 , w 2 , b A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 7 / 39

  8. What are neural networks? Example: input x parameters w 1 , w 2 , b x ∈ R w 1 w 2 h 1 f b ∈ R A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 7 / 39

  9. How to compute the function? Forward propagation/pass, inference, prediction: Given input x and parameters w , b Compute (latent variables/) intermediate results in a feed-forward manner Until we obtain output function f x ∈ R w 1 w 2 h 1 f b ∈ R A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 8 / 39

  10. How to compute the function? Forward propagation/pass, inference, prediction: Given input x and parameters w , b Compute (latent variables/) intermediate results in a feed-forward manner Until we obtain output function f A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 8 / 39

  11. How to compute the function? Example: input x , parameters w 1 , w 2 , b x ∈ R w 1 w 2 h 1 f b ∈ R A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 9 / 39

  12. How to compute the function? Example: input x , parameters w 1 , w 2 , b x ∈ R w 1 w 2 h 1 f = σ ( w 1 · x + b ) h 1 f = w 2 · h 1 b ∈ R Sigmoid function: σ ( z ) = 1 / ( 1 + exp ( − z )) A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 9 / 39

  13. How to compute the function? Example: input x , parameters w 1 , w 2 , b x ∈ R w 1 w 2 h 1 f = σ ( w 1 · x + b ) h 1 f = w 2 · h 1 b ∈ R Sigmoid function: σ ( z ) = 1 / ( 1 + exp ( − z )) x = ln 2, b = ln 3, w 1 = 2, w 2 = 2 h 1 =? f =? A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 9 / 39

  14. How to compute the function? Given parameters, what is f for x = 0, x = 1, x = 2, ... f = w 2 σ ( w 1 · x + b ) A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 10 / 39

  15. How to compute the function? Given parameters, what is f for x = 0, x = 1, x = 2, ... f = w 2 σ ( w 1 · x + b ) 2 1.5 1 f 0.5 0 −5 0 5 x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 10 / 39

  16. Let’s mess with parameters: x ∈ R w 1 w 2 h 1 f h 1 = σ ( w 1 · x + b ) f = w 2 · h 1 σ ( z ) = 1 / ( 1 + exp ( − z )) b ∈ R A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 11 / 39

  17. Let’s mess with parameters: x ∈ R w 1 w 2 h 1 f h 1 = σ ( w 1 · x + b ) f = w 2 · h 1 σ ( z ) = 1 / ( 1 + exp ( − z )) b ∈ R w 1 = 1 . 0 , b changes A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 11 / 39

  18. Let’s mess with parameters: x ∈ R w 1 w 2 h 1 f h 1 = σ ( w 1 · x + b ) f = w 2 · h 1 σ ( z ) = 1 / ( 1 + exp ( − z )) b ∈ R w 1 = 1 . 0 , b changes 1 0.8 0.6 f 0.4 b = −2 0.2 b = 0 b = 2 0 −5 0 5 x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 11 / 39

  19. Let’s mess with parameters: x ∈ R w 1 w 2 h 1 f h 1 = σ ( w 1 · x + b ) f = w 2 · h 1 σ ( z ) = 1 / ( 1 + exp ( − z )) b ∈ R w 1 = 1 . 0 , b changes b = 0 , w 1 changes 1 0.8 0.6 f 0.4 b = −2 0.2 b = 0 b = 2 0 −5 0 5 x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 11 / 39

  20. Let’s mess with parameters: x ∈ R w 1 w 2 h 1 f h 1 = σ ( w 1 · x + b ) f = w 2 · h 1 σ ( z ) = 1 / ( 1 + exp ( − z )) b ∈ R w 1 = 1 . 0 , b changes b = 0 , w 1 changes 1 1 0.8 0.8 0.6 0.6 w 1 = 0 f f 0.4 0.4 w 1 = 0.5 b = −2 w 1 = 1.0 0.2 0.2 b = 0 w 1 = 100 b = 2 0 0 −5 0 5 −5 0 5 x x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 11 / 39

  21. Let’s mess with parameters: x ∈ R w 1 w 2 h 1 f h 1 = σ ( w 1 · x + b ) f = w 2 · h 1 σ ( z ) = 1 / ( 1 + exp ( − z )) b ∈ R w 1 = 1 . 0 , b changes b = 0 , w 1 changes 1 1 0.8 0.8 0.6 0.6 w 1 = 0 f f 0.4 0.4 w 1 = 0.5 b = −2 w 1 = 1.0 0.2 0.2 b = 0 w 1 = 100 b = 2 0 0 −5 0 5 −5 0 5 x x Keep in mind the step function. A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 11 / 39

  22. How to use Neural Networks for binary classification? Feature/Measurement: x Output: How likely is the input to be a cat? 1 0.8 0.6 y 0.4 0.2 0 −5 0 5 x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 12 / 39

  23. How to use Neural Networks for binary classification? Feature/Measurement: x Output: How likely is the input to be a cat? 1 0.8 0.6 f 0.4 0.2 0 −5 0 5 x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 12 / 39

  24. How to use Neural Networks for binary classification? Feature/Measurement: x Output: How likely is the input to be a cat? 1 0.8 0.6 f 0.4 0.2 0 −5 0 5 x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 12 / 39

  25. How to use Neural Networks for binary classification? Feature/Measurement: x Output: How likely is the input to be a cat? 1 0.8 0.6 f 0.4 0.2 0 −5 0 5 x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 12 / 39

  26. How to use Neural Networks for binary classification? Shifted feature/measurement: x Output: How likely is the input to be a cat? Previous features 1 0.8 0.6 f 0.4 0.2 0 −5 0 5 x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 13 / 39

  27. How to use Neural Networks for binary classification? Shifted feature/measurement: x Output: How likely is the input to be a cat? Previous features Shifted features 1 1 0.8 0.8 0.6 0.6 f f 0.4 0.4 0.2 0.2 0 0 −5 0 5 −5 0 5 x x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 13 / 39

  28. How to use Neural Networks for binary classification? Shifted feature/measurement: x Output: How likely is the input to be a cat? Previous features Shifted features 1 1 0.8 0.8 0.6 0.6 f f 0.4 0.4 0.2 0.2 0 0 −5 0 5 −5 0 5 x x Learning/Training means finding the right parameters. A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 13 / 39

  29. So far we are able to scale and translate sigmoids. How well can we approximate an arbitrary function? With the simple model we are obviously not going very far. Features are good Simple classifier 1 0.8 0.6 f 0.4 0.2 0 −5 0 5 x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 14 / 39

  30. So far we are able to scale and translate sigmoids. How well can we approximate an arbitrary function? With the simple model we are obviously not going very far. Features are good Features are noisy Simple classifier More complex classifier 1 1 0.8 0.8 0.6 0.6 f f 0.4 0.4 0.2 0.2 0 0 −5 0 5 −5 0 5 x x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 14 / 39

  31. So far we are able to scale and translate sigmoids. How well can we approximate an arbitrary function? With the simple model we are obviously not going very far. Features are good Features are noisy Simple classifier More complex classifier 1 1 0.8 0.8 0.6 0.6 f f 0.4 0.4 0.2 0.2 0 0 −5 0 5 −5 0 5 x x How can we generalize? A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 14 / 39

  32. Let’s use more hidden variables: b 1 h 1 = σ ( w 1 · x + b 1 ) h 1 h 2 = σ ( w 3 · x + b 2 ) w 1 w 2 x ∈ R f = w 2 · h 1 + w 4 · h 2 f w 3 w 4 h 2 b 2 A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 15 / 39

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Introduction to Deep Learning Outline Deep Learning RNN CNN Attention

Introduction to Deep Learning Outline Deep Learning RNN CNN Attention

Introduction to Deep Learning Outline Deep Learning RNN CNN Attention Transformer Pytorch Introduction Basics Examples RNNs Some slides borrowed from Fei-Fei Li & Justin Johnson &

1.1k views • 55 slides
Presentation about Deep Learning --- Zhongwu xie Contents 1.Brief introduction of Deep learning.

Presentation about Deep Learning --- Zhongwu xie Contents 1.Brief introduction of Deep learning.

Presentation about Deep Learning --- Zhongwu xie Contents 1.Brief introduction of Deep learning. 2.Brief introduction of Backpropagation. 3.Brief introduction of Convolutional Neural Networks. Deep learning I . Introduction to Deep Learning

609 views • 22 slides
Deep learning for NLP: Introduction CS 6956: Deep Learning for NLP Words are a very fantastical

Deep learning for NLP: Introduction CS 6956: Deep Learning for NLP Words are a very fantastical

Deep learning for NLP: Introduction CS 6956: Deep Learning for NLP Words are a very fantastical banquet, just so many strange dishes And yet, we seem to do fine We can understand and generate language effortlessly. Almost 1 Wouldnt it be

1.62k views • 52 slides
Introduction to Deep Learning Princeton University COS 495 Instructor: Yingyu Liang What is deep

Introduction to Deep Learning Princeton University COS 495 Instructor: Yingyu Liang What is deep

Introduction to Deep Learning Princeton University COS 495 Instructor: Yingyu Liang What is deep learning? Short answer: recent buzz word Industry Google Facebook Microsoft Musk Toyota Drug Finance

282 views • 26 slides
Tsing nghua hua University versity Introduction Deep learning has widely used in lots of

Tsing nghua hua University versity Introduction Deep learning has widely used in lots of

Deep ep500 500 BOF 2018 Jidong ong Zhai Tsing nghua hua University versity Introduction Deep learning has widely used in lots of areas Introduction A lot of deep learning frameworks, compute libraries and acceleration devices

371 views • 15 slides
Introduction to Deep Learning Georgia Tech CS 4650/7650 Fall 2020 Outline Deep Learning

Introduction to Deep Learning Georgia Tech CS 4650/7650 Fall 2020 Outline Deep Learning

Introduction to Deep Learning Georgia Tech CS 4650/7650 Fall 2020 Outline Deep Learning CNN RNN Attention Transformer Pytorch Introduction Basics Examples CNNs Some slides borrowed from

575 views • 56 slides
BBM413 Fundamentals of Image Processing Introduction to Deep Learning Erkut Erdem

BBM413 Fundamentals of Image Processing Introduction to Deep Learning Erkut Erdem

BBM413 Fundamentals of Image Processing Introduction to Deep Learning Erkut Erdem Hacettepe University Computer Vision Lab (HUCVL) What is deep learning? Y. LeCun, Y. Bengio, G. Hinton, "Deep Learning" , Nature,

1.12k views • 84 slides
Deep Learning on Massively Parallel Processing Databases Frank McQuillan Feb 2019 2 A Brief

Deep Learning on Massively Parallel Processing Databases Frank McQuillan Feb 2019 2 A Brief

Deep Learning on Massively Parallel Processing Databases Frank McQuillan Feb 2019 2 A Brief Introduction to Deep Learning Artificial Intelligence Landscape Deep Learning 4 Example Deep Learning Algorithms Multilayer Recurrent

651 views • 44 slides
Deep Learning (CNNs) Deep Learning Readings: Matt Gormley Murphy 28 Bishop

Deep Learning (CNNs) Deep Learning Readings: Matt Gormley Murphy 28 Bishop

10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Deep Learning (CNNs) Deep Learning Readings: Matt Gormley Murphy 28 Bishop

985 views • 86 slides
Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms

Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms

Deep 3D Representation Learning for Visual Computing Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms Conclusion 2 Outline Overview of 3D deep learning Background 3D deep learning tasks 3D deep

1.66k views • 122 slides
ACCELERATE DEEP LEARNING WITH NVIDIA'S DEEP LEARNING PLATFORM | STEPHEN JONES | GTC16 DEEP

ACCELERATE DEEP LEARNING WITH NVIDIA'S DEEP LEARNING PLATFORM | STEPHEN JONES | GTC16 DEEP

ACCELERATE DEEP LEARNING WITH NVIDIA'S DEEP LEARNING PLATFORM | STEPHEN JONES | GTC16 DEEP LEARNING EVERYWHERE INTERNET & CLOUD MEDICINE & BIOLOGY MEDIA & ENTERTAINMENT SECURITY & DEFENSE AUTONOMOUS MACHINES Image

307 views • 26 slides
CS535: Deep Learning 1. Introduction Winter 2018 Fuxin Li With materials from Pierre Baldi,

CS535: Deep Learning 1. Introduction Winter 2018 Fuxin Li With materials from Pierre Baldi,

CS535: Deep Learning 1. Introduction Winter 2018 Fuxin Li With materials from Pierre Baldi, Geoffrey Hinton, Andrew Ng, Honglak Lee, Aditya Khosla, Joseph Lim 1 Cutting Edge of Machine Learning: Deep Learning in Neural Networks Engineering

732 views • 48 slides
Advanced Machine Learning - Exercise 3 Deep learning essentials Introduction Whats the plan?

Advanced Machine Learning - Exercise 3 Deep learning essentials Introduction Whats the plan?

Advanced Machine Learning - Exercise 3 Deep learning essentials Introduction Whats the plan? Exercise overview Deep learning in a nutshell Backprop in (painful) detail 2 of 13 Introduction Exercise overview Goal: implement a simple DL

407 views • 22 slides
Deep Learning on GPUs March 2016 What is Deep Learning? GPUs and DL AGENDA DL in practice

Deep Learning on GPUs March 2016 What is Deep Learning? GPUs and DL AGENDA DL in practice

Deep Learning on GPUs March 2016 What is Deep Learning? GPUs and DL AGENDA DL in practice Scaling up DL 2 What is Deep Learning? 3 DEEP LEARNING EVERYWHERE INTERNET & CLOUD MEDICINE & BIOLOGY SECURITY & DEFENSE MEDIA &

1.45k views • 39 slides
Introduction to Deep Learning M S Ram Dept. of Computer Science & Engg. Indian Institute of

Introduction to Deep Learning M S Ram Dept. of Computer Science & Engg. Indian Institute of

Introduction to Deep Learning M S Ram Dept. of Computer Science & Engg. Indian Institute of Technology Kanpur Reading of Chap. 1 from Learning Deep Architectures for AI; Yoshua Bengio; FTML Vol. 2, No. 1 (2009) 1 127 Date: 12 Nov,

570 views • 19 slides
Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning

Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning

Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning Methods) Peter Henderson Statistical Society of Canada Annual Meeting 2018 Contributors Riashat Islam Phil Bachman Doina Precup David Meger Joelle

850 views • 37 slides
Navigating the Bumps in the Road "At each moment, you choose the intentions that will shape

Navigating the Bumps in the Road "At each moment, you choose the intentions that will shape

Navigating the Bumps in the Road "At each moment, you choose the intentions that will shape your experiences and those things, negative or positive, upon which you will focus your attention. If you choose unconsciously, you evolve

544 views • 15 slides
Noise / IR Drop Crosstalk Delay Impacts Timing Timing Failures Crosstalk affects

Noise / IR Drop Crosstalk Delay Impacts Timing Timing Failures Crosstalk affects

Noise / IR Drop Crosstalk Delay Impacts Timing Timing Failures Crosstalk affects interconnect delay of victim net Nets changing in the same direction: speed up Nets changing in opposite directions: slow down Delay and Slew

329 views • 8 slides
Extreme value theory QUAN TITATIVE RIS K MAN AGEMEN T IN P YTH ON Jamsheed Shorish

Extreme value theory QUAN TITATIVE RIS K MAN AGEMEN T IN P YTH ON Jamsheed Shorish

Extreme value theory QUAN TITATIVE RIS K MAN AGEMEN T IN P YTH ON Jamsheed Shorish Computational Economist Extreme values Portfolio losses: extreme values Extreme values: from tail of distribution T ail losses: losses exceeding some value

669 views • 49 slides
TEXTURE MAPPING SAUMITRA BAGCHI DEFINITION Texture: T he feel, appearance, or consistency of a

TEXTURE MAPPING SAUMITRA BAGCHI DEFINITION Texture: T he feel, appearance, or consistency of a

TEXTURE MAPPING SAUMITRA BAGCHI DEFINITION Texture: T he feel, appearance, or consistency of a surface or a substance. DEFINITION Texture mapping is a method for adding detail, surface texture (a bitmap or raster image), or color to a

500 views • 35 slides
Mobile Edge Artificial Intelligence: Opportunities and Challenges Motivations Yuanming Shi

Mobile Edge Artificial Intelligence: Opportunities and Challenges Motivations Yuanming Shi

Mobile Edge Artificial Intelligence: Opportunities and Challenges Motivations Yuanming Shi ShanghaiTech University 1 Why 6G? Fig. credit: Walid 2 What will 6G be? 6G networks: from connected things to connected intelligence

1.74k views • 151 slides
Radio-to-Gamma Ray Monitoring of Mkn 421 and Mkn 501: Source Variability N. Nowak, D. Paneque, U.

Radio-to-Gamma Ray Monitoring of Mkn 421 and Mkn 501: Source Variability N. Nowak, D. Paneque, U.

Radio-to-Gamma Ray Monitoring of Mkn 421 and Mkn 501: Source Variability N. Nowak, D. Paneque, U. Barres de Almeida, N. Strah, D. Tescaro On behalf of the Fermi-LAT, MAGIC, VERITAS and other collaborations and groups involved in the

488 views • 32 slides
Christopher R. H. Hanusa Queens College ? + Bathsheba Sculpture ? + Bathsheba Sculpture

Christopher R. H. Hanusa Queens College ? + Bathsheba Sculpture ? + Bathsheba Sculpture

Christopher R. H. Hanusa Queens College ? + Bathsheba Sculpture ? + Bathsheba Sculpture Henry Segerman ? + Bathsheba Sculpture Henry Segerman Repetition Symmetry Geometry Higher Dimensions Inspiration Nature

1.21k views • 85 slides
3D TECHNOLOGY FOR IMAGING SENSOR AT CEA-LETI Gabriel Pars April 2015 | LAL presentation LETI

3D TECHNOLOGY FOR IMAGING SENSOR AT CEA-LETI Gabriel Pars April 2015 | LAL presentation LETI

3D TECHNOLOGY FOR IMAGING SENSOR AT CEA-LETI Gabriel Pars April 2015 | LAL presentation LETI CONFIDENTIAL G. Pars / CEA-Leti AGENDA Leti brief overview Post-processing Leti 3D technology modules state of the art &

506 views • 46 slides

More recommend