introduction
play

Introduction Lecture slides for Chapter 1 of Deep Learning - PowerPoint PPT Presentation

May 02, 2023 •302 likes •444 views

Introduction Lecture slides for Chapter 1 of Deep Learning www.deeplearningbook.org Ian Goodfellow 2016-09-26 x y Cartesian coordinates Polar coordinates Representations Matter r Figure 1.1 (Goodfellow 2016) Depth: Repeated

  1. Introduction Lecture slides for Chapter 1 of Deep Learning www.deeplearningbook.org Ian Goodfellow 2016-09-26

  2. x y Cartesian coordinates Polar coordinates Representations Matter θ r Figure 1.1 (Goodfellow 2016)

  3. Depth: Repeated Composition Output CAR PERSON ANIMAL (object identity) 3rd hidden layer (object parts) 2nd hidden layer (corners and contours) 1st hidden layer (edges) Visible layer (input pixels) Figure 1.2 (Goodfellow 2016)

  4. Computational Graphs Element Element σ Set Set + + × Logistic Logistic × × Regression Regression σ w x x x 1 x 1 w 2 w 2 x 2 x 2 w 1 w 1 Figure 1.3 (Goodfellow 2016)

  5. Machine Learning and AI Deep learning Example: Shallow Example: Example: Example: autoencoders Logistic Knowledge MLPs regression bases Representation learning Machine learning AI Figure 1.4 (Goodfellow 2016)

  6. Learning Multiple Components Figure 1.5 Output Mapping from Output Output features Additional Mapping from Mapping from layers of more Output features features abstract features Hand- Hand- Simple designed designed Features features program features Input Input Input Input Deep Classic learning Rule-based machine systems Representation learning (Goodfellow 2016) learning

  7. Organization of the Book Figure 1.6 1. Introduction Part I: Applied Math and Machine Learning Basics 3. Probability and 2. Linear Algebra Information Theory 4. Numerical 5. Machine Learning Computation Basics Part II: Deep Networks: Modern Practices 6. Deep Feedforward Networks 7. Regularization 8. Optimization 9. CNNs 10. RNNs 11. Practical 12. Applications Methodology Part III: Deep Learning Research 13. Linear Factor 15. Representation 14. Autoencoders Models Learning 16. Structured 17. Monte Carlo Probabilistic Models Methods 18. Partition 19. Inference Function 20. Deep Generative Models (Goodfellow 2016)

  8. Historical Waves 0.000250 Frequency of Word or Phrase cybernetics 0.000200 (connectionism + neural networks) 0.000150 0.000100 0.000050 0.000000 1940 1950 1960 1970 1980 1990 2000 Year Figure 1.7 (Goodfellow 2016)

  9. Historical Trends: Growing Datasets 10 9 Dataset size (number examples) Canadian Hansard 10 8 Sports-1M WMT 10 7 ImageNet10k 10 6 Public SVHN Criminals 10 5 ImageNet ILSVRC 2014 10 4 MNIST CIFAR-10 10 3 10 2 T vs. G vs. F Rotated T vs. C Iris 10 1 10 0 1900 1950 1985 2000 2015 Figure 1.8 (Goodfellow 2016)

  10. The MNIST Dataset Figure 1.9 (Goodfellow 2016)

  11. Connections per Neuron 10 4 Human Cat 6 Connections per neuron 7 9 4 Mouse 10 3 2 10 5 8 10 2 Fruit fly 3 1 10 1 1950 1985 2000 2015 Figure 1.10 (Goodfellow 2016)

  12. Number of Neurons Number of neurons (logarithmic scale) 10 11 Human 10 10 17 20 10 9 19 16 Octopus 18 10 8 14 10 7 Frog 11 8 10 6 Bee 3 10 5 Ant 10 4 10 3 Leech 13 10 2 2 12 10 1 1 Roundworm 15 6 9 10 0 10 5 10 − 1 7 4 10 − 2 Sponge 1950 1985 2000 2015 2056 Figure 1.11 (Goodfellow 2016)

  13. Solving Object Recognition 0 . 30 ILSVRC classification error rate 0 . 25 0 . 20 0 . 15 0 . 10 0 . 05 0 . 00 2010 2011 2012 2013 2014 2015 Figure 1.12 (Goodfellow 2016)

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION

INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION

TRAINING COURSE ON SOCIAL PROTECTION & FORMALIZATION TRINIDAD AND TOBAGO MARCH 15, 2017 INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION Design of the NIS Assistance from the

547 views • 23 slides
INTRODUCTION Introduction 2/42 INTRODUCTION Alternations I am giving bread to the

INTRODUCTION Introduction 2/42 INTRODUCTION Alternations I am giving bread to the

Patterns of variation in the expression of case and agreement Andrs Brny Leiden University Centre for Linguistics 9 November 2019, BaSIS Workshop a.barany@hum.leidenuniv.nl INTRODUCTION Introduction 2/42 INTRODUCTION

1.37k views • 42 slides
Introduction Introduction Introduction Introduction Outline Motivation Failures

Introduction Introduction Introduction Introduction Outline Motivation Failures

Introduction Introduction Introduction Introduction Outline Motivation Failures Definition and concepts Process and product properties Principles SE Approaches Motivation Software and the economy The economies of

1.03k views • 49 slides
Introduction Introduction Introduction Nationwide Cause for Concern 1

Introduction Introduction Introduction Nationwide Cause for Concern 1

Introduction Introduction Introduction Nationwide Cause for Concern 1 https://www.cdc.gov/std/stats17/infographic.htm Introduction Epidemiology in Western Colorado CPHE Surveillance Data Report, May 2018 Chlamydia Epidemiology in Western

977 views • 39 slides
DAQ introduction Purpose of this talk : (1) Introduction for those who have not been in every

DAQ introduction Purpose of this talk : (1) Introduction for those who have not been in every

DAQ introduction Purpose of this talk : (1) Introduction for those who have not been in every meeting (2) Some ideas of diagrams for section 7.1 (Introduction) of TP. Giles Barr 25 Jan 2018, Pembroke College On DUNE, the reality is there is a

474 views • 10 slides
INTRODUCTION cf. Schneider, Chapter 1 INTRODUCTION THEY ARE OUT TO GET YOU INTRODUCTION WHAT

INTRODUCTION cf. Schneider, Chapter 1 INTRODUCTION THEY ARE OUT TO GET YOU INTRODUCTION WHAT

ADVANCED COMPUTER SECURITY INTRODUCTION cf. Schneider, Chapter 1 INTRODUCTION THEY ARE OUT TO GET YOU INTRODUCTION WHAT IS SECURITY? A systems security policies describe What the system is supposed to do Store and provide access

479 views • 16 slides
Overview Introduction to SMIL Introduction to W3C and XML Introduction to SMIL

Overview Introduction to SMIL Introduction to W3C and XML Introduction to SMIL

Overview Introduction to SMIL Introduction to W3C and XML Introduction to SMIL Writing a SMIL file Adding Media Objects Martin Halvey Clipping media files January 2008 Timing and Synchronising Layout World Wide

387 views • 14 slides
14 Introduction Introduction Bad guys can put malware into Example: hosts

14 Introduction Introduction Bad guys can put malware into Example: hosts

Introduction Introduction source Encapsulation ISO/OSI reference model message M application segment H t H t M transport network datagram H n H n H t M presentation: allow applications to frame link H l H n H t M interpret

583 views • 4 slides
Previous Lecture Introduction to SMIL 2 Introduction to W3C and XML Introduction to SMIL

Previous Lecture Introduction to SMIL 2 Introduction to W3C and XML Introduction to SMIL

Previous Lecture Introduction to SMIL 2 Introduction to W3C and XML Introduction to SMIL Writing a SMIL file Adding Media Objects Martin Halvey Clipping media files January 2008 Timing and Synchronising Layout

322 views • 8 slides
Lecture 1 Andreas Habegger Introduction Zynq Introduction Zynq Introduction Zynq PS vs. PL

Lecture 1 Andreas Habegger Introduction Zynq Introduction Zynq Introduction Zynq PS vs. PL

Introduction Zynq Lecture 1 Andreas Habegger Introduction Zynq Introduction Zynq Introduction Zynq PS vs. PL Processing System Processor Peripherals Data Buses AXI Bus Conclusion BTE5380 - Embedded Systems Octobre 2014 Andreas Habegger

1.81k views • 30 slides
2018.06 01 SMILE5 Introduction S E 5 02 Alpha Cloud M I L 03 Company Introduction 04

2018.06 01 SMILE5 Introduction S E 5 02 Alpha Cloud M I L 03 Company Introduction 04

2018.06 01 SMILE5 Introduction S E 5 02 Alpha Cloud M I L 03 Company Introduction 04 Project References S E 5 PART.1 SMILE5 Introduction M I L Introduction STORION-SMILE5 Residential Energy Storage System Inverter 5kW Output Input

1.06k views • 25 slides
Introduction to CICS Course introduction Course introduction What is CICS? What is an

Introduction to CICS Course introduction Course introduction What is CICS? What is an

Introduction to CICS Course introduction Course introduction What is CICS? What is an application server? Why use an application server? Course introduction Services provided by CICS How CICS applications are defined Scaling CICS to meet

1.84k views • 146 slides
Network Visualization Introduction Presented by Shahed Introduction Introduction Basic

Network Visualization Introduction Presented by Shahed Introduction Introduction Basic

Network Visualization Introduction Presented by Shahed Introduction Introduction Basic building blocks Node Links (relationship between nodes) Spatial information Network data

584 views • 9 slides
Introduction Introduction (1) Main argument of the paper: universities are not only

Introduction Introduction (1) Main argument of the paper: universities are not only

Le jugement des pairs comme outil de management des universits Peer-review as a management tool Christine Musselin (CSO, Sciences Po et CNRS) Avril 2013, Lirrsistible ascension du capitalisme acadmique Introduction Introduction (1)

590 views • 28 slides
Introduction Introduction to storage and to storage and filesystems filesystems Introduction

Introduction Introduction to storage and to storage and filesystems filesystems Introduction

Moreno Baricevic Gilberto Daz Axel Kohlmeyer Stefano Cozzini ULA ICTP CNR-IOM DEMOCRITOS Merida, VENEZUELA Trieste, ITALY Trieste, ITALY Introduction Introduction to storage and to storage and filesystems filesystems Introduction

1.19k views • 51 slides
JABLOTRON JA-100 introduction November 2011 Introduction Todays goals First introduction of

JABLOTRON JA-100 introduction November 2011 Introduction Todays goals First introduction of

JABLOTRON JA-100 introduction November 2011 Introduction Todays goals First introduction of the brand new JA-100 alarm system Visions Basic architecture System components Setting Help us with final validations You are the FIRST ones

1.69k views • 111 slides
Machine Learning for NLP Readings in unsupervised Learning Aurlie Herbelot 2018 Centre for

Machine Learning for NLP Readings in unsupervised Learning Aurlie Herbelot 2018 Centre for

Machine Learning for NLP Readings in unsupervised Learning Aurlie Herbelot 2018 Centre for Mind/Brain Sciences University of Trento 1 Hashing 2 Hashing: definition Hashing is the process of converting data of arbitrary size into

845 views • 48 slides
Modeling and control of gene regulatory networks Madalena Chaves BIOCO 2 RE (Biological control

Modeling and control of gene regulatory networks Madalena Chaves BIOCO 2 RE (Biological control

1 Modeling and control of gene regulatory networks Madalena Chaves BIOCO 2 RE (Biological control of artificial ecosystems) 2 Math 640 Topics in control theory 3 4 5 Genetic networks: transcription and translation DNA mRNA Protein

850 views • 38 slides
Lecture 2 - HW1,numpy arrays, matplotlib, and git 2020.4.14 Review of Python Basics from Lecture

Lecture 2 - HW1,numpy arrays, matplotlib, and git 2020.4.14 Review of Python Basics from Lecture

Lecture 2 - HW1,numpy arrays, matplotlib, and git 2020.4.14 Review of Python Basics from Lecture 1 Topics from Lecture 1 Lists Strings, ints, floats Dicts Objects, attributes, methods (Tuples,) Functions {sets} Importing .py

1.07k views • 84 slides
Workshop on the Role of Information Sciences and Engineering in Sustainability Hyatt Regency

Workshop on the Role of Information Sciences and Engineering in Sustainability Hyatt Regency

Workshop on the Role of Information Sciences and Engineering in Sustainability Hyatt Regency Washington on Capitol Hill February 2-3, 2011 Please send us your comments, notes, etc.: erwin@cra.org http://cra.org/ccc/seesit o ! meet present

511 views • 9 slides
SITplus Update 25 September 2020 Dan Ryan Program Director, SITplus Partnership Co - funded by

SITplus Update 25 September 2020 Dan Ryan Program Director, SITplus Partnership Co - funded by

SITplus Update 25 September 2020 Dan Ryan Program Director, SITplus Partnership Co - funded by the federal government through Horticulture Innovation Australia Concep Proof of Concept Pre commercial trials Commercialisation TECHNOLOGY

958 views • 60 slides
String manipulation and String manipulation and regexes regexes Programming for Statistical

String manipulation and String manipulation and regexes regexes Programming for Statistical

String manipulation and String manipulation and regexes regexes Programming for Statistical Programming for Statistical Science Science Shawn Santo Shawn Santo 1 / 30 1 / 30 Supplementary materials Full video lecture available in Zoom

348 views • 30 slides
PreCalculus Notes MAT 129 Chapter 6: Exponential and Logarithmic Functions David J. Gisch

PreCalculus Notes MAT 129 Chapter 6: Exponential and Logarithmic Functions David J. Gisch

PreCalculus Notes MAT 129 Chapter 6: Exponential and Logarithmic Functions David J. Gisch Department of Mathematics Des Moines Area Community College September 30, 2011 Chapter 6 1 Section 6.1: Composite Functions Section 6.2: One-to-One

1.43k views • 88 slides
CSE 527 Computational Biology http://www.cs.washington.edu/527 Lecture 1: Overview & Bio

CSE 527 Computational Biology http://www.cs.washington.edu/527 Lecture 1: Overview & Bio

CSE 527 Computational Biology http://www.cs.washington.edu/527 Lecture 1: Overview & Bio Review Autumn 2004 Larry Ruzzo Related Courses He who asks is a fool for five Genome 540/541 (Winter/Spring) Intro. To Comp. Mol. Bio.

242 views • 7 slides

More recommend