ee 559 deep learning 1b pytorch tensors
play

EE-559 Deep learning 1b. PyTorch Tensors Fran cois Fleuret - PowerPoint PPT Presentation

Jan 06, 2024 •347 likes •1.33k views

EE-559 Deep learning 1b. PyTorch Tensors Fran cois Fleuret https://fleuret.org/dlc/ [version of: June 14, 2018] COLE POLYTECHNIQUE FDRALE DE LAUSANNE PyTorchs tensors Fran cois Fleuret EE-559 Deep learning / 1b.

  1. EE-559 – Deep learning 1b. PyTorch Tensors Fran¸ cois Fleuret https://fleuret.org/dlc/ [version of: June 14, 2018] ÉCOLE POLYTECHNIQUE FÉDÉRALE DE LAUSANNE

  2. PyTorch’s tensors Fran¸ cois Fleuret EE-559 – Deep learning / 1b. PyTorch Tensors 2 / 37

  3. A tensor is a generalized matrix, a finite table of numerical values indexed along several discrete dimensions. Fran¸ cois Fleuret EE-559 – Deep learning / 1b. PyTorch Tensors 3 / 37

  4. A tensor is a generalized matrix, a finite table of numerical values indexed along several discrete dimensions. • A 1d tensor is a vector ( e.g. a sound sample), • A 2d tensor is a matrix ( e.g. a grayscale image), • A 3d tensor is a vector of identically sized matrices ( e.g. a multi-channel image), • A 4d tensor is a matrix of identically sized matrices ( e.g. a sequence of multi-channel images), • etc. Fran¸ cois Fleuret EE-559 – Deep learning / 1b. PyTorch Tensors 3 / 37

  5. A tensor is a generalized matrix, a finite table of numerical values indexed along several discrete dimensions. • A 1d tensor is a vector ( e.g. a sound sample), • A 2d tensor is a matrix ( e.g. a grayscale image), • A 3d tensor is a vector of identically sized matrices ( e.g. a multi-channel image), • A 4d tensor is a matrix of identically sized matrices ( e.g. a sequence of multi-channel images), • etc. Tensors are used to encode the signal to process, but also the internal states and parameters of the “neural networks”. Fran¸ cois Fleuret EE-559 – Deep learning / 1b. PyTorch Tensors 3 / 37

  6. A tensor is a generalized matrix, a finite table of numerical values indexed along several discrete dimensions. • A 1d tensor is a vector ( e.g. a sound sample), • A 2d tensor is a matrix ( e.g. a grayscale image), • A 3d tensor is a vector of identically sized matrices ( e.g. a multi-channel image), • A 4d tensor is a matrix of identically sized matrices ( e.g. a sequence of multi-channel images), • etc. Tensors are used to encode the signal to process, but also the internal states and parameters of the “neural networks”. Manipulating data through this constrained structure allows to use CPUs and GPUs at peak performance. Fran¸ cois Fleuret EE-559 – Deep learning / 1b. PyTorch Tensors 3 / 37

  7. A tensor is a generalized matrix, a finite table of numerical values indexed along several discrete dimensions. • A 1d tensor is a vector ( e.g. a sound sample), • A 2d tensor is a matrix ( e.g. a grayscale image), • A 3d tensor is a vector of identically sized matrices ( e.g. a multi-channel image), • A 4d tensor is a matrix of identically sized matrices ( e.g. a sequence of multi-channel images), • etc. Tensors are used to encode the signal to process, but also the internal states and parameters of the “neural networks”. Manipulating data through this constrained structure allows to use CPUs and GPUs at peak performance. Compounded data structures can represent more diverse data types. Fran¸ cois Fleuret EE-559 – Deep learning / 1b. PyTorch Tensors 3 / 37

  8. PyTorch is a Python library built on top of Torch’s THNN computational backend. Its main features are: • Efficient tensor operations on CPU/GPU, • automatic on-the-fly differentiation (autograd), • optimizers, • data I/O. Fran¸ cois Fleuret EE-559 – Deep learning / 1b. PyTorch Tensors 4 / 37

  9. PyTorch is a Python library built on top of Torch’s THNN computational backend. Its main features are: • Efficient tensor operations on CPU/GPU, • automatic on-the-fly differentiation (autograd), • optimizers, • data I/O. “Efficient tensor operations” encompass both standard linear algebra and, as we will see later, deep-learning specific operations (convolution, pooling, etc.) Fran¸ cois Fleuret EE-559 – Deep learning / 1b. PyTorch Tensors 4 / 37

  10. PyTorch is a Python library built on top of Torch’s THNN computational backend. Its main features are: • Efficient tensor operations on CPU/GPU, • automatic on-the-fly differentiation (autograd), • optimizers, • data I/O. “Efficient tensor operations” encompass both standard linear algebra and, as we will see later, deep-learning specific operations (convolution, pooling, etc.) A key specificity of PyTorch is the central role of autograd: tensor operations are specified dynamically as Python operations. We will come back to this. Fran¸ cois Fleuret EE-559 – Deep learning / 1b. PyTorch Tensors 4 / 37

  11. >>> from torch import Tensor >>> x = Tensor (5) >>> x.size () torch.Size ([5]) >>> x.fill_ (1.125) 1.1250 1.1250 1.1250 1.1250 1.1250 [torch. FloatTensor of size 5] >>> x.sum () 5.625 >>> x.mean () 1.125 >>> x.std () 0.0 Fran¸ cois Fleuret EE-559 – Deep learning / 1b. PyTorch Tensors 5 / 37

  12. >>> from torch import Tensor >>> x = Tensor (5) >>> x.size () torch.Size ([5]) >>> x.fill_ (1.125) 1.1250 1.1250 1.1250 1.1250 1.1250 [torch. FloatTensor of size 5] >>> x.sum () 5.625 >>> x.mean () 1.125 >>> x.std () 0.0 Fran¸ cois Fleuret EE-559 – Deep learning / 1b. PyTorch Tensors 5 / 37

  13. >>> from torch import Tensor >>> x = Tensor (5) >>> x.size () torch.Size ([5]) >>> x.fill_ (1.125) 1.1250 1.1250 1.1250 1.1250 1.1250 [torch. FloatTensor of size 5] >>> x.sum () 5.625 >>> x.mean () 1.125 >>> x.std () 0.0 Fran¸ cois Fleuret EE-559 – Deep learning / 1b. PyTorch Tensors 5 / 37

  14. >>> from torch import Tensor >>> x = Tensor (5) >>> x.size () torch.Size ([5]) >>> x.fill_ (1.125) 1.1250 1.1250 1.1250 1.1250 1.1250 [torch. FloatTensor of size 5] >>> x.sum () 5.625 >>> x.mean () 1.125 >>> x.std () 0.0 Fran¸ cois Fleuret EE-559 – Deep learning / 1b. PyTorch Tensors 5 / 37

  15. >>> from torch import Tensor >>> x = Tensor (5) >>> x.size () torch.Size ([5]) >>> x.fill_ (1.125) 1.1250 1.1250 1.1250 1.1250 1.1250 [torch. FloatTensor of size 5] >>> x.sum () 5.625 >>> x.mean () 1.125 >>> x.std () 0.0 The default tensor type torch.Tensor is an alias for torch.FloatTensor , but there are others with greater/lesser precision and on CPU/GPU. It can be set to a different type with torch.set default tensor type Fran¸ cois Fleuret EE-559 – Deep learning / 1b. PyTorch Tensors 5 / 37

  16. >>> from torch import Tensor >>> x = Tensor (5) >>> x.size () torch.Size ([5]) >>> x.fill_ (1.125) 1.1250 1.1250 1.1250 1.1250 1.1250 [torch. FloatTensor of size 5] >>> x.sum () 5.625 >>> x.mean () 1.125 >>> x.std () 0.0 The default tensor type torch.Tensor is an alias for torch.FloatTensor , but there are others with greater/lesser precision and on CPU/GPU. It can be set to a different type with torch.set default tensor type In-place operations are suffixed with an underscore. Fran¸ cois Fleuret EE-559 – Deep learning / 1b. PyTorch Tensors 5 / 37

  17. torch.Tensor.narrow creates a new tensor which is a sub-part of an existing tensor, by constraining one of the indexes. It shares its content with the original tensor, and modifying one modifies the other. >>> a = Tensor (4, 5).zero_ () >>> a 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [torch. FloatTensor of size 4x5] >>> a.narrow (1, 2, 2).fill_ (1.0) 1 1 1 1 1 1 1 1 [torch. FloatTensor of size 4x2] >>> a 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 [torch. FloatTensor of size 4x5] Fran¸ cois Fleuret EE-559 – Deep learning / 1b. PyTorch Tensors 6 / 37

  18. torch.Tensor.narrow creates a new tensor which is a sub-part of an existing tensor, by constraining one of the indexes. It shares its content with the original tensor, and modifying one modifies the other. >>> a = Tensor (4, 5).zero_ () >>> a 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [torch. FloatTensor of size 4x5] >>> a.narrow (1, 2, 2).fill_ (1.0) 1 1 1 1 1 1 1 1 [torch. FloatTensor of size 4x2] >>> a 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 [torch. FloatTensor of size 4x5] Fran¸ cois Fleuret EE-559 – Deep learning / 1b. PyTorch Tensors 6 / 37

  19. torch.Tensor.narrow creates a new tensor which is a sub-part of an existing tensor, by constraining one of the indexes. It shares its content with the original tensor, and modifying one modifies the other. >>> a = Tensor (4, 5).zero_ () >>> a 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [torch. FloatTensor of size 4x5] >>> a.narrow (1, 2, 2).fill_ (1.0) 1 1 1 1 1 1 1 1 [torch. FloatTensor of size 4x2] >>> a 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 [torch. FloatTensor of size 4x5] Fran¸ cois Fleuret EE-559 – Deep learning / 1b. PyTorch Tensors 6 / 37

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Introduction to PyTorch Outline Deep Learning RNN CNN Attention

Introduction to PyTorch Outline Deep Learning RNN CNN Attention

Introduction to PyTorch Outline Deep Learning RNN CNN Attention Transformer Pytorch Introduction Basics Examples Introduction to PyTorch What is PyTorch? Open source machine learning

516 views • 25 slides
>>> ELEG5491: Introduction to Deep Learning >>> PyTorch Tutorials Name: GE

>>> ELEG5491: Introduction to Deep Learning >>> PyTorch Tutorials Name: GE

>>> ELEG5491: Introduction to Deep Learning >>> PyTorch Tutorials Name: GE Yixiao Date: February 14, 2019 yxge@link.cuhk.edu.hk [~]$ _ [1/28] >>> WHAT IS PYTORCH? two sets of audiences: * A replacement for

1.64k views • 31 slides
How PyTorch Scales Deep Learning from Experimentation to Production Vincent Quenneville-Blair,

How PyTorch Scales Deep Learning from Experimentation to Production Vincent Quenneville-Blair,

How PyTorch Scales Deep Learning from Experimentation to Production Vincent Quenneville-Blair, PhD. Facebook AI. Overview Compute with PyTorch Model with Neural Networks Ingest Data Use Multiple GPUs and Machines 1 Compute with PyTorch

656 views • 38 slides
How PyTorch Optimizes Deep Learning Computations Vincent Quenneville-Blair, PhD. Facebook AI.

How PyTorch Optimizes Deep Learning Computations Vincent Quenneville-Blair, PhD. Facebook AI.

How PyTorch Optimizes Deep Learning Computations Vincent Quenneville-Blair, PhD. Facebook AI. Overview Compute with PyTorch Model with Neural Networks Ingest Data Use Multiple GPUs and Machines 1 Compute with PyTorch Example: Pairwise

653 views • 44 slides
PyTorch Review Session CS330: Deep Multi-task and Meta Learning 10/29/2020 Rafael Rafailov

PyTorch Review Session CS330: Deep Multi-task and Meta Learning 10/29/2020 Rafael Rafailov

PyTorch Review Session CS330: Deep Multi-task and Meta Learning 10/29/2020 Rafael Rafailov PyTorch Installation https://pytorch.org/ Check if CUDA is available import torch torch.cuda.is_available() Out[55]: True

381 views • 17 slides
AMMI Introduction to Deep Learning 5.3. PyTorch optimizers Fran cois Fleuret

AMMI Introduction to Deep Learning 5.3. PyTorch optimizers Fran cois Fleuret

AMMI Introduction to Deep Learning 5.3. PyTorch optimizers Fran cois Fleuret https://fleuret.org/ammi-2018/ Sat Nov 10 11:27:22 UTC 2018 COLE POLYTECHNIQUE FDRALE DE LAUSANNE The PyTorch module torch.optim provides many

791 views • 14 slides
AMLD Deep Learning in PyTorch 1. Introduction Fran cois Fleuret http://fleuret.org/amld/

AMLD Deep Learning in PyTorch 1. Introduction Fran cois Fleuret http://fleuret.org/amld/

AMLD Deep Learning in PyTorch 1. Introduction Fran cois Fleuret http://fleuret.org/amld/ February 10, 2018 COLE POLYTECHNIQUE FDRALE DE LAUSANNE Why learning Fran cois Fleuret AMLD Deep Learning in PyTorch / 1.

1.57k views • 108 slides
Introduction to Deep Learning Outline Deep Learning RNN CNN Attention

Introduction to Deep Learning Outline Deep Learning RNN CNN Attention

Introduction to Deep Learning Outline Deep Learning RNN CNN Attention Transformer Pytorch Introduction Basics Examples RNNs Some slides borrowed from Fei-Fei Li & Justin Johnson &

1.1k views • 55 slides
Deep learning inanutshell Hype around AI Core data structure: Tensors A.k.a.

Deep learning inanutshell Hype around AI Core data structure: Tensors A.k.a.

T OWARDS T YPESAFE D EEP L EARNING IN S CALA Tongfei Chen Johns Hopkins University 2 Deep learning inanutshell Hype around AI Core data structure: Tensors A.k.a. Multidimensionalarrays ( NdArray ) the cat sat on the mat

680 views • 43 slides
Introduction to Deep Learning Georgia Tech CS 4650/7650 Fall 2020 Outline Deep Learning

Introduction to Deep Learning Georgia Tech CS 4650/7650 Fall 2020 Outline Deep Learning

Introduction to Deep Learning Georgia Tech CS 4650/7650 Fall 2020 Outline Deep Learning CNN RNN Attention Transformer Pytorch Introduction Basics Examples CNNs Some slides borrowed from

575 views • 56 slides
Introd u ction to P y Torch IN TR OD U C TION TO D E E P L E AR N IN G W ITH P YTOR C H Ismail

Introd u ction to P y Torch IN TR OD U C TION TO D E E P L E AR N IN G W ITH P YTOR C H Ismail

Introd u ction to P y Torch IN TR OD U C TION TO D E E P L E AR N IN G W ITH P YTOR C H Ismail Ele z i Ph . D . St u dent of Deep Learning INTRODUCTION TO DEEP LEARNING WITH PYTORCH Ne u ral net w orks INTRODUCTION TO DEEP LEARNING WITH

537 views • 35 slides
Understanding Machine Learning with Language and Tensors Jon Rawski Linguistics Department

Understanding Machine Learning with Language and Tensors Jon Rawski Linguistics Department

Machine Learning Language Tensors Reduplication Results References Understanding Machine Learning with Language and Tensors Jon Rawski Linguistics Department Institute for Advanced Computational Science Stony Brook University 1 Machine

612 views • 46 slides
Understanding Machine Learning with Language and Tensors Jon Rawski Linguistics Department

Understanding Machine Learning with Language and Tensors Jon Rawski Linguistics Department

Machine Learning Language Tensors Reduplication Understanding Machine Learning with Language and Tensors Jon Rawski Linguistics Department Institute for Advanced Computational Science Stony Brook University 1 Machine Learning Language

602 views • 36 slides
Natural Language Processing with Deep Learning CS224N/Ling284 Matthew Lamm Lecture

Natural Language Processing with Deep Learning CS224N/Ling284 Matthew Lamm Lecture

Natural Language Processing with Deep Learning CS224N/Ling284 Matthew Lamm Lecture 3: Word Window Classification, Neural Networks, and PyTorch 1. Course plan: coming up Week 2: We learn neural net fundamentals We

564 views • 34 slides
Comparing TensorFlow 2.0 with PyTorch and PyTorch JIT Tim Lazarus 29 November, 2019 Comparing

Comparing TensorFlow 2.0 with PyTorch and PyTorch JIT Tim Lazarus 29 November, 2019 Comparing

Comparing TensorFlow 2.0 with PyTorch and PyTorch JIT Tim Lazarus 29 November, 2019 Comparing TensorFlow 2.0 with PyTorch and PyTorch JIT Tim Lazarus 29 November, 2019 Static Computation Graphs In a static execution model , we define the

2.1k views • 14 slides
Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms

Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms

Deep 3D Representation Learning for Visual Computing Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms Conclusion 2 Outline Overview of 3D deep learning Background 3D deep learning tasks 3D deep

1.66k views • 122 slides
CSC444: Midterm Review Carlos Scheidegger D3: DATA-DRIVEN DOCUMENTS The essential idea D3

CSC444: Midterm Review Carlos Scheidegger D3: DATA-DRIVEN DOCUMENTS The essential idea D3

CSC444: Midterm Review Carlos Scheidegger D3: DATA-DRIVEN DOCUMENTS The essential idea D3 creates a two-way association between elements of your dataset and entries in the DOM D3 operates on selections methods apply to all elements

1.05k views • 73 slides
Early searches for supersymmetry at the LHC in the all-hadronic channel Tom Whyntie Imperial

Early searches for supersymmetry at the LHC in the all-hadronic channel Tom Whyntie Imperial

Early searches for supersymmetry at the LHC in the all-hadronic channel Tom Whyntie Imperial College London / CMS experiment Introduction How do we look for supersymmetry at the LHC? What are the challenges of the all-hadronic channel? How

569 views • 12 slides
BGV MC Digitization Progress since BGV #27 Plamen Hopchev CERN BE-BI-BL BGV meeting #29 12 Mar

BGV MC Digitization Progress since BGV #27 Plamen Hopchev CERN BE-BI-BL BGV meeting #29 12 Mar

BGV MC Digitization Progress since BGV #27 Plamen Hopchev CERN BE-BI-BL BGV meeting #29 12 Mar 2014 1 / 8 Summary Event Model Classes The needed classes are ready: SciFiChannelID SciFiLiteCluster SciFiCluster Exact same bit assignment

924 views • 8 slides
The Light Channel of the CRESST Experiment Anja Tanzke Max-Planck-Institute for Physics

The Light Channel of the CRESST Experiment Anja Tanzke Max-Planck-Institute for Physics

The Light Channel of the CRESST Experiment Anja Tanzke Max-Planck-Institute for Physics Technische Universit at M unchen May 9th 2014 Table of Contents Introduction to the CRESST Experiment 1 Light Channel 2 Foil 3 Reflectivity of

330 views • 31 slides
Microarchitectural Cryptanalysis Daniel Moghimi Worcester Polytechnic Institute Committee

Microarchitectural Cryptanalysis Daniel Moghimi Worcester Polytechnic Institute Committee

Revisiting Isolated and Trusted Execution via Microarchitectural Cryptanalysis Daniel Moghimi Worcester Polytechnic Institute Committee Members: Prof. Donald R. Brown (Department Head) Prof. Thomas Eisenbarth (Co-advisor) Prof.

1.68k views • 138 slides
Phylogenetics: Parsimony and Likelihood COMP 571 - Spring 2015 Luay Nakhleh, Rice University

Phylogenetics: Parsimony and Likelihood COMP 571 - Spring 2015 Luay Nakhleh, Rice University

Phylogenetics: Parsimony and Likelihood COMP 571 - Spring 2015 Luay Nakhleh, Rice University The Problem Input: Multiple alignment of a set S of sequences Output: Tree T leaf-labeled with S Assumptions Characters are mutually

1.49k views • 84 slides
Compiling Techniques Lecture 4: Automatic Lexer Generation (EaC 2.4) Christophe Dubach 27

Compiling Techniques Lecture 4: Automatic Lexer Generation (EaC 2.4) Christophe Dubach 27

Finite State Automata for Regular Expression From Regular Expression to Generated Lexer Final Remarks Compiling Techniques Lecture 4: Automatic Lexer Generation (EaC 2.4) Christophe Dubach 27 September 2016 Christophe Dubach Compiling

530 views • 21 slides
Reconstructing ancestral sequences through a combined bioinformatics and molecular modelling

Reconstructing ancestral sequences through a combined bioinformatics and molecular modelling

Reconstructing ancestral sequences through a combined bioinformatics and molecular modelling approach Subha Kalyaanamoorthy | OCE Post-Doctoral Fellow 07 November 2013 CSIRO ECOSYSTEM SCIENCES Ancestral Sequence Reconstruction A rearward

617 views • 17 slides

More recommend