tensor layers for compression of deep learning networks
play

TENSOR LAYERS FOR COMPRESSION OF DEEP LEARNING NETWORKS Cris Cecka - PowerPoint PPT Presentation

Jul 30, 2023 •287 likes •937 views

TENSOR LAYERS FOR COMPRESSION OF DEEP LEARNING NETWORKS Cris Cecka Senior Research Scientist, NVIDIA GTC 2018 Tensors Computations and the GPU AGENDA Tensor Networks and Decompositions Tensor Layers in Deep Learning 2 TENSOR COMPUTATIONS

  1. TENSOR LAYERS FOR COMPRESSION OF DEEP LEARNING NETWORKS Cris Cecka Senior Research Scientist, NVIDIA GTC 2018

  2. Tensors Computations and the GPU AGENDA Tensor Networks and Decompositions Tensor Layers in Deep Learning 2

  3. TENSOR COMPUTATIONS AND THE GPU Modern data is inherently multi-dimensional. 3

  4. TENSOR CONTRACTIONS • Core primitive of multilinear algebra • BLAS level 3 — unbounded compute intensity. X

  5. TENSOR LIBRARIES Explicit permutation dominates. Y . Shi, U. N. Niranjan, A. Anandkumar and C. Cecka , "Tensor Contractions with Extended BLAS Kernels on CPU and GPU," 2016 IEEE X 23rd International Conference on High Performance Computing (HiPC), Hyderabad, 2016, pp. 193-202.

  6. CONTRACTIONS : Single GEMM (Provided compact layout) X

  7. BATCHED MATRIX-MATRIX MULTIPLY cublas<T>gemmStridedBatched X

  8. CONTRACTIONS : Single SB-GEMM (Any layout) X

  9. APPLICATION: FFT : Tensor/FFT — vendor optimized cublas<T>gemmStridedBatched : Custom kernel : FMM Communication StridedBatchedGEMM: 75%+ of the runtime 1.5x over cuFFT on 2xV100 2.6x over cuFFT on 8xV100 Cris Cecka. “Low communication FMM-accelerated FFT on GPUs." In Proceedings of the International Conference for High X Performance Computing, Networking, Storage and Analysis (SC '17). ACM, New York, NY, USA.

  10. WHY TENSORS? 10

  11. DENSITY AND SPARSITY H. Anzt, S. Tomov, J. Dongarra , “Energy Efficiency and Performance Frontiers for Sparse Computations on GPU 11 Supercomputers," PMAM 2015.

  12. DENSITY AND SPARSITY In general, need < 5% sparsity for a computational win. Solutions Block-sparse — Locally dense and globally sparse 12

  13. TENSOR DECOMPOSITIONS Decompositions for data sparse representations. 13

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations

Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations

Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations deeplearning.cce2020@gmail.com Deep Networks Intuition Neural networks with multiple hidden layers - Deep networks [Hinton, 2006] Deep Networks Intuition

1.33k views • 28 slides
Introduction to Artificial Intelligence Deep Learning - Tensor Flow Janyl Jumadinova December 2,

Introduction to Artificial Intelligence Deep Learning - Tensor Flow Janyl Jumadinova December 2,

Introduction to Artificial Intelligence Deep Learning - Tensor Flow Janyl Jumadinova December 2, 2016 Credit: Google Workshop Neural Networks 2/24 Neural Networks 3/24 Neural Networks A fully connected NN layer 4/24 Implementation as

1.2k views • 34 slides
CS 4803 / 7643: Deep Learning Topics: Specifying Layers Forward &amp; Backward

CS 4803 / 7643: Deep Learning Topics: Specifying Layers Forward &amp; Backward

CS 4803 / 7643: Deep Learning Topics: Specifying Layers Forward &amp; Backward autodifferentiation (Beginning of) Convolutional neural networks Zsolt Kira Georgia Tech Projects We will release a set of project ideas for those

1.04k views • 50 slides
CS 4803 / 7643: Deep Learning Topics: Specifying Layers Forward &amp; Backward

CS 4803 / 7643: Deep Learning Topics: Specifying Layers Forward &amp; Backward

CS 4803 / 7643: Deep Learning Topics: Specifying Layers Forward &amp; Backward autodifferentiation (Beginning of) Convolutional neural networks Zsolt Kira Georgia Tech Administrivia PS0 released mean of 20.7

1.39k views • 92 slides
Collaborative Channel Pruning for Deep Networks 11th June 2019 Background Model compression

Collaborative Channel Pruning for Deep Networks 11th June 2019 Background Model compression

Collaborative Channel Pruning for Deep Networks 11th June 2019 Background Model compression method Compact network design; Source:https://orbograph.com/ deep-learning-how-will-it-change-healthcare/ Network quantization; Channel or

396 views • 12 slides
Deep Learning for Predictive Maintenance Pawel Morkisz GTC 2017 Agenda Problem

Deep Learning for Predictive Maintenance Pawel Morkisz GTC 2017 Agenda Problem

Deep Learning for Predictive Maintenance Pawel Morkisz GTC 2017 Agenda Problem Introduction and notion of deep neural networks o Convolutional layers o Residual networks (ResNet) One dimensional convolutional networks in failure

276 views • 23 slides
DEEP LEARNING FFR135, Artificial Neural Networks Olof Mogren Chalmers University of Technology

DEEP LEARNING FFR135, Artificial Neural Networks Olof Mogren Chalmers University of Technology

DEEP LEARNING FFR135, Artificial Neural Networks Olof Mogren Chalmers University of Technology October 2016 DEEP LEARNING Artificial neural networks Many layers of abstractions Outperforms traditional methods in: Image

824 views • 61 slides
Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan Zelener Machine Learning Reading Group January 7 th 2016 The Graduate Center, CUNY Objectives Explain some of the trends of deep learning and

680 views • 44 slides
Tensor Methods for Signal Processing and Machine Learning Qibin Zhao Tensor Learning Unit RIKEN

Tensor Methods for Signal Processing and Machine Learning Qibin Zhao Tensor Learning Unit RIKEN

Tensor Methods for Signal Processing and Machine Learning Qibin Zhao Tensor Learning Unit RIKEN AIP 2018-6-9 @ Waseda University 1 Monographs Tensor networks for dimensionality reduction and large optimization Andrzej Cichocki, Namgil

1.34k views • 77 slides
Applications of Tensor Networks: Machine Learning &amp; Quantum Computing E.M. Stoudenmire Aug

Applications of Tensor Networks: Machine Learning &amp; Quantum Computing E.M. Stoudenmire Aug

Applications of Tensor Networks: Machine Learning &amp; Quantum Computing E.M. Stoudenmire Aug 2018 - ICTP Trieste Outline: Yesterday : intro to tensor networks, mainly matrix product states (MPS) computations with MPS intro to

1.78k views • 142 slides
Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and Courville [chapt. 6,7,8]; AIMA [sect. 21.1-21.3]; Sutton and Barto, Reinforcement Learning: an

527 views • 35 slides
Using Tensor Swapping and NVLink to Overcome GPU Memory Limits with TensorFlow Sam Matzek Deep

Using Tensor Swapping and NVLink to Overcome GPU Memory Limits with TensorFlow Sam Matzek Deep

Using Tensor Swapping and NVLink to Overcome GPU Memory Limits with TensorFlow Sam Matzek Deep learning is memory constrained GPUs have limited memory Neural networks are growing deeper and wider Amount and size of data to process is

397 views • 23 slides
Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference

Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference

Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference Engine Song Han CVA group, Stanford University Apr 7, 2016 1 Intro about me and my advisor Fourth year PhD with Prof. Bill Dally at Stanford.

2.06k views • 70 slides
Efficient Neural Network Compression Namhoon Lee University of Oxford 3 May 2019 A Challenge in

Efficient Neural Network Compression Namhoon Lee University of Oxford 3 May 2019 A Challenge in

Efficient Neural Network Compression Namhoon Lee University of Oxford 3 May 2019 A Challenge in Deep Learning: Overparameterization Large neural networks require: memory &amp; computations power consumption A Challenge in Deep Learning:

722 views • 49 slides
Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference

Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference

Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference Engine Song Han CVA group, Stanford University Jan 6, 2015 A few words about us Fourth year PhD with Prof. Bill Dally at Stanford.

966 views • 54 slides
Deep Networks Andrea Passerini passerini@disi.unitn.it Machine Learning Deep Networks Need for

Deep Networks Andrea Passerini passerini@disi.unitn.it Machine Learning Deep Networks Need for

Deep Networks Andrea Passerini passerini@disi.unitn.it Machine Learning Deep Networks Need for Deep Networks Perceptron Can only model linear functions Kernel Machines Non-linearity provided by kernels Need to design appropriate kernels

563 views • 43 slides
Presentation to the Standing Committee on Finance On Budget Bill C-38 and The Abolition of

Presentation to the Standing Committee on Finance On Budget Bill C-38 and The Abolition of

Presentation to the Standing Committee on Finance On Budget Bill C-38 and The Abolition of the Canadian Artists and Producers Professional Relations Tribunal May 31st, 2012 Good afternoon, Thank you for inviting me to testify before you

448 views • 3 slides
arXiv:1712.08207v3 [cs.CL] 21 Jun 2018 decoder networks. When combined with a traditional

arXiv:1712.08207v3 [cs.CL] 21 Jun 2018 decoder networks. When combined with a traditional

Variational Attention for Sequence-to-Sequence Models Hareesh Bahuleyan Lili Mou Olga Vechtomova Pascal Poupart University of Waterloo, Canada { hpallika, ovechtomova, ppoupart } @uwaterloo.ca AdeptMind Research,

265 views • 11 slides
Variational Inequalities Learn about basic networks economics in Network Economics and

Variational Inequalities Learn about basic networks economics in Network Economics and

Mat-2.142 Seminar on optimization Goals of the seminar Variational Inequalities Learn about basic networks economics in Network Economics and variational inequalities. Improve understanding of different Session 1: Introduction

71 views • 6 slides
Manifold Embeddings for Model-Based Reinforcement Learning under Partial Observability By Keith

Manifold Embeddings for Model-Based Reinforcement Learning under Partial Observability By Keith

Outline Introduction Method Experiments Conclusions Manifold Embeddings for Model-Based Reinforcement Learning under Partial Observability By Keith Bush and Joelle Pineau at NIPS 2009 Presented by Chenghui Cai Duke University, ECE

402 views • 18 slides
Strong winds in a coupled wave-atmosphere model during a North Atlantic storm event: evaluation

Strong winds in a coupled wave-atmosphere model during a North Atlantic storm event: evaluation

Strong winds in a coupled wave-atmosphere model during a North Atlantic storm event: evaluation against observations Lucia Pineau-Guillou (Ifremer/LOPS) , Fabrice Ardhuin, Marie-Nolle Bouin, Jean-Luc Redelsperger, Bertrand Chapron, Jean Bidlot (

479 views • 13 slides
Pinot Noir: The Savage Yet Seductive Grape The Beauty and the Beast Dr. Karl Kaiser

Pinot Noir: The Savage Yet Seductive Grape The Beauty and the Beast Dr. Karl Kaiser

Pinot Noir: The Savage Yet Seductive Grape The Beauty and the Beast Dr. Karl Kaiser CCOVI Professional Affiliate Brock University, St. Catharines, ON Introduction to Pinot Noir The Story of the Origin Quotes taken from Pinot

818 views • 38 slides
Junior Open Maubeuge 2017 July 8 th to 11 th , 2017 Schedule of the event: Saturday July 8 th :

Junior Open Maubeuge 2017 July 8 th to 11 th , 2017 Schedule of the event: Saturday July 8 th :

Junior Open Maubeuge 2017 July 8 th to 11 th , 2017 Schedule of the event: Saturday July 8 th : Greetings (pick up from Bruxelles (BRU) or Charleroi (CRL) airports) Sunday July 9 th : Opening ceremony and first games of the competition Monday

157 views • 3 slides
Carson River Basin Wes Kitlasten Murphy Gardner Eric Morway Rich Niswonger Enrique Triana

Carson River Basin Wes Kitlasten Murphy Gardner Eric Morway Rich Niswonger Enrique Triana

Simulating Future Runoff and Water Use in the Carson River Basin Wes Kitlasten Murphy Gardner Eric Morway Rich Niswonger Enrique Triana Funded by: NSF/USDA Overview What does the historical data suggest? What adjustments do our

686 views • 56 slides

More recommend