and inference for convolutional
play

and Inference for Convolutional Neural Networks 1 2 FFT IFFT 3 - PowerPoint PPT Presentation

Aug 12, 2023 •182 likes •648 views

Band-limited Training and Inference for Convolutional Neural Networks 1 2 FFT IFFT 3 4 Mathieu et al.: Fast Training of Convolutional Networks through FFTs Fast Convolutional Nets With fbfft: A GPU Performance Evaluation Data: x

  1. Band-limited Training and Inference for Convolutional Neural Networks 1

  2. 2

  3. FFT IFFT 3

  4. 4

  5. Mathieu et al.: “Fast Training of Convolutional Networks through FFTs” Fast Convolutional Nets With fbfft: A GPU Performance Evaluation Data: x xfft FFT(x) offt Out: o xfft  yfft IFFT(offt) Filter: y FFT(y) yfft

  6. Mathieu et al.: “Fast Training of Convolutional Networks through FFTs” Fast Convolutional Nets With fbfft: A GPU Performance Evaluation Data: x xfft FFT(x) offt Out: o xfft  yfft IFFT(offt) Filter: y FFT(y) yfft

  7. Mathieu et al.: “Fast Training of Convolutional Networks through FFTs” Fast Convolutional Nets With fbfft: A GPU Performance Evaluation Data: x xfft FFT(x) Out: o offt xfft  yfft IFFT(offt) Filter: y FFT(y) yfft

  8. Mathieu et al.: “Fast Training of Convolutional Networks through FFTs” Fast Convolutional Nets With fbfft: A GPU Performance Evaluation Data: x xfft FFT(x) offt Out: o xfft  yfft IFFT(offt) Filter: y FFT(y) yfft

  9. Mathieu et al.: “Fast Training of Convolutional Networks through FFTs” Fast Convolutional Nets With fbfft: A GPU Performance Evaluation cuDNN cu DNN: Subs ubstantial tantial memor ory y wor orkspace space neede ded d for or intermed ermediate iate resul ults. ts. Data: x xfft FFT(x) offt Out: o xfft  yfft IFFT(offt) Filter: y FFT(y) yfft

  10. Band-limiting = masking out high frequencies xfft Data: x xCfft Band-limited FFT(x) (xfft) offt Out: o xCfft  yCfft IFFT(offt) Filter: y yfft Band-limited FFT(y) (yfft) yCfft

  11. xfft Data: x xCfft Band-limited Less memory used FFT(x) (xfft) offt Out: o xCfft  yCfft IFFT(offt) Filter: y yfft Band-limited FFT(y) (yfft) yCfft

  12. xfft Data: x xCfft Band-limited Less memory used FFT(x) (xfft) offt Out: o xCfft  yCfft IFFT(offt) Filter: y yfft Band-limited FFT(y) Faster computation (yfft) yCfft

  13. Preserve enough of the spectrum to retain high accuracy of models. xfft Data: x xCfft Band-limited Less memory used FFT(x) (xfft) offt Out: o xCfft  yCfft IFFT(offt) Filter: y yfft Band-limited FFT(y) Faster computation (yfft) yCfft

  14. 14

  15. 2. Conjugate symmetry 1-j 1+j 15

  16. 2. Conjugate symmetry 1+j 16

  17. 2. Conjugate symmetry DC 1+j 3. Real values 17

  18. 2. Conjugate symmetry DC 1+j 3. Real values 4. No constraints 18

  19. 2. Conjugate symmetry DC 3. Real values 4. No constraints 5. 1 st compression 19

  20. 2. Conjugate symmetry DC 3. Real values 4. No constraints 5. 1 st compression 20

  21. 2. Conjugate symmetry DC 3. Real values 4. No constraints 5. 1 st compression 6. 2 nd compression 21

  22. 2. Conjugate symmetry DC 3. Real values 4. No constraints 5. 1 st compression 6. 2 nd compression 22

  23. Test Accuracy (%) 95 90 ResNet-18 on CIFAR-10 85 0 20 40 60 80 Compression rate (%) 23

  24. 93.5% Test Accuracy (%) 95 90 ResNet-18 on CIFAR-10 85 0 20 40 60 80 Compression rate (%) 24

  25. 93.5% Test Accuracy (%) 95 92% 90 ResNet-18 on CIFAR-10 85 0 20 40 60 80 Compression rate (%) 25

  26. 93.5% Test Accuracy (%) 95 92% 90 ResNet-18 on CIFAR-10 85 0 20 40 60 80 Test Accuracy (%) 80 75.3% 71.2% 70 DenseNet-121 on CIFAR-100 60 0 20 40 60 80 Compression rate (%) 26

  27. ▪ ▪ ▪ ▪ ▪ ▪ ▪ 27

  30. ▪ ▪ ▪ ▪ ▪ ▪ ▪ ▪ 30

  31. Cross-correlate input data and filter: x ∗ c y F x ω = F x n F y ω = F y n x ∗ c y = F −1 (F x ω ʘ F y ω ) Spectrum of convolution: S ω = F x ω ʘ F y ω 𝐍 𝐝 𝛛 = ቊ 𝟐, 𝛛 ≤ 𝐝 𝐏, 𝛛 > 𝐝 x ∗ c y = F −1 [ F x ω ʘ M c ω ) ʘ (F y ω ʘ M c ω ] x ∗ c y = F −1 S ω ʘ M c ω 𝑂−1 |𝑦 𝑜 | 2 = σ 𝜕=0 Energy (Parseval’s theorem): σ 𝑜=0 2𝜌 𝑦 𝜕 | 2 |𝐺 31

  32. 32

  33. DenseNet-121 on CIFAR-100 80 70 Test accuracy (%) 60 50 40 30 20 10 C=50 C=75 0 0 20 40 60 80 Inference Compression Rate (%) 33

  34. DenseNet-121 on CIFAR-100 80 70 Test accuracy (%) 60 50 40 30 20 10 C=0 C=50 C=75 C=85 0 0 20 40 60 80 Inference Compression Rate (%) 34

  35. ResNet-18 on CIFAR-10 performance Normalized 100 (%) 50 GPU memory allocated 0 0 20 40 60 80 performance 100 Normalized 50 (%) Epoch time 0 0 20 40 60 80 Compression rate (%) 35

  36. 100 ResNet-18 on CIFAR-10 80 Test accuracy (%) 60 40 Train Compression Rate (%): 20 C=0 0 0 10 20 30 40 50 60 70 80 Inference Compression Rate (%) 36

  37. 100 ResNet-18 on CIFAR-10 80 Test accuracy (%) 60 40 Train Compression Rate (%): 20 C=0 C=85 0 0 10 20 30 40 50 60 70 80 Inference Compression Rate (%) 37

  38. 100 ResNet-18 on CIFAR-10 80 Test accuracy (%) 60 40 Train Compression Rate (%): 20 C=0 C=85 0 0 10 20 30 40 50 60 70 80 Inference Compression Rate (%) Smooth degradation of accuracy during inference 38

  39. 100 ResNet-18 on CIFAR-10 80 Test accuracy (%) 60 40 Train Compression Rate (%): 20 C=0 C=30 C=50 C=85 0 0 10 20 30 40 50 60 70 80 Inference Compression Rate (%) Apply the same compression rate to training and inference 39

  40. Test Accuracy (%) 95 100 90 50 GPU memory allocated 85 0 0 50 0 20 40 60 80 100 Test Accuracy (%) 80 50 70 Epoch time 0 60 0 20 40 60 80 0 50 Compression rate (%) Compression rate (%) 40

  41. 41

  42. 42

  43. 43

  44. 44

  45. 45

  46. “Speaking of longer term, it would be nice if the community migrated to a fully open sourced implementation for all of this [convolution operations, etc.]. This stuff is just too important to the progress of the field for it to be locked away in proprietary implementations . The more people working together on this the better for everyone. There's plenty of room to compete on the hardware implementation side.” Scott Gray https://github.com/soumith/convnet-benchmarks/issues/93 46

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Adaptive Distributed Convolutional Neural Network Inference at the Network Edge with ADCNN 17

Adaptive Distributed Convolutional Neural Network Inference at the Network Edge with ADCNN 17

Adaptive Distributed Convolutional Neural Network Inference at the Network Edge with ADCNN 17 August 2020 ICPP 2020 Sai Qian Zhang , Jieyu Lin, Qi Zhang Executing DNN Inference Tasks for End Users Option 2: cloud only Option 1: edge only

579 views • 17 slides
Inference Networks, Graph Convolutional Networks Greg Mori School of Computing Science Simon

Inference Networks, Graph Convolutional Networks Greg Mori School of Computing Science Simon

Inference Networks, Graph Convolutional Networks Greg Mori School of Computing Science Simon Fraser University Outline Scene Image annotation with label hierarchies outdoor outdoor indoor man-made natural house sports man-made

682 views • 47 slides
Convolutional Autoencoder (CAE) Prof. Seungchul Lee Industrial AI Lab. Convolutional Autoencoder

Convolutional Autoencoder (CAE) Prof. Seungchul Lee Industrial AI Lab. Convolutional Autoencoder

Convolutional Autoencoder (CAE) Prof. Seungchul Lee Industrial AI Lab. Convolutional Autoencoder Motivation: image to autoencoder ? Convolutional autoencoder extends the basic structure of the simple autoencoder by changing the fully

349 views • 15 slides
Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural

Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural

Transfer Learning with Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural Networks A breakthough Convolutional Neural Networks VGG-16 example Layers of Convolutional filters Bottleneck

625 views • 23 slides
Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN)

Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN)

Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN) A.k.a. CNN or ConvNet Adit Deshpande, A Beginner's Guide To Understanding Convolutional Neural Networks. Digital Images Input array: an images

1.42k views • 72 slides
Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly

Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly

Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly used for what sort of task? a) Recognizing images b) Transcribing speech c) Translating documents d) Playing games Neural network review

431 views • 13 slides
15-780 Graduate Artificial Intelligence: Convolutional and recurrent networks J. Zico Kolter

15-780 Graduate Artificial Intelligence: Convolutional and recurrent networks J. Zico Kolter

15-780 Graduate Artificial Intelligence: Convolutional and recurrent networks J. Zico Kolter (this lecture) and Ariel Procaccia Carnegie Mellon University Spring 2017 1 Outline Convolutional neural networks Applications of convolutional

1.32k views • 32 slides
Incremental and Approximate Inference for Faster Occlusion-based Deep CNN Explanations Supun

Incremental and Approximate Inference for Faster Occlusion-based Deep CNN Explanations Supun

Incremental and Approximate Inference for Faster Occlusion-based Deep CNN Explanations Supun Nakandala , Arun Kumar, and Yannis Papakonstantinou University of California, San Diego Introduction Deep Convolutional Neural Networks (CNNs) are

455 views • 30 slides
Exact Inference Inference Basic task for inference: Compute

Exact Inference Inference Basic task for inference: Compute

Exact Inference Inference Basic task for inference: Compute a posterior distribu8on for some query variables given some observed evidence Sum out

699 views • 30 slides
Convolutional Neural Networks (Part III) 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image

Convolutional Neural Networks (Part III) 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image

Convolutional Neural Networks (Part III) 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image Processing 2016 Prof. Luiz Velho Convolutional Neural Networks 1 Summary & References 08/11 ImageNet Classification with Deep Convolutional

444 views • 27 slides
Convolutional Neural Networks in Speech Lecture 20 CS 753 Instructor: Preethi Jyothi

Convolutional Neural Networks in Speech Lecture 20 CS 753 Instructor: Preethi Jyothi

Convolutional Neural Networks in Speech Lecture 20 CS 753 Instructor: Preethi Jyothi Convolutional Neural Networks (CNNs) Fully connected (dense) layers have no awareness of spatial information Key concept behind convolutional layers is

983 views • 41 slides
Convolutional Neural Nets CS447 Natural Language Processing (J. Hockenmaier)

Convolutional Neural Nets CS447 Natural Language Processing (J. Hockenmaier)

Lecture 8: Convolutional Neural Nets Convolutional Neural Nets CS447 Natural Language Processing (J. Hockenmaier) https://courses.grainger.illinois.edu/cs447/ 1 Convolutional Neural Nets (ConvNets, CNNs) [4 parameters, applied 3

615 views • 24 slides
Inference in Bayesian networks Chapter 14.45 Chapter 14.45 1 Outline Exact inference

Inference in Bayesian networks Chapter 14.45 Chapter 14.45 1 Outline Exact inference

Inference in Bayesian networks Chapter 14.45 Chapter 14.45 1 Outline Exact inference by enumeration Approximate inference by stochastic simulation Chapter 14.45 2 Inference tasks Simple queries: compute posterior marginal P (

450 views • 28 slides
Convolutional Neural Networks 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image Processing

Convolutional Neural Networks 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image Processing

Convolutional Neural Networks 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image Processing 2016 Prof. Luiz Velho Convolutional Neural Networks 1 Summary & References 08/11 ImageNet Classification with Deep Convolutional Neural Networks

648 views • 26 slides
Inference in First-Order Logic C H A P T E R 9 H A S S A N K H O S R A V I S P R I N G 2 0 1 1

Inference in First-Order Logic C H A P T E R 9 H A S S A N K H O S R A V I S P R I N G 2 0 1 1

Inference in First-Order Logic C H A P T E R 9 H A S S A N K H O S R A V I S P R I N G 2 0 1 1 Outline Reducing first-order inference to propositional inference Unification Generalized Modus Ponens Forward chaining Backward

856 views • 58 slides
Semantic Segmentation of the sekleton in bone scintigraphy images with convolutional neural

Semantic Segmentation of the sekleton in bone scintigraphy images with convolutional neural

Semantic Segmentation of the sekleton in bone scintigraphy images with convolutional neural networks Problem Description Convolutional Neural Networks Convolutional Layer Max Pooling Transforming Classification networks to segmentation

157 views • 12 slides
Near Neighbor Search in High Dimensional Data (2) Locality-Sensitive Hashing (continued) LS

Near Neighbor Search in High Dimensional Data (2) Locality-Sensitive Hashing (continued) LS

Near Neighbor Search in High Dimensional Data (2) Locality-Sensitive Hashing (continued) LS Families and Amplification LS Families for Common Distance Measures Anand Rajaraman The Big Picture Candidate pairs : Locality- those pairs

628 views • 44 slides
How to make the first million? How to create a company? Leszek Czarnecki, PhD London, October

How to make the first million? How to create a company? Leszek Czarnecki, PhD London, October

How to make the first million? How to create a company? Leszek Czarnecki, PhD London, October 2011 Leszek Czarnecki, PhD Graduated from Wroclaw University of Technology (Faculty of Sanitary Engineering) Graduated from Wroclaw University of

656 views • 27 slides
association of financial mutuals 53 member companies 20 million policyholders Association of

association of financial mutuals 53 member companies 20 million policyholders Association of

! association of financial mutuals 53 member companies 20 million policyholders Association of Financial Mutuals 125 billion assets fastest growing part of under management the insurance sector HOW DO PEOPLE BUY INSURANCE TODAY?

139 views • 13 slides
Aviva plc plc Aviva 2002 Interim Results 2002 Interim Results 1 August 2002 1 August 2002

Aviva plc plc Aviva 2002 Interim Results 2002 Interim Results 1 August 2002 1 August 2002

Aviva plc plc Aviva 2002 Interim Results 2002 Interim Results 1 August 2002 1 August 2002 Agenda Agenda Business review Business review Richard Harvey Richard Harvey Financial review Financial review Mike Biggs Mike

676 views • 36 slides
Unsupervised Music Understanding based on Nonparametric Bayesian Models Kazuyoshi Yoshii Masataka

Unsupervised Music Understanding based on Nonparametric Bayesian Models Kazuyoshi Yoshii Masataka

Unsupervised Music Understanding based on Nonparametric Bayesian Models Kazuyoshi Yoshii Masataka Goto AIST, Japan Why can we recognize music as music? Is this because we are taught music theory? Definitely No! Why Unsupervised Understanding?

393 views • 22 slides
Communications Strategy Page 1 Scott Edgell General Manager Will Graham Minute Item 4

Communications Strategy Page 1 Scott Edgell General Manager Will Graham Minute Item 4

New recycling and waste collection: Communications Strategy Page 1 Scott Edgell General Manager Will Graham Minute Item 4 Business Support Manager Whats coming up Current service model Key reasons to change New and improved

781 views • 18 slides
Transition Research Programme Allan Colver Professor of Community Child Health Newcastle

Transition Research Programme Allan Colver Professor of Community Child Health Newcastle

Transition Research Programme Allan Colver Professor of Community Child Health Newcastle University Disclaimer and thanks This conference summarises independent research funded by the National Institute for Health Research (NIHR) under

1.8k views • 143 slides
31 January 2015 Primary 6 Outline 1) About the teachers Introduction Preferred mode of

31 January 2015 Primary 6 Outline 1) About the teachers Introduction Preferred mode of

Parents Briefing 31 January 2015 Primary 6 Outline 1) About the teachers Introduction Preferred mode of communication Class rules/ Procedures 2) About the school Address by Principal (Video) General School Matters 3)

1.03k views • 78 slides

More recommend