a framework for new generation ai research
play

a framework for new-generation AI Research Soumith Chintala, Adam - PowerPoint PPT Presentation

May 27, 2023 •358 likes •1.03k views

a framework for new-generation AI Research Soumith Chintala, Adam Paszke, Sam Gross & Team Facebook AI Research Paradigm shifts in AI research Active Research & Tools for AI Today's AI Future AI keeping up with change Today's AI

  2. a framework for new-generation AI Research Soumith Chintala, Adam Paszke, Sam Gross & Team Facebook AI Research

  3. Paradigm shifts in AI research

  4. Active Research & Tools for AI Today's AI Future AI keeping up with change

  5. Today's AI DenseCap by Justin Johnson & group https://github.com/jcjohnson/densecap Future AI Tools for AI Today's AI

  6. Today's AI DeepMask by Pedro Pinhero & group Future AI Tools for AI Today's AI

  7. Today's AI Machine Translation Future AI Tools for AI Today's AI

  8. Today's AI Text Classification (sentiment analysis etc.) Text Embeddings Graph embeddings Machine Translation Ads ranking Future AI Tools for AI Today's AI

  9. Today's AI Model BatchNorm Train Model Conv2d Objective ReLU Data Future AI Tools for AI Today's AI

  10. Today's AI Model BatchNorm Train Model Conv2d Objective ReLU Data BatchNorm New Conv2d Prediction Deploy & Use ReLU Data Future AI Tools for AI Today's AI

  11. Today's AI Static datasets + Static model structure BatchNorm Train Model Conv2d Objective ReLU Data BatchNorm New Conv2d Prediction Deploy & Use ReLU Data Future AI Tools for AI Today's AI

  12. Today's AI Static datasets + Static model structure BatchNorm Train Model Conv2d Objective ReLU Data O ffl ine Learning BatchNorm New Conv2d Prediction Deploy & Use ReLU Data Future AI Tools for AI Today's AI

  13. Current AI Research / Future AI Self-driving Cars Future AI Tools for AI Today's AI

  14. Current AI Research / Future AI Agents trained in many environments Cars Video games Internet Future AI Tools for AI Today's AI

  15. Current AI Research / Future AI Dynamic Neural Networks self-adding new memory or layers changing evaluation path based on inputs Future AI Tools for AI Today's AI

  16. Current AI Research / Future AI BatchNorm Live Conv2d Prediction ReLU data Continued Online Learning Future AI Tools for AI Today's AI

  17. Prediction Data-dependent change in model structure Tools for AI Current AI Research / Future AI ReLU R e L U BatchNorm BatchNorm Conv2d C o n v 2 d ReLU ReLU BatchNorm BatchNorm Future AI Conv2d Conv2d ReLU ReLU B a t m c h r o N N o r h m c t a B Conv2d Conv2d ReLU BatchNorm Conv2d Today's AI Sample -1

  18. Current AI Research / Future AI BatchNorm Conv2d ReLU ReLU C o n v BatchNorm 2 B a d t c h N o r R m e Conv2d BatchNorm L ReLU U Conv2d ReLU BatchNorm Conv2d Sample -2 Prediction B a t c h N Conv2d o r m ReLU Data-dependent change in model structure Future AI Tools for AI Today's AI

  19. Prediction Change in model-capacity at runtime Tools for AI Current AI Research / Future AI ReLU ReLU ReLU BatchNorm BatchNorm BatchNorm Conv2d Conv2d Conv2d Future AI ReLU ReLU ReLU BatchNorm BatchNorm BatchNorm Conv2d Conv2d Conv2d ReLU BatchNorm Conv2d Today's AI Sample

  20. Prediction Change in model-capacity at runtime Tools for AI Current AI Research / Future AI ReLU ReLU ReLU ReLU ReLU BatchNorm BatchNorm BatchNorm BatchNorm BatchNorm Conv2d Conv2d Conv2d Conv2d Conv2d Future AI ReLU ReLU ReLU ReLU ReLU BatchNorm BatchNorm BatchNorm BatchNorm BatchNorm Conv2d Conv2d Conv2d Conv2d Conv2d ReLU BatchNorm Conv2d Today's AI Sample

  21. The need for a dynamic framework • Interop with many dynamic environments - Connecting to car sensors should be as easy as training on a dataset - Connect to environments such as OpenAI Universe • Dynamic Neural Networks - Change behavior and structure of neural network at runtime • Minimal Abstractions - more complex AI systems means harder to debug without a simple API Future AI Tools for AI Today's AI

  22. Tools for AI research and deployment Many machine learning tools and deep learning frameworks Future AI Tools for AI Today's AI

  23. Tools for AI research and deployment Static graph frameworks Dynamic graph frameworks define-by-run define-and-run Future AI Tools for AI Today's AI

  24. Dynamic graph Frameworks • Model is constructed on the fly at runtime • Change behavior, structure of model • Imperative style of programming Future AI Tools for AI Today's AI

  26. Overview automatic di ff erentiation Ndarray library gradient based engine with GPU support optimization package Deep Learning Numpy-alternative Reinforcement Learning

  27. ndarray library with GPU support

  28. ndarray library • np.ndarray <-> torch.Tensor • 200+ operations, similar to numpy • very fast acceleration on NVIDIA GPUs

  29. ndarray library PyTorch Numpy

  30. ndarray / Tensor library

  31. ndarray / Tensor library

  32. ndarray / Tensor library

  33. ndarray / Tensor library

  34. NumPy bridge

  35. NumPy bridge Zero memory-copy very e ffi cient

  36. NumPy bridge

  37. NumPy bridge

  38. Seamless GPU Tensors

  39. Seamless GPU Tensors A full suite of high performance Tensor operations on GPU

  40. GPUs are fast • Buy $700 NVIDIA 1080Ti • 100x faster matrix multiply • 10x faster operations in general on matrices

  41. automatic di ff erentiation engine for deep learning and reinforcement learning

  42. Deep Learning Frameworks - Provide gradient computation - Gradient of one variable w.r.t. any variable in graph MM MM Add Tanh

  43. Deep Learning Frameworks - Provide gradient computation - Gradient of one variable w.r.t. any variable in graph d(i2h)/d(W_h) MM MM Add Tanh

  44. Deep Learning Frameworks - Provide gradient computation - Gradient of one variable w.r.t. any variable in graph - Provide integration with high performance DL libraries like CuDNN MM MM d(h2h)/d(W_h) Add Tanh

  45. PyTorch Autograd from torch.autograd import Variable

  46. PyTorch Autograd from torch.autograd import Variable x = Variable(torch.randn(1, 10)) prev_h = Variable(torch.randn(1, 20)) W_h = Variable(torch.randn(20, 20)) W_x = Variable(torch.randn(20, 10))

  47. PyTorch Autograd from torch.autograd import Variable x = Variable(torch.randn(1, 10)) MM MM prev_h = Variable(torch.randn(1, 20)) W_h = Variable(torch.randn(20, 20)) W_x = Variable(torch.randn(20, 10)) i2h = torch.mm(W_x, x.t()) h2h = torch.mm(W_h, prev_h.t())

  48. PyTorch Autograd from torch.autograd import Variable x = Variable(torch.randn(1, 10)) MM MM prev_h = Variable(torch.randn(1, 20)) W_h = Variable(torch.randn(20, 20)) W_x = Variable(torch.randn(20, 10)) i2h = torch.mm(W_x, x.t()) h2h = torch.mm(W_h, prev_h.t()) next_h = i2h + h2h

  49. PyTorch Autograd from torch.autograd import Variable x = Variable(torch.randn(1, 10)) MM MM prev_h = Variable(torch.randn(1, 20)) W_h = Variable(torch.randn(20, 20)) W_x = Variable(torch.randn(20, 10)) Add i2h = torch.mm(W_x, x.t()) h2h = torch.mm(W_h, prev_h.t()) next_h = i2h + h2h

  50. PyTorch Autograd from torch.autograd import Variable x = Variable(torch.randn(1, 10)) MM MM prev_h = Variable(torch.randn(1, 20)) W_h = Variable(torch.randn(20, 20)) W_x = Variable(torch.randn(20, 10)) Add i2h = torch.mm(W_x, x.t()) h2h = torch.mm(W_h, prev_h.t()) next_h = i2h + h2h Tanh next_h = next_h.tanh()

  51. PyTorch Autograd from torch.autograd import Variable x = Variable(torch.randn(1, 10)) MM MM prev_h = Variable(torch.randn(1, 20)) W_h = Variable(torch.randn(20, 20)) W_x = Variable(torch.randn(20, 10)) Add i2h = torch.mm(W_x, x.t()) h2h = torch.mm(W_h, prev_h.t()) next_h = i2h + h2h Tanh next_h = next_h.tanh() next_h.backward(torch.ones(1, 20))

  52. side by side: TensorFlow and PyTorch

  53. High performance • Integration of: • CuDNN v6 • NCCL • Intel MKL • 200+ operations, similar to numpy • very fast acceleration on NVIDIA GPUs Upcoming feature: 
 Distributed PyTorch

  54. Planned Feature: JIT Compilation

  55. Compilation benefits Out-of-order Automatic Kernel fusion execution work placement Node 1 ReLU CPU 1 2 3 3 1 2 Node 1 GPU 1 Node 0 BatchNorm GPU 0 Node 1 Node 0 GPU 0 GPU 1 Conv2d Node 0 GPU 0

  56. JIT Compilation • Possible in define-by-run Frameworks • The key idea is deferred or lazy evaluation - y = x + 2 - z = y * y - # nothing is executed yet, but the graph is being constructed - print(z) # now the entire graph is executed: z = (x+2) * (x+2) • We can do just in time compilation on the graph before execution

  57. Lazy Evaluation from torch.autograd import Variable x = Variable(torch.randn(1, 10)) prev_h = Variable(torch.randn(1, 20)) MM MM W_h = Variable(torch.randn(20, 20)) W_x = Variable(torch.randn(20, 10)) i2h = torch.mm(W_x, x.t()) Add h2h = torch.mm(W_h, prev_h.t()) next_h = i2h + h2h next_h = next_h.tanh() Tanh next_h.backward(torch.ones(1, 20))

  58. Lazy Evaluation from torch.autograd import Variable x = Variable(torch.randn(1, 10)) prev_h = Variable(torch.randn(1, 20)) MM MM W_h = Variable(torch.randn(20, 20)) W_x = Variable(torch.randn(20, 10)) i2h = torch.mm(W_x, x.t()) Add h2h = torch.mm(W_h, prev_h.t()) next_h = i2h + h2h Graph built but not actually executed next_h = next_h.tanh() Tanh

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Generation of political priority for global health initiatives: A research program, framework,

Generation of political priority for global health initiatives: A research program, framework,

Generation of political priority for global health initiatives: A research program, framework, and case study of maternal mortality* Jeremy Shiffman, Ph.D. Associate Professor of Public Administration The Maxwell School of Syracuse University

780 views • 45 slides
ICTs in education Moving from 1 Generation to 2 Generation models a framework for program

ICTs in education Moving from 1 Generation to 2 Generation models a framework for program

ICTs in education Moving from 1 Generation to 2 Generation models a framework for program success Gurumurthy K IT for Change, Bangalore February 2013 ICT Models in Business First generation computer applications in business - Simple

356 views • 8 slides
a European Framework for the generation of guarantees of origin for green hydrogen CertiHy

a European Framework for the generation of guarantees of origin for green hydrogen CertiHy

CertifHy Developing a European Framework for the generation of guarantees of origin for green hydrogen CertiHy event: October 19th 2016, Brussels Project co-finance by the FCH JU The research leading to these results has received under

895 views • 42 slides
Science Framework, And Next Generation Science Standards Context Education is US is a

Science Framework, And Next Generation Science Standards Context Education is US is a

Science Framework, And Next Generation Science Standards Context Education is US is a controlled at the state and local level. Common Core Math and Language Arts -- 47+ states choosing common standards Next Generation Science

534 views • 24 slides
A Framework for Automatic Generation A Framework for Automatic Generation of Configuration Files

A Framework for Automatic Generation A Framework for Automatic Generation of Configuration Files

A Framework for Automatic Generation A Framework for Automatic Generation of Configuration Files for a Custom of Configuration Files for a Custom Hardware/Software RTOS Hardware/Software RTOS Jaehwan Lee* Lee* Jaehwan Kyeong Keol Keol Ryu

702 views • 29 slides
A Reinforcement Learning Framework for Natural Question Generation using Bi-discriminators

A Reinforcement Learning Framework for Natural Question Generation using Bi-discriminators

A Reinforcement Learning Framework for Natural Question Generation using Bi-discriminators Presenter: Ji, Lu Zhihao Fan 1 , Zhongyu Wei 1 , Siyuan Wang 1 , Yang Liu 2 , Xuanjing Huang 3 1 School of Data Science, Fudan University, China 2

573 views • 22 slides
the EU framework programme for research and innovation DG Research and Innovation The

the EU framework programme for research and innovation DG Research and Innovation The

the EU framework programme for research and innovation DG Research and Innovation The Multiannual Financial Framework 2014-2020: Commissions proposals of 29 June 2011 Smart &amp; inclusive growth ( 491bn) 1. Horizon 2020 Education,

360 views • 15 slides
the EU framework programme for research and innovation The Multiannual Financial Framework

the EU framework programme for research and innovation The Multiannual Financial Framework

the EU framework programme for research and innovation The Multiannual Financial Framework 2014-2020: Commissions proposals of 29 June 2011 1. Smart &amp; inclusive growth (491bn) Horizon Education, Competitive Connecting Cohesion

639 views • 19 slides
MontiCore A Framework for DSL-Development Efficient Editor Generation for Compositional DSLs in

MontiCore A Framework for DSL-Development Efficient Editor Generation for Compositional DSLs in

MontiCore A Framework for DSL-Development Efficient Editor Generation for Compositional DSLs in Eclipse Holger Krahn, Bernhard Rumpe, Steven Vlkel Software Systems Engineering Technische Universitt Braunschweig http://www.sse-tubs.de/

265 views • 22 slides
and Use of Data Implementing the ROMA Cycle in the Next Generation Performance Management

and Use of Data Implementing the ROMA Cycle in the Next Generation Performance Management

Introduction to the Analysis and Use of Data Implementing the ROMA Cycle in the Next Generation Performance Management Framework ROMA Next Generation Video Series Barbara Mooney Frederick Richmond Courtney Kohler ROMA Next Generation

482 views • 34 slides
EEN 336 COMMUNICATION SYSTEMS Research Project Supervised By: Dr. Montasir Qasymeh By:

EEN 336 COMMUNICATION SYSTEMS Research Project Supervised By: Dr. Montasir Qasymeh By:

EEN 336 COMMUNICATION SYSTEMS Research Project Supervised By: Dr. Montasir Qasymeh By: Muhammad Obaidullah - 1030313 Overview Introduction 1 st Generation 2 nd Generation 3 rd Generation 4 th Generation The Future Conclusion

477 views • 18 slides
Column Generation Method Frdric Giroire FG Simplex 1/38 Column Generation in Two Words

Column Generation Method Frdric Giroire FG Simplex 1/38 Column Generation in Two Words

Column Generation Method Frdric Giroire FG Simplex 1/38 Column Generation in Two Words A framework to solve large problems. Main idea: not considering explicitly the whole set of variables. decompose the problem into a

610 views • 40 slides
CoCoSim, a code generation framework for control/command applications An overview of CoCoSim for

CoCoSim, a code generation framework for control/command applications An overview of CoCoSim for

CoCoSim, a code generation framework for control/command applications An overview of CoCoSim for multi-periodic discrete Simulink models Hamza Bourbouh, Pierre-Loc Garoche, Thomas Loquen, Eric Noulard and Claire Pagetti January 31 st 2020

395 views • 23 slides
Framework for a Next- Generation Accountability System 01 Accountability design components 02

Framework for a Next- Generation Accountability System 01 Accountability design components 02

Framework for a Next- Generation Accountability System 01 Accountability design components 02 Relative component accountability percentile 03 Criterion referenced component target AGENDA setting 04 Use of subgroup data for

557 views • 13 slides
Techno-economics for next generation fixed access networks Methodology and framework

Techno-economics for next generation fixed access networks Methodology and framework

Techno-economics for next generation fixed access networks Methodology and framework some key assumptions; Major results of the fixed broadband access evolution scenarios Author: Thomas Monath, T-Systems Enterprise Services GmbH

458 views • 23 slides
A New GDML Generation Framework KEVIN WIERMAN PNNL FNAL K. Wierman, LArSoft Geometry December

A New GDML Generation Framework KEVIN WIERMAN PNNL FNAL K. Wierman, LArSoft Geometry December

A New GDML Generation Framework KEVIN WIERMAN PNNL FNAL K. Wierman, LArSoft Geometry December 6, 2016 1 The Current GDML Generation Scheme Python based soluGons Perl-based GDML Script (uBooNE)

469 views • 11 slides
Convergent Perturbation Theory for the lattice 4 -model Aleksandr Ivanov 1 , Vasily Sazonov 2 ,

Convergent Perturbation Theory for the lattice 4 -model Aleksandr Ivanov 1 , Vasily Sazonov 2 ,

Convergent Perturbation Theory for the lattice 4 -model Aleksandr Ivanov 1 , Vasily Sazonov 2 , Vladimir Belokurov 1 , Eugeny Shavgulidze 1 1 Moscow State University, 2 University of Graz ivanov.as@physics.msu.ru July 15, 2015 Motivation We

389 views • 20 slides
DirectConnect: A Guide to A Direct Care Career Jennifer Rabalais, MA Project Director Institute

DirectConnect: A Guide to A Direct Care Career Jennifer Rabalais, MA Project Director Institute

DirectConnect: A Guide to A Direct Care Career Jennifer Rabalais, MA Project Director Institute on Disability University of New Hampshire DirectConnect 3 year, $2.9 million dollar grant funded by the Department of Labor Addresses

685 views • 31 slides
Lattice Flavour Physics N. Tantalo Rome University Tor Vergata and INFN sez. Tor

Lattice Flavour Physics N. Tantalo Rome University Tor Vergata and INFN sez. Tor

Lattice Flavour Physics N. Tantalo Rome University Tor Vergata and INFN sez. Tor Vergata 22-07-2011 lattice QCD errors in order to improve errors on hadronic matrix elements by using lattice techniques one has to pay (the currency

731 views • 23 slides
The Connected Mobility Global Forecast Quantifying future connected car services April 2016 -

The Connected Mobility Global Forecast Quantifying future connected car services April 2016 -

PTOLEMUS Consulting Group The Connected Mobility Global Forecast Quantifying future connected car services April 2016 - PTOLEMUS intellectual property PTOLEMUS in a nutshell The consulting &amp; research firm for the connected world Our

869 views • 19 slides
CIEL A universal execution engine for distributed data-flow computing Murray, Derek G., et al.

CIEL A universal execution engine for distributed data-flow computing Murray, Derek G., et al.

CIEL A universal execution engine for distributed data-flow computing Murray, Derek G., et al. [1] LSDPO (2017/2018) Paper Presentation Ioana Bica (ib354) Overview 1. Motivation and related work 2. CIELs contributions 3. Dynamic task

582 views • 25 slides
EXPECTATIONS FROM PK-PD MODELLING AND SIMULATION IN THE EVALUATION OF MEDICINES IN CHILDREN Pr

EXPECTATIONS FROM PK-PD MODELLING AND SIMULATION IN THE EVALUATION OF MEDICINES IN CHILDREN Pr

EMEA Workshop on modelling in paediatric medicines London April 14, 2008 EXPECTATIONS FROM PK-PD MODELLING AND SIMULATION IN THE EVALUATION OF MEDICINES IN CHILDREN Pr Grard PONS (MD, PhD), PDCO Vice Chair Paediatrician

521 views • 6 slides
University of Virginia Board of Visitors Richard Chait, Professor Emeritus, Harvard University

University of Virginia Board of Visitors Richard Chait, Professor Emeritus, Harvard University

University of Virginia Board of Visitors Richard Chait, Professor Emeritus, Harvard University August 3, 2018 Nine Steps toward a Good Start 1. Clarify mutual expectations. 2. Get the questions right. 3. Be patient. 4. Avoid individual pleas.

347 views • 15 slides
Investigation of MSWI mechanical properties based on the amount of reactive compounds PhD

Investigation of MSWI mechanical properties based on the amount of reactive compounds PhD

Investigation of MSWI mechanical properties based on the amount of reactive compounds PhD Student: V. Ca pra i Pro f.Dr. H.J.H. Bro uwe rs M.V.A F lo re a K . Sc ho llb a c h General overview I ntro duc tio n a nd mo tiva tio ns

453 views • 13 slides

More recommend