machine learning on a
play

Machine Learning on a Heterogeneous Cluster NaifTarafdar , Giuseppe - PowerPoint PPT Presentation

Oct 16, 2023 •228 likes •712 views

AIgean: An Open Framework for Machine Learning on a Heterogeneous Cluster NaifTarafdar , Giuseppe Di Guglielmo, Philip C Harris, Jeffrey D Krupa, Vladimir Loncar , Dylan S Rankin, Nhan Tran , Zhenbin Wu , Qianfeng Shen and

  1. AIgean: An Open Framework for Machine Learning on a Heterogeneous Cluster NaifTarafdar ¹ , Giuseppe Di Guglielmo², Philip C Harris³, Jeffrey D Krupa³, Vladimir Loncar ⁴ , Dylan S Rankin³, Nhan Tran ⁵ , Zhenbin Wu ⁶ , Qianfeng Shen¹ and Paul Chow¹ University of Toronto¹ Columbia University² Massachusetts Institute of Technology³ CERN ⁴ Fermilab ⁵ University of Illinois ⁶

  2. Take Aways • Galapagos: Platform for multi-FPGA application deployment – A scalable giant FPGA comprised of individual FPGAs • AIgean: Mapping an ML application onto the giant FPGA – Could also be your own applications • Depending on your area of expertise and interest you can use different parts of this project November 13, 2020 H2RC 2020

  3. Machine Learning • One of the most popular topics of research – In many areas, many applications (e.g medical, financial, safety, transportation etc.) – Also within the computing community • Wide usage in world pushes limits of devices – Metrics include performance and energy – Leading many researchers to consider heterogeneity! November 13, 2020 H2RC 2020

  4. Heterogeneity All Around Us This Photo by Unknown author is licensed under CC BY-NC. This Photo by Unknown author is licensed under CC BY-SA-NC. This Photo by Unknown author is licensed under CC BY-NC-ND. November 13, 2020 H2RC 2020

  5. Applying Machine Learning to a Heterogeneous Environment • Challenge: How do you design machine learning algorithms for a heterogenous space? – Hard enough with a homogenous computing environment – Is there a framework for such a thing? • Challenge: If such a framework exists can we get both flexibility and performance? November 13, 2020 H2RC 2020

  6. Outline • Brief Motivation • Overview of machine learning frameworks – Categorized as an abstraction layer stack • Overview of AIgean – HLS4ML – Galapagos • Results November 13, 2020 H2RC 2020

  7. MA MACH CHINE NE LEA EARNING RNING FR FRAM AMEW EWORKS ORKS November 13, 2020 H2RC 2020

  8. Many Popular Examples! • Such as – Tensorflow – PyTorch – Caffe – Intel DLA – Xilinx XfDNN • What do these different frameworks offer? – Depends on who you ask! November 13, 2020 H2RC 2020

  9. Machine Learning Stack Applications & Algorithms Cluster Deployment & Communication Hardware November 13, 2020 H2RC 2020

  10. Machine Learning Stack E.g: Neural net layers, Applications & Algorithms quantization, compression, pruning Cluster Deployment & Communication Hardware November 13, 2020 H2RC 2020

  11. Machine Learning Stack Applications & Algorithms E.g: Physical Connections Cluster Deployment & (PCIe, ethernet etc.), Communication Communication Protocols Hardware November 13, 2020 H2RC 2020

  12. Machine Learning Stack Applications & Algorithms Cluster Deployment & Communication E.g: Hardware circuit Hardware (multipliers, shifters), memory architecture (caching etc.) November 13, 2020 H2RC 2020

  13. Machine Learning Stack • Allows researchers to pick and choose layers Applications & Algorithms they wish to configure Cluster Deployment & Communication • Collapsable/Expandable Hardware for specific application and infrastructure! November 13, 2020 H2RC 2020

  14. AIG AIGEAN AN OVE VERVI VIEW November 13, 2020 H2RC 2020

  15. AIGean Introduction • Like the archipelago and sea • Combines two existing frameworks: – HLS4ML: • HLS IP cores of ML IP – Galapagos • Connects and deploys heterogeneous distributed application across multiple nodes November 13, 2020 H2RC 2020

  16. HLS4ML • Open source project • Input: – Description of FPGA resources • LUT, BRAM, DSP – Description of neural net • PyT orch, Keras, Onyx support • Output: – HLS synthesizable C++ that fits within resource constraints implementing neural net • Tunable HLS code, made to fit the FPGA November 13, 2020 H2RC 2020

  17. Galapagos User can define a FPGA cluster using • cluster description files and AXI-Stream VM kernels FPGA 1 File Network T ool Flow VM VM Kern Kern el el Kernel AXI-Stream AXI-Stream FPGA 2 FPGA 3 November 13, 2020 H2RC 2020

  18. Galapagos User can define a FPGA cluster using • cluster description files and AXI-Stream VM kernels FPGA 1 File Network T ool Flow VM VM Kern Kern el el Kernel AXI-Stream AXI-Stream FPGA 2 FPGA 3 November 13, 2020 H2RC 2020

  19. Galapagos User can define a FPGA cluster using • cluster description files and AXI-Stream VM kernels FPGA 1 File Network T ool Flow VM VM Kern Kern el el Kernel AXI-Stream AXI-Stream FPGA 2 FPGA 3 November 13, 2020 H2RC 2020

  20. Galapagos User can define a FPGA cluster using • cluster description files and AXI-Stream VM kernels FPGA 1 File Network T ool Flow VM VM Kern Kern el el Kernel AXI-Stream AXI-Stream FPGA 2 FPGA 3 November 13, 2020 H2RC 2020

  21. Galapagos Communication Layer • Heterogeneous Stack • Allows users to create flexible heterogeneous clusters Middleware/Network Layer across CPUs/FPGAs Seamlessly prototype by implementing both on CPU and • Hypervisor Layer FPGA Physical Hardware Galapagos ensures functional portability for network – communication Essentially "network-connected" HLS kernels – • For both SW and HW Iterative development, selectively move bottleneck from SW – to hardware without modifying code • Flexibly change communication protocol without modifying user application – TCP, UDP, L1 etc – User application is agnostic to this November 13, 2020 H2RC 2020

  22. Birth of AIgean • HLS4ML creates HLS IP core to maximize FPGA utilization • Galapagos can give a multi-FPGA fabric • Tools combined to deploy neural-net on multi- FPGA Fabric November 13, 2020 H2RC 2020

  23. AIgean Tool Flow Tuning Tunin CPU/FPGA IP Cluster Cluster g Not connected HLS Model Partitio Training ML Model Hls4 ner HDF5 & JSON Layers ml* C++ & TCL Keras, PyTorch AIGean Automated ML2G + Flow HLS ML to Galapag C++ & TCL os Bridge November 13, 2020 H2RC 2020

  24. AIgean Tool Flow Tuning CPU/FPGA IP Cluster Cluster Not connected HLS ML Model Layers Training Partitioner Model Hls4ml* HDF5 & JSON C++ & TCL Keras, PyTorch ML2G + HLS ML to Galapagos Bridge AIGean C++ & TCL Automated Flow November 13, 2020 H2RC 2020

  25. AIgean Tool Flow Tuning CPU/FPGA IP Cluster Cluster Not connected HLS ML Model Layers Training Partitioner Model Hls4ml* HDF5 & JSON C++ & TCL Keras, PyTorch ML2G + HLS ML to Galapagos Bridge AIGean C++ & TCL Automated Flow November 13, 2020 H2RC 2020

  26. HLS4ML Modifications • HLS4ML modified to create independent layers HLS ML Layers as separate HLS IP cores Hls4ml* C++ & TCL Model HDF5 & JSON – Each IP core is a streaming core with each stream per HLS ML to Galapagos Bridge dimension of the particular C++ & TCL layer November 13, 2020 H2RC 2020

  27. HLS4ML Galapagos Bridge • Bridges custom made for the layers used in the network (different bridges needed for different number of dimensions) • If the user has a different application layer then they would need a different bridge Galapagos Kernel ML Layer ML Layer Bridge Bridge … … One 512 bit stream One 512 bit stream one 8 bit stream per dim one 8 bit stream per dim November 13, 2020 H2RC 2020

  28. AIgean Tool Flow Tuning CPU/FPGA IP Cluster Cluster Not connected HLS ML Model Layers Training Partitioner Model Hls4ml* HDF5 & JSON C++ & TCL Keras, PyTorch ML2G + HLS ML to Galapagos Bridge AIGean C++ & TCL Automated Flow November 13, 2020 H2RC 2020

  29. Partitioner Partitioner separates IP cores onto • different FPGAs • Currently using IP resources IP Cluster Not connected estimation from HLS Place and HLS ML route and performing simple greedy Layers Partitioner approach C++ & TCL • Does not place the bridges as that is AIgean specific, and this partitioner is general for all Galapagos IP kernels November 13, 2020 H2RC 2020

  30. AIgean Tool Flow Tuning CPU/FPGA IP Cluster Cluster Not connected HLS ML Model Layers Training Partitioner Model Hls4ml* HDF5 & JSON C++ & TCL Keras, PyTorch ML2G + HLS ML to Galapagos Bridge AIGean C++ & TCL Automated Flow November 13, 2020 H2RC 2020

  31. Machine Learning to Galapagos (ML2G) • Adds the appropriate bridges on the interfaces IP Cluster CPU/FPGA Not connected Cluster of the FPGAs • Creates the local ML2G connections for kernels + on the same FPGA HLS ML to Galapagos Bridge C++ & TCL November 13, 2020 H2RC 2020

  32. RESU SULTS November 13, 2020 H2RC 2020

  33. Experiment Setup • CPUs – Xeon E5-2650 • 24 Cores at 2.2 GHz • FPGAs – Fidus Sidewinder • ZU19EG FPGA – ~1 Million logic cells, 35 MB BRAM, 1968 DSP slices • 100 GB network interface – 100 GB UDP core November 13, 2020 H2RC 2020

  34. Microbenchmarks Link Latency Throughput • Latency send Software to 0.029 ms 0.244 GB/s single flit Hardware • Throughput: Hardware to 0.00017 ms 100 GB/s Hardware maximum Hardware to 0.0203 ms N/A throughput of Software link (varying packet size for software) November 13, 2020 H2RC 2020

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

An Exercise in An Exercise in Machine Learning Machine Learning

An Exercise in An Exercise in Machine Learning Machine Learning

An Exercise in An Exercise in Machine Learning Machine Learning http://www.cs.iastate.edu/~cs573x/bbsilab.html Machine Learning Software Preparing Data Building Classifiers Interpreting Results Machine Learning Software

496 views • 26 slides
Machine Learning By Alex Scarlatos What is Machine Learning? Machine Learning is the process by

Machine Learning By Alex Scarlatos What is Machine Learning? Machine Learning is the process by

Machine Learning By Alex Scarlatos What is Machine Learning? Machine Learning is the process by which computers can be trained through observation, rather than being explicitly programmed. Basically, a program is given input and adjusts its

556 views • 17 slides
Machine Learning: Study of algorithms that improve their performance P at some task T

Machine Learning: Study of algorithms that improve their performance P at some task T

Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University January 12, 2015 Today: Readings: What is machine learning? The Discipline of ML Decision tree learning Mitchell, Chapter 3

872 views • 29 slides
Traditional Machine Learning: Unsupervised Learning Juhan Nam Traditional Machine Learning

Traditional Machine Learning: Unsupervised Learning Juhan Nam Traditional Machine Learning

GCT634/AI613: Musical Applications of Machine Learning (Fall 2020) Traditional Machine Learning: Unsupervised Learning Juhan Nam Traditional Machine Learning Pipeline in Classification Tasks A set of hand-designed audio features are selected

398 views • 26 slides
CS 335 Machine Learning What is Machine Learning? Dan Sheldon Spring 2019 What is Machine

CS 335 Machine Learning What is Machine Learning? Dan Sheldon Spring 2019 What is Machine

1/22/19 CS 335 Machine Learning What is Machine Learning? Dan Sheldon Spring 2019 What is Machine Learning? A Simple Task: Recognize Obama How do you program a computer to Recognize faces? Recommend movies? Decide which web

581 views • 5 slides
Machine Learning Machine Learning: algorithms that use experience to improve their

Machine Learning Machine Learning: algorithms that use experience to improve their

Machine Learning Machine Learning: algorithms that use experience to improve their performance We use machine learning in situations where it is very challenging (or impossible) to define the rules by hand: e.g. face detection

1.04k views • 28 slides
Introduction to Machine Learning COMPSCI 371D Machine Learning COMPSCI 371D Machine

Introduction to Machine Learning COMPSCI 371D Machine Learning COMPSCI 371D Machine

Introduction to Machine Learning COMPSCI 371D Machine Learning COMPSCI 371D Machine Learning Introduction to Machine Learning 1 / 12 Outline 1 Nearest Neighbor Prediction 2 Complexity Considerations 3 The Voronoi Diagram 4 Overfitting

564 views • 12 slides
1 Why Study Machine Learning? Why Study Machine Learning? Cognitive Science The Time is Ripe

1 Why Study Machine Learning? Why Study Machine Learning? Cognitive Science The Time is Ripe

What is Learning? Herbert Simon: Learning is any process by which a system improves performance from CS 391L: Machine Learning experience. Introduction What is the task? Classification Problem solving / planning /

603 views • 6 slides
MACHINE LEARNING, STATISTICAL LEARNING AND PARALLEL COMPUTING INTRODUCTION VS MACHINE LEARNING

MACHINE LEARNING, STATISTICAL LEARNING AND PARALLEL COMPUTING INTRODUCTION VS MACHINE LEARNING

JACOPO DI IORIO, MATTEO FONTANA, DANIELE OCCHIUTO, CLAUDIA VOLPETTI MACHINE LEARNING, STATISTICAL LEARNING AND PARALLEL COMPUTING INTRODUCTION VS MACHINE LEARNING STATISTICAL LEARNING Separate evolutions, but with shared methodologies MERGE

565 views • 55 slides
Apache PredictionIO End-to-End Machine Learning Server with Apache Spark What is Machine

Apache PredictionIO End-to-End Machine Learning Server with Apache Spark What is Machine

Apache PredictionIO End-to-End Machine Learning Server with Apache Spark What is Machine Learning? What is Machine Learning? Examples of machine learning applications: Facial recognition software E-mail spam detectors Automated

302 views • 17 slides
Machine Learning 11 AI Slides (6e) c Lin Zuoquan@PKU 1998-2020 11 1 11 Machine Learning

Machine Learning 11 AI Slides (6e) c Lin Zuoquan@PKU 1998-2020 11 1 11 Machine Learning

Machine Learning 11 AI Slides (6e) c Lin Zuoquan@PKU 1998-2020 11 1 11 Machine Learning 11.1 Learning agents 11.2 Inductive learning 11.3 Deep learning 11.4 Statistical learning 11.5 Reinforcement learning 11.6 Transfer learning

1.7k views • 159 slides
Deep Learning: Intro Juhan Nam Review of Traditional Machine Learning The traditional machine

Deep Learning: Intro Juhan Nam Review of Traditional Machine Learning The traditional machine

GCT634/AI613: Musical Applications of Machine Learning (Fall 2020) Deep Learning: Intro Juhan Nam Review of Traditional Machine Learning The traditional machine learning pipeline Temporal Frame-level Unsupervised Classifier Summary

613 views • 27 slides
Machine Learning for Auto Optimization What is Machine Learning? Definition: Machine

Machine Learning for Auto Optimization What is Machine Learning? Definition: Machine

Machine Learning for Auto Optimization What is Machine Learning? Definition: Machine learning refers to any system where the performance of a machine in performing a task improves by gaining more experience in performing that task .

394 views • 27 slides
MACHINE LEARNING kernels 1 MACHINE LEARNING 2012 MACHINE LEARNING Kernels: Intuition How

MACHINE LEARNING kernels 1 MACHINE LEARNING 2012 MACHINE LEARNING Kernels: Intuition How

MACHINE LEARNING 2012 MACHINE LEARNING MACHINE LEARNING kernels 1 MACHINE LEARNING 2012 MACHINE LEARNING Kernels: Intuition How to separate the red class from the grey class? x 2 360 r x 1 Polar coordinates Data

1.04k views • 44 slides
BBM406 Fundamentals of Machine Learning Lecture 2: Machine Learning by Examples, Nearest

BBM406 Fundamentals of Machine Learning Lecture 2: Machine Learning by Examples, Nearest

photo:@rewardyfahmi // Unsplash BBM406 Fundamentals of Machine Learning Lecture 2: Machine Learning by Examples, Nearest Neighbor Classifier Aykut Erdem // Hacettepe University // Fall 2019 When Do We Use Machine Learning? ML is

1.33k views • 110 slides
Machine Learning Modeling and Learning 15-110 Monday 4/13 Learning Goals Given a

Machine Learning Modeling and Learning 15-110 Monday 4/13 Learning Goals Given a

Machine Learning Modeling and Learning 15-110 Monday 4/13 Learning Goals Given a dataset, identify features which may help predict information about the data Identify common types of machine learning algorithms 2 Machine

553 views • 27 slides
SoC Design SoC Design g L Lecture Lecture 6: IP Cores 6 IP C : IP Cores IP C Shaahin

SoC Design SoC Design g L Lecture Lecture 6: IP Cores 6 IP C : IP Cores IP C Shaahin

SoC Design SoC Design g L Lecture Lecture 6: IP Cores 6 IP C : IP Cores IP C Shaahin Hessabi Shaahin Hessabi Department of Computer Engineering Department of Computer Engineering Sharif University of Technology Sharif University of

328 views • 28 slides
On-hardware debugging of IP cores with free tools Anton Kuzmin 2020-02-02 1 / 12 Intro Why

On-hardware debugging of IP cores with free tools Anton Kuzmin 2020-02-02 1 / 12 Intro Why

Intro Why FPGA (and what the FPGA is all about) Software approach simulation first Going to the hardware Whats next Contact info On-hardware debugging of IP cores with free tools Anton Kuzmin 2020-02-02 1 / 12 Intro Why FPGA (and

770 views • 12 slides
FROM HARDWARE TO 0-DAY Or how to buy a security camera and never use it for its intended purpose

FROM HARDWARE TO 0-DAY Or how to buy a security camera and never use it for its intended purpose

FROM HARDWARE TO 0-DAY Or how to buy a security camera and never use it for its intended purpose PIETRO OLIVA WHOAMI Name: Pietro Oliva Twitter: @0xsysenter Current role: Security researcher (R3) Previous roles: Security

795 views • 30 slides
Securing Industrial IoT Device Attestation, Software Updates, and Data Protection Mauro Conti ,

Securing Industrial IoT Device Attestation, Software Updates, and Data Protection Mauro Conti ,

Securing Industrial IoT Device Attestation, Software Updates, and Data Protection Mauro Conti , University of Padua Slides prepared with the support of Daniele Lain and Moreno Ambrosin SCy-Phy Systems Week 2017 Panel IV: Defences June 6, 2017,

794 views • 25 slides
Integrating OmpSs@FPGA within Eclipse Presentation for EclipseCon 2019 Ruben Cano-Daz and

Integrating OmpSs@FPGA within Eclipse Presentation for EclipseCon 2019 Ruben Cano-Daz and

Integrating OmpSs@FPGA within Eclipse Presentation for EclipseCon 2019 Ruben Cano-Daz and Xavier Martorell Barcelona Supercomputing Center The LEGaTO project has received funding from the European Union's Horizon 2020 research and innovation

665 views • 20 slides
IPBus framework on the zcu102 build instructions updated on 27 July 2019 Kristian Harder

IPBus framework on the zcu102 build instructions updated on 27 July 2019 Kristian Harder

IPBus framework on the zcu102 build instructions updated on 27 July 2019 Kristian Harder (RAL), 18 Oct 2018 1 IPbus concept IPbus protocol: simple packet-based control protocol for reading and modifying memory-mapped resources within

147 views • 11 slides
Memories Introduction Why do we need memory in an FPGA Device? Topics Types of FPGA

Memories Introduction Why do we need memory in an FPGA Device? Topics Types of FPGA

Memories Introduction Why do we need memory in an FPGA Device? Topics Types of FPGA Memories Available Resources Realizing memories in an HDL design Applications Examples Extra: Memory Controller Block Memories

692 views • 10 slides
1 Recovery From Packet Loss Loss is in a broader sense: packet never arrives or arrives

1 Recovery From Packet Loss Loss is in a broader sense: packet never arrives or arrives

Multimedia Networking Last time Principles Classify multimedia Multimedia Networking Applications applications Streaming stored audio and video Identify the network Today services the apps Real-time Multimedia: Internet

326 views • 17 slides

More recommend