Neural Network Overlay Using FPGA DSP Blocks Lenos Ioannou and - PowerPoint PPT Presentation

Nov 10, 2022 •26 likes •96 views

Neural Network Overlay Using FPGA DSP Blocks Lenos Ioannou and Suhaib A. Fahmy School of Engineering, University of Warwick, UK Introduction Long back-end tool compilation hinders rapid deployment of Neural Networks on FPGAs at the edge

Neural Network Overlay Using FPGA DSP Blocks Lenos Ioannou and Suhaib A. Fahmy School of Engineering, University of Warwick, UK
Introduction • Long back-end tool compilation hinders rapid deployment of Neural Networks on FPGAs at the edge • Use of overlays to build abstractions on top of the FPGA • Effectively enabling rapid deployment • Core NN operation, multiply-accumulate, maps well to DSP Blocks • Most FPGA NN implementations operate sub-max frequencies [1] • Can be solved by optimising the overlay around the DSP blocks [3]
Neural Network Test Cases • Trained 3 NNs using Tensorflow [2], each one comprises four layers • Use of ReLU in the intermediate layers • Considering the input bit-widths of the DSP48E2: • 18 bit weights • 27 bit inputs • 48 bit biases
Overlay • Each neuron is mapped to a single DSP block • DSP blocks alternate between two opmodes • Serial data flow • Needs to stall when # neurons > # inputs • Adjustable latency
Implementation Results • Implemented the overlay targeting the Zynq Ultrascale+ ZU7EV • Maintains low resource utilization • Feedforward serial data flow is highly efficient • High operating frequency • Near the DSP blocks’ theoretical maximum
Conclusion • Not offering peak performance in a particular NN implementation • Contribute to the more rapid deployment of NNs on FPGAs at the edge • Prioritise low resource utilization and energy efficiency Future work • Implement a mechanism to handle the data flow and stall accordingly • Expand the overlay for deeper topologies • Integration with a rapid compiler flow
References [1] E. Wu, X. Zhang, D. Berman, and I. Cho, “A high-throughput reconfigurable processing array for neural networks,” in Int. Conference on Field Programmable Logic and Applications (FPL), Sep. 2017. [2] Martin Abadi et al. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. [3] A. K. Jain, D. L. Maskell, and S. A. Fahmy, “Throughput oriented FPGA overlays using DSP blocks,” in 2016 Design, Automation Test in Europe Conference Exhibition (DATE), March 2016, pp. 1628–1633. [4] A. K. Jain, X. Li, P. Singhai, D. L. Maskell, and S. A. Fahmy, “DeCO: A DSP block based FPGA accelerator overlay with low overhead interconnect,” in Proc. Int. Symposium on Field-Programmable Custom Computing Machines (FCCM), 2016, pp. 1–8.

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

An introduction to FPGA-based acceleration of neural networks Marco Pagani 1 What is an FPGA?

An introduction to FPGA-based acceleration of neural networks Marco Pagani 1 What is an FPGA? Field-programmable gate array (FPGA) are integrated circuits designed to be con fi gured after manufacturing for implementing arbitrary logic

429 views • 15 slides

CS5412: OVERLAY NETWORKS Lecture IV Ken Birman Overlay Networks 2 We use the term overlay

CS5412 Spring 2012 (Cloud Computing: Birman) 1 CS5412: OVERLAY NETWORKS Lecture IV Ken Birman Overlay Networks 2 We use the term overlay network when one network (or a network-like data structure) is superimposed upon an underlying

793 views • 57 slides

FPGA-based Training Accelerator Utilizing Sparseness of Convolutional Neural Network Hiroki

FPGA-based Training Accelerator Utilizing Sparseness of Convolutional Neural Network Hiroki Nakahara, Youki Sada, Masayuki Shimoda, Akira Jinguji, Shimpei Sato Tokyo Institute of Technology, JP FPL2019 @Barcelona Challenges in DL Training

369 views • 10 slides

De DeCO: : A DS DSP Block Based FPGA Accelerator Overlay Wi With Low Overhead Interconnect Ab

De DeCO: : A DS DSP Block Based FPGA Accelerator Overlay Wi With Low Overhead Interconnect Ab Abhishek Kumar Ja Jain, Xiangwei Li, Pranjul Singhai, Douglas L. Maskell School of Computer Science and Engineering Nanyang Technological

564 views • 39 slides

Network Intrusion Detection Using Neural Networks on FPGA SoCs Lenos Ioannou and Suhaib A. Fahmy

Network Intrusion Detection Using Neural Networks on FPGA SoCs Lenos Ioannou and Suhaib A. Fahmy School of Engineering, University of Warwick, UK Introduction Mainstream approaches in intrusion detection do not scale well to the embedded

936 views • 7 slides

FPGA-based Convolutional Neural Network Accelerator Ke Xu Xingyu Hou Manqi Yang Wenqi Jiang

FPGA-based Convolutional Neural Network Accelerator Ke Xu Xingyu Hou Manqi Yang Wenqi Jiang Outline Background Software Implementation Python / C implementation of VGG-16 Profiling and acceleration strategy

663 views • 20 slides

Contents Introduction Pipelined FPGA DNN accelerators Roof-line Model and optimizing

FPGA implementation of Quantized (Deep) Neural Network (QNN) Accelerators Biruk Seyoum Phd Student, Scoula Superiore SantAnna, Pisa Contents Introduction Pipelined FPGA DNN accelerators Roof-line Model and optimizing FPGA DNN

587 views • 23 slides

Tinsel: a manythread overlay for FPGA clusters POETS Project (EPSRC) Matthew Naylor, David

Tinsel: a manythread overlay for FPGA clusters POETS Project (EPSRC) Matthew Naylor, David Thomas, Simon Moore, Imperial College London University of Cambridge New compute devices allow ever-larger problems to be solved. But theres

782 views • 32 slides

A Time-Multiplexed FPGA Overlay with Linear Interconnect Xiangwei Li , Douglas L. Maskell

A Time-Multiplexed FPGA Overlay with Linear Interconnect Xiangwei Li , Douglas L. Maskell Abhishek K. Jain, Suhaib A. Fahmy, School of Computer Lawrence Livermore School of Engineering Science and Engineering National Laboratory University

531 views • 27 slides

CompSci 356: Computer Network Architectures Lecture 24: Overlay Networks Chap 9.4 Xiaowei Yang

CompSci 356: Computer Network Architectures Lecture 24: Overlay Networks Chap 9.4 Xiaowei Yang xwy@cs.duke.edu Overview What is an overlay network? Examples of overlay networks End system multicast Unstructured Gnutella,

972 views • 61 slides

Processor Architecture Charles Eric LaForest J. Gregory Steffan ECE, University of Toronto FPGA

Octavo: An FPGA-Centric Processor Architecture Charles Eric LaForest J. Gregory Steffan ECE, University of Toronto FPGA 2012, February 24 Easier FPGA Programming We focus on overlay architectures Nios, MicroBlaze, Vector Processors

519 views • 38 slides

Peer-to-peer overlay networks Structured overlay network: Chord Basics Issue Large-scale

Peer-to-peer overlay networks Structured overlay network: Chord Basics Issue Large-scale distributed computer systems are spread across Each peer is assigned a unique m -bit identifier id . the Internet, yet their constituents need to

229 views • 3 slides

Feasibility of ILA as Network Virtualisation Overlay in multi-domain, multi-tenant Cloud Tako

Feasibility of ILA as Network Virtualisation Overlay in multi-domain, multi-tenant Cloud Tako Marks Bsc. 04-07-2017 System and Network Engineering University of Amsterdam 1 Introduction A Network Virtualisation Overlay (NVO) is a collection

614 views • 29 slides

L2 Overlay L2 Overlay

L2 Overlay L2 Overlay July 30, 2019 A. Gerrard, What Is Kubernetes?

536 views • 24 slides

Neural Information Retrieval Wassila Lalouani 1 Plan Neural network architectures Neural

Neural Information Retrieval Wassila Lalouani 1 Plan Neural network architectures Neural IR tasks Neural IR architecture Feature Representations Neural IR query auto completion Neural IR query suggestion Neural IR document

1.48k views • 18 slides

P4, FPGA, HLS Domain-specific language - network data forwarding Standardized and

P4, FPGA, HLS Domain-specific language - network data forwarding Standardized and maintained by P4 Language Consortium @ p4.org Independent of network protocols (VLAN, VxLAN, INT, ) and target architectures (ASIC, FPGA, NPU,

343 views • 9 slides

Low Power Design Prof. Dr. J. Henkel CES - Chair for Embedded Systems KIT, Germany III. Battery

Low Power Design Prof. Dr. J. Henkel CES - Chair for Embedded Systems KIT, Germany III. Battery Modeling Prof. Jrg Henkel, Low Power Design, SS2014 ces.itec.kit.edu 2 Course overview: topics Components

981 views • 34 slides

Secure Resource Sharing for Embedded Protected Module Architectures Jo Van Bulck imec-DistriNet,

Secure Resource Sharing for Embedded Protected Module Architectures Jo Van Bulck imec-DistriNet, KU Leuven, Celestijnenlaan 200A, B-3001 Belgium MSc Thesis Presentation BELCLIV-CLUSIB, April 21, 2017 Jo Van Bulck Secure Resource Sharing for

515 views • 29 slides

Designing asynchronous circuits with timing conditions Vision statement for possible CAD

Designing asynchronous circuits with timing conditions Vision statement for possible CAD development under Workcraft Original document: A. Yakovlev, Synthesis from timing diagrams: rough notes and examples, Tech Memo, March 7, 1998; written on

683 views • 19 slides

HW/SW Codesign w/ FPGAs The Nature of HW/SW I ECE 522 Hardware Software Codesign with FPGAs

HW/SW Codesign w/ FPGAs The Nature of HW/SW I ECE 522 Hardware Software Codesign with FPGAs Instructor: Prof. Jim Plusquellic Text: A Practical Introduction to Hardware/Software Codesign, 2nd Edition", Patrick Schaumont, Springer,

649 views • 10 slides

Design for Testability Maurcio F. Aniche M.F.Aniche@tudelft.nl Controllability and

Design for Testability Maurcio F. Aniche M.F.Aniche@tudelft.nl Controllability and Observability Controllability determines the work it takes to set up and run test cases and the extent to which individual functions and features of the

268 views • 16 slides

TDDE45 - Lecture 7: Testability Martin Sjlund Department of Computer and Information Science

TDDE45 - Lecture 7: Testability Martin Sjlund Department of Computer and Information Science Linkping University 2020-10-06 Part I Testing What is testing? This course is not a course primarily in Software Testing (TDDD04 HT1).

310 views • 19 slides

UNIT TESTING 3 / 8 1 / 8 Unit testing involves: Lots of small, independent tests Reporting

UNIT TESTING 3 / 8 1 / 8 Unit testing involves: Lots of small, independent tests Reporting passes, failures, and errors Some optional setup and teardown shared across tests Aggregation (combining tests into test suites) 3 / 8 2 / 8 WHY

89 views • 8 slides

Specification-Based Testing 1 Stuart Anderson Stuart Anderson Specification-Based Testing 1

Specification-Based Testing 1 Stuart Anderson Stuart Anderson Specification-Based Testing 1 2011 c 1 Overview Basic terminology A view of faults through failure Systematic versus randomised testing A systematic approach to

492 views • 32 slides