c l b l a s t a t u n e d b l a s l i b r a r y
play

C L B l a s t : A T u n e d B L A S L i b r a - PowerPoint PPT Presentation

Nov 16, 2023 •270 likes •717 views

C L B l a s t : A T u n e d B L A S L i b r a r y f o r F a s t e r D e e p L e a r n i n g C e d r i c N u g t e r e n M a y 1 1 , 2 0 1 7 h t t p : / / g i t h u b

  1. C L B l a s t : A T u n e d B L A S L i b r a r y f o r F a s t e r D e e p L e a r n i n g C e d r i c N u g t e r e n M a y 1 1 , 2 0 1 7 h t t p : / / g i t h u b . c o m / c n u g t e r e n / c l b l a s t h t t p : / / c n u g t e r e n . g i t h u b . i o / c l b l a s t

  2. T h e H e a r t o f D e e p L e a r n i n g C e d r i c N u g t e r e n , T o m T o m C L B l a s t : T u n e d O p e n C L B L A S S l i d e 2 o u t o f 4 3

  3. G E M M i s a t t h e H e a r t o f D e e p L e a r n i n g C e d r i c N u g t e r e n , T o m T o m C L B l a s t : T u n e d O p e n C L B L A S S l i d e 3 o u t o f 4 3

  4. S o w h e r e a r e t h e M a t r i x - M u l t i p l i c a t i o n s ? C e d r i c N u g t e r e n , T o m T o m C L B l a s t : T u n e d O p e n C L B L A S S l i d e 4 o u t o f 4 3

  5. C o n v o l u t i o n s a s M a t r i x M u l t i p l i c a t i o n C e d r i c N u g t e r e n , T o m T o m C L B l a s t : T u n e d O p e n C L B L A S S l i d e 5 o u t o f 4 3

  6. G E M M i s t h e H e a r t o f D e e p L e a r n i n g C e d r i c N u g t e r e n , T o m T o m C L B l a s t : T u n e d O p e n C L B L A S S l i d e 6 o u t o f 4 3

  7. D o e s e v e r y o n e a g r e e ? C e d r i c N u g t e r e n , T o m T o m C L B l a s t : T u n e d O p e n C L B L A S S l i d e 7 o u t o f 4 3

  8. D o e s e v e r y o n e a g r e e ? C e d r i c N u g t e r e n , T o m T o m C L B l a s t : T u n e d O p e n C L B L A S S l i d e 8 o u t o f 4 3

  9. S t i l l t r u e i n 2 0 1 7 ! C e d r i c N u g t e r e n , T o m T o m C L B l a s t : T u n e d O p e n C L B L A S S l i d e 9 o u t o f 4 3

  10. B u t w h y a n e w B L A S L i b r a r y ? ● N V I D I A ’ s c u B L A S i s g r e a t , o r i s i t ? C e d r i c N u g t e r e n , T o m T o m C L B l a s t : T u n e d O p e n C L B L A S S l i d e 1 0 o u t o f 4 3

  11. B u t w h y a n e w B L A S L i b r a r y ? ● N V I D I A ’ s c u B L A S i s g r e a t , o r i s i t ? C e d r i c N u g t e r e n , T o m T o m C L B l a s t : T u n e d O p e n C L B L A S S l i d e 1 1 o u t o f 4 3

  12. B u t w h y a n e w B L A S L i b r a r y ? ● N V I D I A ’ s c u B L A S i s g r e a t , o r i s i t ? – N o t p o r t a b l e , n o t c u s t o m i s a b l e , n o t o p e n - s o u r c e , . . . C e d r i c N u g t e r e n , T o m T o m C L B l a s t : T u n e d O p e n C L B L A S S l i d e 1 2 o u t o f 4 3

  13. B u t w h y a n e w B L A S L i b r a r y ? ● N V I D I A ’ s c u B L A S i s g r e a t , o r i s i t ? – N o t p o r t a b l e , n o t c u s t o m i s a b l e , n o t o p e n - s o u r c e , . . . ● I s A M D ’ s c l B L A S g r e a t ? – N o t p e r f o r m a n c e p o r t a b l e , n o t w e l l e n g i n e e r e d , l a c k o f n e w f e a t u r e s , . . . C e d r i c N u g t e r e n , T o m T o m C L B l a s t : T u n e d O p e n C L B L A S S l i d e 1 3 o u t o f 4 3

  14. I n t r o d u c i n g C L B l a s t ● C L B l a s t : M o d e r n C + + 1 1 O p e n C L B L A S l i b r a r y ● I m p l e m e n t s a l l B L A S r o u t i n e s f o r a l l p r e c i s i o n s ( S , D , C , Z ) ● A c c e l e r a t e s a l l k i n d s o f a p p l i c a t i o n s : – F l u i d d y n a m i c s , q u a n t u m c h e m i s t r y , l i n e a r a l g e b r a , e t c . – T o d a y ’ s f o c u s : d e e p l e a r n i n g C e d r i c N u g t e r e n , T o m T o m C L B l a s t : T u n e d O p e n C L B L A S S l i d e 1 4 o u t o f 4 3

  15. I n t r o d u c i n g C L B l a s t ● C L B l a s t : M o d e r n C + + 1 1 O p e n C L B L A S l i b r a r y ● I m p l e m e n t s a l l B L A S r o u t i n e s f o r a l l p r e c i s i o n s ( S , D , C , Z ) ● A c c e l e r a t e s a l l k i n d s o f a p p l i c a t i o n s : – F l u i d d y n a m i c s , q u a n t u m c h e m i s t r y , l i n e a r a l g e b r a , e t c . – T o d a y ’ s f o c u s : d e e p l e a r n i n g ● A l r e a d y i n t e g r a t e d i n t o v a r i o u s p r o j e c t s : – J O C L B l a s t ( J a v a b i n d i n g s ) – A r r a y F i r e ( G P U a c c e l e r a t e d l i b r a r y a n d a p p l i c a t i o n s ) – O p e n C L f o r k o f C a f e ( g i t h u b . c o m / d i v i d i t i / c k - c a f e ) – O p e n C L f o r k o f T F ( g i t h u b . c o m / h u g h p e r k i n s / t e n s o r l o w - c l ) C e d r i c N u g t e r e n , T o m T o m C L B l a s t : T u n e d O p e n C L B L A S S l i d e 1 5 o u t o f 4 3

  16. I n t r o d u c i n g C L B l a s t c o m m u n i t y a c t i v i t y C I a n d e x t e n s i v e t e s t i n g C e d r i c N u g t e r e n , T o m T o m C L B l a s t : T u n e d O p e n C L B L A S S l i d e 1 6 o u t o f 4 3

  17. B u t … i s i t f a s t ? ● A l l k e r n e l s a r e g e n e r i c a n d t u n a b l e t h a n k s t o i n t e g r a t i o n o f t h e C L T u n e a u t o - t u n e r ( p r e s e n t e d a t l a s t y e a r ’ s G T C ) C e d r i c N u g t e r e n , T o m T o m C L B l a s t : T u n e d O p e n C L B L A S S l i d e 1 7 o u t o f 4 3

  18. B u t … i s i t f a s t ? ● A l l k e r n e l s a r e g e n e r i c a n d t u n a b l e t h a n k s t o i n t e g r a t i o n o f t h e C L T u n e a u t o - t u n e r ( p r e s e n t e d a t l a s t y e a r ’ s G T C ) C e d r i c N u g t e r e n , T o m T o m C L B l a s t : T u n e d O p e n C L B L A S S l i d e 1 8 o u t o f 4 3

  19. B u t … i s i t f a s t ? ● A l l k e r n e l s a r e g e n e r i c a n d t u n a b l e t h a n k s t o i n t e g r a t i o n o f t h e C L T u n e a u t o - t u n e r ( p r e s e n t e d a t l a s t y e a r ’ s G T C ) ● T u n e d o u t - o f - t h e - b o x f o r 4 0 c o m m o n d e v i c e s – F o r n e w d e v i c e s : r u n t h e a u t o - t u n e r w h e n i n s t a l l i n g C L B l a s t C e d r i c N u g t e r e n , T o m T o m C L B l a s t : T u n e d O p e n C L B L A S S l i d e 1 9 o u t o f 4 3

  20. C L B l a s t B e n c h m a r k R e s u l t s A X P Y A X P Y G E M V o d d r e g u l a r r e g u l a r ( i n G B / s ) ( i n G B / s ) ( i n G B / s ) G E M M G E M V G E M M r e g u l a r o d d o d d ( i n G B / s ) ( i n G F L O P S ) ( i n G F L O P S ) ● H i g h e r i s b e t t e r ● M o r e r e s u l t s a t h t t p : / / c n u g t e r e n . g i t h u b . i o / c l b l a s t C e d r i c N u g t e r e n , T o m T o m C L B l a s t : T u n e d O p e n C L B L A S S l i d e 2 0 o u t o f 4 3

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

& Managed Services Cognitive Computing Prescriptive What should happen..! Predictive What

& Managed Services Cognitive Computing Prescriptive What should happen..! Predictive What

Value Capture in Adaptive Workflows & Behavior Change Decision-making in Age of & Managed Services Cognitive Computing Prescriptive What should happen..! Predictive What might happen.. Descriptive What happened .. Human +

353 views • 33 slides
and Other Snakes in the Grass Diane Mueller dianem@activestate.com Trent Mick

and Other Snakes in the Grass Diane Mueller dianem@activestate.com Trent Mick

Python & Finance: US Gov Mandates, Financial Modeling, and Other Snakes in the Grass Diane Mueller dianem@activestate.com Trent Mick trentm@activestate.com Premise Open Govt Data needs Open Tools Tool Vendors need

361 views • 35 slides
Budgeting TEAM WIN-MART Budgeting A master budget is a comprehensive projection of how

Budgeting TEAM WIN-MART Budgeting A master budget is a comprehensive projection of how

Budgeting TEAM WIN-MART Budgeting A master budget is a comprehensive projection of how management expects to conduct all aspects of business over the budget period, usually a fiscal year. The master budget summarizes projected activity by way

447 views • 32 slides
MeetBSD.ir BSD

MeetBSD.ir BSD

MeetBSD.ir BSD list of contents Introduction of speakers History of BSD Kernel models

571 views • 32 slides
Alpha Presentation Change Management Software The Capstone Experience Team Google Benjamin

Alpha Presentation Change Management Software The Capstone Experience Team Google Benjamin

Alpha Presentation Change Management Software The Capstone Experience Team Google Benjamin Bonnen Albert Guo Nick Vogt Department of Computer Science and Engineering Michigan State University Spring 2014 From Students to Professionals

310 views • 10 slides
Agile Software Development Cost Modeling for the US DoD Wilson Rosa, Naval Center for Cost

Agile Software Development Cost Modeling for the US DoD Wilson Rosa, Naval Center for Cost

Agile Software Development Cost Modeling for the US DoD Wilson Rosa, Naval Center for Cost Analysis Ray Madachy, Naval Postgraduate School Bradford Clark, Software Metrics, Inc. Barry Boehm, University of Southern California SEI Software and

806 views • 31 slides
BioBrick-A-Bot: Lego Robot for Automated BioBrick DNA Assembly Gabriel See University of

BioBrick-A-Bot: Lego Robot for Automated BioBrick DNA Assembly Gabriel See University of

BioBrick-A-Bot: Lego Robot for Automated BioBrick DNA Assembly Gabriel See University of Washington Software Team 1 Nov 2009, at MIT 1 Nov 2009 iGEM Team Washington-Software 2009 1 Our Project Automated BioBrick DNA Assembly + + + Lego

555 views • 30 slides
Presentation created by Sanjay S Manvatkar Asst.Prof. BJS College Pune 6. Software Testing

Presentation created by Sanjay S Manvatkar Asst.Prof. BJS College Pune 6. Software Testing

Presentation created by Sanjay S Manvatkar Asst.Prof. BJS College Pune 6. Software Testing Topic Covered in Types of Testing.:- 1. Manual V/S Automated this chapter:- 2. Concept of Verification & Validation. 3. Black Box

412 views • 13 slides
Audited Preliminary Results for the 12 months ended 31 March 2020 24 June 2020 1 Presentation

Audited Preliminary Results for the 12 months ended 31 March 2020 24 June 2020 1 Presentation

Audited Preliminary Results for the 12 months ended 31 March 2020 24 June 2020 1 Presentation Team Euan Fraser John Paton Nick Fienberg Global Chief Executive Officer Chief Financial Officer Chief Commercial Officer Has led Alpha as CEO

485 views • 34 slides
Investor Presentation 9841 Washingtonian Blvd, Suite 390 Gaithersburg MD 20878 Email:

Investor Presentation 9841 Washingtonian Blvd, Suite 390 Gaithersburg MD 20878 Email:

Investor Presentation 9841 Washingtonian Blvd, Suite 390 Gaithersburg MD 20878 Email: info@mPhaseTech.com Stock: XDSL (OTCMKTS) Tel: 973-256-3737 June 2020 mPhase Technologies: AI-Based Technology mPhase is an AI-focused technology company

897 views • 22 slides
Beth Wingates Favorite Proposal Software Tools and Apps Beth Wingate, AF.APMP, APMP Director of

Beth Wingates Favorite Proposal Software Tools and Apps Beth Wingate, AF.APMP, APMP Director of

Beth Wingates Favorite Proposal Software Tools and Apps Beth Wingate, AF.APMP, APMP Director of Education AppMaven@LohfeldConsulting.com Beth.Wingate@APMP.org Here are more of my favorite handy proposal tools and apps. Many are free, and all

312 views • 8 slides
S-100 Maintenance Proposals Miscellaneous revisions, clarifications and corrections S100WG4 25

S-100 Maintenance Proposals Miscellaneous revisions, clarifications and corrections S100WG4 25

S-100 Maintenance Proposals Miscellaneous revisions, clarifications and corrections S100WG4 25 February 1 March 2019 Raphael Malyankar Sponsored by NOAA Overview Proposal: Remove ambiguity in exchange catalogue model about the

151 views • 12 slides
A Software and Systems Process Improvement Initiative in NAVAIR: the Model Based Appraisal (MBA)

A Software and Systems Process Improvement Initiative in NAVAIR: the Model Based Appraisal (MBA)

A Software and Systems Process Improvement Initiative in NAVAIR: the Model Based Appraisal (MBA) Prepared for SSTC Conference 20-24 Apr 2008 Richard Epstein Approved for Public Release by NAVAIR 5 March 2009 Briefing Overview The Problem

227 views • 19 slides
RTI Connext Conference 2017 Vahana: Opening Up Urban Airways with DDS Damien Bardon, Vahana

RTI Connext Conference 2017 Vahana: Opening Up Urban Airways with DDS Damien Bardon, Vahana

RTI Connext Conference 2017 Vahana: Opening Up Urban Airways with DDS Damien Bardon, Vahana Project Avionics Lead October 19, 2017 Agenda Objective A 3 and Vahana Intros Why DDS for Vahana? DDS Architecture Overview Why RTI?

243 views • 20 slides
High Perfor orma mance Re Research Comp omputing - XSE SEDE & Se SeWHiP, David St

High Perfor orma mance Re Research Comp omputing - XSE SEDE & Se SeWHiP, David St

Informa(on Technology Management Council April 29, 2013 High Perfor orma mance Re Research Comp omputing - XSE SEDE & Se SeWHiP, David St Stack - UW-Ea UW-Eau Cla lair ire, e, Pet eter er Bui

320 views • 31 slides
ROPER TECHNOLOGIES OVERVIEW J.P. MORGAN CONFERENCE MARCH 6, 2019 SIMPLE IDEAS. POWERFUL RESULTS.

ROPER TECHNOLOGIES OVERVIEW J.P. MORGAN CONFERENCE MARCH 6, 2019 SIMPLE IDEAS. POWERFUL RESULTS.

A DIVERSIFIED TECHNOLOGY COMPANY ROPER TECHNOLOGIES OVERVIEW J.P. MORGAN CONFERENCE MARCH 6, 2019 SIMPLE IDEAS. POWERFUL RESULTS. SAFE HARBOR STATEMENT The information provided in this presentation contains forward-looking statements within

466 views • 24 slides
Software engineering concepts are needed. Collaborative effort of groups is needed. ABINIT

Software engineering concepts are needed. Collaborative effort of groups is needed. ABINIT

FIRST-PRINCIPLES STUDIES OF MATERIALS : HOW TO ORGANIZE THE SOFTWARE DEVELOPMENT ? The state-of-the-art is in constant progression ... (the following list is not exhaustive, and is biased towards pseudopotentials) 1985 Car-Parinello technique

427 views • 10 slides
eFinancePLUS Enterprise Resource Planning Briefing Speaker name, title Date Important

eFinancePLUS Enterprise Resource Planning Briefing Speaker name, title Date Important

eFinancePLUS Enterprise Resource Planning Briefing Speaker name, title Date Important Notifications The accompanying program, product, investment and other information provided in this presentation is part of on-going negotiations between the

949 views • 60 slides
Real-Time in the Real World: Building a State of the Art Real-Time Analytics Platform INFORMS

Real-Time in the Real World: Building a State of the Art Real-Time Analytics Platform INFORMS

Real-Time in the Real World: Building a State of the Art Real-Time Analytics Platform INFORMS Analytics Conference April 14 th , 2015 Joe DeCosmo Chief Analytics Officer Enova International We are a growing global online Lender Founded

229 views • 18 slides
The Direct Broadcast User Perspective Graeme Martin University of Wisconsin-Madison Space

The Direct Broadcast User Perspective Graeme Martin University of Wisconsin-Madison Space

The Direct Broadcast User Perspective Graeme Martin University of Wisconsin-Madison Space Science and Engineering Center NOAA Satellite Conference, 20 July 2017 Direct Broadcast Direct Broadcast allows users to generate products locally

240 views • 8 slides
Multi-Scale Computing: Integrating Local, Regional, National and International Grids through the

Multi-Scale Computing: Integrating Local, Regional, National and International Grids through the

Multi-Scale Computing: Integrating Local, Regional, National and International Grids through the TIGRE Project Alan Sill, Ph.D TIGRE Senior Scientist, High Performance Computing Center Adjunct Professor of Physics, Texas Tech University Goals

649 views • 20 slides
CAN ESG ADD ALPHA: HIGHLIGHTS FROM MSCI RESEARCH April 25, 2017 AN IMPRACTICAL EXAMPLE OF ESG

CAN ESG ADD ALPHA: HIGHLIGHTS FROM MSCI RESEARCH April 25, 2017 AN IMPRACTICAL EXAMPLE OF ESG

FROM MSCI ESG RESEARCH LLC. CAN ESG ADD ALPHA: HIGHLIGHTS FROM MSCI RESEARCH April 25, 2017 AN IMPRACTICAL EXAMPLE OF ESG RESEARCH RATINGS 2 AGENDA Background and methodology Low-risk / passive investing findingslike size) Factor

283 views • 25 slides
Always ship trunk Managing change in complex websites Paul Hammond Revision Control version 1

Always ship trunk Managing change in complex websites Paul Hammond Revision Control version 1

Always ship trunk Managing change in complex websites Paul Hammond Revision Control version 1 version 2 version 3 version 4 If you re not using any revision control system leave now and start using one But... Coordinating many people

1.17k views • 92 slides
S R V R INVESTING IN THE TECHNOLOGY & INTELLIGENCE REVOLUTION I N D E X WHY SRVR INDEX?

S R V R INVESTING IN THE TECHNOLOGY & INTELLIGENCE REVOLUTION I N D E X WHY SRVR INDEX?

S R V R INVESTING IN THE TECHNOLOGY & INTELLIGENCE REVOLUTION I N D E X WHY SRVR INDEX? FOUNDATION OF TECH Our economy depends on an increasingly sophisticated backbone of servers , fiber optic cable , satellites and network switches to

179 views • 17 slides

More recommend