Leveraging Microsoft Azures GPU N -Series for Compute and - PowerPoint PPT Presentation

S6839 Leveraging Microsoft Azure’s GPU N -Series for Compute and Visualization Karan Batta, Program Manager, Microsoft Azure Alexey Kamenev, Software Engineer, Microsoft Research

Agenda  Azure  HPC in the Cloud  T echnology/Architecture  CNTK Overview & Demo

Vision  GPU based capabilities in cloud infrastructure  High end performance  Not “Swiss - army knife” approach  Deliver and empower developer scenarios  Achieve true “ HPC PC in the Cl e Clou oud ”  Critical workloads

HPC in the Cloud APP exe exe exe exe

Workflow Return Azure results Algorithm Rendering Manage Executable {REST API} Submit job GPU VMs Split job/ Upload data setup execution pipeline Outpu tputs ts GPU Visualization Dynamic Modelling Analytics Virtual Desktops

Where? Financ Fi nce Manufa fact cturin uring g & Oi & Oil/Gas • FX Options • Automotive design • Risk Management • Reservoir modelling • Hedge Fund Management • Manipulation of models & parts Medi dia Re Rende dering ring • Streaming games/video • VFX/Ray-Tracing rendering • Transcoding • CAD applications • Social media analysis • Simulations

T echnology  DDA (Discrete Device Assignment)  Introduced in Windows Server 2016  Pass-through PCIe devices  Allows for close to bare-metal performance

Architecture • Custom Applications Applications • Data and Applications from the Azure Marketplace • Bring your own Image Client OS • Azure VM Marketplace Images GPU • Hyper-V Provisioning • DDA Host OS • NVIDIA M60 GPU (Viz SKU) Hardware • NVIDIA K80 GPU (Compute SKU)

Visualization VMs NV6 NV12 NV24 6 12 24 Cores (E5-2690v3) (E5-2690v3) (E5-2690v3) 1 x M60 GPU (1/2 2 x M60 GPU (1 4 x M60 GPU (2 GPU Physical Card) Physical Card) Physical Cards) Memory 56 GB 112 GB 224 GB Disk ~380 GB SSD ~680 GB SSD ~1.5 TB SSD Network Azure Network Azure Network Azure Network

Compute VMs NC6 NC12 NC24 NC24r 6 12 24 24 Cores (E5-2690v3) (E5-2690v3) (E5-2690v3) (E5-2690v3) 1 x K80 GPU (1/2 2 x K80 GPU (1 4 x K80 GPU (2 4 x K80 GPU (2 GPU Physical Card) Physical Card) Physical Cards) Physical Cards) Memory 56 GB 112 GB 224 GB 224 GB Disk ~380 GB SSD ~680 GB SSD ~1.5 TB SSD ~1.5 TB SSD Azure Network + Network Azure Network Azure Network Azure Network RDMA (RoCE)

CNTK Alexey Kamenev Senior Software Engineer Microsoft Research

CNTK Overview • A deep learning tool that balances • Efficienc ciency: Can train production systems as fast as possible • Perfor formanc mance: Can achieve state-of-the-art performance on benchmark tasks and production systems • Flex exib ibility lity: Can support various tasks such as speech, image, and text, and can try out new ideas quickly • Inspiration: Legos • Each brick is very simple and performs a specific function • Create arbitrary objects by combining many bricks • CNTK enables the creation of existing and novel models by combining simple functions in arbitrary ways. • Historical facts: • Created by Microsoft Speech researchers (Dong Yu et al.) 4 years ago Was quickly extended to handle other workloads (image/text) • • Open-sourced (CodePlex) in early 2015 • Moved to GitHub in Jan 2016

Resources • “Deep Learning in Microsoft with CNTK” – Alexey exey Kame menev nev, , Micr croso soft – Hall l 3 – 4.30pm pm • CNTK (Deep-Learning toolkit) • htt ttps:// ://github github.co .com/ m/Micr Microso osoft/ t/CNTK CNTK • DDA (Direct Device Assignment) • htt ttp:// //blo blogs. gs.techn technet.com et.com/b/vi /b/virtuali tualizatio zation/ar n/archiv chive/ e/2015/11/23/ 015/11/23/disc discrete ete- devic ice-as assi signmen nment-gp gpus. us.as aspx • NVIDIA announcement • htt ttp://nvidianew //nvidianews.nvidia s.nvidia.co .com/ m/news/nvidia ews/nvidia-gpus pus-to to-accel accelerate erate-mic microso soft- azure

Thank you!