Agenda S8539 - Pooling and Orchestrating NVIDIA Jetson for AI and - - PowerPoint PPT Presentation

▶
agenda
SMART_READER_LITE
LIVE PREVIEW

Agenda S8539 - Pooling and Orchestrating NVIDIA Jetson for AI and - - PowerPoint PPT Presentation

Oct 07, 2023 150 likes •374 views

Agenda S8539 - Pooling and Orchestrating NVIDIA Jetson for AI and Deep Learning on the Edge Overview of Composable Infrastructure Fabric and Software Architecture Overview Jetson Pooling and

slide-1
SLIDE 1
slide-2
SLIDE 2

2 PROPRIETARY & CONFIDENTIAL

Agenda

S8539 - Pooling and Orchestrating NVIDIA Jetson for AI and Deep Learning on the Edge

  • Overview ¡of ¡Composable ¡Infrastructure
  • Fabric ¡and ¡Software ¡Architecture ¡Overview
  • Jetson ¡Pooling ¡and ¡Scaling ¡Use ¡Case
  • Summary ¡-­‑ Q&A
slide-3
SLIDE 3

PROPRIETARY & CONFIDENTIAL 3

Agility Always.

Orchestrating the Composable Data Center

slide-4
SLIDE 4

PROPRIETARY & CONFIDENTIAL – 4/27/2016 4

Liqid Delivers Composable Infrastructure

Liqid software and fabric technology enables bare metal provisioning and management of data center resources

Storage Network Compute GPU Fabric

slide-5
SLIDE 5

PROPRIETARY & CONFIDENTIAL 5

Impact of Composable

slide-6
SLIDE 6

6

Trillions of Global Data Transfers Traditional Infrastructure Converged & HyperConvrged Composable Infrastructure

( agile, efficient, scalable)

Disaggregated Resources

( Fabric & Software )

PAST PRESENT FUTURE

Data Center Roadmap

PROPRIETARY & CONFIDENTIAL

slide-7
SLIDE 7

PROPRIETARY & CONFIDENTIAL 7

Liqid enables Composable Infrastructure with a PCIe fabric and software that orchestrates and manages bare-metal servers

Composable Fabric

PCIe is the ONLY common interconnect for ALL data center resources

slide-8
SLIDE 8

Storage Pool Compute Pool Networking Pool

Software Defined Data Center

Dynamically Configure Bare Metal Servers Disaggregated Pools of Physical Resources

Server-1 Server-2 Server-N

Fabric

PROPRIETARY & CONFIDENTIAL

Graphics Pool

slide-9
SLIDE 9

9 PROPRIETARY & CONFIDENTIAL

Jetson Specification

Embedded PCIe Interface (Gen2x4)

slide-10
SLIDE 10

10 PROPRIETARY & CONFIDENTIAL

Benefits:

  • Low Power
  • Small Form Factor
  • Powerful GPU
  • Onboard Compute

Limitations:

  • Limited Storage
  • Limited Networking
  • No Standard Deployment Chassis for Scale Out

Fabric Based Deployment on Jetson:

  • Enables scale-out in data center environments
  • Enables expanded storage
  • Enables expanded networking

Jetson Overview

Storage Pool Networking Pool Fabric

slide-11
SLIDE 11

PROPRIETARY & CONFIDENTIAL 11

Universal PCIe Expansion Chassis

2U 24-Bay Chassis (24x Gen3 x4 Sockets) Off-the-Shelf Storage

(U.2, MLC, 3D, Xpoint, HDD, etc.)

Storage

Off-the-Shelf Networking

(Eth, FC, IB, SAS, etc.)

Networking

PCIe Fabric Switch

GPU

Jetson GPU

slide-12
SLIDE 12

PROPRIETARY & CONFIDENTIAL 12

Universal PCIe Expansion Chassis

2U 24-Bay Chassis (24x Gen3 x4 Sockets)

Storage Networking GPU

Storage Up ¡to ¡24x ¡U.2 ¡SSD ¡= ¡192 ¡TB Networking Up ¡to ¡48x ¡Networking ¡Ports ¡(Eth, ¡FC, ¡IB, ¡etc..) GPU Up ¡to ¡24x ¡Jetson ¡Modules Management ¡SW Liqid ¡OS ¡Orchestration ¡Software

slide-13
SLIDE 13

13 PROPRIETARY & CONFIDENTIAL

Storage Pool Networking Pool GPU Pool Fabric Switch

Machine-1 Machine-2 Machine-N

Fabric

Dynamic GPU Super Cluster

Architecture Overview

Dynamic Bare Metal Machines Creation

slide-14
SLIDE 14

14

Enhanced Jetson Use Cases

PROPRIETARY & CONFIDENTIAL

  • Jetson in Edge Servers
  • Jetson Scale-out in Data Center
slide-15
SLIDE 15

15 PROPRIETARY & CONFIDENTIAL

Edge Computing Use Case

Edge Deployment Requirement:

  • Multi-Server Deployment
  • Low Power
  • Small Form Factor
  • Requires GPU
  • Requires Compute
  • Requires 4TB of Storage Per Server
  • Requires 2x 10Gbe Networking Ports

Option #1) x86 + GPU Server:

  • 1x Tesla P100
  • Intel Xeon 2.5GHz CPU
  • 4TB of SSD
  • 2x 10Gbe Networking ports

Option #2) Jetson Server:

  • 1x Jetson TX2
  • CPU + GPU Combo
  • 4TB of SSD
  • 2x 10Gbe Networking ports
slide-16
SLIDE 16

16 PROPRIETARY & CONFIDENTIAL

Jetson Server vs. x86 Server

Jetson Server x86 + GPU Server GPU 8 W 250 W Compute 0 W 125 W Storage 15 W 15 W Networking 15 W 15 W Total Power 38 W 395 W Server Cost $2,750 $13,500 Form Factor 8x Servers per 2U 1x Server per 2U Watts / Core .029 W .105 W Cost / Core $1.17 $3.07

90% 90% Lo Lower r Po Power 85% 85% Sm Smaller Fo Footpri rint

Jetson Servers (Fabric Based Server) Legacy Servers (2RU Server Chassis)

Jetson Enables Lower Power and Smaller Footprint

slide-17
SLIDE 17

17

Jetson in the Data Center

PROPRIETARY & CONFIDENTIAL

Example of Neural Net Model Built/Trained on Tesla and Deployed on Jetson for Inference

Jetson deployed on PCIe Fabric can also be used for Training

slide-18
SLIDE 18

18

Inference & Training on Jetson

PROPRIETARY & CONFIDENTIAL

Inference on Jetson in the Field Training on Jetson in the Data Center

slide-19
SLIDE 19

19

Jetson Performance

PROPRIETARY & CONFIDENTIAL

Composed Jetson + NVMe SSD Performance: 1.63 GB/s and 160K IOPS

slide-20
SLIDE 20

20 PROPRIETARY & CONFIDENTIAL

Summary

  • Jetson ¡(and ¡Pegasus) ¡are ¡powerful, ¡small ¡form ¡factor
  • Limited ¡in ¡direct ¡connected ¡storage, ¡networking, ¡and ¡scaling ¡capabilities
  • Deploying ¡Jetson ¡on ¡a ¡PCIe ¡fabric ¡enables:
  • Expanded ¡Storage
  • Expanded ¡Networking
  • Standard ¡form ¡factor ¡method ¡of ¡deployment
  • Summary ¡-­‑ Q&A
slide-21
SLIDE 21

21

Thank You