Nucleus: Eight GPU Platform for Visual Simulation David Morgan - - PowerPoint PPT Presentation

▶

Dec 18, 2023 261 likes •504 views

Nucleus: Eight GPU Platform for Visual Simulation David Morgan Principal Engineer Aechelon Technology S9224 Session Trajectory Visual Simulation Background Monsters, Clusters, and Moores Law Nucleus Architecture Challenges

SLIDE 1

Nucleus: Eight GPU Platform for Visual Simulation

David Morgan Principal Engineer Aechelon Technology

S9224

SLIDE 2

Session Trajectory

Visual Simulation Background
Monsters, Clusters, and Moore’s Law
Nucleus Architecture
Challenges
Demo

SLIDE 3

Aechelon Technology

SLIDE 4

Image Generation

SLIDE 5

2000: RealityMonster

5 Racks
8 Graphics “Pipelines” (GPUs)
24 CPUs
9GB RAM (NUMA)
140GB Storage
Single IRIX OS
15kW
$2.8M

SLIDE 6

Scalability Matters

128 CPUs
256GB RAM
16 GPUs

SLIDE 7

2001: GeForce 3

SLIDE 8

2002-Today: PC Clusters

1-3 Racks per IG
1 GPU per node
1U Diskless Renderers
3U Pager w/88TB Storage
Windows OS Per Node
Ethernet Interconnect
7000W (8ch)
Unlimited Scalability

SLIDE 9

Moore’s Law is Dead

Cluster Nucleus 2 2.2 2.4 2.6 2.8 3 3.2 3.4 3.6 3.8 4 6 8 10 12 14 16 18 20 22 24 26 28 GHz Cores Skylake 2017 Broadwell 2016 Haswell 2014 Ivy Bridge 2013 Sandy Bridge 2012

SLIDE 10

2016: 8-GPU Support

SLIDE 11

Multi-GPU is Hard

SLIDE 12

Nucleus

4U
8 Quadro GPUs
One display per GPU
36 CPU Cores
192GB RAM
36TB Storage
One Windows OS
$100-200K
Operates up to 35C
2000W
Limited Scalability

SLIDE 13

Dual Root Complex

SLIDE 14

Single Root Complex

SLIDE 15

GPU Affinity

Exposed in OpenGL through WGL_NV_gpu_affinity extension
Quadro feature necessary to address individual GPUs on Windows
pC-Nova Maps GPU device handles to screens in the Windows

virtual desktop

Beware driver crashes enumerating more than 4 screens per GPU!

SLIDE 16

EDID Management

http://johnsciacca.webs.com/apps/blog/show/16852621-installation-nightmares-9-professional-horror-stories

SLIDE 17

DWM Is…

Independent GPUs’ video

timings phase shift.

Normally correctable by

tracking the phase

“Full-Screen Exclusive

Mode” is gone.

DWM intermediates all

drawing on multi-display systems.

One display is Primary.

SLIDE 18

DWM Is Evil

https://www.pandza.xyz/article/16/dwm,-dxgi,-swap-chains,-latency,-throughput-and-you

GPU 2 GPU 1

SLIDE 19

Workaround: Framelock

Quadro Sync II supports 8 GPUs

per system

Shared oscillator ensures

displays remain in phase with Primary

DWM placated!
Downside: Video timings must

all match

Downside: Wiring is delicate

SLIDE 20

Future Work

GPU Multicast
Or Dual Root Complex?
VR Direct?
Clusters of Nuclei

SLIDE 21

Thanks

Doug Traill
John Chaney
Tim Woodard
Steve Nash
Ian Williams

SLIDE 22

Demo

SLIDE 23