VR Movies Concerts Travel Retail 2 QUADRO PROFESSIONAL VR/AR - - PowerPoint PPT Presentation

▶

Sep 03, 2022 1.45k likes •1.83k views

NVIDIA CLOUDXR David Weinstein, 19 December 2019 Gaming Sports PERSONAL ENTERTAINMENT VR Movies Concerts Travel Retail 2 QUADRO PROFESSIONAL VR/AR MEDICAL ARCHITECTURE DIGITAL CONTENT CREATION DESIGN MANUFACTURING COLLABORATION

SLIDE 1

David Weinstein, 19 December 2019

NVIDIA CLOUDXR™

SLIDE 2

Gaming Movies Sports Concerts Travel Retail

PERSONAL ENTERTAINMENT VR

SLIDE 3

NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.

QUADRO PROFESSIONAL VR/AR

MEDICAL ARCHITECTURE DESIGN MANUFACTURING COLLABORATION DIGITAL CONTENT CREATION

SLIDE 4

SLIDE 5

SLIDE 6

SLIDE 7

FUNDAMENTAL VR GRAPHICS CHALLENGES

TRADITIONAL = 60 MP/S

(1920 X 1080 @ 30 FPS) 1920 1080

VIRTUAL REALITY = 450 MP/S

(3024 X 1680* @ 90 FPS) 1512 1680 1512

Motion to Photon: ≤ 20 ms

Very High Pixel Rendering Throughput Must Maintain Very Low Latency

Pixel Throughput and Perceived Latency

HMD updates pose CPU triggers redraw GPU computes new image HMD flashes display

SLIDE 8

WANTED: THE BEST OF BOTH WORLDS

Highest Quality Graphics + Mobile Freedom

SLIDE 9

CLOUDXR VISION

RTX Servers Content Network Clients XR Streaming to Any Device

SLIDE 10

MOBILE WORLD CONGRESS (LA), OCT 2019

CloudXR Announcement

SLIDE 11

ARCHITECTURE OVERVIEW

Server-Side Driver, Client-Side Application and SDK

OpenVR Runtime (Valve) VR App Virtual HMD Driver Cloud Late & Lens Warp VR Runtime Video Decoder Cloud HMD Video Encoder Virtual Audio Driver Audio Encoder Audio Decoder Internet SERVER driver (Nvidia) CLIENT app (Nvidia)

audio, video, haptics pose, control inputs, video

SLIDE 12

ARCHITECTURE

WIN SERVER CLIENT

INSTALLER

Audio/Video Data (sensors, inputs, QoS) Encode RGB Frame Display

VR App

Decode Video Params (e.g. ROI, optional) HEVC Frame HEVC Frame VR Runtime Controller & inputs Sensor, Pose & Controller Data RGB Frame Sensor & input data Sensor & input data Audio

(e.g., OpenVR)

App Client app Server driver

Time Warp Lens Warp

RGB Frame Driver API VR Runtime Runtime API OpenVR Interfaces

HMD

Audio Playback Audio Capture / Encode

SLIDE 13

QOS

Relevant Internet problems:

Jitter
Missed packets
Bandwidth variation

Reliability & Resilience

SLIDE 14

QOS

Reliability & Resilience Challenges Solutions Control parameters

Latency Custom HEVC profile NVENC knobs Jitter De-jitter buffer Buffer size Missed packets Forward error correction Correction strength Bandwidth variation Video compression Compression rate Server has no display Frame pacing to match client display rate Server render rate

SLIDE 15

WIN SERVER

VR App

Video Params (e.g. ROI, optional)

App

Driver API VR Runtime Runtime API

ARCHITECTURE

INSTALLER

Audio / Video / Data (sensors, inputs, QoS) Encode RGB Frame Decode HEVC Frame Sensor, Pose & Controller data Sensor & input data Audio

Client app Server driver

Audio Capture / Encode Audio Playback Frame Pacing Jitter FIFO CLIENT Display VR Runtime Controller & inputs

(e.g., OpenVR) Time Warp Lens Warp HMD

OpenVR Interfaces

SLIDE 16

LATENCY

Using:

HelloVR sample modified to change color on button press
Latency meter (instrumented switch + photodiode)
Local Cloud with Client on WiFi (average ping time 4 ms)

Button to Photons

SLIDE 17

LATENCY

Results: Server Driver + Client App latency = 12.1 ms. The Jitter FIFO can be downsized at the cost of occasional missed frames.

Button to Photons

Latency [ms] Stdev Local VR 38.3 3.4 CloudXR 81.6 3.3 Diff 43.3

Jitter FIFO + Ping 31.2 Diff – Above 12.1

*Average of 40 clicks

SLIDE 18

WIN SERVER

VR App

Video Params (e.g. ROI, optional)

App

Driver API VR Runtime Runtime API

LATENCY

INSTALLER

Audio / Video / Data (sensors, inputs, QoS) Encode RGB Frame Decode HEVC Frame RGB Frame Sensor & input data Audio

Client app Server driver

Audio Capture / Encode Audio Playback Frame Pacing Jitter FIFO CLIENT Display VR Runtime Controller & inputs

(e.g., OpenVR) Time Warp Lens Warp HMD

3-10 ms 10-50 ms (depending on network) 1-3 ms 3-10 ms

OpenVR Interfaces < 0.5 ms

SLIDE 19

SLIDE 20

Total Flexibility

CLOUDXR CONFIGURATIONS

5G Ethernet WiFi

SLIDE 21

EARLY ACCESS PROGRAM

Test CLOUDXR for VR @ https://developer.nvidia.com/nvidia-cloudxr-sdk-early-access- program

SLIDE 22

NVIDIA 5G MEC XR DEVKIT

SLIDE 23

THE OPPORTUNITY: VR/AR/VGPU OVER 5G

A Convergence of Visual Compute + 5G and a New Edge Topology

Powerful GPUs

Incredible compute capabilities redefining graphics

Bandwidth and Low Latency

Both are required for VR/AR

Compute is at the Edge

Very few hops – Low Latency

SLIDE 24

ENTERPRISE MARKETS

Success Across Verticals

Manufacturing

Engineering, Design, Marketing/Sales

AEC

Entire Design Process, Sales, Moving to Construction

M&E

Virtual Production

Training

Possibly the Largest Market

Enscape Nurulize IC.IDO Autodesk

Location-Based Entertainment

Enterprise/Consumer

ESI

SLIDE 25

ENTERPRISE AR

DIGITAL MODELS ANNOTATED

INSTRUCTION/COLLABORATION

Driving Value Propositions

SLIDE 26

THE FUTURE OF MOBILE XR

Ultra Low Latency
High Bandwidth
Cutting-edge Compute
Cost Effective Delivery
Valuable Use Cases

Requirements:

5G + VR/AR + vGPU at the Edge

SLIDE 27

RTX Performance in a Server

Powered by Quadro RTX 8000/6000
Ray Traced Global Illumination up to 96

GB Scenes

Remoting and Multi-GPU Virtualization

with Quadro virtual Datacenter Work Station (vDWS)

All the technical advantages of Turing

NVIDIA RTX SERVER

SLIDE 28

XR/VGPU 5G EDGE STACK

Software/Hardware Stack Demands a Diverse Effort

Cloud Orchestration

Containerized Edge Applications

Low Latency Streaming

Visual Computing and AI 5G transmission 5G Network

Virtualization

Enabling SDKs Fundamental GPU Drivers

SLIDE 29

COME TRY THE DEMO & REGISTER ONLINE FOR EARLY ACCESS

https://developer.nvidia.com/nvidia- cloudxr-sdk-early-access-program Visit the VR Village on the Expo Floor

SLIDE 30