RENDERING FASTER AND BETTER WITH VRWORKS John Spitzer, April 4, - - PowerPoint PPT Presentation

April 4-7, 2016 | Silicon Valley

John Spitzer, April 4, 2016

RENDERING FASTER AND BETTER WITH VRWORKS

2

HELPING THE ECOSYSTEM BUILD AMAZING VR

VR Ready Hardware Partners Headset Developers Engine Developers App Developers

3

LEADING ENGINES ADOPT NVIDIA VRWORKS

Multi-Res Shading VR SLI Multi-Res Shading VR SLI

4

LEADING HMD MAKERS ADOPT VRWORKS

Direct Mode Context Priority Direct Mode VR SLI

5

VR PLATFORMS FOR CONSUMER, PRO, AND MOBILE

VR workstations Location-based VR CAVEs 360 video capture/streaming M5000M6000

Quadro

Mobile VR headsets Embedded video capture and streaming devices

Tegra

VR gaming and entertainment PCs World’s only VR notebooks Consumer 360 video capture 970 / 980 Ti / Titan X

GeForce

6

Gaming Movies Sports Concerts Travel Retail

CONSUMER APPLICATIONS OF VR ARE HUGE

7

M&E AEC Medical Energy Manufacturing Collaboration

PROFESSIONAL VR MARKET MAY BE EVEN LARGER

8

VR IS RAMPING QUICKLY

Companies Employees Capital Raised

600+ 40,000 $3.8B

VR/AR = $80B revenue by 2025

Source: Goldman Sachs

9

HOW IS VR RENDERING DIFFERENT?

10

HOW IS VR RENDERING DIFFERENT?

High framerate, low latency

High FPS, low latency Stereo Rendering Lens Distortion

11

NVIDIA VRWORKS SDK

WARP AND BLEND

Increase performance via an innovative new way to render for VR Scale performance with multiple GPUs Minimize head tracking latency with asynchronous time warp Plug and play compatibility from GPU to HMD Reduce latency by rendering directly to the front buffer API for geometry and intensity adjustments for seamless VR Provides tear free VR environments by synchronizing multi GPU’s Fine grain control to pin OGL contexts to specific GPU’s Reduces latency for video transfer to and from the GPU

SYNCHRONIZATION GPU AFFINITY GPUDIRECT FOR VIDEO

App Headset Pro only

12

VR SLI

Two eyes...two GPUs!

13

“NORMAL” SLI

GPUs render alternate frames N N+1 N N+1 N N+1 CPU GPU 0 GPU 1 Display Latency

14

VR SLI

Each GPU renders one eye—lower latency NL N+1R N N+1 NR N+1L N N+1 CPU GPU 0 GPU 1 Display Latency

15

VR SLI

GPU affinity masking: full control Shadow maps, GPU physics, etc. Left eye rendering Right eye rendering

16

VR SLI

Broadcasting reduces CPU overhead

R L

Render scene once

17

VR SLI

Per-GPU constant buffers, viewports, scissors

L R

Multi-GPU API

R L

Engine

18

VR SLI

Cross-GPU data transfer via PCI Express

19

MULTI-RESOLUTION SHADING

20

VR HEADSET OPTICS

Distortion and counter-distortion

21

VR HEADSET OPTICS

Distortion and counter-distortion User’s view Image Displayed Optics

22

DISTORTED RENDERING

Render normally, then resample Rendered image Distorted image

23

DISTORTED RENDERING

Over-rendering the outskirts Rendered image Distorted image

24

MULTI-RESOLUTION SHADING

Subdivide the image, and shrink the outskirts

25

MULTI-RESOLUTION SHADING

Fast viewport broadcast on NVIDIA Maxwell GPUs Viewport 1 Viewport 2 Viewport N

...

Geometry Pipeline

26

MULTI-RESOLUTION SHADING

https://developer.nvidia.com/vrworks NVAPI for Maxwell viewport broadcast in DX11/12 Helper code for configuring viewports, shader math, etc DX11 sample app Guide to integrating in an existing engine OpenGL extensions are also available

Public, open-source SDK

27

UE4 INTEGRATION

Postprocessing passes supported:

Bloom & lens flare Tonemapping Temporal AA Reflection environment Height fog Decals Distortion/refraction

https://github.com/NvPhysX/UnrealEngine/tree/MultiRes-4.10

28

MULTI-RESOLUTION SHADING

Infiltrator demo seeing 30-40% increase Everest game seeing up to 40% increase Up to 50% when entirely pixel-bound!

Performance

NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.

29

RENDERING PASSES

Geometry passes: add FastGS; render with 9 viewports & scissors

Base pass, decals, deferred lights, depth prepass, distortion, shadow projection, translucency, velocity

Image-space passes: remap linear/multi-res UVs as needed

Decals, deferred lights, postprocessing, shadow projection, tiled deferred lights

Text/HUD would be done after multi-res

30

IMAGE-SPACE PASSES

Reconstruct 3D position from 2D+depth

Map 2D position to linear before reconstructing

Large filter kernels

Map sample point to linear, apply offset, map back to multi-res

Two common cases

31

UV REMAPPING

32

DEMO TIME!

Unreal Engine 4.11

33

NVIDIA VRWORKS SDK

WARP AND BLEND

Increase performance via an innovative new way to render for VR Scale performance with multiple GPUs Minimize head tracking latency with asynchronous time warp Plug and play compatibility from GPU to HMD Reduce latency by rendering directly to the front buffer API for geometry and intensity adjustments for seamless VR Provides tear free VR environments by synchronizing multi GPU’s Fine grain control to pin OGL contexts to specific GPU’s Reduces latency for video transfer to and from the GPU

SYNCHRONIZATION GPU AFFINITY GPUDIRECT FOR VIDEO

App Headset Pro only

34

Context priority for asynchronous time warp

Reduces latency from head rotation

Frame (Left) Frame (Right) Head Tracking (t) Warped Frame Head Tracking (t+1)

35

Direct Mode

Plug-and-play Compatibility Via Direct Interface with the VR Headset

Operating System

36

Front Buffer Rendering

Reduces Latency By Rendering Directly to the Headset

Scan Out Warp

37

NVIDIA VRWORKS SDK

WARP AND BLEND

Increase performance via an innovative new way to render for VR Scale performance with multiple GPUs Minimize head tracking latency with asynchronous time warp Plug and play compatibility from GPU to HMD Reduce latency by rendering directly to the front buffer API for geometry and intensity adjustments for seamless VR Provides tear free VR environments by synchronizing multi GPUs Fine grain control to pin OGL contexts to specific GPU’s Reduces latency for video transfer to and from the GPU

SYNCHRONIZATION GPU AFFINITY GPUDIRECT FOR VIDEO

App Headset Pro only

38

Warp and blend

API for Geometry, Intensity and Black Offset

• n each display

Adjustments applied to the full desktop image Zero latency Application independent

Image courtesy of Joachim Tesch — Max Planck Institute for Biological Cybernetics

APIs for rendering custom immersive environments

39

Swap Groups and Swap Barriers: HW based swap synchronization within a node or between clusters

Co-ordinate Buffer Swaps

Synchronization

Prevent tearing and image misalignment in VR environments

Frame Lock: Align the scan out of multiple displays, GPU’s, and systems Stereo Lock: Maintain Stereo alignment between multiple systems Synchronize to an internal or external timing source

Synchronize multiple displays

40

GPUDirect™ for video

Enables real-time, low latency video transfers in VR environments 360 degree video capture and stitching Capture, render & output in less than 3 frames Synchronized Transfers Minimal CPU Overhead Sub Frame Transfer Size

41

NVIDIA VRWORKS: PAVING THE WAY FOR VR

WARP AND BLEND

Increase performance via an innovative new way to render for VR Scale performance with multiple GPUs Minimize head tracking latency with asynchronous time warp Plug and play compatibility from GPU to HMD Reduce latency by rendering directly to the front buffer API for geometry and intensity adjustments for seamless VR Provides tear free VR environments by synchronizing multi GPU’s Fine grain control to pin OGL contexts to specific GPU’s Reduces latency for video transfer to and from the GPU

SYNCHRONIZATION GPU AFFINITY GPUDIRECT FOR VIDEO

App Headset Pro only

42

QUESTIONS?

April 4-7, 2016 | Silicon Valley

THANK YOU

JOIN THE NVIDIA DEVELOPER PROGRAM AT developer.nvidia.com/join