Virtual Reality Features of NVIDIA GPUs Mark Kilgard, July 27, 2016 - - PowerPoint PPT Presentation

▶
virtual reality features of nvidia gpus
SMART_READER_LITE
LIVE PREVIEW

Virtual Reality Features of NVIDIA GPUs Mark Kilgard, July 27, 2016 - - PowerPoint PPT Presentation

Jul 23, 2023 26 likes •451 views

Virtual Reality Features of NVIDIA GPUs Mark Kilgard, July 27, 2016 Mark Kilgard My Background Principal System Software Engineer OpenGL driver implementation & API evolution Other interests Programmable shading languages, original Cg

slide-1
SLIDE 1

Mark Kilgard, July 27, 2016

Virtual Reality Features of NVIDIA GPUs

slide-2
SLIDE 2

2

Mark Kilgard

Principal System Software Engineer

OpenGL driver implementation & API evolution Other interests

Programmable shading languages, original Cg GPU shading language Web, vector graphics & path rendering, GPU rendering for 2D graphics

Also known for creating OpenGL Utility Toolkit (GLUT) My Background

slide-3
SLIDE 3

3

GPU Challenges for Virtual Reality Rendering

  • 1. Must render unique view for each eye

Two scene renders per frame

  • 2. Wide field-of-view rendered

More pixels, distributed towards periphery

  • 3. HMD lens magnifies rendered scene

Non-linear inverse lens warp Allows the HMD lens magnification to provide convincing sense of presence

slide-4
SLIDE 4

4

Magnification Effect of HMD Lens

Graph paper viewed & magnified through HMD lens

slide-5
SLIDE 5

5

Head Mounted Display Lens Effect

Barrel Distorted Images

  • n HMD Screen

Magnified Overlapping Wide Field-of-view Binocular View

slide-6
SLIDE 6

6

Focus on Virtual Reality Features of Pascal GPUs

Virtual Reality rendering features in Pascal GPUs

GeForce 1080/1070/1060 NEW Quadro P6000 & P5000

As exposed in standard 3D APIs

slide-7
SLIDE 7

7

Background: NVIDA GPU Architecture Road Map

Our interest NVIDIA GPU architectures of interest: Pascal

What is Pascal mentioned on last slide?

slide-8
SLIDE 8

8

Conventional 3D Single-eye Scene

slide-9
SLIDE 9

9

Conventional 3D Single-eye Scene

slide-10
SLIDE 10

10

Wide Stereo Views of Same Scene

Left Right

slide-11
SLIDE 11

11

Left Right

slide-12
SLIDE 12

12

Right Left

slide-13
SLIDE 13

13

Left & Right Views Are Similar but Different

Left and Right eye views have different occlusion While left & right eye views look similar, must be draw independently

Really only different in a small translation in X

NVIDIA Single Pass Stereo supports generating left & right eye positions

GPU automatically makes left & right eye versions of each primitive Then renders each view to a different framebuffer slice Exposed in OpenGL by NV_stereo_view_rendering extension Exposed for Direct3D 11 by NVAPI SinglePassStereoMode feature

slide-14
SLIDE 14

14

Image Difference of Two Views

− + 0.5 =

Left eye view Right eye view Clamped difference image

slide-15
SLIDE 15

15

Left & Right Views Rendered Efficiently in One Pass

slide-16
SLIDE 16

16

Lens Matched Shading

Left Right

slide-17
SLIDE 17

17

Visualizing Lens Matched Shading

slide-18
SLIDE 18

18

Lens Matched Shading

Left Right

slide-19
SLIDE 19

19

Visualization of Lens Matched Shading Rendering

slide-20
SLIDE 20

20

Still Needs Non-linear Warp

Lens Matched Shading rendering Warped image suitable for HMD display Non-linear image

slide-21
SLIDE 21

21

Still Needs Non-linear Warp

Lens Matched Shading rendering Warped image suitable for HMD display Non-linear image BUT substantially less pixel motion and distortion than conventional VR re-warping

slide-22
SLIDE 22

22

Example HMD Post-rendering Warp

slide-23
SLIDE 23

23

Lens Matched Shading with Window Rectangle Testing

slide-24
SLIDE 24

24

Lens Matched Shading with Window Rectangle Testing

Nothing in black corners is shaded or even rasterized Yellow lines show overlaid 8 inclusive window rectangles Same 8 window rectangles “shared” by each view’s texture array layer

slide-25
SLIDE 25

25

Lens Matched Shading with Window Rectangle Testing

slide-26
SLIDE 26

26

What Didn’t Have to be Rendered

17% of left view pixels 17% of right view pixels

slide-27
SLIDE 27

27

Window Rectangle Testing During Rasterization

New multi-vendor OpenGL extension

EXT_window_rectangles Provides a set of window-space rectangles, 8 for NVIDIA GPUs Rasterizer either includes or excludes rasterization from rectangles

Discards at speed-of-light

As if free to rasterize to irregular rectangular regions

Window Rectangles Test NEW stage

OpenGL’s fragment processing pipeline

slide-28
SLIDE 28

28

Warped Lens Matched Scene

Warped version of Lens Matched Shading to match HMD lens

slide-29
SLIDE 29

29

Warped Lens Matched Shading

slide-30
SLIDE 30

30

Visualizing Warp Window Rectangles

slide-31
SLIDE 31

31

What Didn’t Have to be Warped

17% of left view pixels 17% of right view pixels

slide-32
SLIDE 32

32

VR Rendering Pipeline in Images

LMS Right Eye View Warped Right Eye View LMS Left Eye View Warped Left Eye View Scene LMS = Lens Matched Shading

slide-33
SLIDE 33

33

VR Rendering Pipeline in Images

LMS Right Eye View Warped Right Eye View LMS Left Eye View Warped Left Eye View Scene Displayed within HMD

Single Rendering Pass

Single Pass Stereo + Lens Matched Shading + Window Rectangle Testing

Drawn with Single Triangle

Fragment Shader Warping Window Rectangle Testing Perception to user is linear rendering HMD lens “undoes” warping to provide a perceived wide field-of-view

Pascal does all this efficiently in a single rendering pass! 8 viewports, 1 pass

slide-34
SLIDE 34

34

Traditional 3D Rendering Pipeline

Transform 3D View Downsample Display Frame App Processing

if multisampled uses pinhole camera projection, rasterizing to uniform rectilinear pixel grid monitor directly displays uniform rectilinear pixel grid

Rasterize 3D View

slide-35
SLIDE 35

35

Virtual Reality Rendering Pipeline

Rendering Left & Right Eye Views

Transform Left View Downsample Display Frame to HMD App Processing Rasterize Left View Transform Right View Downsample Rasterize Right View HMD Warp HMD Warp

warps uniform rectilinear grid based on HMD’s device profile

Transform Right View Rasterize Right View Transform Right View Downsample Rasterize Right View Transform Right View HMD Warp Downsample Rasterize Right View Transform Right View

second view rendering

slide-36
SLIDE 36

36

Virtual Reality Rendering Pipeline

With Lens Matched Shading

Transform Left View Downsample Display Frame to HMD App Processing Rasterize Left View Transform Right View Downsample Rasterize Right View HMD Warp HMD Warp

Lens Matched Shading Lens Matched Shading

replicates primitive to 4 projected quadrants less pixel resampling effort & error operating on LMS rasterized images

slide-37
SLIDE 37

37

Virtual Reality Rendering Pipeline

With Single Pass Stereo

Transform Left & Right View Downsample Display Frame to HMD App Processing Rasterize Left View Downsample Rasterize Right View HMD Warp HMD Warp

  • ne transform instance,
  • utputs left & right position
slide-38
SLIDE 38

38

Virtual Reality Rendering Pipeline

Single Pass Stereo + Lens Matched Shading

Downsample Display Frame to HMD Rasterize Left View Downsample Rasterize Right View HMD Warp HMD Warp

Lens Matched Shading Lens Matched Shading

Transform Left & Right View App Processing

+

Advantages

  • 1. Single transform for both views
  • 2. Closer match between rendering & lens images
  • 3. Fewer overall pixels for similar quality

use window rectangle testing to bound lens region, during both rasterization & warp

slide-39
SLIDE 39

39

Pascal GPU Features for VR

Unique to NVIDIA

Pascal renders left & right eye

In single rendering pass Halves CPU load for rendering Better concurrent texture locality

OpenGL: NV_stereo_view_rendering Direct3D: NVAPI SinglePassStereoMode Better matches rendering to HMD lens

Single pass Four separately projected quadrants Less rendering, less resampling error

OpenGL: NV_clip_space_w_scaling Direct3D: NVAPI ModifiedWSupport

slide-40
SLIDE 40

40

Other NVIDIA Virtual Reality Initiatives

slide-41
SLIDE 41

41

More Information

VRWORKS 2.0 Software Development Kit

Available NOW

Pascal OpenGL extensions for Virtual Reality

Specification now public

In official OpenGL.org extension registry

OpenGL Extension Wrangler 2.0 (GLEW) support Example code in VRWORKS 2.0 SDK

VRWORKS also documents NVAPI usage

For Direct3D 11 Includes Examples code

slide-42
SLIDE 42