Advances in Real-Time Automotive Visualisation Ch ris OCo n n o r - - PowerPoint PPT Presentation

▶
advances in real time
SMART_READER_LITE
LIVE PREVIEW

Advances in Real-Time Automotive Visualisation Ch ris OCo n n o r - - PowerPoint PPT Presentation

Mar 06, 2023 18 likes •654 views

Z e r o L i g h t a t G T C p r e s e n t s Advances in Real-Time Automotive Visualisation Ch ris OCo n n o r I n t r o d u c t i o n At ZeroLight, weve created the market -leading visualisation and data platform for the au autom

slide-1
SLIDE 1

Advances in Real-Time Automotive Visualisation

Ch ris O’Co n n o r

Z e r o L i g h t a t G T C p r e s e n t s

slide-2
SLIDE 2

At ZeroLight, we’ve created the market-leading visualisation and data platform for the au autom

  • mot
  • tive

ive indu dustr stry

I n t r o d u c t i o n

slide-3
SLIDE 3

Our Content Maintains the Same High Quality and Re Real al-time ime Configurability Across All Platforms

I n t r o d u c t i o n

slide-4
SLIDE 4

I n t r o d u c t i o n

slide-5
SLIDE 5

Ray Tracing and Next-gen VR

I n t r o d u c t i o n

slide-6
SLIDE 6

This Section Will Cover

R a y T r a c i n g

  • Why ray tracing is important
  • Details of custom tech first shown at GTC18
  • Rasterisation and ray tracing
  • DXR and RTX
slide-7
SLIDE 7

What’s Missing from Automotive Real-time Rendering?

R a y T r a c i n g

  • Audi asked us what's next to improve graphics quality?
  • Started development in 2017
  • Content already had passed look and quality control
  • We had to find a solution that adds quality to the current look
slide-8
SLIDE 8

R a y T r a c i n g

Reference image from Audi.de

slide-9
SLIDE 9

Self-reflection Options

R a y T r a c i n g

  • SSR (Screen Space Reflections) – lots of issues
  • Localised Planar Reflections – great for some areas
  • Compute ray tracing – too slow for real-time
  • Voxels reflections – perfect for GPU acceleration
slide-10
SLIDE 10

Voxel Ray-traced Reflections

R a y T r a c i n g

  • GPU can be used to generate voxels
  • Quick ray cast and look up in Pixel Shader
  • Uses a lot of memory
  • Sparse voxels helps a lot
  • Difficult to update dynamically
slide-11
SLIDE 11

Voxel Ray-traced Reflections

R a y T r a c i n g

slide-12
SLIDE 12

Ray Tracing Demo GTC 2018

R a y T r a c i n g

  • We showed our solution at GTC 2018 on the NVIDIA booth
  • Showing the Audi A3 with accessories
  • Rendering 4K at 60FPS
  • 500 million rays per second (0.5 billion)
  • Used 2x GV100 (Volta GPU)
slide-13
SLIDE 13

Voxel Ray-traced Reflections

R a y T r a c i n g

slide-14
SLIDE 14

Ray Tracing Demo GTC 2018

R a y T r a c i n g

slide-15
SLIDE 15

R a y T r a c i n g

slide-16
SLIDE 16

DXR + RTX

R a y T r a c i n g

  • DXR and RTX hardware accelerated ray tracing
  • Resources all managed by DXR
  • Up to 10 billion rays per second
  • Native support: rays and pixels can be dispatched in parallel
  • Denoiser = fewer rays required per pixel
slide-17
SLIDE 17

Rasterisation and Ray Tracing

R a y T r a c i n g

  • Key to maintaining the look across devices that can’t ray trace
  • Let each part of the GPU do what it does best
  • Ray tracing can be used for shadows, AO, GI and reflections
  • Stage 1: graphical improvement without changing the look
  • Stage 2: runs at high quality in real-time (4K @ 60FPS)
slide-18
SLIDE 18

Rasterisation and Ray Tracing

R a y T r a c i n g

slide-19
SLIDE 19

R a y T r a c i n g

slide-20
SLIDE 20

R a y T r a c i n g

slide-21
SLIDE 21

The Future

R a y T r a c i n g

slide-22
SLIDE 22

The Future

R a y T r a c i n g

slide-23
SLIDE 23

The Future

R a y T r a c i n g

slide-24
SLIDE 24

The Future

R a y T r a c i n g

slide-25
SLIDE 25

N e x t - g e n V R

slide-26
SLIDE 26

This Section Will Cover

N e x t - g e n V R

  • Current VR headsets
  • Optimising for next-gen VR
  • Next-gen VR headsets
  • Quality and results
slide-27
SLIDE 27

Current VR Headsets

N e x t - g e n V R

slide-28
SLIDE 28

Current VR Headsets

N e x t - g e n V R

  • 1080x1200 / 1440x1440 resolution per eye
  • 95˚ - 110˚ FOV
  • 90 FPS
  • 2 rendered viewports, one per eye
slide-29
SLIDE 29

Features of New Headsets

N e x t - g e n V R

  • Higher resolution up to 2560x1440 per eye
  • Up to 210˚ FOV
  • 90 FPS
  • Multiple rendered viewports per eye (wide FOV, foveated

rendering)

slide-30
SLIDE 30

VIVE Pro and VIVE Pro Eye

N e x t - g e n V R

slide-31
SLIDE 31

StarVR One, Pimax 8K, XTAL

N e x t - g e n V R

slide-32
SLIDE 32

Varjo VR-1

N e x t - g e n V R

slide-33
SLIDE 33

What's the Challenge?

N e x t - g e n V R

  • If you just brute force render viewports and pixels, even with the

best GPUs, it will be a challenge to hit performance

  • You must take advantage of GPU software-activated hardware

features

  • NVIDIA VR Works with NVAPI
slide-34
SLIDE 34

VR SLI

N e x t - g e n V R

  • Previously, we used AFR (Alternate Frame Rendering)
  • AFR adds additional 11ms of latency (bad for VR)
  • Great for GPU Utilisation (80%+)
  • VR SLI renders one eye per GPU
  • Copy of buffer back to GPU0 is the bottleneck (PCI Express)
  • Easy to instance per eye, reducing CPU overhead
slide-35
SLIDE 35

VR SLI

N e x t - g e n V R

slide-36
SLIDE 36

VR SLI

N e x t - g e n V R

  • We have to use tricks to copy less data and prevent GPUs

waiting for copies

m_pMultiGPUDevice->CopySubresourceRegion(deviceContext, rtcd.m_RenderTargetTexture, 0, dstGPU, rtcd.m_Left, rtcd.m_Top, 0, rtcd.m_RenderTargetTexture, 0, srcGPU, &srcBox, NVAPI_COPY_ASYNCHRONOUSLY);

Copy Data Async

slide-37
SLIDE 37

VR SLI

N e x t - g e n V R

Viewport cut and copy

slide-38
SLIDE 38

Single Pass Stereo

N e x t - g e n V R

  • Reduce CPU overhead and render in a single pass for both eyes
  • Requires shader modifications
  • Mid-frame effects become more complex

//Standard float4 standardPos = mul(MATRIX_MVP, vertex);

  • utPos.position = standardPos;

//Single Pass Stereo float4 eyePos0 = mul(g_ViewProjMatrix[1], worldPos); float4 eyePos1 = mul(g_ViewProjMatrix[2], worldPos);

  • utPos.position = eyePos0;
  • utPos.posClipRight = eyePos1;
  • utPos.ViewportMask = NV_SINGLERT_VIEWPORT_MASK;
slide-39
SLIDE 39

New with Turing GPUs

N e x t - g e n V R

slide-40
SLIDE 40

NVLINK2

N e x t - g e n V R

  • Reduces copy time from 4ms to 0.1ms (StarVR One)
  • We can now share additional data between the GPUs
slide-41
SLIDE 41

Multi-view Rendering

N e x t - g e n V R

  • Very similar to SPS but supports 4 views
  • Big CPU optimisation on HMDs that require >2 Viewports

//Standard float4 standardPos = mul(MATRIX_MVP, vertex);

  • utPos.position = standardPos;

//Single Pass Stereo float4 eyePos0 = mul(g_ViewProjMatrix[1], worldPos); float4 eyePos1 = mul(g_ViewProjMatrix[2], worldPos);

  • utPos.position = eyePos0;
  • utPos.posClipRight = eyePos1;
  • utPos.ViewportMask = NV_SINGLERT_VIEWPORT_MASK;

//Multi-View Rendering float4 eyePos0 = mul(g_ViewProjMatrix[1], worldPos); float4 eyePos1 = mul(g_ViewProjMatrix[2], worldPos); float4 eyePos2 = mul(g_ViewProjMatrix[3], worldPos); float4 eyePos3 = mul(g_ViewProjMatrix[4], worldPos);

  • utPos.position = eyePos0;
  • utPos.position_v1 = eyePos1;
  • utPos.position_v2 = eyePos2;
  • utPos.position_v3 = eyePos3;
  • utPos.ViewportMask = NV_SINGLERT_VIEWPORT_MASK;
  • utPos.ViewportMask2 = NV_SINGLERT_VIEWPORT_MASK_2;
slide-42
SLIDE 42

Variable Rate Shading

N e x t - g e n V R

  • Vary the pixel density of your render target
  • Big fill rate improvements possible
  • Lens-optimised shading and foveated rendering
  • The larger the render target the more that can be saved
  • Supersampling antialiasing makes VR look great
slide-43
SLIDE 43

Variable Rate Shading

N e x t - g e n V R

slide-44
SLIDE 44

Foveated Rendering

N e x t - g e n V R

slide-45
SLIDE 45

Variable Rate Shading

N e x t - g e n V R

slide-46
SLIDE 46

Variable Rate Shading

N e x t - g e n V R

void Plugin_Initialize() { //Initialize NVAPI NvAPI_Initialize(); //Check VRS Support NV_D3D1x_GRAPHICS_CAPS caps = {}; if (NvAPI_D3D1x_GetGraphicsCapabilities(d3d11NativeDevice, NV_D3D1x_GRAPHICS_CAPS_VER, &caps) == NVAPI_OK && caps.bVariablePixelRateShadingSupported) { isTuringCard = true; } } //Post main camera render void Plugin_DisableVRS() { //Create look up table with shading rates and set vrsViewportDescription.numViewports = 0; NvAPI_D3D11_RSSetViewportsPixelShadingRates(deviceContext, &vrsViewportDescription); }

//Update texture using compute shader or render RenderFullScreenWithShader(VRSTexture)

//Pre main camera Render void Plugin_EnableVRS(void* VRStextureResourceView) { //Create look up table with shading rates and set vrsViewportDescription.numViewports = 2; NvAPI_D3D11_RSSetViewportsPixelShadingRates(deviceContext, &vrsViewportDescription); //Set texture to be used as index into shading rate table NvAPI_D3D11_RSSetShadingRateResourceView(deviceContext, VRStextureResourceView); } //Engine main camera render GameEngineCamera.Render();

slide-47
SLIDE 47

Porsche StarVR One

N e x t - g e n V R

slide-48
SLIDE 48

Porsche StarVR One

N e x t - g e n V R

slide-49
SLIDE 49

Porsche StarVR One

N e x t - g e n V R

slide-50
SLIDE 50

Porsche StarVR One

N e x t - g e n V R

slide-51
SLIDE 51

Porsche StarVR One

N e x t - g e n V R

slide-52
SLIDE 52

N e x t G e n V R

slide-53
SLIDE 53

Porsche StarVR One

N e x t G e n V R

slide-54
SLIDE 54

Audi Varjo VR-1

N e x t - g e n V R

slide-55
SLIDE 55

Audi Varjo VR-1

N e x t - g e n V R

slide-56
SLIDE 56

Audi etron Varjo

N e x t G e n V R

slide-57
SLIDE 57

BMW VIVE Pro Eye

N e x t - g e n V R

slide-58
SLIDE 58

BMW VIVE Pro Eye

N e x t - g e n V R

slide-59
SLIDE 59

BMW VIVE Pro Eye

N e x t - g e n V R

slide-60
SLIDE 60

N e x t G e n V R

slide-61
SLIDE 61

Find Out More

N e x t - g e n V R

https://devblogs.nvidia.com/

slide-62
SLIDE 62

Profiling

N e x t - g e n V R

slide-63
SLIDE 63

Fo Follow

  • w Ou

Our Te Tech h Upda dates es on

  • n Tw

Twitt tter er chris. is.oconno

  • connor@z

r@zeroli erolight ght.com .com @Tec echZer hZeroLig

  • Light

ht Qu Ques estio tions? ns? ww www. w.lin linkedin kedin.com .com/i /in/chr n/chris is-zeroli eroligh ght

T h a n k s !