Real-Time Raytracing in Virtual Production FRANCESCO GIORDANA - - PowerPoint PPT Presentation

▶
real time raytracing in
SMART_READER_LITE
LIVE PREVIEW

Real-Time Raytracing in Virtual Production FRANCESCO GIORDANA - - PowerPoint PPT Presentation

Jan 19, 2023 8 likes •679 views

Genesis: Real-Time Raytracing in Virtual Production FRANCESCO GIORDANA Realtime Architect MARCO GIORDANO Senior Developer Technicolor has the industrys most prominent portfolio of VFX brands, including MPC, MILL FILM and MR. X, Our VFX

slide-1
SLIDE 1

FRANCESCO GIORDANA Realtime Architect MARCO GIORDANO Senior Developer

Genesis: Real-Time Raytracing in Virtual Production

slide-2
SLIDE 2
slide-3
SLIDE 3

Each studio brings their own unique approach to the market while leveraging Technicolor’s R&D teams to harness the new technological demands of compelling Technicolor has the industry’s most prominent portfolio of VFX brands, including MPC, MILL FILM and MR. X, Our VFX studios add VISUAL WONDER and CREATIVE EXPERTISE to film and episodic.

slide-4
SLIDE 4
slide-5
SLIDE 5

LOS ANGELES VANCOUVER TORONTO MONTREAL LONDON MUMBAI BANGALORE ADELAIDE MPC FILM MR X TECHNICOLOR MILL FILM

VISUAL EFFECTS

GLOBAL FOOTPRINT

slide-6
SLIDE 6

GENESIS

slide-7
SLIDE 7
slide-8
SLIDE 8

VR SCOUTING

slide-9
SLIDE 9

MOCAP

slide-10
SLIDE 10

VIRTUAL CAMERA

slide-11
SLIDE 11

LAYERING

slide-12
SLIDE 12

Phases of Virtual Production

1) Scout the environment, find shots, dress your set 2) Capture performance, add animations 3) Shoot virtual cameras 4) Iterate and layer

slide-13
SLIDE 13

LED WALLS

slide-14
SLIDE 14

LED Walls

slide-15
SLIDE 15

Rendering Challenges

1) Advanced lighting for DP on set 2) Good approximation of final render 3) Final pixels in real time

slide-16
SLIDE 16

AREA LIGHTS

slide-17
SLIDE 17

VXGI

slide-18
SLIDE 18

VOLUMETRIC LIGHTING

slide-19
SLIDE 19

DEPTH OF FIELD

slide-20
SLIDE 20

Not enough

  • Need more parity with lighting in post
  • Need raytracing:
  • Live link with a path tracer
  • Raytracing in engine
slide-21
SLIDE 21

Live-Link with Prism

slide-22
SLIDE 22

S9197 – Prism & RTX

Victor Yudin, Lead Software Developer, Mill Film

Wednesday, Mar 20, 9:00 AM - 09:50 AM SJCC Room 230B (Concourse Level)

slide-23
SLIDE 23

Live link with RenderMan XPU

  • Same framework as Prism (Millefiori)
  • Prism is a delegate for Hydra, can replace

with RenderMan delegate

  • USD enables loading the scene both in

engine and in Millefiori

  • USD Shade for transferring materals, but still

needs lots of work (MaterialX, MDL)

slide-24
SLIDE 24

Raytracing in Unity HDRP

slide-25
SLIDE 25
slide-26
SLIDE 26
slide-27
SLIDE 27
slide-28
SLIDE 28
slide-29
SLIDE 29
slide-30
SLIDE 30

DXR INTEGRATION DETAILS

slide-31
SLIDE 31

The goal:

  • Leverage DXR to improve the quality of our

renders

  • Different FPS targets based on the use case

The challenges:

  • New to DX12 and DXR
  • Small team
  • Unity low level integration
  • New tech, with possible bugs to deal with

RAYTRACING IN GENESIS

slide-32
SLIDE 32

AMBIENT OCCLUSION

slide-33
SLIDE 33

AMBIENT OCCLUSION

First effect implemented Plugin: boils down to a C interface dll Faced the first issues with Unity integration:

  • Data marshalling
  • Synchronization
  • Delayed reaction of editor events
  • Issue with resources lifetime
  • Integration of gameworks denoisers
slide-34
SLIDE 34
slide-35
SLIDE 35
slide-36
SLIDE 36
slide-37
SLIDE 37
slide-38
SLIDE 38
slide-39
SLIDE 39
slide-40
SLIDE 40

AO: TIMINGS

5 10 15 20 25 1 spp 4spp 16 spp

AO

raytracing denoise

1 spp 4spp 16 spp raytracing 2.2 4.3 17.4 denoise 2.1 2.1 2.1

slide-41
SLIDE 41
slide-42
SLIDE 42

HDRP INTEGRATION

  • Unity use an AO renderer manager
  • Harder to integrate without drilling down in

the call stack

  • Opted for simpler blit over Unity texture
  • Render moves on as usual
  • No tricks for increasing contrast
slide-43
SLIDE 43

SOFT SHADOWS

slide-44
SLIDE 44

AMBIENT OCCLUSION

slide-45
SLIDE 45

SOFT SHADOWS

  • Heavy use of area lights
  • Previously using in house solution with

raymarched depth map.

slide-46
SLIDE 46
slide-47
SLIDE 47
slide-48
SLIDE 48
slide-49
SLIDE 49
slide-50
SLIDE 50

HDRP INTEGRATION

  • Did not find an easy way to get screen-space shadows
  • Modified HDRP shader binding extra shadow map
  • Multiply final lighting value by shadow
  • Crude approximation
  • Needed to extract texture from denoiser to C# texture to use

it in Unity

slide-51
SLIDE 51

Soft Shadows

Experimenting with several ways to combine multiple are alights

slide-52
SLIDE 52

AMBIENT OCCLUSION

slide-53
SLIDE 53
slide-54
SLIDE 54

Timings

1 spp 4 spp 16 spp raytracing 3.29 10.41 36.11 denoise 4.08 4.01 4.06

5 10 15 20 25 30 35 40 45 1 spp 4 spp 16 spp

Shadows

raytracing denoise

slide-55
SLIDE 55
slide-56
SLIDE 56

REFLECTIONS

slide-57
SLIDE 57

Reflections

Most tricky effect to deal with:

  • No way to evaluate a Unity shader on hit
  • Marshalling textures and textures lifetime
  • Lighting
slide-58
SLIDE 58
slide-59
SLIDE 59
slide-60
SLIDE 60

Reflections

  • One bounce only
  • No texture marshalling
  • Investigating bindless textures
  • Only tracking directional light for simple

Phong model

  • Extra shadow ray
slide-61
SLIDE 61

Timings

2 4 6 8 10 12 14 16 18 20 1 SPP 4 SPP 16 SPP

Reflections

1 SPP 4 SPP 16 SPP 2.3 3.01 17.9

slide-62
SLIDE 62
slide-63
SLIDE 63

RECAP

  • DXR allowed us to step up quality of renders

quite a bit

  • Steep learning curve for DX12 and DXR
  • Tricky Unity integration
  • Resource life time
  • Editor events
  • Render loop sync
slide-64
SLIDE 64

WHAT IS NEXT

  • Investigate new Unity DXR integration
  • Focus more on effects rather than

plumbing

  • Performance, performance, performance
slide-65
SLIDE 65

Special thanks to the twitter community for RTX and game dev help: Matt Pettineo - @MyNameIsMJP Alex Tardif - @longbool Peter Morley - @Biblo_Shwag Sebastian Altonen - @SebAltonen Sebastien Lagarde - @SebLagarde Kostas Anagnostou - @KostasAAA Jesper Flicks - @catlikecoding Special thanks to Nvidia and the outstanding Microsoft Pix Team

slide-66
SLIDE 66

THANK YOU!

slide-67
SLIDE 67

THE ADVENTURE IS ABOUT TO BEGIN… WE’RE HIRING!

APPLY AT

www.moving-picture.com www.mpc-rnd.com

OR EMAIL

recruitment@moving-picture.com