Global Illumination Shadow Layers Franois Desrichard , David - - PowerPoint PPT Presentation

Global Illumination Shadow Layers

François Desrichard, David Vanderhaeghe, Mathias Paulin IRIT, Université de Toulouse, CNRS, INPT, UPS, UT1, UT2J, France

July 12, 2019

Cinematic Lighting Design

Arbitrary Output Variables (AOVs) allow editing without re-rendering We prov rovid ide sh shadow laye yers rs for for com

• mpositing with little

le ove

• verh

rhead

2

Diffuse, specular, normal, depth... Animated geometry Textures, materials Light rig

Scene Composite Render

Shadows

Strong visual cue, subject to artistic expression Tolerant perception of shadow appearance (Hecher et al., 2014, Sattler et al., 2005)

3

Nightmare by Alla Chernova Horizon-Based Ambient Occlusion (HBAO) in Destiny 2

Previous Work

4

Dragging with mouse (Pellacini et al., 2002) On-surface deformation (Ritschel et al., 2010) Rotation, pattern inlay (Obert et al., 2010) Shape simplification (DeCoro et al., 2007)

• Light overestimation
• Empirical matte
• No indirect shadows

Comparison with Available Renderers

5

Main layer = original image Arnold / pbrt / Cycles, no shadow Ours, no shadow Ours, shadow layer Arnold, shadow matte

Comparison with Current Renderers

6

Main layer Arnold, shadow matte Ours, shadow layer Compositing with the shadow layer

Contributions

• Definition of shadow layer under global illumination
• Characterization of the shadow layer in the path space
• Path tracer rendering all layers in a single pass

7

Defining Shadow

Interaction between three components The user defines caster and catcher by their surface How to account for indirect shadows?

8

Light Caster Catcher

Occlusion can be direct or indirect

Screen Space Definition

9

Black body render B Invisible render T Main layer I

As the difference between the two altered renders: S = T - B

Screen Space Definition

10 10

• =

Invisible render T Black body render B Shadow layer S

Adding the shadow layer removes shadows

Screen Space Definition

11 11

= +

Shadow layer with matte No shadow Main layer

Discussion on this Definition

• Agnostic to the rendering algorithm
• Takes into account indirect shadows
• Has a physical meaning: lost radiance
• Self-shadowing cannot be recovered
• No control on light or catcher
• Two additional renders per object

12 12

Path integral formulation for the main layer (Veach, 1997) For the shadow layer of caster C

Translation to the Path Space

13 13

= { all geometric light paths in the scene } = { light paths encountering caster C } = measurement contribution function considering C invisible

• Input: N objects as casters Ci
• Output: the main layer and N shadow layers in a single pass
• The algorithm measures with , but also each with

○ Measuring with ⇔ scattering on all casters ○ Measuring with ⇔ skipping caster Ci

Integration in a Path Tracer

14 14

i i i

}

p = 1 / 2

1. Intersect green

Integration in a Path Tracer

15 15

• 3. Contribute to Sgreen
• 4. Cannot skip blue
• 2. Skip green now

?

Sblue Sgreen

1. Contribute to Sgreen

Integration in a Path Tracer

16 16

• 3. Scatter on blue now
• 4. Contribute to I
• 2. Intersect blue

?

Improved Artistic Control

Per-light separation

17 17

Main layer Separate light sources; shadow ratio I / (I + S)

Improved Artistic Control

Direct and indirect separation

18 18

Main layer Direct and indirect shadow ratio

Improved Artistic Control

Custom catchers and self-shadowing toggle

19 19

Main layer With and without self-shadows

Editing Examples

Balancing shadow strength

20 20

Editing Examples

21 21

+

Hibiscus / Pandanus shadow layer Locally color graded image Main layer

Editing Examples

Shape transformation using the shadow ratio I / (I + S)

22 22

Main layer I (I + S) ⨯ the transformed shadow ratio

Performance Overhead

• Additional intersection tests bring the most overhead
• Managing samples and filtering incurs a per-caster cost
• The sampling budget is now shared among all layers

For 1 to 5 casters, we measure a 1.1 to 1.3 overhead factor

23 23

Limitations

When a shadow is created by multiple casters, it cannot be assigned to

• nly one of them. A possible solution: consider them as one.

With N interacting casters, combinatorial explosion (2N shadow layers)

24 24

Main layer Boxes as separate casters Both boxes as one caster

Limitations

Using the notion of surface to define an object is limiting: how to handle participating media?

25 25

Main layer No scattering No absorption

Conclusion

• Definition of shadow layer under global illumination

○

Editable at the post-processing stage

• Characterization of the shadow layer in the path space

○ Amenable to Monte-Carlo integration algorithms

• Path tracer rendering all layers in a single pass

○ With an overhead in time and render convergence

26 26

Thank you for your attention

https://www.irit.fr/STORM/site/shadow-layers https://github.com/frcsdes/shadow-layers

27 27

References

(DeCoro et al., 2007) DECORO C., COLE F., FINKELSTEIN A., RUSINKIEWICZ S.: Stylized shadows. In Proc. 5th International Symposium on Non-photorealistic Animation and Rendering (New York, NY, USA, 2007), NPAR ’07, ACM, pp. 77–83. doi:10.1145/1274871.1274884. (Hecher et al., 2014) HECHER M., BERNHARD M., MATTAUSCH O., SCHERZER D., WIMMER M.: A comparative perceptual study of soft-shadow algorithms. ACM Trans. Appl. Percept. 11, 2 (July 2014). doi:10.1145/2620029. (Obert et al., 2010) OBERT J., PELLACINI F., PATTANAIK S.: Visibility editing for all-frequency shadow design. Computer Graphics Forum 29, 4 (2010), 1441–1449. doi:10.1111/j.1467-8659.2010.01741.x. (Pellacini et al., 2002) PELLACINI F., TOLE P., GREENBERG D. P.: A user interface for interactive cinematic shadow design. ACM TOG 21, 3 (July 2002), 563–566. doi:10.1145/566654.566617. (Ritschel et al., 2010) RITSCHEL T., THORMÄHLEN T., DACHSBACHER C., KAUTZ J., SEIDEL H.-P.: Interactive on-surface signal

• deformation. ACM TOG 29, 4 (July 2010), 36:1–36:8. doi:10.1145/1778765.1778773.

(Sattler et al., 2005) SATTLER M., SARLETTE R., MÜCKEN T., KLEIN R.: Exploitation of human shadow perception for fast shadow

• rendering. In Proc. 2nd symposium on Applied perception in graphics and visualization (2005), APGV05, Association for Computing

Machinery, p. 131–134. doi:10.1145/1080402.1080426. (Veach, 1997) VEACH E.: Robust Monte Carlo Methods for Light Transport Simulation. PhD thesis, Stanford University, Stanford, CA, USA, 1997. AAI9837162. 28 28

Usually exported using a light path expression: .* <[RT].’tallbox’> .*

• Scattered Layer

29 29

=

Scattered layer Black body render B Main layer I

Scattered Layer

30 30

Participating Media

31 31

Shadow layer test Shadow removal test

Integration in a Path Tracer

32 32

1. Propagation The first time a caster C is encountered, the path now belongs to

• Measure with ⇒ skip the surface and contribute to S

○ C is assigned to the path and always skipped ○ No other caster may be skipped

• Measure with ⇒ scatter normally and contribute to I

○ C will never be skipped ○ Other casters can still be assigned Both outcomes have probability p = 1 / 2 and introduce a factor 1 / p

• 2. Direct lighting

If the path was assigned a caster the shadow ray can skip it, and contributes radiance to the corresponding shadow layer Otherwise:

• An unoccluded shadow ray contributes to I
• An occluded shadow ray can skip a caster that was never

encountered and contribute to its shadow layer Overall, less zero radiance paths

Integration in a Path Tracer

33 33

Future work: locally adapt p to the probability of an encounter?

Skip Probability

34 34

p = 0 p = 1 Global illumination Direct shadows Direct illumination Global shadows p = 1 / 2 Our choice

Scene N Equal sampling Equal time Equal variance

SPP Time SSIM ZRP Time SPP μ-Var Var Time μ-SPP Teaser 2048 15’ 27” 0.903 26% 15’ 2048 0.003 0.01 18’ 24” 2321

1

2048 17’ 27” 0.901 17% 15’ 1824 0.004 0.01 20’ 08” 2318

2

2048 18’ 07” 0.896 12% 15’ 1728 0.004 0.01 21’ 46” 2360

3

2048 18’ 28” 0.896 10% 15’ 1696 0.004 0.01 22’ 14” 2371 Dragon 4096 27’ 53” 0.927 91% 30’ 9856 0.153 0.1 34’ 58” 8122

1

4096 33’ 33” 0.889 90% 30’ 8256 0.153 0.1 41’ 44” 8139 Island 1024 34’ 30” 0.992 87% 30’ 960 0.007 0.01 39’ 36” 1277

1

1024 42’ 35” 0.992 81% 30’ 768 0.007 0.01 51’ 11” 1289

3

1024 46’ 05” 0.992 80% 30’ 704 0.008 0.01 54’ 16” 1310

5

1024 47’ 20” 0.992 79% 30’ 704 0.008 0.01 55’ 56” 1317 Flowers 256 03’ 19” 0.901 15% 10’ 832 0.271 0.5 4’ 01” 291

1

256 03’ 28” 0.899 12% 10’ 784 0.271 0.5 4’ 15” 291

Performance Table

35 35

Performance Table

36 36

Teaser Dragon Island Flowers

Implementation Guidelines

Bidirectional Path Tracing 1. Propagation Altered as path tracing for camera and light sub-paths 1. Integration The connection step can ignore occlusion Only form full paths with coherent propagation history: for instance, a sub-path that scattered on C cannot be connected to one that skipped C

37 37

Implementation Guidelines

Photon Mapping 1. Propagation Altered as path tracing for photons and gathering rays 1. Integration Gathering must remain coherent with propagation history: for instance, photons that scattered on C cannot be gathered by rays that skipped C

38 38

Implementation Guidelines

Metropolis Light Transport The mutation set does not change Propagation and integration are addressed during mutation acceptance 1. Propagation A mutated path is allowed to cross the surface of at most one caster C … 1. Integration … If the resulting history is coherent: it should not scatter on C elsewhere

39 39