Lecture 13 - Bidirectional Welcome! , = (, ) , - - PowerPoint PPT Presentation

β–Ά
lecture 13 bidirectional
SMART_READER_LITE
LIVE PREVIEW

Lecture 13 - Bidirectional Welcome! , = (, ) , - - PowerPoint PPT Presentation

Jun 06, 2023 306 likes β€’834 views

INFOMAGR Advanced Graphics Jacco Bikker - November 2019 - February 2020 Lecture 13 - Bidirectional Welcome! , = (, ) , + , , ,

slide-1
SLIDE 1

𝑱 π’š, π’šβ€² = 𝒉(π’š, π’šβ€²) 𝝑 π’š, π’šβ€² + ΰΆ±

𝑻

𝝇 π’š, π’šβ€², π’šβ€²β€² 𝑱 π’šβ€², π’šβ€²β€² π’†π’šβ€²β€²

INFOMAGR – Advanced Graphics

Jacco Bikker - November 2019 - February 2020

Lecture 13 - β€œBidirectional”

Welcome!

slide-2
SLIDE 2

Today’s Agenda:

β–ͺ Introduction: Forward Path Tracing β–ͺ Virtual Point Lights β–ͺ Photon Mapping β–ͺ Path Guiding

slide-3
SLIDE 3

Backward and Forward Path Tracing

Forward

Advanced Graphics – Bidirectional 3

= + + + + = + + + +

Images: Simon Brown, sjbrown.co.uk/2011/01/03/two-way-path-tracing

t = 2 t = 3 t = 4 t = 5 t = 6 s = 2 s = 3 s = 4 s = 5 s = 6

slide-4
SLIDE 4

Forward Path Tracing

A β€˜normal’ path tracer works back to the lights (valid, Helmholtz). A light tracer or forward path tracer keeps the original propagation direction of light: towards the camera.

Forward

Advanced Graphics – Bidirectional 4

slide-5
SLIDE 5

Forward Path Tracing

A β€˜normal’ path tracer works back to the lights (valid, Helmholtz). A light tracer or forward path tracer keeps the original propagation direction of light: towards the camera. Consequences / issues:

  • 1. β€˜Eye’ must have an area.
  • 2. Or: use Next Event Estimation.
  • 3. If the eye sees a mirror, it will be black.
  • 4. This is a bad idea in an open world scene.
  • 5. Paths hit random pixels (however, on average…).
  • 6. What if the camera is behind glass?

Forward

Advanced Graphics – Bidirectional 5

slide-6
SLIDE 6

Forward Path Tracing

Tracing paths from the light helps when: β–ͺ the light is hard to reach β–ͺ the light cannot be importance sampled (using NEE). Tracing paths from the eye is better when: β–ͺ the camera is hard to reach. Many scenes would benefit from both approaches. Now what? β–ͺ decide on a per-pixel basis? β–ͺ do both and average? (would that even work?) β–ͺ something smarter?

Forward

Advanced Graphics – Bidirectional 6

slide-7
SLIDE 7

Forward Path Tracing

Tracing paths from the light helps when: β–ͺ the light is hard to reach β–ͺ the light cannot be importance sampled (using NEE). Tracing paths from the eye is better when: β–ͺ the camera is hard to reach. Many scenes would benefit from both approaches. Now what? β–ͺ decide on a per-pixel basis? β–ͺ do both and average? (would that even work?) β–ͺ something smarter?

Forward

Advanced Graphics – Bidirectional 7 So… a forward path tracer cannot correctly render a scene in which the camera directly views pure specular objects. Is it possible to construct a scene that cannot be correctly rendered using a backward path tracer?

slide-8
SLIDE 8

Forward Path Tracing

The problem with this scene: β–ͺ When a path hits the torus, it can’t use NEE β–ͺ This is true for light tracing and path tracing. The problematic paths are SDS paths*: E: eye D: diffuse S: specular L: light (a light tracer fails on L(…)SE paths.)

*: Heckbert, Adaptive radiosity textures for bidirectional ray tracing. SIGGRAPH 1990.

Forward

Advanced Graphics – Bidirectional 8

slide-9
SLIDE 9

Path Classification

Advanced Graphics – Bidirectional 9

= + + + + = + + + +

energy returned by 𝑒 = 2, 𝑑 = 2 paths EL - LE energy returned by 𝑒 = 3, 𝑑 = 3 paths EDL - LDE energy returned by 𝑒 = 4, 𝑑 = 4 paths E(D|S)DL L(D|S)DE energy returned by 𝑒 = 5, 𝑑 = 5 paths energy returned by 𝑒 = 6, 𝑑 = 6 paths

Forward

t = 2 t = 3 t = 4 t = 5 t = 6 s = 2 s = 3 s = 4 s = 5 s = 6

slide-10
SLIDE 10

Forward Path Tracing

The problem with this scene: β–ͺ The wood inside the ring benefits from NEE β–ͺ But sometimes much more energy arrives via the metal. Here, NEE correctly samples the direct illumination, but the indirect illumination (via the metal) is poorly represented by the cosine pdf.

Forward

Advanced Graphics – Bidirectional 10

slide-11
SLIDE 11

Today

Paths with high throughput and a low probability yield severe noise. β–ͺ Sometimes it’s better to trace from the light. β–ͺ Sometimes backward nor forward work well. Bidirectional techniques aim to exploit benefits of both.

Forward

Advanced Graphics – Bidirectional 11

slide-12
SLIDE 12

Today’s Agenda:

β–ͺ Introduction: Forward Path Tracing β–ͺ Virtual Point Lights β–ͺ Photon Mapping β–ͺ Path Guiding

slide-13
SLIDE 13

Instant Radiosity*

Idea: Trace 𝑂 particles (where 𝑂 is ~103..105) from the light sources, record non-specular hits. At each bounce, use Russian roulette. Each recorded hit becomes a virtual point light. Now, render the scene as usual (rasterization, or Whitted-style ray tracing). At the first diffuse surface, use the VPLs to estimate indirect light, and the lights themselves for direct illumination. (did we account for all light transport?)

*: A. Keller, Instant Radiosity. SIGGRAPH β€˜97.

VPLs

Advanced Graphics – Bidirectional 13

slide-14
SLIDE 14

Instant Radiosity

VPLs

Advanced Graphics – Bidirectional 14

Images: M. Hasan, SIGGRAPH Asia β€˜09.

slide-15
SLIDE 15

Instant Radiosity

Using VPLs has some interesting characteristics: β–ͺ No noise! Those splotches though… β–ͺ VPLs can bounce: they can represent all indirect light β–ͺ VPLs cannot represent direct light β–ͺ #VPLs < #pixels β–ͺ Evaluating VPLs can be done with or without occlusion β–ͺ VPL visibility can also be evaluated using shadow maps Instant Radiosity is a bidirectional technique: we propagate flux when placing the VPLs, and we propagate im importance when connecting to them. VPLs on glossy surfaces do not work. (why not?)

VPLs

Advanced Graphics – Bidirectional 15

slide-16
SLIDE 16

Today’s Agenda:

β–ͺ Introduction: Forward Path Tracing β–ͺ Virtual Point Lights β–ͺ Photon Mapping β–ͺ Path Guiding

slide-17
SLIDE 17

Photon Mapping*

Idea: what if we got rid of visibility queries altogether? With the photon mapping algorithm, we split rendering in two phases: β–ͺ In phase 1 we deposit flux (Ξ¦) in the scene by tracing a large number of photons; β–ͺ In phase 2, we estimate illumination using the photon map.

*: Henrik Wann Jensen, The photon map in global illumination. Ph.D. dissertation, 1996.

Photons

Advanced Graphics – Bidirectional 17

slide-18
SLIDE 18

Photon Mapping

Phase 1: propagating flux. Photon emission: β–ͺ Point light: emitted in uniformly distributed random directions from the point. β–ͺ Area light: emitted from random positions on the square, with directions limited to a hemisphere. The emission directions are chosen from a cosine distribution. All photons thus have the same power: their density is the only way to express varying brightness.

Photons

Advanced Graphics – Bidirectional 18

slide-19
SLIDE 19

Photon Mapping

Phase 1: propagating flux. Surface interaction: A photon that hits a surface may get absorbed or reflected (Russian roulette).

Photons

Advanced Graphics – Bidirectional 19

slide-20
SLIDE 20

Photon Mapping

Phase 1: propagating flux. Photon storage: At each non-specular path vertex we store the photon:

struct photon { float3 position; // world space position of the photon hit float3 power; // current power level for the photon float3 L; // incident direction };

A photon may be stored multiple times along its path before it gets absorbed. Since the total set of photons represents the illumination, we divide photon power by the total number of stored photons.

Photons

Advanced Graphics – Bidirectional 20

slide-21
SLIDE 21

Photon Mapping

Phase 2: radiance estimation. In the second pass, we render the scene using rasterization or Whitted-style ray tracing; the photon map is used to estimate illumination. At each non-specular path vertex we estimate the reflected radiance: 𝑀 𝑦, πœ•π‘ = ΰΆ±

Ω𝑦

𝑔

𝑠 𝑦, πœ•π‘—, πœ•π‘ 𝑀𝑗 𝑦, πœ•π‘— cos πœ„π‘— π‘’πœ•π‘—

This requires information about the radiance 𝑀𝑗 𝑦, … arriving over the hemisphere Ω𝑦. We estimate this radiance by looking at the photon density at 𝑦: 𝑀 𝑦, πœ•π‘ β‰ˆ 1 πœŒπ‘ 2 ෍

π‘ž=1 𝑂

𝑔

𝑠 𝑦, πœ•π‘ž, πœ•π‘ βˆ†Ξ¦ 𝑦, πœ•π‘ž

Photons

Advanced Graphics – Bidirectional 21

slide-22
SLIDE 22

Photon Mapping

Phase 2: radiance estimation. We estimate this radiance by looking at the photon density at 𝑦: 𝑀 𝑦, πœ•π‘ β‰ˆ 1 πœŒπ‘ 2 ෍

π‘ž=1 𝑂

𝑔

𝑠 𝑦, πœ•π‘ž, πœ•π‘ βˆ†Ξ¦ 𝑦, πœ•π‘ž

Note: β–ͺ We assume that we gathered photons on a disc of radius 𝑠. β–ͺ We assume that the gathered photons belong to the same surface. β–ͺ Each photon within radius 𝑠 has the same influence on the estimate.

Photons

Advanced Graphics – Bidirectional 22

slide-23
SLIDE 23

Photon Mapping

Phase 2: radiance estimation. Instead of using the same weight for each photon we can use a filter: π‘₯π‘žπ‘• = 𝛽 1 βˆ’ 1 βˆ’ π‘“βˆ’π›Ύ π‘’π‘ž

2

2𝑠2

1 βˆ’ π‘“βˆ’π›Ύ , where 𝛽 = 0.918, 𝛾 = 1.953*. Value π‘’π‘ž

2 is the squared distance between photon π‘ž and 𝑦.

Now: 𝑀 𝑦, πœ•π‘ β‰ˆ ෍

π‘ž=1 𝑂

𝑔

𝑠 𝑦, πœ•π‘ž, πœ•π‘ βˆ†Ξ¦ 𝑦, πœ•π‘ž π‘₯π‘žπ‘• .

*: Mark J. Pavicic, Convenient Anti-Aliasing Filters that Minimize Bumpy Sampling. In Graphics Gems I.

Photons

Advanced Graphics – Bidirectional 23

slide-24
SLIDE 24

Photon Mapping

Algorithm characteristics: β–ͺ Low-frequent noise β–ͺ Can be used in a rasterizer β–ͺ Can be used for direct + indirect β–ͺ Still a bidirectional technique

Photons

Advanced Graphics – Bidirectional 24

slide-25
SLIDE 25

Photon Mapping

Algorithm characteristics: β–ͺ Low-frequent noise β–ͺ Can be used in a rasterizer β–ͺ Can be used for direct + indirect β–ͺ Still a bidirectional technique

Photons

Advanced Graphics – Bidirectional 25

slide-26
SLIDE 26

Today’s Agenda:

β–ͺ Introduction: Forward Path Tracing β–ͺ Virtual Point Lights β–ͺ Photon Mapping β–ͺ Path Guiding

slide-27
SLIDE 27

Path Guiding

Advanced Graphics – Bidirectional 27

slide-28
SLIDE 28

Path Guiding

Advanced Graphics – Bidirectional 28

slide-29
SLIDE 29

Advanced Graphics – Bidirectional

Path Guiding

The Way of the Photon

Previously in ADVGR: β–ͺ We importance sampled β–ͺ Aiming for the important samples β–ͺ Blending strategies when needed β–ͺ Going bidirectional if all else fails. Now, what if I told you… There’s a new way. ☺

slide-30
SLIDE 30

Advanced Graphics – Bidirectional

Path Guiding

The Way of the Photon

Importance sampling without prior knowledge:

  • 1. Initialize an array of eight floats

with β€˜EPSILON’

  • 2. Compute a discrete pdf based on

the array

  • 3. Sample proportional to this pdf
  • 4. Add the sampled value to the

array

  • 5. Repeat

Notes:

  • 1. This thing learns.
  • 2. This thing produces the wrong
  • answer. (why?)

` Solution: Use a non-zero uniform base probability for all bins.

slide-31
SLIDE 31

Advanced Graphics – Bidirectional

Path Guiding

The Way of the Photon

slide-32
SLIDE 32

Advanced Graphics – Bidirectional

Path Guiding

The Way of the Photon

slide-33
SLIDE 33

Advanced Graphics – Bidirectional

Path Guiding

The Way of the Photon

slide-34
SLIDE 34

Advanced Graphics – Bidirectional

Path Guiding

Learning Light Transport*

Idea: β–ͺ Store incoming flux at a small number of locations in a scene. β–ͺ Each record stores a small set of directions over the hemisphere. β–ͺ Each path/surface interaction updates nearby records. β–ͺ Path extensions use nearby records to form a discrete pdf. Note: The records merely provide a pdf; as long as it is not zero, it yields the correct result (on averageβ„’). Challenges: β–ͺ Record placement. β–ͺ Updating the records during rendering.

*: Learning Light Transport the Reinforced Way. Dahm, Keller, 2017, and: Q-learned Importance Sampling for Physically Based Light Transport on the GPU. Mastrigt, 2018.

slide-35
SLIDE 35

Advanced Graphics – Bidirectional

Path Guiding

Path Guiding*

Idea: β–ͺ Store incoming flux in a 5D kD-tree (β€˜SD-tree’). β–ͺ 3 spatial dimensions, β–ͺ 2 spherical dimensions stored as a quadtree. Algorithm:

  • 1. Render 1 spp without guiding.
  • 2. Store information about each surface interaction in the SD-tree.
  • 3. Throw away the 1spp, and render 2spp with guiding.
  • 4. Update the SD-tree.
  • 5. Throw away the 2spp, and render 4spp with improved guiding.
  • 6. Repeat.

*: Practical Path Guiding for Efficient Light-Transport Simulation. MΓΌller et al., 2017.

slide-36
SLIDE 36

Advanced Graphics – Bidirectional

Path Guiding

Path Guiding*

Idea: β–ͺ Store incoming flux in a 5D kD-tree (β€˜SD-tree’). β–ͺ 3 spatial dimensions, β–ͺ 2 spherical dimensions stored as a quadtree. Algorithm:

  • 1. Render 1 spp without guiding.
  • 2. Store information about each surface interaction in the SD-tree.
  • 3. Throw away the 1spp, and render 2spp with guiding.
  • 4. Update the SD-tree.
  • 5. Throw away the 2spp, and render 4spp with improved guiding.
  • 6. Repeat.

*: Practical Path Guiding for Efficient Light-Transport Simulation. MΓΌller et al., 2017.

slide-37
SLIDE 37

Advanced Graphics – Bidirectional

Path Guiding

slide-38
SLIDE 38

Advanced Graphics – Bidirectional

Path Guiding

slide-39
SLIDE 39

Advanced Graphics – Bidirectional

Path Guiding

slide-40
SLIDE 40

Advanced Graphics – Bidirectional

Path Guiding

slide-41
SLIDE 41

Advanced Graphics – Bidirectional

Path Guiding

Materials

Main paper: Practical Path Guiding for Efficient Light-Transport Simulation. MΓΌller et al., 2017. Be sure to check out the 2019 SIGGRAPH course:

https://cgg.mff.cuni.cz/~jirka/path-guiding-in-production/2019/presentations/Guiding_In_Production_Course_s2019-2019-08-07-ppg-in-production.pdf

And related materials: https://cgg.mff.cuni.cz/~jirka/path-guiding-in-production/2019/index.htm Of particular interest: Efficient data structures and sampling of many light sources for Next Event Estimation, Andreas Mikolajewski (master thesis, 2018).

slide-42
SLIDE 42

10,989 lights

slide-43
SLIDE 43
slide-44
SLIDE 44

Advanced Graphics – Bidirectional

Path Guiding

1spp ref

slide-45
SLIDE 45

Advanced Graphics – Bidirectional

Path Guiding

1spp + SVGF ref

slide-46
SLIDE 46

Advanced Graphics – Bidirectional

Path Guiding

method ref

slide-47
SLIDE 47

Advanced Graphics – Bidirectional

Path Guiding

method + SVGF ref

slide-48
SLIDE 48

Advanced Graphics – Bidirectional

Path Guiding

ref

slide-49
SLIDE 49

Advanced Graphics – Bidirectional

Path Guiding

method + SVGF

slide-50
SLIDE 50

Today’s Agenda:

β–ͺ Introduction: Forward Path Tracing β–ͺ Virtual Point Lights β–ͺ Photon Mapping β–ͺ Path Guiding

slide-51
SLIDE 51

INFOMAGR – Advanced Graphics

Jacco Bikker - November 2019 – February 2020

END of β€œBidirectional”

next lecture: β€œBRDFs”