Lecture 16 - Bits & Pieces Welcome! , = (, ) - - PowerPoint PPT Presentation

𝑱 𝒚, 𝒚′ = 𝒉(𝒚, 𝒚′) 𝝑 𝒚, 𝒚′ + න

𝑻

𝝇 𝒚, 𝒚′, 𝒚′′ 𝑱 𝒚′, 𝒚′′ 𝒆𝒚′′

INFOMAGR – Advanced Graphics

Jacco Bikker - November 2018 - February 2019

Lecture 16 - “Bits & Pieces”

Welcome!

Today’s Agenda:

▪ On Offsetting Shadows and Reflections ▪ Consistent Normal Interpolation ▪ Path Classification ▪ Packing Normals ▪ Multiple Importance Sampling, One More Time ▪ Research Directions

You have been told…

…To offset your shadow rays by ‘epsilon times R or L’. ▪ How should be chose epsilon? ▪ Is epsilon the same everywhere? ▪ Is this always sufficient?

Offsets

Advanced Graphics – Bits & Pieces 3 p 𝜕𝑝 𝜕𝑗 𝑂

SafeOrigin

When R (or L) is almost parallel to the surface, the epsilon offset fails. Alternative: 𝑃 += cos 𝜄3 𝜁 𝑆 + 1 − cos 𝜄3 𝜁 𝑂

Offsets

Advanced Graphics – Bits & Pieces 4 p 𝜕𝑝 𝜕𝑗 𝑂

SafeOrigin

When R (or L) is almost parallel to the surface, the epsilon offset fails. Alternative: 𝑃 += cos 𝜄3 𝜁 𝑆 + 1 − cos 𝜄3 𝜁 𝑂

Offsets

Advanced Graphics – Bits & Pieces 5 p 𝜕𝑝 𝜕𝑗 𝑂

Today’s Agenda:

▪ On Offsetting Shadows and Reflections ▪ Consistent Normal Interpolation ▪ Path Classification ▪ Packing Normals ▪ Multiple Importance Sampling, One More Time ▪ Research Directions

The Problem

Vertex normal: the average of the normals of all polygons connected to the vertex. Generating / updating vertex normals in O(N): ▪ set each vertex normal to (0,0,0); ▪ loop over polygons; ▪ add normal of each polygon to each polygon vertex; ▪ normalize all vertex normals.

Normals

Advanced Graphics – Bits & Pieces 7

The Problem

Vertex normal: the average of the normals of all polygons connected to the vertex. Using vertex normals: ▪ Möller-Trumbore yields ‘u,v’ coordinates: these are barycentric coordinates; ▪ calculate 𝑥 = 1 – (𝑣 + 𝑤); ▪ now 𝑂𝑗 = 𝑥 ∗ 𝑂0 + 𝑣 ∗ 𝑂1 + 𝑤 ∗ 𝑂2. Mind the side! Advanced Graphics – Bits & Pieces 8

Normals

The Problem

Vertex normal: the average of the normals of all polygons connected to the vertex. For grazing directions, the reflection may go into the surface. Now what? ▪ Use geometric normal instead of interpolated normal? ▪ Clamp dot between R and N to 0? ▪ Just return black? Advanced Graphics – Bits & Pieces 9

Normals

Advanced Graphics – Bits & Pieces 10

Normals

Advanced Graphics – Bits & Pieces 11

Normals

Consistent Normal Interpolation*

Can we come up with an interpolated normal 𝑜𝑑 that behaves properly? ▪ It is smooth ▪ Reflections in 𝑜𝑑 point away from the surface ▪ If all vertex normals are equal to 𝑜, 𝑜𝑑 = 𝑜 ▪ If 𝑒𝑝𝑢 𝑗, 𝑜 = 1, 𝑜𝑑 = 𝑜𝑞 It turns out that this is indeed possible. The paper outlines how to calculate, for a given 𝑗, a vector 𝑠 that satisfies the constraints. 𝑜𝑑 now is simply 𝑜𝑝𝑠𝑛𝑏𝑚𝑗𝑨𝑓 𝑗 + 𝑠 .

*: Reshetov et al., Consistent Normal Interpolation. ACM Transactions on Graphics, 2010.

Advanced Graphics – Bits & Pieces 12

Normals

Advanced Graphics – Bits & Pieces 13

Normals

Advanced Graphics – Bits & Pieces 14

Normals

Advanced Graphics – Bits & Pieces 15

Normals

Advanced Graphics – Bits & Pieces 16

Normals

Normal Mapping

A normal map can be used to modify normals. Advanced Graphics – Bits & Pieces 17

Normals

Normal Mapping

A normal map can be used to modify normals. This is typically done in tangent space. Problem: the orientation of tangent space matters. We need to align 𝑈 and 𝐶 with the texture, in other words: it needs to be based on the U and V vectors over the surface. This is non-

• trivial. For a derivation of the calculation of

T and B, see: learnopengl.com/Advanced-Lighting/Normal-Mapping Advanced Graphics – Bits & Pieces 18

Normals

Today’s Agenda:

▪ On Offsetting Shadows and Reflections ▪ Consistent Normal Interpolation ▪ Path Classification ▪ Packing Normals ▪ Multiple Importance Sampling, One More Time ▪ Research Directions

Paths

Advanced Graphics – Bits & Pieces 20

Splitting Light Transport

Global illumination consists of: ▪ Direct illumination ▪ Indirect illumination Or: ▪ Direct illumination ▪ Indirect illumination with one diffuse bounce ▪ Indirect illumination with multiple bounces

Paths

Advanced Graphics – Bits & Pieces 21

Splitting Light Transport

Light transport consists of: ▪ All paths of all lengths between the camera and the light. Or: ▪ All paths with zero or one diffuse vertices, plus ▪ all paths with at least two diffuse vertices. Or: ▪ All paths that start with ES (e.g. ESDL), plus ▪ all paths that do not start with ES (e.g. EDDL). If we can isolate a certain class of paths, we can handle this class with a dedicated rendering algorithm.

Paths

Advanced Graphics – Bits & Pieces 22

Heckbert’s Path Notation*

Vertex types: ▪ L: light ▪ E: eye ▪ S: specular ▪ D: diffuse (D)+

• ne or more diffuse vertices

(D)* zero or more diffuse vertices (D)? zero or one diffuse vertices (DS’) a diffuse or a specular vertex

*: Heckbert, Adaptive radiosity textures for bidirectional ray tracing. SIGGRAPH 1990.

Paths

Advanced Graphics – Bits & Pieces 23

Splitting Light Transport

If we can isolate a certain class of paths, we can handle this class with a dedicated rendering algorithm. ESDL: Whitted-style EDL: Whitted-style, with next event estimation EDSL: Photon mapping, direct visualization of the photon map EDD(S*)L: Photon mapping, with final gather We can also do this to parts of the path: ▪ Camera ray: rasterization? ▪ Path segments that gather direct light: NEE ▪ Path segments that gather indirect light: baked?

Paths

Advanced Graphics – Bits & Pieces 24

Splitting Light Transport

If we can isolate a certain class of paths, we can handle this class with a dedicated rendering algorithm. ESDL: Whitted-style EDL: Whitted-style, with next event estimation EDSL: Photon mapping, direct visualization of the photon map EDD(S*)L: Photon mapping, with final gather We can also do this to parts of the path: ▪ Camera ray: rasterization? ▪ Path segments that gather direct light: NEE + filter, small kernel ▪ Path segments that gather indirect light: bounce + filter, large kernel

Paths

Advanced Graphics – Bits & Pieces 25

Splitting Light Transport

If we can evaluate a certain path length using multiple techniques, we can use Multiple Importance Sampling to automatically select the best one. E.g. Next Event Estimation: EDL via explicit or implicit technique. Or, BDPT (book, latest edition, 16.3).

Paths

Advanced Graphics – Bits & Pieces 26

Paths

Advanced Graphics – Bits & Pieces 27

Today’s Agenda:

▪ On Offsetting Shadows and Reflections ▪ Consistent Normal Interpolation ▪ Path Classification ▪ Packing Normals ▪ Multiple Importance Sampling, One More Time ▪ Research Directions

Efficiently Storing Normals

A float color pixel contains r, g and b. Wouldn’t it be convenient if the fourth component could store the normal? ➔ Can we store a normal accurately in 32 bits? Observation: For a normal, we need two components (plus a sign). ➔ Can we store the two components in 16 bits each?

Packing

Advanced Graphics – Bits & Pieces 29

Efficiently Storing Normals

For an extensive study on this topic, see: aras-p.info/texts/CompactNormalStorage.html Bottom line: We can encode a normal in 32-bit, and the precision will be excellent: expect approximately four digits of precision. Pack: uint PackNormal( float3 N ) { float f = 65535.0f / sqrtf( 8.0f * N.z + 8.0f ); return (uint)(N.x * f + 32767.0f) + ((uint)(N.y * f + 32767.0f) << 16); }

Packing

Advanced Graphics – Bits & Pieces 30

Efficiently Storing Normals

For an extensive study on this topic, see: aras-p.info/texts/CompactNormalStorage.html Bottom line: We can encode a normal in 32-bit, and the precision will be excellent: expect approximately four digits of precision. Pack: float3 UnpackNormal( uint p ) { float4 nn = make_float4( (float)(p & 65535) * (2.f / 65535.f), (float)(p >> 16) * (2.f / 65535.f), 0, 0 ); nn += make_float4( -1, -1, 1, -1 ); float l = dot( make_float3( nn.x, nn.y, nn.z ), make_float3( -nn.x, -nn.y, -nn.w ) ); nn.z = l, l = sqrtf( l ), nn.x *= l, nn.y *= l; return make_float3( nn ) * 2.0f + make_float3( 0, 0, -1 ); }

Packing

Advanced Graphics – Bits & Pieces 31

Today’s Agenda:

▪ On Offsetting Shadows and Reflections ▪ Consistent Normal Interpolation ▪ Path Classification ▪ Packing Normals ▪ Multiple Importance Sampling, One More Time ▪ Research Directions

MIS

Method 1: direct light sampling

▪ Samples only the part of the hemisphere

• ccupied by unoccluded light sources.

▪ Does so by checking the visibility of a random point on a random light.

Method 2: BRDF sampling

▪ Samples only the part of the hemisphere not occupied by light sources ▪ Does so by rejecting samples that happen to hit a light source. Method 1 and 2 can (should!) be summed to sample the full hemisphere exactly once.

Advanced Graphics – Bits & Pieces 33

MIS

−𝜌 𝜌 −𝜌 𝜌

𝑞𝑒𝑔 = ቐ 1 𝑡𝑝𝑚𝑗𝑒 𝑏𝑜𝑕𝑚𝑓

angle 𝑏𝑠𝑓𝑏 = න

−𝜌 𝜌

𝑑𝑝𝑡𝜄

𝑞𝑒𝑔 = cos 𝜄 𝜌

Advanced Graphics – Bits & Pieces 34

MIS

න 𝑔 𝑦 𝑒𝑦 = 𝐹 𝑔 𝑦 ≈ 1 𝑂 ෍

𝑗=1 𝑂

𝑔 𝑌𝑗 𝐹 𝑔 𝑦 ≈ 1 𝑂 ෍

𝑗=1 𝑂 𝑔 𝑌𝑗

𝑞 𝑌𝑗

𝑚𝑏𝑠𝑕𝑓 𝑤𝑏𝑚𝑣𝑓 𝑡𝑛𝑏𝑚𝑚 𝑤𝑏𝑚𝑣𝑓 = 𝑤𝑓𝑠𝑧 𝑚𝑏𝑠𝑕𝑓 𝑚𝑏𝑠𝑕𝑓 𝑤𝑏𝑚𝑣𝑓 𝑚𝑏𝑠𝑕𝑓 𝑤𝑏𝑚𝑣𝑓 ≈ 1

Advanced Graphics – Bits & Pieces 35

Using two pdfs: randomly pick one.

MIS

𝐹 𝑔 𝑌 ≈ 1 𝑂 ෍

𝑗=1 𝑂 𝑔 𝑌𝑗

𝑞 𝑌𝑗 𝐹 𝑔 𝑌 ≈ ෍

𝑗=1 𝑂

1 𝑂𝑗 ෍

𝑘=1 𝑂𝑗

𝜕𝑗(𝑌𝑗,𝑘) 𝑔 𝑌𝑗,𝑘 𝑞𝑗 𝑌𝑗,𝑘

This is what we did before. Now, we combine the sampling techniques. Weights: σ 𝜕𝑗 = 1 Most basic weight:

1 𝑂

(i.e., a weighted average of the techniques).

iterate over the available techniques / pdfs draw 𝑂𝑗 samples using technique 𝑗 apply a weight to each sample

Advanced Graphics – Bits & Pieces 36

Weights: σ 𝜕𝑗 = 1, most basic weight:

1 𝑂 . Can we do better?

Objective: minimize variance in the estimator. How about this one: This time, the weight is proportional to the pdf (!).

MIS

𝐹 𝑔 ҧ 𝑦 ≈ ෍

𝑗=1 𝑂

1 𝑂𝑗 ෍

𝑘=1 𝑂𝑗

𝜕𝑗(𝑌𝑗,𝑘) 𝑔 𝑌𝑗,𝑘 𝑞𝑗 𝑌𝑗,𝑘

iterate over the available techniques draw 𝑂𝑗 samples using technique 𝑗 apply a weight to each sample

𝜕𝑗(𝑌) = 𝑂𝑗𝑞𝑗(𝑌) σ𝑙 𝑂𝑙𝑞𝑙(𝑌) … = ෍

𝑗=1 𝑂

𝜕𝑗(𝑌𝑗) 𝑔 𝑌𝑗 𝑞𝑗 𝑌𝑗 𝜕𝑗(𝑌) = 𝑞𝑗(𝑌) σ𝑙 𝑞𝑙(𝑌)

For two pdfs:

𝜕𝐵(𝑌) = 𝑞𝐵(𝑌) 𝑞𝐵(𝑌) +𝑞𝐶 (𝑌) 𝜕𝐶(𝑌) = 𝑞𝐶(𝑌) 𝑞𝐵(𝑌) +𝑞𝐶 (𝑌)

Let’s set 𝑂𝑗 to 1 to simplify things:

We are drawing 𝑌 from 𝑞𝐵, and we are looking up the same 𝑌 in 𝑞𝐶.

Advanced Graphics – Bits & Pieces 37

Summarizing:

𝐺 = ෍

𝑗=1 𝑂

𝜕𝑗(𝑌𝑗) 𝑔 𝑌𝑗 𝑞𝑗 𝑌𝑗 = 𝜕𝐵 𝑌𝐵 𝑔 𝑌𝐵 𝑞𝐵 𝑌𝐵 + 𝜕𝐶 𝑌𝐶 𝑔 𝑌𝐶 𝑞𝐶 𝑌𝐶 where 𝜕𝐵(𝑌) = 𝑞𝐵(𝑌) 𝑞𝐵 𝑌 +𝑞𝐶 𝑌 and 𝜕𝐶(𝑌) = 𝑞𝐶(𝑌) 𝑞𝐵(𝑌) +𝑞𝐶 (𝑌) So: for both sampling strategies, we add to the pdf the pdf of the other strategy for the same point on the hemisphere.

MIS

𝜕𝐵 𝑌𝐵 𝑔 𝑌𝐵 𝑞𝐵 𝑌𝐵 𝜕𝐶 𝑌𝐶 𝑔 𝑌𝐶 𝑞𝐶 𝑌𝐶 = 𝑞𝐶(𝑌𝐶) 𝑞𝐵(𝑌𝐶) +𝑞𝐶 (𝑌𝐶) 𝑔 𝑌𝐶 𝑞𝐶 𝑌𝐶 = 𝑔 𝑌𝐶 𝑞𝐵 𝑌𝐶 +𝑞𝐶 (𝑌𝐶) = 𝑞𝐵(𝑌𝐵) 𝑞𝐵(𝑌𝐵) +𝑞𝐶 (𝑌𝐵) 𝑔 𝑌𝐵 𝑞𝐵 𝑌𝐵 = 𝑔 𝑌𝐵 𝑞𝐵 𝑌𝐵 +𝑞𝐶 (𝑌𝐵)

= 𝑔 𝑌𝐵 𝑞𝐵 𝑌𝐵 + 𝑞𝐶(𝑌𝐵) + 𝑔 𝑌𝐶 𝑞𝐵(𝑌𝐶) + 𝑞𝐶 𝑌𝐶 Advanced Graphics – Bits & Pieces 38

Today’s Agenda:

▪ On Offsetting Shadows and Reflections ▪ Consistent Normal Interpolation ▪ Path Classification ▪ Packing Normals ▪ Multiple Importance Sampling, One More Time ▪ Research Directions

Topics

Path tracing BVH GPGPU

• Filtering

Blue noise Materials

• Large scenes

Subdivision surfaces, hair, displacement maps Many lights Geometry compression

Future Work

Advanced Graphics – Bits & Pieces 40

Today’s Agenda:

▪ On Offsetting Shadows and Reflections ▪ Consistent Normal Interpolation ▪ Path Classification ▪ Packing Normals ▪ Multiple Importance Sampling, One More Time ▪ Research Directions

INFOMAGR – Advanced Graphics

Jacco Bikker - November 2018 – February 2019

END of “Bits & Pieces”

next lecture: “Grand Recap”