Realistic Image Synthesis Bidirectional Path Tracing & - PowerPoint PPT Presentation

Realistic Image Synthesis Bidirectional Path Tracing & Reciprocity Philipp Slusallek Karol Myszkowski Gurprit Singh Realistic Image Synthesis SS2020 – Bidirectional Path Tracing Philipp Slusallek

Path Sampling Techniques • Different techniques of sampling paths from both sides – Numbers in parenthesis are # of vertices traced from light/camera, resp. – See later, for Many-Light methods (Virtual Point Light (VPL) methods) Realistic Image Synthesis SS2020 – Bidirectional Path Tracing Philipp Slusallek

Results from Different Techniques • Results from tracing 40 paths per pixel Realistic Image Synthesis SS2020 – Bidirectional Path Tracing Philipp Slusallek

Results from Different Techniques • Results from tracing 40 paths per pixel – f): „Problem of insufficient techniques“ for sampling SDS paths Realistic Image Synthesis SS2020 – Bidirectional Path Tracing Philipp Slusallek

BIDIRECTIONAL PATH TRACING Realistic Image Synthesis SS2020 – Bidirectional Path Tracing Philipp Slusallek

Light & Path Tracing • Problem: – Probability of hitting the camera from the light sources is almost zero – Probability of hitting the light source is often also very small • Next Event Estimator: Try to find a direct connections – Non-optimal (e.g. on mirror surface) – Ignores secondary light sources (e.g. via mirror, at caustics) • Approaches: – Bidirectional Path Tracing • Combination of eye and light paths • Weighted MC sampling for best results • Includes Vertex Connection and Merging (VCM, later) – Metropolis-Sampling [Veach´1997] (see later) • Random variation and mutations of bidirectional paths • Very well suited for very complex light paths • Unbiased but relatively complex algorithms • Uneven convergence Realistic Image Synthesis SS2020 – Bidirectional Path Tracing Philipp Slusallek

Bidirectional Path-Tracing • Idea: Combine Paths from Both Sides – Generate path from the light sources and the camera – Connect paths deterministically (every pair of two hit points) • Different probabilities of generating paths – Compute weighted sum of contributions ( → MIS) • References: – Lafortune et al. , Bidirectional Path- Tracing, [CompuGraphics`93] – Veach, Guibas , Bidirectional Estimators for LightTransport, [EGRW´94, Siggraph´95] Realistic Image Synthesis SS2020 – Bidirectional Path Tracing Philipp Slusallek

Solving the Rendering Equation • Von Neumann Expansion of Measurement Equation 𝐽 𝑞 = න 𝑀 𝑓 𝑦 → 𝑦´ 𝐻 𝑦 → 𝑦´ 𝑋 𝑞 𝑦 → 𝑦´ 𝑒𝐵 𝑦 𝑒𝐵 𝑦´ + 𝑇×𝑇 𝑞 ሺ + න 𝑀 𝑓 𝑦 → 𝑦´ 𝐻 𝑦 → 𝑦´ 𝑔 𝑠 𝑦 → 𝑦´ → 𝑦´´ 𝐻 𝑦´ → x´´ 𝑋 𝑦´ 𝑇×𝑇×𝑇 𝑥𝑗𝑢ℎ 𝐻 𝑦, 𝑧 = 𝑊ሺ𝑦, 𝑧ሻ 𝑑𝑝𝑡𝜄 𝑦 𝑑𝑝𝑡𝜄 𝑧 ሻ → 𝑦´´ 𝑒𝐵 𝑦 𝑒𝐵 𝑦´ 𝑒𝐵ሺ𝑦´´ሻ + ⋯ 𝑦 − 𝑧 2 – Independent estimation of all paths with fixed lengths – Bidirectional generation of paths – Weighted MC integration for each term (MIS) – More efficient by reusing costly paths (i.e. visibility samples) multiple times – Typically: One pair of paths per pixel sample Realistic Image Synthesis SS2020 – Bidirectional Path Tracing Philipp Slusallek

Bidirectional Path-Tracing • Notation Realistic Image Synthesis SS2020 – Bidirectional Path Tracing Philipp Slusallek

Bidirectional Path-Tracing • Generating Light Paths (example) – On the light source    ( , ) || || L x N  = e x x x ( , ) p x  x    =     ( , ) || || L x N d dA e x x x x x  A + • Generating Eye Paths (example) – On the eye/camera (via point in the scene)     ( , ) ( , ) || || g y W y N  = y y y y ( , ) p y y G   =      ( , ) ( , ) || || G g y W y N d dA y y y y y y  A + – g(): 1, if point is visible in this direction Realistic Image Synthesis SS2020 – Bidirectional Path Tracing Philipp Slusallek

Bidirectional Path-Tracing • Extension of Paths at Hit Points – Identical for both directions • Reciprocity of BRDF under reflection (but be careful with refraction!) – Use whatever BRDF sampling technique suits best • But must be a joint probability (conditioned on the previous point) – This does include uniform probability on any surface – (But not a point generated from some other point, e.g. due to occl.) • E.g.  =    +  ( ) ( , , ) || | p f x N + r x i 1 x x + 1 1 i i i Realistic Image Synthesis SS2020 – Bidirectional Path Tracing Philipp Slusallek

Bidirectional Probabilities • Probabilities of Paths 𝝆 in Bidirectional Path Tracing – Different locations of vertex connections (see VCM later) – k : length of paths (# of transports or segments) – m: # of vertices generated from light source ( 0 ≤ 𝑛 ≤ 𝑙 + 1 ) • 0: None • 1: Vertex on light source • 2: Vertex on light source and directional sample • Etc. – Similar for paths from the eye – 𝑞 𝑙,𝑛 ሺ𝜌ሻ : Probability to choose path 𝜌 with method (k,m) Camera LS  𝑞 2,1 ሺ𝜌ሻ 𝑞 2,2 ሺ𝜌ሻ 𝑞 2,3 ሺ𝜌ሻ 𝑞 2,0 ሺ𝜌ሻ Realistic Image Synthesis SS2020 – Bidirectional Path Tracing Philipp Slusallek

Mathematical Formulation • Rendering Equation with Area Parametrization p 2 p 0 p 3 p 1 Realistic Image Synthesis SS2020 – Bidirectional Path Tracing Philipp Slusallek

Mathematical Formulation • Path Formulation (π i : Path of length i) ∞ 𝑀 𝜌 𝑗 𝑞 1 , 𝑞 0 ∞ 𝑀 𝜌 𝑗 – 𝑀 𝑞 1 → 𝑞 0 = σ 𝑗=1 = σ 𝑗=1 𝑗−1 𝐻 𝑞 𝑘+1 → 𝑞 𝑘 𝑔ሺ𝑞 𝑘+1 → 𝑞 𝑘 → 𝑞 𝑘−1 𝑒𝐵ሺ𝑞 2 ሻ ⋯ 𝑒𝐵ሺ𝑞 𝑗 ሻ – 𝑀 𝜌 𝑗 = ׬ 𝐵 ⋯ ׬ 𝐵 𝑀 𝑓 ሺ𝑞 𝑗 → 𝑞 𝑗−1 ሻ ς 𝑘=1 𝐵 ׬ • There are i integrals here • Connection Throughput T(π) of a path π 𝑗−1 𝐻 𝑞 𝑘+1 → 𝑞 𝑘 𝑔ሺ𝑞 𝑘+1 → 𝑞 𝑘 → 𝑞 𝑘−1 ሻ – 𝑈 𝜌 𝑗 = ς 𝑘=1 – 𝑀 𝜌 𝑗 = ׬ 𝐵 ⋯ ׬ 𝐵 𝑀 𝑓 ሺ𝑞 𝑗 → 𝑞 𝑗−1 ሻ𝑈ሺ𝜌 𝑗 ሻ𝑒𝐵ሺ𝑞 2 ሻ ⋯ 𝑒𝐵ሺ𝑞 𝑗 ሻ 𝐵 ׬ • With Measurement – 𝐽 = ׬ 𝐵 𝑞𝑗𝑦𝑓𝑚 𝑀 𝑞 1 → 𝑞 0 𝐻 𝑞 1 → 𝑞 0 𝑋 𝑞 1 → 𝑞 0 𝑒𝐵 𝑞 0 𝑒𝐵ሺ𝑞 1 ሻ 𝐵 ׬ – 𝐽 = σ 𝑗 ׬ 𝐵 ⋯ ׬ 𝐵 𝑀 𝑓 ሺ𝑞 𝑗 → 𝑞 𝑗−1 ሻ𝑈 𝜌 𝑗 𝐻 𝑞 1 → 𝑞 0 𝑋 𝑞 1 → 𝑞 0 𝑒𝐵ሺ𝑞 0 ሻ ⋯ 𝑒𝐵ሺ𝑞 𝑗 ሻ 𝐵 ׬ • There are i integrals here Realistic Image Synthesis SS2020 – Bidirectional Path Tracing Philipp Slusallek

Mathematical Formulation • Path Tracing with Russian Roulette ∞ 𝑀ሺ𝜌 𝑗 ሻ = ∞ 𝑀ሺ𝜌 𝑗 ሻ 1 – 𝑀 𝑞 1 → 𝑞 0 = σ 𝑗=1 𝑟 2 σ 𝑗=2 – Continuation of path with probability 𝑟 2 – And similar for higher path lengths • How to choose sample points – Whatever works, e.g. • Area (uniform): ( 𝑞 𝐵 𝑞 𝑗 = 1 σ 𝑘 𝐵 𝑘 ) Τ • Solid angle, depending on direction cos 𝜄 𝑗 𝑠 2 ) Τ ( 𝑞 𝐵 = 𝑞 𝜕 from previous sample: • Any other joint probability that integrates to one over all surfaces and is non-zero where there could be a contribution • Must be a conditional probability, based on the previous point • Splitting of BRDFs or Emissions – Make sure all path are accounted for ! – Make sure no path is counted multiple times, either ! Realistic Image Synthesis SS2020 – Bidirectional Path Tracing Philipp Slusallek

Example • Light tracing (one eye ray, 1st generation only) Realistic Image Synthesis SS2020 – Bidirectional Path Tracing Philipp Slusallek

Example • Standard MC Path Tracing (same number of paths) Realistic Image Synthesis SS2020 – Bidirectional Path Tracing Philipp Slusallek

Example • Contribution of Different Paths [Not shown: direct connection eye to light + all from light] One reflection One step from the eye (plus direct connection to light) Two reflections l: Two steps from the eye r: One step from the eye, one step from light source Three reflections l: Three steps from the eye m: two steps from the eye, one from light source r: one step from the eye, two from the light sources Realistic Image Synthesis SS2020 – Bidirectional Path Tracing Philipp Slusallek

Bidirectional Path-Tracing • Combination of Estimators – Every option of generating a specific path 𝜌 defines its own estimator with given 𝑞 𝑙,𝑛 ሺ𝜌ሻ – Weighted MC sampling provides new combined estimator of a bi- directionally generated path N N  e l = C w ij C ij = = 1 1 i j – 𝑂 𝑓 : # reflections on eye paths – 𝑂 𝑚 : # reflections on light paths – 𝑥 𝑗𝑘 : weights for combination Realistic Image Synthesis SS2020 – Bidirectional Path Tracing Philipp Slusallek

Combination of Estimators • Example: – Four paths between LS and eye – Weighted with three estimators • A, B, C – Selection with maximum heuristics • Choose 𝑞 𝑌 ሺ𝜌ሻ maximum – Area of rectangles is constant across A, B, C • f/p*p – Width corresponds to 𝑞 𝑌 ሺ𝜌ሻ Realistic Image Synthesis SS2020 – Bidirectional Path Tracing Philipp Slusallek