Probabilistic illumination-aware filtering for Monte Carlo rendering - - PowerPoint PPT Presentation

Probabilistic illumination-aware filtering for Monte Carlo rendering

Ian C. Doidge Mark W. Jones Swansea University, Wales, UK

Wednesday 12th June Computer Graphics International 2013

Monte Carlo Integration

2

− Path Tracing

− Unbiased − Robust

16384 samples 10 hrs

Monte Carlo Integration

3

− Path Tracing

− Unbiased − Robust − Progressive

− High variance:

− Noise − Converges with

1 sample 2 secs 2 samples 4 secs 4 samples 8 secs 8 samples 17 secs 16 samples 34 secs 32 samples 1 min 10 secs 64 samples 2 mins 30 secs 128 samples 5 mins 256 samples 10 mins 512 samples 20 mins 1024 samples 40 mins 2048 samples 1 hr 20 mins 4096 samples 2 hrs 30 mins 8192 samples 5 hrs 16384 samples 10 hrs

Motivation

− Can exploit pixel correlation − But... often breaks down:

− Texture detail − Illumination discontinuities − Glossy reflections

− Filter but detect & preserve edges

− Use data from rendering − Already available

Path Tracing Reference

Related Work

Outlier rejection:

− For ‘cleaning up’ images − Decoro et al. ‘10, Pajot et al. ’11

General image denoising:

− Kalantari & Sen ‘13

Sample reconstruction:

− Sen & Darabi ’12 − Impressive quality, few samples − Unsuitable for progressive rendering

(Ir)radiance caching:

− Ward ‘88, Kontkanen ’04, Krivanek ‘05, Gassenbauer ’09 − Interpolation bias, limited to low frequency detail

Contributions

− Per-pixel clustering framework

− Incident radiance − Layers to separate vertex contributions − Aids detection of discontinuities

− High intensity noise removal

− Pre-process to filtering

− Probabilistic filtering

− Preserves texture detail − Preserves illumination edges − Suitable for progressive rendering

6

Path Tracing Our Method

Algorithm Overview

7

Trace paths Perform clustering High intensity noise removal Filtered image

Iterate

Low variance buffers Probabilistic filtering Unbiased image Compute vertex contributions Radiance Radiance Radiance layers Incident radiance Incident radiance Irradiance layers

Layer based approach

8

1 2 3 = + +

• =

+ + Radiance Irradiance

Layer based approach

9

1 2 3 = + +

− Irradiance contributions:

− Ignores visible texture and BRDF − Already available!

• =

+ +

− Separate vertex contributions:

− Minimise interference − Simplifies feature detection

Radiance Irradiance

− Utilise texture buffer:

− Incident radiance

radiance

Radiance separation

,

, , ·

• , ,

… , ,

• ,
• ·
• , ,

·

Clustering

11

• v = 1

v = 2 v 3

− 1D Luminance domain:

− Improves performance/storage − H.V.S. sensitive to luminance − Preserves visual discontinuities

• Clustering

12

v = 1 v = 2 v 3

− Clusters at each layer:

− Aggregate samples min max − Represents (ir)radiance for sub-integral − Contiguous, non-overlapping in 1D

• Clustering

13

v = 1 v = 2 v 3

Cluster{ float[3] radiance float[3] irradiance float min float max int numSamples : 31 bits int outlier : 1 bit }

Clustering

14

• v = 1

v = 2 v 3

− Heuristic based on:

− Absolute sample difference − Cluster gradient

− Add new contributions

− Extend existing clusters

Clustering

Clustering

− Based on:

− Absolute sample difference − Cluster gradient

− Add new contributions

− Extend existing clusters

Clustering

− Based on:

− Absolute sample difference − Cluster gradient

− Add new contributions

− Extend existing clusters

− Merge clusters

− Reduce memory costs − Dynamic approximation of integral − Always preserves largest discontinuities

Algorithm Overview

18

Trace paths Perform clustering High intensity noise removal Filtered image

Iterate

Low variance buffers Probabilistic filtering Unbiased image Compute layer contributions Radiance Radiance Radiance layers Incident radiance Incident radiance Irradiance layers

− Key differences:

− Ignore texture variation − Layer based rejection

Lower rejection rates Energy preservation More data

Noise Removal

19

− Density estimation approach

− Image space neighbourhood

− DeCoro et al. ’10

− Pixel-wise estimates

− Pajot et al. ’11

− Filtering Pre-process:

− Prefer sparse data to no data − Accept if isolated and low energy

Illumination-aware filtering

− Extends cross-bilateral filter:

− Eisemann & Durand ’04 − Petschnigg et al. ’04 − Low variance range buffer

− Poisson probability distribution:

− Derive weights from cluster statistics − Probabilistic detection of illumination edges

!

Illumination-aware filtering

21

Filtered image Find similar clusters Radiance Low var. buffers Calculate filter weights Scale cluster contributions Final pixel radiance Incident radiances

− Extend min and max of cluster from pixel p − Find overlapping clusters in

neighbouring contributing pixels, c

− Compute poisson probabilities

using sample counts

− Repeat for each cluster in pixel p

Find similar clusters

• !

6 5 1

− Product of Gaussian, geometric and poisson

weights:

Calculate filter weights

23

Radiance Radiance Low var. buffers Geometric weights Compute

• geom. weights

Radiance Radiance Poisson weights Filter kernel weights · ·

• Gaussian

Geometric Poisson Gaussian weights

Scale cluster contributions

− Radiance is a product of:

− Texture , BRDF coefficient , incident radiance ,

− Preserve detail in from pixel − Filter and using neighbouring pixels − Weight with kernel

· ·

• · ·

·

Results

Path Tracing Our Method

25

Cross-Bilateral Reference

− Multiple coloured lights − Hard and soft gradients − Chromatic and luminance

noise

16 samples per pixel

Results

Path Tracing (22 samples) Reference Equal time comparison after 45 seconds

26

Our method (16 samples)

Results

Reference

27

Cross-Bilateral (Smooth noise) Cross-Bilateral (Preserve texture) Our method Equal time comparison after 45 seconds

− Self-shadowing − Complex geometry

Results

Path Tracing Our method Cross-Bilateral Equal time comparison after 45 seconds

28

Reference

Conclusion

− Fast image-based incident radiance filtering

− Preserve illumination edges and gradients − Preserve texture detail − Remove high intensity noise

− Future Work

− Perceptual clustering heuristic − Layer specific parameters − Automatic kernel radius

29

Thank you for listening! Questions? (and Answers)

30

Thank you for listening! Questions? (and Answers)

31

32

Future Improvements

− Aliasing:

− Yang et al. ’11, Anti-Aliasing Recovery, ACM Trans. Graph.

− Convergence:

− Variance based adaptive filter widths

− Noise removal parameters:

− Difficult to establish for low sample counts − What is the pixels normal distribution?

33