Reconstruction II Neural Networks in Monte Carlo Rendering Philipp - PowerPoint PPT Presentation

Reconstruction II Neural Networks in Monte Carlo Rendering Philipp Slusallek Karol Myszkowski Gurprit Singh � 1 Realistic Image Synthesis SS2018

Previous Lecture � 2 Realistic Image Synthesis SS2018

Slide from Kartic Subr � 3 Realistic Image Synthesis SS2018

Depth of field Slide from Jakko Lehtinen � 4

1 scanline 
 Slide from Jakko Lehtinen � 5

Lens u Slide from Jakko Lehtinen � 6 Screen x

Visibility: SameSurface The trajectories of 
 samples originating from a single 
 apparent surface never intersect. 
 Slide from Jakko Lehtinen � 7

Hachisuka et al. [2008] � 8 Realistic Image Synthesis SS2018

Hachisuka et al. [2008] � 9 Realistic Image Synthesis SS2018

Sen and Darabi [2012] � 10 Realistic Image Synthesis SS2018

Pixels,Random Params,Features The algorithm computes the statistical dependency of (c-f) on the random parameters in ( b ) Sen and Darabi [2012] � 11 Realistic Image Synthesis SS2018

Gaussian Filtering Paris et al. [2009] � 12 Realistic Image Synthesis SS2018

Bilateral Filtering Paris et al. [2009] � 13 Realistic Image Synthesis SS2018

Bilateral vs Gaussian Filtering Paris et al. [2009] � 14 Realistic Image Synthesis SS2018

À la Carte • Introduction to Multi-Layer perceptrons (Neural Networks) • Machine Learning for Filtering Monte Carlo Noise [Kalantari et al. 2015] � 15 Realistic Image Synthesis SS2018

Motivation Bako et al. [2017] � 16 Realistic Image Synthesis SS2018

History of Neural Networks • In 1943, McCulloch and Pitts created a computational model for neural networks • In 1975, Werbos's back propagation algorithm generally accelerated the training of multi-layer networks. • In 1980s, Recurrent Neural Networks were developed � 17 Realistic Image Synthesis SS2018

Multi-Layer Perceptrons � 18 Realistic Image Synthesis SS2018

Classifiers y j = f ( w j x j + b j ) � 19 Realistic Image Synthesis SS2018

Classifiers y j = f ( w j x j + b j ) � 20 Realistic Image Synthesis SS2018

Complex Classifiers y j = f ( w j x j + b j ) Complex classifier � 21 Realistic Image Synthesis SS2018

Complex Classifiers Complex classifier What features can produce this decision rule ? � 22 Realistic Image Synthesis SS2018

Perceptron Classifier x 1 Classifier x 2 x 3 x 4 x 5 . . . 1 � 23 Realistic Image Synthesis SS2018

Perceptron Classifier w 1 x 1 Classifier x 2 w 2 Output x 3 y = f ( w 1 x 1 + w 2 x 2 + ... + w 0 ) x 4 x 5 . . w 0 . 1 � 24 Realistic Image Synthesis SS2018

Multi-layer Perceptron x 1 1 � 25 Realistic Image Synthesis SS2018

Multi-layer Perceptron f X x 1 f X 1 f X � 26 Realistic Image Synthesis SS2018

Multi-layer Perceptron f X w 11 w 10 x 1 w 21 f X w 20 1 w 31 f X w 30 � 27 Realistic Image Synthesis SS2018

Multi-layer Perceptron f X w 11 w 10 x 1 w 21 f X w 20 1 w 31 f X w 30 + x 1 w 11 w 10 + x 1 w 21 w 20 + x 1 w 31 w 30 � 28 Realistic Image Synthesis SS2018

Multi-layer Perceptron f X w 11 w 10 x 1 w 21 f X w 20 1 w 31 f X w 30 + = f ( ( y 1 x 1 w 11 w 10 ( ( = f + x 1 y 2 w 21 w 20 ( + ( = f x 1 w 31 w 30 y 3 � 29 Realistic Image Synthesis SS2018

Multi-layer Perceptron f X w 11 w 1 w 10 x 1 w 21 w 2 f Output X X w 20 1 w 31 w 3 f X w 30 + = f ( ( y 1 x 1 w 11 w 10 ( ( = f + x 1 y 2 w 21 w 20 ( + ( = f x 1 w 31 w 30 y 3 � 30 Realistic Image Synthesis SS2018

Multi-layer Perceptron Input Hidden layers Output layers features f X w 11 w 1 w 10 x 1 w 21 w 2 f Output X X w 20 1 w 31 w 3 f X w 30 y 1 w 1 + = f ( ( y 2 y 1 x 1 w 11 w 10 w 2 ( ( = f + x 1 y 2 w 21 w 20 y 3 w 3 ( + ( = f x 1 w 31 w 30 y 3 � 31 Realistic Image Synthesis SS2018

Multi-layer Perceptron Input Hidden layers Output layers features f X w 11 "Features" are outputs of perceptrons w 1 w 10 x 1 w 21 w 2 f X Output X w 20 1 w 31 w 3 f X w 30 Perceptrons Matrix of second layer weights Matrix of first layer weights w 1 w 11 w 10 w 2 w 21 w 20 w 3 w 31 w 30 � 32 Realistic Image Synthesis SS2018

Features of MLPs Input features Perceptron: Step function with linear decision boundary � 33 Realistic Image Synthesis SS2018

Features of MLPs 2-layer: These outputs are now input features to the next layer "Features" are now decision boundaries (partitions) Layer 1 All linear combination of those partitions give complex partitions � 34 Realistic Image Synthesis SS2018

Features of MLPs These complex outputs become the features for the new layer Layer 2 Layer 1 � 35 Realistic Image Synthesis SS2018

Features of MLPs Deep Neural Networks Layer 2 Layer 1 � 36 Realistic Image Synthesis SS2018

Computational Graph representation of Neural Networks � 37 Realistic Image Synthesis SS2018

Neural Networks Fully connected layers ReLU W 1 0 x 1 N × N N × 1 � 38 Realistic Image Synthesis SS2018

Neural Networks Fully connected layers ReLU W 1 0 x 1 x 2 N × N N × 1 N × 1 � 39 Realistic Image Synthesis SS2018

Neural Networks Fully connected layers ReLU ReLU W 1 W 2 ... 0 x 1 x 2 N × N N × 1 N × N N × 1 � 40 Realistic Image Synthesis SS2018

Neural Networks data Fully connected layers ReLU ReLU W 1 W 2 ... 0 x 1 x 2 N × N N × 1 N × N N × 1 N represents number of pixels in an image � 41 Realistic Image Synthesis SS2018

Neural Networks Unstructured data Fully connected layers ReLU ReLU W 1 W 2 ... 0 x 1 x 2 Computational ReLU Graph * max � 42 Realistic Image Synthesis SS2018

Neural Networks Unstructured data Fully connected layers ReLU ReLU W 1 W 2 ... 0 x 1 x 2 Computational ReLU ReLU Graph * * ... max max � 43 Realistic Image Synthesis SS2018

Two-layer model R R W 1 W 2 . Fully connected layers ReLU ReLU * * max max What can be a loss function ? � 44 Realistic Image Synthesis SS2018

Two-layer model R R W 1 W 2 . Fully connected layers Reference ReLU ReLU * * max max What can be a loss function ? � 45 Realistic Image Synthesis SS2018

Two-layer model R R W 1 W 2 . Fully connected layers Reference ReLU ReLU * * max max What can be a loss function ? � 46 Realistic Image Synthesis SS2018

Two-layer model R R W 1 W 2 . Fully connected layers Reference ReLU ReLU * * max L2 Loss max What can be a loss function ? � 47 Realistic Image Synthesis SS2018

Two-layer model R R W 1 W 2 . Reference 2 ReLU - - max L2 Loss What can be a loss function ? � 48 Realistic Image Synthesis SS2018

Two-layer model R R W 1 W 2 . Reference 2 ReLU - - max L2 Loss What can be a loss function ? � 49 Realistic Image Synthesis SS2018

Two-layer model: Back propagation Source link � 50 Realistic Image Synthesis SS2018

Two-layer model: Back propagation Gradient Descent Algorithm for back propagation Random initialization Global cost minimum � 51 Realistic Image Synthesis SS2018

Back Propagation Slides courtesy: Stanford Online Course � 52 Realistic Image Synthesis SS2018

Back Propagation Slides courtesy: Stanford Online Course � 53 Realistic Image Synthesis SS2018

Back Propagation Slides courtesy: Stanford Online Course � 54 Realistic Image Synthesis SS2018

Back Propagation Slides courtesy: Stanford Online Course � 55 Realistic Image Synthesis SS2018

Back Propagation Slides courtesy: Stanford Online Course � 56 Realistic Image Synthesis SS2018

Back Propagation Slides courtesy: Stanford Online Course � 57 Realistic Image Synthesis SS2018

Back Propagation Slides courtesy: Stanford Online Course � 58 Realistic Image Synthesis SS2018

Back Propagation Slides courtesy: Stanford Online Course � 59 Realistic Image Synthesis SS2018

Back Propagation Slides courtesy: Stanford Online Course � 60 Realistic Image Synthesis SS2018

Machine Learning for Filtering Monte Carlo Noise Kalantari et al. [SIGGRAPH 2015] � 61 Realistic Image Synthesis SS2018

Reconstruction / Denoising Filter weights , Pixel neighborhood � 62 Realistic Image Synthesis SS2018

Filter weights Filter weights For cross Bilateral filters: Pixel neighborhood � 63 Realistic Image Synthesis SS2018

Filter weights Filter weights For cross Bilateral filters: Pixel neighborhood � 64 Realistic Image Synthesis SS2018

Filter weights Filter weights For cross Bilateral filters: Pixel neighborhood Sen and Darabi [2012] � 65 Realistic Image Synthesis SS2018

Filter weights For cross Bilateral filters: Pixel screen coordinates Mean sample color value Scene features � 66 Realistic Image Synthesis SS2018

Filter weights For cross Bilateral filters: Pixel screen coordinates Mean sample color value What are the optimal parameters ? Scene features � 67 Realistic Image Synthesis SS2018