DeepBRDF: A Deep Representation for Manipulating Measured BRDF - - PowerPoint PPT Presentation

▶
deepbrdf a deep representation for manipulating measured
SMART_READER_LITE
LIVE PREVIEW

DeepBRDF: A Deep Representation for Manipulating Measured BRDF - - PowerPoint PPT Presentation

Apr 29, 2023 397 likes •898 views

DeepBRDF: A Deep Representation for Manipulating Measured BRDF Bingyang Hu 1 Jie Guo 1* Yanjun Chen 1 Mengtian Li 1 Yanwen Guo 1 1 State Key Lab for Novel Software Technology, Nanjing University *guojie@nju.edu.cn Material Apperance + Material

slide-1
SLIDE 1

DeepBRDF: A Deep Representation for Manipulating Measured BRDF

Bingyang Hu1 Jie Guo1* Yanjun Chen1 Mengtian Li1 Yanwen Guo1

*guojie@nju.edu.cn

1State Key Lab for Novel Software Technology, Nanjing University

slide-2
SLIDE 2

Material Apperance

+ Material

slide-3
SLIDE 3

General Reflectance Function

  • A real material’s surface reflectance function is a very complex function of

16 variables.

general reflectance function (GRF)

slide-4
SLIDE 4

Taxonomy

slide-5
SLIDE 5

BRDF

  • Bidirectional Reflectance Distribution Function
  • BRDF 𝑔

describes surface reflection at a point 𝑦 for light incident from

direction 𝜕 𝜄, 𝜒 reflected into direction 𝜕 𝜄, 𝜒

slide-6
SLIDE 6

Raymond et al. – Multi‐Scale Rendering of Scratched Materials using a Structured SV‐BRDF

  • Model. 2016

stanschaap.com

Material Apperance

slide-7
SLIDE 7

Material Apperance

Heitz et al. ‐ Multiple‐Scattering Microfacet BSDFs with the Smith Model, SIGGRAPH 2016

slide-8
SLIDE 8

Material Apperance

Vincent Schussler et al. ‐ Microfacet‐based normal mapping for robust Monte Carlo path tracing, SIGGRAPH Asia 2017

slide-9
SLIDE 9

Material Apperance

Yan et al. –Rendering Specular Microgeometry with Wave Optics, SIGGRAPH 2018

slide-10
SLIDE 10

BRDF Acquisition

2020/10/29 Physically Based Rendering ‐ Jie Guo

Qimaging Retiga 1300 (10‐bit 1300x1300 firewire camera) lamp sample controlled turntable

slide-11
SLIDE 11

Measured BRDF

MERL BRDFs UTIA BRDFs

slide-12
SLIDE 12

Measured BRDF

slide-13
SLIDE 13

‐ Memory footprint ‐ Computational cost

Measured BRDF

+ Accurate Reduce the dimensionality

slide-14
SLIDE 14

A data‐driven reflectance model [Matusik 2003]

Related Work

slide-15
SLIDE 15

On Optimal, Minimal BRDF Sampling for Reflectance Acquisition [Nielsen 2015](IPCA) A data‐driven reflectance model [Matusik 2003]

Related Work

slide-16
SLIDE 16

On Optimal, Minimal BRDF Sampling for Reflectance Acquisition [Nielsen 2015](IPCA) A data‐driven reflectance model [Matusik 2003]

Related Work

An intuitive control space for material appearance [Serrano et.al 2016]

slide-17
SLIDE 17

On Optimal, Minimal BRDF Sampling for Reflectance Acquisition [Nielsen 2015](IPCA) A data‐driven reflectance model [Matusik 2003]

Related Work

An intuitive control space for material appearance [Serrano et.al 2016] Connecting measured brdfs to analytic brdfs by data‐driven diffuse‐specular separation. [Sun et.al 2018]

slide-18
SLIDE 18

On Optimal, Minimal BRDF Sampling for Reflectance Acquisition [Nielsen 2015] A data‐driven reflectance model [Matusik 2003] An intuitive control space for material appearance [Serrano et.al 2016]

Related Work

Linear dimensionality reducer

Connecting measured brdfs to analytic brdfs by data‐driven diffuse‐specular separation. [Sun et.al 2018]

slide-19
SLIDE 19

Related Work

Fitting measured BRDF to analytic models

slide-20
SLIDE 20

Related Work

[Ngan 2005] Experimental Analysis of BRDF Models

Fitting measured BRDF to analytic models

Ward Blinn‐Phong Lafortune Measured

slide-21
SLIDE 21

‐ The fitting process is time‐consuming and unstable. ‐ For some materials, they are not accurate.

Related Work

Ward Blinn‐Phong Lafortune Measured

slide-22
SLIDE 22

Our Approach

  • Deep learning based

dimensionality reducer to explore a nonlinear low‐dimensional manifold for measure BRDFs

slide-23
SLIDE 23

Basic Idea

slide-24
SLIDE 24

Network Architecture

Encoder Decoder

BRDF Slice Convolution Fully connected Residual Fully connected Deconvolution Convolution Convolution Deconvolution Deconvolution

slide-25
SLIDE 25

𝜍 ln 𝜍 𝜗 𝜍 𝜗 1

Network Architecture

Encoder Decoder

BRDF Slice Convolution Fully connected Residual Fully connected Deconvolution Convolution Convolution Deconvolution Deconvolution

slide-26
SLIDE 26

Loss Function

Decoder Encoder

AutoEncoder

slide-27
SLIDE 27

Loss Function

slide-28
SLIDE 28

Geometric Interpretation

slide-29
SLIDE 29

Quality Analysis

 Visual quality comparisons against PCA and improved PCA (IPCA)  Quantitative evaluation in terms of RelAE is provided for each reconstructed result

slide-30
SLIDE 30

Quality Analysis

 Visual quality comparisons against PCA and improved PCA (IPCA)  Quantitative evaluation in terms of RelAE is provided for each reconstructed result

slide-31
SLIDE 31

Quality Analysis

 Visual quality comparisons against PCA and improved PCA (IPCA)  Quantitative evaluation in terms of RelAE is provided for each reconstructed result

slide-32
SLIDE 32

Quality Analysis

  • Reconstruction error comparison of our DeepBRDF against PCA

and IPCA with varying dimensions.

slide-33
SLIDE 33

Quality Analysis

 From left to right in each group of closeups, we compare the method of Sun et al. [SJR18], ours and the reference, with corresponding RelAE.

slide-34
SLIDE 34

 Evaluation on other BRDF data (not in MERL dataset)

Quality Analysis

slide-35
SLIDE 35

Application

 Measured BRDF editing  Single Image BRDF Recovery

Application Applications

slide-36
SLIDE 36

Measured BRDF Editing

Measured BRDF Low‐dim. representation Attributes

DeepBRDF (𝒁 (10D Latent Vector) Diffuse albedo(𝛽) Specular albedo(𝛽) roughness(𝑕)

Perceptual appearance

slide-37
SLIDE 37

Train a Back Propagation (BP) regression network to establish the relationship between 𝒁 (latent vector) and 𝜷𝜷𝒕 ∈ ℝ𝟒, 𝜷𝒆 ∈ ℝ𝟒, 𝒉 ∈ ℝ

Measured BRDF Editing

slide-38
SLIDE 38

Measured BRDF Editing

 Linear interpolation between RED‐METALLIC‐PAINT and RED‐ FABRIC using IPCA and our DeepBRDF, respectively

slide-39
SLIDE 39

Measured BRDF Editing

 Editing diffuse albedo

GRAY‐PLASTIC

slide-40
SLIDE 40

Measured BRDF Editing

Ours [[SGM*16]  Editing the roughness of RED‐PLASTIC

RED‐PLASTIC

slide-41
SLIDE 41

 Editing the roughness of SPECULAR‐YELLOWPHENOLIC with IPCA‐ based representation (bottom row) and DeepBRDF‐based representation (top row) DeepBRDF IPCA

Measured BRDF Editing

slide-42
SLIDE 42

Single Image BRDF Recovery

Image BRDF

?

slide-43
SLIDE 43

Decoder

Image BRDF

DeepBRDF

Single Image BRDF Recovery

slide-44
SLIDE 44

Single Image BRDF Recovery

 A new CNN is trained to map the input image to the latent space

  • f DeepBRDF.
slide-45
SLIDE 45

Single Image BRDF Recovery

 Comparison with the method of Ye et

  • al. [YLD*18] in

homogeneous BRDF recovery.

slide-46
SLIDE 46

Single Image BRDF Recovery

 BRDF recovery results for real‐world images.

slide-47
SLIDE 47

Conclusion

  • We have presented DeepBRDF, a deep‐learning‐based representation for

Measured BRDF.

  • We have apply the DeepBRDF to edit measured BRDFs.
  • We have apply the DeepBRDF to the task of single image BRDF recovery.
slide-48
SLIDE 48

Thank you! Q&A