Between Renderers with MDL Jan Jordan Software Product Manager MDL - - PowerPoint PPT Presentation

Jan Jordan Software Product Manager MDL Lutz Kettner Director Advanced Rendering and Materials

Sharing Physically Based Materials Between Renderers with MDL

March 18, GTC San Jose 2019

2

Agenda

Introduction to NVIDIA Material Definition Language MDL Matching the appearance of a single material within different rendering techniques Defining physically-based materials MDL ecosystem Become part of the ecosystem

3

Introduction

4

The NVIDIA Material Definition Language (MDL) is technology developed by NVIDIA to define physically-based materials for physically-based rendering solutions.

5

Iray Photoreal

6

Iray Photoreal

7

Iray Photoreal

8

courtesy Harley Davidson

9

10

11

NVIDIA vMaterials with Iray Photoreal

12

Iray Photoreal

13

MDL in Substance Designer

14

Substance Designer

15

Matching the Appearance of a Single Material Within Different Rendering Techniques

16

Pathtracer Interactive Raytracer Realtime Rasterizer

One Scene for Different Renderers

Share scene and MDL materials for a consistent look Switching renderers with no scene modifications

17

Iray Realtime OpenGL Rasterizer Iray Interactive Ray Tracer, Direct Illumination Iray Photoreal Path Tracer

18

Material Implementation Material Definition Texturing

Traditional Shading Language Parts

Texture lookups Procedurals Uv-transforms Projectors Noise functions Math functions Light loops / trace N rays OIT / ray-continuation Ray marching Glossy reflection Transparency Translucency

19

Rasterizer Declarative Material Definition Procedural Program- ming Language

Texture lookups Procedurals Uv-transforms Projectors Noise functions Math functions Light loops / OIT Glossy reflection Transparency Translucency

Renderer

Raytracer

Trace N rays

Pathtracer

Ray-marching

20

Rasterizer Declarative Material Definition Procedural Program- ming Language

Light loops / OIT

Renderer

Raytracer

Trace N rays

Pathtracer

Ray-marching

21

Rasterizer

Light loops / OIT

Renderer

Raytracer

Trace N rays

Pathtracer

Ray-marching

MDL is not a Shading Language MDL defines what to compute, not how to compute it – no programmable shading – no light loops or access to illumination – no trace call – no sampling – no camera dependence Declarative Material Definition Procedural Program- ming Language

22

material volume geometry surface

MDL Material Model

backface

…

23

material volume geometry surface

MDL Material Model

scattering

bsdf

backface

…

24

material volume geometry surface emission

MDL Material Model

scattering

bsdf

intensity emission

edf

backface

…

25

material volume geometry surface emission

MDL Material Model

scattering

bsdf

intensity emission

edf

scattering_coefficient absorption_coefficient scattering

vdf

backface

…

26

material volume geometry surface emission

MDL Material Model

scattering

bsdf

intensity emission

edf

scattering_coefficient absorption_coefficient scattering

vdf

cutout_opacity displacement normal

backface

…

27

material volume geometry surface emission

MDL Material Model

thin_walled ior scattering

bsdf

intensity emission

edf

scattering_coefficient absorption_coefficient scattering

vdf

cutout_opacity displacement normal

backface

…

28

Diffuse Transmission Specular Reflection

• Spec. Refl.+Transm.

Measured BSDF Glossy (various) Backscatter Glossy

Bidirectional Scattering Distribution Functions

MDL Elemental Distribution Functions

Diffuse Reflection

29

Emissive Distribution Functions Volume Distribution Functions

MDL Elemental Distribution Functions

Henyey-Greenstein Diffuse Spot IES Profile

30

MDL Distribution Function Modifiers

Tint Thin Film Directional Factor Measured Curve Factor

31

MDL Distribution Functions Combiners

Normalized Mix Clamped Mix Weighted Layer Fresnel Layer Measured Curve Layer Custom Curve Layer

32

MDL Layered Material Example

34

Defining Physically-based Materials With Source Code

35

struct material { bool thin_walled; material_surface surface; material_surface backface; color ior; material_volume volume; material_geometry geometry; };

MDL is a ‘C’ like language. The material viewed as a struct

Defining a Material Using MDL

36

struct material { bool thin_walled; material_surface surface; material_surface backface; color ior; material_volume volume; material_geometry geometry; }; struct material_surface { bsdf scattering; material_emission emission; };

MDL is a ‘C’ like language. The material and its components viewed as a struct

Defining a Material Using MDL

37

struct material { bool thin_walled = false; material_surface surface = material_surface(); material_surface backface = material_surface(); color ior = color(1.0); material_volume volume = material_volume(); material_geometry geometry = material_geometry(); }; struct material_surface { bsdf scattering = bsdf(); material_emission emission = material_emission(); };

MDL is a ‘C’ like language. The material and its components viewed as a struct

Defining a Material Using MDL

38

material();

Material struct is already fully defined

Defining a Material Using MDL

39

Material struct is already fully defined

material();

Defining a Material Using MDL

40

Creating new materials

material name ( material-parameters ) = material ( material-arguments );

Defining a Material Using MDL

42

material plaster( ) = material( surface: material_surface( scattering: df::diffuse_reflection_bsdf() ) );

Defining a Material Using MDL

43

material plaster ( color plaster_color = color(.7)) = material( surface: material_surface ( scattering: df::diffuse_reflection_bsdf ( tint: plaster_color ) ) );

New materials can have parameters

Defining a Material Using MDL

44

material plastic( color diffuse_color = color(.15,0.4,0.0), float roughness = 0.05 ) = material( surface: material_surface( scattering: df::fresnel_layer ( ior: color(1.5), layer: df::simple_glossy_bsdf ( roughness_u: glossy_roughness ), base: df::diffuse_reflection_bsdf ( tint: diffuse_color ) ) ) );

Create complex materials by layering

Defining a Material Using MDL

46

January 10th update: more on procedural texturing and displacement Upcoming: advanced volumes

MDL Handbook

www.mdlhandbook.com

47

MDL Procedural Programming Language

Function results feed into material and function parameters “Shader graphs” are equivalent to function call graphs

C-like language for function definitions

value

color_constructor texture_coordinate

texture_space`: 0 position

summed_perlin_noise

plaster_color

Material plaster

48

Defining a Function Using MDL

float summed_perlin_noise ( float3 point, int level_count=4, float level_scale=0.5, float point_scale=2.0, bool turbulence=false) { float scale = 0.5, noise_sum = 0.0; float3 level_point = point; for (int i = 0; i < level_count; i++) { float noise_value = perlin_noise(level_point); if (turbulence) noise_value = math::abs(noise_value); else noise_value = 0.5 + 0.5 * noise_value; noise_sum += noise_value * scale; scale *= level_scale; level_point *= point_scale; } return noise_sum; }

MDL Handbook Functions allow control flow like loops, switches, conditionals

49

Defining a Function Using MDL

material perlin_noise_material() = plaster( plaster_color: color( summed_perlin_noise( point: state::texture_coordinate(0) ) ) )

Call graph of functions substitute shader graphs

value

color_constructor texture_coordinate

texture_space`: 0 position

summed_perlin_noise

plaster_color

Material plaster

50

MDL Module System

MDL is a programming language allowing dependencies among modules and materials import nvidia::vMaterials::Design::Metal::chrome::*; We use search paths to resolve imports

MDL is program code

51

MDL Module System

MDL is a programming language allowing dependencies among modules and materials import nvidia::vMaterials::Design::Metal::chrome::*; We use search paths to resolve imports C:\Users\Jan\Documents\mdl\nvidia\vMaterials\Design\Metal\chrome.mdl

MDL is program code

search path MDL package space nvidia::vMaterials::Design::Metal::chrome

52

.mdl

MDL 1.5 Preview

One material – fully self contained in one file Includes textures, previews, etc. in the file Renaming and copying works … work just like textures

MDL Encapsulated File Format (MDLE)

unpack refactor MDL SDK .mdle main.mdl resources/… thumbnails/… .jpg .jpg .mdl

53

MDL 1.5 Preview

Localization of all MDL string annotations Based on OASIS standard XLIFF 1.2: XML Localisation Interchange File Format

http://docs.oasis-open.org/xliff/xliff-core/xliff-core.html

Package and module XLIFF files in MDL file hierarchy Example C:\Users\%USERNAME%\Documents\mdl\ MDL search path nvidia\vMaterials\fr.xlf French vMaterial package XLIFF file nvidia\vMaterials\AEC\Glass\Mirror_fr.xlf French Mirror module XLIFF file

Internationalization (i18n)

54

MDL 1.5 Preview

Hair shading

struct material { … hair_bsdf hair; }; hair_bsdf chiang_hair_bsdf { float diffuse_reflection_weight = 0.0; color diffuse_reflection_tint = color(1.0); float2 roughness_R = float2(0.0); float2 roughness_TT = roughness_R; float2 roughness_TRT = roughness_TT; float cuticle_angle = 0.0; color absorption_coefficient = color(); float ior = 1.55; };

Combined hair BSDF model

55

MDL 1.5 Preview

Microfacet coloring to support flip-flop car paints and more

1D measured curve (MDL >=1.4) 2D measured curve (new in MDL 1.5)

56

Material Catalogs Designed for Parallelism Measured Materials

Additional MDL Benefits

Spatially Varying BRDF AxF from X-Rite Measure Isotropic BSDF Modules and packages Archives Little data dependencies Side-effect free functions

57

MDL Ecosystem

58

MDL – Past, Present and Future

2011 2012 2013 2014 2015 2016 2017 2018 2019

MDL 0.x MDL 1.5 MDL 1.0 Iray 2013 MDL 1.1 Public Specification Catia V6 Bunkspeed MDL 1.4 Open Source SDK Public SDK Unreal Studio 4.20 Advisory Council Holodeck Solidworks Visualize MDL 1.3 Vray Adobe Dimension ESI IC.IDO vMaterials Nvidia Iray Plugins MDL 1.2 JIT Compile mental ray (3ds May, Maya) Substance Designer Daz 3d Megascans Vizoo

59

MDL Advisory Council

Joint direction of MDL and the MDL eco system Include expertise other companies have gained in the field and with MDL

Companies sharing our vision of MDL

60

NVIDIA Iray

Iray 2019 roadmap

Iray RTX 2019

• Release in May
• RTX support, up to 5 times speedup!
• MDL 1.5 support for
• MDLE
• localization
• 2d measured curve

61

SOLIDWORKS Visualize

MDL import since 10/2018, tweaking + viewport preview coming

62

Epic Unreal Studio

MDL support through DATASMITH

“Real-time workflows for enterprise” www.unrealengine.com/studio

63

Vizoo xTex

MDL export in the next release

“Vizoo is the number one supplier of Soft-and Hardware solutions for the physically accurate digitization of material swatches in the fashion industry.“

www.vizoo3d.com

64

MEDULR

Online MDL editor and material library preparing for opening

Discover, create and share materials. We're building a global community to create the worlds largest material library.

www.medulr.com

65

Quixel Megascans

“Incredible scans and tools for creatives.”

quixel.com

66

Substance Designer Adobe Dimensions

Creation Libraries/Scan Rendering Applications inc Renderers

Unreal Studio Adobe Stock Vray Max/Maya NVIDIA Holodeck Substance Source NVIDIA VMaterials NVIDIA Iray UE4

MDL Ecosystem

Chaosgroup Vray Iray DCC Plugins Dassault Catia V6 Daz 3D Studio Solidworks Vis. Siemens NX11 Other Iray Prod‘s Vizoo xTex Quixel Megascans medulr

67

Focus on Material Exchange

Freely choose where to author material content

Substance Designer Iray for Rhino

create consume modify

Chaosgroup V-RAY

68

NVIDIA vMaterials 1.6 – SIGGRAPH 2019

~1700 MDL materials verified for accuracy - FREE TO USE

69

70

Become Part of the Ecosystem

71

Become Part of the Ecosystem

Integrate MDL enabled renderer

MDL is included

Write your own compiler

Based on the freely available MDL Specification

Use the MDL SDK

Published under the NVIDIA Designworks License and …

72

Write Your Own Compiler

MDL Specification

Language specification document Free to use http://www.nvidia.com/mdl/

MDL conformance test suite

Syntactic conformance tests Semantic conformance tests Available on request

73

https://developer.nvidia.com/designworks

RENDERING

Multi-Display Capture SDK Warp and Blend GPUDirect for Video Video Codec SDK Iray SDK OptiX SDK MDL SDK vMaterials NV Pro Pipeline GVDB Voxels VXGI GRID SW MGMT SDK NVAPI/NVWMI PhysX

PHYSICS VOXELS VIDEO MANAGEMENT DISPLAY

74

MDL SDK 2019 (.0.1)

MDL 1.4 (1.5 feature previews) DB for MDL definitions DAG view on materials

several compilation modes

MDL editing Code generators

PTX, LLVM IR, x86, HLSL, GLSL (fcts.

• nly)

Distiller and texture baker Samples Documentation and tutorials

Features

Editor Renderer API Samples Distill MDL source Database of content Generate code Bake textures Docs MDL SDK Compile Material Resolve, parse, store Optimized DAG view on material

75

MDL SDK 2019 – What is New

Preview of MDL 1.5 features:

• Localisation
• MDL encapsulated format

Improved BSDF reference implementation (libbsdf) including measured brdf and emissive distribution functions Additional distilling mode (transmissive PBR) HLSL backend (2019.0.1) Automatic derivatives for texture filtering Open source release available on Github

• Includes exclusive MDL core compiler API

More samples

• Updated CUDA sample for transmissive

materials

• (CPU rendering sample)?
• MDL browser sample

Features

76

MDL and RTX

• Anisotropic glossy reflections
• True refractive and volumetric materials
• Measured BRDF
• Proper translucency
• Complex glossy lobe shape and color

MDL materials make RTX shine!

Materials tricky for todays game engines become feasible with RTX

77

MDL SDK and RTX

Microsoft DXR

• HLSL back-end with MDL SDK 2019.0.1

and sample path tracer in the SDK

NVIDIA OptiX

• PTX back-end since MDL SDK 2018.1

sample program available as part of Optix 5.1 & 6 Integrating MDL with an RTX based renderer is simple!

The MDL SDK directly generates material code for use in RTX enabled renderer

78

Automatic Derivatives for Texture Filtering

OptiX sample renderer integration: Derivatives off

79

Automatic Derivatives for Texture Filtering

OptiX sample renderer integration: Derivatives on

80

MDL in Realtime Rendering

1. Ubershader 2. Compilation: on-demand shader generation 3. Distillation to fixed material model All based on MDL SDK

Three approaches

81

Distillation to Fixed Material Model

Fixed Material Model MDL Material

Complex BSDF layering Complex procedurals Simple BSDF structure One texture per parameter

Distillation

82

Distillation to Fixed Material Model

Fixed Material Model MDL Material

Complex BSDF layering Complex procedurals Simple BSDF structure One texture per parameter

Distillation

fct bsdf bsdf fct fct fct fct fct fct fct fct bsdf bsdf fct fct fct fct bsdf bsdf bsdf

83

Distillation to Fixed Material Model

Fixed Material Model MDL Material

Complex BSDF layering Complex procedurals Simple BSDF structure One texture per parameter

Distillation

tex tex tex tex tex

Material

diffuse specular glossy kurtosis normal fct bsdf bsdf fct fct fct fct fct fct fct fct bsdf bsdf fct fct fct fct bsdf bsdf bsdf

84

Distillation to Fixed Material Model

Fixed Material Model MDL Material

Complex BSDF layering Complex procedurals Simple BSDF structure One texture per parameter

Distillation

tex tex tex tex tex

Material

diffuse specular glossy kurtosis normal fct bsdf bsdf fct fct fct fct fct fct fct fct bsdf bsdf fct fct fct fct bsdf bsdf bsdf

Approximate render result: Some materials will look quite different

85

Distillation to Fixed Material Model

Fixed Material Model MDL Material

Complex BSDF layering Complex procedurals Simple BSDF structure One texture per parameter

Distillation

tex tex tex tex tex

Material

diffuse specular glossy kurtosis normal fct bsdf bsdf fct fct fct fct fct fct fct fct bsdf bsdf fct fct fct fct bsdf bsdf bsdf

Fast projection of material instances: Realtime editing Approximate render result: Some materials will look quite different

86

Distillation to Fixed Material Model

Fixed Material Model MDL Material

Complex BSDF layering Complex procedurals Simple BSDF structure One texture per parameter

Distillation

tex tex tex tex tex

Material

diffuse specular glossy kurtosis normal fct bsdf bsdf fct fct fct fct fct fct fct fct bsdf bsdf fct fct fct fct bsdf bsdf bsdf

Fast projection of material instances: Realtime editing Approximate render result: Some materials will look quite different Flexible framework to target different fixed models not a fixed MDL subset (no “MDL lite”)

87

Distillation to Fixed Material Model

Results on vMaterials

• riginal

Fresnel( glossy, diffuse) diffuse-only

88

MDL Distilling

Multiple distilling targets (diffuse only, diffuse_glossy, UE4, new: transmissive PBR)

Released as part of Iray/MDL SDK

Original: Iray MDL Projection: Dassault Stellar with Enterprise PBR

89

May the Source Be with You

https://github.com/NVIDIA/MDL-SDK BSD 3-clause license Full MDL SDK

• 48 modules, 570 files, 310 KLOC
• Excluding

MDL Distilling and texture baking GLSL compiler back-end

• Added MDL Core API
• Includes MDL Core Definitions and more

4 releases shipped since SIGGRAPH 2018

NVIDIA Open Sourced the MDL SDK

Feature image courtesy of Adobe, created by art director Vladimir Petkovic.

90

MDL Core API

Higher-level API for easy integration Reference counted interfaces Mutable objects In-memory store Texture and resource importer API close to the compiler Objects managed in arenas Immutable objects Stateless compiler Callbacks

A Lower-level Compiler API in the MDL SDK MDL SDK API MDL Core API

91

Starting Material MDL Ecosystem What is MDL

MDL Takeaways

Declarative Material Definition Procedural Programming Language Open Source release MDL Specification MDL Handbook MDL SDK MDL Backend Examples Conformance Test Suite NVIDIA vMaterials MDL Advisory Council

92

Further Information on MDL

Documents

NVIDIA Material Definition Language

฀ Technical Introduction ฀ Handbook ฀ Language Specification

GTC On-Demand

• n-demand-gtc.gputechconf.com

www.nvidia.com/mdl raytracing-docs.nvidia.com/mdl/index.html MDL@GTC

Mon 9 AM SJCC 230B

Sharing Physically Based Materials Between Renderers with MDL Integrating the NVIDIA Material Definition Language MDL in Your Application A New PBR Material Serving Mobile, Web, Real-Time Engines and Ray Tracing Multi-Platform Photo-Real Rendering: Utilizing NVIDIA'S MDL and Allegorithmic's Substance Suite for Product Imaging Real-Time Ray Tracing with MDL Materials

Mon 10 AM SJCC 230B Mon 11 AM Hilton Hotel Almaden 2 Tue 9 AM Hilton Hotel Almaden 2 Thu 10 AM SJCC 230C