SLIDE 1
Rishabh Battulwar Research and Development at Digital Domain VFx - - PowerPoint PPT Presentation
Rishabh Battulwar Research and Development at Digital Domain VFx - - PowerPoint PPT Presentation
Data-Driven Future in Visual Effects Rishabh Battulwar Research and Development at Digital Domain VFx pipelines Fluids Hair Smoke Fire Rigid Body Facial Animation What s important? Natural Facial Shapes Realistic Appearance Facial
SLIDE 2
SLIDE 3
SLIDE 4
VFx pipelines
Fluids Fire Smoke Rigid Body Hair
SLIDE 5
SLIDE 6
What’s important?
Facial Animation
Natural Facial Shapes Realistic Appearance
SLIDE 7
Facial Motion
Highly Non-Linear Organic Person Specific
SLIDE 8
Facial Animation Goal
SLIDE 9
Traditional Model
Ekman and Friesen (1978)
Facial Action Coding System (FACS)
50-60 Primary Shapes
Empirical Interpretations GENERIC Facial Shape Basis* Codify Facial Expressions Inspired by Facial Musculature
SLIDE 10
FACS Model
FACS shapes Animation Rig Setup 50-60 Primary Shapes More than 1000 Combination Shapes
SLIDE 11
FACS Model
50-60 Primary Shapes More than 1000 Combination Shapes Primary Shape Primary Shape Hand-CORRECTED Combo Shape
An example of combination shape in FACS model
[A] [B] [A+B]*
SLIDE 12
FACS Model
FACS shapes Animation Rig Setup 50-60 Primary Shapes More than 1000 Combination Shapes
SLIDE 13
FACS Model
FACS based process
SLIDE 14
Geometric Approach
Low-Resolution Capture
SLIDE 15
Geometric Approach
MASQUERADE
Geometric Mesh Deformer Output
SLIDE 16
Geometric Approach
Correspondence Mapping Performance Transfer using Differential Geometry
SLIDE 17
Geometric Approach
MASQUERADE
Geometric Mesh Deformer Output Quick Result !
SLIDE 18
Geometric Approach
MASQUERADE
Geometric Mesh Deformer Output NO ! Medium-scale & Hi-resolution detail ! NOT ! True to the Person Quick Result !
SLIDE 19
NO ! Medium-scale & Hi-resolution detail ! NOT ! True to the Person
SLIDE 20
NO ! Medium-scale & Hi-resolution detail ! NOT ! True to the Person
SLIDE 21
NO ! Medium-scale & Hi-resolution detail ! NOT ! True to the Person
SLIDE 22
Data Preparation
Raw Data Capture Rigs
SLIDE 23
Data Preparation
High-Resolution Captures
Capturing Facial Range of Motion Offline Processing
SLIDE 24
Data Preparation
More High-Resolution Captures
Capturing Mesoscopic Detail Texture Maps
SLIDE 25
MASQUERADE
Data-driven Shape Model
Taking sparse facial capture to high-resolution data
Moser et. al. ‘17 - Masquerade
SLIDE 26
Medium-scale & Hi-frequency detail ! True to the Person !
SLIDE 27
Medium-scale & Hi-frequency detail ! True to the Person !
SLIDE 28
Medium-scale & Hi-frequency detail ! True to the Person !
SLIDE 29
Issues in Geometric Approach
Specific features (lip shapes) Collapsing Geometry
SLIDE 30
Synthetic Data-driven Corrections on CG Creatures
Sculpted Corrections for CG Creatures No FACS-based BlendShape-Rig
SLIDE 31
Data-driven Corrections on CG Creatures
Pre-Correction Post-Correction Source
Hendler et. al. ‘17 - Direct Drive
SLIDE 32
More examples - Wrinkle Map Regression
SLIDE 33
Comparison
Takes Several Months Shape-space modeled iteratively Takes 1-2 Weeks Shape-space built from physical data
SLIDE 34
Final Result
Markered face input Large-Scale Geomteric Deformation using low-resolution capture Training Data!! Final High-Resolution Output Mesh Overview of Data-Driven Facial Animation
SLIDE 35
Final Result
SLIDE 36