Data-Driven Future in Visual Effects Rishabh Battulwar Research and Development at Digital Domain
VFx pipelines Fluids Hair Smoke Fire Rigid Body
Facial Animation What ’ s important? Natural Facial Shapes Realistic Appearance
Facial Motion Highly Non-Linear Organic Person Specific
Facial Animation Goal
Traditional Model Facial Action Coding System (FACS) Codify Facial Expressions Inspired by Facial Musculature GENERIC Facial Shape Basis* Ekman and Friesen (1978) Empirical Interpretations 50-60 Primary Shapes
FACS Model 50-60 Primary Shapes More than 1000 Combination Shapes Animation Rig Setup FACS shapes
FACS Model An example of combination shape in FACS model 50-60 Primary Shapes Primary Shape Primary Shape Hand-CORRECTED Combo Shape More than 1000 Combination Shapes [A+B]* [A] [B]
FACS Model 50-60 Primary Shapes More than 1000 Combination Shapes Animation Rig Setup FACS shapes
FACS Model FACS based process
Geometric Approach Low-Resolution Capture
Geometric Approach MASQUERADE Geometric Mesh Deformer Output
Geometric Approach Correspondence Mapping Performance Transfer using Differential Geometry
Geometric Approach Quick Result ! MASQUERADE Geometric Mesh Deformer Output
Geometric Approach Quick Result ! NO ! Medium-scale & Hi-resolution detail ! NOT ! True to the Person MASQUERADE Geometric Mesh Deformer Output
NO ! NOT ! Medium-scale & True to the Person Hi-resolution detail !
NO ! NOT ! Medium-scale & True to the Person Hi-resolution detail !
NO ! NOT ! Medium-scale & True to the Person Hi-resolution detail !
Data Preparation Raw Data Capture Rigs
Data Preparation Capturing Facial Range of Motion Offline Processing High-Resolution Captures
Data Preparation Capturing Mesoscopic Detail Texture Maps More High-Resolution Captures
Data-driven Shape Model Taking sparse facial capture to high-resolution data Moser et. al. ‘17 - Masquerade MASQUERADE
Medium-scale & True to the Person ! Hi-frequency detail !
Medium-scale & True to the Person ! Hi-frequency detail !
Medium-scale & True to the Person ! Hi-frequency detail !
Issues in Geometric Approach Collapsing Geometry Specific features (lip shapes)
Synthetic Data-driven Corrections on CG Creatures Sculpted Corrections for CG Creatures No FACS-based BlendShape-Rig
Data-driven Corrections on CG Creatures Source Pre-Correction Post-Correction Hendler et. al. ‘17 - Direct Drive
More examples - Wrinkle Map Regression
Comparison Takes 1-2 Weeks Takes Several Months Shape-space built from Shape-space physical data modeled iteratively
Final Result Overview of Data-Driven Facial Animation Markered Large-Scale face input Final High-Resolution Geomteric Deformation using Output Mesh low-resolution capture Training Data!!
Final Result
Acknowledgements Thanks to ~ David Corral, David Mclean, Rickey Cloudsdale, Lucio Moser, Geoff Wedig, Mark Williams, Ron Griswold, Nafees Bin Zafar, Doug Roble, Jeremy Buttell, Derek Crosby, Ron Miller, Darren Hendler and the entire Digital Domain Team! ~
Recommend
More recommend
