Real-time Performance-Based Facial Animation Mark Pauly - PowerPoint PPT Presentation

Real-time Performance-Based Facial Animation Mark Pauly

Eurographics 2012, Cagliari, Italy Introduction 2

Eurographics 2012, Cagliari, Italy Overview Facial Tracking using Blendshape Model • rigid transformation + blendshape weights per frame Personalized Blendshape Model • static face capture • example based facial rigging Tracking • rigid & nonrigid registration • animation prior 3

Eurographics 2012, Cagliari, Italy Eurographics 2012, Cagliari, Italy Animation Pipeline

Eurographics 2012, Cagliari, Italy Facial Expression Model Static Face Capture [Weise et al, 2009] 5

Eurographics 2012, Cagliari, Italy Facial Expression Model Face Fitting [Li et al, 2009] 6

Eurographics 2012, Cagliari, Italy Facial Expression Model Texture Reconstruction Poisson Image Editing [Pérez et al, 2003] 7

Eurographics 2012, Cagliari, Italy Facial Expression Model Example-based Facial Rigging generic template personalized expression model [Li et al, 2010] 8

Eurographics 2012, Cagliari, Italy Rigid Tracking 9

Eurographics 2012, Cagliari, Italy Non-rigid Tracking Estimate blendshape weights • similar to scans & color image • plausible animation MAP estimation • non-rigid registration based on geometry & texture • animation prior from existing animation sequences 10

Eurographics 2012, Cagliari, Italy Eurographics 2012, Cagliari, Italy N-Dim Expression Space

Eurographics 2012, Cagliari, Italy Eurographics 2012, Cagliari, Italy N-Dim Expression Space

Eurographics 2012, Cagliari, Italy Eurographics 2012, Cagliari, Italy N-Dim Expression Space

Eurographics 2012, Cagliari, Italy Eurographics 2012, Cagliari, Italy Animation Manifold

Eurographics 2012, Cagliari, Italy Eurographics 2012, Cagliari, Italy Probabilistic Expression Prior

Eurographics 2012, Cagliari, Italy Eurographics 2012, Cagliari, Italy Probabilistic Animation Prior

Eurographics 2012, Cagliari, Italy Eurographics 2012, Cagliari, Italy Temporal Joint Probabilistic Distribution

Eurographics 2012, Cagliari, Italy Eurographics 2012, Cagliari, Italy MAP Estimation

Eurographics 2012, Cagliari, Italy Demo www.faceshift.com