real time facial animation
play

Real-time Facial Animation Hao Li Mark Pauly ILM EPFL High-End - PowerPoint PPT Presentation

Mar 14, 2024 •157 likes •1.67k views

Real-time Facial Animation Hao Li Mark Pauly ILM EPFL High-End 3D Scanning High-End 3D Scanning Low-Cost Passive Scanning (AGI soft) stereo pair Low-Cost Passive Scanning (AGI soft) stereo pair 3D scan Low-Cost Active Scanning +

  1. Real-time Facial Animation Hao Li Mark Pauly ILM EPFL

  2. High-End 3D Scanning

  3. High-End 3D Scanning

  4. Low-Cost Passive Scanning (AGI soft) stereo pair

  5. Low-Cost Passive Scanning (AGI soft) stereo pair 3D scan

  6. Low-Cost Active Scanning + Temporal Upsampling Microsoft Kinect & Kinect Fusion

  7. Rigging & Animation

  8. Rigging & Animation

  9. Typical Facial Animation Workflow in Industry

  10. Typical Facial Animation Workflow in Industry 3D Scanning

  11. Typical Facial Animation Workflow in Industry Modeling + 3D Scanning Fitting

  12. Typical Facial Animation Workflow in Industry Modeling + Light-weight 3D Scanning Fitting Rigging

  13. Typical Facial Animation Workflow in Industry Modeling + Light-weight Motion 3D Scanning Fitting Rigging Capture

  14. Typical Facial Animation Workflow in Industry Modeling + Light-weight Motion Cleanup & 3D Scanning Fitting Rigging Capture Key-Framing

  15. Typical Facial Animation Workflow in Industry Modeling + Light-weight Motion Cleanup & 3D Scanning Fitting Rigging Capture Key-Framing Modeling

  16. Typical Facial Animation Workflow in Industry Modeling + Light-weight Motion Cleanup & 3D Scanning Fitting Rigging Capture Key-Framing Complex Modeling Rigging

  17. Typical Facial Animation Workflow in Industry Modeling + Light-weight Motion Cleanup & 3D Scanning Fitting Rigging Capture Key-Framing Complex Modeling Retargeting Rigging

  18. Typical Facial Animation Workflow in Industry Modeling + Light-weight Motion Cleanup & 3D Scanning Fitting Rigging Capture Key-Framing Complex Modeling Retargeting Rigging

  19. Typical Facial Animation Workflow in Industry Modeling + Light-weight Motion Cleanup & 3D Scanning Fitting Rigging Capture Key-Framing Complex Key-Framing + Modeling Retargeting Rigging Proc.+Sim.

  20. Markerless Facial Capture

  21. 3D range sensor

  22. 3D range sensor

  23. 3D range sensor Motion can be Captured at the Same Resolution as the Geometry

  24. USC ICT Light Stage 5

  25. USC ICT Light Stage 5

  26. Goal

  27. Goal

  28. Template-Based Tracking

  29. Template-Based Tracking

  30. Template-Based Tracking analyze deformation

  31. Template-Based Tracking analyze deformation

  32. Template-Based Tracking transfer deformation

  33. Template-Based Tracking transfer deformation

  34. Template-Based Tracking transfer deformation

  35. Template-Based Tracking transfer deformation

  36. Correspondences Problem

  37. Correspondences Problem

  38. Correspondences Problem

  39. Correspondences Problem

  40. Non-Rigid Registration

  41. Pair of 3D Scans

  42. Pair of 3D Scans source

  43. Pair of 3D Scans target source

  44. Correspondences are Lost

  45. Correspondences are Lost

  46. Correspondences are Lost ?

  47. Overlapping Regions are Lost

  48. Overlapping Regions are Lost overlapping regions

  49. Overlapping Regions are Lost missing data overlapping regions

  50. Overlapping Regions are Lost

  51. Overlapping Regions are Lost

  52. Non-Rigid Registration

  53. Non-Rigid Registration

  54. Three Ingredients

  55. Three Ingredients source

  56. Three Ingredients source target

  57. Three Ingredients source target

  58. Three Ingredients source target detect overlap

  59. Three Ingredients source target detect overlap

  60. Three Ingredients source target detect correspond overlap

  61. Three Ingredients source target detect correspond overlap

  62. Three Ingredients source target detect correspond deform overlap

  63. Three Ingredients registration target detect correspond deform overlap

  64. Challenges detect correspond deform overlap

  65. Challenges detect correspond deform overlap

  66. Challenges detect correspond deform overlap

  67. Challenges deformation detect correspond deform overlap

  68. Challenges detect correspond deform overlap

  69. Challenges detect correspond deform overlap

  70. Challenges ambiguity detect correspond deform overlap

  71. Challenges detect correspond deform overlap

  72. Challenges detect correspond deform overlap

  73. Challenges detect correspond deform overlap

  74. Challenges detect correspond deform overlap

  75. Challenges ? detect correspond deform overlap

  76. Challenges detect correspond deform overlap

  77. Challenges correspond detect deform overlap

  78. Observation correspond detect deform overlap

  79. Observation correspond detect deform overlap

  80. Observation correspond helps detect deform overlap

  81. Observation correspond helps helps detect deform overlap

  82. Observation correspond detect deform overlap

  83. Observation correspond detect deform overlap

  84. Observation correspond detect deform overlap global optimization via local refinement

  85. Iterative Global Optimization correspond detect deform overlap

  86. Iterative Global Optimization correspond detect overlap deform

  87. Iterative Global Optimization correspond detect overlap deform

  88. Iterative Global Optimization correspond detect overlap deform

  89. Iterative Global Optimization correspond detect overlap deform

  90. Iterative Global Optimization correspond detect overlap deform

  91. Iterative Global Optimization closest point detect overlap deform

  92. Iterative Global Optimization closest point detect overlap deform

  93. Iterative Global Optimization closest point pruning deform

  94. Iterative Global Optimization closest point pruning deform

  95. Iterative Global Optimization closest point pruning deform

  96. Iterative Global Optimization closest point pruning deform

  97. Iterative Global Optimization closest point pruning deform

  98. Iterative Global Optimization closest point pruning global optimization

  99. Iterative Global Optimization closest point pruning global optimization

  100. Iterative Global Optimization closest point pruning global optimization converges?

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Real-time Performance-Based Facial Animation Mark Pauly Eurographics 2012, Cagliari, Italy

Real-time Performance-Based Facial Animation Mark Pauly Eurographics 2012, Cagliari, Italy

Real-time Performance-Based Facial Animation Mark Pauly Eurographics 2012, Cagliari, Italy Introduction 2 Eurographics 2012, Cagliari, Italy Overview Facial Tracking using Blendshape Model rigid transformation + blendshape weights per

542 views • 19 slides
Performance-Driven Facial Animation Performance-based Facial Animation Creating an animation

Performance-Driven Facial Animation Performance-based Facial Animation Creating an animation

Performance-Driven Facial Animation Performance-based Facial Animation Creating an animation of a realistic and expressive human face is one of greatest challenges in computer graphics. The human face is an extremely complex

492 views • 28 slides
Real-time Animation and Real-time Animation and Rendering of Ocean Whitecaps Rendering of Ocean

Real-time Animation and Real-time Animation and Rendering of Ocean Whitecaps Rendering of Ocean

Real-time Animation and Real-time Animation and Rendering of Ocean Whitecaps Rendering of Ocean Whitecaps Jonathan Dupuy, Eric Bruneton INRIA Grenoble Rhne-Alpes, Universit de Grenoble et CNRS, Laboratoire Jean Kuntzmann Universit de

558 views • 26 slides
Real-time 3D Eyelids Tracking From Semantic Edges Quan Wen, Feng Xu, Ming Lu, Jun-Hai Yong

Real-time 3D Eyelids Tracking From Semantic Edges Quan Wen, Feng Xu, Ming Lu, Jun-Hai Yong

Real-time 3D Eyelids Tracking From Semantic Edges Quan Wen, Feng Xu, Ming Lu, Jun-Hai Yong Tsinghua University Background Facial capture and animation is crucial in many applications Face capture in computer games Face animation in movies

944 views • 79 slides
Data-Driven Animation Full-body animation Skin deformation Facial animation Motion

Data-Driven Animation Full-body animation Skin deformation Facial animation Motion

Data-Driven Animation Full-body animation Skin deformation Facial animation Motion Representation A pose is represented by root translation, root orientation, and joint angles q = ( q 1 , , q n ) Motion consists of a

368 views • 24 slides
FMP Pitch Introduction The path I am keen to work with is an action animation and facial

FMP Pitch Introduction The path I am keen to work with is an action animation and facial

Joe Agilada M00486462 FMP Pitch Introduction The path I am keen to work with is an action animation and facial expression animation such as lip syncing. But for my fjnal major project I want to create a VR narrative animation where character

423 views • 13 slides
Facial Expressions & Rigging CSE169: Computer Animation Instructor: Steve Rotenberg UCSD,

Facial Expressions & Rigging CSE169: Computer Animation Instructor: Steve Rotenberg UCSD,

Facial Expressions & Rigging CSE169: Computer Animation Instructor: Steve Rotenberg UCSD, Spring 2016 Facial Muscles Universal Expression Groups Sadness Anger Happiness Fear Disgust Surprise FACS Facial Action

1.03k views • 59 slides
Facial Expressions & Rigging CSE169: Computer Animation Instructor: Steve Rotenberg UCSD,

Facial Expressions & Rigging CSE169: Computer Animation Instructor: Steve Rotenberg UCSD,

Facial Expressions & Rigging CSE169: Computer Animation Instructor: Steve Rotenberg UCSD, Winter 2020 Facial Muscles Universal Expression Groups Sadness Anger Happiness Fear Disgust Surprise FACS Facial

1.09k views • 51 slides
Chapter 8 - Animation Animation before the time of 3DCG Animation techniques

Chapter 8 - Animation Animation before the time of 3DCG Animation techniques

Chapter 8 - Animation Animation before the time of 3DCG Animation techniques keyframing bone animation & motion capture simulation Animation principles LMU Mnchen Medieninformatik Heinrich Hussmann

812 views • 47 slides
Real Time Facial Expression Recognition using Eigen Faces - By Aadesh M Bagmar (U12 CO 092)

Real Time Facial Expression Recognition using Eigen Faces - By Aadesh M Bagmar (U12 CO 092)

Summer Intern Project on Real Time Facial Expression Recognition using Eigen Faces - By Aadesh M Bagmar (U12 CO 092) Project Guide: Dr. M.A.Zaveri Computer Engineering Department Brief The project aims at developing a robust system for real

206 views • 18 slides
WE MAKE DIGITAL HUMANS CONTENTS SCANNING 3D EXTRAS - FACIAL SCANNING - HAIR + CLOTH -

WE MAKE DIGITAL HUMANS CONTENTS SCANNING 3D EXTRAS - FACIAL SCANNING - HAIR + CLOTH -

WE MAKE DIGITAL HUMANS CONTENTS SCANNING 3D EXTRAS - FACIAL SCANNING - HAIR + CLOTH - BODY SCANNING - REAL-TIME INTEGRATION MODELLING/TEXTURING - LIGHTING + RENDERING - CAMERA ANIMATION - MODELLING - HEAD TRACKING -

728 views • 42 slides
animation 1 animation shape specification as a function of time 2 animation representation

animation 1 animation shape specification as a function of time 2 animation representation

animation 1 animation shape specification as a function of time 2 animation representation many ways to represent changes with time intent artistic motion physically-plausible motion efficiency typically not a major problem control most

756 views • 74 slides
Animation structure and content of the internal of changes over time? representation. Why?

Animation structure and content of the internal of changes over time? representation. Why?

Animation Animation: can it facilitate? Congruence Principle A useful graphic is congruent to the Does animation help the understanding Is animation useful? Animation structure and content of the internal of changes over time?

423 views • 4 slides
Realistic Modeling for Realistic Modeling for Facial Animation Facial Animation Yuencheng Lee,

Realistic Modeling for Realistic Modeling for Facial Animation Facial Animation Yuencheng Lee,

Realistic Modeling for Realistic Modeling for Facial Animation Facial Animation Yuencheng Lee, Lee, Demetri Demetri Yuencheng Terzopoulos, and Keith Waters. , and Keith Waters. Terzopoulos Overview Overview Desire Desire A

349 views • 13 slides
Annotated Facial Landmarks in the Wild A Large-scale, Real-world Database for Facial Landmark

Annotated Facial Landmarks in the Wild A Large-scale, Real-world Database for Facial Landmark

Institute for Computer Graphics and Vision, Graz University of Technology, Austria Annotated Facial Landmarks in the Wild A Large-scale, Real-world Database for Facial Landmark Localization Martin Kstinger, Paul Wohlhart, Peter M. Roth,

310 views • 14 slides
Animation Presented by Sancho McCann Animation Is animation useful? Why?

Animation Presented by Sancho McCann Animation Is animation useful? Why?

Animation Presented by Sancho McCann Animation Is animation useful? Why? Principles of animation Principles applied Animation: can it facilitate? Does animation help the understanding of changes over time? A picture

4.12k views • 49 slides
Generating Optimized Code with GlobalISel Or: GlobalISel going beyond "it works" 1

Generating Optimized Code with GlobalISel Or: GlobalISel going beyond "it works" 1

Generating Optimized Code with GlobalISel Or: GlobalISel going beyond "it works" 1 LLVM Dev Meeting 2019 Volkan Keles, Daniel Sanders Apple Agenda What is GlobalISel? GlobalISel Combiner and Helpers Testing and

1.38k views • 123 slides
13.1 Dynamic Geometry Processing I Hao Li http://cs621.hao-li.com 1 Problem Classification 2

13.1 Dynamic Geometry Processing I Hao Li http://cs621.hao-li.com 1 Problem Classification 2

Spring 2019 CSCI 621: Digital Geometry Processing 13.1 Dynamic Geometry Processing I Hao Li http://cs621.hao-li.com 1 Problem Classification 2 Correspondence Classification 3 Correspondence Classification 4 3-D Reconstruction acquisition

817 views • 69 slides
Predicting Secondary Structures of Protein and Global Optimization Piotr Berman and Jieun Jeong

Predicting Secondary Structures of Protein and Global Optimization Piotr Berman and Jieun Jeong

Predicting Secondary Structures of Protein and Global Optimization Piotr Berman and Jieun Jeong DIMACS Workshop June 11, 2005 Page 1 A major goal of bioinformatics: find protein structure (shape) from the

531 views • 24 slides
Global Optimization Global constant propagation Liveness analysis 2 Local

Global Optimization Global constant propagation Liveness analysis 2 Local

Lecture Outline Global flow analysis Global Optimization Global constant propagation Liveness analysis 2 Local Optimization Global Optimization Recall the simple basic-block optimizations These optimizations can be extended

434 views • 14 slides
Global Optimization based on Contractor Programming: Application to H synthesis Jordan Ninin

Global Optimization based on Contractor Programming: Application to H synthesis Jordan Ninin

Introduction Global Optimization Application to H control synthesis under structural constraints Conclusion Global Optimization based on Contractor Programming: Application to H synthesis Jordan Ninin with Dominique Monnet and Benoit

828 views • 61 slides
Image Space Gathering Austin Robison Predictive Rendering Phenomenological Rendering Robustness

Image Space Gathering Austin Robison Predictive Rendering Phenomenological Rendering Robustness

Image Space Gathering Austin Robison Predictive Rendering Phenomenological Rendering Robustness Robustness Speed Speed Controllable Accuracy Appearance Example Phenomena Soft Shadows Depth of Field Motion Blur Color Bleeding

388 views • 24 slides
Computer Graphics Seminar MTAT.03.305 Spring 2019 Raimond Tunnel Previously... v 2 M = M 1

Computer Graphics Seminar MTAT.03.305 Spring 2019 Raimond Tunnel Previously... v 2 M = M 1

Computer Graphics Seminar MTAT.03.305 Spring 2019 Raimond Tunnel Previously... v 2 M = M 1 M 2 M 3 ... Data v 0 v 1 v 2 ( v w ) v x , v y , v z P V M v Vertex v 1 transformations v 0 v w v w v 2 v 2 Vertex shader

1.1k views • 81 slides
Effects needed for Realism Effects needed for Realism Computer Graphics (Spring 2008) Computer

Effects needed for Realism Effects needed for Realism Computer Graphics (Spring 2008) Computer

Effects needed for Realism Effects needed for Realism Computer Graphics (Spring 2008) Computer Graphics (Spring 2008) (Soft) Shadows Reflections (Mirrors and Glossy) COMS 4160, Lecture 16: Ray Tracing Transparency (Water, Glass)

163 views • 5 slides

More recommend