Virtual and Augmented Reality Motion Capture and Animation (01) RNDr. Martin Madaras, PhD. madaras@skeletex.xyz
Outline Principles of animation Keyframe animation Articulated figures Skeletal animation Motion capture Projects… 2
How the lectures should look like #1 Ask questions, please!!! - Be communicative - www.slido.com #VAR01 - More active you are, the better for you! - 3
Manual animation Stop-motion animation e.g. Coraline, Wallace & Gromit, etc. 4
Computer Animation What is animation? Make objects change over time according to scripted actions What is simulation? Predict how object change over time according to physical laws 5
Computer Animation Animation pipeline 3D Modeling Articulation Motion specification Motion simulation Shading Lighting Rendering Postprocessing 6
Keyframe Animation Define character poses at specific time steps called “keyframes” 7
Inbetweening (“ tweening ”) Computing missing values based on existing surrounding values 45 0 8
Inbetweening (“ tweening ”) Linear (constant) Ease-in, ease-out 9
Keyframe Animation Inbetweening: Linear interpolation – usually not enough continuity 10
Spline Continuity How to ensure curves are “smooth” Generally we have three levels of continuity C0 - The curves meet C0 & C1 - The tangents are shared C0 & C1 & C2 - The “speed” is the same 11
C0 Continuity Zero order parametric continuity 12
C0 & C1 Continuity First order parametric continuity 13
C0 & C1 & C2 Continuity Second order parametric continuity 14
Implications for animation Linear interpolation is only C0 Movement changes instantly at keyframes Very unnatural looking We need at least C0 & C1 continuity Hermite interpolation Spline interpolation “Smoothstep” function 15
Smoothstep 16
Smoothstep float smootherstep(float edge0, float edge1, float x) { // Scale, and clamp x to 0..1 range x = clamp((x - edge0)/(edge1 - edge0), 0.0, 1.0); // Evaluate polynomial return x*x*x*(x*(x*6 - 15) + 10); } 17
Keyframe Animation Inbetweening: Spline interpolation – may be visually good enough May not follow physical laws 18
Keyframe Animation Inbetweening: Inverse kinematics or dynamics 19
Articulated Figures Character poses described by set of rigid bodies connected by “joints” 20
Articulated Figures Well suited for humanoid characters 21
Keyframe Animation Inbetweening: Compute angles between keyframes 22
Example: Walk Cycle Inbetweening: Compute angles between keyframes 23
Example: Walk Cycle Hip joint orientation 24
Example: Walk Cycle Knee joint orientation 25
Example: Walk Cycle Ankle joint orientation 26
Animation Hierarchies Animate objects in relation to their parent Sun matrix is Ms Earth matrix is MsMe Moon matrix is MsMeMm 27
Kinematics and Dynamics Kinematics Considers only motion Determined by positions, velocities, accelerations Dynamics Considers underlaying forces Capture motion from initial positions and physics 28
Example: 2-Link Structure Two links connected by rotational joints 29
Forward Kinematics Animators specifies angles Θ 1 and Θ 2 Computer finds position of end effector: X 30
Forward Kinematics Joint motion can be specified by spline curves Joint motion can be specified by spline curves 31
Forward Kinematics Joint motions can be specified by initial conditions Joint motion can be specified by spline curves 32
Example: 2-Link Structure What if animator knows position of “end effector” 33
Inverse Kinematics Animator specifies end effector positions: X Computer finds joint angles Θ 1 and Θ 2 34
Inverse Kinematics End-effector positions can be specified by splines 35
Inverse Kinematics Problem with more complex structures System with equation is usually under-defined Multiple solutions 36
Inverse Kinematics Solution for more complex structures Find best solution (eg. minimize energy in motion) Non-linear optimization 37
Inverse Kinematics Forward Kinematics Specify conditions (joint angles) Compute conditions of end effectors Inverse Kinematics “Goal - directed” motion Specify goal positions of end effectors Compute conditions required to achieve goal 38
Character Animation (Linear Blend Skinning) (Skeletal Animation) 39
Skeletal Animation Hierarchical graph structure called Skeleton Nodes and edges (bones) 40
Skeletal Animation Graph structure can be disconnected in space 41
Real-time skeletal skinning https://www.youtube.com/watch?v=DfIfcQiC2oA 42
Facial animation Facial expressions Lips to speech synchronization Controllers Skinning Morphing http://www.anzovin.com/products/tfm1maya.html 43
Reusable animation One skeleton – different models http://www.studiopendulum.com/alterego/ 44
Motion capture Markers on actor’s body Optical / magnetic sensors 3D reconstruction of markers’ position Motion mapping to virtual character 45
Outline Principles of animation Keyframe animation Articulated figures Skeletal animation Motion capture Projects… 46
How the lectures should look like #2 Ask questions, please!!! - Be communicative - www.slido.com #VAR01 - More active you are, the better for you! - 47
Motion Capture Capturing of actor’s movement Natural animation of CG characters 48
Optical Mocap Estimation of marker positions 49
Optical Mocap Problems Complicated setup Problems with occlusions Limited space 50
Inertial Mocap Inertial Measurement Unit Bone rotations from IMU sensors 51
Mocap Suit Problems Custom implementation of walking algorithm Inaccurate for sliding, complex jumping etc. Recalibration needed over time 52
Mocap Walking Algorithm 53
Walking Algorithm Fail-Case 54
Optinertial Optical-Inertial Synchronization of MoCap Suit with Single Camera Setup for Reliable Position Tracking VISIGRAPP 2018 ( Adam Riečický , Martin Madaras, Michal Piovarči ) 55
Motivation Combine optical and inertial data Use camera image to find the actor in the scene Use data streamed from the mocap suit 56
Main Idea Synchronize inertial mocap suit and camera data Suit joint rotations Camera-based position estimation Overcome disadvantage of each method Get reliable position tracking and mocap rotations How to merge and synchronize the data ? 57
Related work 58
RGB Camera-based Mocap / Tracking Multi-view RGB – sum of spatial Gaussians Cheung et al. 2003 Stoll et al. 2011 RGB camera probabilistic methods Wojek et al. 2009 Optical flow + Chamfer matching Dimitrijevic et al. 2006 Katz and Aghajan 2008 59
Related Hybrid Methods Laser scanner + robot + IMUs Ziegler et al. 2011 Template-based + IMUs + RGB cameras Pons-Moll et al. 2017 von Marcard et al. 2016 Depth camera +IMUs + template Helten et al. 2013 60
Related Work Problems Mesh templates Depth cameras Multiple RGB cameras Complicated setup Position estimation only Our Optinertial system needs only RGB or monochromatic camera and the suit data 61
Optical-Inertial Synchronization 62
Our Idea 63
Optinertial Pipeline Chamfer matching 64
Actor Image Distance transform image Background subtraction / contrast suit color 65
Optinertial Pipeline Template image rendering 66
Mocap Suits High number of IMUs vs. affordable price 67
Virtual Scene Real-time update of the virtual scene Frame rendering Camera calibration / pose estimation 68
Camera Calibration OpenCV ChArUco board Estimation of origin/camera position in world space 69
Base-mesh Rendering SQM algorithm (skeleton, radii) Baerentzen et al. 2012 70
Base-mesh Rendering Actor image database Skinning of base-mesh 71
Optinertial Pipeline Minimizing DT for the best match 72
Actor Movement Detection Find the best match for actor shift vector Distance transform minimization 73
Image-space Optimization Silhouette + kernel integration 74
Optimization Implementation Parallel matrix multiplication on GPU The best match gives the actor motion vector 75
Optinertial Pipeline Skeleton root translation 76
Results - Video 77
Evaluation 78
Evaluation Scene HTC Vive ground truth tracking 79
Evaluation Paths 80
Data Smoothing Post-processing Gaussian filtering 81
Recommend
More recommend
![Advanced Animation [Boukhayma 2015] Topics 1. Advanced & non-rigid capture techniques 2.](https://c.sambuz.com/1011882/advanced-animation-s.jpg)
