SLIDE 1
Example-Based Skeleton Extraction Scott Schaefer Can Yuksel - - PowerPoint PPT Presentation
Example-Based Skeleton Extraction Scott Schaefer Can Yuksel - - PowerPoint PPT Presentation
Example-Based Skeleton Extraction Scott Schaefer Can Yuksel Example-Based Deformation Examples Previous Work Mesh-based Inverse Kinematics [Sumner et al. 2005], [Der et al. 2006] Example-based deformation method Non-linear
SLIDE 2
SLIDE 3
Previous Work
Mesh-based Inverse Kinematics [Sumner et al.
2005], [Der et al. 2006]
Example-based deformation method Non-linear minimization Not real-time
Image taken from [Sumner et al. 2005]
SLIDE 4
Objective
Determine parameters for skeletal animation
from set of examples
Bone transformations Vertex weights Bone connectivity Joint locations Root node
Examples
SLIDE 5
Previous Work
Skeleton from shape decomposition
[Katz, Tal 2003] [de Aguilar et al. 2004] [Theobalt et al. 2004] [Katz et al. 2005] [Lien et al. 2006]
Image taken from [Lien et al. 2006]
SLIDE 6
Previous Work
Skeleton from shape decomposition
[Katz, Tal 2003] [de Aguilar et al. 2004] [Theobalt et al. 2004] [Katz et al. 2005] [Lien et al. 2006]
Image taken from [Lien et al. 2006]
SLIDE 7
Previous Work
Example-based skinning
[Wang et al. 2002] [Mohr, Gleicher 2003]
Image taken from [Wang et al. 2002]
SLIDE 8
Previous Work
Skinning Mesh Animations [James, Twigg
2005]
Robust face clustering for bone estimation Non-negative least squares for weights Compresses animations for display on GPU No connectivity/hierarchy/joint positions
Image taken from [James, Twigg 2005]
SLIDE 9
Previous Work
Skeletons from real world data
[Kurihara, Miyata 2004] [Anguelov et al. 2004] [Kirk et al. 2005]
Image taken from [Kirk et al. 2005] Image taken from [Kurihara, Miyata 2004]
SLIDE 10
Outline
Bone estimation Skinning Finding bone connectivity Estimating joint locations
SLIDE 11
Outline
Bone estimation Skinning Finding bone connectivity Estimating joint locations
SLIDE 12
Outline
Bone estimation Skinning Finding bone connectivity Estimating joint locations
SLIDE 13
Outline
Bone estimation Skinning Finding bone connectivity Estimating joint locations
SLIDE 14
Outline
Bone estimation Skinning Finding bone connectivity Estimating joint locations
SLIDE 15
Bone Estimation
Cluster faces that transform in the same rigid
manner
SLIDE 16
Rigid Error Functions
∑∫
− +
= i t k i i T I R R
dt t p T t Rp
T
2 ,
) ( ) ( min T t Rpi + ) (
SLIDE 17
Rigid Error Functions
T t Rpi + ) ( p R p T
k −
= ∑∫ ∑∫ =
i t i t k i
dt dt t p k
p
) (
where
SLIDE 18
Rigid Error Functions
T t Rpi + ) (
k i k i k i t T k i i
p t p t p dt t p t p M − = ∫ = ) ( ) ( ˆ where ) ( ˆ ) ( ˆ 0
SLIDE 19
Rigid Error Functions
T t Rpi + ) (
I R R S S RS M
T T
= = = and where
[Shoemake et al. 1992], [Alexa et al. 2000], [Müller et al. 2005]
SLIDE 20
Rigid Error Functions
Eigen decomposition of 4x4 matrix [Horn 1987] T t Rpi + ) (
SLIDE 21
Rigid Error Functions
Compact representation (17 floats) T t Rpi + ) (
SLIDE 22
Skin Weights
Weight properties:
Match the motion of the example poses Translation invariant Four or less bone
weights per vertex
Positive weights
∑
+ =
i i i i
T p R p ) ( ˆ α
SLIDE 23
Skin Weights
SLIDE 24
Skin Weights
SLIDE 25
Skin Weights
SLIDE 26
Skin Weight Validation
SLIDE 27
Bone Connectivity
Vertex weights indicate information about
bone connectivity!!!
5 . 4 . 1 .
SLIDE 28
Bone Connectivity
Vertex weights indicate information about
bone connectivity!!!
5 . 4 . 1 .
STRONG!!!
SLIDE 29
Bone Connectivity
Vertex weights indicate information about
bone connectivity!!!
5 . 4 . 1 .
weak…
SLIDE 30
Bone Connectivity
Extract maximal spanning tree to determine
connectivity
For each vertex
For each bone weight Let be the
maximum weight
Add weight to edge
between (max, j)
j
α max j,
max
α
SLIDE 31
Joint Locations
SLIDE 32
Joint Locations
Joint has same position with respect to both
bone transformations
SLIDE 33
Joint Locations
Many joints act as hinges and bend along an
axis
Yields infinite number of solutions to
minimization!!!
SLIDE 34
Joint Locations
Vertex weights indicate information about
joint locations!!!
5 . 5 .
SLIDE 35
Joint Locations
Vertex weights indicate information about
joint locations!!!
5 . 5 .
Joint close to this vertex
SLIDE 36
Joint Locations
Vertex weights indicate information about
joint locations!!!
1 . 9 .
Joint not close to this vertex
SLIDE 37
Joint Locations
Vertex weights indicate information about
joint locations!!!
1 . 9 .
Joint not close to this vertex
Minimize distance to estimated joint location
SLIDE 38
Root of Skeleton
Very arbitrary… We choose bone closest to center of mass Center of mass important
in physical calculations so makes sense to set root there
SLIDE 39
Results
Examples
SLIDE 40
Results
Examples
SLIDE 41
Results
Examples
SLIDE 42
Results
SLIDE 43
Results
Examples
SLIDE 44
Results
Examples
SLIDE 45
Results
Examples
SLIDE 46
Results
SLIDE 47
Results
Examples
SLIDE 48
Results
Examples
SLIDE 49
Results
Examples
SLIDE 50
Results
SLIDE 51
Results
Examples
SLIDE 52
Results
Examples
SLIDE 53
Results
Examples
SLIDE 54
Results
SLIDE 55
Results
29 bones 19 bones 9 bones
SLIDE 56
Performance
0.00s 1.85s 1.57s 19 9 9996 Lion 0.02s 32.23s 32.29s 22 23 84638 Elephant 0.00s 2.53s 5.21s 12 9 30000 Armadillo 0.00s 4.01s 2.11s 24 9 14410 Cat 0.00s 11.14s 5.53s 29 23 16843 Horse 0.00s 6.46s 9.96s 19 46 15789 Hand Skeleton Extraction Skinning Face Clustering Bones Example Poses Faces Model
SLIDE 57
Performance
0.00s 1.85s 1.57s 19 9 9996 Lion 0.02s 32.23s 32.29s 22 23 84638 Elephant 0.00s 2.53s 5.21s 12 9 30000 Armadillo 0.00s 4.01s 2.11s 24 9 14410 Cat 0.00s 11.14s 5.53s 29 23 16843 Horse 0.00s 6.46s 9.96s 19 46 15789 Hand Skeleton Extraction Skinning Face Clustering Bones Example Poses Faces Model
Face Clustering 10x-50x faster than Mean Shift Clustering [James, Twigg 2005]
SLIDE 58
Importing Skeletons
Maya-script importer Manipulate directly in Maya No plugins/special software needed Works with existing
production pipelines
SLIDE 59