SLIDE 1
Fast Automatic Skinning Transformations
Alec Jacobson Ilya Baran Ladislav Kavan Jovan Popović Olga Sorkine
August 8, 2012
#
Real-time performance critical for interactive design and animation
August 8, 2012 Alec Jacobson 2
Fast Automatic Skinning Transformations Alec Jacobson ETH Zurich - - PowerPoint PPT Presentation
Fast Automatic Skinning Transformations Alec Jacobson ETH Zurich - - PowerPoint PPT Presentation
Fast Automatic Skinning Transformations Alec Jacobson ETH Zurich Ilya Baran Disney Research Zurich Ladislav Kavan ETH Zurich Jovan Popovi Adobe Systems, Inc. Olga Sorkine ETH Zurich August 8, 2012 Real-time performance critical for
SLIDE 2
SLIDE 3
#
Real-time performance critical for interactive design and animation
August 8, 2012 Alec Jacobson 3
2D 3D
SLIDE 4
# August 8, 2012 Alec Jacobson 4
We want speeds measured in microseconds
80k triangles 20µs per iteration
SLIDE 5
# August 8, 2012 Alec Jacobson 5
We want speeds measured in microseconds
80k triangles 20µs per iteration
SLIDE 6
#
This means speed comparable to rendering
August 8, 2012 Alec Jacobson 6
30fps
SLIDE 7
#
Linear Blend Skinning preferred for real-time performance
August 8, 2012 Alec Jacobson 7
place skeleton in shape
SLIDE 8
#
Linear Blend Skinning preferred for real-time performance
August 8, 2012 Alec Jacobson 8
place skeleton in shape compute/paint weights
SLIDE 9
#
Linear Blend Skinning preferred for real-time performance
August 8, 2012 Alec Jacobson 9
place skeleton in shape compute/paint weights deform bones
SLIDE 10
#
Linear Blend Skinning preferred for real-time performance
August 8, 2012 Alec Jacobson 10
place skeleton in shape compute/paint weights deform bones
v0
i = m
X
j=1
wj(vi)Tj ✓vi 1 ◆
SLIDE 11
#
Linear Blend Skinning preferred for real-time performance
August 8, 2012 Alec Jacobson 11
place skeleton in shape compute/paint weights deform bones
SLIDE 12
#
Linear Blend Skinning preferred for real-time performance
August 8, 2012 Alec Jacobson 12
place skeleton in shape compute/paint weights deform bones
SLIDE 13
#
Linear Blend Skinning preferred for real-time performance
August 8, 2012 Alec Jacobson 13
place skeleton in shape compute/paint weights deform bones
SLIDE 14
#
Linear Blend Skinning preferred for real-time performance
August 8, 2012 Alec Jacobson 14
place skeleton in shape compute/paint weights deform bones 15 bones * 3x4 matrix = 180 degrees of freedom
SLIDE 15
#
LBS generalizes to different handle types
August 8, 2012 Alec Jacobson 15
skeletons
v0
i = m
X
j=1
wj(vi)Tj ✓vi 1 ◆
SLIDE 16
#
regions
LBS generalizes to different handle types
August 8, 2012 Alec Jacobson 16
skeletons
v0
i = m
X
j=1
wj(vi)Tj ✓vi 1 ◆
SLIDE 17
#
regions
LBS generalizes to different handle types
August 8, 2012 Alec Jacobson 17
skeletons
v0
i = m
X
j=1
wj(vi)Tj ✓vi 1 ◆
SLIDE 18
#
regions
LBS generalizes to different handle types
August 8, 2012 Alec Jacobson 18
skeletons points
v0
i = m
X
j=1
wj(vi)Tj ✓vi 1 ◆
SLIDE 19
#
regions
LBS generalizes to different handle types
August 8, 2012 Alec Jacobson 19
skeletons points
v0
i = m
X
j=1
wj(vi)Tj ✓vi 1 ◆
SLIDE 20
#
User specifies subset of parameters,
- ptimize to find remaining ones
August 8, 2012 Alec Jacobson 20
Full optimization Mesh vertex positions
SLIDE 21
#
User specifies subset of parameters,
- ptimize to find remaining ones
August 8, 2012 Alec Jacobson 21
Full optimization Reduced model Skinning degrees of freedom
SLIDE 22
#
User specifies subset of parameters,
- ptimize to find remaining ones
August 8, 2012 Alec Jacobson 22
Full optimization Reduced model Matrix form
SLIDE 23
#
User specifies subset of parameters,
- ptimize to find remaining ones
August 8, 2012 Alec Jacobson 23
Full optimization Reduced model Matrix form Reduced optimization
SLIDE 24
#
Enforce user constraints as linear equalities
August 8, 2012 Alec Jacobson 24
User constraints Reduced optimization Full Position only Unconstrained
SLIDE 25
#
Enforce user constraints as linear equalities
August 8, 2012 Alec Jacobson 25
User constraints Reduced optimization Full Position only Unconstrained
SLIDE 26
#
Enforce user constraints as linear equalities
August 8, 2012 Alec Jacobson 26
User constraints Reduced optimization Full Position only Unconstrained
SLIDE 27
#
We reduce any as-rigid-as-possible energy
August 8, 2012 Alec Jacobson 27
Full energies
SLIDE 28
#
We reduce any as-rigid-as-possible energy
August 8, 2012 Alec Jacobson 28
Full energies
triangles
Liu et al. 08
“spokes”
Sorkine & Alexa 07
“spokes and rims”
Chao et al. 10
tetrahedra
Chao et al. 10
SLIDE 29
#
Global step: Fix , minimize with respect to
We reduce any as-rigid-as-possible energy
August 8, 2012 Alec Jacobson 29
Full energies
V0 R
Local step: Fix , minimize with respect to R
V0
Local/Global optimization
SLIDE 30
#
Global step: large, sparse linear solve
We reduce any as-rigid-as-possible energy
August 8, 2012 Alec Jacobson 30
Full energies Local step: Fix , minimize with respect to R
V0
Local/Global optimization
V0 = A1b
precompute
SLIDE 31
#
We reduce any as-rigid-as-possible energy
August 8, 2012 Alec Jacobson 31
Full energies Local step: 3x3 SVD for each rotation in R Local/Global optimization Global step: large, sparse linear solve V0 = A1b
SLIDE 32
#
Global step: small, dense linear solve
We reduce any as-rigid-as-possible energy
August 8, 2012 Alec Jacobson 32
Full energies Local step: 3x3 SVD for each rotation in R Local/Global optimization
T = ˜ A−1˜ b
precompute
Similar to: [Huang et al. 06] [Der et al. 06] [Au et al. 07] [Hildebrandt et al. 12]
Substitute
SLIDE 33
#
Direct reduction of elastic energies brings speed up and regularization…
August 8, 2012 Alec Jacobson 33
SLIDE 34
#
Direct reduction of elastic energies brings speed up and regularization…
August 8, 2012 Alec Jacobson 34
Full ARAP solution
SLIDE 35
#
Direct reduction of elastic energies brings speed up and regularization…
August 8, 2012 Alec Jacobson 35
Full ARAP solution Our smooth subspace solution
SLIDE 36
#
Local step: 3x3 SVD for each rotation in Global step: small, dense linear solve
We reduce any as-rigid-as-possible energy
August 8, 2012
But #rotations ~ full mesh discretization
Alec Jacobson 36
Full energies
R
Local/Global optimization
T = ˜ A−1˜ b
Substitute
SLIDE 37
#
Global step: small, dense linear solve
We reduce any as-rigid-as-possible energy
August 8, 2012 Alec Jacobson 37
Full energies
R
Local/Global optimization Substitute
T = ˜ A−1˜ b
Local step: 3x3 SVD for each rotation in Cluster
Ek
SLIDE 38
#
Rotation evaluations may be reduced by clustering in weight space
August 8, 2012 Alec Jacobson 38
Full energies
triangles
Liu et al. 08
“spokes”
Sorkine & Alexa 07
“spokes and rims”
Chao et al. 10
tetrahedra
Chao et al. 10
SLIDE 39
#
Rotation evaluations may be reduced by k-means clustering in weight space
August 8, 2012 Alec Jacobson 39
Full energies
xj = w1(vj) w2(vj) . . . wm(vj)
weight space
SLIDE 40
#
Rotation evaluations may be reduced by clustering in weight space
August 8, 2012 Alec Jacobson 40
Full energies r = 2 r = 4 r = 64
SLIDE 41
#
Rotation evaluations may be reduced by clustering in weight space
August 8, 2012 Alec Jacobson 41
Full energies r = 2 r = 4 r = 64
SLIDE 42
#
Global step: small, dense linear solve
We reduce any as-rigid-as-possible energy
August 8, 2012 Alec Jacobson 42
Full energies
R
Local/Global optimization
T = ˜ A−1˜ b
Local step: 3x3 SVD for each rotation in
#rotations ~ #T , independent of full mesh resolution
Substitute Cluster
Ek
SLIDE 43
#
Real-time automatic degrees of freedom
August 8, 2012 Alec Jacobson 43
SLIDE 44
#
Real-time automatic degrees of freedom
August 8, 2012 Alec Jacobson 44
SLIDE 45
#
With more and more user constraints we fall back to standard skinning
August 8, 2012 Alec Jacobson 45
SLIDE 46
#
With more and more user constraints we fall back to standard skinning
August 8, 2012 Alec Jacobson 46
SLIDE 47
#
With more and more user constraints we fall back to standard skinning
August 8, 2012 Alec Jacobson 47
SLIDE 48
#
With more and more user constraints we fall back to standard skinning
August 8, 2012 Alec Jacobson 48
SLIDE 49
#
Extra weights would expand subspace…
August 8, 2012 Alec Jacobson 49
SLIDE 50
#
Extra weights would expand subspace…
August 8, 2012 Alec Jacobson 50
SLIDE 51
#
Extra weights would expand subspace…
August 8, 2012 Alec Jacobson 51
SLIDE 52
#
Overlapping b-spline “bumps” in weight space
August 8, 2012 Alec Jacobson 52
farthest point sampling
xj = w1(vj) w2(vj) . . . wm(vj)
weight space
SLIDE 53
#
Overlapping b-spline “bumps” in weight space
August 8, 2012 Alec Jacobson 53
b-spline basis parameterized by distance in weight space
in weight space
xj = w1(vj) w2(vj) . . . wm(vj)
weight space
SLIDE 54
#
Overlapping b-spline “bumps” in weight space
August 8, 2012 Alec Jacobson 54
b-spline basis parameterized by distance in weight space
in weight space
xj = w1(vj) w2(vj) . . . wm(vj)
weight space
SLIDE 55
#
no extra weights
Extra weights expand deformation subspace
August 8, 2012 Alec Jacobson 55
15 extra weights
SLIDE 56
#
no extra weights
Extra weights expand deformation subspace
August 8, 2012 Alec Jacobson 56
15 extra weights
SLIDE 57
#
Subspace now rich enough for fast variational modeling
August 8, 2012 Alec Jacobson 57
Full non-linear optimization [Botsch et al. 2006] Our reduced method
SLIDE 58
#
Subspace now rich enough for fast variational modeling
August 8, 2012 Alec Jacobson 58
Full non-linear optimization [Botsch et al. 2006] Our reduced method
SLIDE 59
#
Subspace now rich enough for fast variational modeling
August 8, 2012 Alec Jacobson 59
Full non-linear optimization [Botsch et al. 2006] Our reduced method
SLIDE 60
#
Subspace now rich enough for fast variational modeling
August 8, 2012 Alec Jacobson 60
Full non-linear optimization [Botsch et al. 2006] Our reduced method
SLIDE 61
#
Subspace now rich enough for fast variational modeling
August 8, 2012 Alec Jacobson 61
Full non-linear optimization [Botsch et al. 2006] Our reduced method
SLIDE 62
#
Subspace now rich enough for fast variational modeling
August 8, 2012 Alec Jacobson 62
Full non-linear optimization [Botsch et al. 2006] Our reduced method
SLIDE 63
#
Final algorithm is simple and FAST
August 8, 2012 Alec Jacobson 63
Precomputation per shape+rig
- Compute any additional weights
- Construct, prefactor system matrices
For a 50K triangle mesh: 12 seconds 2.7 seconds
SLIDE 64
#
Final algorithm is simple and FAST
August 8, 2012 Alec Jacobson 64
Precomputation per shape+rig
- Compute any additional weights
- Construct, prefactor system matrices
Precomputation when switching constraint type
- Re-factor global step system
For a 50K triangle mesh: 12 seconds 2.7 seconds 6 milliseconds
SLIDE 65
#
Final algorithm is simple and FAST
August 8, 2012 Alec Jacobson 65
Precomputation per shape+rig
- Compute any additional weights
- Construct, prefactor system matrices
Precomputation when switching constraint type
- Re-factor global step system
~30 iterations global: #weights by #weights linear solve local: #rotations SVDs
For a 50K triangle mesh: 12 seconds 2.7 seconds 6 milliseconds
22 microseconds
[McAdams et al. 2011]
SLIDE 66
# August 8, 2012 Alec Jacobson 66
Demo
Lightning FAST automatic skinning transformations
SLIDE 67
#
Extra weights and disjoint skeletons make flexible control easy
August 8, 2012 Alec Jacobson 67
From Cartoon Animation by Preston Blair
SLIDE 68
#
Extra weights and disjoint skeletons make flexible control easy
August 8, 2012 Alec Jacobson 68
SLIDE 69
#
Extra weights and disjoint skeletons make flexible control easy
August 8, 2012 Alec Jacobson 69
SLIDE 70
#
Extra weights and disjoint skeletons make flexible control easy
August 8, 2012 Alec Jacobson 70
SLIDE 71
#
Our reduction preserves nature of different energies, at no extra cost
August 8, 2012 Alec Jacobson 71
Surface ARAP Volumetric ARAP
V0
surf = MsurfT
V0
vol = MvolT
SLIDE 72
#
Our reduction preserves nature of different energies, at no extra cost
August 8, 2012 Alec Jacobson 72
Surface ARAP Volumetric ARAP
V0
surf = MsurfT
V0
vol = MvolT
SLIDE 73
#
Simple drag-only interface for point handles
August 8, 2012 Alec Jacobson 73
SLIDE 74
#
Simple drag-only interface for point handles
August 8, 2012 Alec Jacobson 74
SLIDE 75
#
Simple drag-only interface for point handles
August 8, 2012 Alec Jacobson 75
SLIDE 76
#
Skinning rig enables FAST deformation
- Substitute to reduce DOFs
August 8, 2012 Alec Jacobson 76
SLIDE 77
#
Skinning rig enables FAST deformation
- Substitute to reduce DOFs
- Cluster rotations to reduce energy eval.
August 8, 2012 Alec Jacobson 77
SLIDE 78
#
Skinning rig enables FAST deformation
- Substitute to reduce DOFs
- Cluster rotations to reduce energy eval.
- Additional weights to expand subspace
August 8, 2012 Alec Jacobson 78
SLIDE 79
#
Skinning rig enables FAST deformation
- Substitute to reduce DOFs
- Cluster rotations to reduce energy eval.
- Additional weights to expand subspace
August 8, 2012 Alec Jacobson 79
Each innovation takes advantage of input skinning rig
SLIDE 80
#
Future work and discussion
- Alternative additional weights: sparsity?
- Joint limits, balance, etc.
August 8, 2012 Alec Jacobson 80
SLIDE 81
#
Acknowledgements
We are grateful to Peter Schröder, Emily Whiting, and Maurizio Nitti. We thank Eftychios Sifakis for his open source fast 3×3 SVD code. This work was supported in part by an SNF award 200021_137879 and by a gift from Adobe Systems.
August 8, 2012 Alec Jacobson 81
SLIDE 82