SCAPE: Shape Completion SCAPE: Shape Completion and Animation of - - PowerPoint PPT Presentation

1 1

SCAPE: Shape Completion SCAPE: Shape Completion and Animation of People and Animation of People

By Dragomir Anguelov, Praveen Srinivasan, By Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Daphne Koller, Sebastian Thrun, Jim Rodgers, James Davis Rodgers, James Davis From SIGGRAPH 2005 From SIGGRAPH 2005

Presentation for CS468 by Emilio Ant Presentation for CS468 by Emilio Antú únez nez

2 2

Motivation Motivation

 It is difficult to get high­resolution body

It is difficult to get high­resolution body scans scans

 It is even harder at video rates

It is even harder at video rates

 By building up a human model, you

By building up a human model, you could synthesize a high­resolution could synthesize a high­resolution scan from sparse/incomplete data scan from sparse/incomplete data

 Accurate model is most easily created

Accurate model is most easily created by learning from sample scans by learning from sample scans

3 3

Pre-existing Work in Pre-existing Work in Deformable Human Models I Deformable Human Models I

 Deformations described relative to a

Deformations described relative to a template shape template shape

 Pose deformations given relative to

Pose deformations given relative to local joints in an articulated model local joints in an articulated model

 Body­shape deformations described

Body­shape deformations described using displacement vectors from PCA using displacement vectors from PCA

4 4

Pre-existing Work in Pre-existing Work in Deformable Human Models II Deformable Human Models II

 Pose and shape deformations rarely

Pose and shape deformations rarely addressed together addressed together

 Most similar work by Sumner and

Most similar work by Sumner and Popovi Popović ć

– Retargets pose deformation to another Retargets pose deformation to another mesh mesh – Does not learn a model Does not learn a model

5 5

Paper Contributions Paper Contributions

 Learning an affine deformation model

Learning an affine deformation model for both pose and shape for both pose and shape

 Shape completion for scan of an

Shape completion for scan of an arbitrary human target arbitrary human target

 Body shape manipulation for motion

Body shape manipulation for motion capture animation capture animation

6 6

Presentation Overview Presentation Overview

 Data Acquisition

Data Acquisition

 Learning the Human Model

Learning the Human Model

 Applications

Applications

– Shape Completion Shape Completion – Motion Capture Animation Motion Capture Animation

 Limitations

Limitations

7 7

Data Format / Assumptions Data Format / Assumptions

 Each input model is a deformation of a

Each input model is a deformation of a fixed­topology triangle mesh fixed­topology triangle mesh

 Models divided into three categories

Models divided into three categories

– One template model One template model – Template subject in different poses Template subject in different poses – Different people in (roughly) same pose Different people in (roughly) same pose

 Articulated skeleton assigned to each

Articulated skeleton assigned to each mesh mesh

8 8

Data Acquisition and Data Acquisition and Processing Processing

9 9

Learning the Human Model Learning the Human Model

 Pose and shape deformations

Pose and shape deformations described per­triangle using linear described per­triangle using linear transformations transformations

 Pose transformations learned from

Pose transformations learned from template subject in different poses template subject in different poses

 Body shape transformations learned

Body shape transformations learned by comparing different subjects to by comparing different subjects to template template

10 10

Pose Deformation I Pose Deformation I

 Rigid (skeletal)

Rigid (skeletal) deformations are deformations are represented separately represented separately from non­rigid ones from non­rigid ones

 Transformations are

Transformations are given in relative given in relative coordinate system coordinate system where one of the where one of the corners is fixed at the corners is fixed at the

• rigin
• rigin

O Qk Rl[k] template triangle final triangle

11 11

Pose Deformation II Pose Deformation II

 Triangle edges are not

Triangle edges are not forced to be consistent forced to be consistent

 Final synthesized

Final synthesized mesh reduces the mesh reduces the least­squares error least­squares error between mesh points between mesh points and triangle and triangle deformations deformations

O Qk Rl[k] template triangle final triangle

12 12

Learning Pose Deformation Learning Pose Deformation Model I Model I

 Rigid rotation is known from skeleton

Rigid rotation is known from skeleton

 Non­rigid transformation is

Non­rigid transformation is underdefined underdefined

 Q matrix is computed by requiring

Q matrix is computed by requiring adjacent triangles’ non­rigid adjacent triangles’ non­rigid transformations to be similar transformations to be similar

13 13

Learning Pose Deformation Learning Pose Deformation Model II Model II

 Non­rigid deformation modeled as an

Non­rigid deformation modeled as an affine function of adjacent joint angles affine function of adjacent joint angles

 In practice, some of the degrees of

In practice, some of the degrees of freedom are removed for constrained freedom are removed for constrained joings joings

14 14

Pose Deformation Learning Pose Deformation Learning Results Results

15 15

Body-Shape Deformation Body-Shape Deformation

 Body shape is modeled as an additional

Body shape is modeled as an additional linear transform, S linear transform, S

 S is underdetermined (like Q)

S is underdetermined (like Q)

 Again, solved using a smoothness

Again, solved using a smoothness constraint constraint

16 16

Learning the Shape Learning the Shape Deformation Model Deformation Model

 The matrix coefficients for all body

The matrix coefficients for all body shape transformations are vectorized shape transformations are vectorized

 Principal component analysis is used

Principal component analysis is used to parameterize the shape transform to parameterize the shape transform vectors vectors

17 17

Shape Deformation Shape Deformation Learning Results Learning Results

18 18

Shape Completion I Shape Completion I

 Assuming you know some of the node

Assuming you know some of the node positions, estimate the others positions, estimate the others

 Must estimate pose and body shape

Must estimate pose and body shape

 This optimization is highly nonlinear in the

This optimization is highly nonlinear in the pose pose

 Empirically found that optimizing over all

Empirically found that optimizing over all variables at once produces bad results variables at once produces bad results

 Instead, SCAPE iterates solving

Instead, SCAPE iterates solving

19 19

Shape Completion II Shape Completion II

 Empirically found that optimizing over all

Empirically found that optimizing over all variables at once produces bad results variables at once produces bad results

 Instead, SCAPE iterates, solving each of

Instead, SCAPE iterates, solving each of these in order: these in order:

– Pose Pose – Mesh estimate Mesh estimate – Body shape Body shape

 Results in a “completed” mesh and a

Results in a “completed” mesh and a “predicted” mesh “predicted” mesh

20 20

Partial View Completion Partial View Completion

 Skeletal and point­correspondences

Skeletal and point­correspondences may be off if too much data is missing may be off if too much data is missing

 Iterate between the shape completion

Iterate between the shape completion algorithm previously described and algorithm previously described and remapping the point correspondences remapping the point correspondences

21 21

Partial View Completion Partial View Completion Results Results

22 22

Motion Capture Animation Motion Capture Animation

 Motion capture data provides the pose

Motion capture data provides the pose data data

 Body shape parameters can be set

Body shape parameters can be set arbitrarily arbitrarily

 Since markers are generally placed on

Since markers are generally placed on body surface (not in the bones), mesh body surface (not in the bones), mesh is constrained to lie in the space of is constrained to lie in the space of body shapes encoded by the model body shapes encoded by the model

23 23

Motion Capture Animation Motion Capture Animation Results Results

24 24

Limitations Limitations

 Assumes that pose deformation and

Assumes that pose deformation and body shape are mostly independent body shape are mostly independent

 Models only pose deformations from

Models only pose deformations from skeletal motion skeletal motion

25 25

Conclusion Conclusion

 SCAPE learns simple body model

SCAPE learns simple body model which distinguishes pose and body which distinguishes pose and body shape deformations shape deformations

 Creates reasonable shape

Creates reasonable shape completions, even when large features completions, even when large features are missing are missing

 Allows for flexible reconstruction of

Allows for flexible reconstruction of moving model from motion capture moving model from motion capture data data