[PPT] - Towards a Generative Model of Natural Motion C. Karen Liu PowerPoint Presentation

SLIDE 1

Towards a Generative Model of Natural Motion

C. Karen Liu

University of Southern California

SLIDE 2

SLIDE 3

space of motion sequences space of motion sequences space of motion sequences space of motion sequences

Manual methods

SLIDE 4

space of motion sequences

Manual methods

space of motion sequences

Manual methods

space of motion sequences space of motion sequences

Data-driven methods

SLIDE 5

space of motion sequences

Data-driven methods

space of motion sequences

Data-driven methods

space of motion sequences

Data-driven methods

space of motion sequences

Data-driven methods

SLIDE 6

space of motion sequences space of motion sequences

Physics-based methods

space of motion sequences

Physics-based methods

space of motion sequences

Physics-based methods

highly dynamic motion

SLIDE 7

space of motion sequences

Physics-based methods

highly dynamic motion

space of motion sequences

Physics-based methods

highly dynamic motion

space of motion sequences

Physics-based methods

highly dynamic motion variational optimization

space of motion sequences

Physics-based methods

highly dynamic motion variational optimization

SLIDE 8

space of motion sequences

Physics-based methods

highly dynamic motion variational optimization

space of motion sequences

Physics-based methods

highly dynamic motion variational optimization

space of motion sequences

Physics-based methods

highly dynamic motion variational optimization

space of motion sequences

Physics-based methods

highly dynamic motion variational optimization

SLIDE 9

space of motion sequences

Physics-based methods

highly dynamic motion variational optimization

space of motion sequences

Physics-based methods

highly dynamic motion variational optimization

space of motion sequences

Physics-based methods

highly dynamic motion variational optimization

space of motion sequences

Generative model

highly dynamic motion low energy motion

SLIDE 10

space of motion sequences

Generative model

highly dynamic motion low energy motion

space of motion sequences

Generative model

highly dynamic motion low energy motion

space of motion sequences

Generative model

highly dynamic motion low energy motion

space of motion sequences

Generative model

highly dynamic motion low energy motion

SLIDE 11

space of motion sequences

Generative model

highly dynamic motion low energy motion

highly dynamic motion

SLIDE 12

SLIDE 13

SLIDE 14

space of motion sequences

highly dynamic motion

space of motion sequences

highly dynamic motion low energy motion

space of motion sequences

highly dynamic motion low energy motion

space of motion sequences

highly dynamic motion low energy motion

SLIDE 15

space of motion sequences

highly dynamic motion low energy motion

space of motion sequences

highly dynamic motion low energy motion

space of motion sequences

highly dynamic motion low energy motion

space of motion sequences

highly dynamic motion low energy motion

SLIDE 16

space of motion sequences

highly dynamic motion low energy motion

low energy motion

input motion

SLIDE 17

input motion input constraints input motion

utput motion

input constraints

Physical model

SLIDE 18

E(X) =

j

Q2

mj

E(X) =

j

Q2

mj

muscle force usage

E(X) =

j

Q2

mj

muscle force usage motion

E(X) =

j

Q2

mj

muscle force usage motion

SLIDE 19

E(X) =

j

Q2

mj

muscle force usage motion

There is a distinct preference for using specific joints rather than others, due to variations in join strength, stability, and

ther factors [Full et al. 2002].

E(X) =

j

Q2

mj

muscle force usage motion

E(X) =

j

αjQ2

mj

E(X) =

j

αjQ2

mj

SLIDE 20

E(X) =

j

αjQ2

mj

E(X) =

j

αjQ2

mj

Lagrangian dynamics

SLIDE 21

Lagrangian dynamics

d dt ∂T ∂ ˙ q − ∂T ∂q = Qint + Qext

Lagrangian dynamics

d dt ∂T ∂ ˙ q − ∂T ∂q = Qint + Qext

Lagrangian dynamics

d dt ∂T ∂ ˙ q − ∂T ∂q = Qint + Qext

Lagrangian dynamics

d dt ∂T ∂ ˙ q − ∂T ∂q =

Qm + Qp +Qext

SLIDE 22

Lagrangian dynamics

d dt ∂T ∂ ˙ q − ∂T ∂q =

Qm + Qp +Qext

Hooke’s law Hooke’s law Hooke’s law

Biological systems use passive elements, such as tendons and ligaments, to store and release energy, thereby reducing total power consumption [Alexandar 1998].

SLIDE 23

Hooke’s law

Biological systems use passive elements, such as tendons and ligaments, to store and release energy, thereby reducing total power consumption [Alexandar 1998]. Animals vary stiffness of their joints when performing different tasks [Farley and Morgenroth 1999].

Hooke’s law

Qp = −ks(q − ¯ q) − kd ˙ q

Hooke’s law Lagrangian dynamics

d dt ∂T ∂ ˙ q − ∂T ∂q =

Qm + Qp +Qext

SLIDE 24

d dt ∂T ∂ ˙ q − ∂T ∂q =

Qm + Qp +
Qg

Lagrangian dynamics

Qg d dt ∂T ∂ ˙ q − ∂T ∂q =

Qm + Qp +
Qg + Qc

Lagrangian dynamics

Qg Qc d dt ∂T ∂ ˙ q − ∂T ∂q =

Qm + Qp +
Qg + Qc + Qs

Lagrangian dynamics

Qg Qs Qc

Lagrangian dynamics

Qg Qs Qc Qm = d dt ∂T ∂ ˙ q − ∂T ∂q − Qp − Qg − Qc − Qs

SLIDE 25

Qp = −ks(q − ¯ q) − kd ˙ q

Lagrangian dynamics Hooke’s law

E(X) =

j

αjQ2

mj

Qm = d dt ∂T ∂ ˙ q − ∂T ∂q − Qp − Qg − Qc − Qs

Qp = −ks(q − ¯ q) − kd ˙ q

Lagrangian dynamics Hooke’s law

E(X) =

j

αjQ2

mj

Qm = d dt ∂T ∂ ˙ q − ∂T ∂q − Qp − Qg − Qc − Qs

αj

Qp = −ks(q − ¯ q) − kd ˙ q

Lagrangian dynamics Hooke’s law

E(X) =

j

αjQ2

mj

Qm = d dt ∂T ∂ ˙ q − ∂T ∂q − Qp − Qg − Qc − Qs

αj

ks ¯ q kd Qp = −ks(q − ¯ q) − kd ˙ q

Lagrangian dynamics Hooke’s law

E(X) =

j

αjQ2

mj

Qm = d dt ∂T ∂ ˙ q − ∂T ∂q − Qp − Qg − Qc − Qs

αj

ks ¯ q kd

α ks kd ¯ q θ = { , , , }

SLIDE 26

Qp = −ks(q − ¯ q) − kd ˙ q

Lagrangian dynamics Hooke’s law

E(X) =

j

αjQ2

mj

Qm = d dt ∂T ∂ ˙ q − ∂T ∂q − Qp − Qg − Qc − Qs

αj

ks ¯ q kd

α ks kd ¯ q θ = { , , , }

style preference of muscle usage stiffness of passive elements

{

emotional state emotional state individual

SLIDE 27

emotional state individual activity

α ks kd ¯ q θ = { , , , }

style preference of muscle usage stiffness of passive elements

{

α ks kd ¯ q θ = { , , , }

style preference of muscle usage stiffness of passive elements

{

X∗ = argmin E(X; θ)

X ∈ C

α ks kd ¯ q θ = { , , , }

style preference of muscle usage stiffness of passive elements

{

X∗ = argmin E(X; θ)

X ∈ C

SLIDE 28

α ks kd ¯ q θ = { , , , }

style preference of muscle usage stiffness of passive elements

{

X∗ = argmin E(X; θ)

X ∈ C

α ks kd ¯ q θ = { , , , }

style preference of muscle usage stiffness of passive elements

{

X∗ = argmin E(X; θ)

X ∈ C

α ks kd ¯ q θ = { , , , }

style preference of muscle usage stiffness of passive elements

{

X∗ = argmin E(X; θ)

X ∈ C

α ks kd ¯ q θ = { , , , }

style preference of muscle usage stiffness of passive elements

{

X∗ = argmin E(X; θ)

X ∈ C

SLIDE 29

α ks kd ¯ q θ = { , , , }

style preference of muscle usage stiffness of passive elements

{

θ X E(X, )

X∗ = argmin E(X; θ)

X ∈ C

α ks kd ¯ q θ = { , , , }

style preference of muscle usage stiffness of passive elements

{

θ

X∗

X E(X, )

X∗ = argmin E(X; θ)

X ∈ C

α ks kd ¯ q θ = { , , , }

style preference of muscle usage stiffness of passive elements

{

θ

X∗ X∗

X E(X, )

X∗ = argmin E(X; θ)

X ∈ C

α ks kd ¯ q θ = { , , , }

style preference of muscle usage stiffness of passive elements

{

θ

X∗ X∗ X∗

X E(X, )

X∗ = argmin E(X; θ)

X ∈ C

SLIDE 30

α ks kd ¯ q θ = { , , , }

style preference of muscle usage stiffness of passive elements

{

X∗ = argmin E(X; θ)

X ∈ C

α ks kd ¯ q θ = { , , , }

style preference of muscle usage stiffness of passive elements

{

has about 150 degrees of freedom θ

X∗ = argmin E(X; θ)

X ∈ C

Nonlinear inverse optimization Nonlinear inverse optimization

Given target motion and constraints , XT C

SLIDE 31

Nonlinear inverse optimization

Given target motion and constraints , XT C

Nonlinear inverse optimization

Given target motion and constraints , XT C determine style parameters θ

Nonlinear inverse optimization

Given target motion and constraints , XT C determine style parameters θ

Nonlinear inverse optimization

Given target motion and constraints , XT C determine style parameters θ

X E(X; θ)

XT

SLIDE 32

Nonlinear inverse optimization

Given target motion and constraints , XT C determine style parameters θ

X E(X; θ)

XT

Nonlinear inverse optimization

Given target motion and constraints , XT C determine style parameters θ

X E(X; θ)

XT

What does not work?

Least square difference

What does not work?

SLIDE 33

Least square difference

What does not work?

||XT − argmin E(X; θ)||2

X ∈ C

Least square difference

What does not work?

||XT − argmin E(X; θ)||2

X ∈ C

inefficient

Least square difference

What does not work?

||XT − argmin E(X; θ)||2

X ∈ C

inefficient
not robust

What does not work?

SLIDE 34

What does not work?

Maximum likelihood

What does not work?

Maximum likelihood

p(X|θ) = e−E(X;θ)

X∈C e−E(X;θ)dX

What does not work?

Maximum likelihood

p(X|θ) = e−E(X;θ)

X∈C e−E(X;θ)dX
intractable

G(θ) = E(XT ; θ) − min

X∈C E(X; θ)

˜ G(θ) = E(XT ; θ) − E(XS; θ) E(XT ; θ) > E(XS; θ) Given a motion , s. t. XS Approximate with G(θ)

Nonlinear inverse optimization

E(XT ; θ) = min

X∈C E(X; θ)

G(θ) = E(XT ; θ) − min

X∈C E(X; θ)

− d dθG(θ) ≈ − d dθ ˜ G(θ) ˜ G(θ) = E(XT ; θ) − E(XS; θ) E(XT ; θ) > E(XS; θ) Given a motion , s. t. XS Approximate with G(θ)

XT

X E(X; θ)

XS ← min

X∈C E(X; θ)

XT XS

X E(X; θ)

SLIDE 37

− d dθG(θ) ≈ − d dθ ˜ G(θ) XS ← min

X∈C E(X; θ)

XT XS

X E(X; θ)

θnew = θ − d dθ ˜ G(θ) XS ← min

X∈C E(X; θ)

− d dθG(θ) ≈ − d dθ ˜ G(θ)

XT XS

X E(X; θ)

θnew = θ − d dθ ˜ G(θ) XS ← min

X∈C E(X; θ)

− d dθG(θ) ≈ − d dθ ˜ G(θ)

XT XS

X E(X; θ)

θnew = θ − d dθ ˜ G(θ) XS ← min

X∈C E(X; θ)

− d dθG(θ) ≈ − d dθ ˜ G(θ)

X E(X; θ)

XT

SLIDE 38

XS

θnew = θ − d dθ ˜ G(θ) XS ← min

X∈C E(X; θ)

− d dθG(θ) ≈ − d dθ ˜ G(θ)

X E(X; θ)

XT XS

θnew = θ − d dθ ˜ G(θ) − d dθG(θ) ≈ − d dθ ˜ G(θ) XS ← min

X∈C E(X; θ)

− d dθG(θ) ≈ − d dθ ˜ G(θ)

X E(X; θ)

XT XS

θnew = θ − d dθ ˜ G(θ) θnew = θ − d dθ ˜ G(θ) XS ← min

X∈C E(X; θ)

− d dθG(θ) ≈ − d dθ ˜ G(θ)

X E(X; θ)

XT XS

θnew = θ − d dθ ˜ G(θ) θnew = θ − d dθ ˜ G(θ) XS ← min

X∈C E(X; θ)

− d dθG(θ) ≈ − d dθ ˜ G(θ)

X E(X; θ)

XT

SLIDE 39

θnew = θ − d dθ ˜ G(θ) XS ← min

X∈C E(X; θ)

− d dθG(θ) ≈ − d dθ ˜ G(θ)

X E(X; θ)

XT

θnew = θ − d dθ ˜ G(θ) XS ← min

X∈C E(X; θ)

− d dθG(θ) ≈ − d dθ ˜ G(θ)

X E(X; θ)

XTXS

θnew = θ − d dθ ˜ G(θ) − d dθG(θ) ≈ − d dθ ˜ G(θ) XS ← min

X∈C E(X; θ)

− d dθG(θ) ≈ − d dθ ˜ G(θ)

X E(X; θ)

XTXS

θnew = θ − d dθ ˜ G(θ) θnew = θ − d dθ ˜ G(θ) XS ← min

X∈C E(X; θ)

− d dθG(θ) ≈ − d dθ ˜ G(θ)

X E(X; θ)

XTXS

SLIDE 40

θnew = θ − d dθ ˜ G(θ) θnew = θ − d dθ ˜ G(θ) XS ← min

X∈C E(X; θ)

− d dθG(θ) ≈ − d dθ ˜ G(θ)

X E(X; θ)

XTXS

θnew = θ − d dθ ˜ G(θ) θnew = θ − d dθ ˜ G(θ) XS ← min

X∈C E(X; θ)

− d dθG(θ) ≈ − d dθ ˜ G(θ)

X E(X; θ)

XT

SLIDE 41

SLIDE 42

space of motion sequences

highly dynamic motion low energy motion

single character multiple tasks

highly dynamic motion low energy motion

single character multiple tasks

highly dynamic motion low energy motion

multiple characters multiple tasks

SLIDE 43

SLIDE 44

Applications Contributions

SLIDE 45

Contributions

Physics-based representation for

motion style

Contributions

Physics-based representation for

motion style

Learning algorithm for extracting

style from a single motion sequence

Contributions

Physics-based representation for

motion style

Learning algorithm for extracting

style from a single motion sequence

Generative model for motion

synthesis

Contributions

Physics-based representation for

motion style

Learning algorithm for extracting

style from a single motion sequence

Generative model for motion

synthesis

Support of theories in biomechanics

SLIDE 46

Future work Generalize to other animals Robotics controllers Motion-based interface

SLIDE 47

Acknowledgements

University of Washington Grail Lab Zoran Popovi Aaron Hertzmann Michael Cohen Steve Seitz National Science Foundation Alfred Sloan Fellowship NVIDIA Fellowship Microsoft Research Electronic Arts