Locomotion Animation Gavriel State, Senior Director, Systems - - PowerPoint PPT Presentation

▶
locomotion animation
SMART_READER_LITE
LIVE PREVIEW

Locomotion Animation Gavriel State, Senior Director, Systems - - PowerPoint PPT Presentation

Aug 30, 2022 270 likes •626 views

Deep Learning for Locomotion Animation Gavriel State, Senior Director, Systems Software March 26, 2018 Deep Learning Animation: PFNN Breakthrough paper on using motion capture + DL to drive locomotion animation

slide-1
SLIDE 1

Gavriel State, Senior Director, Systems Software March 26, 2018

Deep Learning for Locomotion Animation

slide-2
SLIDE 2

2

Deep Learning Animation: PFNN

  • Breakthrough paper on using motion capture + DL to drive locomotion animation
  • http://theorangeduck.com/page/phase-functioned-neural-networks-character-control
slide-3
SLIDE 3

3

slide-4
SLIDE 4

4

Applications

Games

slide-5
SLIDE 5

5

Applications

Crowds

slide-6
SLIDE 6

6

Applications

Auto Simulation

Image from the SYNTHIA dataset

slide-7
SLIDE 7

7

Applications

Human/Robot interaction safety

Mimus, Madeline Gannon / ATONATON (2016)

slide-8
SLIDE 8

8

Applications

Holodeck

slide-9
SLIDE 9

9

How does it work?

  • Gather Motion Capture data
  • Lots of free data available from CMU: http://mocap.cs.cmu.edu/
  • Many thanks to Fox VFX Lab for our capture above

Motion Capture

slide-10
SLIDE 10

10

How does it work?

  • Additional data needed:
  • Gait (running, walking, crouching, etc)
  • Phase
  • Footstep positions

Metadata labeling

slide-11
SLIDE 11

11

How does it work?

  • Generate many different

height fields that can fit a given set of character positions

  • More robust than just

capturing the actual height field, since it gives the network more potential data to fit with

Terrain Fitting

slide-12
SLIDE 12

12

How does it work?

  • Weights in the network

are different depending

  • n the phase parameter
  • Four sets of weights

trained

  • Mid-cycle weights

calculated by spline interpolation or precomputed (requires custom inferencing code or lots of memory)

Phase Functioned Neural Network

slide-13
SLIDE 13

13

How does it work?

Runtime Inferencing

slide-14
SLIDE 14

14

slide-15
SLIDE 15

15

PFNN On GPU

slide-16
SLIDE 16

16

slide-17
SLIDE 17

17

What’s Wrong With This Picture?

slide-18
SLIDE 18

18

Here’s a Hint

slide-19
SLIDE 19

19

slide-20
SLIDE 20

20

DeepLoco: Physics + RL

  • Another major recent work adds physics and high level control:

DeepLoco: Dynamic Locomotion Skills Using Hierarchical Deep Reinforcement Learning

Xue Bin Peng (1) Glen Berseth (1) KangKang Yin (2) Michiel van de Panne (1) (1)University of British Columbia (2)National University of Singapore

  • http://www.cs.ubc.ca/~van/papers/2017-TOG-deepLoco/
slide-21
SLIDE 21

21

How does it work?

  • Simulation engine + RL
  • Bullet Physics Engine, rewards
  • Low level controller
  • Uses phase, like PFNN, but simpler
  • Activates PD controller
  • High Level controller
  • Generates ‘footstep plan’ based on goals gH
  • Customizable for different tasks

DeepLoco RL System

slide-22
SLIDE 22

22

slide-23
SLIDE 23

23

Early RL Results

DeepLoco-style Reward Function

slide-24
SLIDE 24

24

slide-25
SLIDE 25

25

Physics + Mocap + RL

slide-26
SLIDE 26

26

slide-27
SLIDE 27

27

Physics + RL + Uneven Terrain No Mocap

Ministry of Silly Walks

slide-28
SLIDE 28

28

slide-29
SLIDE 29

29

Physics + RL + Uneven Terrain + Mocap

Ministry of Getting Closer

slide-30
SLIDE 30

30

slide-31
SLIDE 31

31

Additional Research

Character Interaction

slide-32
SLIDE 32

32

slide-33
SLIDE 33

33

DEMO: Deep Learning Animation

slide-34
SLIDE 34

Questions?