[PPT] - Perception and Animation of Body and Hand Motions for Realistic PowerPoint Presentation

SLIDE 1

Perception and Animation of Body and Hand Motions for Realistic Virtual Characters Sophie J¨

rg

Postdoctoral Researcher Carnegie Mellon University

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 2

Virtual Characters Are Increasingly Common

Final Fantasy (2001) Museumguides [Swartout10] NBA2K11 (2011) The Sims 3 (2009) Beowulf (2007) Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 3

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 4

People Are Very Good At Recognizing Human Motions

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 5

More Detail Is Not Always Better

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 6

Not All Virtual Characters Are Engaging

The Polar Express (2004) Mars Needs Moms (2011)

The Polar Express: . . . cold and distanced animation. . . (EmanualLevy.Com) . . . More like Night of the Living Dead . . . (Bangor Daily News) Mars Needs Moms: . . . waxy, inexpressive faces. . . (New York Times) . . . the dehumanized child characters . . . (ColeSmithey.com) . . . affectless, mechanical cartoons, . . . (Film.com)

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 7

The Uncanny Valley

Hypothesized Uncanny Valley graph, redrawn from [Mori 70]

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 8

The Uncanny Valley

Hypothesized Uncanny Valley graph, redrawn from [Mori 70]

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 9

The Uncanny Valley

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 10

Research Goals

Understand how people perceive and interpret

human-like motions.

Develop better algorithms to create virtual

characters.

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 11

Hand and Finger Movements

integral and crucial part of language and communication

[Kendon04, McNeill92]

very hard to capture and often animated separately [Majkowska

et al. 04] or only a few degrees of freedom are captured [Kitagawa and Windsor 08]

GreenDot Project, Williams et al. 10 Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 12

How to Animate Fingers

Motion capture
Keyframe animation
Algorithms

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 13

How to Animate Fingers

Motion capture → not always accurate [Kahlesz et al. 2004],

needs post-processing [Kitagawa and Windsor 2008]

Keyframe animation
Algorithms

CyberGlovesII Capturing A Christmas Carol Markerset for hands Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 14

How to Animate Fingers

Motion capture
Keyframe animation → time-consuming
Algorithms

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 15

How to Animate Fingers

Motion capture
Keyframe animation
Algorithms → rule-based [ElKoura03], physics-based [Pollard05,

Liu08, Liu09], data-driven [Majkowska06]

Liu08 Liu09 Pollard05 Majkowska06 ElKoura03 Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 16

Finger Perception: Threshold Experiment

Are errors in finger motion noticeable?

We investigate synchronization errors between finger and body motion, which are common when hand and body are captured separately.

published in The Perception of Finger Motions, APGV 2010

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 17

“Is this motion MODIFIED or NOT MODIFIED?”

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 18

Design

Four actions:

Count, Drink, Snap, Point

Three versions of each action Four levels of synchronization error:

0.1s, 0.2s, 0.3s, 0.4s

Total of 60 different clips

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 19

Result: Errors As Small As 0.1s Can Be Recognized

proportion of motions rated as unmodified level of error

Count

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 20

Result: Error Perception Depends on the Motion

proportion of motions rated as unmodified level of error

Point

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 21

Theshold Experiment Summary

Participants recognize errors as little as 0.1 seconds

֒ → Correct timing of hand motions is very important

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 22

Finger Perception: Interpretation Experiment

Does Correct Finger Motion Matter?

Investigate the impact of incorrect finger motion on the interpretation of a scene

published in The Perception of Finger Motions, APGV 2010

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 23

New Approach: Vignettes

Animated movies that tell a story and convey emotional content. Previous studies:

simple scenarios
little or no affective content
existing videos

Our approach:

tell a story
elicit an emotional response
ask higher level questions, e.g., interpretation of a scene

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 24

Why Vignettes? Closer to real experience. Investigate higher-level questions. Find effects that are impossible to measure with short snippets of motion.

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 25

Stimuli: Original Motion Captured

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 26

Stimuli: Finger Joints Delayed by 0.5 Seconds

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 27

Design

Between subjects design: each group only saw one of the versions

original: 47 participants
delayed: 69 participants

Questionnaire:

empathy with the character
factual questions
describe the scene
quality

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 28

Data Analysis

Five-point scale response data: between groups ANOVA

significant differences? empathy questions

no

factual questions

no

quality

no Descriptions in free form text:

riginal

delayed character using ctrl-alt-del

21% 1%

computer freezing

40% 15%

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 29

Interpretation Experiment Summary Finger motion can alter the interpretation of a scene, even without altering its perceived quality.

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 30

Finger Perception: Animation Method Experiment

What animation method should we recommend?

riginal: motion captured

no hands: no motion, stiff hands in a natural pose keyframe: imitating motion capture with limited number of keyframes random: small idle motion captured movements

published in The Perception of Finger Motions, APGV 2010

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 31

“In which clip was the motion of better quality?”

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 32

Stimuli and Design

Four types of finger animations:

original, no hands, keyframe, random

Four scenes out of a narration Two types of viewpoint:

long shot, close-up (hip to neck)

Pairwise comparisons of animation types 48 pairs of clips in total

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 33

Result: Motion Capture Rated Best

proportion of motions rated as having the better quality animation type

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 34

Animation Method Experiment Summary The motion captured movements are considered to have the best quality

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 35

Finger Perception Summary Subtle errors in finger motions can be perceived. They can change the interpretation of a movie. Motion captured movements are considered to have the best quality. Finger motions are hard to capture.

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 36

Facial Animation and Sound

Investigate the effect of facial animation and sound on the emotional content and interpretation of a scene

Used three vignettes, about 30 seconds each

Published in The Saliency of Anomalies in Animated Human Characters, TAP 2010

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 37

Create Stimuli

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 38

Stimuli: Milk – Original

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 39

Results

Sound and motion interacted with content in complex ways. Results could not have been found with short stimuli. Even unrelated choices of stimuli (e.g., sound/no sound) can influence results.

Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012

SLIDE 40

Questions?

Recognizing Human Motions The Uncanny Valley Finger Perception: Threshold Experiment Finger Perception: Interpretation Experiment Finger Perception: Animation Method Experiment Finger Motion Synthesis Facial Animation and Sound Future Research Sophie J¨

rg

Perception and Animation of Virtual Characters Feb 16, 2012