Paper Summaries Any takers? Articulated Figures III Motion Capture - - PDF document

1

Articulated Figures III

Motion Capture

Paper Summaries

Any takers?

Projects

Question about presentations

We have approx 19 projects

Presentations: 15 minutes (max) per project Sign up for time via e-mail (first come / first served)

Midquarter report due next Wednesday

Grad Reports

We have 10

Please indicate topic by end of day (e-mail) 20 minutes per presentation Week 9

Projects

Exam Week Presentation day:

Thursday, March 2nd 12:30pm -- 2:30pm Room 70-3445

Assignments

Assignment 1: Keyframing

Due last Wednesday Grading in progress (mostly done)

Assignment 2: Dynamics

Due Wednesday, January 25th (Wed)

Assignment 3: Group motion

Particle systems / Flocking: Due Feb 6th

Assignment 4: moCap

To be given today Due February 20

Plan for today

Next 2 weeks: Articulated Figures

Monday: Forward Kinematics Wednesday: Inverse Kinematics Monday: Motion Capture Wednesday: Advanced algorithms

Then

Monday: Character animation

2

Motivation Films

Early examples of motion capture

Motivational Film

Brilliance (1985)

It’s all Apple’s fault! Robert Abel & Associates first entirely computer generated TV ad Debuted at Super Bowl XI X (1985) Who said motion capture was a new

technology?

Motivational Film

Don’t Touch Me (1989)

Diana Walczak and Jeff

Kleiser

Synthespian, (Synthetic

Thesbians)

Dojo – First female

Synthespian

http://www.kwcc.com/

Plan For Today

Topics

Motion Capture Walking Assignment # 4 (maybe)

Role of Animation

Degrees of freedom

Number of parameters

whose values must be defined in order to fully position the articulated figure

Purpose of animation

Provide values to each of the DOF

for each time step.

Motion Capture

The idea between motion capture

You want realistic human motion?

Go to the source No, not Newton this time… Use an actual human

3

Rotoscoping

• Used to trace motion of live

actors, frame by frame into an animation

• Invented by Max Fleischer in

1916

• First used in Koko the Clown

cartoons

• Used extensively by Disney in

Snow White

Motion Capture in CG

First introduced by Abel and Associates

for “Brilliance”

Motion Capture

What motion capture gives us:

Sampled values for each DOF in time.

Since captured directly from human

motion

Subtleties of motion come for free. Difficult for an animator to keyframe these

subtleties

Motion Capture

Watt/Policarpo

Motion Capture

Types of motion capture systems

Optical

Incorporate directionally-reflective balls referred to as

markers which attach to the performer.

Three (at least) digital video cameras that track markers. Provides most flexibility for performers. Problem: Markers may be occluded from cameras views.

Optical Motion Capture

Motion Analysis Corp

I Robot Final Fantasy

Entirely motion capture

Polar Express

video

4

Motion Capture

Types of motion capture systems

Prosthetic

set of armatures attached all over the

performer’s body

The armatures are connected to each other by

using a series of rotational and linear encoders.

Accurate, though cumbersome for the

performer

Prosthetic Motion Capture

Gypsy 4

By MetaMotion

Motion Capture Systems

Types of motion capture systems

Acoustic

An array of audio transmitters are strapped to

various parts of the performers body.

Three receivers are triangulated to provide a

point in 3D space.

No occlusion problem. Cables are cumbersome to performers Ambient sound may interfere

Motion Capture Systems

Types of motion capture systems

ElectroMagnetic

Much like acoustic except magnetic

transmitters/receivers used instead of acoustic

No occlusion problem. Cables are cumbersome to performers Though now wireless solutions are available Metal / other magnetic fields may interfere.

Electromagnetic Motion Capture

MotionStar 2

Ascension

Technologies

Motion Capture Systems

Types of motion capture systems

Fiber Optic Sensors

Flexible FO sensors strapped to various parts of

the performers body.

Sensors can directly measure joint rotations Used in conjunction with electromagentic

sensor for head and torso.

5

Fiber Optics Motion Capture

Shapewrap II

Measurand

Capturing Human Motion

Minimal set of

recording points

Frey, et. al

Motion capture Systems

Challenges:

Signal is not perfect

Noisy missing data not perfectly aligned with joints

Retargeting

Data is only valid for virtual character who

possesses same scale as real character.

Motion Capture Systems

Challenges:

Even if motion capture data was perfect, we still

have the following challenges:

Re-use – use the motion for a slightly different purpose Creating impossible motion – Motion capture won’t do it,

but may be desired in animation

Change of intent – we can’t always predict what motion

we will need

Take Home Message: Motion Capture captures a

particular, single motion.

Motion Capture Systems

Examples

From Eurographics Computer

Animation and Simulation EGCAS'96

From The Polar Express

Motion Capture Data

So what CAN we do with motion

capture data?

We can

speed up slow down time warp Motion warp

However, one must remember that

Captured data is Sampled Data.

6

Sampling Theory

Signal - function that conveys

information

Audio signal (1D - function of time) Image (2D - function of space)

Continuous vs. Discrete

Continuous - defined for all values in range Discrete - defined for a set of discrete

points in range.

Sampling Theory

Point Sampling

start with continuous signal calculate values of signal at discrete,

evenly spaced points (sampling)

convert back to continuous signal for

display or output (reconstruction)

Sampling Theory

Foley/VanDam

Sampling Theory

Sampling can be described as creating a

set of values representing a function evaluated at evenly spaced samples n i i f fn , , 2 , 1 , ) ( K = ∆ =

∆ = interval between samples = range / n.

1 2

…

n

Sampling Theory

Sampling Rate = number of samples per unit

∆ = 1 f

Sampling Theory

Example -- CD Audio

sampling rate of 44,100 samples/sec ∆ = 1 sample every 2.26x10-5 seconds

7

Sampling Theory

Rich mathematical foundation for

sampling theory

Hope to give an “intuitive” notion of

these mathematical concepts

Sampling Theory

Spatial vs frequency domains

Most well behaved functions can be

described as a sum of sin waves (possibly

• ffset) at various frequencies

Describing a function by the contribution

(and offset) at each frequency is describing the function in the frequency domain

Higher frequencies equate to greater detail

Sampling Theory

Foley/VanDam

Sampling Theory

Nyquist Theorum

A signal can be properly reconstructed if

the signal is sampled at a frequency (rate) that is greater than twice the highest frequency component of the signal.

Sampling Theory

Nyquist Theory

Said another way, if you have a signal with

highest frequency component at fh, you need at lease 2fh samples to represent this signal accurately.

Sampling Theory

Example -- CD Audio

sampling rate of 44,100 samples/sec ∆ = 1 sample every 2.26x10-5 seconds

8

Sampling Theory

Nyquist Theory -- examples

CDs can accurately reproduce sounds with

frequencies as high as 22,050 Hz.

Sampling Theory

Aliasing

Failure to follow the Nyquist Theorum

results in aliasing.

Aliasing is when high frequency

components of a signal appear as low frequency due to inadequate sampling.

Sampling Theory

Aliasing - example Foley/VanDam

Sampling Theory

Annoying audio aliasing applet Example of aliasing in animation.

Sampling Theory

Anti-Aliasing

What to do in an aliasing situation

Increase your sampling rate (supersampling) Decrease the frequency range of your signal

(Filtering)

How do we determine the contribution of

each frequency on our signal?

Sampling Theory

Fourier analysis

Given f(x) we can generate a function F(u)

which indicates how much contribution each frequency u has on the function f.

F(u) is the Fourier Transform Fourier Transform has an inverse

9

Sampling Theory

Fourier Transforms

Fourier Transform Inverse Fourier Transform f(x) F(u) f(x)

Sampling Theory

How do we calculate the Fourier

Transform?

Use Mathematics For discrete functions, use the Fast Fourier

Transform algorithm (FFT)

Sampling Theory

Anti-Aliasing

What to do in an aliasing situation

Increase your sampling rate (supersampling) Decrease the frequency range of your signal

(Filtering)

Since we already have the data sampled,

we can’t supersample motion capture data

Thus, we need to filter

Sampling Theory

Filtering -- Frequency domain

Place function into frequency domain F(u) simple multiplication with box filter S(u)

⎩ ⎨ ⎧ ≤ ≤ − = elsewhere , when , 1 ) ( k u k u S

Sampling Theory

Filtering - frequency domain Foley/VanDam

Sampling Theory

Filtering -- Spatial Domain

Convolution

∫

∞ ∞ −

− = ∗ = τ τ τ d x g f x g x f x h ) ( ) ( ) ( ) ( ) (

Taking a weighted average of the neighborhood around each point of f, weighted by g centered at that point.

10

Sampling Theory

Convolution Applet

Sampling Theory

Convolution and Filtering

Convolution in the spatial domain is

equivalent to multiplication in the frequency domain

Use Fourier Transform to convert filter

from spatial to frequency & visa versa

Sampling Theory

Convolving with a sinc function in the spatial

domain is the same as using a box filter in the frequency domain

Foley/VanDam

Sampling Theory

Anti-aliasing -- Filtering

Removes high component frequencies from

a signal.

Removing high frequencies results in

removing detail from the signal.

Can be done in the frequency or spatial

domain

Sampling Theory

Filtering - Convolution Foley/VanDam

Motion capture data

So what does all this mean w.r.t. motion

capture data?

To avoid aliasing must filter before modifying data

in time

Motion capture sampling rates can be as high as 144

samples / sec

Filtering can also remove “noisy” data by

removing high frequency components.

Questions? Break!

11

Retargeting Motion Capture Data

In general, moCap data is useful for a

single articulated figure.

Retargeting Motion Capture Data

An inverse kinematic problem

• Eg. Walking – want feet on floor.

[Gleicher98] takes a spacetime

constraints approach.

Spacetime Constraints

The problem turns into a constrained

• ptimization problem

Find values Sj that minimize R subject to Ci

(Sj) = 0

Si = DOF and forces for all time steps Ci = constraints R = minimization criteria

Given these, there are well known

numerical techniques to solve

Spacetime Constraints and Retargeting

Si = joint angles Constraints:

Joint constraints (elbows don’t bend

backwards)

Environment constrains (must not go

through floor)

Motion constraints (char must pick up box

at frame 50)

Spacetime Constraints and Retargeting

Minimization criteria

Minimize “noticable change” from original

data

Minimize difference of angles from original

data

Minimize high frequency content of

changes

Spacetime Constraints and Retargeting

Video

12

Motion capture data formats

No “standard” moCap data format

Defacto standards from motion capture

system manufacturers

Must specify both structure of skeleton

as well as sampled data for each joint.

Motion capture data formats

Popular formats

Acclaim File Format

.asf (Acclaim skeleton format) .amc (Acclaim motion capture)

Biovision

.bva (BioVision animation) .bvh (BioVision Hierarchical)

C3D

Independent Binary format with programmer support. http://www.c3d.org

Acclaim

.asf file .amc file

.bvh file Questions?