Motion capture Captures the subtlety of human motion Keyframing - - PowerPoint PPT Presentation

Motion capture

• Captures the subtlety of human motion
• Keyframing difficult and not as successful
• Cons:

– We can use only prerecorded material (with certain adaptations)-limiting – Data valid only for same scale avatar-need for retargeting – How to satisfy constraints

Applications of MoCap

• TV and film
• Computer games (85%): no need for top

quality motion data

– Motion of points on the surface does not lead to successful motion of skeleton

• Virtual reality (control of avatars,

autonomous avatars)

Low level processing

• Requires effort
• Noise removal
• Filling the gaps when marker is lost
• Eliminating confusion when markers

coincide

• Conversion from positional data to joints

rotations for driving a skeleton

• Storage of a vocabulary of sequences
• Blending and adaptation in real time to

generate new sequences

• Time consuming ops like retargeting off

line.

Rotation around the three local coordinate axes

The Biovision BVH

• Hierarchical representation of skeleton
• Translation-rotation to the root node
• Rotation to other nodes (joints)
• Offset between nodes (link dimension)

MoCap data editing

• MoCap data can be manipulated up to a

certain point before their integrity and naturalness becomes compromised.

• If editing does not provide the desired

sequence then we should capture a new sequence.

Changing the speed of motion

• K>1 speeding up (decimation)
• K<1 slowing down (interpolation)
• Speeding up might also require

interpolation if k non-integer

Changing the speed of motion

Changing the speed of motion

Changing the speed of motion

Changing the speed of motion

Blending

• Usually to provide transition between sequences
• Fade in/out parts of two sequences

Blending

• For blending sequences represented as B-

splines, simply blend control points

• Blending has no theoretical validity
• Blending a run and a walk might not

generate a valid/realistic transition

• Works well when sequences are aligned and

similar

Blending

• Similar: different manifestations of the

same action e.g. walk

• Aligned key poses (e.g. foot on ground)
• ccurring at the same time.
• Need for non-uniform re-sampling: time

warping

Time warping

Multi-target interpolation

• Blending two sequences (all over their

length) to create a new sequence

– Interpolation between a tired and brisk walking sequence -> sequence with both flavours.

• Sequences need to be aligned

No alignment: uncoordinated motion, feet might not even touch ground

Time warping

• Warping of sequences can create new

sequences without blending

• Warp between brisk walk and drunken,

slow walk

– Drunken->brisk = drunken walk at brisk pace – Brisk->drunken = brisk walk at drunken pace

Time warping

• Difficult step: how to find alignment

automatically

• It might be impossible/inapropriate to find a

suitable time warp

– Complex motions (gymnastics), completely different motions

• Blending after warping in such cases will

create unfeasible / unrealistic motion

Time warping

• Similarities with contour triangulation &

speech recognition

• Sequences A and B of length n
• To each vertex of one signal, assign (at

least) one vertex in the other signal so as to minimize a cost function measuring their “difference”

• Try different correspondences between

points in A and B, use dynamic programming

Time warping

• Finding correspondences is equivalent to

finding an optimal path on a 2D grid

Time warping

• Cost: how much work it takes to deform
• ne signal into the other.
• Sum of local stretching and bending

– Stretching: difference in distances between two consecutive samples on the two curves – Bending: difference in angle between three consecutive samples on the two curves

Time warping

Time warping

• Once correspondences are found actual

warping should be performed (B to A)

• substitution: 1:1 correspondence of

successive samples. (diagonal line)

• deletion: multiple samples of B map to

a sample of A. (horizontal line)

• insertion: a sample of B maps to

multiple samples of A (vertical line)

Time warping

Time warping

• Substitution: Bj corresponds to Ai.

– Bwi = Bj.

• Deletion (Bj ,Bj+1 ,…Bj+k) correspond to Ai,

– Bwi =mean (Bj ,Bj+1 ,…Bj+k)

• Insertion Bj corresponds to (Ai ,Ai+1 ,…Ai+k)

– (Bwi ,Bwi+1 ,…Bwi+k) evaluated by fitting a spline around Bj

Motion warping

• Making local adjustments to the amplitude
• f motion data

– Alter walking motion so that character bends to walk through a doorway – Increase the height of a jump to clear new

• bstacle
• Use of a displacement function / map
• f’(t) = f(t) + d(t)

Motion warping

• Must interpolate to ‘spread’ the effect

Motion warping

• For real time motion control:

– Predict interaction with object or obstacle – Pick up appropriate displacement map depending on object/obstacle position – Compose new motion

Blending: Verbs and Adverbs

• Extension of blending between two

sequences.

• Verb: a certain type of motion e.g. walk
• Take a number of example motions from

the verb and interpolate to obtain a continuous space of motions in this verb

Blending: Verbs and Adverbs

• Parametrize verb by adverbs (adverb space)

– Happy-sad, knowledgeable-clueless

• Use sparse data interpolation methods (e.g.

using RBFs) over the control points of the examples

• For a certain frame i use pi for all examples

to fit a continuous function (combination of RBFs)

Blending: Verbs and Adverbs

• Examine recorded motions and place them

in appropriate positions on the adverb plane according to their characteristics

• Examples should be time aligned (time-

warped): similar key poses to occur at the same time.

MoCap signal processing

• MoCap data are 1-D discrete time data,

samples of a continuous signal

• Classic 1-D signal processing principles

apply

• However MoCap data are angular (Euler

angles)

• Classical theory for data on the line does

not apply in the strict sense.

• Acceptable if small movements occur

MoCap signal processing

• Aliasing: sampling frequency must be at

least twice the largest frequency of the signal

– Lowpass filtering

• Problems in change of speed operations

MoCap signal processing

• Data have a finite length, start and end

abruptly

• This introduces high frequencies to the

signal: leakage

• A data weighting function or a non-

rectangular window should be used on the MoCap data

MoCap signal processing

• Hanning window
• w(t)=cos2(πt/T0), |t|<=T0/2
• Windowing is also needed in concatenation
• f a motion sequence to create a cyclic

(repeated) motion when the start/end of the sequence does not match

– High frequencies are introduced if no windowing is used.

MoCap signal processing

• Filtering with appropriate filters can change

the character of the motion

• Walking sequence

– Increase middle freqs: smoothed but exaggerated – Increase high freqs: nervous, jerky – Increase low freqs: attenuated, reduced joint movement

Constraint based approaches

• Change mocap data in order to satisfy

constraints

• Different from what we had so far
• Motion fitting or re-targeting

Constraint based approaches

• Adapt a sequence to a character of different

scale

• Fit “switch on light” data to a different

character

– Joint angles have to be changed in a non-linear manner – Some angles might not have to be changed at all – Possible collision detection

Constraint based approaches

• Adapt a sequence to make a new sequence
• Adapt a “grasp object” sequence to “grasp a

different object” sequence for the same character

– Can use collision detection and IK to modify MoCap data and prevent penetration

Kinematic constraints approach

• Joint limits, end effector positions
• Retargetting a “turn on switch” motion to

another character.

• Basic question: what is more important to

be maintained end effector positions or joint angles?

Kinematic constraints approach

• Assign importance metric to different constraints

– Root position is usually not important – The position of end effector is important if it is interacting with an object – The position of end effector is important if it is near the

• bject, moving towards the object
• Use of heuristic rules to specify importance
• Change aspects that are not important.

Dynamic constraints approach

• Space-time constraints
• Use forces and apply constraints on them
• Minimize some energy function
• Evaluate the trajectories