Multitouch Puppetry Creating coordinated 3D motion for an - - PowerPoint PPT Presentation

Multitouch Puppetry

Creating coordinated 3D motion for an articulated arm

DFKI Embodied Agents Research Group Cluster of Excellence Multimodal Computing and Interaction Saarbrücken, Germany

Michael Kipp Quan Nguyen

Motivation

• Non-experts perform 3D character

animation in realtime... from their desktops

• Why?

➡ animations for games / online worlds / fun ➡ „pose“ scenes for movie / theater ➡ dance choreography ➡ produce sign language ➡ teleoperate robots

Character Animation

• Two ways:

➡ Pose-to-pose:

precise, final production

➡ Straight ahead:

creative, improvisational

• Straight-ahead animation ideally has

parallel realtime control of all degrees of freedom

Performance Animation

• Use physical widgets, recognized

by camera, to create motions

[Oore et al. 02, Dontcheva et al. 03] ➡ special hardware ➡ fatigue

• Use simple mouse input, exploit

correlations between body parts

[Neff et al. 07]

• Shortcomings

➡ „layering“ of motion ➡ no solutions for hand shape

Problem & Scope

shoulder elbow wrist hand shape

Problem & Scope

hand shape (many DOFs) Move wrist and use Inverse Kinematics (IK) for resolving other joints => 3 DOFs hand

• rientation

(1 DOF) Arm swivel (1 DOF)

Multitouch Interface

• As many input DOFs as possible

➡ bimanual, multi-finger

• Easy to learn

➡ standard interaction techniques (mostly)

• Directness of control

➡ touch & output co-located ➡ no direct manipulation (occlusion of hand shape)

Dominant Hand: Arm

x/y z swivel

Nondominant Hand: Hand

• Index finger: hand shape

➡ High-dimensional space ➡ Few hand shapes suffice

(open, fist, thumbs up...)

➡ 2D morph map

• 2-Finger rotation: hand orientation

Nondominant Hand: Hand

shape hand

• rient.

7 DOF bimanual control

spread out closed base shapes specific

Technically:

• max. 3 shapes for

interpolation similar shapes close

Design guidelines:

good „pass-through“ in center memorable „themes“

User Study

User Study

• Long-term study: 7 sessions (2 weeks)
• Participants: 6 students (3+3 male/female),

age 21-30

• Hardware: 15.4“ screen
• Conditions: multitouch vs. mouse
• Three layered design to capture performance &

user experience

• Within-subject: participants compared mouse

and multitouch (measures, questionnaire)

Control condition

• Mouse interface
• Motion → frontal plane
• Wheel → depth
• Keyboard key → wheel mode

➡ arm swivel ➡ hand rotation

Task 1: Match Pose

• Test precision of interface with

matching task (docking)

• Target pose shown with ghost arm

(10 poses per session)

• Bimanual control

Hand orientation Hand shape Hand position Arm swivel

Task 1: Match Pose

Task 2: Trace Trajectory

• Encourage fluid, coordinated

3D motion

• Traces from motion

captured gestures (10 per session)

• Moving highlight indicates

direction

• Unimanual control

Hand position Arm swivel

Task 2: Trace Trajectory

Task 3: Creative Exploration

• Qualitative test of complete interface
• No „fair“ control condition
• Task: Create „appropriate“ motion for

➡ song (strong beat) ➡ song (slow ballad) ➡ voice track (male harsh) ➡ voice track (female melodic)

Hand shape Hand orientation Hand position Arm swivel

Non-Expert Subjects in the Creative Task Full multitouch interface

P i l

• t

s t u d y

Results

• Measure completion time (tasks 1+2)
• Pilot studies

➡ Mouse outperformed MT

• Hypothesis

➡ Mouse beats MT in first sessions

but both converge after...

Mouse Multitouch

pose matching tracing

Final Results

Measuring Coordination

• Existing measures

➡ efficiency [Zhai, Milgram 98] and parallelism [Balakrishnan, Hinckley 99] require an optimal target path ➡ integrality [Jacob et al. 94] ignores quantity

• Suggestion

➡ assume that maximum coordination happens if distance travelled along each dimension is equal ➡ penalize digression from this optimum ➡ compare distance travelled in each dimension

min(dxy,dz) max(dxy,dz)

xy z

• ptimum

1 N

N −1

∑

Coordination in Session 7

Multitouch yields significantly higher coordination for both tasks 1 + 2 (p < .05)

Questionnaires

• 16 questions
• 5-point differential scale: -2 Mouse ... +2 MT
• Bonferroni correction: alpha of .003
• For 6 Q significant differences from 0, all in favour of MT:

➡ Which interface was more useful? ➡ Which interface allowed a faster solution of the tasks? ➡ With which interface were you more satisfied? ➡ Which one would you recommend to others? ➡ Which device was more fun to use? ➡ Which interface do you prefer?

Discussion

• Performance & user experience

➡ Big leap from session 1 to 2 =>

multi-session studies

➡ MT feels faster, although objectively equal,

probably due to higher coordination

➡ Comfortable only in session 2

• MT yields higher coordination

➡ < 0.1 bad coordination ➡ > 0.2 already good

Conclusion

• Goal: Continuous, parallel control of high-dimensional spaces
• Multitouch interface for coordinated arm motion
• Create & eval complex interaction techniques

for concrete application (3D) tasks

• Contributions

➡ bimanual 7-DOF input arm animation interface ➡ 3-layered study design ➡ Novel coordination measure

• Future

➡ prototyping tool (dataflow) ➡ scale up to whole body

T h a n k y

• u

!

[Neff et al. 06]

Try it!