Overview Overview Movement Paradigms System Architecture - - PDF document

▶
overview
SMART_READER_LITE
LIVE PREVIEW

Overview Overview Movement Paradigms System Architecture - - PDF document

Dec 28, 2023 210 likes •386 views

<latexit

slide-1
SLIDE 1

xP

<latexit sha1_base64="3iSalipRGSrk925a/EoIH3I+Gk=">AB83icbVDLSgMxFL3js9ZX1aWbYBFclZkq6LoxmUF+4DOWDJpg3NZEKSEcvQ3DjQhG3/ow7/8ZMOwtPRA4nHMv9+SEkjNtXPfbWVldW9/YLG2Vt3d29/YrB4dtnaSK0BZJeK6IdaUM0FbhlOu1JRHIecdsLxTe53HqnSLBH3ZiJpEOhYBEj2FjJf3rwY2xGYZg1p/1K1a25M6Bl4hWkCgWa/cqXP0hIGlNhCMda9zxXmiDyjDC6bTsp5pKTMZ4SHuWChxTHWSzFN0apUBihJlnzBopv7eyHCs9SQO7WSeUC96ufif10tNdBVkTMjUEHmh6KUI5OgvA0YIoSwyeWYKYzYrICtMjK2pbEvwFr+8TNr1mndeq9dVBvXR0lOIYTOAMPLqEBt9CEFhCQ8Ayv8Oakzovz7nzMR1ecYucI/sD5/AFS25He</latexit>

Expanded Virtual Puppeteering

Bernard Lupiac / Luiz Velho

VISGRAF Lab - IMPA

slide-2
SLIDE 2

Overview

  • Overview
  • Movement Paradigms
  • System Architecture
  • Presentation and Networking
  • Media Project Authoring
  • Proof-of-Concept

Video

slide-3
SLIDE 3

Movement Paradigms

  • Animations
  • Inverse Kinematics
  • Physics-Based Simulation

Animations

  • Animation Clips


(pre-recorded)

  • Blend Trees


(transitions)

  • Interactive Control


(selection)

Animator Component Animator Controller Animator Clip

slide-4
SLIDE 4

Inverse Kinematics

  • Articulated Structures


(human skeleton)

  • Control of End Effectors


(hands, feet)

  • IK Pass

Physically Based Simulation

  • Interaction with Environment


(collision detection)

  • Forces and Torques


(gravity, etc..)

slide-5
SLIDE 5

X-Puppet Architecture

  • Gesture Recognizer
  • Movement Modes
  • Presentation and Network
slide-6
SLIDE 6

Gesture Recognizer

Mode Selection

  • Input: Leap Motion
  • Detection: SVM


(ML Library)

Movement Modes

  • Head
  • Hands and Arms
  • Jump
  • Locomotion
slide-7
SLIDE 7

Head

Simple Inverse Kinematics


  • Palm of Hand Direction

Hands and Arms

Complex Inverse Kinematics

  • Hand Position and Rotation
  • Head follows Hands
slide-8
SLIDE 8

Jump

Simple Animation

  • Cycles btw Animations

  • Idle Standing
  • Jump / Fall : Hands Up Velocity
  • Idle Ground
  • Get Up : Palm Facing Upwards

Locomotion

Complex Animation
 (Blend Tree)

  • Walking and Running
  • Control Widget
slide-9
SLIDE 9

Locomotion - Control

  • Joystick Paradigm

  • Reference Sphere : Dead Spot
  • Hand Position Vector : Velocity / Direction
  • Locomotion Animation : Speed

Presentation & Network

Synchronization of Objects : Holojam SDK

  • Class Structure

Manipulator instance

Host Movement Processing Leap XR Service Provider (external) Host Hand States Leap Gesture Recognizer Holojam Server Targets IK Mannequin Controller Mannequin Controller Movement Manager Animator Puppet Master (external) Hand Getter

Server

Any instance with a Mannequin

slide-10
SLIDE 10

Presence / Visualization

  • Performers
  • Play Interaction
  • Audience

Performers

  • Leap Motion
  • Attached to User’s Forehead
  • Feedback
  • Hands
  • Mode
  • Etc..
slide-11
SLIDE 11

HUD

  • Screen Feedback (Heads-Up Display)

Bezel + Text Minimap

Play Interaction

  • Director
  • Live Cinema
  • Other Characters
  • Avatars
  • VR Mode
slide-12
SLIDE 12

Audience

  • Theatre Screen
  • Virtual Reality
  • Augmented Reality

Media Project Authoring

  • Gesture Sets
  • Puppets
slide-13
SLIDE 13

Gesture Sets

  • Built-In Gesture Recognizer
  • Default : 10 Gestures
  • Changing / Adding Gestures
  • Training / Compiling
  • Robustness and Reliability
  • Distinct Hand Poses
  • Redundancy (use two hands)

Puppets

  • Humanoid


(compatible rigged model)

  • Enforce T-Pose
  • Add Animator Component
  • Add Targets IK
  • Add Leap Control
  • General


(custom skeleton)

  • Adapt Everything!
slide-14
SLIDE 14

Applications Proof of Concept

  • The Framework in Action
  • “O Boneco”
  • Test and Refine Expressiveness
  • Short Piece
  • Collaboration with Professional Puppeteers
  • Bando Criação Cegonha
slide-15
SLIDE 15

O Boneco

  • Creative Team
  • Screenplay and Direction : Vida de Oliveira
  • Puppeteering Animation : Miguel Araujo
  • Agile Methodology
  • Iterative Workflow
  • Test / Improve
  • Weekly Rehearsals
  • Feedback
  • Development
slide-16
SLIDE 16

Web Portal

  • https://www.visgraf.impa.br/projects/puppet/

The Show

slide-17
SLIDE 17

– o Boneco

“See you soon!”