[PPT] - Transferring Human Skills to Humanoid Robots Dongheui Lee PowerPoint Presentation

SLIDE 1

Transferring Human Skills to Humanoid Robots

Dongheui Lee

dhlee@tum.de b

Dynamic Human-Robot-Interaction for Automation Systems (HRI) Lab

Department of Electrical Engineering and Information Technology Technical University of Munich Technical University of Munich

SLIDE 2

Transferring Human Skills to Humanoid Robots

Movements

learning

Manipulation

whole body

Pysical HRI

contact
learning

motion

recognition
reproduction
whole body

coordination

grasping skills
interaction
contact

estabilishment

physical

coaching

reproduction
interaction

force control policy coaching

haptic assistance

in collaboration

SLIDE 3

Programming by Demonstration Mirror Neurons Mirror Neurons

Observing man picking feed Monkey picking feed

Monkey Brain: F5 Activities of Mirror Neuron （F5） [Gallese et al.1996] [Rizzolatti et al. 1996].

SLIDE 4

Programming by Demonstration Mathematical formulation of Mirror Neurons Mathematical formulation of Mirror Neurons

Mimesis Model
Probabilistic representation for

Probabilistic representation for spatiotemporal data

Learning, recognition,

Learning, recognition, generation (a bidirectional computational model)

a11 a a a22 aN-1 N-1 aNN

Mimesis from partial
bservation

aN1 S1 S2 SN-1 SN

…

a12 aN-1 N b1 b2 bN 1 bN b1 b2 bN-1 bN

[Lee and Nakamura IJRR 2010]

SLIDE 5

Programming by Demonstration Mathematical formulation of Mirror Neurons Mathematical formulation of Mirror Neurons

Mimesis Model
Probabilistic representation for

Probabilistic representation for spatiotemporal data

Learning, recognition,

Learning, recognition, generation (a bidirectional computational model)

Mimesis from partial
bservation

[Lee and Nakamura IJRR 2010] [Lee and Nakamura IJRR 2010]

SLIDE 6

Motion Recostruction from Monocular Vision

SLIDE 7

Recognition from Optical Flow

Biological Movement [Johansson 1975] Aim to recognize and recover the motion from the optical flow of feature points

SLIDE 8

{C} {I} {D} {

2D Optical Flow

CT

p

Iλ

D x

λ

C x

λ

θ

λ

→ → →

DT

x

λ

x

θ

Motion Primitives in Joint Space { Joint angle, angular velocity, Demonstrator Cartesian Camera Cartesian Image Cartesian

HMM

base-joint velocity}

HMM Human behavior Humanoid behavior

Generation Recognition

3D Whole

Conditioned by Observation from partial

bservation

Partial Observation: Occluded Monocular Image 3D Whole body Motion

SLIDE 9

Motion Recostruction from Monocular Vision

[Lee and Nakamura IROS 2007]

Human perception of biological movements Human perception of biological movements

[Lee and Nakamura IROS 2007]

Activity recognition
6 motions

magenta : True Model blue: Recovered Model

Motion Capturing 56DOF

SLIDE 10

Transferring Human Skills to Humanoid Robots

Movements

learning

Manipulation

full body

Pysical HRI

contact
learning

motion

recognition
reproduction
full body

coordination

grasping skills
interaction
contact

estabilishment

physical

coaching

reproduction
interaction

force control policy coaching

haptic assistance

in collaboration

SLIDE 11

Grasping Skill Learning from Motion and Force Data

Learning grasping skills from motion and force patterns
Teleoperation using Cyberglove, Flock of Birds, &

C b (H ti F db k) Cybergrasp (Haptic Feedback)

SLIDE 12

Grasping Skill Reproduction

Parallel position (PD) and force (PI) control

} { ) ( ) , ( ) (

∫

+ + + = − = + + dt e k f J e k e k f J q g q q q C q q M

f f d T d T

& & & & & τ τ } {

∫

+ + + = dt e k f J e k e k

f f d p d p p

τ

z x

SLIDE 13

Learning Interaction vs. Internal Forces

Interaction force learned Static internal force learne Static internal force learne demonstrations

SLIDE 14

Generalization Capability: Radius

SLIDE 15

Transferring other manipulation skills

Mechanism for Association of Whole Body Motion from

Tool Knowledge

Tool in Body Schema [Maravita and Iriki 2004]

Tool in Body Schema [Maravita and Iriki 2004]
e.g. Distal-type neurons
[Lee et al IROS2008] [Kunori,Lee,NakamuraIROS2009]
Learning interaction control policies
Dynamic movement primitives for parallel position and

force control

Deformable objects, sculpting tasks

j , p g

[Koropouli, Lee, Hirche, 2011 IROS]

SLIDE 16

Transferring Human Skills to Humanoid Robots

Movements

learning

Manipulation

whole body

Pysical HRI

contact
learning

motion

recognition
reproduction
whole body

coordination

grasping skills
interaction
contact

estabilishment

physical

coaching

reproduction
interaction

force control policy coaching

haptic assistance

in collaboration

SLIDE 17

Simple Human Robot Interaction

AUTOMATICA 2010

Collaboration with Dr Ott Dr Albu-Schaeffer Haddadin DLR Collaboration with Dr. Ott, Dr. Albu Schaeffer, Haddadin, DLR

SLIDE 18

Imitation Learning Imitation Learning

Teaching Execution

Physical interaction

g

Mimetic Communication [Lee et al IJRR 2010]

SLIDE 19

Motivation: Motion Interaction

From the Movie “Terminator 2 Judgment Day”

Issues for pHRI:

H

ti i it ti

M

k C t l

Human motion imitation Marker Control Learn/Recognize/Generate Motion Primitives Mimesis Model Learn/Recognize/Generate Interaction Rules Mimetic Communication Model

ea / ecog e/Ge e ate te act o u es et c Co u cat o

de

Contact transition Real-time motion adaptation Application : High-Five like interaction

SLIDE 20

Motion Imitation by Marker Control

Dynamics of the humanoid’s upper body on a free-floating base body:

⎛ ⎞ ⎛ ⎞ ⎛ ⎞ && & ( ) ( , , ) q q M q C q q x x x f τ ⎛ ⎞ ⎛ ⎞ ⎛ ⎞ + = ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎝ ⎠ ⎝ ⎠ ⎝ ⎠ && & & & && &

Virtual Springs:

2

1 ( ) ( ) V q x k r r q x

Measured marker position Marker pos. of the simulation

,

( , ) ( , ) 2

i i d i i

V q x k r r q x = −

,

( ) ( )( ( , ))

T i i d i i i M

q D q k J q r r q x f x τ

∀ ∈

⎛ ⎞ ⎛ ⎞ = − + − ⎜ ⎟ ⎜ ⎟ ⎝ ⎠ ⎝ ⎠ ∑ & &

SLIDE 21

Motion Imitation by Marker Control

Upper body Control: Marker trajectory following
Lower body Control: Balancing, Hip orientation and Height following

[Ott, Lee, Nakamura, “Motion Capture based Human Motion Recognition and Imitation by Direct Marker Control”, Humanoids 2008]

SLIDE 22

Full Body Motion Imitation

SLIDE 23

Motion Learning Interaction Learning

HMM

interactive primitive

HMM

interactive primitive

HMM

motion

HMM HMM

motion primitive

HMM HMM

motion primitive

HMM

motion primitive recognition generation

behavior behavior behavior behavior behavior human robot

Mimetic Communication Model

human robot

l i i i & i learning, recognition & generation

f interaction primitives
How to react to human’s action
Contact location & timing

SLIDE 24

Physical Contact Establishment

Adaptation of the robot’s motion to the

desired contact point in real-time: 1) Use additional spring (red) connected to the desired contact point. 2) Project the forces of the hand’s marker springs (green) i b l d h h d i i

, \

( ) ( )( ( , ))

T i i d i i i M H

q D q k J q r r q x f x τ

∀ ∈

⎛ ⎞ ⎛ ⎞ = − + − ⎜ ⎟ ⎜ ⎟ ⎝ ⎠ ⎝ ⎠

∑

& &

into a subspace related to the hand orientation.

, , , , ,

(1 ) ( )

k h k k w k T h k k R L w k

F F J q T δ δ

=

⎝ ⎠ ⎝ ⎠ + − ⎛ ⎞ + ⎜ ⎟ ⎝ ⎠

∑

Position control (Position based) Impedance control

Distance information

smooth transition

contact/non-contact

Position control (Position based) Impedance control

Limiting the contact forces Implementing “smooth” contact

SLIDE 25

Experiments

12 motion primitives and 8 interaction primitives
Implementation to humanoid robot (38DOF), 30DOF is controlled.
Position based Impedance Control to the Upper body

p pp y [Lee, Ott, Nakamura, ICRA 2009] [Lee, Ott, Nakamura, IJRR 2010]

SLIDE 26

Imitation Learning Imitation Learning

Teaching Execution

Physical interaction

g

Mimetic Communication [Lee et al IJRR 2010] Physical coaching [Lee & Ott, Autonomous Robots 2011] [ ]

SLIDE 27

Demonstration Technique Observational Demo Observational Demo Kinesthetic Demo Kinesthetic Demo

synchronized whole body motion

Unsynchronized body motion
Accidental disturbance

correspondence problem No correspondence problem

SLIDE 28

Overview

SLIDE 29

E

Multi observations (reproduced &

∑ ∑

− E T t e ij e

e

t

1 1 1

) ( ξ ω

∑ ∑

E T e s e e

e

t

t

) ( ) ( γ ω

∑

=

e e i e i 1

) 1 ( γ ω π

i hti f t (reproduced &

bserved)

∑ ∑ ∑ ∑

= − = = =

=

E e T t e i e e t ij ij

e

t a

1 1 1 1 1

) ( ) ( γ ω ξ

∑ ∑ ∑ ∑

= = = =

=

E e T t e i e e t i i s

e

t t

t

1 1 1 1

) ( ) ( ) ( γ ω γ ω μ

weighting factor

∑ ∑

T E T

1

=

∑

E e

ω

∑ ∑ ∑ ∑

= = = =

− − = Σ

E e T t e i e T i s e s i s E e T t e s e i e i s

e e

t t

t
t

1 1 1 1

) ( ) ) ( )( ) ( )( ( γ ω μ μ γ ω

1 = e

SLIDE 30

Motion Primitives

– No preprocessing for – No preprocessing for learning & recognition – Correlation of temporal & i l d

48 1 3

N t N t

T Σ + + − + = μ

& spatial data

HMM

4

N

T + μ

Data

) , | ( ) ( λ ζ O S q P t

t i t i

= =

) ( ) ( ) (

1

t

t

t

i

s N i i s

∑

=

= ζ

N

) (

1 t

s

) (

2 t

s

No slower filter for

Gaussian regression

) ( ) ( ) (

| 1 |

t t t

i t s i i t s

Σ = Σ

∑

=

ζ ) (

1

– No slower filter for smoothing trajectory

Gaussian regression

SLIDE 31

SLIDE 32

Compliant Control

Requirements

1. Precise tracking in free motion for motion primitive generation 2. Compliant interaction with low stiffness during teaching 2. Compliant interaction with low stiffness during teaching 3. Refinement tube: Limit the allowed deviation from the motion primitive

Integration into customized impedance controller

) ~ ( ~ ) , ( ) ( ) ( q s q D q q q C q q M q g

d d

− − + + = & & & & & τ δ Σ ) ( 3

|t s

ε δ + Σ = ) ( 3

|

t

t s

SLIDE 33

Experiments – Physical Coaching

Impedance Control and Motion Refinement Tube

Without tube With tube

Undesired accidental disturbance Guide for easy physical coaching

SLIDE 34

Incremental Refinement

M ti R t ti f Motion Retargeting from human body motion M ti R fi t b Motion Refinement by Kinesthetic Coaching Refined Robot Motion

SLIDE 35

Incremental Learning : Unsupervised Segmentation and Clustering Segmentation and Clustering

Segmentation Clustering Beh Graph [Kulic, Lee, Ott, Nakamura, IJRR 2011] Segmentation Clustering

Beh. Graph

SLIDE 36

Parallel Learning, Prediction, Execution

Segmentation Clustering

Beh. Graph

P ll l i f Parallel execution of

nline learning and

prediction

SLIDE 37

Experiment in 2D Virtual Scenario

2D virtual scenario
No initial knowledge
No initial knowledge
As learning proceeds, prediction

starts

Robot behavior is changed from

“passive follower” to “load h i ” sharing”

SLIDE 38

Experiment

Risk Sensitive Stochastic Optimal Control for Haptic Assistance [Medina, Lee, Hirche, ICRA 2012] [Medina, Lee, Hirche, ICRA 2012]

SLIDE 39

Conclusion

Manipulation Pysical HRI Movements

grasping skills

from position and force patterns

learning pHRI

tasks (give-me- five)

mirror neuron

mimesis model force patterns five)

physical coaching

for incremental learning model

self vs others

: motion skills learning & learning

Human intention

recogntiion for collaboration learning & recognition

S f d h l Safe and Autonomous Physical Human-Aware Robot Interaction

SLIDE 40

Thank you Thank you

for your attention

Additional Questions? Q Email: dhlee@tum.de