Mobile Robotics Lab Workshop on Grasp Planning and Task Learning by - - PowerPoint PPT Presentation

▶
mobile robotics lab workshop on grasp planning and task
SMART_READER_LITE
LIVE PREVIEW

Mobile Robotics Lab Workshop on Grasp Planning and Task Learning by - - PowerPoint PPT Presentation

Nov 14, 2023 379 likes •666 views

Mobile Robotics Lab Workshop on Grasp Planning and Task Learning by Imitation IROS 2010 Taipei, Taiwan Institute of Systems and Robotics Institute of Systems and Robitcs http://paloma.isr.uc.pt University of Coimbra Symbolic level

slide-1
SLIDE 1

University of Coimbra

Institute of Systems and Robitcs

http://paloma.isr.uc.pt

Institute of Systems and Robotics

Mobile Robotics Lab Workshop on Grasp Planning and Task Learning by Imitation

IROS 2010 – Taipei, Taiwan

slide-2
SLIDE 2

University of Coimbra

Symbolic level generalization of in-hand manipulation tasks from human demonstrations using tactile information

Ricardo Martins, Diego R. Faria and Jorge Dias

Ricardo Martins

http://www.isr.uc.pt/~rmartins/ Taipei, Taiwan 18th August 2010

slide-3
SLIDE 3

University of Coimbra

Contents

  • Introduction
  • Approach Overview
  • Contact Templates Definition
  • Contact Templates Detection
  • Conclusions
  • Future Work
slide-4
SLIDE 4

University of Coimbra

Development of robotic systems able to interact in new dynamic, unpredictable environments:

  • Domestic, healthcare, entertainment, education.
  • New challenges: Interaction Robots & Humans, Interaction Robots & Environment
  • Development of mobile robotic platforms:
  • Multimodal sensing capabilities:

+ Active vision + Audition + Multi Articulated Arms + Dexterous hands + …

  • Emergence of integration recognition, interaction and learning issues.

I - Introduction

slide-5
SLIDE 5

University of Coimbra

Evolution:

simple gripper human inspired articulated hands

  • Development of human inspired robotic hands:

Anatomy: Move from simple grippers towards human inspired articulated hands. Physiology: Introduction of sensing devices (tactile, temperature, force/torque sensors)

I - Introduction

slide-6
SLIDE 6

University of Coimbra

  • The neuroscience literature (Johansson and Flanagan, 2009) proposes a

decomposition of a typical human manipulation movement on different stages

  • reach -load -lift -hold -replace -unload

I - Introduction

Reach Load Lift Holding Replace Unload Digits contact

  • bject

Object lifts off surface Object approachs manipulation height Object contacts surface Digits release

  • bject
  • In-Hand Manipulation
  • The internal consecutive regrasping and release of the object
  • Perform object reorientation, fine positioning or more complex interaction.
slide-7
SLIDE 7

University of Coimbra

  • Representation of the general contact signatures for manipulation tasks developed
  • n some daily activities

I - Introduction

  • N. Kamakura, Te no ugoki, Te no katachi (in Japanese), Ishiyaku Publishers, Tokyo, Japan, 1989.
slide-8
SLIDE 8

University of Coimbra

  • Challenge:
  • How to learn and encode the human like skills/capabilities?
  • Develop a compact and flexible representation for:

+ task recognition + task planning + task synthesis

  • Imitation Learning à

à Task representation

I - Introduction

slide-9
SLIDE 9

University of Coimbra

II – Approach Overview

  • Generalization at a trajectory level:

Adapted from Billard et al, 2008

slide-10
SLIDE 10

University of Coimbra Adapted from Billard et al, 2008

II – Approach Overview

  • Generalization at a symbolic level:
slide-11
SLIDE 11

University of Coimbra

Overview of the global structure of the proposed system

II – Approach Overview

Environment Multifingered Robotic Hand Effector Hand Tactile Sensing Array Sensing Devices Contact State Sequence Posterior Tactile Discretization Sensory Processing Contact State Primitives Templates Pior Knowledge Task Constraints Gain/Loss Function Sensation Perception Action Decision Rule Bayesion Approach

  • The human subjects performs an in-hand manipulation task using a instrumented

glove (hand joint flexure sensors and tactile sensors)

  • Elementary primitives sequence extraction among a pre-defined primitive set.
  • Verification if the raw sequence of primitives extracted from the demonstration

respects the constraints of that class.

slide-12
SLIDE 12

University of Coimbra

III –Contact Template Definition

Primitive model definition

  • The output of the 360 tactile sensing

elements are grouped on 15 regions.

  • Each region corresponds to different areas
  • f the hand.
  • A variable Ti is assigned to each of this

regions: T = { T1, T2,…, T15}

slide-13
SLIDE 13

University of Coimbra

III –Contact Template Definition

} , , {

15 ,..., 1

HighActive LowActive NotActive Ti

i

∈ ∀ =

T1 T2 T6 T10 T14 T3 T4 T5 T7 T8 T12 T9 T13 T11 T15

Primitive model definition

  • A variable Ti is assigned to each of this

regions: T = { T1, T2,…, T15}

  • Domain definition of each variable:
  • NotActive à 0 < Ti < 25
  • LowActive à 26 < Ti< 190
  • HighActive à 190 < Ti < 255
slide-14
SLIDE 14

University of Coimbra

III –Contact Template Definition

Primitive set definition

  • The set of primitives comprises a total of 7

templates.

  • A primitive is designed by the variable E.
  • The domain definition of E is:

E ε {primitive1, primitive2, …, primitive7}

Primitive1 Primitive2 Primitive3 Primitive4 Primitive5 Primitive6

  • Primitive7 àno contact between the hand and the object.
slide-15
SLIDE 15

University of Coimbra

III –Contact Template Definition

Primitive set training

  • Estimation of the parameters T of each of

the predefined primitives E

  • Human demonstration of each of the 7 pre-

defined primitives à static contact configuration of the human hand and the

  • bject.
  • From each primitive demonstration a

probabilistic distribution is built à P(T/E)

Primitive1 Primitive2 Primitive3 Primitive4 Primitive5 Primitive6

slide-16
SLIDE 16

University of Coimbra

III – Contact Template Definition

Experimental Results

  • Estimation of the parameters T of each of the pre-defined contact state templates
  • Five demonstrations of each grasp configurations

Trial 01 Trial 02 Trial 03 Trial 04 Trial 05 T1 H H H H L T2 H H H H L T3 N N N N N T4 N N N N N T5 N N N N N T6 N N N N N T7 N N N N N T8 N N N N N T9 N N N N N T10 N N N N N T11 N N N N N T12 N N N N N T13 N N N N N T14 N N N N N T15 N N N N N

Primitive 1 trainning result

tk P(T=tk / E=primitive1) (H,H,N,N,N,N,N,N,N,N,N, N,N,N,N) 4/5 (L,L,N,N,N,N,N,N,N,N,N,N ,N,N,N) 1/5 Other

Primitive 1 - Conditional Probability Density Function N-NotActive L-LowActive H-HighActive Primitive1

slide-17
SLIDE 17

University of Coimbra

III – Contact Template Definition

Experimental Results

tk P(T=tk / E=primitive2) (N,L,H,H,L,L,H,H,L,N,N,N, N,N,H) 2/5 (N,L,H,H,L,N,L,L,N,N,N,N, N,N,H) 1/5 (N,L,H,H,N,N,L,L,N,N,N,N, N,N,H) 2/5 Other

Primitive 2 - Conditional Probability Density Function N-NotActive L-LowActive H-HighActive

tk P(T=tk / E=primitive3) (H,H,H,H,L,H,H,L,L,H,L,L, N,L,N) 3/5 (H,H,H,H,N,H,H,L,N,H,L,L, N,L,N) 2/5 Other

Primitive 3 - Conditional Probability Density Function

tk P(T=tk / E=primitive4) (H,N,H,N,N,N,L,N,N,N,N,N ,N,N,N) 2/5 (H,N,H,N,N,N,N,N,N,N,N, N,N,N,N) 3/5 Other

Primitive 4 - Conditional Probability Density Function Primitive2 Primitive3 Primitive4

slide-18
SLIDE 18

University of Coimbra

III – Contact Template Definition

Experimental Results

Primitive 5 - Conditional Probability Density Function N-NotActive L-LowActive H-HighActive

tk P(T=tk / E=primitive6) (L,L,L,L,L,H,H,H,L,N,N,N, N,N,H) 2/5 (L,L,L,L,N,H,H,H,L,N,N,N, N,N,H) 3/5 Other

Primitive 6 - Conditional Probability Density Function

tk P(T=tk / E=primitive7) (N,N,N,N,N,N,N,N,N,N,N, N,N,N,N) 1 Other

Primitive 7 - Conditional Probability Density Function Primitive5 Primitive6

tk P(T=tk / E=primitive5) (H,H,H,H,H,N,N,N,N,N,N, N,N,N,N) 1/5 (H,H,H,L,L,N,N,N,N,N,N,N ,N,N,N) 2/5 (H,H,L,N,N,N,N,N,N,N,N,N ,N,N,N) 2/5 Other

slide-19
SLIDE 19

University of Coimbra

IV – Contact Template Detection

Primitives detection on raw data input

  • The raw data input produced by the human demonstration is integrated during equal

time intervals of length t.

  • The integrated data during each time slot t à Tt
  • The primitive with the maximum likelihood is the template assigned to that timeslot.

) ( ) ( ) / ( ) / ( T T T

t t t

P E P E P E P =

From the primitives demonstration trainning session P(E)=1/7

∑ =

= = = = = = = = =

7 1 15 1 15 1 15 2 1

) ( ) / ) ,..., / ( ) ( ) / ) ,..., ( ( )) ,..., , ( / (

j j j t i i t i t

primitive E P primitive E t t P primitive E P primitive E t t P t t t primitive E P T T T

slide-20
SLIDE 20

University of Coimbra

IV – Contact Template Detection

Experimental Setup

  • Human demonstrator
  • Right handed instrumented data glove
  • Tactile sensing
  • Tactile sensing array data acquisition

rate: 500Hz

  • Cup
slide-21
SLIDE 21

University of Coimbra

IV - Contact Template Detection

Experimental Results

  • Task I – Reorientation of the mug in order to place the grasp of the mug in a

configuration suitable to be grasped by the handle by the subject.

slide-22
SLIDE 22

University of Coimbra

IV – Contact Template Detection

Experimental Results

Timeslot (ms) Trial 01 Estimation 0-500 (N,N,N,N,N,N,N,N,N,N,N,N,N,N,N) primitive7 500-1000 (N,N,N,N,N,N,N,N,N,N,N,N,N,N,N) primitive7 1000-1500 (H,H,N,N,N,N,N,N,N,N,N,N,N,N,N) primitive1 1500-2000 (N,N,N,N,N,N,N,N,N,N,N,N,N,N,N) primitive7 2000-2500 H,H,H,L,L,N,N,N,N,N,N,N,N,N,N) primitive5 2500-3000 (N,N,N,N,N,N,N,N,N,N,N,N,N,N,N) primitive7 3000-3500 H,H,L,N,N,N,N,N,N,N,N,N,N,N,N) primitive5 3500-4000 H,H,H,L,L,N,N,N,N,N,N,N,N,N,N) primitive5 4000-4500 (N,N,N,N,N,N,N,N,N,N,N,N,N,N,N) primitive7 4500-5000 H,H,L,N,N,N,N,N,N,N,N,N,N,N,N) primitive5 5000-5500 (N,N,N,N,N,N,N,N,N,N,N,N,N,N,N) primitive7 Timeslot (ms) Trial 02 Estimation 0-500 (N,N,N,N,N,N,N,N,N,N,N,N,N,N,N) primitive7 500-1000 H,H,H,H,H,N,N,N,N,N,N,N,N,N,N) primitive5 1000-1500 (N,N,N,N,N,N,N,N,N,N,N,N,N,N,N) primitive7 1500-2000 (H,H,H,H,H,N,N,N,N,N,N,N,N,N,N) primitive5 2000-2500 (N,N,N,N,N,N,N,N,N,N,N,N,N,N,N) primitive7 2500-3000 (H,H,L,N,N,N,N,N,N,N,N,N,N,N,N) primitive5 3000-3500 (N,H,N,N,N,N,N,N,N,N,N,N,N,N,N) Not recognized 3500-4000 (N,N,N,N,N,N,N,N,N,N,N,N,N,N,N) primitive7

Task I – Trial 02 Task I – Trial 01

Primitive1 Primitive5 Primitive5

slide-23
SLIDE 23

University of Coimbra

IV – Contact Template Detection

Experimental Results

  • Task II – Grasping the mug without reorientation and elevate it.
slide-24
SLIDE 24

University of Coimbra

IV – Contact Template Detection

Experimental Results

Timeslot (ms) Trial 01 Estimation 0-500 (N,N,N,N,N,N,N,N,N,N,N,N,N,N,N) primitive7 500-1000 (H,H,L,N,N,N,N,N,N,N,N,N,N,N,N) primitive5 1000-1500 (H,H,H,H,N,H,H,L,N,H,L,L,N,L,N) primitive3 1500-2000 (H,H,H,H,L,H,H,L,L,H,L,L,N,L,N) primitive3 2000-2500 (H,H,H,H,L,H,H,L,L,H,L,L,N,L,N) primitive3 Timeslot (ms) Trial 02 Estimation 0-500 (N,N,N,N,N,N,N,N,N,N,N,N,N,N,N) primitive7 500-1000 (H,H,H,H,L,H,H,L,L,H,L,L,N,L,N) primitive3 1000-1500 (H,H,H,H,L,H,H,L,L,H,L,L,N,L,N) primitive3 1500-2000 (H,H,H,H,L,H,H,L,L,H,L,L,N,L,N) primitive3

Task II – Trial 02 Task II – Trial 01

Primitive3 Primitive5 Primitive3

slide-25
SLIDE 25

University of Coimbra

V – Conclusions

  • Typically the first segments correspond to template of primitive7- no contact hand
  • bject.
  • Task I
  • Sequence of grasp and release of the object.
  • Fingers involved on the reorientation are predominantly the thumb, index and

middle fingers.

  • The magnitude of the tactile inputs and number of fingers mobilized is higher

during the initial grasp-release cycles.

  • Task II
  • The movement is decomposed in primitives that involve the participation of

high extensions of fingers surface.

  • Simple prehension movement – transport – contact signatures still the same

along the time

slide-26
SLIDE 26

University of Coimbra

VI – Future Work

  • Perform tests on additional types of tasks
  • Automatic definition of the primitive set
  • Implementation of the task primitive sequence representation constraints

extraction.

slide-27
SLIDE 27

University of Coimbra

ISR-UC End

Institute of Systems and Robitcs

http://paloma.isr.uc.pt

  • Institute of Systems and Robotics