Robot learning from few demonstrations by exploiting the structure - - PowerPoint PPT Presentation

Robot learning from few demonstrations by exploiting the structure and geometry of data Sylvain Calinon

Senior Researcher Idiap Research Institute, Martigny, Switzerland

Lecturer EPFL, Lausanne, Switzerland External Collaborator IIT, Genoa, Italy

Research Groups:

• Speech & Audio Processing
• Perception & Activity Understanding
• Computer Vision & Learning
• Social Computing
• Biometric Person Recognition
• Applied Machine Learning
• Natural Language Processing
• Robot Learning & Interaction
• Computational Bioimaging
• Uncertainty Quantification and Optimal Design

Artificial Intelligence for Society

MARTIGNY

Research Education Technology transfer

Learning from demonstration as an intuitive interface to transfer skills to robots

Learning from demonstration - Challenges

Finding Priors that are expressive enough to be used in a wide range of tasks

Prior 1: Movements are smooth and continuous Prior 2: Actions often relate to

• bjects, tools or body landmarks

Prior 3: Data spaces in robotics have geometries and structures

Movement generation as a mix of clustering, subspace analysis and optimal control

Walking Walking Running

We look for a compact and modular representation

• f continuous movements and skills that can learn

from few interactions (with user and environment), that can exploit variation and coordination, and that can adapt to new situations in a fast manner.

[Tanwani and Calinon, IEEE RA-L 1(1), 2016]

Learning of motions from few demonstrations

Global sharing of local coordination patterns with: Dictionary of coordination patterns: center covariance matrix

System plant

Use low control commands! Track path!

state variable (position+velocity) control command (acceleration) tracking weight matrix control weight matrix

Approach: Using control formalism in task space to solve analytically a basic form of model predictive control (MPC) with a double integrator as constant linear system

Learning minimal intervention controllers

[Tanwani and Calinon, IEEE RA-L 1(1), 2016]

System plant

Use low control commands! Track path!

Learning minimal intervention controllers

[Tanwani and Calinon, IEEE RA-L 1(1), 2016]

 Analytical solution to generate

motion control by following a minimal intervention principle

Transition and state duration (HSMM)

Stepwise reference with:

Learning minimal intervention controllers

[Calinon, Bruno and Caldwell, ICRA’2014]

Application: Editing motions with variations

[Berio, Calinon and Leymarie, IROS’2016] [Berio, Calinon and Leymarie , MOCO’2017]

User interface to edit and generate natural and dynamic motions by considering variation and coordination Compliant controller to retrieve safe and human-like motions

Daniel Berio Frederic Fol Leymarie

Personalized assistance using haptic and visual information, with compliant controllers following a minimal intervention principle

[Pignat and Calinon, RAS 93, 2017]

Dressing skills require some aspects to be time-independent, while other aspects are time- dependent for the generation of movements.

I-DRESS project

Emmanuel Pignat

Prior 1: Movements are smooth and continuous Prior 2: Actions often relate to

• bjects, tools or body landmarks

Prior 3: Data spaces in robotics have geometries and structures

Prior 2: Actions often relate to objects, tools

• r body landmarks

Photo: Basilio Noris

Task-parameterized motions

 Generic approach, but

limited generalization capability

Regression with a context variable c:

• Learning of
• Retrieval with

Track path in coordinate system j

Control in multiple coordinate systems

Use low control commands!

1 2 2 2 2 1 2

Set of demonstrations Reproduction in new situation

New position and

• rientation of coordinate

systems 1 and 2 Two candidate coordinate systems (P=2)

[Calinon, HFR’2016]

1 2

[Calinon, HFR’2016]

In many robotics problems, the parameters describing the task or situation can be interpreted as coordinate systems

Control in multiple coordinate systems

Track path in coordinate system j

Control in multiple coordinate systems

Use low control commands!

 Learning of a controller

(instead of learning a trajectory) that adapts to new situations while regulating the gains according to the precision and coordination patterns required by the task

[Calinon, HFR’2016]

Control in multiple coordinate systems

Track path in coordinate system j Use low control commands!

Control in multiple coordinate systems

 Retrieval of control commands

in the form of trajectory distributions, facilitating exploration and adaptation (in either control or state space)

[Calinon, HFR’2016]

Track path in coordinate system j Use low control commands!

SNSF, CHIST-ERA (2015-2018)

[Canal, G., Pignat, E., Alenya, G, Calinon, S. and Torras, C., ICRA’2018]

I-DRESS project

SNSF, CHIST-ERA (2015-2018)

[Canal, Pignat, Alenya, Calinon and Torras, ICRA’2018]

I-DRESS project

http://dexrov.eu

EC, H2020 (2015-2018)

Exploitation in shared control

Robot side Teleoperator side

• nly Gaussian ID

is transmitted

Dr Andras Kupcsik

[Havoutis and Calinon, Autonomous Robots, 2018]

Dr Ioannis Havoutis

Adaptation to different object shapes

[Calinon, Alizadeh and Caldwell, IROS’2013]

Coordinate system as task parameter

[Silvério et al., IROS’2015] [Rozo et al., IROS’2015] [Rozo et al., IEEE T-RO 32(3), 2016]

Bimanual coordination and co-manipulation

Dr Leonel Rozo Dr João Silvério

Candidate hierarchy Candidate hierarchy Demonstration Reproduction

Demonstration Reproduction

[Silvério, Calinon, Rozo and Caldwell (2018), Arxiv 1707.06791] [Calinon, ISRR’15]

Priority on left hand Learning & generalizing tasks prioritization

Candidate hierarchy Candidate hierarchy Demonstration Reproduction

Demonstration Reproduction

Priority on right hand Learning & generalizing tasks prioritization

[Silvério, Calinon, Rozo and Caldwell (2018), Arxiv 1707.06791] [Calinon, ISRR’15]

Candidate hierarchy Candidate hierarchy Demonstration Reproduction

Demonstration Reproduction

Equal priority Learning & generalizing tasks prioritization

[Silvério, Calinon, Rozo and Caldwell (2018), Arxiv 1707.06791] [Calinon, ISRR’15]

Prior 1: Movements are smooth and continuous Prior 2: Actions often relate to

• bjects, tools or body landmarks

Prior 3: Data spaces in robotics have geometries and structures

Prior 3: Data spaces in robotics have geometries and structures

Motivation of using Riemannian manifolds

Interpolation on Riemannian manifolds

Orientation (unit quaternions) Rigid body motions (position+orientation) Covariance features, inertia and gain matrices, manipulability ellipsoids, trajectory distributions (symmetric positive definite matrices)

Clustering on Riemannian manifolds

Orientation (unit quaternions) Rigid body motions (position+orientation) Covariance features, inertia and gain matrices, manipulability ellipsoids, trajectory distributions (symmetric positive definite matrices)

Regression on Riemannian manifolds

→ Regression for orientation data (unit quaternions on )

Gaussian mixture regression (GMR) to compute from the joint distribution encoded as a GMM

[Zeestraten, Havoutis, Silvério, Calinon and Caldwell, IEEE RA-L 2(3), 2017]

Four demonstrations of coordinated bimanual movement Regression with orientation and position data

Four reproductions with perturbations by the user Regression with orientation and position data

[Zeestraten, Havoutis, Silvério, Calinon and Caldwell, IEEE RA-L 2(3), 2017]

TACT-HAND

Surface electromyography (sEMG) measurements Transformation in spatial covariances (SPD matrices)

[Jaquier and Calinon, IROS 2017]

Regression with sEMG sensory data

Control of the corresponding hand pose

Noémie Jaquier

SNSF, D-A-CH (2016-2019)

Comparison: standard GMR vs geometric GMR

sEMG data from Ninapro database processed as spatial covariances:

12 4

Input Output

[Jaquier and Calinon, IROS 2017]

Manipulability ellipsoid tracking

[N. Jaquier, L. Rozo, D.G. Caldwell and S. Calinon, RSS’2018]

Noémie Jaquier

Conclusion

Combining statistical learning techniques and model predictive control provides a generative approach to the transfer of skills and movements Statistical learning in multiple coordinate systems can be exploited to learn robot skills and movements from few demonstrations, with adaptation to new situations Robotics is rich in structures and geometries that can be exploited to acquire skills and movements from a small set of interactions (with user or environment)

Some advertisements!

Open Postdoc and PhD positions at the Robot Learning & Interaction Group at Idiap (no information on the web yet: contact me by email sylvain.calinon@idiap.ch) Mo-TUT-2: From Least Squares Regression to High-dimensional Motion Primitives (AM) Organizers: Freek Stulp (DLR, Germany), Sylvain Calinon (Idiap Research Institute, Switzerland), Gerhard Neumann (University of Lincoln, USA) Fr-WS5: Robots for Assisted Living Organizers: Sylvain Calinon (Idiap Research Institute, Switzerland), Sanja Dogramadzi (University of the West of England, UK), Carme Torras (CSIC- UPC, Spain), Tomohiro Shibata (Kyushu Institute of Technology, Japan), Yiannis Demiris (Imperial College London, UK) Abstract submission deadline: July 6th, 2018 IROS’2018 events (Madrid, Tutorial on Oct 1st and Workshop on Oct 5th) https://www.iros2018.org/workshops

Robot Learning & Interaction Group at Idiap: Contact:

sylvain.calinon@idiap.ch http://calinon.ch

Source codes (Matlab/Octave, C++ and Python):

http://www.idiap.ch/software/pbdlib/

Photo: Basilio Noris

Thibaut Kulak Emmanuel Pignat Noémie Jaquier Dr Andras Kupcsik Dr Antonio Paolillo Hakan Girgin Nicolas Desprès