Human-centered manipulation and navigation with Robot DE NIRO - - PowerPoint PPT Presentation

01/10/2018 Fabian Falck – Robot DE NIRO – IROS workshop on Manipulation Intelligence 1

Human-centered manipulation and navigation with Robot DE NIRO

1 Department of Computing, Imperial College London 2 Dyson School of Design Engineering, Imperial College London

Towards Robots that Exhibit Manipulation Intelligence IROS workshop

October 1, 2018

Fabian Falck 1 Sagar Doshi 1 Nico Smuts 1 John Lingi 1 Kim Rants 1 Petar Kormushev 1

01/10/2018 Fabian Falck – Robot DE NIRO – IROS workshop on Manipulation Intelligence 2

Robot DE NIRO – Motivation and Hardware Design

QUICKIE base with feedback controller Baxter robot arms Microsoft Kinect depth camera Stereovision cameras Mount position of 2D Hokuyo LIDAR

• Macrosocial trends of aging and

long-lived populations

• Related work: focused on helping

the elderly live independently [1] [2] [3] [4]

• Ethical concerns [5] [6]:

– human isolation – loss of control and personal liberty – deception and infantilization

• DE NIRO: “Support the

supporter” (the caregiver) and

• ffer direct human-robot

interaction with the care recipient

Sources: [1] Care-O-Bot 3 [2] ASIMO [3] HRP-3 [4] DOMEO RobuMate [5] Sparrow et al. [6] Wallach et al.

01/10/2018 Fabian Falck – Robot DE NIRO – IROS workshop on Manipulation Intelligence 3

Software implementation and Play Fetch routine

• ROS as middleware
• Finite-state machine to manage the control flow

01/10/2018 Fabian Falck – Robot DE NIRO – IROS workshop on Manipulation Intelligence 4

Perception and user interaction

• Face recognition: ResNet model pre-trained on faces applied to

video frames retrieved by the Kinect camera, reaching an accuracy

• f 99.38% on a standard benchmark [1]
• Speech recognition: Offline library CMU Sphinx [2]. Jspeech

Grammar to allow reliable voice commands in a specific format.

• Speech output: eSpeak [3] yielding a high reliability, rapid

response time and an offline implementation.

fetch me give me a an the coffee water apple juice

<command>

<article> <object>

Sources: [1] “dlib face recognition documentation,” http://dlib.net/dnn [2] https://cmusphinx.github.io/wiki/ [3] http://espeak.sourceforge.net/index.html

01/10/2018 Fabian Falck – Robot DE NIRO – IROS workshop on Manipulation Intelligence 5

Object Recognition and Manipulation

• Target objects are localized using 2D fiducial markers [1]
• Inverse kinematics solver to compute each of the seven joint angle

trajectories needed to reach an object [2]

• Safety: dynamic awareness procedure reacts to changes of the location of

the target object during grasping and actively avoids collisions

Sources: [1] https://github.com/chili-epfl/ros_markers [2] http://sdk.rethinkrobotics.com/wiki/IK_Service_-_Code_Walkthrough

01/10/2018 Fabian Falck – Robot DE NIRO – IROS workshop on Manipulation Intelligence 6

Navigation stack

• Static mapping: SLAM-based approach using the LIDAR sensor to

detect spatial boundaries and 2D artifacts [1]

• Localization: Dynamic map overlaid onto static map (for collision

avoidance)

• Trajectory planning: “Timed elastic band” approach [2] [3].

Sources: [1] http://wiki.ros.org/hector_mapping [2][3] http://wiki.ros.org/teb_local_planner

01/10/2018 Fabian Falck – Robot DE NIRO – IROS workshop on Manipulation Intelligence 7

Conclusion

• nonholonomic design
• maximum payload of 2.2 kg per arm
• currently limited to forward motion
• nly due to limited sensor capabilities

(possible deadlocks in corners)

• increased awareness and safety

through 360-degree camera rig

• 3D LIDAR
• teleoperation through virtual reality

headset and body tracking markers

Limitations Future work

01/10/2018 Fabian Falck – Robot DE NIRO – IROS workshop on Manipulation Intelligence 8

Thank you! https://www.imperial.ac.uk/robot-intelligence/software

Code + Documentation + Videos