SLIDE 1
Agenda
- Overview
○ Innovation Team Presentation ○ Mummer Research Project
- Problematics
- Proposed solution
- Live Demo
- Question
Humanoid Companion Embedded GPUs can make your robotic companion - - PowerPoint PPT Presentation
Humanoid Companion Embedded GPUs can make your robotic companion - - PowerPoint PPT Presentation
Smarter Humanoid Companion Embedded GPUs can make your robotic companion more alive Hi Alex! Alexandre Mazel Innovation Software Director Oct 2017 Agenda Overview Innovation Team Presentation Mummer Research Project
SLIDE 2
SLIDE 3
Team Presentation
Part of the Innovation Department, which includes hardware, electronics, collaborative projects and design.
AI Lab Fundamental Research on Developmental Robotics 3 Permanent 3 PhD student 3 Intern Protolab Applied Research 4 Permanent 1 PhD student Innovation Software
SLIDE 4
Goal: Prospection
Enhance our Humanoid Robots for more natural Human-Robot Interaction
- Explore future uses
- Test and embed new algorithms
- Hardware improvement
- Provide versatile platforms for research
SLIDE 5
Goals:
- Make Pepper navigates in malls
- Entertain visitors/customers
Experimentation field:
- Ideapark mall in Lempäälä (Finland)
- Huge: More than 150 stores, restaurants and cafes
within 100.000 m2
- Crowdy: 7 million visitors (2013)
SLIDE 6
SLIDE 7
SLIDE 8
SoftBank Robotics Europe VTT TechnicalResearch Center of Finland Ideapark University of Glasgow Heriot-Watt University Idiap Research Institute LAAS-CNRS
SLIDE 9
Challenges
- Obstacle avoidance
- Quick person detection (<1s)
- Self Localization
- Data confidentiality
SLIDE 10
Navigation Sensors
Laser (45 points, up to 3m) RGB Camera (55°H, 44°V) Sonar Depth Camera (58°H, 45°V) RGB Camera (55°H, 44°V)
SLIDE 11
Current limitations
SLIDE 12
Current limitations
SLIDE 13
Proposed solution: case study
RGB Fish Eye (100°H, 180°V)
SLIDE 14
Robot POV
SLIDE 15
ConvNet Learning for obstacle avoidance
- pretrained AlexNet using the LSVRC-2010 ImageNet (1.3M Images)
- learning FC7, FC8 and binary classification
Passable Non passable
SLIDE 16
Collection of training data
SLIDE 17
Region of Interest
SLIDE 18
Results of the learning process
- 3400 images
- Learning rate: 0.001
- dropout rate: 0.5
- batches of size: 40
- duration of one epoch: 70 sec (using a Geforce GTX 1070)
SLIDE 19
Proposed solution: case study
RGB Fish Eye (100°H, 180°V)
JetsonTM TX2
DC-DC 29V-12V
SLIDE 20
Embedding JetsonTM TX2
USB Ethernet (or wifi)
SLIDE 21
Video Technical Demo
SLIDE 22
NB: Using the trained tensorflow model as is Action Time (s) Acquire image 0.016 Computing difference 0.005 Undistort and rotation (numpy) 0.049 Inference 0.033 Total Time 0.103 (9.7 fps)
Embedding JetsonTM TX2
SLIDE 23
Battery Draining Measure
NB: based on one test only Standard Pepper - no movement 11h32 Pepper with gpu processing and infering every frame - no movement 10h21
SLIDE 24
Advantages
- Dodges obstacles
- Fully autonomous (no cloud, no wifi)
- Quick training - can be done multiple times
- Can be learned directly on site
- Confidentiality is preserved
SLIDE 25
To be continued
Next steps:
- add more classes (left/right/center)
- optimisation (int8, tensorRT, …)
- autonomous & continuous learning on the fly
Future work:
- Navigation: Localisation/VSlam
- Skeleton estimation (2D) (Zhe Cao, Tomas Simon, Shih-En Wei, Yaser Sheikh)
- Face features extraction
- Speech Recognition (Caldi)
SLIDE 26
Live Demo
SLIDE 27
Conclusion
SLIDE 28
Acknowledgement
Based on work from:
- Abdelhak Loukkal (2017)
- Michael Guerzhoy and Davi Frossard (2016)
Reference:
- Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton, ImageNet
Classification with Deep Convolutional Neural Networks (2015)
SLIDE 29