Self-Supervised Deep Learning for Robotic Grasping Lars Berscheid | - - PowerPoint PPT Presentation

▶
self supervised deep learning for robotic grasping
SMART_READER_LITE
LIVE PREVIEW

Self-Supervised Deep Learning for Robotic Grasping Lars Berscheid | - - PowerPoint PPT Presentation

Nov 22, 2022 400 likes •564 views

Self-Supervised Deep Learning for Robotic Grasping Lars Berscheid | KUKA Roboter GmbH | 10/10/2017 Self-Supervised Deep Learning for Robotic Grasping | Lars Berscheid | 10/10/2017 | www.kuka.com 1 From Robotic Grasping to Bin Picking - Bin

slide-1
SLIDE 1

Self-Supervised Deep Learning for Robotic Grasping | Lars Berscheid | 10/10/2017 | www.kuka.com 1

Self-Supervised Deep Learning for Robotic Grasping

Lars Berscheid | KUKA Roboter GmbH | 10/10/2017

slide-2
SLIDE 2

Self-Supervised Deep Learning for Robotic Grasping | Lars Berscheid | 10/10/2017 | www.kuka.com 2

From Robotic Grasping to Bin Picking

  • Bin Picking: Grasps from an unsorted bin (often just one object type)
  • Essential for logistic applications, household robotics and so much more…
  • Traditional approaches are:
  • Not robust
  • Or not flexible
  • And often slow
slide-3
SLIDE 3

Self-Supervised Deep Learning for Robotic Grasping | Lars Berscheid | 10/10/2017 | www.kuka.com 3

  • Fig. 2: Planar Grasps with 4 parameters

Planar coordinates (x, y, a) and gripper position d

  • Fig. 1: KUKA LBR iiwa robot arm
  • Depth camera at flange
  • Force feedback gripper

Robotic System

slide-4
SLIDE 4

Self-Supervised Deep Learning for Robotic Grasping | Lars Berscheid | 10/10/2017 | www.kuka.com 4

  • Fig. 3: Depth Image Example

Red: Relevant window (outer),

  • approx. position of the grippers

(inner)

Sliding Window Approach

slide-5
SLIDE 5

Self-Supervised Deep Learning for Robotic Grasping | Lars Berscheid | 10/10/2017 | www.kuka.com 5

Neural Net

  • Fig. 4: Net Architecture

Variable input 10 rotations (transformed input) 3 gripper position Performance (20ms for 48000 poses) on a NVIDIA GTX 1070

Training Inference Input 32 × 32 110 × 110 Output 1 × 1 × 3 40 × 40 × 3

Window Depth Image Neural Net Grasp Probability

slide-6
SLIDE 6

Self-Supervised Deep Learning for Robotic Grasping | Lars Berscheid | 10/10/2017 | www.kuka.com 6

  • Fig. 5: Heatmap Neural Net

Red: High Grasping Probability Blue: Low Grasping Probability Averaged over rotation and gripper position. During inference, one of the 5 best poses is chosen randomly.

Neural Net

slide-7
SLIDE 7

Self-Supervised Deep Learning for Robotic Grasping | Lars Berscheid | 10/10/2017 | www.kuka.com 7

Training

  • Up to 12500 grasp recordings (≈35h, 10s)
  • Active learning
  • Adapt gripper force
  • Data augmentation
  • Asymmetric error measure
  • Automated recording and training
  • Fig. 6: Test Error depending
  • n Training Set Size
slide-8
SLIDE 8

Self-Supervised Deep Learning for Robotic Grasping | Lars Berscheid | 10/10/2017 | www.kuka.com 8

slide-9
SLIDE 9

Self-Supervised Deep Learning for Robotic Grasping | Lars Berscheid | 10/10/2017 | www.kuka.com 9

Results

m out of n Grasping rate [%] Time for n grasps [s] Number Grasp tries 1 out of 1 99,5 ± 0,5 10,7 ± 0,1 210 5 out of 5 98,2 ± 1,2 56,7 ± 0,7 112 5 out of 10 95,2 ± 2,2 59,0 ± 1,1 105 10 out of 20 92,2 ± 2,2 117,2 ± 2,2 141

  • Tab. 1: Grasping rate at different scenarios: m objects in the bin, n objects are taken out without

replacement, grasping rate is percentage of successful grasps. Random grasping rate ≈3%

slide-10
SLIDE 10

Self-Supervised Deep Learning for Robotic Grasping | Lars Berscheid | 10/10/2017 | www.kuka.com 10

slide-11
SLIDE 11

Self-Supervised Deep Learning for Robotic Grasping | Lars Berscheid | 10/10/2017 | www.kuka.com 11

Generalisation

  • Fig. 7: Grasps of novel objects
slide-12
SLIDE 12

Self-Supervised Deep Learning for Robotic Grasping | Lars Berscheid | 10/10/2017 | www.kuka.com 12

Specific Grasps

  • Fig. 9: Markings of grasped objects (blue)
slide-13
SLIDE 13

Self-Supervised Deep Learning for Robotic Grasping | Lars Berscheid | 10/10/2017 | www.kuka.com 13

Specific Grasps

Window Depth-Image Point in image Neural Net Grasp probability of specific object

  • Fig. 10: Net architecture for specific grasps

Dataset Generation Create random points in image space → only if objects is grasped and point is within the marking, it is a successful grasp.

slide-14
SLIDE 14

Self-Supervised Deep Learning for Robotic Grasping | Lars Berscheid | 10/10/2017 | www.kuka.com 14

slide-15
SLIDE 15

Self-Supervised Deep Learning for Robotic Grasping | Lars Berscheid | 10/10/2017 | www.kuka.com 15

Self-Supervised Deep Learning for Robotic Grasping

  • Learns grasping rates of 90-95% for bin picking in under 35h
  • High performance with under 20ms per grasp inference
  • Automated data recording and training
  • Generalizes to many object types
  • Specific grasps of objects