Learning to Place New Objects Yun Jiang , Changxi Zheng, Marcus Lim - PowerPoint PPT Presentation

Learning to Place New Objects Yun Jiang , Changxi Zheng, Marcus Lim and Ashutosh Saxena Cornell University

Placing Novel Objects  Robotic grasping has been studied for decades. How to place objects after grasping?  Example scenarios:  Arranging grocery in a fridge.  Loading a dish-rack.  Organizing a house. 2 Jiang, Zheng, Lim and Saxena

Placing Problem Formulation  Input: point cloud of objects and placing areas (from Kinect).  Output: a stable and preferred placement specified by 3D location/orientation of the object 3 Jiang, Zheng, Lim and Saxena

Challenges  Where to place?  complex (non-flat) areas  Semantic preferences: shoes not on tables  Which orientation to place in?  Depends on different placing areas. 4 Jiang, Zheng, Lim and Saxena

Previous Work  Where to place?  Flat and clutter-free surface [M.J.Schuster et al., 2010] M.J.Schuster et al., 2010  How to execute placing?  Given desired location to place,  Path planning [Lozano-Perez et al., 2002].  Tactile feedback [Edsinger and Kemp, 2002]. A.Edsinger and C.C.Kemp, 2002 5 Jiang, Zheng, Lim and Saxena

Learning Approach  Placing single object 6 Jiang, Zheng, Lim and Saxena

Stability Features  Supporting contacts (12)  Falling distance, Eigen values, center of mass, etc. contacts 7 Jiang, Zheng, Lim and Saxena

Stability Features  Supporting contacts (12)  Caging features (37)  Height  Horizontal distance from the placing area to the object (a) side view (b) top view 8 Jiang, Zheng, Lim and Saxena

Semantic Features  Supporting contacts (12)  Caging features (37)  Histogram features (96)  # points from object, placing area and their ratio (a) side view (b) top view 9 Jiang, Zheng, Lim and Saxena

Max-margin learning  Features for each placement  Labels  Soft-margin support vector machine (SVM)  Shared-sparsity in the parameters. 10 Jiang, Zheng, Lim and Saxena

Learning Approach 11 Jiang, Zheng, Lim and Saxena

Learning Experiments  Finding best location and orientation  7 placing areas and 19 objects  A total of 620 placements 12 Jiang, Zheng, Lim and Saxena

Robotic Experiments 13 Jiang, Zheng, Lim and Saxena

Robotic Experiments Objects/environments are never seen before by the robot! T otal 400 robotic trials. 14 Jiang, Zheng, Lim and Saxena

Placing multiple objects  Task: placing multiple objects into multiple placing areas  Challenges  (1) stability (2) semantic preference  (3) linear stacking (4) non-overlap  Max-margin learning  Inference as a linear programming problem  [ Jiang, Zheng, Lim and Saxena, IJRR 2012 ] 15 Jiang, Zheng, Lim and Saxena

Experiments: Full Scene Semantics  Finding semantically preferred placing area  98 objects from 16 categories  11 different placing areas 16 Jiang, Zheng, Lim and Saxena

Video 17 Jiang, Zheng, Lim and Saxena

Learning Object Arrangements using Human Context  Learn human-object relationships  Arrange objects meaningfully Jiang, Lim and Saxena, ICML’12 18 Jiang, Zheng, Lim and Saxena

Thank you!  Learning to Place New Objects. Jiang, Zheng, Lim and Saxena, ICRA’12 .  Learning to Place New Objects in a Scene. Jiang, Zheng, Lim and Saxena, IJRR ’12  Learning Object Arrangements in 3D Scenes using Human Context. Jiang, Lim and Saxena, ICML’12  Dataset for object placing & arrangements  98 objects from 16 categories, 40 placing areas  180 human labeled arrangements on 20 rooms and 47 objects http://pr.cs.cornell.edu/placingobjects/ 19 Jiang, Zheng, Lim and Saxena