BADGr: A Toolbox for Box-based Approximation, Decomposition and - - PowerPoint PPT Presentation

▶
badgr a toolbox for box based approximation decomposition
SMART_READER_LITE
LIVE PREVIEW

BADGr: A Toolbox for Box-based Approximation, Decomposition and - - PowerPoint PPT Presentation

Apr 24, 2023 329 likes •591 views

BADGr: A Toolbox for Box-based Approximation, Decomposition and Grasping Kai Huebner khubner@kth.se KTH Royal Institute of Technology, Stockholm, Sweden International Conference on Intelligent RObots and Systems 2010 Slide 1 Workshop on

slide-1
SLIDE 1

BADGr: A Toolbox for Box-based Approximation, Decomposition and Grasping

Kai Huebner

khubner@kth.se

KTH – Royal Institute of Technology, Stockholm, Sweden

International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 1

slide-2
SLIDE 2

Overview

Overview

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Motivation Boxgrasping Framework Overview BADGr System Architecture Conclusion

International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 2

slide-3
SLIDE 3

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Motivation

EU Project PACO-PLUS

EU project PACO-PLUS: Perception, Action and Cognition through Learning

  • f Object-Action-Complexes (February 2006 - July 2010).

Focus on formalizing the interplay between objects and actions. Describe an object not by its appearance only, but also by which actions it allows.

International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 3

slide-4
SLIDE 4

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Motivation

Interactions with Objects

Problem: before actions can be employed or analyzed on a higher-level, the

  • bject must already be interacted with on a lower level, through grasping.

Grasping is an important module in a number of robot applications. (4 Grasping sessions = 22 talks at IROS 2010 conference)

International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 4

slide-5
SLIDE 5

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Motivation

Grasping

A good grasp is classically defined as stable object-gripper situation such that the object can be successfully lifted for further manipulation. Thus, grasp planners have to take into account a number of properties of the object (e.g. shape) and the action (e.g. hand kinematics) and more [1].

[1] Song et al., IROS 2010, Grasping II, TuET2.4.

International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 5

slide-6
SLIDE 6

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Motivation

Object Shape Representations

From a sensor point of view, an object can be described to a reference, or to its pure appearance, i.e. 2D or 3D features, whereof one is 3D shape. Even with the simple representation of box constellations we can access rough shape, size, pose, task, not only of objects, but also object parts [2].

[2] Huebner & Kragic, IROS 2008.

International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 6

slide-7
SLIDE 7

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Boxgrasping Framework Overview

Framework Overview

3D Point Cloud Bounding Box 3D Shape Approximation Box Approximation Box Decomposition Box Faces ⇒ Grasp Hypotheses Heuristical Validity Checks (Task, Occlusion, Reachability, ...) Pre-Grasp Geometrical Heuristics Huebner et al. [3] Huebner & Kragic [2]

From this motivation, we developed a flexible framework enabling box approximation, decomposition, and grasp hypotheses generation,

◮ taking a point cloud and generating a restricted set of pre-grasps, ◮ where input can be from any source providing basic 3D data, and ◮ output are also shape representations besides grasp poses.

International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 7

slide-8
SLIDE 8

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Boxgrasping Framework Overview

Framework Overview

Machine Vision Real Scene 2D/3D Scene Processor Object Recognition 3D Segment Graphical Modeling Sim. Scene 3D Model 3D Point Cloud Bounding Box 3D Shape Approximation Box Approximation Box Decomposition Box Faces ⇒ Grasp Hypotheses Heuristical Validity Checks (Task, Occlusion, Reachability, ...) Pre-Grasp Geometrical Heuristics Rasolzadeh et al. [4] Huebner et al. [5] Huebner et al. [3] Huebner & Kragic [2]

From this motivation, we developed a flexible framework enabling box approximation, decomposition, and grasp hypotheses generation,

◮ taking a point cloud and generating a restricted set of pre-grasps, ◮ where input can be from any source providing basic 3D data, and ◮ output are also shape representations besides grasp poses.

International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 7

slide-9
SLIDE 9

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Boxgrasping Framework Overview

Framework Overview

Machine Vision Real Scene 2D/3D Scene Processor Object Recognition 3D Segment Graphical Modeling Sim. Scene 3D Model 3D Point Cloud Bounding Box 3D Shape Approximation Box Approximation Box Decomposition Box Faces ⇒ Grasp Hypotheses Heuristical Validity Checks (Task, Occlusion, Reachability, ...) Pre-Grasp Geometrical Heuristics Stability Learning Neural Network Learning (Stability) Pre-Grasp Method Evaluation Evaluation (Justification) Task Constraint Learning Bayesian Network Learning (Task) Pre-Grasp Rasolzadeh et al. [4] Huebner et al. [5] Huebner et al. [3] Huebner & Kragic [2] Huebner et al. [3] Geidenstam et al. [6] Song et al. [1]

From this motivation, we developed a flexible framework enabling box approximation, decomposition, and grasp hypotheses generation,

◮ taking a point cloud and generating a restricted set of pre-grasps, ◮ where input can be from any source providing basic 3D data, and ◮ output are also shape representations besides grasp poses.

International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 7

slide-10
SLIDE 10

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Boxgrasping Framework Overview

Framework Overview

Machine Vision Real Scene 2D/3D Scene Processor Object Recognition 3D Segment Graphical Modeling Sim. Scene 3D Model 3D Point Cloud Bounding Box 3D Shape Approximation Box Approximation Box Decomposition Box Faces ⇒ Grasp Hypotheses Heuristical Validity Checks (Task, Occlusion, Reachability, ...) Pre-Grasp Geometrical Heuristics Stability Learning Neural Network Learning (Stability) Pre-Grasp Method Evaluation Evaluation (Justification) Task Constraint Learning Bayesian Network Learning (Task) Pre-Grasp Rasolzadeh et al. [4] Huebner et al. [5] Huebner et al. [3] Huebner & Kragic [2] Huebner et al. [3] Geidenstam et al. [6] Song et al. [1]

2008 Implementation of box approximation & decomposition (Huebner et al., ICRA) Implementation of pre-grasp selection strategies (Huebner & Kragic, IROS) 2009 Integration on Armar-III, KIT (Huebner et al., ICAR) Integration and learning of 2D grasping strategies (Geidenstam et al., RSS) 2010 Application for task-constraint learning (Song et al., IROS) Publication of open source software package BADGr (Huebner, IROS-WS)

International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 7

slide-11
SLIDE 11

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping BADGr System Architecture

System Architecture

(( Point Cloud )) ((Bound Box )) (( Pre- Grasp)) BoxGrasping generate fit split

◮ Input: object model, configurations, [gripper model, interface] ◮ Output: box shape representations, pre-grasp representations ◮ Tools: BoxGrasping, BoxXmlViewer, TaskLabeling http://www.csc.kth.se/~khubner/badgr/

International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 8

slide-12
SLIDE 12

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping BADGr System Architecture

System Architecture

(( Point Cloud )) ((Bound Box )) (( Pre- Grasp)) Expert BoxGrasping ((Object)) <Configuration> <Gripper> < Shape Features > < Grasp Features > setup parametrize generate fit split

◮ Input: object model, configurations, [gripper model, interface] ◮ Output: box shape representations, pre-grasp representations ◮ Tools: BoxGrasping, BoxXmlViewer, TaskLabeling http://www.csc.kth.se/~khubner/badgr/

International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 8

slide-13
SLIDE 13

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping BADGr System Architecture

System Architecture

(( Point Cloud )) ((Bound Box )) (( Pre- Grasp)) Expert User BoxXmlViewer TaskLabeling BoxGrasping ((Object)) <Configuration> <Gripper> < Shape Features > < Grasp Features > (( Task Label )) setup parametrize generate analyze provide fit split

◮ Input: object model, configurations, [gripper model, interface] ◮ Output: box shape representations, pre-grasp representations ◮ Tools: BoxGrasping, BoxXmlViewer, TaskLabeling http://www.csc.kth.se/~khubner/badgr/

International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 8

slide-14
SLIDE 14

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping BADGr System Architecture

System Architecture

(( Point Cloud )) ((Bound Box )) (( Pre- Grasp)) Expert User BoxXmlViewer TaskLabeling BoxGrasping ((Gripper)) ((Object)) ((Scene)) ((Stability, Contacts)) Environment <Configuration> <Gripper> < Shape Features > < Grasp Features > (( Task Label )) Interface extract execute extract setup provide parametrize parametrize generate analyze provide produce fit split

◮ Input: object model, configurations, [gripper model, interface] ◮ Output: box shape representations, pre-grasp representations ◮ Tools: BoxGrasping, BoxXmlViewer, TaskLabeling http://www.csc.kth.se/~khubner/badgr/

International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 8

slide-15
SLIDE 15

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Download & Documentation

◮ Demos, Download & Documentation: http://www.csc.kth.se/~khubner/badgr/

International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 9

slide-16
SLIDE 16

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Download & Documentation

Literature

[1]

  • D. Song, K. Huebner, V. Kyrki, and D. Kragic, “Learning Task Constraints for Robot Grasping using Graphical

Models,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2010. [2]

  • K. Huebner and D. Kragic, “Selection of Robot Pre-Grasps using Box-Based Shape Approximation,” in

IEEE/RSJ International Conference on Intelligent Robots and Systems, 2008, pp. 1765–1770. [3]

  • K. Huebner, S. Ruthotto, and D. Kragic, “Minimum Volume Bounding Box Decomposition for Shape

Approximation in Robot Grasping,” in IEEE International Conference on Robotics and Automation, 2008, pp. 1628–1633. [4]

  • B. Rasolzadeh, M. Björkman, K. Huebner, and D. Kragic, “An Active Vision System for Detecting, Fixating and

Manipulating Objects in Real World,” International Journal of Robotics Research, vol. 29, no. 2–3, pp. 133–154, 2010. [5]

  • K. Huebner, K. Welke, M. Przybylski, N. Vahrenkamp, T. Asfour, D. Kragic, and R. Dillmann, “Grasping Known

Objects with Humanoid Robots: A Box-Based Approach,” in 14th International Conference on Advanced Robotics, 2009. [6]

  • S. Geidenstam, K. Huebner, D. Banksell, and D. Kragic, “Learning of 2D grasping strategies from box-based 3D
  • bject approximations,” in Robotics: Science and Systems, Seattle, USA, June 2009, pp. 9–16.

[7]

  • J. Bohg, C. Barck-Holst, K. Huebner, M. Ralph, B. Rasolzadeh, D. Song, and D. Kragic, “Towards

Grasp-Oriented Visual Perception for Humanoid Robots,” International Journal of Humanoid Robotics, Special Issue on Active Vision of Humanoids, vol. 6, no. 3, pp. 387–434, 2009. International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 10

slide-17
SLIDE 17

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Download & Documentation

Literature

[1]

  • D. Song, K. Huebner, V. Kyrki, and D. Kragic, “Learning Task Constraints for Robot Grasping using Graphical

Models,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2010. Grasping II, TuET2.4 [2]

  • K. Huebner and D. Kragic, “Selection of Robot Pre-Grasps using Box-Based Shape Approximation,” in

IEEE/RSJ International Conference on Intelligent Robots and Systems, 2008, pp. 1765–1770. [3]

  • K. Huebner, S. Ruthotto, and D. Kragic, “Minimum Volume Bounding Box Decomposition for Shape

Approximation in Robot Grasping,” in IEEE International Conference on Robotics and Automation, 2008, pp. 1628–1633. [4]

  • B. Rasolzadeh, M. Björkman, K. Huebner, and D. Kragic, “An Active Vision System for Detecting, Fixating and

Manipulating Objects in Real World,” International Journal of Robotics Research, vol. 29, no. 2–3, pp. 133–154, 2010. [5]

  • K. Huebner, K. Welke, M. Przybylski, N. Vahrenkamp, T. Asfour, D. Kragic, and R. Dillmann, “Grasping Known

Objects with Humanoid Robots: A Box-Based Approach,” in 14th International Conference on Advanced Robotics, 2009. [6]

  • S. Geidenstam, K. Huebner, D. Banksell, and D. Kragic, “Learning of 2D grasping strategies from box-based 3D
  • bject approximations,” in Robotics: Science and Systems, Seattle, USA, June 2009, pp. 9–16.

[7]

  • J. Bohg, C. Barck-Holst, K. Huebner, M. Ralph, B. Rasolzadeh, D. Song, and D. Kragic, “Towards

Grasp-Oriented Visual Perception for Humanoid Robots,” International Journal of Humanoid Robotics, Special Issue on Active Vision of Humanoids, vol. 6, no. 3, pp. 387–434, 2009. International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 10

slide-18
SLIDE 18

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Conclusion

Conclusion

Open source framework for applications in different fields:

◮ Simplistic shape representation for clustering any kind of 3D data, plus

various other representations (hierarchies, 2.5D projections, etc.)

◮ Part-based representations of textured objects allows usage of shape

descriptors (Zernike optional, but also others)

◮ All representations allow creation of grasp databases for machine

learning, e.g., stability learning or task constraint learning. Issues and possible future work:

◮ Known vs. Unknown objects: completeness of data. ◮ Simulation vs. Reality: grasp stability learning.

International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 11

slide-19
SLIDE 19

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Conclusion

Conclusion

Open source framework for applications in different fields:

◮ Simplistic shape representation for clustering any kind of 3D data, plus

various other representations (hierarchies, 2.5D projections, etc.)

◮ Part-based representations of textured objects allows usage of shape

descriptors (Zernike optional, but also others)

◮ All representations allow creation of grasp databases for machine

learning, e.g., stability learning or task constraint learning. Issues and possible future work:

◮ Known vs. Unknown objects: completeness of data. ◮ Simulation vs. Reality: grasp stability learning.

International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 11

slide-20
SLIDE 20

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Conclusion

Conclusion

Open source framework for applications in different fields:

◮ Simplistic shape representation for clustering any kind of 3D data, plus

various other representations (hierarchies, 2.5D projections, etc.)

◮ Part-based representations of textured objects allows usage of shape

descriptors (Zernike optional, but also others)

◮ All representations allow creation of grasp databases for machine

learning, e.g., stability learning or task constraint learning. Issues and possible future work:

◮ Known vs. Unknown objects: completeness of data. ◮ Simulation vs. Reality: grasp stability learning.

International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 11

slide-21
SLIDE 21

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Conclusion

Conclusion

Open source framework for applications in different fields:

◮ Simplistic shape representation for clustering any kind of 3D data, plus

various other representations (hierarchies, 2.5D projections, etc.)

◮ Part-based representations of textured objects allows usage of shape

descriptors (Zernike optional, but also others)

◮ All representations allow creation of grasp databases for machine

learning, e.g., stability learning or task constraint learning. Issues and possible future work:

◮ Known vs. Unknown objects: completeness of data. ◮ Simulation vs. Reality: grasp stability learning.

International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 11

slide-22
SLIDE 22

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Conclusion

Conclusion

Open source framework for applications in different fields:

◮ Simplistic shape representation for clustering any kind of 3D data, plus

various other representations (hierarchies, 2.5D projections, etc.)

◮ Part-based representations of textured objects allows usage of shape

descriptors (Zernike optional, but also others)

◮ All representations allow creation of grasp databases for machine

learning, e.g., stability learning or task constraint learning. Issues and possible future work:

◮ Known vs. Unknown objects: completeness of data. ◮ Simulation vs. Reality: grasp stability learning.

International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 11

slide-23
SLIDE 23

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Conclusion

Conclusion

Open source framework for applications in different fields:

◮ Simplistic shape representation for clustering any kind of 3D data, plus

various other representations (hierarchies, 2.5D projections, etc.)

◮ Part-based representations of textured objects allows usage of shape

descriptors (Zernike optional, but also others)

◮ All representations allow creation of grasp databases for machine

learning, e.g., stability learning or task constraint learning. Issues and possible future work:

◮ Known vs. Unknown objects: completeness of data. ◮ Simulation vs. Reality: grasp stability learning.

International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 11

slide-24
SLIDE 24

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Acknowledgements

Thanks

Thanks for your attention! EU project IST-FP7-IP-215821 EU project IST-FP6-IP-027657 Swedish Foundation for Strategic Research

International Conference on Intelligent RObots and Systems 2010 Workshop on “Grasp Planning and Task Learning by Imitation” Slide 12