A Deliberation Layer for Instantiating Robot Execution Plans from - - PowerPoint PPT Presentation

▶
a deliberation layer for instantiating robot execution
SMART_READER_LITE
LIVE PREVIEW

A Deliberation Layer for Instantiating Robot Execution Plans from - - PowerPoint PPT Presentation

Feb 24, 2023 95 likes •298 views

A Deliberation Layer for Instantiating Robot Execution Plans from Abstract Task Descriptions June 10th, 2013 D. Di Marco , R.J.M. Janssen , A.C. Perzylo , M.J.G. van de Molengrafu , P. Levi Eindhoven University of

slide-1
SLIDE 1

A Deliberation Layer for Instantiating Robot Execution Plans from Abstract Task Descriptions

June 10th, 2013

  • D. Di Marco∗, R.J.M. Janssen◦, A.C. Perzylo⊛, M.J.G. van de Molengrafu◦, P. Levi∗
  • Eindhoven University of Technology, The Netherlands

⊛ Technische Universität München, Germany ∗ Universität Stuttgart, Germany

dimarco@ipvs.uni-stuttgart.de

slide-2
SLIDE 2

Motivation

Primary question: How can task descriptions be shared among different service robot platforms in different environments?

  • Represent task descriptions in an abstract way (action recipe)
  • Decouple from environment & robot hardware specifics
  • Store information in a globally accessible database

How use these to generate execution plans tailored to the current situation?

2 / 14 A Deliberation Layer for Instantiating Robot Execution Plans from Abstract Task Descriptions

slide-3
SLIDE 3

Action Recipe

BringSomethingSomewhere PickupSomething NavigateToHandover HandoverSomething RetractArmToParking

Abstract (i.e. independent of robot platform and environment) task descriptions1

  • Formulated in OWL, based on a

common ontology

  • Can be composed of other recipes or

primitive actions, sub-actions are partially ordered

  • Annotated with description of

requirements, to check whether a given robot is capable of executing these . Example Recipe . .

Class: BringSomethingSomewhere EquivalentTo: (knowrob:toLocation some robotPose-handover1) SubClassOf: roboearth:IntentionalAction ... Individual: ServeADrinkOrder10 Types: knowrob:PartialOrdering-Strict Facts: knowrob:occursBeforeInOrdering MoveBaseToGraspPose, knowrob:occursAfterInOrdering GraspBottle

  • 1M. Tenorth et al.: ”The RoboEarth language: Representing and Exchanging

Knowledge about Actions, Objects, and Environments”, ICRA 2012

3 / 14 A Deliberation Layer for Instantiating Robot Execution Plans from Abstract Task Descriptions

slide-4
SLIDE 4

Action Recipe

BringSomethingSomewhere PickupSomething NavigateToHandover HandoverSomething RetractArmToParking

Abstract (i.e. independent of robot platform and environment) task descriptions1

  • Formulated in OWL, based on a

common ontology

  • Can be composed of other recipes or

primitive actions, sub-actions are partially ordered

  • Annotated with description of

requirements, to check whether a given robot is capable of executing these . Example Recipe . .

Class: BringSomethingSomewhere EquivalentTo: (knowrob:toLocation some robotPose-handover1) SubClassOf: roboearth:IntentionalAction ... Individual: ServeADrinkOrder10 Types: knowrob:PartialOrdering-Strict Facts: knowrob:occursBeforeInOrdering MoveBaseToGraspPose, knowrob:occursAfterInOrdering GraspBottle

  • 1M. Tenorth et al.: ”The RoboEarth language: Representing and Exchanging

Knowledge about Actions, Objects, and Environments”, ICRA 2012

3 / 14 A Deliberation Layer for Instantiating Robot Execution Plans from Abstract Task Descriptions

slide-5
SLIDE 5

Action Instantiation

  • Task: navigate into the next

room

  • How to infer that ”Navigate”

has to be instantiated as ”open door, pass open door”? ⇒ we need more knowledge

4 / 14 A Deliberation Layer for Instantiating Robot Execution Plans from Abstract Task Descriptions

slide-6
SLIDE 6

Action Recipe Annotation

BringSomethingSomewhere PickupSomething NavigateToHandover HandoverSomething RetractArmToParking already there? door between? default case NAVIGATE-TASK ?robot ?goal No Operation

NavInReachOfDoorButton PushDoorButton NavigateToTarget OpenDoor

MoveToPrimitive

HTN decompositions for actions with preconditions & effects, fully encoded in OWL and linked to recipes.

5 / 14 A Deliberation Layer for Instantiating Robot Execution Plans from Abstract Task Descriptions

slide-7
SLIDE 7

Plan Instantiation Process

Symbolic Plan CPL Execution Plan HTN Planner Semantic Map Recipe HTN Annotations HTN world state description HTN Planning Domain SRDL Robot Description Action Recipes

  • Semantic Maps: basic topology
  • f environment
  • SRDL: Semantic Robot

Description Language1

  • Off-the-shelf HTN planner:

SHOP21

  • CRAM Plan Language (CPL) to

specify execution plans

  • 1L. Kunze et al.: ”Towards semantic robot description languages”. ICRA 2011
  • 2D. Nau et al.: ”SHOP2: An HTN planning system”. JAIR, 2003

6 / 14 A Deliberation Layer for Instantiating Robot Execution Plans from Abstract Task Descriptions

slide-8
SLIDE 8

Plan Instantiation Process

Symbolic Plan CPL Execution Plan HTN Planner Semantic Map Recipe HTN Annotations HTN world state description HTN Planning Domain SRDL Robot Description Action Recipes

OWL CRAM-PL SHOP2

  • Semantic Maps: basic topology
  • f environment
  • SRDL: Semantic Robot

Description Language1

  • Off-the-shelf HTN planner:

SHOP21

  • CRAM Plan Language (CPL) to

specify execution plans

  • 1L. Kunze et al.: ”Towards semantic robot description languages”. ICRA 2011
  • 2D. Nau et al.: ”SHOP2: An HTN planning system”. JAIR, 2003

6 / 14 A Deliberation Layer for Instantiating Robot Execution Plans from Abstract Task Descriptions

slide-9
SLIDE 9

CRAM Plan Language1

Extension of Common Lisp for robot execution plans

  • Control structures for executing/synchronizing tasks in parallel
  • Implementation of the Rete algorithm for Prolog-like

reasoning facilities

  • Designators: symbolic description of objects, locations, task

parts

  • Based on ideas from Firby2
  • Created from action specifications in recipe
  • Get resolved at the latest possible instant during plan execution
  • Independent of Hardware/Environment
  • 1M. Beetz, L. Mösenlechner, M. Tenorth, ”CRAM - A Cognitive Robot Abstract

Machine for Everyday Manipulation in Human Environments”. IROS, 2010

2R.J. Firby, ”Adaptive execution in complex dynamic worlds”. PhD thesis, Yale

University, 1989

7 / 14 A Deliberation Layer for Instantiating Robot Execution Plans from Abstract Task Descriptions

slide-10
SLIDE 10

Architecture

Process Modules

Cloud Storage

SHOP2 Planner Domain SeRQL Interface Local Inference System

Local Know- ledge Base Local World Model

Comm Query Response CRAM executive Parametrized CPL Plans Human Machine Interface Detections ROS Actionlib Calls ROS component layer Problem Planning Domain Extraction Query Response CPL Plan Generation SHOP2 Plan Query Response

Semantic Maps SRDL Robot Descr. Action Recipes Recipe HTN Annotations

Simulated Perception Task Definition 8 / 14 A Deliberation Layer for Instantiating Robot Execution Plans from Abstract Task Descriptions

slide-11
SLIDE 11

Simulation Experiment

  • Gazebo simulator1 used
  • Fraunhofer IPA’s Care-o-Bot 3-4 (1 arm, 1 tray) and TU/e’s

Amigo (2 arms) prototype

  • Task description: ”bring something somewhere”
  • Both robots use same top-level recipe, but need to select

different decompositions

1http://gazebosim.org 9 / 14 A Deliberation Layer for Instantiating Robot Execution Plans from Abstract Task Descriptions

slide-12
SLIDE 12

Experiment Environment

The door can be opened by touching the green button. The used arm has to be free.

10 / 14 A Deliberation Layer for Instantiating Robot Execution Plans from Abstract Task Descriptions

slide-13
SLIDE 13

Differences in generated Plans

NAVIGATE-TASK

MoveTo

No door: navigation between rooms trivial

11 / 14 A Deliberation Layer for Instantiating Robot Execution Plans from Abstract Task Descriptions

slide-14
SLIDE 14

Differences in generated Plans

NAVIGATE-TASK

MoveArmTo MoveTo MoveTo FREE-ARMS-FROM-ENVIRONMENT FREE-ARM-FOR-GRASPING ChangeGripperState TuckArm FREE-ARMS-FROM-ENVIRONMENT NoOP NoOP NoOP OPEN-DOOR

2 manipulators: grasp object with lefu, open door with right arm

11 / 14 A Deliberation Layer for Instantiating Robot Execution Plans from Abstract Task Descriptions

slide-15
SLIDE 15

Differences in generated Plans

NAVIGATE-TASK

MoveArmTo MoveTo MoveTo FREE-ARMS-FROM-ENVIRONMENT FREE-ARM-FOR-GRASPING ChangeGripperState TuckArm FREE-ARMS-FROM-ENVIRONMENT PickupObjectFromTray NoOP PutObjectOnTray OPEN-DOOR

1 manipulator + tray: put object on tray, open door, pickup object

11 / 14 A Deliberation Layer for Instantiating Robot Execution Plans from Abstract Task Descriptions

slide-16
SLIDE 16

Video

  • Video Demo -

12 / 14 A Deliberation Layer for Instantiating Robot Execution Plans from Abstract Task Descriptions

slide-17
SLIDE 17

Conclusion

  • Increase applicability of abstract task descriptions using
  • ff-the-shelf HTN planning
  • Separation of sources of knowledge
  • Experiment required about 4K lines of OWL
  • Classic symbolic reasoning, e.g. no temporal aspects

Future work:

  • Apply approach to more difficult scenarios
  • Integrate plan generation and task execution for more robust

behavior

13 / 14 A Deliberation Layer for Instantiating Robot Execution Plans from Abstract Task Descriptions

slide-18
SLIDE 18

Discussion

Thank you for your attention

14 / 14 A Deliberation Layer for Instantiating Robot Execution Plans from Abstract Task Descriptions