Librarian Y1 Final Report Oct 29, 2013, Personal Robotics Lab, CMU - - PowerPoint PPT Presentation

▶
librarian y1 final report
SMART_READER_LITE
LIVE PREVIEW

Librarian Y1 Final Report Oct 29, 2013, Personal Robotics Lab, CMU - - PowerPoint PPT Presentation

Jul 18, 2023 309 likes •646 views

Librarian Y1 Final Report Oct 29, 2013, Personal Robotics Lab, CMU Y1 Statement of Work Trimester 1: Infrastructure Setup and Initial Evaluation D1-1: Planning environment with kinematic model of Toyota robot, objects and

slide-1
SLIDE 1

Librarian Y1 Final Report

Oct 29, 2013, Personal Robotics Lab, CMU

slide-2
SLIDE 2

Y1 Statement of Work

  • Trimester 1: “Infrastructure Setup and Initial Evaluation”
  • D1-1: Planning environment with kinematic model of Toyota robot,
  • bjects and environment
  • D1-2: Perception and manipulation pipeline for task
  • D1-3: Support TEMA team for hardware drivers and system integration
  • Trimester 2: “Retrieving and Placing with non-prehensile physics-

based actions”

  • D2-1: Tech. for object sliding with quasi-static pushing physics
  • D2-2: Tech. for object pivoting and control
  • D3-3: Tech. for data driven error recovery for pivoting
  • Trimester 3: “Behavioral Framework for task planning”
  • D3-1: Full book manipulation pipeline on HERB and TEMA robot
  • D3-2: Error detection and recovery behavior engine
slide-3
SLIDE 3

Research Contributions

slide-4
SLIDE 4

Pregrasp Manipulation as Trajectory Optimization

Jennifer King, Matthew Klingensmith, Christopher Dellin, Mehmet Dogar, Prasanna Velagapudi, Nancy Pollard, and Siddhartha Srinivasa, "Pregrasp Manipulation as Trajectory Optimization," In Proc. of Robotics: Science and Systems, June, 2013.

1 2 3 4 5 6 7 8

Seconds (s)

Computation Time

100 200 300 400 500 600 700

Cost

10 20 30 40 50 60 70 80 90 100

Percentage (%)

Success Rate

606.67 531.11 64.28 83.33 6.643 3.91

  • Joint optimization of

reconfiguration and transport in multistage plans

  • Reconfiguration =

Non-prehensile push

  • Transport =

Pick-and-place

slide-5
SLIDE 5

Pregrasp Manipulation as Trajectory Optimization

  • Lattice planner to solve reconfiguration plans
  • CHOMP to solve transport plans
  • Strategy:
  • Compute reconfiguration to find start-configuration cost functional
  • Use CHOMP to optimize composite trajectory cost, including start cost

Reconfiguration (pushing) Transport (pick-and-place)

Reconfiguration Cost Transport Cost

slide-6
SLIDE 6

Pregrasp Manipulation as Trajectory Optimization

  • Push-planning under quasi-static physics
  • Contact forces map to velocity via normal of limit surface
  • Friction-cones bound contact forces to limit-surface region
  • Bounded limit surface region form velocity constraints corresponding

to a Dubins car

slide-7
SLIDE 7

Pregrasp Manipulation as Trajectory Optimization

  • Framework can be applied to general two-stage manipulation planning
  • Ex: optimizing pickup location of drill to minimize joint torques
slide-8
SLIDE 8

Hybrid Symbolic/Physical planning for manipulation

  • Python metaprogramming

wrapper for task planning

  • Binds physical planning and

execution with symbolic representations

  • Integrates with state-of-the-

art FastDownward planner

  • Generalized task libraries for

HERB and TIM

Initial state:

aware_of_object(bookcase) &

  • bj_at(Herb2,bookcase_place) &
  • bj_in(dracula,bookcase) &

is_loc(test_loc3) & is_loc(bookcase_place) & aware_of_object(dracula) & is_clear(bookcase)…

Plan:

grasp_from_bookshelf('dracula') goto_place_obj_at('dracula','hando ff') hand_obj_at('dracula','handoff') teleport_book('dracula','table') edge_grasp('dracula','table') goto('bookcase_place') place_in_bookcase('dracula')

Goals:

  • bj_handed_at('dracula','handoff'),
  • bj_on('dracula','desk'),
  • bj_in('dracula','bookcase')
slide-9
SLIDE 9

Machine Learning for Error Avoidance

slide-10
SLIDE 10

Predicting Errors using Machine Learning

Hypothesis: some errors can be avoided by learning relationships with single physical variables

slide-11
SLIDE 11

Predicting Errors using Machine Learning

Hypothesis: some errors can be avoided by learning relationships with single physical variables Classification tree:

  • simple constraints
  • built-in feature selection
slide-12
SLIDE 12

Constraint Learning from Subactions

  • Train error classifiers for every

'sub-action‘ within each PDDL action

  • Apply classifiers as planning

constraints or costs in CBiRRT, CHOMP, etc.

Learned constraints for grasp_from_bookshelf (Dataset of 20 trials on HERB):

======================================================= Action: PlanToNamedConfiguration1_start robot|relative|dracula|locX<=-1.120: False (100.00%) robot|relative|dracula|locX>-1.120 | robot|locY>1.655: True (100.00%) | robot|locY<=1.655 | | robot|relative|dracula|locX<=-1.075: True (100.00%) | | robot|relative|dracula|locX>-1.075: False (100.00%) ======================================================= Action: PlanToNamedConfiguration1_end right_ee|locX<=-1.025: False (100.00%) right_ee|locX>-1.025 | right_ee|relative|dracula|locY>-0.435: True (100.00%) | right_ee|relative|dracula|locY<=-0.435 | | right_ee|relative|dracula|locZ<=-0.925: True (100.00%) | | right_ee|relative|dracula|locZ>-0.925 | | | right_ee|locX<=-1.005: True (100.00%) | | | right_ee|locX>-1.005: False (100.00%)

grasp_from_bookshelf:

PlanToNamedConfiguration RotateSegway right_arm.PlanToConfiguration DriveStraightUntilForce …

slide-13
SLIDE 13

PDDLpy w/ Backtracking

Current symbolic state Current PDDL Plan Execution History

slide-14
SLIDE 14

State Estimation with Contact Sensors

  • Manipulation depends on

physical interaction with objects

  • Physics simulations inherently

introduce uncertainty

  • Cameras and range sensors

suffer from hand occlusions

  • Tactile sensors are highly

discriminative:

  • Accurately identify contact vs.

no contact

  • Cannot perfectly localize the
  • bject’s full pose

Using sensor feedback to track the object’s belief state

Michael C. Koval, Mehmet R. Dogar, Nancy S. Pollard, Siddhartha S. Srinivasa . “Pose Estimation for Contact Manipulation with Manifold Particle Filters ,” In Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013

slide-15
SLIDE 15

Conventional Particle Filter

Represents using a set of weighted samples Algorithm: 1. Sample from 2. Weight by 3. Resample according to the particles’ weights “Forward simulate, then weight using the observation”

slide-16
SLIDE 16

Degenerate Proposal Distribution

  • During contact almost everywhere
  • must be in non-penetrating contact with the hand
  • n the lower-dimensional contact manifold
  • Conventional PF performs worse as tactile sensors improve!

x y θ

slide-17
SLIDE 17

Manifold Particle Filter

Algorithm: 1. Decide if there was contact 2. If , then sample from the dual proposal 3. Otherwise, sample from the conventional proposal “Sample from the observation, then weight using history”

slide-18
SLIDE 18
slide-19
SLIDE 19

Results: CPF vs MPF

  • MPF always significantly outperforms the CPF
  • MPF improves as tactile sensors become higher resolution
slide-20
SLIDE 20

Infrastructure Contributions

slide-21
SLIDE 21

TIMpy/HERBpy/PrPy Infrastructure

  • Common python layer between TIM

and HERB

  • Shared planning and execution

between platforms

  • Standalone simulation environments

to allow remote development

  • Core components:
  • OpenRAVE
  • ROS
  • Python
slide-22
SLIDE 22

HERB/Tim Infrastructure

OWD/Schunk controller segwayrmp/ neobotix moped3d ROS Nodes OWD/IPA BHD Controller Base Controller Marker System OpenRAVE OpenRAVE Multicontroller Grab PushGrasp OpenDoor HerbPy PrPy PDDLPy PressButton Python API Task Planning HERBpy/TIMpy PrPy

slide-23
SLIDE 23

OpenRAVE URDF Loader

  • Plugin that allows direct

import of URDF files

  • Status:
  • URDF conversion to

OpenRAVE Kinbody

  • Model loading

Positioning relative joints

  • Issues:
  • Mesh simplification still

manual

  • Requires additional YAML

parameter file

slide-24
SLIDE 24
  • r_rviz – OpenRAVERViz Bridge
slide-25
SLIDE 25

Personal Robotics APT server

(packages.personalrobotics.ri.cmu.edu)

  • Live for internal use, moving to public
slide-26
SLIDE 26

Y2 / Phase II Overview

slide-27
SLIDE 27

Y2 - Timeline

Task Nov-Jan Feb Book Manipulation Multi-hand Grasping

  • Various books (small, large, soft cover, etc.)
  • In the presence of clutter
  • Grasping book between bookends on the table
  • Work with Jen on Grasping/Manipulation capture region

with dual arm control Evaluation Reflection TIM integration Error Handling Showing error recovery and improvement

  • Using data from book manip show error recovery
  • Show the changes in planning based on historical error
  • Show overall performance increase (less errors and faster

planning) Evaluation Reflection TIM integration Multimodal Manipulation Investigation of tactile sensors and finger variations

  • Test syntouch tactile sensors and port to error detection
  • Explore finger variations to assist in book manipulation

and grasping Evaluation Reflection TIM integration

Phase 2.1 Nov 2013-Feb 2014

slide-28
SLIDE 28

Y2 - Timeline

Task Mar-May Jun Book Manipulation Multi-hand Grasping

  • All books
  • More dual arm constrained motions with clutter
  • Dual arm grasping in cabinets
  • Synchronous dual arm movements holding same item

Evaluation Reflection TIM integration Error Handling Streamline error handling and learned feedback

  • Improve code to show streamlined process
  • Show multimodal consideration in error handling
  • Show overall performance increase (less errors and faster

planning) Evaluation Reflection TIM integration Multimodal Manipulation Investigation of tactile sensors and finger variations

  • Use tactile sensors in the loop for book manipulation
  • Compare various hands for manipulation (potentially in

simulation) Evaluation Reflection TIM integration

Phase 2.2 Mar 2014-June 2014

slide-29
SLIDE 29

Y2 - Timeline

Task July-September October General Manipulation Multi-hand Grasping

  • Start working on grocery sorting and placement
  • Work on scooping and grasping in obscured locations
  • Show transferrable grasping strategies from our book

grasping Evaluation Reflection TIM integration FINAL REPORT Error Handling Showing error recovery and improvement

  • Expand data collection to include new tasks
  • Continue to verify and test error handling and

improvements

  • Look at and determine if there are any newer better

processes for dealing with errors Evaluation Reflection TIM integration FINAL REPORT Multimodal Manipulation Investigation of tactile sensors and finger variations

  • Detect slippage using tactile sensors on fingers and prevent

errors (e.g. increase gripping force and /or bringing a second hand to support the object from dropping) Evaluation Reflection TIM integration FINAL REPORT

Phase 2.3 July 2014-Oct 2014

slide-30
SLIDE 30

Y2 – Statement of Work

  • Trimester 1: “Generalizing Book Manipulation”
  • D1-1:

Generalized pushing across object shapes

  • D1-2:

Error avoidance task-based framework

  • D1-3:

Preliminary results on MPF with tactile sensing

  • Trimester 2: “Physics-based feedback driven primitives”
  • D2-1:

Multi-hand primitives for physics-based manipulation

  • D2-2:

Physics-aware symbolic planning

  • D3-3:
  • Tech. for MPF for object pose and physics parameters
  • Trimester 3: “Framework for task and primitive planning”
  • D3-1:

Full book manipulation, grocery bag pipeline on HERB and TEMA robot

  • D3-2:

Integrated task and primitive planning framework

slide-31
SLIDE 31

Y2 - Personnel

  • Professor Siddhartha Srinivasa (1 month)
  • Project Scientist Pras Velagapudi (50%)
  • PhD student Jennifer King (100%)
  • PhD student Michael Koval (50%)
  • MS student Evan Shapiro (100%)
slide-32
SLIDE 32

Librarian Y1 Final Report

Oct 29, 2013, Personal Robotics Lab, CMU