Librarian Y1 Final Report Oct 29, 2013, Personal Robotics Lab, CMU - PowerPoint PPT Presentation

Librarian Y1 Final Report Oct 29, 2013, Personal Robotics Lab, CMU

Y1 Statement of Work • Trimester 1: “Infrastructure Setup and Initial Evaluation” • D1-1: Planning environment with kinematic model of Toyota robot, objects and environment • D1-2: Perception and manipulation pipeline for task • D1-3: Support TEMA team for hardware drivers and system integration • Trimester 2: “Retrieving and Placing with non-prehensile physics- based actions” • D2-1: Tech. for object sliding with quasi-static pushing physics • D2-2: Tech. for object pivoting and control • D3-3: Tech. for data driven error recovery for pivoting • Trimester 3: “Behavioral Framework for task planning” • D3-1: Full book manipulation pipeline on HERB and TEMA robot • D3-2: Error detection and recovery behavior engine

Research Contributions

Pregrasp Manipulation as Trajectory Optimization Jennifer King, Matthew Klingensmith, Christopher Dellin, Mehmet Dogar, Prasanna Velagapudi, Nancy Pollard, and Siddhartha Srinivasa, "Pregrasp Manipulation as Trajectory Optimization," In Proc. of Robotics: Science and Systems, June, 2013. • Joint optimization of reconfiguration and transport in multistage plans • Reconfiguration = Non-prehensile push • Transport = Pick-and-place Success Rate Cost Computation Time 100 700 606.67 6.643 8 83.33 90 531.11 600 7 Percentage (%) 80 64.28 Seconds (s) 6 70 500 5 60 3.91 400 50 4 300 40 3 30 200 2 20 100 1 10 0 0 0

Pregrasp Manipulation as Trajectory Optimization • Lattice planner to solve reconfiguration plans • CHOMP to solve transport plans • Strategy: • Compute reconfiguration to find start-configuration cost functional • Use CHOMP to optimize composite trajectory cost, including start cost Reconfiguration (pushing) Transport (pick-and-place) Reconfiguration Transport Cost Cost

Pregrasp Manipulation as Trajectory Optimization • Push-planning under quasi-static physics • Contact forces map to velocity via normal of limit surface • Friction-cones bound contact forces to limit-surface region • Bounded limit surface region form velocity constraints corresponding to a Dubins car

Pregrasp Manipulation as Trajectory Optimization • Framework can be applied to general two-stage manipulation planning • Ex: optimizing pickup location of drill to minimize joint torques

Hybrid Symbolic/Physical planning for manipulation • Python metaprogramming Initial state: Goals: wrapper for task planning aware_of_object(bookcase) & obj_handed_at('dracula','handoff'), obj_at(Herb2,bookcase_place) & obj_on('dracula','desk'), obj_in(dracula,bookcase) & obj_in('dracula','bookcase') is_loc(test_loc3) & • Binds physical planning and is_loc(bookcase_place) & execution with symbolic aware_of_object(dracula) & is_clear(bookcase)… representations • Integrates with state-of-the- Plan: art FastDownward planner grasp_from_bookshelf('dracula') goto_place_obj_at('dracula','hando ff') hand_obj_at('dracula','handoff') teleport_book('dracula','table') • Generalized task libraries for edge_grasp('dracula','table') HERB and TIM goto('bookcase_place') place_in_bookcase('dracula')

Machine Learning for Error Avoidance

Predicting Errors using Machine Learning Hypothesis: some errors can be avoided by learning relationships with single physical variables

Predicting Errors using Machine Learning Hypothesis: some errors can be avoided by learning relationships with single physical variables Classification tree: • simple constraints • built-in feature selection

Constraint Learning from Subactions • Train error classifiers for every Learned constraints for grasp_from_bookshelf 'sub-action‘ within each PDDL (Dataset of 20 trials on HERB): action ======================================================= • Apply classifiers as planning Action: PlanToNamedConfiguration1_start robot|relative|dracula|locX<=-1.120: False (100.00%) constraints or costs in CBiRRT, robot|relative|dracula|locX>-1.120 | robot|locY>1.655: True (100.00%) CHOMP, etc. | robot|locY<=1.655 | | robot|relative|dracula|locX<=-1.075: True (100.00%) | | robot|relative|dracula|locX>-1.075: False (100.00%) ======================================================= grasp_from_bookshelf: Action: PlanToNamedConfiguration1_end right_ee|locX<=-1.025: False (100.00%) right_ee|locX>-1.025 PlanToNamedConfiguration | right_ee|relative|dracula|locY>-0.435: True (100.00%) | right_ee|relative|dracula|locY<=-0.435 | | right_ee|relative|dracula|locZ<=-0.925: True (100.00%) RotateSegway | | right_ee|relative|dracula|locZ>-0.925 | | | right_ee|locX<=-1.005: True (100.00%) | | | right_ee|locX>-1.005: False (100.00%) right_arm.PlanToConfiguration DriveStraightUntilForce …

PDDLpy w/ Backtracking Execution History Current symbolic state Current PDDL Plan

State Estimation with Contact Sensors Michael C. Koval, Mehmet R. Dogar, Nancy S. Pollard, Siddhartha S. Srinivasa . “Pose Estimation for Contact Manipulation with Manifold Particle Filters ,” In Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013 Using sensor feedback to track • Manipulation depends on physical interaction with objects the object’s belief state • Physics simulations inherently introduce uncertainty • Cameras and range sensors suffer from hand occlusions • Tactile sensors are highly discriminative: • Accurately identify contact vs. no contact • Cannot perfectly localize the object’s full pose

Conventional Particle Filter Represents using a set of weighted samples Algorithm: 1. Sample from 2. Weight by 3. Resample according to the particles’ weights “Forward simulate, then weight using the observation”

Degenerate Proposal Distribution • During contact almost everywhere must be in non-penetrating contact with the hand • on the lower-dimensional contact manifold • θ x y • Conventional PF performs worse as tactile sensors improve!

Manifold Particle Filter Algorithm: 1. Decide if there was contact 2. If , then sample from the dual proposal 3. Otherwise, sample from the conventional proposal “Sample from the observation, then weight using history”

Results: CPF vs MPF • MPF always significantly outperforms the CPF • MPF improves as tactile sensors become higher resolution

Infrastructure Contributions

TIMpy/HERBpy/PrPy Infrastructure Common python layer between TIM • and HERB Shared planning and execution • between platforms Standalone simulation environments • to allow remote development Core components: • OpenRAVE • ROS • Python •

HERB/Tim Infrastructure Task Planning PDDLPy Grab PushGrasp OpenDoor PressButton Python API HERBpy/TIMpy HerbPy PrPy PrPy OpenRAVE OpenRAVE Multicontroller OWD/IPA BHD Controller Base Controller Marker System ROS Nodes segwayrmp/ OWD/Schunk controller moped3d neobotix

OpenRAVE URDF Loader • Plugin that allows direct import of URDF files • Status: • URDF conversion to OpenRAVE Kinbody • Model loading Positioning relative joints • Issues: • Mesh simplification still manual • Requires additional YAML parameter file

or_rviz – OpenRAVE � RViz Bridge

Personal Robotics APT server (packages.personalrobotics.ri.cmu.edu) • Live for internal use, moving to public

Y2 / Phase II Overview

Y2 - Timeline Phase 2.1 Nov 2013-Feb 2014 Task Nov-Jan Feb Evaluation Book Multi-hand Grasping � Manipulation Various books (small, large, soft cover, etc.) Reflection � In the presence of clutter TIM integration � Grasping book between bookends on the table � Work with Jen on Grasping/Manipulation capture region with dual arm control Error Handling Showing error recovery and improvement Evaluation � Reflection Using data from book manip show error recovery � Show the changes in planning based on historical error TIM integration � Show overall performance increase (less errors and faster planning) Multimodal Investigation of tactile sensors and finger variations Evaluation � Manipulation Test syntouch tactile sensors and port to error detection Reflection � Explore finger variations to assist in book manipulation TIM integration and grasping

Y2 - Timeline Phase 2.2 Mar 2014-June 2014 Task Mar-May Jun Book Multi-hand Grasping Evaluation � Manipulation All books Reflection � More dual arm constrained motions with clutter TIM integration � Dual arm grasping in cabinets � Synchronous dual arm movements holding same item Error Handling Streamline error handling and learned feedback Evaluation � Improve code to show streamlined process Reflection � Show multimodal consideration in error handling TIM integration � Show overall performance increase (less errors and faster planning) Multimodal Investigation of tactile sensors and finger variations Evaluation � Manipulation Use tactile sensors in the loop for book manipulation Reflection � Compare various hands for manipulation (potentially in TIM integration simulation)