Grope and Hope Matthew T. Mason Dagstuhl Workshop on Multimodal - - PowerPoint PPT Presentation

SLIDE 1

Grope and Hope

Matthew T. Mason Dagstuhl Workshop on Multimodal Manipulation Under Uncertainty October 2015

SLIDE 2

Thanks!

SLIDE 3

collaborators and sponsors

Robbie Paolini Alberto Rodriguez Nikhil Chavan-Dafle Annie Holladay Weiwei Wan Laura Herlant Siddhartha Srinivasa Mike Erdmann Marc Raibert Tomas Lozano-Perez Zhiwei Zhang Erol Sahin with thanks to 
 Jeff Trinkle for the title National Science Foundation DARPA Army Research Office ABB

SLIDE 4

Simple Complex Specialized General

Factory grippers Human hand Anthropomorphic robot hands Human-operated simple hands

Anthropomorphism Toolomorphism

SLIDE 5

An inspiration: the pickup tool

SLIDE 6

Pickup tool philosophy

• Let the fingers fall where they may
• Instead of “put the fingers in the right

place”.

• Grasp first, ask questions later
• Instead of knowing pose in advance, and

avoiding object motion during grasp.

• Grope and hope

SLIDE 7

Dagstuhl 2009

Simplicity, Generality, Robotics

SLIDE 8

SLIDE 9

Most significant grasp types.

SLIDE 10

• SVM
• Gaussian kernel
• Minimize misclassifications and maximize

margin for correctly classified examples.

12.6 25.2 37.7 12.6 25.2 37.7 50.2 0.1 6.4 12.6 18.9 25.2 31.4 37.7

SLIDE 11

Singulation

SVM, Gaussian kernel, 
 leave half out cross validation,
 mean accuracy: 92.9%

Orientation

K nearest neighbors, 
 leave one out cross-validation,
 mean error: 8.4°

SLIDE 12

The MLab Hand’s 
 approach to haptic perception

• Skin receptors?


No

• Intrinsic tactile sensing? [Bicchi & Dario 1987]


A generalization


SLIDE 13

Hyperintrinsic tactile sensing

• Exclude unstable poses
• Use proprioceptors to get the last few bits
• Is wholly intrinsic tactile sensing possible?

Determine pose with zero sensory bits?

SLIDE 14

SLIDE 15

15

a ul t

• as

Figure 2: The Flex Feeder with a rotatable fence. The verification stage at the end of the plan causes it to iterate until a desired state is achieved. We define the expected cost of a plan as the cost of the plan times the expected number of iterations. The expected number of iterations for a plan that succeeds with probability p is l/p. If the desired state is ti’, we can define the probability that a plan is successful as p =fn(H*Iu). Then the expected cost of the plan is C(u)/p. A Bayes’ plan is one with minimal expected cost (note that there may be more than one Bayes’ plan).

• 4. Application to Parallel-Jaw Grasping

Figure 2 bearing reduces friction between the jaws (drawing Brown).

A “frictionless” parallel-jaw gripper. A linear

by Ben

4.1.

Assumptions In the remainder of the paper we apply the Bayesian framework to the problem of grasping with a parallel-jaw gripper, which we think

• f as a two-dimensional problem. We assume that:
• 1. The gripper has two linear jaws arranged in parallel.
• 2. Objects to be grasped are known rigid planar polygons.
• 3. The object’s initial position is unconstrained as long as it lies

somewhere between the two jaws. The object remains between the jaws throughout grasping; hence any polygon is equivalent to its convex hull.

• 4. All motion occurs i

n the plane and is slow enough that inertial forces are negligable. The scope of this quasi-stutic model is discussed in (Mason, 1986b) and (Peshkin, 1986).

• 5. The direction of squeezing is always perpendicular to the jaws.
• 6. There is zero friction between object and the jaws.
• 7. Both jaws make contact simultaneously (pure squeezing). Once

contact is made between a jaw and the object, the two surfaces remain in contact throughout the grasp. A grasp continues until further motion would deform the object. The first four assumptions were used by Brost (1988). Taylor et

• al. (1987). and Mason et al. (1988). The latter three assumptions

simplify the analysis and improve the combinatorics of the search. By restricting gripper motion to be perpendicular to the gripper jaws, we obtain a one-dimensional space of actions to search. By using a frictionless gripper (see Figure 2) we eliminate “wedging” of the

• bject, leaving a finite set of stable object orientations. By assuming

simultaneous

contact by the jaws (pure squeezing) we eliminate the dificult analysis of pushing motions, and obtain predictions that are independent of the support friction. We next specify the six-tuple < 0

.ft. A.

G.

• C. t

i ’ > that defines an instance of a parallel-jaw grasping problem. Figure 3: The diameter function for the five-sided object shown at the right in its zero orientation. During a squeeze, the object rotates so as to reduce the diameter, terminating when the diameter reaches a local minimum. The probability distribution after the first squeezing step is shown as a dotted histogram. 4.2. The Transfer Function G In this paper we assume that the jaws make simultaneous contact and neither jaw loses contact-pure

• squeezing. The effect of a squeeze

is predicted using the diameter

function, which gives the minimum

jaw separation as a function of the orientation of the object relative to the jaws. For polygonal objects the diameter function is piecewise sinusoidal (see Figure 4.1). During a squeeze, the object passively rotates to reduce the diameter, terminating when the diameter reaches a local minimum. I

f we assume a linear spring force between the

jaws, the square of the diameter defines a potential function where local minima are “stable” in the sense that small deviations produce restoring forces. The diameter function has a period of i

i ,

so that it is impossible to remove a 180 degree ambiguity in object orientation through squeezing alone. See Appendix A for details on the diameter function. A different ambiguity arises when the object’s orientation is close to a local maximum in the diameter function. With a frictional gripper, the object would be wedged. Even with a frictionless gripper, an unstable equilibrium exists at the local maximum. Assuming that the prior probability density of orientations is continuous, the initial squeeze will encounter an unstable equilibrium with probability zero. Thereafter we avoid ambiguous actions. Note that the analysis of grasp mechanics using the diameter func- tion is not limited to polygonal objects. Any two-dimensional object can be analyzed if we can compute its diameter function.

• 43. The State Space (-1 and Prior Probabilityfl

The space of object orientations is the uncountable set of all pla- nar angles, i.e. the half-open interval from zero to 27i. After the first squeeze step the object rotates into one of its stable orientations (where at least one edge of the object is aligned with the gripper, see above). The state space for the planning problem is this finite set of

• bject orientations.

We consider the initial orientation of the object to be a random vari- able on the space of rotations. After the first squeezing action, the

1266 Figure 8. Grippers 15 I

I I

I I

I I

tJ

;

(a) (b) (c)

axis of ' rotation(

gravity

g

1

Figure 2: Notation for club throwing. characterized by the following equations: where T, is the time elapsed during the carry, e, is the angular velocity at the release, ' U , is the linear velocity at the release, Tf is the time of flight of the club, and h is the maximum height of the center of mass above the release point.

3.1

Carrying conditions

The forces and torques (measured in the coordinate system fixed

• n the club) which must be applied to the club during the carry

are given by the following equations: where m is the mass of the club and p is its radius of gyration. The angle 311 and the contacts between the foot and the club should be chosen to keep these forces inside the wrench friction cone.

3.2

Release conditions

Here we will consider the simplest type of release: simultaneous breaking of all contacts between the foot and the club. For this to be possible, each contact point on the foot must be able to accelerate away from the club during the release. Equivalently, fY \

4

\ \ I1 = 1

L

W = -10 deg Figure 3: Forces applied to the club during the carry of a one rotation throw. i i i s normal to the club. Figure 4: Constraints on the contact locations: (a) finite club length; (b) release constraints; (c) the combined constraints and the normal forces and torques during the trajectory (normalized to the unit circle). each contact normal (into the club) must have a positive angular sense about the axis of rotation. If this condition is satisfied, the club instantaneously breaks contact for sufficiently large decelerations of the foot. (Note that this is only a necessary local condition for simultaneous release. The complete paths of the foot and the club as it spins away must be considered to ensure a clean release.)

3.3 Choosing contacts

Given a club of length 0.5, a club angle 4 = -10 degrees, a leg such that T = 1, and a goal to throw the club so that it rotates

• nce in the air (

T I . = l), how should the foot contact the club? Figure 3 shows the evolution of the forces which must be applied to the club during the carry. The coefficient of friction between the foot and the club must be large enough to include these forces in the total wrench friction cone. In this case, the coefficient of friction must be at least 0.7 for a successful carry. The choice of contact points on the club is constrained by the length of the club, the release condition, and the forces and torques which must be applied to the club during the carry. To visualize these constraints, we can plot them in a force-torque space, where the force axis is the force in the direction f i normal to the club. The limited club length imposes a constraint on the relative values of the normal contact force and the torque about the center of mass of the club. These constraints are illustrated in Figure 4(a), where the shaded forces are unattainable. The release condition provides an additional constraint, shown in Figure 4(b). These constraints are combined in Figure 4(c). Each point on the unit circle which is not in a shaded region represents a permissible contact. Also plotted are the normal forces and torques (normalized to the unit circle) that must be applied to the

155

Figure 2: Inchworm mode. Figure 3: Cylinder rolling mode.

describes some variations of the robot. Finally we discuss

SLIDE 16

Then versus now

• I used to work on:
• Now:

Simple effectors can do a lot!

SLIDE 17

Back to the present

• MLab hand can pick. Highlighters at least.
• Can it place?
• Can it do in-hand manipulation?
• Can it change grasp types?

SLIDE 18

The MLab Hand

SLIDE 19

Task: take a marker from bin and put it on stage using only haptic sense data

SLIDE 20

Ground truth cameras Grasp Place

SLIDE 21

SLIDE 22

The last challenge:
 changing grasps

SLIDE 23

A grasp taxonomy

CUTKOSKY: DESIGN OF HANDS FOR MANUFACTURING TASKS

273

emphasis on

Grasp

emphasis on Securfty,Stablllty dexter@. sensltluuy

I

Power

clnmplng required

Non-Ihehensile Prehknsile

thin

Hook. Uatfam. push 15 h t d p i n e h

Prismatic Circ'ular

(wrap aynu".

Bngen lradlal symmetly.

surround part)

kgcm surround pad

Precision

Specltlc Grasp8

italic labels - grasp attributes

b o l d I . o o ~ - g r p ~

4 b

Increasing Power Increasing D

e x t e r i t y .

and Object Size

Decreasing Object Size

• Fig. 4. A partial taxonomy of manufacturing grasps, modified from a taxonomy presented in 141. The drawings of hands were

provided by M. .

I .

Dowling and are reprinted with permission of the Robotics Institute, Carnegie-Mellon University.

are the most precise. However, the trend is not strictly

• followed. A Spherical Power grasp may be either more or less

dextrous than a Medium Wrap, depending on the size of the

• sphere. Moving from top to bottom, the trend is from general

task considerations, such as whether clamping is required, to details of geometry and sensing. Again, the trend is not strictly

• bserved. For example, a small, flat object may provoke the

choice of a Lateral Pinch grip near the top of the tree. The role of task forces and torques on grip choice is most apparent when the hand shifts between grips during a task. For example, in unscrewing a knob the hand shifts from Grasp 11 to Grasp 13. Similarly, when holding a tool, as in Grasp 3, the hand shifts to Grasp 5 as the task-related forces decrease and may adopt Grasp 6, a precision grasp, if the forces become still smaller. The role of object size is most apparent when similar tasks are performed with different tools. For example, in light assembly work Grasps 12 and 13 approach Grasp 14, and finally Grasp 9, as the objects become very small. A related observation, brought out more clearly in developing the grasp expert system discussed in Section IV, is that sequences can be traced in the taxonomy, corresponding to adjustments that the machinists make in response to shifting constraints.

D.

Limitations o

f the Taxonomy While the taxonomy in Fig. 4 has proven to be a useful tool for classifying and comparing manufacturing grasps, it suffers from a number of limitations. To begin with, it is incomplete. For example, there are numerous everyday grasps, such as the grasp that people use in writing with a pencil or in marking items with a scribe (Figs. 7 and 8) that are not included. It was also found that the machinists in our study adopted numerous variations on the grasps in Fig. 4, partly in response to particular task or geometry constraints and partly due to personal preferences and differences in the size and strength of their hands. Such individual grasps could usually be identified as "children" of the grasps in Fig. 4. To examine such issues further, and to clarify the roles of dexterity, sensitivity and stability in grasp choice, an expert system was constructed for choosing grasps from initial information about the task requirements and object shape.

• M. Cutkosky and P

. Wright. Modeling manufacturing grips and correlations with the design of robotic hands. ICRA 1986.

SLIDE 24

Can a simple hand 
 change grasps?

• It seems to require a lot of dexterity!
• The “dexterous hand” approach to in-hand

manipulation: at least 9 motors

• Can a simple effector do it? Of course!

SLIDE 25

p in Hand d T h r

• w

t

• F

i n g e r t i p T h r

• w

t

• P

a l m T h r

• w

t

• P

a l m T h r

• w

t

• F

i n g e r t i p Roll to Palm T h r

• w

t

• F

i n g e r t i p Drop Throw and Flip P i c k P i c k D r

• p

D r

• p

P i c k

Hand Drop in

P l a c e P i c k Place and Pick D r

• p

P l a c e P l a c e Topple R

• l

l t

• G

r

• u

n d P l a c e P l a c e T h r

• w

t

• P

a a l m m T h r

• w

t

• F

i n g e r t i p T p Pick P l a c e Topple

SLIDE 26

`

SLIDE 27

References, page 1

• 1. Akella, Srinivas, Wesley H. Huang, Kevin
• M. Lynch, and Matthew T. Mason. “Parts

Feeding on a Conveyor with a One Joint Robot.” Algorithmica 26 (2000): 313-344.

• 2. Bicchi, A., and P. Dario. “Intrinsic tactile

sensing for artificial hands.” In Proc. 4th

• Int. Symp. on Robotics Research.1987.
• 3. Chavan Dafle, Nikhil, A. Rodriguez, R.

Paolini, Bowei Tang, S.S. Srinivasa, M. Erdmann, M.T. Mason, I. Lundberg, H. Staab, and T. Fuhlbrigge. “Extrinsic dexterity: In-hand manipulation with external forces.” IEEE International Conference on Robotics and Automation (ICRA), 2014,

• 4. Christiansen, Alan D., Matthew T. Mason,

and Tom M. Mitchell. “Learning Reliable Manipulation Strategies Without Initial Physical Models.” Robotics and Autonomous Systems 8 (1991): 7-18.

• 5. Cutkosky, M., and P. Wright. “Modeling

manufacturing grips and correlations with the design of robotic hands.” In Robotics and Automation. Proceedings. 1986 IEEE International Conference on.

• 6. Erdmann, MA, and MT Mason. “An

exploration of sensorless manipulation.” IEEE Journal of Robotics and Automation 4, no. 4 (1988): 369-379.

• 7. Goldberg, Kennneth, and Matthew T.
• Mason. “Bayesian grasping.” IEEE

International Conference on Robotics and Automation (ICRA) (1990): 1264-1269.

• 8. Lynch, Kevin M., and Matthew T. Mason.

“Dynamic nonprehensile manipulation: Controllability, Planning and Experiments.” International Journal of Robotics Research 18, no. 1 (January 1999): 64-92.

SLIDE 28

References, page 2

• 9. Lynch, Kevin M., and Matthew T. Mason.

“Stable Pushing: Mechanics, Controllability, and Planning.” International Journal of Robotics Research 15, no. 6 (December 1996): 533-556. 10.Mason, Matthew T. “Mechanics and Planning of Manipulator Pushing Operations.” 5, no. 3 (Fall 1986): 53-71. 11.Mason, Matthew T., Dinesh K. Pai, Daniela Rus, Lee R. Taylor, and Michael A.

• Erdmann. “A Mobile Manipulator.” In

International Conference on Robotics and Automation, 2322-2327. IEEE, 1999. 12.Mason, Matthew T, Alberto Rodriguez, Siddhartha S Srinivasa, and Andres S

• Vazquez. “Autonomous manipulation with a

general-purpose simple hand.” The International Journal of Robotics Research 31, no. 5 (April 2012): 688-703. 13.Mason, M.T., and J.K. Salisbury Jr. Robot Hands and the Mechanics of Manipulation. Cambridge MA: MIT Press, 1985. 14.Paolini, Robert, Alberto Rodriguez, Siddhartha S. Srinivasa, and Matthew T.

• Mason. “A data-driven statistical framework

for post-grasp manipulation.” The International Journal of Robotics Research 33, no. 4 (2014): 600-615. 15.Rodriguez, Alberto, Matthew T. Mason, Siddhartha S. Srinivasa, Matthew Bernstein, and Alex Zirbel. “Abort and Retry in Grasping.” In IEEE International Conference

• n Intelligent Robots and Systems (IROS).

2011. 16.Zeglin, Garth, Alberto Rodriguez, and Matthew T. Mason. “A Simple and Compliant Force Sensing Palm for the MLab Simple Hand.” In 2013 IEEE International Conference on Robotics and Automation (ICRA), 2359-2365. Karlsruhe, Germany, May 2013.

SLIDE 29

das Ende