Bimanual Regrasping Introduction from Unimanual Machine Learning - - PowerPoint PPT Presentation

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Bimanual Regrasping from Unimanual Machine Learning

Benjamin Balaguer and Stefano Carpin University of California, Merced

IEEE International Conference on Robotics and Automation Grasping and Manipulation Session (WeD02)

May 16, 2012

1 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Regrasping An Overview

Regrasping Problem

Modify an object’s configuration

when limits/constraints are reached

Regrasping in the robotics literature

In-hand Regrasping

Depends on dexterous end-effector

On-surface Regrasping

Slow and non-human-like

Proposed solution: bimanual regrasping

Requires dual manipulators Human inspired Focus on speed and efficiency

2 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Regrasping An Overview

Regrasping Problem

Modify an object’s configuration

when limits/constraints are reached

Regrasping in the robotics literature

In-hand Regrasping

Depends on dexterous end-effector

On-surface Regrasping

Slow and non-human-like

Proposed solution: bimanual regrasping

Requires dual manipulators Human inspired Focus on speed and efficiency

2 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Regrasping An Overview

Regrasping Problem

Modify an object’s configuration

when limits/constraints are reached

Regrasping in the robotics literature

In-hand Regrasping

Depends on dexterous end-effector

On-surface Regrasping

Slow and non-human-like

Proposed solution: bimanual regrasping

Requires dual manipulators Human inspired Focus on speed and efficiency

2 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Algorithm Overview

Problem Definition: Given an object only reachable by the right manipulator, move it to an area only reachable by the left manipulator

3 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Algorithm Overview

Problem Definition: Given an object only reachable by the right manipulator, move it to an area only reachable by the left manipulator Problem Solution: Minimize execution time

Cast as an optimization problem

3 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Algorithm Overview

Problem Definition: Given an object only reachable by the right manipulator, move it to an area only reachable by the left manipulator Problem Solution: Minimize execution time

Cast as an optimization problem

3 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Algorithm Overview

Problem Definition: Given an object only reachable by the right manipulator, move it to an area only reachable by the left manipulator Problem Solution: Minimize execution time

Cast as an optimization problem

3 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Algorithm Overview

Problem Definition: Given an object only reachable by the right manipulator, move it to an area only reachable by the left manipulator Problem Solution: Minimize execution time

Cast as an optimization problem

3 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Algorithm Overview

Problem Definition: Given an object only reachable by the right manipulator, move it to an area only reachable by the left manipulator Problem Solution: Minimize execution time

Cast as an optimization problem

3 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Reusing Unimanual Grasper Image Processing [Saxena et al. 2008]

Start from object-extracted image-space Apply Canny edge detector Apply [Saxena et al. 2008], keeping ∀pi, P(zi = 1) > 0.9 Use heuristic to choose two good grasping points

4 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Reusing Unimanual Grasper Image Processing [Saxena et al. 2008]

Start from object-extracted image-space Apply Canny edge detector Apply [Saxena et al. 2008], keeping ∀pi, P(zi = 1) > 0.9 Use heuristic to choose two good grasping points

4 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Reusing Unimanual Grasper Image Processing [Saxena et al. 2008]

Start from object-extracted image-space Apply Canny edge detector Apply [Saxena et al. 2008], keeping ∀pi, P(zi = 1) > 0.9 Use heuristic to choose two good grasping points

4 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Reusing Unimanual Grasper Image Processing [Saxena et al. 2008]

Start from object-extracted image-space Apply Canny edge detector Apply [Saxena et al. 2008], keeping ∀pi, P(zi = 1) > 0.9 Use heuristic to choose two good grasping points arg maxi,j |P(zi=1)+P(zj=1)|

2

pi − pj

• ∀i, j ∈ R

Constraint: PG

i (z),PG j (z) ≥ 5cm

4 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Reusing Unimanual Grasper Grasp Synthesis [Balaguer et al. 2010]

Classification Orientation Estimation Nearest Neighbor Search Training Data

Point Cloud Image Right Point Left Point

Image Processing

5 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Moving the Object to a Regrasping Configuration Mathematical Derivation

Assume regrasping is given by PG

Ropt and RG Ropt

V G = (RG

Ropt)[(RG Rini)T(PG Lini − PG Rini)]

PG

Lopt = PG Ropt + V G

RG

Lopt = (RG Ropt)[(RG Rini)T(RG Lini)]

6 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Moving the Object to a Regrasping Configuration Mathematical Derivation

Assume regrasping is given by PG

Ropt and RG Ropt

V G = (RG

Ropt)[(RG Rini)T(PG Lini − PG Rini)]

PG

Lopt = PG Ropt + V G

RG

Lopt = (RG Ropt)[(RG Rini)T(RG Lini)]

6 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Moving the Object to a Regrasping Configuration Mathematical Derivation

Assume regrasping is given by PG

Ropt and RG Ropt

V G = (RG

Ropt)[(RG Rini)T(PG Lini − PG Rini)]

PG

Lopt = PG Ropt + V G

RG

Lopt = (RG Ropt)[(RG Rini)T(RG Lini)]

6 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Moving the Object to a Regrasping Configuration Mathematical Derivation

Assume regrasping is given by PG

Ropt and RG Ropt

V G = (RG

Ropt)[(RG Rini)T(PG Lini − PG Rini)]

PG

Lopt = PG Ropt + V G

RG

Lopt = (RG Ropt)[(RG Rini)T(RG Lini)]

6 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Moving the Object to a Regrasping Configuration Mathematical Derivation

Assume regrasping is given by PG

Ropt and RG Ropt

V G = (RG

Ropt)[(RG Rini)T(PG Lini − PG Rini)]

PG

Lopt = PG Ropt + V G

RG

Lopt = (RG Ropt)[(RG Rini)T(RG Lini)]

6 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Moving the Object to a Regrasping Configuration Mathematical Derivation

Assume regrasping is given by PG

Ropt and RG Ropt

V G = (RG

Ropt)[(RG Rini)T(PG Lini − PG Rini)]

PG

Lopt = PG Ropt + V G

RG

Lopt = (RG Ropt)[(RG Rini)T(RG Lini)]

6 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Moving the Object to a Regrasping Configuration Mathematical Derivation

Assume regrasping is given by PG

Ropt and RG Ropt

V G = (RG

Ropt)[(RG Rini)T(PG Lini − PG Rini)]

PG

Lopt = PG Ropt + V G

RG

Lopt = (RG Ropt)[(RG Rini)T(RG Lini)]

6 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Optimization Algorithm

Searching the 6D space of RG

Ropt and PG Ropt

Nelder-Mead Optimization Algorithm

Exploits a simplex (no need for derivatives) Each vertex represents a potential solution Solutions are rated with a cost function

f (x) = max(|qRini − qRopt|) + max(|qLstr − qLopt|)

Four geometrical operations minimize vertices’ cost

Reflection, Expansion, Contraction, Deflation

7 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Optimization Algorithm

Searching the 6D space of RG

Ropt and PG Ropt

Nelder-Mead Optimization Algorithm

Exploits a simplex (no need for derivatives) Each vertex represents a potential solution Solutions are rated with a cost function

f (x) = max(|qRini − qRopt|) + max(|qLstr − qLopt|)

Four geometrical operations minimize vertices’ cost

Reflection, Expansion, Contraction, Deflation

7 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Optimization Algorithm

Searching the 6D space of RG

Ropt and PG Ropt

Nelder-Mead Optimization Algorithm

Exploits a simplex (no need for derivatives) Each vertex represents a potential solution Solutions are rated with a cost function

f (x) = max(|qRini − qRopt|) + max(|qLstr − qLopt|)

Four geometrical operations minimize vertices’ cost

Reflection, Expansion, Contraction, Deflation

7 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Algorithm Comparison

8 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Algorithm Speed

Component Part Time(ms) Image Processing Acquisition 40 Denoising 42 Pixel Selection 130 Grasp Synthesis Classification 80 Orientation Estimation 21 Nearest Neighbor 16 Optimization

• 366

Total 695

9 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

End-to-End Scenario Real Robot Examples

10 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Conclusions

Bimanual regrasping algorithm Exploit unimanual grasping algorithm Optimization framework

Most efficient Best solutions, on average

Future Work

Optimize final object’s rotation Consider tasks whith multiple regrasping phases

11 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Conclusions

Bimanual regrasping algorithm Exploit unimanual grasping algorithm Optimization framework

Most efficient Best solutions, on average

Future Work

Optimize final object’s rotation Consider tasks whith multiple regrasping phases

11 / 12

Bimanual Regrasping Balaguer et al. Introduction Algorithm

Overview Vision Grasp Synthesis Optimization

Experiments

Comparison Efficiency Examples

Conclusion

Thank You... Any Questions?

12 / 12