Summary
We consider the problem of grasping novel objects, specifically, ones that are being seen for the first time through vision. We present a learning algorithm which predicts, as a function of the images, the position at which to grasp the object. This is done without building or requiring a 3-d model of the object. Our algorithm is trained via supervised learning, using synthetic images for the training set. Using our robotic arm, we successfully demonstrate this approach by grasping a variety of differently shaped objects, such as duct tape, markers, mugs, pens, wine glasses, knife-cutters, jugs, keys, toothbrushes, books, and others, including many object types not seen in the training set.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bicchi, A., Kumar, V.: Robotic grasping and contact: a review. In: ICRA (2000)
Bower, T., Broughton, J., Moore, M.: Demonstration of intention in the reaching behaviour of neonate humans. Nature 228, 679–681 (1970)
Castiello, U.: The neuroscience of grasping. Nature Reviews Neuroscience 6 (2005)
Glassner, A.S.: An Introduction to Ray Tracing. Morgan Kaufmann Publishers, Inc., San Francisco (1989)
Mason, M.T., Salisbury, J.K.: Manipulator grasping and pushing operations. In: Robot Hands and the Mechanics of Manipulation, The MIT Press, Cambridge, MA (1985)
Michels, J., Saxena, A., Ng, A.Y.: High speed obstacle avoidance using monocular vision and reinforcement learning. In: ICML (2005)
Miller, A., Knoop, S., Christensen, H., Allen, P.: Automatic grasp planning using shape primitives. In: ICRA (2003)
Neuronics: Katana user manual (2004), http://www.neuronics.ch/
Pelossof, R., Miller, A., Allen, P., Jebara, T.: An svm learning approach to robotic grasping. In: ICRA (2004)
Piater, J.H.: Learning visual features to predict hand orientations. In: ICML Workshop on Machine Learning of Spatial Knowledge (2000)
Platt, R., Fagg, A.H., Grupen, R.: Manipulation gaits: Sequences of grasp control tasks. In: ICRA (2004)
Platt, R., Fagg, A.H., Grupen, R.: Reusing schematic grasping policies. In: IEEE-RAS International Conference on Humanoid Robots, Tsukuba, Japan (2005)
Saxena, A., Chung, S.H., Ng, A.Y.: Learning depth from single monocular images. In: NIPS 18 (2005)
Saxena, A., Driemeyer, J., Kearns, J., Ng, A.Y.: Robotic grasping of novel objects. In: NIPS (2006)
Shin-ichi, T., Satoshi, M.: Living and working with robots. Nipponia (2000)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Saxena, A., Driemeyer, J., Kearns, J., Osondu, C., Ng, A.Y. (2008). Learning to Grasp Novel Objects Using Vision. In: Khatib, O., Kumar, V., Rus, D. (eds) Experimental Robotics. Springer Tracts in Advanced Robotics, vol 39. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-77457-0_4
Download citation
DOI: https://doi.org/10.1007/978-3-540-77457-0_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-77456-3
Online ISBN: 978-3-540-77457-0
eBook Packages: EngineeringEngineering (R0)