Data-driven grasping


This paper propose a novel framework for a data driven grasp planner that indexes partial sensor data into a database of 3D models with known grasps and transfers grasps from those models to novel objects. We show how to construct such a database and also demonstrate multiple methods for matching into it, aligning the matched models with the known sensor data of the object to be grasped, and selecting an appropriate grasp to use. Our approach is experimentally validated in both simulated trials and trials with robots.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 99

This is the net price. Taxes to be calculated in checkout.


  1. Aleotti, J., & Caselli, S. (2007). Robot grasp synthesis from virtual demonstration and topology-preserving environment reconstruction. In IEEE international conference on intelligent robots and systems.

  2. Aydin, Y., & Nakajima, M. (1999). Database guided computer animation of human grasping using forward and inverse kinematics. Computers & Graphics, 23(1).

  3. Balasubramanian, R., Xu, L., Brook, P. D., Smith, J. R., & Matsuoka, Y. (2010). Human-guided grasp measures improve grasp robustness on physical robot. In IEEE international conference on robotics and automation.

  4. Belongie, S., & Malik, J. (2000). Matching with shape contexts. In Workshop on content-based access of image and video libraries.

  5. Biederman, I. (1995). Visual object recognition. In S. F. Kosslyn & D. N. Osherson (Eds.), An invitation to cognitive science: Vol. 2 (2nd ed.). Chap. 4.

  6. Bohg, J., & Kragic, D. (2010). Learning grasping points with shape context. Robotics and Autonomous Systems, 58(4).

  7. Bowers, D. L., & Lumia, R. (2003). Manipulation of unmodeled objects using intelligent grasping schemes. IEEE Transactions on Fuzzy Systems, 11(3).

  8. Chen, D.-Y., Ouhyoung, M., Tian, X. P., & Shen, Y. T. (2003). On visual similarity based 3d model retrieval. In Eurographics.

  9. Ciocarlie, M., & Allen, P. K. (2009). Hand posture subspaces for dexterous robotic grasping. The International Journal of Robotics Research, 28, 851–867.

  10. Ciocarlie, M., Goldfeder, C., & Allen, P. K. (2007a). Dimensionality reduction for hand-independent dexterous robotic grasping. In IEEE international conference on intelligent robots and systems.

  11. Ciocarlie, M., Lackner, C., & Allen, P. K. (2007b). Soft finger model with adaptive contact geometry for grasping and manipulation tasks. In IEEE world haptics.

  12. Dueck, G., & Scheuer, T. (1990). Threshold accepting: a general purpose optimization algorithm appearing superior to simulated annealing. Journal of Computational Physics, 90(1).

  13. Ferrari, C., & Canny, J. (2002). Planning optimal grasps. In IEEE international conference on robotics and automation.

  14. Gelfand, N., Mitra, N. J., Guibas, L. J., & Pottmann, H. (2005). Robust global registration. In Symposium on geometry processing.

  15. Glover, J., Rus, D., & Roy, N. (2008). Probabilistic models of object geometry for grasp planning. In Robotics: science and systems.

  16. Goldfeder, C., & Allen, P. K. (2008). Autotagging to improve text search for 3d models. In Joint conference on digital libraries.

  17. Goldfeder, C., Ciocarlie, M., Dang, H., & Allen, P. K. (2009a). The Columbia grasp database. In IEEE international conference on robotics and automation.

  18. Goldfeder, C., Ciocarlie, M., Peretzman, J., Dang, H., & Allen, P. K. (2009b). Data-driven grasping with partial sensor data. In IEEE international conference on intelligent robots and systems.

  19. Huber, D. F., & Hebert, M. (2003). Fully automatic registration of multiple 3d data sets. Image and Vision Computing, 21(7).

  20. Jaccard, P. (1901). Étude comparative de la distribution florale dans une portion des alpes et des jura. Bulletin del la Société Vaudoise des Sciences Naturelles, 37.

  21. Johnson, A. E., & Hebert, M. (1997). Surface registration by matching oriented points. In 3-D digital imaging and modeling.

  22. Kullback, S., & Leibler, R. A. (1951). On information and sufficiency. Annals of Mathematical Statistics, 22.

  23. Landy, M. S., & Graham, N. (2004). Visual perception of texture. In L. M. Chalupa & J. S. Werner (Eds.), The visual neurosciences. Chap. 2.

  24. Leung, T., & Malik, J. (2001). Representing and recognizing the visual appearance of materials using three-dimensional textons. International Journal of Computer Vision, 43(1).

  25. Li, Y., & Pollard, N. S. (2005). A shape matching algorithm for synthesizing humanlike enveloping grasps. In Humanoid robots.

  26. Liu, T., Moore, A., Gray, A., & Yang, K. (2004). An investigation of practical approximate nearest neighbor algorithms. In Conference on neural information processing systems.

  27. Lowe, D. G. (1999). Object recognition from local scale-invariant features. In IEEE international conference on computer vision.

  28. Makadia, A., Patterson, A., & Daniilidis, K. (2006). Fully automatic registration of 3D point clouds. In IEEE conference on computer vision and pattern recognition.

  29. Miller, A., & Allen, P. K. (2004). Graspit!: a versatile simulator for robotic grasping. IEEE Robotics and Automation Magazine, 11(4).

  30. Morales, A., Asfour, T., Azad, P., Knoop, S., & Dillmann, R. (2006). Integrated grasp planning and visual object localization for a humanoid robot with five-fingered hands. In IEEE international conference on intelligent robots and systems.

  31. Novotni, M., & Klein, R. (2003). 3D Zernike descriptors for content based shape retrieval. In ACM symposium on solid modeling and applications.

  32. Nowak, E., Jurie, F., & Triggs, B. (2006). Sampling strategies for bag-of-features image classification. In European conference on computer vision.

  33. Ohbuchi, R., Osada, K., Furuya, T., & Banno, T. (2008). Salient local visual featuers for shape-based 3D model retrieval. In IEEE international conference on shape modeling and applications.

  34. Papadakis, P., Pratikakis, I., Theoharis, T., & Perantonis, S. (2010). PANORAMA: a 3D shape descriptor based on panoramic views for unsupervised 3d object retrieval. International Journal of Computer Vision, Special Issue on: 3D Object Retrieval, 89(2).

  35. Pelossof, R., Miller, A., Allen, P. K., & Jebara, T. (2004). An svm learning approach to robotic grasping. In IEEE international conference on robotics and automation.

  36. Platt, R., Fagg, A. H., & Grupen, R. (2002). Nullspace composition of control laws for grasping. In IEEE international conference on robotics and automation.

  37. Romea, A. C., & Srinivasa, S. (2010). Efficient multi-view object recognition and full pose estimation. In IEEE international conference on robotics and automation.

  38. Rother, C., Kolmogorov, V., & Blake, A. (2004). Grabcut: Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics, 23.

  39. Rusinkiewicz, S., & Levoy, M. (2001). Efficient variants of the icp algorithm. In IEEE international conference on 3-D digital imaging and modeling.

  40. Salvi, J., Matabosch, C., Fofi, D., & Forest, J. (2007). A review of recent range image registration methods with accuracy evaluation. Image and Vision Computing, 25(5).

  41. Saxena, A., Driemeyer, J., & Ng, A. Y. (2008). Robotic grasping of novel objects using vision. International Journal of Robotics Research.

  42. Shilane, P., Min, P., Kazhdan, M., & Funkhouser, T. (2004). The Princeton shape benchmark. In IEEE international conference on shape modeling and applications.

  43. Sivamani, R. K., Goodman, J., Gitis, N. V., & Maibach, H. I. (2003). Coefficient of friction: tribological studies in man—an overview. Skin Research and Technology, 9(3).

  44. Srinivasa, S., Ferguson, D., Helfrich, C., Berenson, D., Romea, A. C., Diankov, R., Gallagher, G., Hollinger, G., Kuffner, J., & Vandeweghe, J. M. (2010). HERB: a home exploring robotic butler. Autonomous Robots, 28(1).

  45. Torres, M. M., Romea, A. C., & Srinivasa, S. (2010). Moped: a scalable and low latency object recognition and pose estimation system. In IEEE international conference on robotics and automation.

  46. Triggs, B., Mclauchlan, P. F., Hartley, R. I., & Fitzgibbon, A. W. (2000). Lecture notes in computer science: Vol. 1883. Bundle adjustment—a modern synthesis.

Download references

Author information

Correspondence to Corey Goldfeder.

Additional information

This work was funded in part by NIH BRP grant 1RO1 NS 050256-01A2 and a Google research grant. We would like to thank Siddhartha Srinivasan, Dmitry Berenson and Mehmet Dogar from Intel Pittsburgh Lab for their help in providing access to the HERB system.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

(MP4 35636 KB)

(MP4 35636 KB)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Goldfeder, C., Allen, P.K. Data-driven grasping. Auton Robot 31, 1–20 (2011) doi:10.1007/s10514-011-9228-1

Download citation


  • Grasping
  • Robotics
  • Data-driven