Model-Free Segmentation and Grasp Selection of Unknown Stacked Objects

  • Umar Asif
  • Mohammed Bennamoun
  • Ferdous Sohel
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8693)


We present a novel grasping approach for unknown stacked objects using RGB-D images of highly complex real-world scenes. Specifically, we propose a novel 3D segmentation algorithm to generate an efficient representation of the scene into segmented surfaces (known as surfels) and objects. Based on this representation, we next propose a novel grasp selection algorithm which generates potential grasp hypotheses and automatically selects the most appropriate grasp without requiring any prior information of the objects or the scene. We tested our algorithms in real-world scenarios using live video streams from Kinect and publicly available RGB-D object datasets. Our experimental results show that both our proposed segmentation and grasp selection algorithms consistently perform superior compared to the state-of-the-art methods.


3D segmentation grasp selection 

Supplementary material

978-3-319-10602-1_43_MOESM1_ESM.mp4 (29.7 mb)
Electronic Supplementary Material (MP4 30,416 KB)


  1. 1.
    Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Susstrunk, S.: Slic superpixels compared to state-of-the-art superpixel methods. PAMI 34(11), 2274–2282 (2012)CrossRefGoogle Scholar
  2. 2.
    Asif, U., Bennamoun, M., Sohel, F.: Real-time pose estimation of rigid objects using rgb-d imagery. In: ICIEA, pp. 1692–1699 (2013)Google Scholar
  3. 3.
    Bohg, J., Kragic, D.: Learning grasping points with shape context. Robotics and Autonomous Systems 58(4), 362–377 (2010)CrossRefGoogle Scholar
  4. 4.
    Borst, C., Fischer, M., Hirzinger, G.: Grasp planning: How to choose a suitable task wrench space. In: ICRA, vol. 1, pp. 319–325 (2004)Google Scholar
  5. 5.
    Bowers, D.L., Lumia, R.: Manipulation of unmodeled objects using intelligent grasping schemes. IEEE Transactions on Fuzzy Systems 11(3), 320–330 (2003)CrossRefGoogle Scholar
  6. 6.
    Collet, A., Martinez, M., Srinivasa, S.S.: The moped framework: Object recognition and pose estimation for manipulation. IJRR 30(10), 1284–1306 (2011)Google Scholar
  7. 7.
    Curtis, N., Xiao, J.: Efficient and effective grasping of novel objects through learning and adapting a knowledge base. In: IROS, pp. 2252–2257 (2008)Google Scholar
  8. 8.
    Dogar, M.R., Hsaio, K., Ciocarlie, M., Srinivasa, S.: Physics-based grasp planning through clutter (2012)Google Scholar
  9. 9.
    Ekvall, S., Kragic, D.: Integrating object and grasp recognition for dynamic scene interpretation. In: ICAR, pp. 331–336 (2005)Google Scholar
  10. 10.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. IJCV 59(2), 167–181 (2004)CrossRefGoogle Scholar
  11. 11.
    Fenzi, M., Dragon, R., Leal-Taixé, L., Rosenhahn, B., Ostermann, J.: 3d object recognition and pose estimation for multiple objects using multi-prioritized ransac and model updating. In: Pattern Recognition, pp. 123–133 (2012)Google Scholar
  12. 12.
    Huebner, K., Ruthotto, S., Kragic, D.: Minimum volume bounding box decomposition for shape approximation in robot grasping. In: ICRA, pp. 1628–1633 (2008)Google Scholar
  13. 13.
    Jiang, Y., Moseson, S., Saxena, A.: Efficient grasping from rgbd images: Learning using a new rectangle representation. In: ICRA, pp. 3304–3311 (2011)Google Scholar
  14. 14.
    Kootstra, G., Popović, M., Jørgensen, J.A., Kuklinski, K., Miatliuk, K., Kragic, D., Krüger, N.: Enabling grasping of unknown objects through a synergistic use of edge and surface information. IJRR 31(10), 1190–1213 (2012)Google Scholar
  15. 15.
    Lai, K., Bo, L., Ren, X., Fox, D.: A large-scale hierarchical multi-view rgb-d object dataset. In: ICRA, pp. 1817–1824 (2011)Google Scholar
  16. 16.
    Le, Q.V., Kamm, D., Kara, A.F., Ng, A.Y.: Learning to grasp objects with multiple contact points. In: ICRA, pp. 5062–5069 (2010)Google Scholar
  17. 17.
    Lenz, I., Lee, H., Saxena, A.: Deep learning for detecting robotic grasps. arXiv preprint arXiv:1301.3592 (2013)Google Scholar
  18. 18.
    Li, X., Guskov, I.: 3d object recognition from range images using pyramid matching. In: ICCV. pp. 1–6 (2007)Google Scholar
  19. 19.
    Miller, A.T., Allen, P.K.: Graspit! a versatile simulator for robotic grasping. IEEE Robotics & Automation Magazine 11(4), 110–122 (2004)CrossRefGoogle Scholar
  20. 20.
    Mishra, A.K., Shrivastava, A., Aloimonos, Y.: Segmenting “simple” objects using rgb-d. In: ICRA. pp. 4406–4413 (2012)Google Scholar
  21. 21.
    Morales, A., Sanz, P.J., Del Pobil, A.P., Fagg, A.H.: Vision-based three-finger grasp synthesis constrained by hand geometry. Robotics and Autonomous Systems 54(6), 496–512 (2006)CrossRefGoogle Scholar
  22. 22.
    Pelossof, R., Miller, A., Allen, P., Jebara, T.: An svm learning approach to robotic grasping. In: ICRA, vol. 4, pp. 3512–3518 (2004)Google Scholar
  23. 23.
    Platt, R.: Learning grasp strategies composed of contact relative motions. In: 7th IEEE-RAS International Conference on Humanoid Robots, pp. 49–56 (2007)Google Scholar
  24. 24.
    Rao, D., Le, Q.V., Phoka, T., Quigley, M., Sudsang, A., Ng, A.Y.: Grasping novel objects with depth segmentation. In: IROS, pp. 2578–2585 (2010)Google Scholar
  25. 25.
    Recatalá, G., Chinellato, E., Del Pobil, Á.P., Mezouar, Y., Martinet, P.: Biologically-inspired 3d grasp synthesis based on visual exploration. Autonomous Robots 25(1-2), 59–70 (2008)CrossRefGoogle Scholar
  26. 26.
    Richtsfeld, A., Morwald, T., Prankl, J., Zillich, M., Vincze, M.: Segmentation of unknown objects in indoor environments. In: IROS, pp. 4791–4796 (2012)Google Scholar
  27. 27.
    Richtsfeld, A., Mörwald, T., Prankl, J., Zillich, M., Vincze, M.: Learning of perceptual grouping for object segmentation on rgb-d data. Journal of Visual Communication and Image Representation 25(1), 64–73 (2014)CrossRefGoogle Scholar
  28. 28.
    Richtsfeld, M., Vincze, M., et al.: Grasping of unknown objects from a table top. In: Workshop on Vision in Action: Efficient Strategies for Cognitive Agents in Complex Environments (2008)Google Scholar
  29. 29.
    Rosales, C., Porta, J.M., Ros, L.: Global optimization of robotic grasps. In: Proceedings of Robotics: Science and Systems VII (2011)Google Scholar
  30. 30.
    Rusu, R.B., Bradski, G., Thibaux, R., Hsu, J.: Fast 3d recognition and pose using the viewpoint feature histogram. In: IROS, pp. 2155–2162 (2010)Google Scholar
  31. 31.
    Saxena, A., Driemeyer, J., Kearns, J., Osondu, C., Ng, A.Y.: Learning to grasp novel objects using vision. In: ISER (2006)Google Scholar
  32. 32.
    Saxena, A., Driemeyer, J., Ng, A.Y.: Robotic grasping of novel objects using vision. IJRR 27(2), 157–173 (2008)Google Scholar
  33. 33.
    Shapiro, L., Stockman, G.C.: Computer vision. In: Computer Vision 2001, pp. 69–75. Prentice Hall (2001)Google Scholar
  34. 34.
    Sun, M., Bradski, G., Xu, B.-X., Savarese, S.: Depth-encoded hough voting for joint object detection and shape recovery. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 658–671. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  35. 35.
    Tegin, J., Ekvall, S., Kragic, D., Wikander, J., Iliev, B.: Demonstration-based learning and control for automatic grasping. Intelligent Service Robotics 2(1), 23–30 (2009)CrossRefGoogle Scholar
  36. 36.
    Vedaldi, A., Soatto, S.: Quick shift and kernel methods for mode seeking. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 705–718. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  37. 37.
    Veksler, O., Boykov, Y., Mehrani, P.: Superpixels and supervoxels in an energy optimization framework. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 211–224. Springer, Heidelberg (2010)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Umar Asif
    • 1
  • Mohammed Bennamoun
    • 1
  • Ferdous Sohel
    • 1
  1. 1.School of Computer Science & Software EngineeringThe University of Western Australia, CrawleyPerthAustralia

Personalised recommendations