Skip to main content
SpringerLink
Log in

Robotic Grasp Synthesis Using Deep Learning Approaches: A Survey

  • Conference paper
  • First Online:
Mathematical Modeling, Computational Intelligence Techniques and Renewable Energy

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1287))

Abstract

Deep learning methods are derived and inspired from the structure and activities of a brain. In an intricate state, learning from the past experiences helps human to accomplish the task in efficient way. This paper addresses such deep learning approaches practiced in the robotic grasp detection for an unknown object. A grasp is defined as lifting of an object by the end effector of robotic manipulator without tumbling. To realize this task, a robot must be able to identify the grasp area of the target object autonomously by means of visual or touch feedback. During the last decade, there is an enormous improvements in hardware computational capabilities, and hence, deep learning methods play a vital role in making robots autonomous. Review of such deep learning methods, used in robotic grasping, is carried out in this work. Deep learning is a subset of machine learning with more neurons then artificial neural networks. Primarily four types of network architectures in deep learning are identified and discussed in this work. They are unsupervised pre-trained network, convolution neural network, recurrent neural network and recursive neural network architectures. Types and availability of general training datasets are discussed wherein deep learning models can be trained and benchmarked. Unavailability of extensive training datasets for the specific task application is a major hurdle in implementing deep learning methods. Performance of these deep learning methods is also debated in this work. The potential research directions are thus proposed in the paper.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Billard, A., Calinon, S., Dillmann, R.: Learning from humans. In: Springer Handbook of Robotics, 2nd edn. (2016)

    Google Scholar 

  2. Zhou, X., Lan, X., Zhang, H., Tian, Z., Zhang, Y., Zheng, N.: Fully Convolutional Grasp Detection Network with Oriented Anchor Box. IEEE Conf. Intell. Robot. Syst. (2018)

    Google Scholar 

  3. Murali, A. et al.: Learning to Grasp Without Seeing. arXiv:abs/1805.0 (2018)

  4. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015)

    Article  Google Scholar 

  5. Ruiz-del-Solar, J., Loncomilla, P., Soto, N.: A Survey on Deep Learning Methods for Robot Vision, pp. 1–43 (2018). arXiv:abs/1803.1

    Google Scholar 

  6. Lenz, I., Lee, H., Saxena, A.: Deep learning for detecting robotic grasps. In: 1st International Conference on Learning Representations, ICLR 2013—Work. Track Proc. (2013)

    Google Scholar 

  7. Caldera, S., Rassau, A., Chai, D.: Review of deep learning methods in robotic grasp detection. Multimodal Technol. Interact. 2, 57 (2018)

    Article  Google Scholar 

  8. Kumra, S., Kanan, C.: Robotic grasp detection using deep convolutional neural networks. IEEE Int. Conf. Intell. Robot. Syst. 769–776 (2017)

    Google Scholar 

  9. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT press (2016)

    Google Scholar 

  10. Redmon, J., Angelova, A.: Real-time grasp detection using convolutional neural networks. Proceedings of IEEE International Conference on Robotics and Automation, June 2015, pp. 1316–1322 (2015)

    Google Scholar 

  11. Saxena, A., Driemeyer, J., Kearns, J., Ng, A.Y.: Robotic grasping of novel objects. Adv. Neural Inf. Process. Syst. 1209–1216 (2007)

    Google Scholar 

  12. Saxena, A., Driemeyer, J., Ng, A.Y.: Robotic grasping of novel objects using vision. Int. J. Rob. Res. 27, 157–173 (2008). https://doi.org/10.1177/0278364907087172

    Article  Google Scholar 

  13. Zhang, F., Leitner, J., Milford, M., Upcroft, B., Corke, P.: Towards vision-based deep reinforcement learning for robotic motion control. In: ACRA (2015)

    Google Scholar 

  14. Jiang, Y., Moseson, S., Saxena, A.: Efficient grasping from RGBD images: Learning using a new rectangle representation. In: Proceedings of IEEE International Conference on Robotics and Automation (2011)

    Google Scholar 

  15. Park, D., Chun, S.Y.: Classification based Grasp detection using spatial transformer network. arXiv:abs/1803.0 (2018)

    Google Scholar 

  16. Trottier, L., Giguere, P., Chaib-Draa, B.: Sparse dictionary learning for identifying grasp locations. In: Proceedings of 2017 IEEE Winter Conference on Applications of Computer Vision (2017)

    Google Scholar 

  17. Wang, Z., Li, Z., Wang, B., Liu, H.: Robot grasp detection using multimodal deep convolutional neural networks. Adv. Mech. Eng. 8, 1–12 (2016)

    Google Scholar 

  18. Pinto, L., Gupta, A.: Supersizing self-supervision: Learning to grasp from 50 K tries and 700 robot hours. In: Proceedings of IEEE International Conference on Robotics and Automation, June 2016, pp. 3406–3413 (2016)

    Google Scholar 

  19. Mahler, J., Goldberg, K.: Learning deep policies for robot bin picking by simulating robust grasping sequences. In: Proceedings of 1st Annual Conference on Robot Learning, vol 78, pp. 515–524 (2017)

    Google Scholar 

  20. Viereck, U., Pas, A. ten, Saenko, K., Platt, R.: Learning a visuomotor controller for real world robotic grasping using simulated depth images, pp. 1–10 (2017). arXiv:abs/1706.0

    Google Scholar 

  21. Calandra, R., et al.: More than a feeling: learning to grasp and regrasp using vision and touch. IEEE Robot. Autom. Lett. 3, 3300–3307 (2018)

    Article  Google Scholar 

  22. Bicchi, A., Kumar, V.: Robotic grasping and contact: a review. In: Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings vol 1, pp 348–353 (2000)

    Google Scholar 

  23. Bohg, J., Morales, A., Asfour, T., Kragic, D.: Data-driven grasp synthesis—a survey. IEEE Trans. Robot. 30, 289–309 (2014). https://doi.org/10.1109/TRO.2013.2289018

    Article  Google Scholar 

  24. Chu, F.J., Xu, R., Vela, P.A.: Real-world multiobject, multigrasp detection. IEEE Robot. Autom. Lett. 3, 3355–3362 (2018). https://doi.org/10.1109/LRA.2018.2852777

    Article  Google Scholar 

  25. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, December 2016

    Google Scholar 

  26. Guo, D., Sun, F., Liu, H., Kong, T., Fang, B., Xi, N.: A hybrid deep architecture for robotic grasp detection. In: Proceedings of IEEE International Conference on Robotics and Automation, pp. 1609–1614 (2017)

    Google Scholar 

  27. Ten Pas, A., Gualtieri, M., Saenko, K., Platt, R.: Grasp pose detection in point clouds. Int. J. Rob. Res. 36, 1455–1473 (2017). https://doi.org/10.1177/0278364917735594

    Article  Google Scholar 

  28. Lu, Q., Chenna, K., Sundaralingam, B., Hermans, T.: Planning multi-fingered grasps as probabilistic inference in a learned deep network, pp. 1–16 (2018). arXiv:abs/1804.0

    Google Scholar 

  29. Johns, E., Leutenegger, S., Davison, A.J.: Deep learning a grasp function for grasping under gripper pose uncertainty. IEEE Int. Conf. Intell. Robot. Syst. (2016)

    Google Scholar 

  30. Mahler, J. et al.: Dex-Net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics. Robot. Sci. Syst. 13 (2017)

    Google Scholar 

  31. Zeng, A. et al.: Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching. Int. J. Rob. Res. (2019)

    Google Scholar 

  32. Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)

    Article  Google Scholar 

  33. Goodfellow, I. et al.: Measuring invariances in deep networks. In: Advances in neural information processing systems, vol. 22. pp. 646–654. Curran Associates, Inc. (2009)

    Google Scholar 

  34. Tobin, J. et al.: Domain randomization for transferring deep neural networks from simulation to the real world. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 23–30 (2017)

    Google Scholar 

  35. CornellUniversity: Robot learning lab: Learning to grasp, http://pr.cs.cornell.edu/grasping/rect_data/data.php. Accessed on 17/09/2019

  36. Joe Watson, J.H., Iida, F.: Real-world, real-time robotic grasping with convolution neural network. In: Conference Towards Autonomous Robotic Systems, pp. 627–636 (2017)

    Google Scholar 

  37. Quigley, M., Conley, K., Gerkey, B.P., Faust, J., Foote, T., Leibs, J., Wheeler, R.C., Ng, A.Y.: ROS: an open-source robot operating system. In: ICRA 2009 (2009)

    Google Scholar 

  38. Bradski, G.R., Kaehler, A.: Learning OpenCV—computer vision with the OpenCV library: software that sees. O’Reilly Media, Inc. (2008)

    Google Scholar 

  39. Park, D., Seo, Y., Chun, S.Y.: Real-time, highly accurate robotic grasp detection using fully convolutional neural network with rotation ensemble module (2019)

    Google Scholar 

  40. Washington-Dataset, web:rgbd-dataset.cs.washington.edu. Accessed on 19/09/2019

    Google Scholar 

  41. Levine, S., et al.: Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. Int. J. Rob. Res. 37, 421–436 (2018)

    Article  Google Scholar 

  42. Bousmalis, K. et al.: Using simulation and domain adaptation to improve efficiency of deep robotic grasping. In: Proceedings of IEEE International Conference on Robotics and Automation, pp. 4243–4250 (2018)

    Google Scholar 

  43. Russakovsky, O. et al.: ImageNet large scale visual recognition challenge. CoRR. abs/1409.0 (2014)

    Google Scholar 

  44. Szegedy, C. et al.: Going deeper with convolutions. CoRR. (2014)

    Google Scholar 

  45. Szegedy, C. et al.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-First AAAI Conference on Artificial Intelligence (2017)

    Google Scholar 

  46. Szegedy, C. et al.: Rethinking the inception architecture for computer vision (2016)

    Google Scholar 

  47. Xia, Y., Wang, J.: A dual neural network for kinematic control of redundant robot manipulators. IEEE Trans. Syst. Man, Cybern. Part B. 31, 147–154 (2001)

    Google Scholar 

  48. Polydoros, A.S. et al.: Real-time deep learning of robotic manipulator inverse dynamics. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (2015)

    Google Scholar 

  49. Gonzalez, T.F.: Handbook of approximation algorithms and metaheuristics. In: Handbook of Approximation Algorithms and Metaheuristics, pp. 1–1432 (2007)

    Google Scholar 

  50. Krizhevsky, A., Sutskever, I., Hinton, G.: Imagenet classification with deep convolutional neural. In: Neural Information Processing Systems. pp. 1–9 (2014)

    Google Scholar 

  51. Ruder, S.: An overview of gradient descent optimization algorithms (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Technology, Pandit Deendayal Petroleum University, Gandhinagar, India

    Dhaval R. Vyas & Anilkumar Markana

  2. Indian Institute of Technology, Gandhinagar, India

    Nitin Padhiyar

Authors
  1. Dhaval R. Vyas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anilkumar Markana
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nitin Padhiyar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dhaval R. Vyas .

Editor information

Editors and Affiliations

  1. Department of Mathematics, School of Technology, Pandit Deendayal Petroleum University, Gandhinagar, India

    Manoj Sahni

  2. University of Technology Sydney, Sydney, NSW, Australia

    José M. Merigó

  3. Department of Mathematics, School of Technology, Pandit Deendayal Petroleum University, Gandhinagar, India

    Brajesh Kumar Jha

  4. Department of Management Control and Information Systems, University of Chile, Santiago, Chile

    Rajkumar Verma

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Vyas, D.R., Markana, A., Padhiyar, N. (2021). Robotic Grasp Synthesis Using Deep Learning Approaches: A Survey. In: Sahni, M., Merigó, J.M., Jha, B.K., Verma, R. (eds) Mathematical Modeling, Computational Intelligence Techniques and Renewable Energy. Advances in Intelligent Systems and Computing, vol 1287. Springer, Singapore. https://doi.org/10.1007/978-981-15-9953-8_11

Download citation

Publish with us

Policies and ethics

Search

Navigation