Abstract
Deep learning methods are derived and inspired from the structure and activities of a brain. In an intricate state, learning from the past experiences helps human to accomplish the task in efficient way. This paper addresses such deep learning approaches practiced in the robotic grasp detection for an unknown object. A grasp is defined as lifting of an object by the end effector of robotic manipulator without tumbling. To realize this task, a robot must be able to identify the grasp area of the target object autonomously by means of visual or touch feedback. During the last decade, there is an enormous improvements in hardware computational capabilities, and hence, deep learning methods play a vital role in making robots autonomous. Review of such deep learning methods, used in robotic grasping, is carried out in this work. Deep learning is a subset of machine learning with more neurons then artificial neural networks. Primarily four types of network architectures in deep learning are identified and discussed in this work. They are unsupervised pre-trained network, convolution neural network, recurrent neural network and recursive neural network architectures. Types and availability of general training datasets are discussed wherein deep learning models can be trained and benchmarked. Unavailability of extensive training datasets for the specific task application is a major hurdle in implementing deep learning methods. Performance of these deep learning methods is also debated in this work. The potential research directions are thus proposed in the paper.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Billard, A., Calinon, S., Dillmann, R.: Learning from humans. In: Springer Handbook of Robotics, 2nd edn. (2016)
Zhou, X., Lan, X., Zhang, H., Tian, Z., Zhang, Y., Zheng, N.: Fully Convolutional Grasp Detection Network with Oriented Anchor Box. IEEE Conf. Intell. Robot. Syst. (2018)
Murali, A. et al.: Learning to Grasp Without Seeing. arXiv:abs/1805.0 (2018)
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015)
Ruiz-del-Solar, J., Loncomilla, P., Soto, N.: A Survey on Deep Learning Methods for Robot Vision, pp. 1–43 (2018). arXiv:abs/1803.1
Lenz, I., Lee, H., Saxena, A.: Deep learning for detecting robotic grasps. In: 1st International Conference on Learning Representations, ICLR 2013—Work. Track Proc. (2013)
Caldera, S., Rassau, A., Chai, D.: Review of deep learning methods in robotic grasp detection. Multimodal Technol. Interact. 2, 57 (2018)
Kumra, S., Kanan, C.: Robotic grasp detection using deep convolutional neural networks. IEEE Int. Conf. Intell. Robot. Syst. 769–776 (2017)
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT press (2016)
Redmon, J., Angelova, A.: Real-time grasp detection using convolutional neural networks. Proceedings of IEEE International Conference on Robotics and Automation, June 2015, pp. 1316–1322 (2015)
Saxena, A., Driemeyer, J., Kearns, J., Ng, A.Y.: Robotic grasping of novel objects. Adv. Neural Inf. Process. Syst. 1209–1216 (2007)
Saxena, A., Driemeyer, J., Ng, A.Y.: Robotic grasping of novel objects using vision. Int. J. Rob. Res. 27, 157–173 (2008). https://doi.org/10.1177/0278364907087172
Zhang, F., Leitner, J., Milford, M., Upcroft, B., Corke, P.: Towards vision-based deep reinforcement learning for robotic motion control. In: ACRA (2015)
Jiang, Y., Moseson, S., Saxena, A.: Efficient grasping from RGBD images: Learning using a new rectangle representation. In: Proceedings of IEEE International Conference on Robotics and Automation (2011)
Park, D., Chun, S.Y.: Classification based Grasp detection using spatial transformer network. arXiv:abs/1803.0 (2018)
Trottier, L., Giguere, P., Chaib-Draa, B.: Sparse dictionary learning for identifying grasp locations. In: Proceedings of 2017 IEEE Winter Conference on Applications of Computer Vision (2017)
Wang, Z., Li, Z., Wang, B., Liu, H.: Robot grasp detection using multimodal deep convolutional neural networks. Adv. Mech. Eng. 8, 1–12 (2016)
Pinto, L., Gupta, A.: Supersizing self-supervision: Learning to grasp from 50 K tries and 700 robot hours. In: Proceedings of IEEE International Conference on Robotics and Automation, June 2016, pp. 3406–3413 (2016)
Mahler, J., Goldberg, K.: Learning deep policies for robot bin picking by simulating robust grasping sequences. In: Proceedings of 1st Annual Conference on Robot Learning, vol 78, pp. 515–524 (2017)
Viereck, U., Pas, A. ten, Saenko, K., Platt, R.: Learning a visuomotor controller for real world robotic grasping using simulated depth images, pp. 1–10 (2017). arXiv:abs/1706.0
Calandra, R., et al.: More than a feeling: learning to grasp and regrasp using vision and touch. IEEE Robot. Autom. Lett. 3, 3300–3307 (2018)
Bicchi, A., Kumar, V.: Robotic grasping and contact: a review. In: Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings vol 1, pp 348–353 (2000)
Bohg, J., Morales, A., Asfour, T., Kragic, D.: Data-driven grasp synthesis—a survey. IEEE Trans. Robot. 30, 289–309 (2014). https://doi.org/10.1109/TRO.2013.2289018
Chu, F.J., Xu, R., Vela, P.A.: Real-world multiobject, multigrasp detection. IEEE Robot. Autom. Lett. 3, 3355–3362 (2018). https://doi.org/10.1109/LRA.2018.2852777
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, December 2016
Guo, D., Sun, F., Liu, H., Kong, T., Fang, B., Xi, N.: A hybrid deep architecture for robotic grasp detection. In: Proceedings of IEEE International Conference on Robotics and Automation, pp. 1609–1614 (2017)
Ten Pas, A., Gualtieri, M., Saenko, K., Platt, R.: Grasp pose detection in point clouds. Int. J. Rob. Res. 36, 1455–1473 (2017). https://doi.org/10.1177/0278364917735594
Lu, Q., Chenna, K., Sundaralingam, B., Hermans, T.: Planning multi-fingered grasps as probabilistic inference in a learned deep network, pp. 1–16 (2018). arXiv:abs/1804.0
Johns, E., Leutenegger, S., Davison, A.J.: Deep learning a grasp function for grasping under gripper pose uncertainty. IEEE Int. Conf. Intell. Robot. Syst. (2016)
Mahler, J. et al.: Dex-Net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics. Robot. Sci. Syst. 13 (2017)
Zeng, A. et al.: Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching. Int. J. Rob. Res. (2019)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)
Goodfellow, I. et al.: Measuring invariances in deep networks. In: Advances in neural information processing systems, vol. 22. pp. 646–654. Curran Associates, Inc. (2009)
Tobin, J. et al.: Domain randomization for transferring deep neural networks from simulation to the real world. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 23–30 (2017)
CornellUniversity: Robot learning lab: Learning to grasp, http://pr.cs.cornell.edu/grasping/rect_data/data.php. Accessed on 17/09/2019
Joe Watson, J.H., Iida, F.: Real-world, real-time robotic grasping with convolution neural network. In: Conference Towards Autonomous Robotic Systems, pp. 627–636 (2017)
Quigley, M., Conley, K., Gerkey, B.P., Faust, J., Foote, T., Leibs, J., Wheeler, R.C., Ng, A.Y.: ROS: an open-source robot operating system. In: ICRA 2009 (2009)
Bradski, G.R., Kaehler, A.: Learning OpenCV—computer vision with the OpenCV library: software that sees. O’Reilly Media, Inc. (2008)
Park, D., Seo, Y., Chun, S.Y.: Real-time, highly accurate robotic grasp detection using fully convolutional neural network with rotation ensemble module (2019)
Washington-Dataset, web:rgbd-dataset.cs.washington.edu. Accessed on 19/09/2019
Levine, S., et al.: Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. Int. J. Rob. Res. 37, 421–436 (2018)
Bousmalis, K. et al.: Using simulation and domain adaptation to improve efficiency of deep robotic grasping. In: Proceedings of IEEE International Conference on Robotics and Automation, pp. 4243–4250 (2018)
Russakovsky, O. et al.: ImageNet large scale visual recognition challenge. CoRR. abs/1409.0 (2014)
Szegedy, C. et al.: Going deeper with convolutions. CoRR. (2014)
Szegedy, C. et al.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-First AAAI Conference on Artificial Intelligence (2017)
Szegedy, C. et al.: Rethinking the inception architecture for computer vision (2016)
Xia, Y., Wang, J.: A dual neural network for kinematic control of redundant robot manipulators. IEEE Trans. Syst. Man, Cybern. Part B. 31, 147–154 (2001)
Polydoros, A.S. et al.: Real-time deep learning of robotic manipulator inverse dynamics. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (2015)
Gonzalez, T.F.: Handbook of approximation algorithms and metaheuristics. In: Handbook of Approximation Algorithms and Metaheuristics, pp. 1–1432 (2007)
Krizhevsky, A., Sutskever, I., Hinton, G.: Imagenet classification with deep convolutional neural. In: Neural Information Processing Systems. pp. 1–9 (2014)
Ruder, S.: An overview of gradient descent optimization algorithms (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Vyas, D.R., Markana, A., Padhiyar, N. (2021). Robotic Grasp Synthesis Using Deep Learning Approaches: A Survey. In: Sahni, M., Merigó, J.M., Jha, B.K., Verma, R. (eds) Mathematical Modeling, Computational Intelligence Techniques and Renewable Energy. Advances in Intelligent Systems and Computing, vol 1287. Springer, Singapore. https://doi.org/10.1007/978-981-15-9953-8_11
Download citation
DOI: https://doi.org/10.1007/978-981-15-9953-8_11
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-9952-1
Online ISBN: 978-981-15-9953-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)