Deep Grasping Prediction with Antipodal Loss for Dual Arm Manipulators

  • Yunlong Dong
  • Xiangdi Liu
  • Bidan Huang
  • Chunlin Ji
  • Jianfeng Xu
  • Han Ding
  • Ye YuanEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11740)


The cooperative manipulators can execute a wide range of tasks, such as carrying large or heavy payloads, which are difficult for a single manipulator. Dual arm manipulators are in typically operative configuration to mimic human, which are of highly flexibility and dexterity. In this paper, we propose a novel coarse-to-fine deep learning model along with investigating the grasp prior loss based on the well-known antipodal force-closure property. The proposed deep learning model predicts the contact configurations in grasping over-loaded and over-sized objects for dual arm manipulators directly from raw RGB images. We first apply detection network to locate the coarse bounding box of objects, further apply a fine-predicting network on the bounding box clipped images to precisely generate two contact configurations via minimizing regression loss and the antipodal grasp prior loss. Extensive experimental results under dense clutter and occlusion strongly demonstrate the effectiveness and robustness of the proposed method.



This work is supported by National Natural Science Foundation of China under Grant 91748112. The authors would like to thank Wei Li, Xiuchuan Tang and Linan Deng in the School of Artificial Intelligence and Automation, Huazhong University of Science and Technology for helping in the experiments.


  1. 1.
    Almeida, D., Karayiannidis, Y.: Folding assembly by means of dual-arm robotic manipulation. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 3987–3993. IEEE (2016)Google Scholar
  2. 2.
    Chaumette, F., Hutchinson, S.: Visual servo control. I. Basic approaches. IEEE Robot. Autom. Mag. 13(4), 82–90 (2006)CrossRefGoogle Scholar
  3. 3.
    Chen, I.M., Burdick, J.W.: Finding antipodal point grasps on irregularly shaped objects. IEEE Trans. Robot. Autom. 9(4), 507–512 (1993). Scholar
  4. 4.
    Gualtieri, M., ten Pas, A., Saenko, K., Platt, R.: High precision grasp pose detection in dense clutter. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 598–605. IEEE (2016)Google Scholar
  5. 5.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  6. 6.
    Hsu, P.: Coordinated control of multiple manipulator systems. IEEE Trans. Robot. Autom. 9(4), 400–410 (1993)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Johns, E., Leutenegger, S., Davison, A.J.: Deep learning a grasp function for grasping under gripper pose uncertainty. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4461–4468. IEEE (2016)Google Scholar
  8. 8.
    Kinga, D., Adam, J.B.: A method for stochastic optimization. In: International Conference on Learning Representations (ICLR), vol. 5 (2015)Google Scholar
  9. 9.
    Korpela, C., Orsag, M., Oh, P.: Towards valve turning using a dual-arm aerial manipulator. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3411–3416, September 2014.
  10. 10.
    Lenz, I., Lee, H., Saxena, A.: Deep learning for detecting robotic grasps. Int. J. Robot. Res. 34(4–5), 705–724 (2015)CrossRefGoogle Scholar
  11. 11.
    Lin, M., Chen, Q., Yan, S.: Network in network. arXiv preprint arXiv:1312.4400 (2013)
  12. 12.
    Mahler, J., et al.: Dex-Net 2.0: deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics. In: Robotics: Science and Systems (RSS) (2017)Google Scholar
  13. 13.
    Park, C., Park, K.: Design and kinematics analysis of dual arm robot manipulator for precision assembly. In: 6th IEEE International Conference on Industrial Informatics, INDIN 2008, pp. 430–435. IEEE (2008)Google Scholar
  14. 14.
    Paszke, A., et al.: Automatic differentiation in pytorch (2017)Google Scholar
  15. 15.
    Pinto, L., Gupta, A.: Supersizing self-supervision: learning to grasp from 50k tries and 700 robot hours. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 3406–3413. IEEE (2016)Google Scholar
  16. 16.
    Quigley, M., et al.: ROS: an open-source robot operating system. In: ICRA Workshop on Open Source Software (2009)Google Scholar
  17. 17.
    Redmon, J., Farhadi, A.: Yolo9000: better, faster, stronger. arXiv preprint arXiv:1612.08242 (2016)
  18. 18.
    Siciliano, B., Sciavicco, L., Villani, L., Oriolo, G.: Robotics: Modelling, Planning and Control. Springer, Heidelberg (2010). Scholar
  19. 19.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  20. 20.
    Smith, C., Karayiannidis, Y., Nalpantidis, L., et al.: Dual arm manipulation: a survey. Robot. Autonom. Syst. 60(10), 1340–1353 (2012)CrossRefGoogle Scholar
  21. 21.
    Uchiyama, M., et al.: Development of a flexible dual-arm manipulator testbed for space robotics. In: Proceedings of the IEEE International Workshop on Intelligent Robots and Systems, Towards a New Frontier of Applications, IROS 1990, pp. 375–381. IEEE (1990)Google Scholar
  22. 22.
    Varley, J., Weisz, J., Weiss, J., Allen, P.: Generating multi-fingered robotic grasps via deep learning. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4415–4420. IEEE (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Yunlong Dong
    • 1
  • Xiangdi Liu
    • 1
  • Bidan Huang
    • 2
  • Chunlin Ji
    • 3
  • Jianfeng Xu
    • 4
  • Han Ding
    • 4
  • Ye Yuan
    • 1
    Email author
  1. 1.School of Artificial Intelligence and Automation and State Key Laboratory of Digital Manufacturing Equipment and TechnologyHuazhong University of Science and TechnologyWuhanChina
  2. 2.Tencent Robotics XShenzhenChina
  3. 3.Kuang-Chi Institute of Advanced TechnologyShenzhenChina
  4. 4.School of Mechanical Science and Engineering and State Key Laboratory of Digital Manufacturing Equipment and TechnologyHuazhong University of Science and TechnologyWuhanChina

Personalised recommendations