Skip to main content
SpringerLink
Log in

Planning Multi-fingered Grasps with Reachability Awareness in Unrestricted Workspace

  • Short Paper
  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

In the unstructured real-world, general object grasping with multi-fingered robot hands requires robots to plan stable and reachable grasps for a plethora of unknown objects and be efficient in arbitrary workspace. Motivated by these rigorous demands, we propose a reachability-aware multi-fingered grasp planning framework that can synthesize feasible high-DOF grasps for novel objects in unrestricted workspace. The framework includes an end-to-end voxel-based Grasp Prediction Network (GPN) for generating stable multi-fingered grasp configurations and a learned Grasping Reachability Evaluator (GRE) to estimate kinematic reachability. The GPN takes as input a voxel representation of the scene and directly outputs the predicted collision-free multi-fingered grasps and the associated grasping stability. The GRE then evaluates the grasping reachability of each generated grasp configuration. Based on the predicted grasping reachability and stability, the most feasible grasp is ultimately executed by the robot. In contrast to previous methods that plan successful grasps within restricted workspace solely based on grasping stability, our approach further incorporates a concept of reachability into the online multi-fingered grasp planning via neural networks. The real-world experiments show that our approach outperforms several comparable methods and achieves an average completion rate of 82.2% for unseen cluttered objects in unrestricted scenarios. All these results indicate that the proposed approach can facilitate general multi-fingered robotic grasping in unstructured environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data Availability

Data and materials used or analyzed during the current study are available from the corresponding author on reasonable request.

Code Availability

Not applicable.

References

  1. Ozawa, R., Tahara, K.: Grasp and dexterous manipulation of multi-fingered robotic hands: a review from a control view point. Adv. Robot. 31(19-20), 1030–1050 (2017)

    Article  Google Scholar 

  2. Miller, A.T., Allen, P.K.: Graspit! a versatile simulator for robotic grasping. IEEE Robotics & Automation Magazine 11(4), 110–122 (2004)

    Article  Google Scholar 

  3. Li, S., Zhang, S., Fu, Y., Xiong, Y., Xie, Z.: Grasp2hardness: fuzzy hardness inference of cylindrical objects for grasp force adjustment of force sensor-less robots. Intell. Serv. Robot. 14(2), 129–141 (2021)

    Article  Google Scholar 

  4. Lenz, I., Lee, H., Saxena, A.: Deep learning for detecting robotic grasps. Int. J. Robot. Res. 34(4-5), 705–724 (2015)

    Article  Google Scholar 

  5. Varley, J., Weisz, J., Weiss, J., Allen, P.: Generating multi-fingered robotic grasps via deep learning. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4415–4420. IEEE (2015)

  6. Liu, M., Pan, Z., Xu, K., Ganguly, K., Manocha, D.: Generating grasp poses for a high-dof gripper using neural networks. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1518–1525. IEEE (2019)

  7. Kiatos, M., Malassiotis, S., Sarantopoulos, I.: A geometric approach for grasping unknown objects with multifingered hands. IEEE Trans. Robot. 37(3), 735–746 (2020)

    Article  Google Scholar 

  8. Zhao, Z., Shang, W., He, H., Li, Z.: Grasp prediction and evaluation of multi-fingered dexterous hands using deep learning. Robot. Auton. Syst. 129, 103550 (2020)

    Article  Google Scholar 

  9. Mnyussiwalla, H., Seguin, P., Vulliez, P., Gazeau, J.: Evaluation and selection of grasp quality criteria for dexterous manipulation. J. Intell. Robot. Syst. 104(2), 1–31 (2022)

    Article  Google Scholar 

  10. Berenson, D., Srinivasa, S.S.: Grasp synthesis in cluttered environments for dexterous hands. In: Humanoids 2008-8th IEEE-RAS International Conference on Humanoid Robots, pp. 189–196. IEEE (2008)

  11. Miller, A.T., Knoop, S., Christensen, H.I., Allen, P.K.: Automatic grasp planning using shape primitives. In: 2003 IEEE International Conference on Robotics and Automation (Cat. No. 03CH37422), vol. 2, pp. 1824–1829. IEEE (2003)

  12. Hang, K., Stork, J.A., Kragic, D.: Hierarchical fingertip space for multi-fingered precision grasping. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1641–1648. IEEE (2014)

  13. Weisz, J., Allen, P.K.: Pose error robust grasping from contact wrench space metrics. In: 2012 IEEE International Conference on Robotics and Automation, pp. 557–562. IEEE (2012)

  14. Lu, Q., Van der Merwe, M., Sundaralingam, B., Hermans, T.: Multifingered grasp planning via inference in deep neural networks: Outperforming sampling by learning differentiable models. IEEE Robotics & Automation Magazine 27(2), 55–65 (2020)

    Article  Google Scholar 

  15. Van der Merwe, M., Lu, Q., Sundaralingam, B., Matak, M., Hermans, T.: Learning continuous 3d reconstructions for geometrically aware grasping. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 11516–11522. IEEE (2020)

  16. Li, S., Li, Z., Han, K., Li, X., Xiong, Y., Xie, Z.: An end-to-end spatial grasp prediction model for humanoid multi-fingered hand using deep network. In: 2021 6th International Conference on Control, Robotics and Cybernetics (CRC), pp. 130–136. IEEE (2021)

  17. Schmidt, P., Vahrenkamp, N., Wächter, M., Asfour, T.: Grasping of unknown objects using deep convolutional neural networks based on depth images. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 6831–6838. IEEE (2018)

  18. Song, F., Zhao, Z., Ge, W., Shang, W., Cong, S.: Learning optimal grasping posture of multi-fingered dexterous hands for unknown objects. In: 2018 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 2310–2315. IEEE (2018)

  19. Kappler, D., Bohg, J., Schaal, S.: Leveraging big data for grasp planning. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 4304–4311. IEEE (2015)

  20. Shao, L., Ferreira, F., Jorda, M., Nambiar, V., Luo, J., Solowjow, E., Ojea, J.A., Khatib, O., Bohg, J.: Unigrasp: Learning a unified model to grasp with multifingered robotic hands. IEEE Robotics and Automation Letters 5(2), 2286–2293 (2020)

    Article  Google Scholar 

  21. Choi, C., Schwarting, W., DelPreto, J., Rus, D.: Learning object grasping for soft robot hands. IEEE Robotics and Automation Letters 3(3), 2370–2377 (2018)

    Article  Google Scholar 

  22. Breyer, M., Chung, J.J., Ott, L., Siegwart, R., Nieto, J.: Volumetric grasping network: Real-time 6 dof grasp detection in clutter. arXiv:2101.01132 (2021)

  23. Ruehl, S.W., Hermann, A., Xue, Z., Kerscher, T., Dillmann, R.: Graspability: A description of work surfaces for planning of robot manipulation sequences. In: 2011 IEEE International Conference on Robotics and Automation, pp. 496–502. IEEE (2011)

  24. Qiu, S., Kermani, M.R.: Inverse kinematics of high dimensional robotic arm-hand systems for precision grasping. Journal of Intelligent & Robotic Systems 101(4), 1–15 (2021)

    Article  Google Scholar 

  25. Lavalle, S., Kuffner, J. Jr: Rapidly-Exploring Random trees: Progress and prospects of algorithmic and computational robotics: New directions. Kluwer academic publishers: Norwell MA (2000)

  26. Jeong, I.-B., Lee, S.-J., Kim, J.-H.: Quick-rrt*: Triangular inequality-based implementation of rrt* with improved initial solution and convergence rate. Expert Syst. Appl. 123, 82–90 (2019)

    Article  Google Scholar 

  27. Wu, B., Akinola, I., Gupta, A., Xu, F., Varley, J., Watkins-Valls, D., Allen, P.K.: Generative attention learning: a ”general” framework for high-performance multi-fingered grasping in clutter. Autonomous Robots (2020)

  28. Zeng, A., Song, S., Welker, S., Lee, J., Rodriguez, A., Funkhouser, T.: Learning synergies between pushing and grasping with self-supervised deep reinforcement learning. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4238–4245. IEEE (2018)

  29. ten Pas, A., Gualtieri, M., Saenko, K., Platt, R.: Grasp pose detection in point clouds. Int. J. Robot. Res. 36(13-14), 1455–1473 (2017)

    Article  Google Scholar 

  30. Porges, O., Stouraitis, T., Borst, C., Roa, M.A.: Reachability and capability analysis for manipulation tasks. In: ROBOT2013: First Iberian Robotics Conference, pp. 703–718. Springer (2014)

  31. Akinola, I., Varley, J., Chen, B., Allen, P.K.: Workspace aware online grasp planning. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2917–2924. IEEE (2018)

  32. Lou, X., Yang, Y., Choi, C.: Learning to generate 6-dof grasp poses with reachability awareness. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 1532–1538. IEEE (2020)

  33. Hinton, G.E.: Learning multiple layers of representation. Trends Cogn. Sci. 11(10), 428–434 (2007)

    Article  Google Scholar 

  34. Coumans, E., Bai, Y.: Pybullet, a python module for physics simulation for games robotics and machine learning (2016)

  35. Kasper, A., Xue, Z., Dillmann, R.: The kit object models database: an object model database for object recognition, localization and manipulation in service robotics. Int. J. Robot. Res. 31(8), 927–934 (2012)

    Article  Google Scholar 

  36. Calli, B., Walsman, A., Singh, A., Srinivasa, S., Abbeel, P., Dollar, A.M.: Benchmarking in manipulation research: The ycb object and model set and benchmarking protocols. arXiv:arXiv1502.03143 (2015)

  37. Singh, A., Sha, J., Narayan, K.S., Achim, T., Abbeel, P.: Bigbird: A large-scale 3d database of object instances. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 509–516. IEEE (2014)

  38. Porges, O., Lampariello, R., Artigas, J., Wedler, A., Borst, C., Roa, M.A.: Reachability and dexterity: analysis and applications for space robotics. In: Workshop on Advanced Space Technologies for Robotics and Automation-ASTRA (2015)

  39. Miller, A.T., Allen, P.K.: Examples of 3d grasp quality computations. In: Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No. 99CH36288C), vol. 2, pp. 1240–1246. IEEE (1999)

  40. Michel, O.: Cyberbotics ltd. webotsTM: professional mobile robot simulation. Int. J. Adv. Robot. Syst. 1(1), 5 (2004)

    Article  Google Scholar 

  41. Chitta, S., Sucan, I., Cousins, S.: Moveit![ros topics]. IEEE Robotics & Automation Magazine 19(1), 18–19 (2012)

    Article  Google Scholar 

  42. Zhou, Q.-Y., Park, J., Koltun, V.: Open3d: A modern library for 3d data processing. arXiv:1801.09847 (2018)

  43. Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F.J., Marín-jiménez, M.J.: Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recogn. 47(6), 2280–2292 (2014)

    Article  Google Scholar 

Download references

Funding

This study was funded by the Key-Area Research and Development Program of Guangdong Province, China (grant number 2019B010154003).

Author information

Author notes

    Authors and Affiliations

    1. School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, 430074, China

      Zhuo Li, Shiqi Li, Ke Han & Xiao Li

    2. UBTECH ROBOTICS CORP LTD, Shenzhen, 518000, China

      Youjun Xiong & Zheng Xie

    Authors
    1. Zhuo Li
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Shiqi Li
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Ke Han
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Xiao Li
      View author publications

      You can also search for this author in PubMed Google Scholar

    5. Youjun Xiong
      View author publications

      You can also search for this author in PubMed Google Scholar

    6. Zheng Xie
      View author publications

      You can also search for this author in PubMed Google Scholar

    Contributions

    Conceptualization: Zhuo Li, Shiqi Li; Methodology: Zhuo Li, Shiqi Li and Ke Han; Software: Zhuo Li and Xiao Li; Writing - original draft preparation: Zhuo Li; Writing - review and editing: Ke Han and Xiao Li; Resources: Youjun Xiong and Zheng Xie; Supervision: Shiqi Li.

    Corresponding author

    Correspondence to Shiqi Li.

    Ethics declarations

    Ethics approval

    This article does not contain any studies with human participants or animals performed by any of the authors.

    Consent to participate

    Not applicable.

    Consent for Publication

    Not applicable.

    Conflict of Interests

    The authors declare that they have no conflict of interest.

    Additional information

    Publisher’s Note

    Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

    Shiqi Li, Ke Han and Xiao Li are contributed equally to this work.

    Rights and permissions

    Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Li, Z., Li, S., Han, K. et al. Planning Multi-fingered Grasps with Reachability Awareness in Unrestricted Workspace. J Intell Robot Syst 107, 39 (2023). https://doi.org/10.1007/s10846-023-01829-y

    Download citation

    • Received:

    • Accepted:

    • Published:

    • DOI: https://doi.org/10.1007/s10846-023-01829-y

    Keywords

    Search

    Navigation