Skip to main content
SpringerLink
Log in

Unknown Object Grasping Based on Adaptive Dynamic Force Balance

  • Short Paper
  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Autonomous grasping in dynamic and unstructured environments is challenging in robotics. We aim to solve the problem by proposing an grasping algorithm for unknown objects based on adaptive dynamic force balance. Firstly, Principal Component Analysis(PCA) is used to adjust the axis direction of the camera coordinate system, so that it is consistent with the object coordinate system. The multi-plane projection policy and contour detection algorithm are used to extract the 2D contour information of unknown object, which can reduce the computational complexity. Secondly, a graspable area generating algorithm is established by estimating the size of target object and the distance of the gripper. Several candidate areas are adaptively generated by detecting the concave and convex points on the edge. The optimal grasping area is dynamically selected by minimizing the angle obtained from force balance analysis. The grasping pose is generated by using the depth image to complete autonomous grasping operation. In order to verify the effectiveness of proposed algorithm, a 6-DoF robot grasping platform is built based on eye-in-hand calibration. The experimental results show that compared with the current grasping algorithms, the proposed algorithm can effectively obtain the optimal grasping area of unknown objects without GPUs, which can achieve higher execution efficiency and adaptability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Code Availability

Not applicable.

References

  1. de Souza, J.P.C., Costa, C.M., Rocha, L.F., Arrais, R., Moreira, A.P., Pires, E.S., Boaventura-Cunha, J.: Reconfigurable grasp planning pipeline with grasp synthesis and selection applied to picking operations in aerospace factories. Robot. Comput. Integr. Manuf. 102032, 67 (2021)

    Google Scholar 

  2. Vazquez, J., Giacomini, M., Escareno, J., Rubio, E., Sossa, H.: Optimal grasping points identification for a rotational four-fingered aerogripper. In: 2015 Workshop on Research, Education and Development of Unmanned Aerial Systems (RED-UAS), pp. 272–277. IEEE (2015)

  3. Figueroa, N., Faraji, S., Koptev, M., Billard, A.: A dynamical system approach for adaptive grasping, navigation and co-manipulation with humanoid robots. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 7676–7682 (2020)

  4. Qian, K., Jing, X., Duan, Y., Zhou, B., Fang, F., Xia, J., Ma, X.: Grasp pose detection with affordance-based task constraint learning in single-view point clouds. J. Intell. Robot. Syst. 100, 145–163 (2020)

    Article  Google Scholar 

  5. Islam, F., Salzman, O., Agarwal, A., Likhachev, M.: Provably constant-time planning and replanning for real-time grasping objects off a conveyor belt. arXiv:2101.07148 (2021)

  6. Bohg, J., Morales, A., Asfour, T., Kragic, D.: Data-driven grasp synthesis—a survey. IEEE Trans. Robot. 30(2), 289–309 (2013)

    Article  Google Scholar 

  7. Sui, D., Zhu, Y., Zhao, S., Wang, T., Agrawal, S.K., Zhang, H., Zhao, J.: A bioinspired soft swallowing gripper for universal adaptable grasping Soft Robotics (2020)

  8. Yang, Y., Vella, K., Holmes, D.P.: Grasping with kirigami shells. Science Robotics 6(54), eabd6426 (2021)

    Article  Google Scholar 

  9. Watanabe, T., Morino, K., Asama, Y., Nishitani, S., Toshima, R.: Variable-grasping-mode gripper with different finger structures for grasping small-sized items IEEE Robotics and Automation Letters (2021)

  10. Wu, C., Song, T., Wu, Z., Cao, Q., Fei, F., Yang, D., Xu, B., Song, A.: Development and evaluation of an adaptive multi-dof finger with mechanical-sensor integrated for prosthetic hand. Micromachines 12(1), 33 (2021)

    Article  Google Scholar 

  11. Pinto, L., Gupta, A.: Supersizing self-supervision: learning to grasp from 50k tries and 700 robot Hours. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 3406–3413 (2016)

  12. Mahler, J., Liang, J., Niyaz, S., Laskey, M., Doan, R., Liu, X., Ojea, J.A., Goldberg, K.: Dex-net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics. arXiv:1703.09312 (2017)

  13. Lippiello, V., Ruggiero, F., Siciliano, B., Villani, L.: Visual grasp planning for unknown objects using a multifingered robotic hand. IEEE/ASME Trans. Mechatron. 18(3), 1050–1059 (2012)

    Article  Google Scholar 

  14. Bohg, J., Johnson-Roberson, M., León, B., Felip, J., Gratal, X., Bergström, N., Kragic, D., Morales, A.: Mind the gap-robotic grasping under incomplete observation. In: 2011 IEEE international conference on robotics and automation, pp. 686–693 (2011)

  15. Makhal, A., Thomas, F., Gracia, A.P.: Grasping unknown objects in clutter by superquadric representation. In: 2018 Second IEEE International Conference on Robotic Computing (IRC), pp. 292–299 (2018)

  16. Vezzani, G., Pattacini, U., Natale, L.: A grasping approach based on superquadric models. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 1579–1586 (2017)

  17. Lee, H.K., Kim, M.H., Lee, S.R.: 3d optimal determination of grasping points with whole geometrical modeling for unknown objects. Sens. Actuator A: Phys. 107(2), 146–151 (2003)

    Article  Google Scholar 

  18. Ala, R., Kim, D.H., Shin, S.Y., Kim, C., Park, S.K.: A 3d-grasp synthesis algorithm to grasp unknown objects based on graspable boundary and convex segments. Inform. Sci. 295, 91–106 (2015)

    Article  Google Scholar 

  19. ten Pas, A., Platt, R.: Using geometry to detect grasp poses in 3d point clouds. In: Robotics research, pp. 307–324. Springer (2018)

  20. Suzuki, T., Oka, T.: Grasping of unknown objects on a planar surface using a single depth image. In: 2016 IEEE International Conference on Advanced Intelligent Mechatronics (AIM), pp. 572–577 (2016)

  21. Lei, Q., Wisse, M.: Fast grasping of unknown objects using force balance optimization. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2454–246 (2014)

  22. Lei, Q., Wisse, M.: Fast grasping of unknown objects using cylinder searching on a single point cloud. In: Ninth International Conference on Machine Vision (ICMV 2016), Vol. 10341, P. 1X00000. p. 034108 International Society for Optics and Photonics (2017)

  23. Lei, Q., Chen, G., Wisse, M.: Fast grasping of unknown objects using principal component analysis. Aip Advances 7(9), 095126 (2017)

    Article  Google Scholar 

  24. Bao, J., Jia, Y., Cheng, Y., Xi, N.: Saliency-guided detection of unknown objects in rgb-d indoor scenes. Sensors 15(9), 21054–21074 (2015)

    Article  Google Scholar 

  25. James, J.W., Lepora, N.F.: Slip detection for grasp stabilization with a multifingered tactile robot hand. IEEE Trans. Robot. 37(2), 506–519 (2020)

    Article  Google Scholar 

  26. Calandra, R., Owens, A., Jayaraman, D., Lin, J., Yuan, W., Malik, J., Adelson, E.H., Levine, S.: More than a feeling: Learning to grasp and regrasp using vision and touch. IEEE Robot. Autom. Lett. 3(4), 3300–3307 (2018)

    Article  Google Scholar 

  27. Lei, Q., Wisse, M.: Unknown object grasping by using concavity. In: 2016 14Th International Conference on Control, Automation, Robotics and Vision (ICARCV), pp. 1–8. IEEE (2016)

  28. Jiang, Y., Moseson, S., Saxena, A.: Efficient grasping from rgbd images: learning using a new rectangle representation. In: 2011 IEEE international conference on robotics and automation, pp. 3304–3311 (2011)

  29. Depierre, A., Dellandréa, E., Chen, L.: Jacquard: a large scale dataset for robotic grasp detection. In: 2018 IEEE/ RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3511–3516 (2018)

  30. Coumans, E., Bai, Y.: Pybullet, a python module for physics simulation for games robotics and machine learning (2016)

  31. Chu, F.J., Xu, R., Vela, P.A.: Real-world multiobject, multigrasp detection. IEEE Robot. Autom. Lett. 3 (4), 3355–3362 (2018)

    Article  Google Scholar 

  32. Zhang, H., Lan, X., Bai, S., Zhou, X., Tian, Z., Zheng, N.: Roi-based robotic grasp detection for object overlapping scenes. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4768–4775 (2019)

  33. Zhang, H., Lan, X., Zhou, X., Tian, Z., Zhang, Y., Zheng, N.: Visual manipulation relationship network for autonomous robotics. In: 2018 IEEE–RAS 18Th international conference on humanoid robots (Humanoids), pp. 118–125 (2018)

  34. Fang, H.S., Wang, C., Gou, M., Lu, C.: Graspnet-1billion: a large-scale benchmark for general object grasping. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)

  35. Eppner, C., Mousavian, A., Fox, D.: Acronym: A large-scale grasp dataset based on simulation. arXiv:2011.09584 (2020)

  36. Ebert, F., Finn, C., Lee, A.X., Levine, S.: Self-supervised visual planning with temporal skip connections. In: CoRL, pp. 344–356 (2017)

  37. Ebert, F., Dasari, S., Lee, A.X., Levine, S., Finn, C.: Robustness via retrying: closed-loop robotic manipulation with self-supervised learning. In: Conference on robot learning, pp. 983–993. PMLR (2018)

  38. Morrison, D., Corke, P., Leitner, J.: Egad! an evolved grasping analysis dataset for diversity and reproducibility in robotic manipulation. IEEE Robot. Autom. Lett. 5(3), 4368–4375 (2020)

    Article  Google Scholar 

  39. Zhang, H., Yang, D., Wang, H., Zhao, B., Lan, X., Ding, J., Zheng, N.: Regrad:, A large-scale relational grasp dataset for safe and object-specific robotic grasping in clutter. arXiv:2104.014118(2021)

  40. Lenz, I., Lee, H., Saxena, A.: Deep learning for detecting robotic grasps. Int. J Rob. Res. 34(4-5), 705–724 (2015)

    Article  Google Scholar 

  41. Guo, D., Sun, F., Liu, H., Kong, T., Fang, B., Xi, N.: A hybrid deep architecture for robotic grasp detection. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 1609–1614 (2017)

  42. Kumra, S., Kanan, C.: Robotic grasp detection using deep convolutional neural networks. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 769–776 (2017)

  43. Morrison, D., Corke, P., Leitner, J.: Closing the loop for robotic grasping:, A real-time, generative grasp synthesis approach. arXiv:1804.05172 (2018)

  44. Karaoguz, H., Jensfelt, P.: Object detection approach for robot grasp detection. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 4953–4959. IEEE (2019)

  45. Weng, Y., Sun, Y., Jiang, D., Tao, B., Liu, Y., Yun, J., Zhou, D.: Enhancement of real-time grasp detection by cascaded deep convolutional neural networks. Concurr. Comput. Pract. Exp. 33(5), e5976 (2021)

    Article  Google Scholar 

  46. Wang, D., Liu, C., Chang, F., Li, N., Li, G.: High-performance pixel-level grasp detection based on adaptive grasping and grasp-aware network IEEE Transactions on Industrial Electronics (2021)

  47. Gualtieri, M., Ten Pas, A., Saenko, K., Platt, R.: High precision grasp pose detection in dense clutter. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 598–605 (2016)

  48. Schmidt, P., Vahrenkamp, N., Wächter, M., Asfour, T.: Grasping of unknown objects using deep convolutional neural networks based on depth images. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 6831–6838 (2018)

  49. Ni, P., Zhang, W., Bai, W., Lin, M., Cao, Q.: A new approach based on two-stream cnns for novel objects grasping in clutter. J. Intell. Robotic Syst. 94(1), 161–177 (2019)

    Article  Google Scholar 

  50. Jeng, K.Y., Liu, Y.C., Liu, Z.Y., Wang, J.W., Chang, Y.L., Su, H.T., Hsu, W.H.: Gdn: A coarse-to-fine (c2f) representation for end-to-end 6-dof grasp detection. arXiv:2010.10695 (2020)

  51. Ni, P., Zhang, W., Zhu, X., Cao, Q.: Pointnet++ grasping: learning an end-to-end spatial grasp generation algorithm from sparse point clouds. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 3619–3625 (2020)

  52. Qin, Y., Chen, R., Zhu, H., Song, M., Xu, J., Su, H.: S4g: amodal single-view single-shot se (3) grasp detection in cluttered scenes. In: Conference on Robot Learning, pp. 53–65. PMLR (2020)

  53. Breyer, M., Chung, J.J., Ott, L., Siegwart, R., Nieto, J.: Volumetric grasping network: real-time 6 dof grasp detection in clutter. In: Conference on Robot Learning. PMLR (2020)

  54. Sundermeyer, M., Mousavian, A., Triebel, R., Fox, D.: Contact-graspnet: Efficient 6-dof grasp generation in cluttered scenes. arXiv:2103.14127 (2021)

  55. Herzog, A., Pastor, P., Kalakrishnan, M., Righetti, L., Asfour, T., Schaal, S.: Template-based learning of grasp selection. In: 2012 IEEE international conference on robotics and automation, pp. 2379–2384 (2012)

  56. Pas, A.t., Platt, R.: Using geometry to detect grasps in 3d point clouds. arXiv:1501.03100(2015)

  57. Jain, S., Argall, B.: Grasp detection for assistive robotic manipulation. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 2015–2021 (2016)

  58. Jabalameli, A., Ettehadi, N., Behal, A.: Edge-based recognition of novel objects for robotic grasping. arXiv:1802.08753 (2018)

  59. Zapata-Impata, B.S., Gil, P., Pomares, J., Torres, F.: Fast geometry-based computation of grasping points on three-dimensional point clouds, international journal advanced robotic system 16(1) (2019)

  60. Geng, W., Cao, Z., Li, Z., Yu, Y., Jing, F., Yu, J.: A robotic grasping approach with elliptical cone-based potential fields under disturbed scenes, International Journal of Advanced Robotic Systems 18(1) (2021)

  61. Pfanne, M., Chalon, M., Stulp, F., Ritter, H., Albu-Schäffer, A.: Object-level impedance control for dexterous in-hand manipulation. IEEE Robot. Autom. Lett. 5(2), 2987–2994 (2020)

    Article  Google Scholar 

  62. Li, R., Li, Y., Su, X.: A two-step method for 4-pin form-closure gripper with grasping force optimization Asian Journal of Control (2021)

  63. Liu, B., Wang, T.: An effcient convex hull algorithm for planar point set based on recursive method. Acta Automatica Sinica 38(8), 1375–1379 (2012)

    Article  Google Scholar 

  64. Moreira, A., Santos, M.Y.: Concave Hull: A k-nearest neighbours approach for the computation of the region occupied by a set of points. In: GRAPP (2007)

Download references

Funding

This work is supported by Fundation of Key Laboratory of Equipment Reliability(WD2C20205500306), Major Science and Technology Projects of Liaoning Province(No.2021JH1/10400049) and Fundamental Research Funds for the Central Universities (N182608004, N2004022).

Author information

Authors and Affiliations

  1. Faculty of Robot Science and Engineering, Northeastern University, Shenyang, China

    He Cao & Xin Chen

  2. College of Information Science and Engineering, Northeastern University, Shenyang, China

    Yunzhou Zhang & Yanli Shang

  3. College of Intelligence Science and Technology, National University of Defense Technology, Changsha, Hunan, 410073, China

    Guoji Shen

Authors
  1. He Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yunzhou Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guoji Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yanli Shang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

He Cao and Yanli Shang conducted literature survey and drafted the manuscript; Yunzhou Zhang, Guoji Shen and Xin Chen revised the manuscript.

Corresponding author

Correspondence to Guoji Shen.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for Publication

Not applicable.

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cao, H., Zhang, Y., Shen, G. et al. Unknown Object Grasping Based on Adaptive Dynamic Force Balance. J Intell Robot Syst 105, 10 (2022). https://doi.org/10.1007/s10846-021-01546-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10846-021-01546-4

Keywords

Search

Navigation