Skip to main content
SpringerLink
Log in

Human-robot Collision Detection Based on the Improved Camshift Algorithm and Bounding Box

  • Regular Papers
  • Robot and Applications
  • Published:
International Journal of Control, Automation and Systems Aims and scope Submit manuscript

Abstract

Aiming at the problem of collision detection and collision point information evaluation in the process of human-robot collaboration, the binocular camera is used as an external sensor to observe. Collision detection is realized by tracking the motion of human-robot through the color information of joints. Firstly, the Camshift algorithm is used to track the position of the manipulator joints and the human arm joints based on the color information. In order to solve the factors that may cause target loss during tracking process, such as shelter and background color similar problems, Kalman filter is integrated on the basis of Camshift algorithm. A similarity threshold is set to judge whether there is interference in the tracking process. The tracking experiment proved that the Kalman filter is effective and enhances the robustness of the tracking algorithm. Secondly, a bounding box collision detection method based on space domain is designed. The sphere bounding box and the cylindrical bounding box is used as the human-robot bounding boxes. The equations of the distance between different boxes are derived and the position of the collision point on the manipulator is calculated. Finally, an experimental environment is built for verification. The distance error of the collision is within 0–10 mm, and the position error between the calculated collision point and the pre-determined collision point is within 10%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. A.-N. Sharkawy and N. Aspragathos, “Human-robot collision detection based on neural networks,” International Journal of Mechanical Engineering and Robotics Research, vol. 7, no. 2, pp. 150–157, 2018.

    Article  Google Scholar 

  2. F. Min, G. Wang, and N. Liu, “Collision detection and identification on robot manipulators based on vibration analysis,” Sensors, vol. 19, no. 5, p. 1080, 2019.

    Article  Google Scholar 

  3. S. Haddadin, A. De Luca, and A. Albu-Schäffer, “Robot collisions: A survey on detection, isolation, and identification,” IEEE Transactions on Robotics, vol. 33, no. 6, pp. 1292–1312, 2017.

    Article  Google Scholar 

  4. M. Zhiju and W. Haibin, “Sensorless collision detection for robot manipulator,” Journal of Test and Measurement Technology, vol. 27, no. 3, pp. 254–259, 2013.

    Google Scholar 

  5. Z. Hanjie, Collision Detection of Robotic Manipulators, Harbin Insititute of Technology, 2017.

  6. A. Cirillo, P. Cirillo, G. De Maria, C. Natale, and S. Pirozzi, “A proximity/contact-force sensor for human safety in industrial robot environment,” Proc. of IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pp. 1272–1277, 2013.

    Google Scholar 

  7. A. de Luca and F. Flacco, “Integrated control for pHRI: Collision avoidance, detection, reaction and collaboration,” Proc. of 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob), pp. 288–295, 2012.

    Google Scholar 

  8. T. Ren, Y. Dong, D. Wu, and K. Chen, “Collision detection and identification for robot manipulators based on extended state observer,” Control Engineering Practice, vol. 79, pp. 144–153, 2018.

    Article  Google Scholar 

  9. Z. Wang, L. Liu, W. Yan, J. He, B. Cui, and Z. Li, “Research on collision point identification based on six-axis force/torque sensor,” Journal of Sensors, vol. 2020, p. 8881263, 2020.

    Article  Google Scholar 

  10. J. O’Neill, J. Lu, R. Dockter, and T. Kowalewski, “Practical, stretchable smart skin sensors for contact-aware robots in safe and collaborative interactions,” Proc. of IEEE International Conference on Robotics and Automation (ICRA), pp. 624–629, 2015.

    Google Scholar 

  11. Y. Xu, F. Jiang, S. Newbern, A. Huang, C.-M. Ho, and Y.-C. Tai, “Flexible shear-stress sensor skin and its application to unmanned aerial vehicles,” Sensors and Actuators A: Physical, vol. 105, no. 3, pp. 321–329, 2003.

    Article  Google Scholar 

  12. T. Someya, T. Sekitani, S. Iba, Y. Kato, H. Kawaguchi, and T. Sakurai, “A large-area, flexible pressure sensor matrix with organic field-effect transistors for artificial skin applications,” Proceedings of the National Academy of Sciences, vol. 101, no. 27, pp. 9966–9970, 2004.

    Article  Google Scholar 

  13. Z. Xiuli and H. Chunyan, “Design and collision experiment of tactile sensing device for collaborative robot,” School of Mechanical, Electronic and Control Engineering, Beijing Jiaotong University, vol. 43, no. 4, p. 88, 2019.

    Google Scholar 

  14. A. De Luca, A. Albu-Schaffer, S. Haddadin, and G. Hirzinger, “Collision detection and safe reaction with the DLR-III lightweight manipulator arm,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1623–1630, 2006.

    Google Scholar 

  15. F. Flacco, T. Kröger, A. de Luca, and O. Khatib, “A depth space approach to human-robot collision avoidance,” Proc. of IEEE International Conference on Robotics and Automation, pp. 338–345, 2012.

    Google Scholar 

  16. J. Pan, S. Chitta, and D. Manocha, “FCL: A general purpose library for collision and proximity queries,” Proc. of IEEE International Conference on Robotics and Automation, pp. 3859–3866, 2012.

    Google Scholar 

  17. D. M. Ebert and D. D. Henrich, “Safe human-robot-cooperation: Image-based collision detection for industrial robots,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 2, pp. 1826–1831, 2002.

    Article  Google Scholar 

  18. D. Exner, E. Bruns, D. Kurz, A. Grundhöfer, and O. Bimber, “Fast and robust CAMShift tracking,” Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, pp. 9–16, 2010.

    Google Scholar 

  19. W. Guan, Z. Liu, S. Wen, H. Xie, and X. Zhang, “Visible light dynamic positioning method using improved Camshift-Kalman algorithm,” IEEE Photonics Journal, vol. 11, no. 6, pp. 1–22, 2019.

    Article  Google Scholar 

  20. A. H. Mazinan and A. Amir-Latifi, “Applying mean shift, motion information and Kalman filtering approaches to object tracking,” ISA Transactions, vol. 51, no. 3, pp. 485–497, 2012.

    Article  Google Scholar 

  21. S.-K. Weng, C.-M. Kuo, and S.-K. Tu, “Video object tracking using adaptive Kalman filter,” Journal of Visual Communication and Image Representation, vol. 17, no. 6, pp. 1190–1208, 2006.

    Article  Google Scholar 

  22. C. Ericson, Real-time Collision Detection, CRC Press, 2004.

  23. T. Urano, T. Matsui, T. Nakata, and H. Mizoguchi, “Human pose recognition by memory-based hierarchical feature matching,” Proc. of IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No. 04CH37583), vol. 7, pp. 6412–6416, 2004.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Mechanical and Electrical Engineering, China Jiliang University, No. 258, Xueyuan Road, Hangzhou, 310018, China

    Shuangning Lu, Zhouda Xu & Binrui Wang

Authors
  1. Shuangning Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhouda Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Binrui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Binrui Wang.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Shuangning Lu received his B.E. degree in software engineering from Chengdu University of Information Technology in 2018. His research interests include collaborative robot and collision detection.

Zhouda Xu received his M.Eng. degree in control science and engineering from China Jiliang University in 2020. His research intrests include collaborative robot and wire and cable detection.

Binrui Wang received his Ph.D. degree in pattern recognition and intelligent system from the School of Information Science and Engineering at Northeastern University in 2005. Currently, he is a professor in the College of Mechanical and Electrical Engineering at China Jiliang University, China. His research interests include intelligent control algorithm and humanoid robot.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, S., Xu, Z. & Wang, B. Human-robot Collision Detection Based on the Improved Camshift Algorithm and Bounding Box. Int. J. Control Autom. Syst. 20, 3347–3360 (2022). https://doi.org/10.1007/s12555-021-0280-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12555-021-0280-0

Keywords

Search

Navigation