Skip to main content
SpringerLink
Log in

MGBM-YOLO: a Faster Light-Weight Object Detection Model for Robotic Grasping of Bolster Spring Based on Image-Based Visual Servoing

  • Short paper
  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

The rapid detection and accurate positioning of bolster spring with complex geometric features in cluttered background is highly important for the grasping task of bolster spring in overhauling workshop. To achieve a better trade-off among positioning accuracy, running time and model size, two MobileNetv3 and GhostNet-based modified YOLOv3 (MGBM-YOLO) models are proposed in this paper: MGBM-YOLO-Large and MGBM-YOLO-Small, which are applied to the robotic grasping system of bolster spring based on image-based visual servoing (IBVS). The proposed MGBM-YOLO models are trained and evaluated on Pascal VOC2007 and Pascal VOC2012 datasets. The results show that compared with the original YOLOv3 model based on Darknet53, the sizes of the proposed Large and Small models are reduced by 59% and 66% respectively, and the model parameters are reduced by 63% and 66% respectively. The detection speed is 4× and 5.5× faster respectively when the difference of mAP is small. To improve the convergence speed of IBVS system, an online depth estimation method based on the area feature of detection bounding box of bolster spring is proposed. The comparative experiments of visual servoing grasping are conducted on the robotic grasping platform of bolster spring. The experimental results show that the two MGBM-YOLO detection models proposed in this paper can meet the requirements of fast detection and positioning of bolster spring robotic grasping system based on IBVS. The proposed method of feature depth online estimation is of great significance to accelerate the convergence speed of visual servoing system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data Availability

Not applicable.

References

  1. Xin, J.S., Shang, Y.J., Xue, H., et al.: Dynamic reliability sensitivity analysis of bolster spring for heavy-haul freight Car. J. Lanzhou Jiaotong Univ. 39(06), 86–91 (2020)

    Google Scholar 

  2. Chaumette, F., Hutchinson, S.: Visual servo control, part I: basic approaches[J]. IEEE Robot. Autom. Mag. 13(4), 82–90 (2006)

    Article  Google Scholar 

  3. Malis, E., Ezio, S., Chaumette, F., et al.: 2-1/2-D visual Servoing[J]. IEEE Trans Rob Autom. (1999)

  4. Espiau B.: Effect of Camera Calibration Errors on Visual Servoing in Robotics[C]// Preprints of the Third International Symposium on Experimental Robotics. (1993)

  5. Li., D.F., Liu, H.L., Wei, T., et al.: Robotic grasping method of bolster spring based on image-based visual servoing with YOLOv3 object detection algorithm[J]. Proc. Inst. Mech. Eng. Part C-J. Mech. Eng. Sci. (2021)

  6. Howard, A., Sandler, M., Chu, G., et al.: Searching for Mobilenetv3[C]// 2019 IEEE/CVF international conference on computer vision (ICCV). IEEE, 1314–1324 (2020)

  7. Han, K., Wang, Y., Tian, Q., et al.: GhostNet: More Features from Cheap Operations[C]// 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, 1580–1589 (2020)

  8. Lin, M., Chen, Q., Yan, S.: Network in network[J]. Comput. Therm. Sci. (2013)

  9. Szegedy, C., Liu, W., Jia, Y., et al.: Going Deeper with Convolutions[C]// IEEE Computer Society (2014)

  10. Iandola, F. N, Han, S., Moskewicz, M. W., et al.: SqueezeNet: AlexNet-level accuracy with 50× fewer parameters and < 0.5 MB model size[J]. (2016)

  11. Howard, A. G., Zhu, M., Chen, B., et al.: Mobilenets: Efficient Convolutional Neural Networks for Mobile Vision Applications[J]. (2017)

  12. Sandler, M., Howard, A., Zhu, M., et al.: Mobilenetv2: inverted residuals and linear bottlenecks[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 4510–4520 (2018)

  13. Zhang, X., Zhou, X., Lin, M., et al.: ShuffleNet: an extremely efficient convolutional neural network for Mobile devices[C]// 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 6848–6856 (2018)

  14. Ma, N., Zhang, X., Zheng, H. T, et al.: ShuffleNet v2: Practical Guidelines for Efficient CNN Architecture Design[C]// European Conference on Computer Vision (ECCV). Springer, Cham, 116–131 (2018)

  15. Xu, D.: A tutorial for monocular visual servoing. Acta Aotumatica Sinica. 44(10), 1729–1746 (2018)

    Google Scholar 

  16. Yang, Y., Song, Y., Pan, H., et al.: Visual servo simulation of EAST articulated maintenance arm robot[J]. Fusion Eng. Des. 104(Mar.), 28–33 (2016)

    Article  Google Scholar 

  17. Tongloy, T., Boonsang, S.: An Image-Based Visual Servo Control System Based on an Eye-in-Hand Monocular Camera for Autonomous Robotic Grasping[C]// International Conference on Instrumentation. IEEE (2016)

  18. Dong, G.Q., Zhu, Z.H.: Kinematics-based incremental visual servo for robotic capture of non-cooperative target[J]. Robot. Auton. Syst. 112, 221–228 (2019)

    Article  Google Scholar 

  19. Xie, W.F., Li, Z., Tu, X.W., et al.: Switching control of image-based visual Servoing with laser pointer in robotic manufacturing systems[J]. IEEE Trans. Ind. Electron. 56(2), 520–529 (2009)

    Article  Google Scholar 

  20. Mehta, S.S., Ton, C., Rysz, M., et al.: New approach to visual servo control using terminal constraints[J]. J. Frankl. Inst. 356(10), 5001–5026 (2019)

    Article  MathSciNet  Google Scholar 

  21. Zhong, X.G., Zhong, X.Y., Hu, H.S., et al.: Adaptive neuro-filtering based visual servo control of a robotic manipulator[J]. IEEE Access. 7, 76891–76901 (2019)

    Article  Google Scholar 

  22. Yang, Y., Zhang, W.S., He, Z.W., et al.: High-speed rail pole number recognition through deep representation and temporal redundancy[J]. Neurocomputing. 415, 201–214 (2020)

    Article  Google Scholar 

  23. Álvaro, A.G., Álvarez-García, J.A., Soria-Morillo, L.M.: Evaluation of deep neural networks for traffic sign detection system[J]. Neurocomputing. 316, 332–344 (2018)

    Article  Google Scholar 

  24. Redmon, J., Divvala, S., Girshick, R., et al.: You only look once: unified, real-time object detection[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 779–788 (2016)

  25. Redmon, J., Farhadi, A.: YOLO9000: Better, Faster, Stronger[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 7263–7271 (2017)

  26. Redmond, J., Farhadi, A.: YOLOv3: An incremental improvement[J]. arXiv preprint arVix: 1804.02767 (2018)

  27. Liu, J., Wang, X.W.: Tomato diseases and pests detection based on improved Yolo V3 convolutional neural network[J]. Front. Plant Sci. 11, 898 (2020)

    Article  Google Scholar 

  28. Xu, D.Q., Wu, Y.Q.: MRFF-YOLO: a multi-receptive fields fusion network for remote sensing target detection[J]. Remote Sens. 12, 3118 (2020)

    Article  Google Scholar 

  29. Corke, P.: Robotics, Vision and Control[M]// Robotics, Vision and Control - Fundamental Algorithms in MATLAB®. Springer, Berlin (2011)

    MATH  Google Scholar 

  30. Zhang, Z.Y.: A flexible new technique for camera calibration[J]. IEEE Trans. Pattern Anal. Mach. Intell. (2000)

Download references

Code Availability

Not applicable.

Author information

Authors and Affiliations

  1. Engineering Research Center of Advanced Driving Energy-Saving Technology, Ministry of Education, Chengdu, 610031, People’s Republic of China

    Huanlong Liu, Dafa Li, Bin Jiang, Jianyi Zhou, Tao Wei & Xinliang Yao

  2. School of Mechanical Engineering, Southwest Jiaotong University, Chengdu, 610031, People’s Republic of China

    Huanlong Liu, Dafa Li, Jianyi Zhou, Tao Wei & Xinliang Yao

  3. Graduate School of Tangshan, Southwest Jiaotong University, Tangshan, 063000, People’s Republic of China

    Bin Jiang

Authors
  1. Huanlong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dafa Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bin Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jianyi Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tao Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xinliang Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Huanlong Liu: Conceptualization, Methodology. Dafa Li: Software, Experimental validation, Data curation, Writing-Reviewing and Editing. Bin Jiang: Data curation, Visualization. Jianyi Zhou: Investigation. Tao Wei: Investigation. Xinliang Yao: Investigation.

Corresponding author

Correspondence to Dafa Li.

Ethics declarations

Conflict of Interest

No conflict of interest exits in the submission of this article, and the article is approved by all authors for publication.

Ethics Approval

Not applicable.

Consent to Participate

Not applicable.

Consent for Publication

Not applicable.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, H., Li, D., Jiang, B. et al. MGBM-YOLO: a Faster Light-Weight Object Detection Model for Robotic Grasping of Bolster Spring Based on Image-Based Visual Servoing. J Intell Robot Syst 104, 77 (2022). https://doi.org/10.1007/s10846-021-01564-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10846-021-01564-2

Keywords

Search

Navigation