Skip to main content
SpringerLink
Log in

An inspection method of rail head surface defect via bimodal structured light sensors

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Rail defects have long threatened the safety of railway vehicles. Existing inspection methods still have some problems and flaws that can not meet practical application. In this paper, we propose a rail surface defect inspection method based on bimodal structured light sensors, termed Rail Surface Defect Inspection Network (RSDINet), which can detect and measure defect in bimodal rail images. To verify effect of the method, we establish a bimodal image dataset of intensity and depth images collected by the constructed bimodal structured light sensors data acquisition system. To solve the irregularity of rail surface defect shape, we propose a feature extraction backbone network by introducing deformable convolution. Moreover, RSDINet adopts a parallel feature extraction strategy to process bimodal images respectively. Specifically, we apply different backbone networks to bimodal images respectively for different image characteristics to enhance the feature representation ability of network. Then, our RSDINet fuses multi-scale feature of bimodal images respectively and carries out multi-scale rail surface defect detection and measurement. It is worth noting that the proposed RSDINet can accomplish these two tasks end-to-end simultaneously. Experiments demonstrate that based on the established dataset, the method achieves 87.17 mAP and 39.07 mSAP for detection and measurement respectively at 6.2 FPS on a single GPU, which has a better performance than previous SOTA methods and shows a promising potential for application in high-speed railway.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Data availability

Some or all data, models, or code generated or used during the study are available from the corresponding author by request (Shengchun Wang).

References

  1. Dou Y, Huang Y, Li Q, Luo S (2014) A fast template matching-based algorithm for railway bolts detection. Int J Mach Learn Cybern 5(6):835–844

    Article  Google Scholar 

  2. Li Q, Ren S (2012) A real-time visual inspection system for discrete surface defects of rail heads. IEEE Trans Instrum Meas 61(8):2189–2199

    Article  Google Scholar 

  3. Huber-Mörk R, Nölle M, Oberhauser A, Fischmeister E (2010) Statistical rail surface classification based on 2d and 21/2d image analysis. In: International conference on advanced concepts for intelligent vision systems. Springer, Berlin, pp 50–61

  4. Yu H, Li Q, Tan Y, Gan J, Wang J, Geng YA, Jia L (2019) A coarse-to-fine model for rail surface defect detection. IEEE Trans Instrum Meas 68(3):656–666

    Article  Google Scholar 

  5. Li Q, Ren S (2012) A visual detection system for rail surface defects. IEEE Trans Syst Man Cybern Part C (Appl Rev) 42(6):1531–1542

    Article  Google Scholar 

  6. Molodova M, Li Z, Núñez A, Dollevoet R (2014) Automatic detection of squats in railway infrastructure. IEEE Trans Intell Transp Syst 15(5):1980–1990

    Article  Google Scholar 

  7. Wang J, Li Q, Gan J, Yu H, Yang X (2020) Surface defect detection via entity sparsity pursuit with intrinsic priors. IEEE Trans Ind Inform 16(1):141–150

    Article  Google Scholar 

  8. Chen L, Liang Y, Wang K (2010) Inspection of rail surface defect based on machine vision system. In: The 2nd international conference on information science and engineering. IEEE, Hangzhou, China, pp 3793–3796

  9. Molodova M, Li Z, Nunez A, Dollevoet R (2013) Monitoring the railway infrastructure: detection of surface defects using wavelets. In: International IEEE conference on intelligent transportation systems. IEEE, The Hague, pp 1316–1321

  10. Gan J, Li Q, Wang J, Yu H (2017) A hierarchical extractor-based visual rail surface inspection system. IEEE Sens J 17(23):7935–7944

    Article  Google Scholar 

  11. Xu K (2010) 3D detection technique of surface defects for steel rails based on linear lasers. Chin J Mech Eng. https://doi.org/10.3901/JME.2010.08.001

    Article  Google Scholar 

  12. Ke X, Zhou P, Hu C (2012) 3D detection technique of surface defects for heavy rail based on binocular stereo vision. Proc SPIE Int Soc Opt Eng 8417:07

    Google Scholar 

  13. Ren S, Li Q, Xu G, Han Q, Feng Q (2011) Research on robust fast algorithm of rail surface defect detection. Zhongguo Tiedao Kexue/China Railw Fence 32(1):25–29

    Google Scholar 

  14. Zhao HW, Huang YP, Wang SC, Qing-Yong LI (2014) Rail surface defect detection algorithm based on spatial filtering. Comput Sci 41(1):130–137

    Google Scholar 

  15. Gao JQ, Liu GH (2017) 3D defect detection technology for rail surface with multi-camera line structure light. Mach Des Manuf 3:170–172

    Google Scholar 

  16. Li P, Wang P, Chen P, Xu H (2018) Rail corrugation detection based on 3D structured light and wavelet analysis. Railw Stand Des 62(8):33–38

    Google Scholar 

  17. Krizhevsky A, Sutskever I, Hinton G (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90

    Article  Google Scholar 

  18. Faghih-Roohi S, Siamak H, Núez A, Babuska R, Schutter BD (2016) Deep convolutional neural networks for detection of rail surface defects. In: International joint conference on neural networks (IJCNN 2016). IEEE, Vancouver, pp 2584–2589

  19. Shang L, Yang Q, Wang J, Li S, Lei W (2018) Detection of rail surface defects based on CNN image recognition and classification. In: 2018 20th International conference on advanced communication technology (ICACT). IEEE, Chuncheon, pp 45–51

  20. Song Y, Zhang H, Liu L, Zhang H (2019) Rail surface defect detection method based on YOLOv3 deep learning networks. In: 2018 Chinese automation congress (CAC). IEEE, Xi’an, pp 1563–1568

  21. He Y, Song K, Meng Q, Yan Y (2020) An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE Trans Instrum Meas 69(4):1493–1504

    Article  Google Scholar 

  22. Yi L, Li G, Jiang M (2017) An end-to-end steel strip surface defects recognition system based on convolutional neural networks. Steel Res Int 88(2):176–187

    Article  Google Scholar 

  23. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2018) DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  Google Scholar 

  24. Dai J, Qi H, Xiong Y, Li Y, Zhang G, Hu H, Wei Y (2017) Deformable convolutional networks. In: 2017 IEEE international conference on computer vision (ICCV). IEEE, Venice, pp 764–773

  25. Everingham M, Gool LV, Williams CKI, Zisserman WA (2010) The Pascal visual object classes (VOC) challenge. Int J Comput Vis 88(2):303–338

    Article  Google Scholar 

  26. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: 2014 IEEE conference on computer vision and pattern recognition. IEEE, Columbus, pp 580–587

  27. Girshick R (2015) Fast R-CNN. In: 2015 IEEE international conference on computer vision (ICCV). IEEE, Santiago, pp 1440–1448

  28. Ren S, He K, Girshick R, Sun J (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  29. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, Las Vegas, pp 779–788

  30. Redmon J, Farhadi A (2018) Yolov3: an incremental improvement. arXiv e-prints. arXiv:1804.02767 [cs.CV]

  31. Bochkovskiy A, Wang CY, Liao HYM (2020) Yolov4: optimal speed and accuracy of object detection. arXiv e-prints. arXiv:2004.10934 [cs.CV]

  32. Redmon J, Farhadi A (2017) Yolo9000: better, faster, stronger. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, Honolulu, pp 6517–6525

  33. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) SSD: single shot multibox detector. In: Computer vision—ECCV 2016. Springer, Cham, pp 21–23

  34. Li B, Ouyang W, Sheng L, Zeng X, Wang X (2019) Gs3d: an efficient 3d object detection framework for autonomous driving. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, Long Beach

  35. Chen X, Kundu K, Zhu Y, Ma H, Fidler S, Urtasun R (2018) 3D object proposals using stereo imagery for accurate object class detection. IEEE Trans Pattern Anal Mach Intell 40(5):1259–1272

    Article  Google Scholar 

  36. Qi CR, Liu W, Wu C, Su H, Guibas LJ (2018) Frustum pointnets for 3D object detection from RGB-D data. In: 2018 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, Salt Lake City

  37. Zhou Y, Tuzel O (2018) Voxelnet: End-to-end learning for point cloud based 3D object detection. In: 2018 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, Salt Lake City

Download references

Acknowledgements

Work described in this paper was supported by National Natural Science Foundation of China—China National Railway Group Co., LTD. High-speed Railway Basic Research Fund under Grant no. U1934215, Research and Development Plan of China Academy of Railway Sciences Co. LTD under Grant no. 2021IMXM04.

Author information

Author notes

  1. Jiajun Zheng and Le Wang contributed equally to this work.

Authors and Affiliations

  1. School of Electronics and Communication Engineering, Sun Yat-Sen University, No. 66, Gongchang Road, Guangming District, Shenzhen, 518107, Guangdong, China

    Jiajun Zheng, Liang Wang & Jiaxu Zhang

  2. Railway Infrastructure Research Institute, China Academy of Railway Science Corporation Limited, No. 2 Daliushu Road, Haidian District, Beijing, 100081, China

    Le Wang, Junbo Liu, Hao Wang & Shengchun Wang

Authors
  1. Jiajun Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Le Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junbo Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shengchun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Liang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jiaxu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Shengchun Wang or Liang Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zheng, J., Wang, L., Liu, J. et al. An inspection method of rail head surface defect via bimodal structured light sensors. Int. J. Mach. Learn. & Cyber. 14, 1903–1920 (2023). https://doi.org/10.1007/s13042-022-01736-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-022-01736-y

Keywords

Search

Navigation