Skip to main content
SpringerLink
Log in

YDRSNet: an integrated Yolov5-Deeplabv3 + real-time segmentation network for gear pitting measurement

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

The traditional calculation methods of the pitting area ratio include artificial vision inspection and rubbing measurement based on cutting tooth. However, these methods have the disadvantages of low efficiency, high cost and static measurement. The non-contact computer vision measurement technology can achieve continuous monitoring without interfering with the machine operation, and have satisfactory detection accuracy. In this paper, we propose an integrated Yolov5-Deeplabv3 + real-time segmentation network (YDRSNet) for gear pitting measurement. The two-stage network is constructed by using Yolov5 and an improve Deeplabv3 + , which can be applied to process the video samples in real time and overcome the problem of sample imbalance. Considering that the second-stage network implements a binary classification task, the dice loss is applied to replace the Cross-entropy loss for reducing the amount of calculation and solving the problem of sample imbalance effectively. Moreover, a DC-Focus module is embedded into the second-stage network for reducing the information loss caused by down sampling. Compared with the existing typical segmentation algorithms, the proposed YDRSNet has stronger segmentation ability, and it can segment the effective tooth surface and different levels of pitting quickly and accurately. The proposed methodology provides a feasible way for online measuring the pitting area ratio and detecting the degree of gear failure.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017). SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12), 2481–2495. https://doi.org/10.1109/TPAMI.2016.2644615

    Article  Google Scholar 

  • Bochkovskiy, A., Wang, C. Y., & Liao, H. Y. M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv: 2004.10934.

  • GitHub - wudashuo/yolov5: https://github.com/wudashuo/yolov5

  • Cha, Y., Choi, W., Suh, G., Mahmoudkhani, S., & Büyüköztürk, O. (2018). Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Computer-Aided Civil and Infrastructure Engineering, 33(9), 731–747. https://doi.org/10.1111/mice.12334

    Article  Google Scholar 

  • Chen, L. C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018). Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision-ECCV2018 (pp. 801–818).

  • Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018a). DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4), 834–848. https://doi.org/10.1109/TPAMI.2017.2699184

    Article  Google Scholar 

  • Chollet, F. (2017). Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1251–1258).

  • Drever, L., Roa, W., McEwan, A., & Robinson, D. (2007). Iterative threshold segmentation for PET target volume delineation: Iterative threshold segmentation for PET. Medical Physics (lancaster), 34(4), 1253–1265. https://doi.org/10.1118/1.2712043

    Article  Google Scholar 

  • He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2020b). Mask R-CNN. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(2), 386–397. https://doi.org/10.1109/TPAMI.2018.2844175

    Article  Google Scholar 

  • He, Y., Song, K., Meng, Q., & Yan, Y. (2020a). An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE Transactions on Instrumentation and Measurement, 69(4), 1493–1504. https://doi.org/10.1109/TIM.2019.2915404

    Article  Google Scholar 

  • Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324. https://doi.org/10.1109/5.726791

    Article  Google Scholar 

  • Li, S., Zhao, X., & Zhou, G. (2019). Automatic pixel-level multiple damage detection of concrete structure using fully convolutional network. Computer-Aided Civil and Infrastructure Engineering, 34(7), 616–634. https://doi.org/10.1111/mice.12433

    Article  Google Scholar 

  • Lin, G., Liu, F., Milan, A., Shen, C., & Reid, I. (2020). RefineNet: Multi-path refinement networks for dense prediction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(5), 1228–1242. https://doi.org/10.1109/TPAMI.2019.2893630

    Article  Google Scholar 

  • Liu, Z., Zhang, M., Liu, F., & Zhang, B. (2021). Multidimensional feature fusion and ensemble learning-based fault diagnosis for the braking system of heavy-haul train. IEEE Transactions on Industrial Informatics, 17(1), 41–51. https://doi.org/10.1109/TII.2020.2979467

    Article  Google Scholar 

  • Lutz, B., Kisskalt, D., Mayr, A., Regulin, D., Pantano, M., & Franke, J. (2020). In-situ identification of material batches using machine learning for machining operations. Journal of Intelligent Manufacturing, 32(5), 1485–1495. https://doi.org/10.1007/s10845-020-01718-3

    Article  Google Scholar 

  • Meister, S., Wermes, M. A. M., Stüve, J., & Groves, R. M. (2021). Review of image segmentation techniques for layup defect detection in the Automated Fiber Placement process: A comprehensive study to improve AFP inspection. Journal of Intelligent Manufacturing, 32(8), 2099–2119. https://doi.org/10.1007/s10845-021-01774-3

    Article  Google Scholar 

  • Menotti, D., Chiachia, G., Pinto, A., Robson Schwartz, W., Pedrini, H., Xavier Falcao, A., & Rocha, A. (2015). Deep representations for iris, face, and fingerprint spoofing detection. IEEE Transactions on Information Forensics and Security, 10(4), 864–879. https://doi.org/10.1109/TIFS.2015.2398817

    Article  Google Scholar 

  • Ni, F., Zhang, J., & Chen, Z. (2019). Pixel-level crack delineation in images with convolutional feature fusion. Structural Control and Health Monitoring, 26(1), e2286. https://doi.org/10.1002/stc.2286

    Article  Google Scholar 

  • Niu, M., Song, K., Huang, L., Wang, Q., Yan, Y., & Meng, Q. (2021). Unsupervised saliency detection of rail surface defects using stereoscopic images. IEEE Transactions on Industrial Informatics, 17(3), 2271–2281. https://doi.org/10.1109/TII.2020.3004397

    Article  Google Scholar 

  • Qin, Y., Li, C., Cao, F., & Chen, H. (2020a). A fault dynamic model of high-speed angular contact ball bearings. Mechanism and Machine Theory, 143, 103627. https://doi.org/10.1016/j.mechmachtheory.2019.103627

    Article  Google Scholar 

  • Qin, Y., Mao, Y., Tang, B., Wang, Y., & Chen, H. (2019). M-band flexible wavelet transform and its application to the fault diagnosis of planetary gear transmission systems. Mechanical Systems and Signal Processing, 134, 106298. https://doi.org/10.1016/j.ymssp.2019.106298

    Article  Google Scholar 

  • Qin, Y., Xiang, S., Chai, Y., & Chen, H. (2020b). Macroscopic-microscopic attention in lstm networks based on fusion features for gear remaining life prediction. IEEE Transactions on Industrial Electronics, 67(12), 10865–10875. https://doi.org/10.1109/TIE.2019.2959492

    Article  Google Scholar 

  • Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I., & Savarese, S. (2019). Generalized intersection over union: A metric and a loss for bounding box regression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 658–666).

  • Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015 234–241.

  • Shelhamer, E., Long, J., & Darrell, T. (2017). Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4), 640–651. https://doi.org/10.1109/TPAMI.2016.2572683

    Article  Google Scholar 

  • Shi, Y., Cui, L., Qi, Z., Meng, F., & Chen, Z. (2016). Automatic road crack detection using random structured forests. IEEE Transactions on Intelligent Transportation Systems, 17(12), 3434–3445. https://doi.org/10.1109/TITS.2016.2552248

    Article  Google Scholar 

  • Sudre, C. H., Li, W., Vercauteren, T., Ourselin, S., & Cardoso, M. J. (2017). Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In Deep learning in medical image analysis and multimodal learning for clinical decision support (pp. 240–248). Springer.

  • Hanshen Tang, Augustin, E., Suen, C.Y., Baret, O., & Cheriet, M. (2004). Recognition of unconstrained legal amounts handwritten on Chinese bank checks. In: Proceedings of the 17th International Conference on Pattern Recognition, ICPR 2004, 2, 610–613 Vol.2. https://doi.org/10.1109/ICPR.2004.1334322

  • Wang, T., Chu, F., & Han, Q. (2017). Fault diagnosis for wind turbine planetary ring gear via a meshing resonance based filtering algorithm. ISA Transactions, 67, 173–182. https://doi.org/10.1016/j.isatra.2016.11.008

    Article  Google Scholar 

  • Wang, X., Qin, Y., Wang, Y., Xiang, S., & Chen, H. (2019). ReLTanh: An activation function with vanishing gradient resistance for SAE-based DNNs and its application to rotating machinery fault diagnosis. Neurocomputing (amsterdam), 363, 88–98. https://doi.org/10.1016/j.neucom.2019.07.017

    Article  Google Scholar 

  • Wang, Y. W., Ni, Y. Q., & Wang, X. (2020). Real-time defect detection of high-speed train wheels by using Bayesian forecasting and dynamic model. Mechanical Systems and Signal Processing, 139, 106654. https://doi.org/10.1016/j.ymssp.2020.106654

    Article  Google Scholar 

  • Xi, D., Qin, Y., Luo, J., Pu, H., & Wang, Z. (2021). Multipath fusion mask R-CNN with double attention and its application into gear pitting detection. IEEE Transactions on Instrumentation and Measurement, 70, 1–11. https://doi.org/10.1109/TIM.2021.3049276

    Article  Google Scholar 

  • Xi, D., Qin, Y., & Wang, Y. (2020). Vision measurement of gear pitting under different scenes by deep mask R-CNN. Sensors (basel, Switzerland), 20(15), 4298. https://doi.org/10.3390/s20154298

    Article  Google Scholar 

  • Xiang, S., Qin, Y., Zhu, C., Wang, Y., & Chen, H. (2020). Long short-term memory neural network with weight amplification and its application into gear remaining useful life prediction. Engineering Applications of Artificial Intelligence, 91, 103587. https://doi.org/10.1016/j.engappai.2020.103587

    Article  Google Scholar 

  • Xie, L., Xiang, X., Xu, H., Wang, L., Lin, L., & Yin, G. (2021). FFCNN: A deep neural network for surface defect detection of magnetic tile. IEEE Transactions on Industrial Electronics, 68(4), 3506–3516. https://doi.org/10.1109/TIE.2020.2982115

    Article  Google Scholar 

  • Xu, H., Lv, X., Wang, X., Ren, Z., Bodla, N., & Chellappa, R. (2021). Deep regionlets: Blended representation and deep learning for generic object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(6), 1914–1927. https://doi.org/10.1109/TPAMI.2019.2957780

    Article  Google Scholar 

  • Xu, Y., Bao, Y., Chen, J., Zuo, W., & Li, H. (2019). Surface fatigue crack identification in steel box girder of bridges by a deep fusion convolutional neural network based on consumer-grade camera images. Structural Health Monitoring, 18(3), 653–674. https://doi.org/10.1177/1475921718764873

    Article  Google Scholar 

  • Yang, M., Yu, K., Zhang, C., Li, Z., & Yang, K. (2018). Denseaspp for semantic segmentation in street scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition-CVPR2018 (pp. 3684–3692).

  • Zhang, S., Zhang, Q., Gu, J., Su, L., Li, K., & Pecht, M. (2021). Visual inspection of steel surface defects based on domain adaptation and adaptive convolutional neural network. Mechanical Systems and Signal Processing, 153, 107541. https://doi.org/10.1016/j.ymssp.2020.107541

    Article  Google Scholar 

  • Zhang, W., Wang, X., You, W., Chen, J., Dai, P., & Zhang, P. (2020). RESLS: Region and edge synergetic level set framework for image segmentation. IEEE Transactions on Image Processing, 29, 57–71. https://doi.org/10.1109/TIP.2019.2928134

    Article  Google Scholar 

  • Zhang, X., Rajan, D., & Story, B. (2019). Concrete crack detection using context-aware deep semantic segmentation network. Computer-Aided Civil and Infrastructure Engineering, 34(11), 951–971. https://doi.org/10.1111/mice.12477

    Article  Google Scholar 

Download references

Acknowledgements

The work described in this paper was supported by the National Key R&D Program of China (no. 2018YFB2001300), National Natural Science Foundation of China (nos. 62033001 and 52175075), and Project supported by graduate scientific research and innovation foundation of Chongqing, China (no. CYB21010).

Author information

Authors and Affiliations

  1. State Key Laboratory of Mechanical Transmission, Chongqing University, Chongqing, 400044, China

    Dejun Xi, Yi Qin & Sijun Wang

  2. College of Mechanical and Vehicle Engineering, Chongqing University, Chongqing, 400044, China

    Dejun Xi, Yi Qin & Sijun Wang

Authors
  1. Dejun Xi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yi Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sijun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yi Qin.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xi, D., Qin, Y. & Wang, S. YDRSNet: an integrated Yolov5-Deeplabv3 + real-time segmentation network for gear pitting measurement. J Intell Manuf 34, 1585–1599 (2023). https://doi.org/10.1007/s10845-021-01876-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-021-01876-y

Keywords

Search

Navigation