Skip to main content
SpringerLink
Log in

Intelligent identification and classification of sewer pipeline network defects based on improved RegNetY network

  • Original Paper
  • Published:
Journal of Civil Structural Health Monitoring Aims and scope Submit manuscript

Abstract

The operation of a sewer pipeline network affects the urban drainage effect, and identifying the defects of the pipe network is very significant to accurately evaluate the condition of the pipeline network. Closed-circuit television detection has been widely used in urban sewer pipeline network inspection. However, manual discrimination requires professionals to have enough knowledge reserves, which is subjective, laborious, and time-consuming. It is necessary to realize automation and intelligence of defect classification of the sewer pipeline network. An intelligent identification model for sewer pipeline network defects is constructed based on an improved RegNetY network and gradient-weighted class activation mapping (Grad-CAM). The defects are classified and visualized. First, data augmentation is used to deal with data imbalance. Then, LeakyReLU is used as the activation function to improve the RegNetY network. Finally, the training features of the improved RegNetY network are visualized using Grad-CAM. The engineering application results show that the improved RegNetY is more competitive than other types of deep neural networks in defect classification, which can improve the inspection efficiency of engineers. The Grad-CAM can visualize the specific location of defects, which is beneficial to understanding the defect information in images intuitively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Data availability

Data will be made available on request.

References

  1. Lyu H, Ni GH, Tian FQ (2018) Impacts of drainage pipe network complexity on urban stormwater modeling. J Hydroelectr Eng 37(11):97–106. https://doi.org/10.11660/slfdxb.20181110. (in Chinese)

    Article  Google Scholar 

  2. Zhang ML, Shen YM, Shen D (2007) Contrast study of unsteady flow mathematical model for rain pipeline networks of urban drainage system. J Hydroelectr Eng 26(5):80–85. https://doi.org/10.3969/j.issn.1003-1243.2007.05.016. (in Chinese)

    Article  Google Scholar 

  3. Jagai JS, Deflorio-Barker S, Lin CJ et al (2017) Sanitary sewer overflows and emergency room visits for gastrointestinal illness: analysis of Massachusetts data, 2006–2007. Environ Health Perspect 125(11):117007. https://doi.org/10.1289/EHP2048

    Article  Google Scholar 

  4. Wang JL, Deng YL, Li Y et al (2020) A review on detection and defect identification of drainage pipeline. Sci Technol Eng 20(33):13520–13528. https://doi.org/10.3969/j.issn.1671-1815.2020.33.002. (in Chinese)

    Article  Google Scholar 

  5. He M, Qi CC, Chen JX et al (2021) Intelligent auxiliary classification method for drainage pipeline defects based on inspection images. Sci Technol Eng 21(35):15144–15151. https://doi.org/10.3969/j.issn.1671-1815.2021.35.035. (in Chinese)

    Article  Google Scholar 

  6. Duanshun L, Anran C, Shuai G (2019) Sewer damage detection from imbalanced CCTV inspection data using deep convolutional neural networks with hierarchical classification. Autom Constr 101:199–208. https://doi.org/10.1016/j.autcon.2019.01.017

    Article  Google Scholar 

  7. Srinath SK, Dulcy MA, Mohammad RJ et al (2018) Automated defect classification in sewer closed circuit television inspections using deep convolutional neural networks. Autom Constr 91:273–283. https://doi.org/10.1016/j.autcon.2018.03.028

    Article  Google Scholar 

  8. Cheng JCP, Wang M (2018) Automated detection of sewer pipe defects in closed-circuit television images using deep learning techniques. Autom Constr 95:155–171. https://doi.org/10.1016/j.autcon.2018.08.006

    Article  Google Scholar 

  9. Wang M, Cheng JCP (2020) A unified convolutional neural network integrated with conditional random field for pipe defect segmentation. Comput Aided Civ Infrastruct Eng 35(2):162–177. https://doi.org/10.1111/mice.12481

    Article  Google Scholar 

  10. Zhou QQ, Situ ZX, Teng S et al (2021) Intelligent detection and classification of drainage pipe defects based on convolutional neural networks. China Water Wastewater 37(21):114–118. https://doi.org/10.19853/j.zgjsps.1000-4602.2021.21.018. (in Chinese)

    Article  Google Scholar 

  11. Kruachottikul P, Cooharojananone N, Phanomchoeng G et al (2021) Deep learning-based visual defect-inspection system for reinforced concrete bridge substructure: a case of Thailand’s department of highways. J Civ Struct Heal Monit 11(4):949–965. https://doi.org/10.1007/s13349-021-00490-z

    Article  Google Scholar 

  12. Ren QB, Li MC, Shen Y et al (2021) Pixel-level shape segmentation and feature quantification of hydraulic concrete cracks based on digital images. J Hydroelectr Eng 40(2):234–246. https://doi.org/10.11660/slfdxb.20210224. (in Chinese)

    Article  Google Scholar 

  13. Sharma S, Sen S (2020) One-dimensional convolutional neural network-based damage detection in structural joints. J Civ Struct Health Monit 10(5):1057–1072. https://doi.org/10.1007/s13349-020-00434-z

    Article  Google Scholar 

  14. Ruffels A, Gonzalez I, Karoumi R (2020) Model-free damage detection of a laboratory bridge using artificial neural networks. J Civ Struct Health Monit 10(2):183–195. https://doi.org/10.1007/s13349-019-00375-2

    Article  Google Scholar 

  15. Tran VP, Tran TS, Lee HJ et al (2020) One stage detector (RetinaNet)-based crack detection for asphalt pavements considering pavement distresses and surface objects. J Civ Struct Health Monit 11(1):205–222. https://doi.org/10.1007/s13349-020-00447-8

    Article  Google Scholar 

  16. Sunil KS, Paul WF (2005) Neuro-fuzzy network for the classification of buried pipe defects. Autom Constr 15(1):73–83. https://doi.org/10.1016/j.autcon.2005.02.005

    Article  Google Scholar 

  17. Su TC, Yang MD, Wu TC et al (2011) Morphological segmentation based on edge detection for sewer pipe defects on CCTV images. Expert Syst Appl 38(10):13094–13114. https://doi.org/10.1016/j.eswa.2011.04.116

    Article  Google Scholar 

  18. Alaa H, Mazen A, Firas A et al (2018) Automated defect detection tool for closed circuit television (CCTV) inspected sewer pipelines. Autom Constr 89:99–109. https://doi.org/10.1016/j.autcon.2018.01.004

    Article  Google Scholar 

  19. Xie Q, Li D, Xu J et al (2019) Automatic detection and classification of sewer defects via hierarchical deep learning. IEEE Trans Autom Sci Eng 16(4):1836–1847. https://doi.org/10.1109/TASE.2019.2900170

    Article  Google Scholar 

  20. Krizhevsky A, Sutskever I, Hinton G (2012) ImageNet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25(2):1097–1105. https://doi.org/10.1145/3065386

    Article  Google Scholar 

  21. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. In: Proceedings of the international conference on learning representations, pp 1–14. https://doi.org/10.48550/arXiv.1409.1556

  22. Wang HP, An GF, Xie GY (2014) Interpretation of technical regulations for inspection and evaluation of urban drainage pipelines (CJJ181—2012). Water Supply Sewerage 50(2):124–127. https://doi.org/10.3969/j.issn.1002-8471.2014.02.032. (in Chinese)

    Article  Google Scholar 

  23. He K, Zhang X, Ren S et al (2016) Deep residual learning for image recognition. In: Conference on computer vision and pattern recognition, pp 770–778. https://doi.org/10.1109/CVPR.2016.90

  24. Tan M, Le QV (2021) EfficientNetV2: smaller models and faster training. In: Proceedings of the 38th international conference on machine learning, pp 10096–10106. https://doi.org/10.48550/arXiv.2104.00298

  25. Sandler M, Howard A, Zhu M et al (2018) MobileNetV2: inverted residuals and linear bottlenecks. In: Conference on computer vision and pattern recognition (CVPR), pp 4510–4520. https://doi.org/10.1109/CVPR.2018.00474

  26. Huang G, Liu Z, Laurens V et al (2017) Densely connected convolutional networks. In: Conference on computer vision and pattern recognition, pp 2261–2269. https://doi.org/10.1109/CVPR.2017.243

  27. Ma N, Zhang X, Zheng T et al (2018) ShuffleNet V2: practical guidelines for efficient CNN architecture design. In: Proceedings of the 15th European conference on computer vision, 2018, pp 122–138. https://doi.org/10.48550/arXiv.1807.11164

  28. Radosavovi I, Kosaraju RP, Girshick R et al (2020) Designing network design spaces. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 10425–10433. https://doi.org/10.1109/CVPR42600.2020.01044

  29. Bowen B, Otkrist G, Nikhil N et al (2017) Designing neural network architectures using reinforcement learning. In: Proceedings of international conference on learning representations, pp 1–18. https://doi.org/10.48550/arXiv.1611.02167

  30. Li L, Zhang S, Wang B (2021) Apple leaf disease identification with a small and imbalanced dataset based on lightweight convolutional networks. Sensors 22(1):173–173. https://doi.org/10.3390/s22010173

    Article  Google Scholar 

  31. Jie H, Li S, Gang S et al (2019) Squeeze-and-excitation networks. IEEE Trans Pattern Anal Mach Intell 42(8):2011–2023. https://doi.org/10.1109/TPAMI.2019.2913372

    Article  Google Scholar 

  32. Selvaraju RR, Cogswell M, Das A et al (2020) Grad-CAM: visual explanations from deep networks via gradient-based localization. Int J Comput Vis 128(2):336–359. https://doi.org/10.1007/s11263-019-01228-7

    Article  Google Scholar 

  33. Zhou B, Khosla A, Lapedriza A et al (2016) Learning deep features for discriminative localization. In: IEEE conference on computer vision and pattern recognition, pp 2921–2929. https://doi.org/10.1109/CVPR.2016.319

  34. Hou QS, Xing JS (2020) SSD object detection algorithm based on KL loss and Grad-CAM. Acta Electron Sin 48(12):2409–2416. https://doi.org/10.3969/j.issn.0372-2112.2020.12.017. (in Chinese)

    Article  Google Scholar 

  35. Feng XS, Shen Y, Wang DQ (2021) A survey on the development of image data augmentation. Comput Sci Appl 11(2):370–382. https://doi.org/10.12677/CSA.2021.112037. (in Chinese)

    Article  Google Scholar 

Download references

Acknowledgements

This research was jointly funded by the National Natural Science Foundation of China (Grant nos. 51879185 and 52179139), and the Open Fund of Hubei Key Laboratory of Construction and Management in Hydropower Engineering (Grant no. 2020KSD06).

Author information

Authors and Affiliations

  1. State Key Laboratory of Hydraulic Engineering Simulation and Safety, Tianjin University, Tianjin, 300350, China

    Mingze Li, Mingchao Li & Qiubing Ren

  2. Power China Zhongnan Engineering Corporation Limited, Changsha, 410014, Hunan, China

    Hao Liu & Chengzhao Liu

Authors
  1. Mingze Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mingchao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiubing Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chengzhao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qiubing Ren.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, M., Li, M., Ren, Q. et al. Intelligent identification and classification of sewer pipeline network defects based on improved RegNetY network. J Civil Struct Health Monit 13, 547–560 (2023). https://doi.org/10.1007/s13349-022-00660-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13349-022-00660-7

Keywords

Search

Navigation