Skip to main content
SpringerLink
Log in

Deep learning-based siltation image recognition of water conveyance tunnels using underwater robot

  • Original Paper
  • Published:
Journal of Civil Structural Health Monitoring Aims and scope Submit manuscript

Abstract

Siltation is a significant element that affects the efficiency and safety of water conveyance tunnels. One efficient inspection technique is optical vision inspection carried out by underwater robots. However, efficient processing is required to handle the volume of images that underwater robots collect. Convolutional neural networks (CNNs), have demonstrated considerable promise in computer vision, however it is challenging to implement these models in underwater robots. In this paper, we propose a classification framework for multiple siltation types based on siltation images of water conveyance tunnels using the structure-optimized MobileNet v3, namely SRNet. An underwater robotic image acquisition device is used to acquire the siltation images for training and testing. Out of 6000 images collected from 7 water conveyance tunnels, 4172 are used to train the proposed SRNet network. The remaining 1828 images are used to test it. Furthermore, multiple learning strategies are used to optimize the entire training process. Compared with other deep learning models, the proposed method shows great superiority in terms of recognition results, computational cost and model size. The proposed method effectively weighs model accuracy and complexity and can be used for rapid and accurate identification of siltation in water conveyance tunnel health monitoring.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data availability

Data will be made available on request.

References

  1. Yang F, Cao SR, Qin G (2018) Mechanical behavior of two kinds of prestressed composite linings: A case study of the Yellow River Crossing Tunnel in China. Tunn Undergr Space Technol 79:96–109

    Article  Google Scholar 

  2. Xu ZG, Xian MT, Li XF, Zhou W, Wang JM, Wang YP, Chai JR (2021) Risk assessment of water inrush in karst shallow tunnel with stable surface water supply: Case study. Geomechan Eng 25:495–508

    Google Scholar 

  3. Panthi KK, Basnet CB (2019) Evaluation of earthquake impact on magnitude of the minimum principal stress along a shotcrete lined pressure tunnel in Nepal. J Rock Mech Geotech Eng 11:920–934

    Article  Google Scholar 

  4. Farhadian H, Hassani AN, Katibeh H (2017) Groundwater inflow assessment to Karaj water conveyance tunnel, Northern Iran. KSCE J Civ Eng 21:2429–2438

    Article  Google Scholar 

  5. Dabling M, Lambson D (2018) Olmsted flowline seismic retrofit. Pipelines Conference, Toronto, Canada, 135–141.

  6. Montero R, Victores JG, Martinez S, Jardon A, Balaguer C (2015) Past, present and future of robotic tunnel inspection. Autom Constr 59:99–112

    Article  Google Scholar 

  7. Jorge VAM, Gava PDD, Silva JRBF, Mancilha TM, Vieira W, Adabo GJ, Nascimento CL (2021) VITA1: An unmanned underwater vehicle prototype for operation in underwater tunnels. The 15th Annual IEEE International Systems Conference (SysCon), Vancouver, BC, Canada, 1–8.

  8. Moughamian R, McLeod M (2019) Pardee tunnel inspection and condition assessment. Conference on Pipeline Engineering - Concepts in Harmony (PIPELINES), Nashville, TN, 279–286.

  9. Wang XB, Sun YS, Wan L, Bian HY, Ran XR (2021) Design and reliability analysis of a tunnel-detection AUV based on a heterogeneous dual CPU hot redundancy system. Electronics 10:22

    Article  Google Scholar 

  10. De Cerqueira Gava PD, Jorge VAM, Nascimento CL, Adabo GJ (2020) AUV cruising auto pilot for a long straight confined underwater tunnel. IEEE 14th International Systems Conference (SysCon), Montreal, QC, Canada, 1–8.

  11. Heffron RE (1998) The use of submersible remotely operated vehicles for the inspection of water-filled pipelines and tunnels, Pipeline Division Conference. In: Conjunction with the Prestressed Concrete Cylinder Pipe (PCCP) Users Forum on Pipelines in the Constructed Environment, San Diego, CA, 397-404

  12. Yu SN, Jang JH, Han CS (2007) Auto inspection system using a mobile robot for detecting concrete cracks in a tunnel. Autom Constr 16:255–261

    Article  Google Scholar 

  13. Huang HW, Sun Y, Xue YD, Wang F (2017) Inspection equipment study for subway tunnel defects by grey-scale image processing. Adv Eng Inform 32:188–201

    Article  Google Scholar 

  14. Ai Q, Yuan Y (2019) Rapid acquisition and identification of structural defects of metro tunnel. Sensors 19:4278

    Article  Google Scholar 

  15. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444

    Article  Google Scholar 

  16. Azimi M, Eslamlou AD, Pekcan G (2020) Data-driven structural health monitoring and damage detection through deep learning: State-of-the-art review. Sensors 20:2778

    Article  Google Scholar 

  17. Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, et al. (2016), Tensorflow: Large-scale machine learning on heterogeneous distributed systems, arXiv: 1603.04467.

  18. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L, Desmaison A, et al. (2019) Pytorch: An imperative style, high-performance deep learning library, arXiv: 1912.01703.

  19. Chollet F (2015) Keras. https://keras.io.

  20. LeCun Y, Bottou L, Bengio Y, Haffner P (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 2278–2324.

  21. Simonyan K, Zisserman A (2015). Very deep convolutional networks for large-scale image recognition. International Conference on Learning Representations, San Diego, CA, USA.

  22. Szegedy C, Liu W, Jia YQ, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1–9.

  23. Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Commun ACM 60:84–90

    Article  Google Scholar 

  24. Makantasis K, Protopapadakis E, Doulamis A, Dulamis N, Loupos C (2015) Deep convolutional neural networks for efficient vision based tunnel inspection. The 11th IEEE International Conference on Intelligent Computer Communication and Processing (ICCP), Cluj Napoca, ROMANIA, 335–342.

  25. Xue YD, Li YC (2018) A fast detection method via region-based fully convolutional neural networks for shield tunnel lining defects, Computer-Aided Civil and Infrastructure. Engineering 33:638–654

    Google Scholar 

  26. Ren YP, Huang JS, Hong ZY, Lu W, Yin J, Zou LJ, Shen XH (2020) Image-based concrete crack detection in tunnels using deep fully convolutional networks. Constr Build Mater 234:117367

    Article  Google Scholar 

  27. Zhao S, Zhang DM, Huang HW (2020) Deep learning-based image instance segmentation for moisture marks of shield tunnel lining. Tunn Undergr Space Technol 95:1–11

    Article  Google Scholar 

  28. Protopapadakis E, Voulodimos A, Doulamis A, Doulamis N, Stathaki T (2019) Automatic crack detection for tunnel inspection using deep learning and heuristic image post-processing. Appl Intell 49:2793–2806

    Article  Google Scholar 

  29. Protopapadakis E, Doulamis N (2015) Image based approaches for tunnels' defects recognition via robotic inspectors. The 11th International Symposium on Visual Computing (ISVC), Las Vegas, NV, 706–716.

  30. Protopapadakis E, Stentoumis C, Doulamis N, Doulamis A, Loupos K, Makantasis K, Kopsiaftis G, Amditis A (2016) Autonomous robotic inspection in tunnels. ISPRS Ann Photogram Remote Sens Spatial Inform Sci 3:167

    Article  Google Scholar 

  31. Zhao S, Shadabfar M, Zhang DM, Chen JY, Huang HW (2021) Deep learning-based classification and instance segmentation of leakage-area and scaling images of shield tunnel linings. Struct Control Health Monit 28:1–22

    Article  Google Scholar 

  32. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: Efficient convolutional neural networks for mobile vision applications, arXiv: 1704.04861.

  33. Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016) SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size, arXiv: 1602.07360.

  34. Zhang X, Zhou X, Lin M, Sun J (2017) Shufflenet: An extremely efficient convolutional neural network for mobile devices, arXiv: 1707.01083.

  35. Howard A, Sandler M, Chu G, Chen LC, Chen B, Tan M, Wang W, Zhu Y, Pang R, Vasudevan V, Le QV, Adam H (2019) Searching for MobileNetV3, arXiv: 1905.02244.

  36. Balaguer C, Montero R, Victores J, Martínez S, Jardón A (2014) Towards fully automated tunnel inspection: A survey and future trends. Proceedings of the International Symposium on Automation and Robotics in Construction & Mining, Sydney, Australia.

  37. Loupos K, Doulamis AD, Stentoumis C, Protopapadakis E, Makantasis K, Doulamis ND, Amditis A, Chrobocinski P, Victores J, Montero R, Menendez E, Balaguer C, Lopez R, Cantero M, Navarro R, Roncaglia A, Belsito L, Camarinopoulos S, Komodakis N, Singh P (2018) Autonomous robotic system for tunnel structural inspection and assessment. Int J Intell Robot Appl 2:43–66

    Article  Google Scholar 

  38. Li D, Xie Q, Gong X, Yu Z, Xu J, Sun Y, Wang J (2021) Automatic defect detection of metro tunnel surfaces using a vision-based inspection system. Adv Eng Inform 47:12

    Article  Google Scholar 

  39. Huang H, Li Q, Zhang D (2018) Deep learning based image recognition for crack and leakage defects of metro shield tunnel. Tunn Undergr Space Technol 77:166–176

    Article  Google Scholar 

  40. Huang Z, Sun H (2019) An application of remotely operated vehicle to underwater inspection of deep-buried long tunnel. J Yangtze River Sci Res Inst 36:170–174

    Google Scholar 

  41. Feng CC, Zhang H, Li YL, Wang S, Wang HR (2021) Efficient real-time defect detection for spillway tunnel using deep learning. J Real-Time Image Proc 18:2377–2387

    Article  Google Scholar 

  42. Sifre L, Mallat S (2014) Rigid-motion scattering for image classification author, Ecole Polytechnique.

  43. Hu J, Shen L, Albanie S, Sun G, Wu E (2017) Squeeze-and-Excitation networks, arXiv: 1709.01507.

  44. Tan M, Chen B, Pang R, Vasudevan V, Sandler M, Howard A, Le QV (2018) Mnasnet: Platform-aware neural architecture search for mobile, arXiv: 1807.11626.

  45. He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition, arXiv: 1512.03385.

  46. Ramachandran P, Zoph B, Le QV (2017) Searching for activation functions, arXiv: 1710.05941.

  47. Elfwing S, Uchibe E, Doya K (2017) Sigmoid-weighted linear units for neural network function approximation in reinforcement learning, arXiv: 1702.03118.

  48. Loshchilov I, Hutter F (2016) SGDR: Stochastic gradient descent with warm restarts, arXiv: 1608.03983.

  49. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2015) Rethinking the Inception architecture for computer vision, arXiv: 1512.00567.

  50. Zhang H, Cisse M, Dauphin YN, Lopez-Paz D (2017) Mixup: Beyond empirical risk minimization, arXiv: 1710.09412.

  51. Baur C, Albarqouni S, Navab N (2017) Semi-supervised deep learning for fully convolutional networks. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 311–319.

  52. Doulamis N, Doulamis A (2014) Semi-supervised deep learning for object tracking and classification. IEEE International Conference on Image Processing (ICIP), Paris, France, 848–852.

  53. Tarvainen A, Valpola H (2017) Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. The 31st Annual Conference on Neural Information Processing Systems (NIPS), Long Beach, CA, 1195–1204.

Download references

Acknowledgements

Thanks to South to North Water Diversion Central Route Information Technology Co., Ltd. for providing the underwater video of the water conveyance tunnels for research purposes. This work is supported by the National Key Research & Development Program of China (2016YFC0401600), the National Natural Science Foundation of China (51979027, 52079022, 51769033 and 51779035).

Author information

Authors and Affiliations

  1. Faculty of Infrastructure Engineering, Dalian University of Technology, Experimental Building 3, Room 217, 2 Linggong Road, Dalian, 116024, China

    Xinbin Wu & Junjie Li

  2. College of Water Resources and Hydropower, Hohai University, Nanjing, 210098, China

    Junjie Li

Authors
  1. Xinbin Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Junjie Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xinbin Wu.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, X., Li, J. Deep learning-based siltation image recognition of water conveyance tunnels using underwater robot. J Civil Struct Health Monit 14, 801–816 (2024). https://doi.org/10.1007/s13349-023-00754-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13349-023-00754-w

Keywords

Search

Navigation