Abstract
It is of great significance to quickly detect underwater cracks as they can seriously threaten the safety of underwater structures. Research to date has mainly focused on the detection of above-water-level cracks and hasn’t considered the large scale cracks. In this paper, a large-scale underwater crack examination method is proposed based on image stitching and segmentation. In addition, a purpose of this paper is to design a new convolution method to segment underwater images. An improved As-Projective-As-Possible (APAP) algorithm was designed to extract and stitch keyframes from videos. The graph convolutional neural network (GCN) was used to segment the stitched image. The GCN’s m-IOU is 24.02% higher than Fully convolutional networks (FCN), proving that GCN has great potential of application in image segmentation and underwater image processing. The result shows that the improved APAP algorithm and GCN can adapt to complex underwater environments and perform well in different study areas.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Li W, Chen G, Ge J, Yin X, Li K. High sensitivity rotating alternating current field measurement for arbitrary-angle underwater cracks. NDT & E International, 2016, 79: 123–131
Fang H, Duan M. Off-shore Operation Facilities: Equipment and Procedures. Boston: Gulf Professional Publishing, 2014, 537–686
Sun J, Xue C, Yu Y. Research on feature-based underwater image mosaic technology. Ship Electronic Engineering, 2017, 37(6): 118–121
Rabczuk T, Belytschko T. Cracking particles: A simplified meshfree method for arbitrary evolving cracks. International Journal for Numerical Methods in Engineering, 2004, 61(13): 2316–2343
Zhang Y M, Yang X Q, Wang X Y, Zhuang X Y. A micropolar peridynamic model with non-uniform horizon for static damage of solids considering different nonlocal enhancements. Theoretical and Applied Fracture Mechanics, 2021, 113: 102930
Zhang Y, Mang H A. Global cracking elements: A novel tool for Galerkin-based approaches simulating quasi-brittle fracture. International Journal for Numerical Methods in Engineering, 2020, 121(11): 2462–2480
Martinez J, Rey J, Hidalgo M C, Garrido J, Rojas D. Influence of measurement conditions on the resolution of electrical resistivity imaging: The example of abandoned mining dams in the La Carolina District (Southern Spain). International Journal of Mineral Processing, 2014, 133: 67–72
Xue X, Yang X. Earthquake safety assessment of an arch dam using an anisotropic damage model for mass concrete. Computers and Concrete, 2014, 13(5): 633–648
Zhang Y M, Zhuang X Y. Cracking elements method for dynamic brittle fracture. Theoretical and Applied Fracture Mechanics, 2019, 102: 1–9
Zhang Y M, Zhuang X Y. Cracking elements: A self-propagating Strong Discontinuity embedded Approach for quasi-brittle fracture. Finite Elements in Analysis and Design, 2018, 144: 84–100
Rezaiee-Pajand M, Tavakoli F H. Crack detection in concrete gravity dams using a genetic algorithm. Proceedings of the Institution of Civil Engineers. Structures and Buildings, 2015, 168(3): 192–209
Su H, Li J, Hu J, Wen Z. Analysis and back-analysis for temperature field of concrete arch dam during construction period based on temperature data measured by DTS. IEEE Sensors Journal, 2013, 13(5): 1403–1412
Lai S L, Lee D H, Wu J H, Dong Y M. Detecting the cracks and seepage line associated with an earthquake in an earth dam using the nondestructive testing technologies. Journal of the Chinese Institute of Engineers, 2014, 37(4): 428–437
Luo D, Yue Y, Li P, Ma J X, Zhang L L, Ibrahim Z, Ismail Z. Concrete beam crack detection using tapered polymer optical fiber sensors. Measurement, 2016, 88: 96–103
Shi P, Fan X, Ni J, Khan Z, Li M. A novel underwater dam crack detection and classification approach based on sonar images. PLoS One, 2017, 12(6): e0179627
Xiong P, Xingu Z, Chao Z, Anhua C, Tianyu Z. A UAV-based machine vision method for bridge crack recognition and width quantification through hybrid feature learning. Construction and Building Materials, 2021, 299: 123896
Bang S, Park S, Kim H, Kim H. Encoder—decoder network for pixel-level road crack detection in black-box images. Computer-Aided Civil and Infrastructure Engineering, 2019, 34(8): 713–727
Xu G. Research and implementation of concrete apparent crack detection algorithm based on deep learning. Thesis for the Master’s Degree. Shanghai: Shanghai Jiao Tong University, 2020 (in Chinese)
Belytschko T, Lu Y Y, Gu L. Element-free Galerkin methods. International Journal for Numerical Methods in Engineering, 1994, 37(2): 229–256
Ukai M. Development of image processing technique for detection of tunnel wall deformation using continuously scanned image. Quarterly Report of RTRI, 2000, 41(3): 120–126
Ukai M. Advanced inspection system of tunnel wall deformation using image processing. Quarterly Report of RTRI, 2007, 48(2): 94–98
Lu G F, Zhao Q C, Liao J G, He Y B. Pavement crack identification based on automatic threshold iterative method. In: Seventh International Conference on Electronics and Information Engineering. Nanjing: International Society for Optics and Photonics, 2017
Talab A M A, Huang Z C, Xi F, Liu H M. Detection crack in image using Otsu method and multiple filtering in image processing techniques. Optik (Stuttgart), 2016, 127(3): 1030–1033
Xiao Y, Li J. Crack detection algorithm based on the fusion of percolation theory and adaptive canny operator. In: 2018 37th Chinese Control Conference (CCC). Wuhan: IEEE, 2018, 4295–4299
Feng C C, Zhang H, Wang H R, Wang S, Li Y L. Automatic pixellevel crack detection on dam surface using deep convolutional network. Sensors (Basel), 2020, 20(7): 2069
Nguyen-Thanh V M, Anitescu C, Alajlan N, Rabczuk T, Zhuang X Y. Parametric deep energy approach for elasticity accounting for strain gradient effects. Computer Methods in Applied Mechanics and Engineering, 2021, 386: 114096
Nguyen-Thanh V M, Zhuang X Y, Rabczuk T. A deep energy method for finite deformation hyperelasticity. European Journal of Mechanics. A, Solids, 2020, 80: 103874
Guo H W, Zhuang X Y, Rabczuk T. A deep collocation method for the bending analysis of Kirchhoff plate. Computers, Materials & Continua, 2019, 59(2): 433–456
Zhuang X Y, Guo H W, Alajlan N, Zhu H, Rabczuk T. Deep autoencoder based energy method for the bending, vibration, and buckling analysis of Kirchhoff plates with transfer learning. European Journal of Mechanics. A, Solids, 2021, 87: 104225
Guo H W, Zhuang X Y, Chen P W, Alajlan N, Rabczuk T. Stochastic deep collocation method based on neural architecture search and transfer learning for heterogeneous porous media. Engineering with Computers, 2022, 1–26
Samaniego E, Anitescu C, Goswami S, Nguyen-Thanh V M, Guo H, Hamdia K, Zhuang X, Rabczuk T. An energy approach to the solution of partial differential equations in computational mechanics via machine learning: Concepts, implementation and applications. Computer Methods in Applied Mechanics and Engineering, 2020, 362: 112790
Anitescu C, Atroshchenko E, Alajlan N, Rabczuk T. Artificial neural network methods for the solution of second order boundary value problems. Computers, Materials & Continua, 2019, 59(1): 345–359
Kang F, Wu Y R, Li J J, Li H J. Dynamic parameter inverse analysis of concrete dams based on Jaya algorithm with Gaussian processes surrogate model. Advanced Engineering Informatics, 2021, 49: 101348
Kang F, Liu X, Li J J. Temperature effect modeling in structural health monitoring of concrete dams using kernel extreme learning machines. Structural Health Monitoring, 2020, 19(4): 987–1002
Cha Y, Choi W, Büyüköztürk O. Deep learning-based crack damage detection using convolutional neural networks. Computer-Aided Civil and Infrastructure Engineering, 2017, 32(5): 361–378
Kim B, Cho S. Automated vision-based detection of cracks on concrete surfaces using a deep learning technique. Sensors (Basel), 2018, 18(10): 3452
Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 3431–3440
Dung C V, Anh L D. Autonomous concrete crack detection using deep fully convolutional neural network. Automation in Construction, 2019, 99: 52–58
Zhang X X, Rajan D, Story B. Concrete crack detection using context-aware deep semantic segmentation network. Computer-Aided Civil and Infrastructure Engineering, 2019, 34(11): 951–971
Zhang C B, Chang C C, Jamshidi M. Concrete bridge surface damage detection using a single-stage detector. Computer-Aided Civil and Infrastructure Engineering, 2020, 35(4): 389–409
Liu J W, Yang X, Lau S, Wang X, Luo S, Lee V C S, Ding L. Automated pavement crack detection and segmentation based on two-step convolutional neural network. Computer-Aided Civil and Infrastructure Engineering, 2020, 35(11): 1291–1305
Zhang Y, Yuen K V. Crack detection using fusion features-based broad learning system and image processing. Computer-Aided Civil and Infrastructure Engineering, 2021, 36(12): 1568–1584
Wang L L, Spencer B F, Li J J, Hu P. A fast image-stitching algorithm for characterization of cracks in large-scale structures. Smart Structures and Systems, 2021, 27(4): 593–605
Wu L J, Lin X, Chen Z C, Lin P J, Cheng S Y. Surface crack detection based on image stitching and transfer learning with pretrained convolutional neural network. Structural Control and Health Monitoring, 2021, 28(8): e2766
Burt P J, Adelson E H. The laplacian pyramid as a compact image code. IRE Transactions on Communications Systems, 1983, 31(4): 532–540
Lowe D G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60(2): 91–110
Bay H, Ess A, Tuytelaars T, Van Gool L. Speeded up robust features (SURF). Computer Vision and Image Understanding, 2008, 110(3): 346–359
Rublee E, Rabaud V, Konolige K, Bradski G. ORB: An efficient alternative to SIFT or SURF. In: IEEE International Conference on Computer Vision. 2016, 2564–2571
Zhu Z, German S, Brilakis I J. Detection of large-scale concrete columns for automated bridge inspection. Automation in Construction, 2010, 19(8): 1047–1055
Won J, Park J W, Shim C, Park M W. Bridge-surface panoramic-image generation for automated bridge-inspection using deepmatching. Structural Health Monitoring, 2021, 20(4): 1689–1703
Zhang R W, He D H, Li Y P, Huang L, Bao X J. Synthetic imaging through wavy water surface with centroid evolution. Optics Express, 2018, 26(20): 26009–26019
Qin G Q. Research on underwater photogrammetry for surface easurement of satellite antenna in simulated zero-gravity conditions. Dissertation for the Doctoral Degree. Zhengzhou: The PLA Information Engineering University, 2011 (in Chinese)
Zaragoza J, Chin T J, Tran Q H, Brown M S, Suter D. As-projective-as-possible image stitching with moving DLT. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(7): 1285–1298
Liu Q. Research on aerial image stitching technology based on improved SURF and APAP. Thesis for the Master’s Degree. Dalian: Dalian University of Technology, 2021 (in Chinese)
Landrieu L, Simonovsky M. Large-scale point cloud semantic segmentation with superpoint graphs. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT: IEEE, 2018, 4558–4567
Yuan Y H, Wang J D. Ocnet: Object context network for scene parsing. 2018, arXiv:1809.00916
Ding K M, Chen S P, Meng F. A novel perceptual hash algorithm for multispectral image authentication. Algorithms, 2018, 11(1): 1–14
Zhang L, Li X T, Arnab A, Yang K Y, Tong, Y H, Torr P H S. Dual graph convolutional network for semantic segmentation. 2019, arXiv:1909.06121
Simo-Serra E, Trulls E, Ferraz L, Kokkinos I, Fua P, Moreno-Noguer F. Discriminative learning of deep convolutional feature point descriptors. In: 2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2015, 118–126
Yang J X, Zhang Y B, Huang L H, Guo D C, Yang Y K. A novel diamond search algorithm for fast block motion estimation. In: International Conference on Image Processing and Pattern Recognition in Industrial Engineering. SPIE, 2010, 737–743
Zhao X H, Wang Y, Du Z S, Ye X F. Research on the image enhancement technology of underwater image of super cavitation vehicle. In: 2019 IEEE International Conference on Mechatronics and Automation (ICMA). Tianjin: IEEE, 2019, 1520–1524
Zhao X W, Jin T, Chi H, Qu S. Modeling and simulation of the background light in underwater imaging under different illumination conditions. Acta Physica Sinica, 2015, 64(10): 104201
Baheti B, Innani S, Gajre S, Talbar S. Eff-unet: A novel architecture for semantic segmentation in unstructured environment. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Seattle, WA: IEEE, 2020
Yuan Y H, Chen X L, Wang J D. Object-contextual representations for semantic segmentation. In: European Conference on Computer Vision. Springer, 2019
Acknowledgements
Thanks to South to North Water Diversion Central Route Information Technology Co., Ltd. for providing the underwater videos of the water conveyance tunnels for research purposes. Thanks to CISPDR Corporation for providing the underwater video of the dam for research purposes. This work was supported by the National Natural Science Foundation of China (Grant Nos. 51979027, 52079022, 51769033 and 51779035).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Cao, W., Li, J. Detecting large-scale underwater cracks based on remote operated vehicle and graph convolutional neural network. Front. Struct. Civ. Eng. 16, 1378–1396 (2022). https://doi.org/10.1007/s11709-022-0855-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11709-022-0855-8