Abstract
Large structures, such as bridges, highways, etc., need to be inspected to evaluate their actual physical and functional condition, to predict future conditions, and to help decision makers allocating maintenance and rehabilitation resources. The assessment of civil infrastructure condition is carried out through information obtained by inspection and/or monitoring operations. Traditional techniques in structural health monitoring (SHM) involve visual inspection related to inspection standards that can be time-consuming data collection, expensive, labor intensive, and dangerous. To address these limitations, machine vision-based inspection procedures have increasingly been investigated within the research community. In this context, this paper proposes and compares four different computer vision procedures to identify damage by image processing: Otsu method thresholding, Markov random fields segmentation, RGB color detection technique, and K-means clustering algorithm. The first method is based on segmentation by thresholding that returns a binary image from a grayscale image. The Markov random fields technique uses a probabilistic approach to assign labels to model the spatial dependencies in image pixels. The RGB technique uses color detection to evaluate the defect extensions. Finally, K-means algorithm is based on Euclidean distance for clustering of the images. The benefits and limitations of each technique are discussed, and the challenges of using the techniques are highlighted. To show the effectiveness of the described techniques in damage detection of civil infrastructures, a case study is presented. Results show that various types of corrosion and cracks can be detected by image processing techniques making the proposed techniques a suitable tool for the prediction of the damage evolution in civil infrastructures.
Similar content being viewed by others
References
Arthur D, Vassilvitskii S (2007), “K-means++: The Advantages of Careful Seeding,” Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms, 7–9 January, 2007, New Orleans, pp. 1027–1035.
Attard L, Debono CJ, Valentino G, Di Castro M, Masi A and Scibile L (2019), “Automatic Crack Detection Using Mask R-CNN,” Proceeding of 2019 11th International Symposium on Image and Signal Processing and Analysis (ISPA), IEEE, pp. 152–157.
Bauer J, Sünderhauf N and Protzel P (2007), “Comparing Several Implementations of Two Recently Published Feature Detectors,” IFAC Proceedings Volumes, 40(15): 143–148.
Bondada V, Pratihar DK and Kumar CS (2018), “Detection and Quantitative Assessment of Corrosion on Pipelines Through Image Analysis,” Procedia Computer Science, 133: 804–811.
Burney SA and Tariq H (2014), “K-Means Cluster Analysis for Image Segmentation,” International Journal of Computer Applications, 96(4): 1–8.
Cha YJ, Choi W and Büyüköztürk O (2017), “Deep Learning-Based Crack Damage Detection Using Convolutional Neural Networks,” Computer-Aided Civil and Infrastructure Engineering, 32(5): 361–378.
Chen LC, Zhu Y, Papandreou G, Schroff F and Adam H (2018), “Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation,” Proceedings of the European Conference on Computer Vision (ECCV), pp. 801–818.
Cho KY and Kim S (2005), “Morphological Analysis and Classification of Types of Surface Corrosion Damage by Digital Image Processing,” Corrosion Science, 47(1): 1–15.
Crognale M, Gattulli V, Ivorra S and Potenza F (2020), “An Integrated Vibration-Image Procedure for Damage Identification in Steel Trusses,” Proceedings of the XI International Conference on Structural Dynamic, EURODYN, 23–25 November, 2020, 1: 1011–1026.
Deng W, Mou Y, Kashiwa T, Escalera S, Nagai K, Nakayama K, Matsuo Y and Prendinger H (2020), “Vision Based Pixel-Level Bridge Structural Damage Detection Using a Link ASPP Network,” Automation in Construction, 110: 102973.
Domaneschi M, Cimellaro G, De Iuliis M and Marano G (2021), “Laboratory Investigation of Digital Image Correlation Techniques for Structural Assessment,” Bridge Maintenance, Safety, Management, Life-Cycle Sustainability and Innovations, CRC Press, pp. 3260–3266.
Dung CV (2019), “Autonomous Concrete Crack Detection Using Deep Fully Convolutional Neural Network,” Automation in Construction, 99: 52–58.
Farhang SH, Rezaifar O, Sharbatdar MK and Ahmady Fard A (2021), “Evaluation of Different Methods of Machine Vision in Health Monitoring and Damage Detection of Structures,” Journal of Rehabilitation in Civil Engineering, 9(4): 93–132.
Farrar CR and Worden K (2012), Structural Health Monitoring: A Machine Learning Perspective, John Wiley & Sons, USA.
Feng X, Xiao L, Li W, Pei L, Sun Z, Ma Z, Shen H and Ju H (2020), “Pavement Crack Detection and Segmentation Method Based on Improved Deep Learning Fusion Model,” Mathematical Problems in Engineering, 2020(1):1–22.
Gavilán M, Balcones D, Marcos O, Llorca DF, Sotelo MA, Parra I, Ocaña M, Aliseda P, Yarza P and Amírola A (2011), “Adaptive Road Crack Detection System by Pavement Classification,” Sensors, 11(10): 9628–9657.
Hallermann N and Morgenthal G (2014), “Visual Inspection Strategies for Large Bridges Using Unmanned Aerial Vehicles (UAV),” Proc. of 7th IABMAS, International Conference on Bridge Maintenance, Safety and Management, pp. 661–667.
He K, Gkioxari G, Dollar P and Girshick R (2017), “Mask R-CNN,” Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969.
Housner G, Bergman LA, Caughey TK, Chassiakos AG, Claus RO, Masri SF, Skelton RE, Soong T, Spencer B and Yao JT (1997), “Structural Control: Past, Present, and Future,” Journal of Engineering Mechanics, 123(9): 897–971.
Hu Y and Zhao CX (2010), “A Novel LBP Based Methods for Pavement Crack Detection,” Journal of Pattern Recognition Research, 5(1): 140–147.
Jáuregui DV and White KR (2003), “Implementation of Virtual Reality in Routine Bridge Inspection,” Transportation Research Record, 1827(1): 29–35.
Jin Lim H, Hwang S, Kim H and Sohn H (2021), “Steel Bridge Corrosion Inspection with Combined Vision and Thermographic Images,” Structural Health Monitoring, 20(6): 3424–3435.
Karaaslan E, Bagci U and Catbas FN (2021), “Attention-Guided Analysis of Infrastructure Damage with Semi-Supervised Deep Learning,” Automation in Construction, 125: 103634.
Kaseko MS and Ritchie SG (1993), “A Neural Network-Based Methodology for Pavement Crack Detection and Classification,” Transportation Research Part C: Emerging Technologies, 1(4): 275–291.
Kato Z and Zerubia J (2012), “Markov Random Fields in Image Segmentation,” Foundations and Trends in Signal Processing, 5(1–2): 1–155.
Kazemi Majd F, Fallahi N and Gattulli V (2021), “Detection of Corrosion Defects in Steel Bridges by Machine Vision,” Proceedings of International Conference of the European Association on Quality Control of Bridges and Structures, Springer, pp. 830–834.
Kim B and Cho S (2019), “Image-Based Concrete Crack Assessment Using Mask and Region-Based Convolutional Neural Network,” Structural Control and Health Monitoring, 26(8): e2381.
Koch C, Georgieva K, Kasireddy V, Akinci B and Fieguth P (2015), “A Review on Computer Vision Based Defect Detection and Condition Assessment of Concrete and Asphalt Civil Infrastructure,” Advanced Engineering Informatics, 29(2): 196–210.
Lee BJ and Lee HD (2004), “Position-Invariant Neural Network for Digital Pavement Crack Analysis,” Computer-Aided Civil and Infrastructure Engineering, 19(2): 105–118.
Lee S and Kalos N (2015), “Bridge Inspection Practices Using Non-Destructive Testing Methods,” Journal of Civil Engineering and Management, 21(5): 654–665.
Lei X, Liu C, Li L and Wang G (2020), “Automated Pavement Distress Detection and Deterioration Analysis Using Street View Map,” IEEE Access, 8: 76163–76172.
Lin (2022), “Image Segmentation Based on Markov Random Fields,” https://www.mathworks.com/matlabcentral/fileexchange/33592-image-segmentation-based-on-markov-random-fields.
Lohade DM and Chopade P (2016), “Real Time Metal Inspection for Surface and Dimensional Defect Detection Using Image Processing Techniques,” Proceedings of 3rd International Conference on Electrical, Electronics, Engineering Trends, Communication, Optimization and Sciences (EEECOS-2016), pp. 873–877.
Long J, Shelhamer E and Darrell T, “Fully Convolutional Networks for Semantic Segmentation,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440.
Mathavan S, Rahman M and Kamal K (2015), “Use of a Self-Organizing Map for Crack Detection in Highly Textured Pavement Images,” Journal of Infrastructure Systems, 21(3): 1–11.
Nguyen TS, Begot S, Duculty F, Bardet JC and Avila M (2010), “Pavement Cracking Detection Using an Anisotropy Measurement,” Proceedings of 11th IASTED International Conference on Computer Graphics and Imaging (CGIM), pp. 80–87.
Oliveira H and Correia PL (2010), “Automatic Crack Detection on Road Imagery Using Anisotropic Diffusion and Region Linkage,” Proceedings of 2010 18th European Signal Processing Conference, IEEE, pp. 274–278.
Otsu N (1979), “A Threshold Selection Method from Gray-Level Histograms,” IEEE Transactions on Systems, Man, and Cybernetics, 9(1): 62–66.
Park SE, Eem SH and Jeon H (2020), “Concrete Crack Detection and Quantification Using Deep Learning and Structured Light,” Construction and Building Materials, 252: 119096.
Potenza F, Federici F, Lepidi M, Gattulli V, Graziosi F and Colarieti A (2015), “Long-Term Structural Monitoring of the Damaged Basilica S. Maria di Collemaggio Through a Low-Cost Wireless Sensor Network,” Journal of Civil Structural Health Monitoring, 5(5): 655–676.
Potenza F, Rinaldi C, Ottaviano E and Gattulli V (2020), “A Robotics and Computer-Aided Procedure for Defect Evaluation in Bridge Inspection,” Journal of Civil Structural Health Monitoring, 10(3): 471–484.
Qidwai U and Chen CH (2009), Digital Image Processing: An Algorithmic Approach with MATLAB, Chapman and Hall/CRC, UK.
Ranjan R and Gulati T (2014), “Condition Assessment of Metallic Objects Using Edge Detection,” Int. J. Adv. Res. Comput. Sci. Softw. Eng, 4(5): 253–258.
Ren S, He K, Girshick R and Sun J (2017), “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks,” IEEE Transactions on Advances in Neural Information Processing Systems Machine Intelligence, 39(6): 1137–1149. DOI: https://doi.org/10.1109/TPAMI.2016.2577031
Silva LA, Sanchez San Blas H, Peral García D, Sales Mendes A and Villarubia González G (2020), “An Architectural Multi-Agent System for a Pavement Monitoring System with Pothole Recognition in UAV Images,” Sensors, 20(21): 6205.
Silva WRLD and Lucena DSD (2018), “Concrete Cracks Detection Based on Deep Learning Image Classification,” Proceedings of the Eighteenth International Conference of Experimental Mechanics, MDPI, 2(8): 489.
Sinaga KP and Yang MS (2020), “Unsupervised K-Means Clustering Algorithm,” IEEE Access, 8: 80716–80727.
Solomon C and Breckon T (2011), Fundamentals of Digital Image Processing: A Practical Approach with Examples in Matlab, John Wiley & Sons, USA.
Sun Y, Yang Y, Yao G, Wei F and Wong M (2021), “Autonomous Crack and Bughole Setection for Concrete Surface Image Based on Deep Learning,” IEEE Access, 9: 85709–85720.
Tanaka N and Uematsu K (1998), “A Crack Detection Method in Road Surface Images Using Morphology,” MVA, 98: 17–19.
Yamane T and Chun PJ (2020), “Crack Detection from a Concrete Surface Image Based on Semantic Segmentation Using Deep Learning,” Journal of Advanced Concrete Technology, 18(9): 493–504.
Yeum CM and Dyke SJ (2015), “Vision-Based Automated Crack Detection for Bridge Inspection,” Computer-Aided Civil and Infrastructure Engineering, 30(10): 759–770.
Ying L and Salari E (2010), “Beamlet Transform-Based Technique for Pavement Crack Detection and Classification,” Computer-Aided Civil and Infrastructure Engineering, 25(8): 572–580.
Zalama E, Gómez-García-Bermejo J, Medina R and Llamas J (2014), “Road Crack Detection Using Visual Features Extracted by Gabor Filters,” Computer-Aided Civil and Infrastructure Engineering, 29(5): 342–358.
Zhang C, Chang CC and Jamshidi M (2020), “Concrete Bridge Surface Damage Detection Using a Single-Stage Detector,” Computer-Aided Civil and Infrastructure Engineering, 35(4): 389–409.
Acknowledgment
Part of the research leading to these results has received funding from the research project DESDEMONA – Detection of Steel Defects by Enhanced MONitoring and Automated procedure for self-inspection and maintenance (grant agreement number RFCS-2018_800687) supported by EU Call RFCS-2017. Other funding resources was in part sponsored by the NATO Science for Peace and Security Programme under grant id. G5924.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Crognale, M., De Iuliis, M., Rinaldi, C. et al. Damage detection with image processing: a comparative study. Earthq. Eng. Eng. Vib. 22, 333–345 (2023). https://doi.org/10.1007/s11803-023-2172-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11803-023-2172-1