Skip to main content
SpringerLink
Log in

Synthesizing the performance of deep learning in vision-based pavement distress detection

  • Technical Paper
  • Published:
Innovative Infrastructure Solutions Aims and scope Submit manuscript

Abstract

Deep learning (DL) has proven its efficacy in extracting useful distress information from image-based data of infrastructure assets, such as pavements. Despite the overwhelming research on this topic, state-of-the-art DL approaches fail to perform satisfactorily on independent datasets as noted from an object detection-based competition. Besides, a lack of clarification in computing DL performance measures and inadequate discussion on DL implementation framework still exist. To this end, this paper contributes to the body of knowledge by synthesizing the performance of DL models from the existing relevant literature using the ‘random effect meta-analysis’ approach. Meta-analysis requires an estimate of the uncertainty in the reported performance measure (i.e., F1-score) to assign weights to individual studies and compute an overall performance measure for a group of studies. Hence, this paper introduces a statistical approach to calculate the uncertainty in the reported F1-score to compute the within-study variance. The methods, statistics, and results presented in this paper will help understand the requisites for future studies on DL in pavement distress evaluation, ultimately improving pavement asset management.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Herold M, Roberts D, Noronha V, Smadi O (2008) Imaging spectrometry and asphalt road surveys. Transp Res Part C Emerg Technol 16:153–166. https://doi.org/10.1016/j.trc.2007.07.001

    Article  Google Scholar 

  2. Bang S, Park S, Kim H, Kim H (2019) Encoder–decoder network for pixel-level road crack detection in black-box images. Comput Aided Civ Infrastruct Eng 34:713–727

    Article  Google Scholar 

  3. Zhang A, Wang KC, Fei Y et al (2019) Automated pixel-level pavement crack detection on 3D asphalt surfaces with a recurrent neural network. Comput Aided Civ Infrastruct Eng 34:213–229

    Article  Google Scholar 

  4. Huyan J, Li W, Tighe S et al (2020) CrackU-net: a novel deep convolutional neural network for pixelwise pavement crack detection. Struct Control Health Monit 27:2551

    Article  Google Scholar 

  5. Hascoet T, Zhang Y, Persch A, et al (2020) FasterRCNN monitoring of road damages: competition and deployment. In: 2020 IEEE International conference on big data (Big Data. IEEE, pp 5545–5552

  6. Naddaf-Sh S, Naddaf-Sh MM, Kashani AR, Zargarzadeh H (2020) An efficient and scalable deep learning approach for road damage detection. In: 2020 IEEE International conference on big data big data. IEEE, pp 5602–5608

  7. Mandal V, Mussah AR, Adu-Gyamfi Y (2020) Deep learning frameworks for pavement distress classification: A comparative analysis. In: 2020 IEEE international conference on big data Big Data. IEEE, pp 5577–5583

  8. Arya D, Maeda H, Ghosh SK, et al (2020) Global road damage detection: State-of-the-art solutions. In: 2020 IEEE international conference on big data (Big Data. IEEE, pp 5533–5539

  9. Hsieh YA, Tsai YJ (2020) Machine learning for crack detection: review and model performance comparison. J Comput Civ Eng 34:04020038

    Article  Google Scholar 

  10. Hoang ND (2018) Detection of surface crack in building structures using image processing technique with an improved Otsu method for image thresholding. Adv Civ Eng. https://doi.org/10.1155/2018/3924120

    Article  Google Scholar 

  11. Wang W, Su C (2020) Convolutional neural network-based pavement crack segmentation using pyramid attention network. IEEE Access 8:206548–206558

    Article  Google Scholar 

  12. Mathavan S, Vaheesan K, Kumar A et al (2017) Detection of pavement cracks using tiled fuzzy Hough transform. J Electron Imag 26:053008

    Article  Google Scholar 

  13. Ghosh R, Smadi O (2021) Automated detection and classification of pavement distresses using 3D pavement surface images and deep learning. Transp Res Record. 03611981211007481

  14. Yamashita R, Nishio M, Do RKG, Togashi K (2018) Convolutional neural networks: an overview and application in radiology. Insights Imaging 9:611–629

    Article  Google Scholar 

  15. Tang S, Chen Z (2020) Scale-space data augmentation for deep transfer learning of crack damage from small sized datasets. J Nondestr Eval 39:1–18

    Article  Google Scholar 

  16. Abdellatif M, Peel H, Cohn AG, Fuentes R (2021) Combining block-based and pixel-based approaches to improve crack detection and localisation. Autom Constr 122:103492

    Article  Google Scholar 

  17. Bolya D, Foley S, Hays J, Hoffman J (2020) Tide: a general toolbox for identifying object detection errors. In: Computer Vision–ECCV 2020: 16th European Conference. Springer, Glasgow, UK, pp 16 558–573

  18. Zhang Y, Chen B, Wang J et al (2020) APLCNet: automatic pixel-level crack detection network based on instance segmentation. IEEE Access 8:199159–199170

    Article  Google Scholar 

  19. Tiu E (2019) Metrics to evaluate your semantic segmentation model|by Ekin Tiu|Towards Data Science. https://towardsdatascience.com/metrics-to-evaluate-your-semantic-segmentation-model-6bcb99639aa2. Accessed 20 Aug 2023

  20. Zhang K, Zhang Y, Cheng HD (2020) Self-supervised structure learning for crack detection based on cycle-consistent generative adversarial networks. J Comput Civ Eng 34:04020004

    Article  Google Scholar 

  21. Mei Q, Gül M (2020) A cost effective solution for pavement crack inspection using cameras and deep neural networks. Constr Build Mater 256:119397. https://doi.org/10.1016/j.conbuildmat.2020.119397

    Article  Google Scholar 

  22. Lang H, Lu JJ, Lou Y, Chen S (2020) Pavement cracking detection and classification based on 3D image using multiscale clustering model. J Comput Civ Eng 34:04020034

    Article  Google Scholar 

  23. Pan Y, Chen X, Sun Q, Zhang X (2021) Monitoring asphalt pavement aging and damage conditions from low-altitude UAV imagery based on a CNN approach. Can J Remote Sens 47:432–449

    Article  Google Scholar 

  24. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst 28:91–99

    Google Scholar 

  25. Tran TS, Tran VP, Lee HJ et al (2020) A two-step sequential automated crack detection and severity classification process for asphalt pavements. Int J Pav Eng 23:2019

    Article  Google Scholar 

  26. Lõuk R, Riid A, Pihlak R, Tepljakov A (2020) Pavement defect segmentation in orthoframes with a pipeline of three convolutional neural networks. Algorithms 13:198

    Article  Google Scholar 

  27. Alfarraj O (2020) Internet of things with bio-inspired co-evolutionary deep-convolution neural-network approach for detecting road cracks in smart transportation. Neural Comput Appl.

  28. Feng X, Xiao L, Li W et al (2020) Pavement crack detection and segmentation method based on improved deep learning fusion model. Math Prob Eng. https://doi.org/10.1155/2020/8515213

    Article  Google Scholar 

  29. Maeda H, Sekimoto Y, Seto T et al (2018) Road damage detection and classification using deep neural networks with smartphone images. Comput Aided Civ Infrastruct Eng 33:1127–1141

    Article  Google Scholar 

  30. Borenstein M, Hedges LV, Higgins JP, Rothstein HR (2021) Introduction to meta-analysis, 2nd edn. Wiley

    Book  Google Scholar 

  31. Varghese V, Chikaraishi M, Urata J (2020) Deep learning in transport studies: a meta-analysis on the prediction accuracy. J Big Data Anal Transp. 2:199

    Article  Google Scholar 

  32. Bashar MZ, Torres-Machi C (2021) Performance of machine learning algorithms in predicting the pavement international roughness index. Transp Res Record 2675:226

    Article  Google Scholar 

  33. Papadimitriou E, Theofilatos A (2017) Meta-analysis of crash-risk factors in freeway entrance and exit areas. J Transp Eng Part A Syst 143:04017050

    Article  Google Scholar 

  34. Alruqi WM, Hallowell MR (2019) Critical success factors for construction safety: review and meta-analysis of safety leading indicators. J Constr Eng Manag 145:04019005

    Article  Google Scholar 

  35. Elvik R (2005) Introductory guide to systematic reviews and meta-analysis. Transp Res Rec 1908:230–235

    Article  Google Scholar 

  36. Deeks JJ, Altman DG (2001) Effect measures for meta-analysis of trials with binary outcomes. Syst Rev Health Care Meta-Anal Context. https://doi.org/10.1002/9780470693926.ch16

    Article  Google Scholar 

  37. Zhang X, Rajan D, Story B (2019) Concrete crack detection using context-aware deep semantic segmentation network. Comput Aided Civ Infrastruct Eng 34:951–971

    Article  Google Scholar 

  38. Liu J, Yang X, Lau S et al (2020) Automated pavement crack detection and segmentation based on two-step convolutional neural network. Comput Aided Civ Infrastruct Eng 35:1291–1305

    Article  Google Scholar 

  39. Majidifard H, Jin P, Adu-Gyamfi Y, Buttlar WG (2020) Pavement image datasets: a new benchmark dataset to classify and densify pavement distresses. Transp Res Rec 2674:328–339

    Article  Google Scholar 

  40. Peraka NSP, Biligiri KP, Kalidindi SN (2021) Development of a multi-distress detection system for asphalt pavements: transfer learning-based approach. Transp Res Record 75:538

    Article  Google Scholar 

  41. Roberts R, Giancontieri G, Inzerillo L, Mino G (2020) Towards low-cost pavement condition health monitoring and analysis using deep learning. Appl Sci 10:319

    Article  Google Scholar 

  42. Ibragimov E, Lee HJ, Lee JJ, Kim N (2020) Automated pavement distress detection using region based convolutional neural networks. Int J Pav Eng 23:1981

    Article  Google Scholar 

  43. Lei X, Liu C, Li L, Wang G (2020) Automated pavement distress detection and deterioration analysis using street view map. IEEE Access 8:76163–76172

    Article  Google Scholar 

  44. Chen SY, Zhang Y, Zhang YH et al (2019) Embedded system for road damage detection by deep convolutional neural network. Math Biosci Eng MBE 16:7982–7994

    Article  Google Scholar 

  45. Zhang K, Cheng HD, Zhang B (2018) Unified approach to pavement crack and sealed crack detection using pre-classification based on transfer learning. J Comput Civ Eng 32:04018001

    Article  Google Scholar 

  46. Yu B, Meng X, Yu Q (2021) Automated pixel-wise pavement crack detection by classification-segmentation networks. J Transp Eng Part B Pav 147:04021005

    Article  Google Scholar 

  47. Qiao W, Liu Q, Wu X et al (2021) Automatic pixel-level pavement crack recognition using a deep feature aggregation segmentation network with a scSE attention mechanism module. Sensors 21:2902

    Article  Google Scholar 

  48. Fei Y, Wang KC, Zhang A et al (2019) Pixel-level cracking detection on 3D asphalt pavement images through deep-learning-based CrackNet-V. IEEE Trans Intell Transp Syst 21:273–284

    Article  Google Scholar 

  49. Yu Y, Guan H, Li D et al (2022) CCapFPN: a context-augmented capsule feature pyramid network for pavement crack detection. IEEE Trans Intell Transp Syst 23:3324–3335. https://doi.org/10.1109/TITS.2020.3035663

    Article  Google Scholar 

  50. Qu Z, Mei J, Liu L, Zhou DY (2020) Crack detection of concrete pavement with cross-entropy loss function and improved VGG16 network model. IEEE Access 8:54564–54573

    Article  Google Scholar 

  51. Chun C, Ryu SK (2019) Road surface damage detection using fully convolutional neural networks and semi-supervised learning. Sensors 19:5501

    Article  Google Scholar 

  52. Jia G, Song W, Jia D, Zhu H (2019) Sample generation of semi-automatic pavement crack labelling and robustness in detection of pavement diseases. Electron Lett 55:1235–1238

    Article  Google Scholar 

  53. Lajeunesse MJ (2021) Fixed effect, homogeneity tests, and random-effects meta-analysis in Microsoft Excel. figshare. Online resource. https://doi.org/10.6084/m9.figshare.14138087.v1

Download references

Author information

Authors and Affiliations

  1. Department of Civil, Construction and Environmental Engineering, Iowa State University, 813 Bissell Rd, Ames, IA, 50011, USA

    Zia U. A. Zihan

  2. Department of Civil, Construction and Environmental Engineering, Center for Transportation Research and Education, Iowa State University, 374 Town Engineering Building, Ames, IA, 50011, USA

    Omar Smadi

  3. Department of Statistics, Iowa State University, 2438 Osborn Dr, Ames, IA, 50011, USA

    Miranda Tilberg

  4. Department of Construction Engineering, Faculty of Engineering, Zagazig University, Zagazig, 44519, Egypt

    Mohamed S. Yamany

  5. Iowa State University, Ames, IA50011, USA

    Mohamed S. Yamany

Authors
  1. Zia U. A. Zihan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Omar Smadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Miranda Tilberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mohamed S. Yamany
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohamed S. Yamany.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

For this type of study, formal consent is not required.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zihan, Z.U.A., Smadi, O., Tilberg, M. et al. Synthesizing the performance of deep learning in vision-based pavement distress detection. Innov. Infrastruct. Solut. 8, 299 (2023). https://doi.org/10.1007/s41062-023-01250-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s41062-023-01250-2

Keywords

Search

Navigation