Skip to main content
SpringerLink
Log in

Deep Learning-Based Crack Detection: A Survey

  • Original Research Paper
  • Published:
International Journal of Pavement Research and Technology Aims and scope Submit manuscript

Abstract

Cracks are an acute distress in an asphalt pavement, which must be detected and quantified to diagnose the pavement’s health. Hence, many researchers have developed methods to detect cracks based on three main techniques: image processing, machine learning (ML), and deep learning (DL). Among these three techniques, DL has been recognised as an excellent method for crack detection because it assures high accuracy with an adequate analysis time. However, choosing an appropriate DL algorithm to identify cracks in an asphalt pavement is challenging for both transportation agencies and researchers. This study has identified the bigger picture of DL methods for crack identification in asphalt pavement. The authors evaluated several DL-based crack identification algorithms from the literature, such as crack classification, crack object detection, pixel-level crack segmentation, generative adversarial networks (GANs) for crack segmentation, and crack identification using unsupervised learning. Moreover, 26 DL-based crack detection models (25 supervised learning models and one unsupervised learning model) were analysed on the same dataset to test the performance of each model using consistent assessment metrics. The testing results suggest that ResNet and DenseNet are the best options for crack classification, while Faster R-CNN should be used for crack object detection and pix2pix is suggested for crack segmentation. It is also recommended that semi-supervised and unsupervised learning be further studied to efficiently detect cracks in an asphalt pavement.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Zimmerman, K. A. (2017). Pavement management systems: Putting data to work. NCHRP Synthesis of Highway Practice. Transportation Research Board. Project 20-05, Topic 47-08.

  2. ASTM, D. (2011). Standard practice for roads and parking lots pavement condition index surveys. ASTM International.

    Google Scholar 

  3. Haas, R., & Hudson, W. R. (1978). Pavement management systems. Monograph.

    Google Scholar 

  4. Subirats, P., Dumoulin, J., Legeay, V., & Barba, D. (2006). Automation of pavement surface crack detection using the continuous wavelet transform. In Proc. 2006 International Conference on Image Processing. IEEE. pp. 3037–3040. Atlanta, GA, USA.

  5. Ahmed, N. B. C., Lahouar, S., Souani, C., & Besbes, K. (2017). Automatic crack detection from pavement images using fuzzy thresholding. In Proc. International conference on control, automation and diagnosis (ICCAD). IEEE. pp. 528–537. Hammamet, Tunisia.

  6. Ying, L., & Salari, E. (2010). Beamlet transform-based technique for pavement crack detection and classification. Computer-Aided Civil and Infrastructure Engineering. Wiley Online Library, 25, 572–580.

    Google Scholar 

  7. Zhu, S., Xia, X., Zhang, Q., & Belloulata, K. (2007). An image segmentation algorithm in image processing based on threshold segmentation. In Proc. 2007 third international IEEE conference on signal-image technologies and internet-based system. IEEE. pp. 673–678. Shanghai, China.

  8. Oliveira, H., & Correia, P. L. (2009). Automatic road crack segmentation using entropy and image dynamic thresholding. In Proc. 17th European signal processing conference. IEEE. pp. 622–626. Glasgow, Scotland.

  9. Zhao, H., Qin, G., & Wang, X. (2010). Improvement of canny algorithm based on pavement edge detection. In Proc. 2010 3rd International congress on image and signal processing. IEEE. pp. 964–967. Yantai, China.

  10. Cao, W., Liu, Q., & He, Z. (2020). Review of pavement defect detection methods. IEEE Access, 8(19313368), 14531–14544.

    Article  Google Scholar 

  11. Hoang, N. D., Nguyen, Q. L., & Tien Bui, D. (2018). Image processing-based classification of asphalt pavement cracks using support vector machine optimized by artificial bee colony. Journal of Computing in Civil Engineering, 32(5), 04018037.

    Article  Google Scholar 

  12. Bhat, S., Naik, S., Gaonkar, M., Sawant, P., Aswale, S., & Shetgaonkar, P. (2020). A survey on road crack detection techniques. In Proc. 2020 International conference on emerging trends in information technology and engineering (ic-ETITE). IEEE. pp. 1–6. Vellore, India, India.

  13. Wang, J., Wan, K., Gao, X., Cheng, X., Shen, Y., Wen, Z., & Piran, M. J. (2020). Energy and materials-saving management via deep learning for wastewater treatment plants. IEEE Access, 8, 191694–191705.

    Article  Google Scholar 

  14. Butt, U. A., Mehmood, M., Shah, S. B. H., Amin, R., Shaukat, M. W., Raza, S. M., & Piran, M. (2020). A review of machine learning algorithms for cloud computing security. Electronics. Multidisciplinary Digital Publishing Institute, 9, 1379.

    Google Scholar 

  15. Goodfellow, I., Yoshua, B., & Aaron, C. (2016). Deep learning (pp. 326–329), MIT press.

    MATH  Google Scholar 

  16. LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989). Backpropagation applied to handwritten zip code recognition. Neural Computation, 1(4), 541–551.

    Article  Google Scholar 

  17. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84–90.

    Article  Google Scholar 

  18. Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.

  19. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., et al. (2015). Going deeper with convolutions. In Proc. Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1–9. Boston, MA, USA.

  20. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proc. Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778.

  21. Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017). Densely connected convolutional networks. In Proc. Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 4700–4708. Honolulu, HI, USA.

  22. Wang, X., & Hu, Z. (2017). Grid-based pavement crack analysis using deep learning. In Proc. 2017 4th international conference on transportation information and safety (ICTIS). IEEE. pp. 917–924.

  23. Chen, F. C., & Jahanshahi, M. R. (2017). NB-CNN: Deep learning-based crack detection using convolutional neural network and Naïve Bayes data fusion. IEEE Transactions on Industrial Electronics, 65(5), 4392–4400.

    Article  Google Scholar 

  24. Li, B., Wang, K. C., Zhang, A., Yang, E., & Wang, G. (2020). Automatic classification of pavement crack using deep convolutional neural network. International Journal of Pavement Engineering, 21(4), 457–463.

    Article  Google Scholar 

  25. Girshick, R. (2015). Fast r-cnn. In Proceedings of the IEEE international conference on computer vision. pp. 1440–1448.

  26. Ren, S., He, K., Girshick, R., & Sun, J. (2017). Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031.

    Article  Google Scholar 

  27. Redmon, J., & Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767.

  28. He, K., Gkioxari, G., Dollár, P., & Girshick, R. (2017). Mask r-cnn. In PProc. Proceedings of the IEEE international conference on computer vision. pp. 2961–2969.

  29. Li, S., Gu, X., Xu, X., Xu, D., Zhang, T., Liu, Z., & Dong, Q. (2021). Detection of concealed cracks from ground penetrating radar images based on deep learning algorithm. Construction and Building Materials, 273, 121949.

    Article  Google Scholar 

  30. Chen, F. & Jahanshahi, M. R. (2018). NB-CNN: Deep learning-based crack detection using convolutional neural network and Naïve Bayes data fusion. IEEE Transactions on Industrial Electronics, 65(5), 4392–4400. https://doi.org/10.1109/TIE.2017.2764844.

    Article  Google Scholar 

  31. Gou, C., Peng, B., Li, T., & Gao, Z. (2019). Pavement crack detection based on the improved Faster-RCNN. In Proc. 2019 IEEE 14th international conference on intelligent systems and knowledge engineering (ISKE). IEEE. pp. 962–967. Dalian, Liaoning, China.

  32. Du, Y., Pan, N., Xu, Z., Deng, F., Shen, Y., & Kang, H. (2020). Pavement distress detection and classification based on YOLO network. International Journal of Pavement Engineering. https://doi.org/10.1080/10298436.2020.1714047.

    Article  Google Scholar 

  33. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In Proc. International Conference on Medical image computing and computer-assisted intervention. pp. 234–241. Munich, Germany.

  34. Isola, P., Zhu, J. Y., Zhou, T., & Efros, A. A. (2017). Image-to-image translation with conditional adversarial networks. In Proc. Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1125–1134.

  35. Zhang, K., Zhang, Y., & Cheng, H. D. (2020). CrackGAN: Pavement crack detection using partially accurate ground truths based on generative adversarial learning. IEEE Transactions on Intelligent Transportation Systems,22(2), 1306–1319.

    Article  Google Scholar 

  36. Kim, B., & Cho, S. (2019). Image-based concrete crack assessment using mask and region-based convolutional neural network. Structural Control and Health Monitoring, 26(8), 2381.

    Article  Google Scholar 

  37. Zhang, K., Zhang, Y., & Cheng, H. D. (2020). Self-supervised structure learning for crack detection based on cycle-consistent generative adversarial networks. Journal of Computing in Civil Engineering, 34(3), 04020004.

    Article  Google Scholar 

  38. Li, G., Wan, J., He, S., Liu, Q., & Ma, B. (2020). Semi-supervised semantic segmentation using adversarial learning for pavement crack detection. IEEE Access, 8, 51446–51459.

    Article  Google Scholar 

  39. Wang, W., Wang, M., Li, H., Zhao, H., Wang, K., He, C., & Chen, J. (2019). Pavement crack image acquisition methods and crack extraction algorithms: A review. Journal of Traffic and Transportation Engineering, 6(6), 535–556.

    Google Scholar 

  40. Cao, W., Liu, Q., & He, Z. (2020). Review of pavement defect detection methods. IEEE Access, 8, 14531–14544.

    Article  Google Scholar 

  41. Hsieh, Y. A., & Tsai, Y. J. (2020). Machine learning for crack detection: Review and model performance comparison. Journal of Computing in Civil Engineering, 34(5), 04020038.

    Article  Google Scholar 

  42. Miller, J. S., & Bellinger, W. Y. (2003). Distress Identification Manual for the Long-Term Pavement Performance Program (Fourth Revised Edition). Office of Infrastructure Research and Development,Federal Highway Administration. FHWA-RD-03-031.

  43. Zhang, L., Yang, F., Zhang, Y. D., & Zhu, Y. J. (2016). Road crack detection using deep convolutional neural network. In Proc. 2016 IEEE international conference on image processing (ICIP). IEEE. pp. 3708–3712. Phoenix, AZ, USA.

  44. Maeda, H., Sekimoto, Y., Seto, T., Kashiyama, T., & Omata, H. (2018). Road damage detection using deep neural networks with images captured through a smartphone. arXiv preprint arXiv:1801.09454.

  45. Yusof, N. A. M., Ibrahim, A., Noor, M. H. M., Tahir, N. M., Yusof, N. M., Abidin, N. Z., & Osman, M. K. (2019). Deep convolution neural network for crack detection on asphalt pavement. In Proc. Journal of Physics: Conference Series. IOP Publishing. pp. 012020. Penang Island, Malaysia.

  46. Majidifard, H., Jin, P., Adu-Gyamfi, Y., & Buttlar, W. G. (2019). PID: A new benchmark dataset to classify and densify pavement distresses. arXiv preprint arXiv:1910.11123.

  47. Shatnawi, N. (2018). Automatic pavement cracks detection using image processing techniques and neural network. International Journal of Advanced Computer Science and Applications (IJACSA), 9(9), 399–402.

    Google Scholar 

  48. Naddaf-Sh, M., SeyedSaeid H., Jing Z., Nicholas A. B., & Hassan Z. (2019). Real-time road crack mapping using an optimized convolutional neural network. Complexity, 2019 2470735.

    Article  Google Scholar 

  49. Tran, V. P., Tran, T. S., Lee, H. J., Kim, K. D., Baek, J., & Nguyen, T. T. (2020). One stage detector (RetinaNet)-based crack detection for asphalt pavements considering pavement distresses and surface objects. Journal of Civil Structural Health Monitoring,11(1), 205–222.

    Article  Google Scholar 

  50. Mei, Q., & Mustafa, G. (2020). A cost effective solution for pavement crack inspection using cameras and deep neural networks. Construction and Building Materials, 256, 119397.

    Article  Google Scholar 

  51. Zhang, A., Wang, K. C., Fei, Y., Liu, Y., Chen, C., Yang, G., & Qiu, S. (2018). Automated pixel-level pavement crack detection on 3D asphalt surfaces with a recurrent neural network. Computer-Aided Civil and Infrastructure Engineering, 34(3), 213–229.

    Article  Google Scholar 

  52. Wu, L., Mokhtari, S., Nazef, A., Nam, B., & Yun, H. B. (2019). Improvement of crack-detection accuracy using a novel crack defragmentation technique in image-based road assessment. Journal of Computing in Civil Engineering, 30(1), 04014118.

    Article  Google Scholar 

  53. Tsai, Y. J., Jiang, C., & Wang, Z. (2012). Pavement crack detection using high-resolution 3D line laser imaging technology. In Proc. 7th RILEM international conference on cracking in pavements. Springer, Dordrecht. pp. 169–178.

  54. Zhang, A., Wang, K. C., Li, B., Yang, E., Dai, X., Peng, Y., & Chen, C. (2017). Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network. Computer-Aided Civil and Infrastructure Engineering, 32(10), 805–819.

    Article  Google Scholar 

  55. Tong, Z., Gao, J., & Zhang, H. (2017). Recognition, location, measurement, and 3D reconstruction of concealed cracks using convolutional neural networks. Construction and Building Materials, 146, 775–787.

    Article  Google Scholar 

  56. Stehman, S. V. (1997). Selecting and interpreting measures of thematic classification accuracy. Remote Sensing of Environment, 62(1), 77–89.

    Article  Google Scholar 

  57. Yang, F., Zhang, L., Yu, S., Prokhorov, D., Mei, X., & Ling, H. (2020). Feature pyramid and hierarchical boosting network for pavement crack detection. IEEE Transactions on Intelligent Transportation Systems, 21, 1525–1535.

    Article  Google Scholar 

  58. Zhang, M., Liu, Y., Luo, S., & Gao, S. (2020). Research on Baidu Street View Road crack information extraction based on deep learning method. In Proc. Journal of physics: conference series. IOP Publishing. vol. 1616. pp. 012086. Kunming, China.

  59. Lin, T. Y., Goyal, P., Girshick, R., He, K., & Dollár, P. (2020). Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(2).

  60. Gibb, S., La, H. M., & Louis, S. (2018). A genetic algorithm for convolutional network structure optimization for concrete crack detection. In Proc. 2018 IEEE congress on evolutionary computation (CEC). IEEE. pp. 1–8.

  61. Li, S., & Zhao, X. (2018). Convolutional neural networks-based crack detection for real concrete surface. Sensors and Smart Structures Technologies for Civil, Mechanical, and Aerospace Systems 2018. Vol. 10598. International Society for Optics and Photonics.

  62. Li, Y., Li, H., & Wang, H. (2018). Pixel-wise crack detection using deep local pattern predictor for robot application. Sensors, 18(9), 3042.

    Article  Google Scholar 

  63. Cha, Y. J., Choi, W., & Büyüköztürk, O. (2017). Deep learning-based crack damage detection using convolutional neural networks. Computer-Aided Civil and Infrastructure Engineering, 32(5), 361–378.

    Article  Google Scholar 

  64. Pauly, L., Hogg, D., Fuentes, R., & Peel, H. (2017). Deeper networks for pavement crack detection. In Proc. Proceedings of the 34th ISARC. IAARC. pp. 479–485. Taipei, Taiwan.

  65. Park, J. K., Kwon, B. K., Park, J. H., & Kang, D. J. (2016). Machine learning-based imaging system for surface defect inspection. International Journal of Precision Engineering and Manufacturing-Green Technology., 3(3), 303–310.

    Article  Google Scholar 

  66. Zhang, K., Cheng, H. D., & Zhang, B. (2018). Unified approach to pavement crack and sealed crack detection using preclassification based on transfer learning. Journal of Computing in Civil Engineering. American Society of Civil Engineers, 32(2), 04018001.

    Article  MathSciNet  Google Scholar 

  67. Li, B., Wang, K. C., Zhang, A., Yang, E., & Wang, G. (2018). Automatic classification of pavement crack using deep convolutional neural network. International Journal of Pavement Engineering, 21(4), 457–463.

    Article  Google Scholar 

  68. Ahmed, TU., Hossain, MS., Alam, MJ., Andersson, K. (2019). An integrated CNN-RNN framework to assess road crack. In Proc. 2019 22nd international conference on computer and information technology (ICCIT). IEEE. pp. 1–6. Dhaka, Bangladesh, Bangladesh.

  69. Tran, T. S., Tran, V. P., Lee, H. J., Flores, J. M., & Le, V. P. (2020). A two-step sequential automated crack detection and severity classification process for asphalt pavements. International Journal of Pavement Engineering. https://doi.org/10.1080/10298436.2020.1836561.

    Article  Google Scholar 

  70. Gou, C., Peng, B., Li, T., & Gao, Z. (2019). Pavement crack detection based on the improved Faster-RCNN. In Proc. 2019 IEEE 14th international conference on intelligent systems and knowledge engineering (ISKE). IEEE. pp. 962–967. Dalian, China, China.

  71. Li, J., Zhao, X., & Li, H. (2019). Method for detecting road pavement damage based on deep learning. Health Monitoring of Structural and Biological Systems XIII. International Society for Optics and Photonics, 10972, 109722D.

  72. Du, Y., Pan, N., Xu, Z., Deng, F., Shen, Y., & Kang, H. (2021). Pavement distress detection and classification based on YOLO network. International Journal of Pavement Engineering, 22(13), 1659–1672.

    Article  Google Scholar 

  73. Mandal, V., Uong, L., & Adu-Gyamfi, Y. (2018). Automated road crack detection using deep convolutional neural networks. 2018 IEEE international conference on big data (big data). IEEE. pp. 5212–5215.

  74. Liu, Z., Cao, Y., Wang, Y., & Wang, W. (2019). Computer vision-based concrete crack detection using U-net fully convolutional networks. Automation in Construction, 104, 129–139.

    Article  Google Scholar 

  75. Liu, W., Huang, Y., Li, Y., & Chen, Q. (2019). FPCNet: Fast pavement crack detection network based on encoder-decoder architecture. arXiv preprint arXiv:1907.02248

  76. Wu, Y., Yang, W., Pan, J., & Chen, P. (2021). Asphalt pavement crack detection based on multi-scale full convolutional network. Journal of Intelligent and Fuzzy Systems, 40(1), 1495–1508.

    Article  Google Scholar 

  77. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., et al. (2014). Generative adversarial networks. arXiv preprint arXiv:1406.2661.

  78. Li, G., Wan, J., He, S., Liu, Q., & Ma, B. (2020). Semi-supervised semantic segmentation using ddversarial learning for pavement crack detection. IEEE Access, 8, 51446–51459.

    Article  Google Scholar 

  79. Shim, S., Kim, J., Cho, G. C., & Lee, S. W. (2020). Multi-scale and adversarial learning-based semi-supervised semantic segmentation approach for crack detection in concrete structures. IEEE Access, 8, 170939–170950.

    Article  Google Scholar 

  80. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C. Y., & Berg, A. C. (2016). Ssd: Single shot multibox detector. In Proc. European conference on computer vision. pp. 21–37. Amsterdam, The Netherlands.

Download references

Funding

This work was supported by the Korea Agency for Infrastructure Technology Advancement (KAIA) grant funded by the Ministry of Land, Infrastructure and Transport (Grant 21POQW-B152690-03) and Sejong University.

Author information

Authors and Affiliations

  1. Civil and Environmental Engineering Department, Sejong University, Seoul, 05006, South Korea

    Son Dong Nguyen, Thai Son Tran & Hyun Jong Lee

  2. IRIS Technology Company, 209 Neungdong-ro, Gwangjin-gu, Seoul, 05006, South Korea

    Van Phuc Tran

  3. Computer Science and Engineering Department, Sejong University, Seoul, 05006, South Korea

    Md. Jalil Piran

  4. Faculty of Civil Engineering, University of Transport and Communications, Ha Noi, Vietnam

    Van Phuc Le

Authors
  1. Son Dong Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thai Son Tran
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Van Phuc Tran
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hyun Jong Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Md. Jalil Piran
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Van Phuc Le
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Son Dong Nguyen.

Ethics declarations

Conflict of Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nguyen, S.D., Tran, T.S., Tran, V.P. et al. Deep Learning-Based Crack Detection: A Survey. Int. J. Pavement Res. Technol. 16, 943–967 (2023). https://doi.org/10.1007/s42947-022-00172-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42947-022-00172-z

Keywords

Search

Navigation