Skip to main content
SpringerLink
Log in

A Review of the Application of CNN-Based Computer Vision in Civil Infrastructure Maintenance

  • Conference paper
  • First Online:
Proceedings of the 25th International Symposium on Advancement of Construction Management and Real Estate (CRIOCM 2020)

Abstract

Computer-vision and deep-learning techniques are being increasingly applied to the maintenance of civil infrastructure, such as inspecting, monitoring, and assessing infrastructure conditions, which overcome time-consuming and laborious compared with traditional technology. In this paper, the research progress of deep learning, the developments of convolutional neural network (CNN)-based computer vision in improving accuracy, reliability and generalized object detection capability and its application in civil infrastructure maintenance are reviewed. The main objectives are as follows: (1) clarify the application of deep learning in computer vision to help researchers systematically understand deep learning; (2) review the application of computer vision in civil infrastructure maintenance to help researchers pay more attention to its advantages; (3) encourage relevant personnel to use this research as a reference, take deep learning as an important method at the forefront of engineering management, generate more innovations in the construction field, and promote the development of the construction industry.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 329.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Perez, H., Tah, J. H. M., & Mosavi, A. (2019). Deep learning for detecting building defects using convolutional neural networks. Sensors, 19(16), 22.

    Article  Google Scholar 

  2. Voulodimos, A., et al. (2018). Deep learning for computer vision: A brief review. Computational Intelligence and Neuroscience, 13.

    Google Scholar 

  3. Zhong, B. T., et al. (2019). Mapping computer vision research in construction: Developments, knowledge gaps and implications for research. Automation in Construction, 107.

    Google Scholar 

  4. Zhou, S. L., & Song, W. (2020). Deep learning-based roadway crack classification using laser-scanned range images: A comparative study on hyperparameter selection. Automation in Construction, 114, 17.

    Article  Google Scholar 

  5. Cheng, J. C. P., & Wang, M. Z. (2018). Automated detection of sewer pipe defects in closed-circuit television images using deep learning techniques. Automation in Construction, 95, 155–171.

    Article  Google Scholar 

  6. Wang, M. Z., & Cheng, J. C. P. (2020). A unified convolutional neural network integrated with conditional random field for pipe defect segmentation. Computer-Aided Civil and Infrastructure Engineering, 35(2), 162–177.

    Article  Google Scholar 

  7. Cha, Y. J., Choi, W., & Buyukozturk, O. (2017). Deep learning-based crack damage detection using convolutional neural networks. Computer-Aided Civil and Infrastructure Engineering, 32(5), 361–378.

    Article  Google Scholar 

  8. Lee, K. H., Byun, N., & Shin, D. (2020). A damage localization approach for Rahmen Bridge based on convolutional neural network. Ksce Journal of Civil Engineering, 24(1), 1–9.

    Article  Google Scholar 

  9. Zhang, J., et al. (2020). Automatic detection of moisture damages in asphalt pavements from GPR data with deep CNN and IRS method. Automation in Construction, 113, 11.

    Article  Google Scholar 

  10. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the Acm, 60(6), 84–90.

    Article  Google Scholar 

  11. Hubel, D. H., & Wiesel, T. N. (1962). Receptive fields, binocular interaction and functional architecture in cats visual cortex. Journal of Physiology-London, 160(1), 106.

    Google Scholar 

  12. Wang, M. Z., & Cheng, J. C. P. (2018). Development and improvement of deep learning based automated defect detection for sewer pipe inspection using faster R-CNN. In I. F. C. Smith & B. Domer (Eds.), Advanced computing strategies for engineering, Pt Ii (pp. 171–192). Springer International Publishing Ag.

    Chapter  Google Scholar 

  13. Guo, Y. M., et al. (2016). Deep learning for visual understanding: A review. Neurocomputing, 187, 27–48.

    Article  Google Scholar 

  14. Wu, H., Liu, Q., & Liu, X. D. (2019). A review on deep learning approaches to image classification and object segmentation. Cmc-Computers Materials and Continua, 60(2), 575–597.

    Article  Google Scholar 

  15. Simonyan, K., & Zisserman, A. J. A. E.-P. (2014). Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556.

  16. Szegedy, C., et al. (2015). Going deeper with convolutions. 2015 Ieee Conference on Computer Vision and Pattern Recognition (pp. 1–9). Ieee.

    Google Scholar 

  17. Kaiming, H., et al. (2015). Deep residual learning for image recognition (pp. 12).

    Google Scholar 

  18. Wang, N. N., et al. (2018). Damage classification for masonry historic structures using convolutional neural networks based on still images. Computer-Aided Civil and Infrastructure Engineering, 33(12), 1073–1089.

    Article  Google Scholar 

  19. Redmon, J., & Farhadi, A. (2018). YOLOv3: An incremental improvement (pp. 1–6).

    Google Scholar 

  20. Redmon, J., Farhadi, A., & IEEE. (2017). YOLO9000: Better, faster, stronger. In 30th Ieee Conference on Computer Vision and Pattern Recognition (pp. 6517–6525).

    Google Scholar 

  21. Redmon, J., et al. (2016). You only look once: unified, real-time object detection. In 2016 Ieee Conference on Computer Vision and Pattern Recognition (pp. 779–788).

    Google Scholar 

  22. Liu, W., et al. (2015). SSD: Single shot MultiBox detector. arXiv:1512.02325.

  23. Ren, S. Q., et al. (2017). Faster R-CNN: Towards real-time object detection with region proposal networks. Ieee Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137–1149.

    Article  Google Scholar 

  24. Kumar, S. S., et al. (2020). Deep learning-based automated detection of sewer defects in CCTV videos. Journal of Computing in Civil Engineering, 34(1), 13.

    Article  Google Scholar 

  25. Li, L.J., et al. (2019). Dam surface crack detection based on deep learning. In Proceedings of the 2019 International Conference on Robotics, Intelligent Control and Artificial Intelligence (pp. 738–743). Assoc Computing Machinery.

    Google Scholar 

  26. Zhang, C. B., Chang, C. C., & Jamshidi, M. (2020). Concrete bridge surface damage detection using a single-stage detector. Computer-Aided Civil and Infrastructure Engineering, 35(4), 389–409.

    Article  Google Scholar 

  27. Zhang, C. B., & Chang, C. C. (2019). Surface damage detection for concrete bridges using single-stage convolutional neural networks. In P. Fromme & Z. Su (Eds.), Health Monitoring of Structural and Biological Systems (pp. Xiii). Spie-Int Soc Optical Engineering.

    Google Scholar 

  28. Mondal, T. G., et al. (2020). Deep learning-based multi-class damage detection for autonomous post-disaster reconnaissance. Structural Control and Health Monitoring, 27(4), 15.

    Google Scholar 

  29. Shelhamer, E., Long, J., & Darrell, T. (2017). Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4), 640–651.

    Article  Google Scholar 

  30. Li, Y., et al. (2017). Fully convolutional instance-aware semantic segmentation. 30th Ieee Conference on Computer Vision and Pattern Recognition (pp. 4438–4446). Ieee.

    Google Scholar 

  31. Guo, L., et al. (2020). Automatic crack distress classification from concrete surface images using a novel deep-width network architecture. Neurocomputing, 397, 383–392.

    Article  Google Scholar 

  32. Zhang, X. X., Rajan, D., & Story, B. (2019). Concrete crack detection using context-aware deep semantic segmentation network. Computer-Aided Civil and Infrastructure Engineering, 34(11), 951–971.

    Article  Google Scholar 

  33. Kim, B., & Cho, S. (2019). Image-based concrete crack assessment using mask and region-based convolutional neural network. Structural Control and Health Monitoring, 26(8), 15.

    Google Scholar 

  34. Kim, B., & Cho, S. (2018). Automated vision-based detection of cracks on concrete surfaces using a deep learning technique. Sensors, 18(10), 18.

    Article  Google Scholar 

  35. Xie, Q., et al. (2019). Automatic detection and classification of sewer defects via hierarchical deep learning. Ieee Transactions on Automation Science and Engineering, 16(4), 1836–1847.

    Article  Google Scholar 

  36. Browne, M., & Ghidary, S. S. (2003). Convolutional neural networks for image processing: An application in robot vision. In T. D. Gedeon & L. C. C. Fung (Eds.), Ai 2003: Advances in artificial intelligence (pp. 641–652). Springer.

    Google Scholar 

  37. Hassan, S. I., et al. (2019). Underground sewer pipe condition assessment based on convolutional neural networks. Automation in Construction, 106, 12.

    Article  Google Scholar 

  38. Meijer, D., et al. (2019). A defect classification methodology for sewer image sets with convolutional neural networks. Automation in Construction, 104, 281–298.

    Article  Google Scholar 

  39. Yin, X. F., et al. (2020). A deep learning-based framework for an automated defect detection system for sewer pipes. Automation in Construction, 109, 17.

    Article  Google Scholar 

  40. Kim, I. H., et al. (2018). Application of crack identification techniques for an aging concrete bridge inspection using an unmanned aerial vehicle. Sensors, 18(6), 14.

    Article  Google Scholar 

  41. Xu, H. Y., et al. (2019). Automatic bridge crack detection using a convolutional neural network. Applied Sciences-Basel, 9(14), 14.

    Google Scholar 

  42. Jiang, S., & Zhang, J. (2020). Real-time crack assessment using deep neural networks with wall-climbing unmanned aerial system. Computer-Aided Civil and Infrastructure Engineering, 35(6), 549–564.

    Article  Google Scholar 

  43. Dinh, K., Gucunski, N., & Duong, T. H. (2018). An algorithm for automatic localization and detection of rebars from GPR data of concrete bridge decks. Automation in Construction, 89, 292–298.

    Article  Google Scholar 

  44. Jung, H. J., et al. (2019). Bridge inspection and condition assessment using unmanned aerial vehicles (UAVs): Major challenges and solutions from a practical perspective. Smart Structures and Systems, 24(5), 669–681.

    Google Scholar 

  45. Kang, D. H., & Cha, Y. J. (2018). Autonomous UAVs for structural health monitoring using deep learning and an ultrasonic beacon system with geo-tagging. Computer-Aided Civil and Infrastructure Engineering, 33(10), 885–902.

    Article  Google Scholar 

  46. Liu, H., & Zhang, Y. F. Bridge condition rating data modeling using deep learning algorithm. Structure and Infrastructure Engineering, 14.

    Google Scholar 

  47. Modarres, C., et al. (2018). Convolutional neural networks for automated damage recognition and damage type identification. Structural Control and Health Monitoring, 25(10), 17.

    Article  Google Scholar 

  48. Zhu, J. S., & Song, J. B. (2020). An intelligent classification model for surface defects on cement concrete bridges. Applied Sciences-Basel, 10(3), 15.

    Google Scholar 

  49. Chun, C., & Ryu, S. K. (2019). Road surface damage detection using fully convolutional neural networks and semi-supervised learning. Sensors, 19(24), 15.

    Article  Google Scholar 

  50. Augustauskas, R., & Lipnickas, A. (2020). Improved pixel-level pavement-defect segmentation using a deep autoencoder. Sensors, 20(9), 21.

    Article  Google Scholar 

  51. Eguchi, M., et al. (2019). A simplified method of detecting spot surface defects by using Quasi-3D data from a conventional road profiler. Transportation Research Record, 2673(11), 377–387.

    Article  Google Scholar 

  52. Cao, W. M., Liu, Q. F., & He, Z. Q. (2020). Review of pavement defect detection methods. Ieee Access, 8, 14531–14544.

    Article  Google Scholar 

  53. Zhu, S. P., et al. (2007). An image segmentation algorithm in image processing based on threshold segmentation. In Sitis 2007: Proceedings of the International Conference on Signal Image Technologies & Internet Based Systems (pp. 673+). Ieee Computer Soc.

    Google Scholar 

  54. Huili, Z., Guofeng, Q., & Xingjian, W. (2010). Improvement of canny algorithm based on pavement edge detection. In Proceedings of the 2010 3rd International Congress on Image and Signal Processing (CISP 2010) (pp. 964–967).

    Google Scholar 

  55. Zhou, Y., et al. (2016). Seed-based approach for automated crack detection from pavement images. Transportation Research Record, 2589, 162–171.

    Article  Google Scholar 

  56. Qu, Z., et al. (2020). Crack detection of concrete pavement with cross-entropy loss function and improved VGG16 network model. Ieee Access, 8, 54564–54573.

    Article  Google Scholar 

  57. Riid, A., et al. (2019). Pavement distress detection with deep learning using the orthoframes acquired by a mobile mapping system. Applied Sciences-Basel, 9(22), 22.

    Google Scholar 

  58. Cha, Y. J., et al. (2018). Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Computer-Aided Civil and Infrastructure Engineering, 33(9), 731–747.

    Article  Google Scholar 

  59. Deng, L., et al. (2020). Region-based CNN method with deformable modules for visually classifying concrete cracks. Applied Sciences-Basel, 10(7), 18.

    Google Scholar 

  60. Lee, J., et al. (2019). Learning to detect cracks on damaged concrete surfaces using two-branched convolutional neural network. Sensors, 19(21), 18.

    Article  Google Scholar 

  61. Li, S. Y., & Zhao, X. F. (2019). Image-based concrete crack detection using convolutional neural network and exhaustive search technique. Advances in Civil Engineering, 2019, 12.

    Google Scholar 

  62. Wei, F. J., et al. (2019). Instance-level recognition and quantification for concrete surface bughole based on deep learning. Automation in Construction, 107, 13.

    Article  Google Scholar 

  63. Ryu, E., et al. (2020). Automated detection of surface cracks and numerical correlation with thermal-structural behaviors of fire damaged concrete beams. International Journal of Concrete Structures and Materials, 14(1), 12.

    Article  Google Scholar 

  64. Ye, X. W., Jin, T., & Chen, P. Y. (2019). Structural crack detection using deep learning-based fully convolutional networks. Advances in Structural Engineering, 22(16), 3412–3419.

    Article  Google Scholar 

  65. Wei, S. Y., Bao, Y. Q., & Li, H. (2020). Optimal policy for structure maintenance: A deep reinforcement learning framework. Structural Safety, 83, 13.

    Article  Google Scholar 

  66. Perry, B. J., et al. (2020). Streamlined bridge inspection system utilizing unmanned aerial vehicles (UAVs) and machine learning. Measurement, 164, 14.

    Article  Google Scholar 

  67. Liu, L. L., et al. (2017). Transfer learning on convolutional activation feature as applied to a building quality assessment robot. International Journal of Advanced Robotic Systems, 14(3), 12.

    Article  Google Scholar 

  68. Fang, W. L., et al. (2020). Computer vision for behaviour-based safety in construction: A review and future directions. Advanced Engineering Informatics, 43, 13.

    Article  Google Scholar 

  69. Fang, W. L., et al. (2020). Computer vision applications in construction safety assurance. Automation in Construction, 110, 10.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Sino-Australia Joint Research Center in BIM and Smart Construction, Shenzhen University, Shenzhen, China

    Ruying Cai, Jingru Li, Geng Li, Dongdong Tang & Yi Tan

Authors
  1. Ruying Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jingru Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Geng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dongdong Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yi Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yi Tan .

Editor information

Editors and Affiliations

  1. Central China Normal University, Wuhan, China

    Xinhai Lu

  2. Central China Normal University, Wuhan, China

    Zuo Zhang

  3. University of Hong Kong, Hong Kong, China

    Weisheng Lu

  4. Zhejiang University of Finance and Economics, Hangzhou, China

    Yi Peng

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cai, R., Li, J., Li, G., Tang, D., Tan, Y. (2021). A Review of the Application of CNN-Based Computer Vision in Civil Infrastructure Maintenance. In: Lu, X., Zhang, Z., Lu, W., Peng, Y. (eds) Proceedings of the 25th International Symposium on Advancement of Construction Management and Real Estate. CRIOCM 2020. Springer, Singapore. https://doi.org/10.1007/978-981-16-3587-8_42

Download citation

Publish with us

Policies and ethics

Search

Navigation