Skip to main content
SpringerLink
Log in

An effective detection and classification of road damages using hybrid deep learning framework

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The monitoring of road surfaces is a critical thing in transport infrastructure management. The manual reporting process increases the processing delay and causes challenges in accuracy. Detecting road surface damages is important for improving the quality of transportation and avoiding several issues normal people face in daily life. Therefore, an automated monitoring system is needed to compute road surface conditions for effective road maintenance regularly. Accurately detecting and classifying road damage images become a challenging task for researchers. Thus, the proposed work introduced a hybrid deep learning framework for detecting and classifying road damage images. At first, the input images are acquired from the dataset and pre-processed with an adaptive intensity limited histogram equalization algorithm. This pre-processing method enhances the contrast of the given input images and eliminates the noise presented in the image. Then, the damage detection is performed in the segmentation stage using an adaptive density based fuzzy c-means clustering method. Features from the segmented images are extracted using Laplacian edge detection with Gaussian operator and hybrid wavelet-Walsh transform approaches. Subsequently, the dimensionality of the feature set is reduced by using the Adaptive Horse herd Optimization (AHO) algorithm. Finally, the road damages are detected and classified using the proposed Hybrid Deep Capsule autoencoder based Convolutional Neural network (Hybrid DCACN) with Improved Whale Optimization (IWO) model. The experimental validation is done using the RDD2020 dataset, and the performance metrics are evaluated to show the efficacy of the proposed model. The proposed work attains 98.815% accuracy, and the obtained results outperform the existing approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

Data availability

Data sharing not applicable to this article.

References

  1. Anand A, Tripathy SS, Kumar RS (2015) An improved edge detection using morphological Laplacian of Gaussian operator. In 2015 2nd International conference on signal processing and integrated networks (SPIN) 532-536

  2. Arman MS, Hasan MM, Sadia F, Shakir AK, Sarker K, Himu FA (2020) Detection and classification of road damage using R-CNN and faster R-CNN: a deep learning approach. In international conference on cyber security and computer science. Springer, Cham 730-741

  3. Arya D, Maeda H, Ghosh SK, Toshniwal D, Mraz A, Kashiyama T, Sekimoto Y (2020) Transfer learning-based road damage detection for multiple countries. Computer Vision and Pattern Recognitionp, 1–15. https://doi.org/10.48550/arXiv.2008.13101

  4. Arya D, Maeda H, Ghosh SK, Toshniwal D, Mraz A, Kashiyama T, Sekimoto Y (2021) Deep learning-based road damage detection and classification for multiple countries. Autom Constr 132:103935

    Article  Google Scholar 

  5. Arya D, Maeda H, Ghosh SK, Toshniwal D, Sekimoto Y (2021) RDD2020: an annotated image dataset for automatic road damage detection using deep learning. Data in brief 36:1–7

  6. Cao MT, Tran QV, Nguyen NM, Chang KT (2020) Survey on performance of deep learning models for detecting road damages using multiple dashcam image resources. Adv Eng Inform 46:101182

    Article  Google Scholar 

  7. Chowdhary CL, Acharjya DP (2016) Breast cancer detection using intuitionistic fuzzy histogram hyperbolization and possibilitic fuzzy C-mean clustering algorithms with texture feature based classification on mammography images. In: Proceedings of the international conference on advances in information communication technology & computing (20):1–6

  8. Chowdhary CL, Acharjya DP (2016) A hybrid scheme for breast cancer detection using intuitionistic fuzzy rough set technique. Int J Healthcare Inf Syst Inf (IJHISI) 11(2):38–61

    Article  Google Scholar 

  9. Chowdhary CL, Acharjya DP (2017) Clustering algorithm in possibilistic exponential fuzzy C-mean segmenting medical images. J Biomimetics, Biomaterials Biomed Eng Trans Tech Publications Ltd 30:12–23

    Article  Google Scholar 

  10. Chowdhary CL, Acharjya DP (2018) Segmentation of mammograms using a novel intuitionistic possibilistic fuzzy c-mean clustering algorithm. In: In Nature Inspired Computing. Springer, Singapore, pp 75–82

    Chapter  Google Scholar 

  11. Chowdhary CL, Acharjya DP (2020) Segmentation and feature extraction in medical imaging: a systematic review. Procedia Comput Sci 167:26–36

    Article  Google Scholar 

  12. Chowdhary CL, Sai GVK, Acharjya DP (2016) Decrease in false assumption for detection using digital mammography. Comput Intell Data Mining, Springer, New Delhi 2:325–333

    Google Scholar 

  13. Chun C, Ryu SK (2019) Road surface damage detection using fully convolutional neural networks and semi-supervised learning. Sensors 19(24):5501

    Article  Google Scholar 

  14. Devi HS, Singh KM (2020) Red-cyan anaglyph image watermarking using DWT, Hadamard transform and singular value decomposition for copyright protection. J Inf Secur Appl 50:102424

    Google Scholar 

  15. Dung CV (2019) Autonomous concrete crack detection using deep fully convolutional neural network. Autom Construct 99:52–58

    Article  Google Scholar 

  16. Fan R, Liu M (2019) Road damage detection based on unsupervised disparity map segmentation. IEEE Trans Intell Transp Syst 21(11):4906–4911

    Article  Google Scholar 

  17. Gao Y, Wang Z, Li H, Pan J (2021) Gaussian collaborative fuzzy c-means clustering. Int J Fuzzy Syst 23(7):2218–2234

    Article  Google Scholar 

  18. Griffiths D, Boehm J (2019) A review on deep learning techniques for 3D sensed data classification. Remote Sens 11(12):1–29

  19. Guo L, Li R, Jiang B (2020) A cascade broad neural network for concrete structural crack damage automated classification. IEEE Trans Industrial Inf 17(4):2737–2742

    Article  Google Scholar 

  20. He Y, Wu M, Sun M, Wang L (2021) Towards pavement crack detection and classification based on machine leaning methods. In international conference on autonomous unmanned systems, springer, Singapore, 1-9

  21. Hegde V, Trivedi D, Alfarrarjeh A, Deepak A, Kim SH, Shahabi C (2020) Yet another deep learning approach for road damage detection using ensemble learning. In: 2020 IEEE international conference on big data (big data). IEEE, pp 5553–5558

  22. Hou Y, Li Q, Zhang C, Lu G, Ye Z, Chen Y, Cao D (2021) The state-of-the-art review on applications of intrusive sensing, image processing techniques, and machine learning methods in pavement monitoring and analysis. Engineering 7(6):845–856

    Article  Google Scholar 

  23. Huyan J, Li W, Tighe S, Xu Z, Zhai J (2020) CrackU-net: a novel deep convolutional neural network for pixelwise pavement crack detection. Struct Control Health Monit 27(8):e2551

    Article  Google Scholar 

  24. Kanwal S, Rasheed MI, Pitafi AH, Pitafi A, Ren M (2020) Road and transport infrastructure development and community support for tourism: The role of perceived benefits, and community satisfaction. Tour Manag 77:104014

    Article  Google Scholar 

  25. Maeda H, Kashiyama T, Sekimoto Y, Seto T, Omata H (2021) Generative adversarial network for road damage detection. Comput-Aided Civ Infrastruct Eng 36(1):47–60

    Article  Google Scholar 

  26. Mei Q, Gül M (2020) A cost effective solution for pavement crack inspection using cameras and deep neural networks. Constr Build Mater 256:119397

    Article  Google Scholar 

  27. MiarNaeimi F, Azizyan G, Rashki M (2021) Horse herd optimization algorithm: a nature-inspired algorithm for high-dimensional optimization problems. Knowl-Based Syst 213:106711

    Article  Google Scholar 

  28. Mohan A, Singh AK, Kumar B, Dwivedi R (2021) Review on remote sensing methods for landslide detection using machine and deep learning. Trans Emerg Telecommun Technol 32(7):e3998

    Google Scholar 

  29. Pandey AK, Iqbal R, Maniak T, Karyotis C, Akuma S, Palade V (2022) Convolution neural networks for pothole detection of critical road infrastructure. Comput Electr Eng 99:1–12

  30. Park S, Bang S, Kim H, Kim H (2019) Patch-based crack detection in black box images using convolutional neural networks. J Comput Civ Eng 33(3):04019017

    Article  Google Scholar 

  31. Pham V, Pham C, Dang T (2020) Road damage detection and classification with detectron2 and faster r-cnn. In 2020 IEEE International Conference on Big Data (Big Data). IEEE 5592–5601

  32. Ping P, Yang X, Gao Z (2020) A deep learning approach for street pothole detection. In 2020 IEEE sixth international conference on big data computing service and applications (BigDataService). IEEE 198-204

  33. Shim S, Kim J, Lee SW, Cho GC (2021) Road surface damage detection based on hierarchical architecture using lightweight auto-encoder network. Autom Construct 130:103833

    Article  Google Scholar 

  34. Shim S, Kim J, Lee SW, Cho GC (2022) Road damage detection using super-resolution and semi-supervised learning with generative adversarial network. Autom Constr 135:104139

    Article  Google Scholar 

  35. Shtayat A, Moridpour S, Best B, Shroff A, Raol D (2020) A review of monitoring systems of pavement condition in paved and unpaved roads. J Traffic Transport Eng (English Edition)

  36. Wang Q, Mao J, Zhai X, Gui J, Shen W, Liu Y (2021) Improvements of YoloV3 for road damage detection. J Phys: Conf Series, IOP publishing 1903(1):012008

    Google Scholar 

  37. Wei C, Li S, Wu K, Zhang Z, Wang Y (2021) Damage inspection for road markings based on images with hierarchical semantic segmentation strategy and dynamic homography estimation. Automat Construct 131:103876

    Article  Google Scholar 

  38. Wu H, Yao L, Xu Z, Li Y, Ao X, Chen Q, Meng B (2019) Road pothole extraction and safety evaluation by integration of point cloud and images derived from mobile mapping sensors. Adv Eng Inf 42:100936

    Article  Google Scholar 

  39. Xu H, Su X, Wang Y, Cai H, Cui K, Chen X (2019) Automatic bridge crack detection using a convolutional neural network. Appl Sci 9(14):2867

    Article  Google Scholar 

  40. Xu H, Chen B, Qin J (2021) A CNN-based length-aware cascade road damage detection approach. Sensors 21(3):689

    Article  Google Scholar 

  41. Yang F, Zhang L, Yu S, Prokhorov D, Mei X, Ling H (2019) Feature pyramid and hierarchical boosting network for pavement crack detection. IEEE Trans Intell Transport Syst 21(4):1525–1535

  42. Yang J, Zhang L, Chen C, Li Y, Li R, Wang G, Zeng Z (2020) A hierarchical deep convolutional neural network and gated recurrent unit framework for structural damage detection. Inf Sci 540:117–130

    Article  Google Scholar 

  43. Yik YK, Alias NE, Yusof Y, Isaak S (2021) A real-time pothole detection based on deep learning approach. J Phys Conf Series, IOP Publishing 1828(1):012001

    Article  Google Scholar 

  44. Yuan Y, Yuan Y, BakerT KLM, Hogrefe D (2021) FedRD: privacy-preserving adaptive federated learning framework for intelligent hazardous road damage detection and warning. Futur Gener Comput Syst 125:385–398

    Article  Google Scholar 

Download references

Funding

No funding is provided for the preparation of manuscript.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, School of Computing, Sathyabama Institute of Science and Technology, Chennai, India

    D. Deepa

  2. Department of Information Technology, School of Computing, Sathyabama Institute of Science and Technology, Chennai, India

    A. Sivasangari

Authors
  1. D. Deepa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Sivasangari
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors have equal contributions in this work.

Corresponding author

Correspondence to D. Deepa.

Ethics declarations

Conflict of interest

Authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Consent to participate

All the authors involved have agreed to participate in this submitted article.

Consent to publish

All the authors involved in this manuscript give full consent for publication of this submitted article.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Deepa, D., Sivasangari, A. An effective detection and classification of road damages using hybrid deep learning framework. Multimed Tools Appl 82, 18151–18184 (2023). https://doi.org/10.1007/s11042-022-14001-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-14001-9

Keywords

Search

Navigation