Skip to main content
SpringerLink
Log in

RCNet: road classification convolutional neural networks for intelligent vehicle system

  • Original Research Paper
  • Published:
Intelligent Service Robotics Aims and scope Submit manuscript

Abstract

Vision-based techniques for intelligent vehicles in heterogeneous road environments are gaining significant attention from researchers and industrialists. Unfortunately, the mechanisms in this domain suffer from limited performance due to scene complexity, varying road structure, and improper illumination conditions. These challenging situations may lead an intelligent vehicle into dangerous situations such as collisions or road accidents and may cause higher mortality. The application of intelligent methods and other machine learning techniques for road surface classification is little explored in the existing literature. Thus, we propose a convolutional neural network-based road classification network (RCNet) for the accurate classification of road surfaces. This procedure includes the classification of five major categories of road surfaces: curvy, dry, ice, rough, and wet roads. The experimental results reveal the behavior of the proposed RCNet under various optimizer techniques. The standard performance evaluation measures have been used to test and validate the proposed method on the Oxford RobotCar dataset. RCNet achieves classification accuracy, precision, and sensitivity of 99.90%, and 99.97% of specificity. Results of implemented work are significantly higher than available state-of-the-art techniques and show accurate and effective performance in the complex road environment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. https://youtu.be/3rHREEoYzBk.

  2. https://youtu.be/E6ry6-Fu_xU.

  3. https://youtu.be/cHyu7gP4G0o.

References

  1. “Road traffic deaths, Global Health Observatory data repository by World Health Organization,” 2019. [Online]. Available: http://apps.who.int/gho/data/node.main.A997

  2. Bagloee SA, Tavana M, Asadi M, Oliver T (2016) Autonomous vehicles: challenges, opportunities, and future implications for transportation policies. J Modern Trans 24(4):284–303

    Article  Google Scholar 

  3. Cui Y, Wu J, Xu H, Wang A (2020) Lane change identification and prediction with roadside LiDAR data. Opt Laser Technol 123(July 2019):105934

    Article  Google Scholar 

  4. Castorena J, Agarwal S (2018) Ground-edge-based lidar localization without a reflectivity calibration for autonomous driving. IEEE Robot Autom Lett 3(1):344–351

    Article  Google Scholar 

  5. Changalvala R, Malik H (2019) LiDAR data integrity verification for autonomous vehicle. IEEE Access 7:138018–138031

    Article  Google Scholar 

  6. Fernandez MG et al (2018) Synthetic aperture radar imaging system for landmine detection using a ground penetrating radar on board a unmanned aerial vehicle. IEEE Access 6:45100–45112

    Article  Google Scholar 

  7. Okamoto K, Itti L, Tsiotras P (2019) Vision-based autonomous path following using a human driver control model with reliable input-feature Value Estimation. IEEE Trans Intell Veh 4(3):497–506

    Article  Google Scholar 

  8. Drews P, Williams G, Goldfain B, Theodorou EA, Rehg JM (2019) Vision-based high-speed driving with a deep dynamic observer. IEEE Robot Autom Lett 4(2):1564–1571

    Article  Google Scholar 

  9. Gupta A, Choudhary A (2018) A framework for camera-based real-time lane and road surface marking detection and recognition. IEEE Trans Intell Veh 3(4):476–485

    Article  Google Scholar 

  10. Cheng L, Zhang X, Shen J (2019) Road surface condition classification using deep learning. J Vis Commun Image Represent 64:102638

    Article  Google Scholar 

  11. Hoang TM, Nam SH, Park KR (2019) Enhanced detection and recognition of road markings based on adaptive region of interest and deep Learning. IEEE Access 7(2):109817–109832

    Article  Google Scholar 

  12. Chen B, Gong C, Yang J (2019) Importance-aware semantic segmentation for autonomous vehicles. IEEE Trans Intell Transp Syst 20(1):137–148

    Article  Google Scholar 

  13. Wang Q, Gao J, Yuan Y (2018) Embedding structured contour and location prior in siamesed fully convolutional networks for road Detection. IEEE Trans Intell Transp Syst 19(1):230–241

    Article  Google Scholar 

  14. Zhang Z, Liu Q, Wang Y (2018) Road extraction by deep residual U-net. IEEE Geosci Remote Sens Lett 15(5):749–753

    Article  Google Scholar 

  15. Liu H, Han X, Li X, Yao Y, Huang P, Tang Z (2019) Deep representation learning for road detection using Siamese network. Multimed Tools Appl 78(17):24269–24283

    Article  Google Scholar 

  16. You J (2019, December) Weather data integrated mask R-CNN for automatic road surface condition monitoring. In 2019 IEEE visual communications and image processing (VCIP) (pp. 1-4). IEEE

  17. Geng L, Sun J, Xiao Z, Zhang F, Wu J (2018) Combining CNN and MRF for road detection. Comput Electr Eng 70:895–903

    Article  Google Scholar 

  18. Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional Encoder–Decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  Google Scholar 

  19. Peng B, Li Y, He L, Fan K, Tong L (2018) Road segmentation of UAV RS image using adversarial network with multi-scale context aggregation. Int. Geosci. Remote Sens. Symp. 2018:6935–6938

    Google Scholar 

  20. qiang Geng H, Zhang H, bing Xue Y, Zhou M, ping XuG, Gao Z (2017) Semantic image segmentation with fused CNN features. Optoelectron. Lett. 13(5):381–385

    Article  Google Scholar 

  21. Varona B, Monteserin A, Teyseyre A (2020) A deep learning approach to automatic road surface monitoring and pothole detection. Pers Ubiquit Comput 24:519–534. https://doi.org/10.1007/s00779-019-01234-z

    Article  Google Scholar 

  22. Hong M, Guo J, Dai Y, Yin Z (2019) A novel FMH model for road extraction from high-resolution remote sensing images in urban areas. Procedia Comput Sci 147:49–55

    Article  Google Scholar 

  23. Bendjaballah M, Graovac S (2016, June) “One Approach to Detection and Extraction of On-Road Obstacles Based on Image Processing”. In International Conference on Robotics in Alpe-Adria Danube Region (pp. 96-104). Springer, Cham

  24. Lak AM, Zoej MJV, Mokhtarzade M (2016) A new method for road detection in urban areas using high-resolution satellite images and Lidar data based on fuzzy nearest-neighbor classification and optimal features. Arab J Geosci 9(5):1–11

    Article  Google Scholar 

  25. Bakhtiari HRR, Abdollahi A, Rezaeian H (2017) Semi automatic road extraction from digital images. Egypt J Remote Sens Sp Sci 20(1):117–123

    Google Scholar 

  26. Ochman M (2019) Hybrid approach to road detection in front of the vehicle. IFAC-PapersOnLine 52(8):245–250

    Article  Google Scholar 

  27. Wang W, Yang N, Zhang Y, Wang F, Cao T, Eklund P (2016) A review of road extraction from remote sensing images. J. Traffic Transp Eng (English Ed., ) 3(3):271–282

    Article  Google Scholar 

  28. Kinattukara T, Verma B (2014) A Neural Ensemble Approach for Segmentation and Classification of Road Images. In: Loo CK, Yap KS, Wong KW, Beng Jin AT, Huang K (eds) Neural Information Processing. ICONIP 2014. Lecture Notes in Computer Science, vol 8836. Springer, Cham. https://doi.org/10.1007/978-3-319-12643-2_23.

  29. Hoang ND, Nguyen QL (2019) A novel method for asphalt pavement crack classification based on image processing and machine learning. Eng Comput 35(2):487–498

    Article  Google Scholar 

  30. Zakeri H, Nejad FM, Fahimifar A (2017) Image based techniques for crack detection, classification and quantification in asphalt pavement: a review. Arch Comput Methods Eng 24(4):935–977

    Article  MATH  Google Scholar 

  31. Tang I, Breckon TP (2011) Automatic road environment classification. IEEE Trans Intell Transp Syst 12(2):476–484

    Article  Google Scholar 

  32. Almazan EJ, Qian Y, Elder JH (2016) Road segmentation for classification of road weather conditions. In: European Conference on Computer Vision. Springer, Cham, pp 96–108

  33. Yang T, Liu J, Huang W, Yang L, Chui C K (2013) “Frontiers of intelligent autonomous systems,” vol. 466, pp. 297-308

  34. Xu F, Wang H, Hu B et al (2020) Road Boundaries Detection based on Modified Occupancy Grid Map Using Millimeter-wave Radar. Mob Netw Appl 25:1496–1503. https://doi.org/10.1007/s11036-019-01378-5

    Article  Google Scholar 

  35. Han J, Kim D, Lee M, Sunwoo M (2014) Road boundary detection and tracking for structured and unstructured roads using A 2D LIDAR sensor. Int J Automot Technol 15(4):611–623

    Article  Google Scholar 

  36. Stasinopoulos S, Zhao M (2015) “Laser-Based Obstacle Avoidance and Road Quality Detection for Autonomous Bicycles,” In Lecture Notes in Electrical Engineering, pp. 215-223

  37. Byun J, Na K, Seo B, Roh M (2015) Drivable Road Detection with 3D Point Clouds Based on the MRF for Intelligent Vehicle. In: Mejias L, Corke P, Roberts J (eds) Field and Service Robotics. Springer Tracts in Advanced Robotics, vol 105. Springer, Cham. https://doi.org/10.1007/978-3-319-07488-7_4

  38. Gorges C, Öztürk K, Liebich R (2019) Impact detection using a machine learning approach and experimental road roughness classification. Mech Syst Signal Process 117:738–756

    Article  Google Scholar 

  39. Qin Y, Wang Z, Xiang C, Hashemi E, Khajepour A, Huang Y (2019) Speed independent road classification strategy based on vehicle response: theory and experimental validation. Mech Syst Signal Process 117:653–666

    Article  Google Scholar 

  40. Gui K, Ye L, Ge J, Cheikh FA, Huang L (2019) Road surface condition detection utilizing resonance frequency and optical technologies. Sensors Actuators A Phys 297:111540

    Article  Google Scholar 

  41. Yang CH, Yun DG, Kim JG et al (2020) Machine Learning Approaches to Estimate Road Surface Temperature Variation along Road Section in Real-Time for Winter Operation. Int J ITS Res 18:343–355. https://doi.org/10.1007/s13177-019-00203-3

    Article  Google Scholar 

  42. Nunes DE, Mota VFS (2019) A participatory sensing framework to classify road surface quality. J Internet Serv Appl 10(1):1–16

    Article  Google Scholar 

  43. Jonsson P, Casselgren J, Thornberg B (2015) Road surface status classification using spectral analysis of NIR camera images. IEEE Sens J 15(3):1641–1656

    Article  Google Scholar 

  44. Maddern W, Pascoe G, Linegar C, Newman P (2017) 1 year, 1000 km: The Oxford Robotcar dataset. Int J Robotics Res 36(1):3–15

    Article  Google Scholar 

  45. Nair RR, David E, Rajagopal S (2019) A robust anisotropic diffusion filter with low arithmetic complexity for images. Eurasip J Image Video Process 1:2019

    Google Scholar 

  46. Naveed K, Shaukat B, Ehsan S, Mcdonald-Maier KD, Ur Rehman N (2019) Multiscale image denoising using goodness-of-fit test based on EDF statistics. PLoS One 14(5):1–25

    Article  Google Scholar 

  47. Liu F, Lin G, Shen C (2017) Discriminative training of deep fully connected continuous CRFs with task-specific loss. IEEE Trans Image Process 26(5):2127–2136

    Article  MathSciNet  MATH  Google Scholar 

  48. Lapin M, Hein M, Schiele B (2018) Analysis and optimization of loss functions for multiclass, top-k, and multilabel classification. IEEE Trans Pattern Anal Mach Intell 40(7):1533–1554

    Article  Google Scholar 

  49. Peng H, Liu CL (2019) Discriminative feature selection via employing smooth and robust hinge loss. IEEE Trans Neural Networks Learn Syst 30(3):788–802

    Article  MathSciNet  Google Scholar 

  50. Ioffe S, Szegedy C (2015) “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” 32nd International Conference on Machine Learning. ICML 2015, vol. 1, pp. 448-456

  51. Ruder S (2016) An overview of gradient descent optimization algorithmsar. Xiv preprint arXiv:1609.04747.

  52. Singh D, Mohan CK (2019) Deep spatio-temporal representation for detection of road accidents using stacked autoencoder. IEEE Trans Intell Transp Syst 20(3):879–887

    Article  MathSciNet  Google Scholar 

  53. Yasrab R, Gu N, Zhang X (2017) An encoder-decoder based convolution neural network (CNN) for future advanced driver assistance system (ADAS). Appl Sci 7(4):312

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Technology, National Institute of Technology, Raipur, India

    Deepak Kumar Dewangan & Satya Prakash Sahu

Authors
  1. Deepak Kumar Dewangan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Satya Prakash Sahu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Deepak Kumar Dewangan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dewangan, D.K., Sahu, S.P. RCNet: road classification convolutional neural networks for intelligent vehicle system. Intel Serv Robotics 14, 199–214 (2021). https://doi.org/10.1007/s11370-020-00343-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11370-020-00343-6

Keywords

Search

Navigation