Skip to main content

Advertisement

SpringerLink
Log in

Towards the design of vision-based intelligent vehicle system: methodologies and challenges

  • Review Article
  • Published:
Evolutionary Intelligence Aims and scope Submit manuscript

Abstract

Rapid growth in technology has changed the way humans live. Ongoing development in the automobile industry is creating intelligent vehicles and this mode of transportation will assist human society. The need for this survey arises to identify the scope of an intelligent vehicle through a computer vision approach equipped with recent technological trends. In this article, the major technological phases of intelligent vehicles are analyzed and discussed. The operational mechanism in these phases is mostly based on vision sensors that facilitate these vehicles to perceive the heterogeneous and dynamic environments and help them to make appropriate decisions. This study identifies various state-of-art techniques and phase-wise datasets used in the literature. It highlights the advancement in different phases, challenges, and scopes for the design and development of intelligent vehicles system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Okumura B et al (2016) Challenges in perception and decision making for intelligent automotive vehicles: a case study. IEEE Trans Intell Veh 1(1):20–32. https://doi.org/10.1109/tiv.2016.2551545

    Article  Google Scholar 

  2. “GHO | By category | Road traffic deaths—Data by country, https://apps.who.int/gho/data/node.main.A997,” WHO, Accessed: Aug. 21, 2019. [Online]. Available: https://apps.who.int/gho/data/node.main.A997

  3. Azizi A, Entessari F, Osgouie KG, Rashnoodi AR (2014) Introducing neural networks as a computational intelligent technique. Mech Mater 464:369–374. https://doi.org/10.4028/www.scientific.net/AMM.464.369

    Article  Google Scholar 

  4. Pandey P, Dewangan KK and Dewangan DK (2018) Enhancing the quality of satellite images using fuzzy inference system. In: 2017 International Conference on Energy, Communication, Data Analytics and Soft Computing, ICECDS 2017. https://doi.org/10.1109/ICECDS.2017.8390024

  5. A. Azizi, “Applications of Artificial Intelligence Techniques to Enhance Sustainability of Industry 4 . 0 : Design of an Artificial Neural Network Model as Dynamic Behavior Optimizer of Robotic Arms,” vol. 2020, 2020.

  6. Zhao Y, Noorbakhsh A, Koopialipoor M, Azizi A (2020) A new methodology for optimization and prediction of rate of penetration during drilling operations. Eng Comput 36(2):587–595. https://doi.org/10.1007/s00366-019-00715-2

    Article  Google Scholar 

  7. Chaudhuri A, Sahu TP (2022) Multi-objective feature selection based on quasi-oppositional based Jaya algorithm for microarray data. Knowl-Based Syst 236:107804. https://doi.org/10.1016/j.knosys.2021.107804

    Article  Google Scholar 

  8. Ali U, Dewangan KK, Dewangan DK (2018) Distributed denial of service attack detection using ant bee colony and artificial neural network in cloud computing. Adv Intell Syst Comput 652:165–175. https://doi.org/10.1007/978-981-10-6747-1_19

    Article  Google Scholar 

  9. Azizi A (2019) Hybrid artificial intelligence optimization technique. https://doi.org/10.1007/978-981-13-2640-0

  10. Razmjooy N (2021) A computational intelligence perspective on multimodal image registration for unmanned aerial vehicles (UAVs). Springer Nature, Switzerland

    Google Scholar 

  11. Fernandes SR, Razmjooy N (2021) Nondestructive diagnosis and analysis of computed microtomography images via texture descriptors. Springer Nature, Switzerland (2019)

    Book  Google Scholar 

  12. Guo G, Razmjooy N (2019) A new interval differential equation for edge detection and determining breast cancer regions in mammography images. Syst Sci Control Eng. https://doi.org/10.1080/21642583.2019.1681033

    Article  Google Scholar 

  13. Sharma A, Gupta R, Lakshmanan K, Gupta A (2021) Transition based discount factor for model free algorithms in reinforcement learning. Symmetry. https://doi.org/10.3390/sym13071197

    Article  Google Scholar 

  14. Dewangan DK, Sahu SP, Sairam B, Agrawal A (2021) VLDNet: vision-based lane region detection network for intelligent vehicle system using semantic segmentation. Computing 103(12):2867–2892. https://doi.org/10.1007/s00607-021-00974-2

    Article  MathSciNet  Google Scholar 

  15. Elon Musk statement about Self Driving Car. https://9to5google.com/2015/10/16/elon-musk-says-that-the-lidar-google-uses-in-its-self-driving-car-doesnt-make-sense-in-a-car-context/ (accessed Jan 08, 2022)

  16. Belyankina ER and Moreva NA (2021) Unmanned cars. pp 185–187. Available: https://www.elibrary.ru/item.asp?id=46600583

  17. Geng L, Sun J, Xiao Z, Zhang F, Wu J (2018) Combining CNN and MRF for road detection. Comput Electr Eng 70:895–903. https://doi.org/10.1016/j.compeleceng.2017.11.026

    Article  Google Scholar 

  18. Zhu X, Zhang X, Zhang XY, Xue Z, Wang L (2019) A novel framework for semantic segmentation with generative adversarial network. J Vis Commun Image Represent 58:532–543. https://doi.org/10.1016/j.jvcir.2018.11.020

    Article  Google Scholar 

  19. Qingqun Ning CC, Zhu J (2018) Very fast semantic image segmentation using hierarchical dilation and feature refining. Cognit Comput 10:62–72

    Article  Google Scholar 

  20. Brostow GJ, Shotton J, Fauqueur J, Cipolla R (2008) Segmentation and recognition using structure from motion point clouds. Lect Notes Comput Sci 5302:44–57. https://doi.org/10.1007/978-3-540-88682-2-5 (LNCS, no. PART 1)

    Article  Google Scholar 

  21. Brostow GJ, Fauqueur J, Cipolla R (2009) Semantic object classes in video: a high-definition ground truth database. Pattern Recognit Lett 30(2):88–97. https://doi.org/10.1016/j.patrec.2008.04.005

    Article  Google Scholar 

  22. Cordts M et al (2016) The Cityscapes Dataset for Semantic Urban Scene Understanding. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit 2016:3213–3223. https://doi.org/10.1109/CVPR.2016.350

    Article  Google Scholar 

  23. Cordts M et al (2015) The cityscapes dataset. CVPR Work Futur Datasets Vis

  24. Fritsch J, Kuhnl T, Geiger A (2013) A new performance measure and evaluation benchmark for road detection algorithms. IEEE Conf Intell Transp Syst Proce ITSC. https://doi.org/10.1109/ITSC.2013.6728473

    Article  Google Scholar 

  25. Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495. https://doi.org/10.1109/TPAMI.2016.2644615

    Article  Google Scholar 

  26. Yadav S and Arora C (2017) Deep cnn with color lines model for unmarked road segmentation Indian Institute of Technology Delhi New Delhi 110016 Indraprastha Institute of Information Technology Delhi New Delhi 110020. pp 585–589

  27. Zhou Y, Lyu Y, Huang X (2019) RoadNet: an 80-mW hardware accelerator for road detection. IEEE Embed Syst Lett 11(1):21–24. https://doi.org/10.1109/LES.2018.2841199

    Article  Google Scholar 

  28. Dewangan DK, Sahu SP (2021) Road detection using semantic segmentation-based convolutional neural network for intelligent vehicle system. In: Data engineering and communication technology. Lecture notes on data engineering and communications technologies. Springer, Singapore, pp 629–637

  29. Song X, Rui T, Zhang S, Fei J, Wang X (2018) A road segmentation method based on the deep auto-encoder with supervised learning. Comput Electr Eng 68:381–388. https://doi.org/10.1016/j.compeleceng.2018.04.003

    Article  Google Scholar 

  30. Han X, Lu J, Zhao C, You S, Li H (2018) Semisupervised and weakly supervised road detection based on generative adversarial networks. IEEE Signal Process Lett 25(4):551–555. https://doi.org/10.1109/LSP.2018.2809685

    Article  Google Scholar 

  31. H. qiang Geng, H. Zhang, Y. bing Xue, M. Zhou, G. ping Xu, and Z. Gao, “Semantic image segmentation with fused CNN features,” Optoelectron. Lett., vol. 13, no. 5, pp. 381–385, 2017, doi: https://doi.org/10.1007/s11801-017-7086-6.

  32. Dewangan DK, Sahu SP (2021) RCNet: road classification convolutional neural networks for intelligent vehicle system. Intell Serv Robot 14(2):199–214. https://doi.org/10.1007/s11370-020-00343-6

    Article  Google Scholar 

  33. Peng B, Li Y, He L, Fan K, Tong L (2018) Road segmentation of UAV RS image using adversarial network with multi-scale context aggregation. Int Geosci Remote Sens Symp 2018:6935–6938. https://doi.org/10.1109/IGARSS.2018.8517641

    Article  Google Scholar 

  34. Liu X, Deng Z (2018) Segmentation of drivable road using deep fully convolutional residual network with pyramid pooling. Cognit Comput 10(2):272–281. https://doi.org/10.1007/s12559-017-9524-y

    Article  MathSciNet  Google Scholar 

  35. Kakegawa S, Matono H, Kido H, Shima T (2018) Road surface segmentation based on vertically local disparity histogram for stereo camera. Int J Intell Transp Syst Res 16(2):90–97. https://doi.org/10.1007/s13177-017-0140-8

    Article  Google Scholar 

  36. Valente M, Stanciulescu B (2017) Real-time method for general road segmentation. IEEE Intell Veh Symp Proc. https://doi.org/10.1109/IVS.2017.7995758

    Article  Google Scholar 

  37. Shi J, Fu F, Wang Y and Wang J (2016) A novel path segmentation method for autonomous road following. In: ICSPCC 2016 - IEEE Int. Conf. Signal Process. Commun. Comput. Conf. Proc., pp 1–6. https://doi.org/10.1109/ICSPCC.2016.7753701

  38. Xia X, Zhao J, Li X, and Wang H (2016) Segmentation based on the unstructured road with shadow. In: Proc. - 2016 8th Int. Conf. Intell. Human-Machine Syst. Cybern. IHMSC 2016, vol. 1. pp 501–504. https://doi.org/10.1109/IHMSC.2016.71

  39. Cheng M, Zhang Y, Su Y, Alvarez JM, Kong H (2018) Curb detection for road and sidewalk detection. IEEE Trans Veh Technol 67(11):10330–10342. https://doi.org/10.1109/TVT.2018.2865836

    Article  Google Scholar 

  40. Yang W, Fang B, Tang YY (2018) Fast and accurate vanishing point detection and its application in inverse perspective mapping of structured road. IEEE Trans Syst Man Cybernet Syst 48(5):755–766. https://doi.org/10.1109/TSMC.2016.2616490

    Article  Google Scholar 

  41. Bao J, Zhang Y, Su X, Zheng R (2018) Unpaved road detection based on spatial fuzzy clustering algorithm. EURASIP J Image Video Process 1:2018. https://doi.org/10.1186/s13640-018-0260-3

    Article  Google Scholar 

  42. De Cristóforis P, Nitsche MA, Krajník T, Mejail M (2016) Real-time monocular image-based path detection: a GPU-based embedded solution for on-board execution on mobile robots. J Real-Time Image Process 11(2):335–348. https://doi.org/10.1007/s11554-013-0356-z

    Article  Google Scholar 

  43. Song W, Yang Y, Fu M, Li Y, Wang M (2018) Lane detection and classification for forward collision warning system based on stereo vision. IEEE Sens J 18(12):5151–5163. https://doi.org/10.1109/JSEN.2018.2832291

    Article  Google Scholar 

  44. Dewangan DK, Sahu SP (2021) Deep learning-based speed bump detection model for intelligent vehicle system using raspberry Pi. IEEE Sens J 21(3):3570–3578. https://doi.org/10.1109/JSEN.2020.3027097

    Article  Google Scholar 

  45. Ye YY, Hao XL, Chen HJ (2018) Lane detection method based on lane structural analysis and CNNs. IET Intell Transp Syst 12(6):513–520. https://doi.org/10.1049/iet-its.2017.0143

    Article  Google Scholar 

  46. Chen J, Ruan Y, Chen Q (2018) A precise information extraction algorithm for lane lines. China Commun 15(10):210–219. https://doi.org/10.1109/CC.2018.8485482

    Article  Google Scholar 

  47. Ju Han Yoo DHK, Lee S-W, Park S-K (2017) A robust lane detection method based on vanishing point estimation using the relevance of line segments. IEEE Trans Intell Transp Syst 18(12):3254–3266

    Article  Google Scholar 

  48. De Paula MB, Jung CR (2015) Automatic detection and classification of road lane markings using onboard vehicular cameras. IEEE Trans Intell Transp Syst 16(6):3160–3169. https://doi.org/10.1109/TITS.2015.2438714

    Article  Google Scholar 

  49. Su Y, Zhang Y, Lu T, Yang J, Kong H (2018) Vanishing point constrained lane detection with a stereo camera. IEEE Trans Intell Transp Syst 19(8):2739–2744. https://doi.org/10.1109/TITS.2017.2751746

    Article  Google Scholar 

  50. Shin BS, Tao J, Klette R (2015) A superparticle filter for lane detection. Pattern Recognit 48(11):3333–3345. https://doi.org/10.1016/j.patcog.2014.10.011

    Article  Google Scholar 

  51. Niu J, Lu J, Xu M, Lv P, Zhao X (2016) Robust lane detection using two-stage feature extraction with curve fitting. Pattern Recognit 59:225–233. https://doi.org/10.1016/j.patcog.2015.12.010

    Article  Google Scholar 

  52. Moon YY, Geem ZW, Han GT (2018) Vanishing point detection for self-driving car using harmony search algorithm. Swarm Evol Comput 41(February):111–119. https://doi.org/10.1016/j.swevo.2018.02.007

    Article  Google Scholar 

  53. Dewangan DK, Sahu SP (2021) Lane detection for intelligent vehicle system using image processing techniques. Data science. Transactions on computer systems and networks. Springer, Singapore

    Google Scholar 

  54. Yi SC, Chen YC, Chang CH (2015) A lane detection approach based on intelligent vision. Comput Electr Eng 42(2):23–29. https://doi.org/10.1016/j.compeleceng.2015.01.002

    Article  Google Scholar 

  55. Li C, Dai B, Wang R, Fang Y, Yuan X, Wu T (2016) Multi-lane detection based on omnidirectional camera using anisotropic steerable filters. IET Intell Transp Syst 10(5):298–307. https://doi.org/10.1049/iet-its.2015.0144

    Article  Google Scholar 

  56. Piao J, Shin H (2017) Robust hypothesis generation method using binary blob analysis for multi-lane detection. IET Image Process 11(12):1210–1218. https://doi.org/10.1049/iet-ipr.2016.0506

    Article  Google Scholar 

  57. Zheng F, Luo S, Song K, Yan CW, Wang MC (2018) Improved lane line detection algorithm based on hough transform. Pattern Recognit Image Anal 28(2):254–260. https://doi.org/10.1134/S1054661818020049

    Article  Google Scholar 

  58. Wang Y, Shen D, Teoh EK (2000) Lane detection using spline model. Pattern Recognit Lett 21(8):677–689. https://doi.org/10.1016/S0167-8655(00)00021-0

    Article  Google Scholar 

  59. Wang Y, Dahnoun N, Achim A (2012) A novel system for robust lane detection and tracking. Signal Process 92(2):319–334. https://doi.org/10.1016/j.sigpro.2011.07.019

    Article  Google Scholar 

  60. Obradović D, Konjović Z, Pap E, Rudas IJ (2013) Linear fuzzy space based road lane model and detection. Knowl-Based Syst 38:37–47. https://doi.org/10.1016/j.knosys.2012.01.002

    Article  MATH  Google Scholar 

  61. Dewangan DK, Sahu SP (2021) Driving behaviour analysis of intelligent vehicle system for lane detection using vision-sensor. IEEE Sens J 21(5):6367–6375. https://doi.org/10.1109/JSEN.2020.3037340

    Article  Google Scholar 

  62. Son J, Yoo H, Kim S, Sohn K (2015) Real-time illumination invariant lane detection for lane departure warning system. Expert Syst Appl 42(4):1816–1824. https://doi.org/10.1016/j.eswa.2014.10.024

    Article  Google Scholar 

  63. Aly H, Basalamah A, Youssef M (2016) Robust and ubiquitous smartphone-based lane detection. Pervasive Mob Comput 26:35–56. https://doi.org/10.1016/j.pmcj.2015.10.019

    Article  Google Scholar 

  64. Aparna, Bhatia Y, Rai R, Gupta V, Aggarwal N, Akula A (2019) Convolutional neural networks based potholes detection using thermal imaging. J King Saud Univ Comput Inf Sci. https://doi.org/10.1016/j.jksuci.2019.02.004

    Article  Google Scholar 

  65. Dewangan DK, Sahu SP (2021) PotNet: pothole detection for autonomous vehicle system using convolutional neural network. Electron Lett 57(2):53–56. https://doi.org/10.1049/ell2.12062

    Article  Google Scholar 

  66. Varona B, Monteserin A, Teyseyre A (2019) A deep learning approach to automatic road surface monitoring and pothole detection. Pers Ubiquitous Comput. https://doi.org/10.1007/s00779-019-01234-z

    Article  Google Scholar 

  67. Koch C, Brilakis I (2011) Pothole detection in asphalt pavement images. Adv Eng Inform 25(3):507–515. https://doi.org/10.1016/j.aei.2011.01.002

    Article  Google Scholar 

  68. Murthy SBS, Varaprasad G (2014) Detection of potholes in autonomous vehicle. IET Intell Transp Syst 8(6):543–549. https://doi.org/10.1049/iet-its.2013.0138

    Article  Google Scholar 

  69. Jang DW, Park RH (2016) Pothole detection using spatio-Temporal saliency. IET Intell Transp Syst 10(9):605–612. https://doi.org/10.1049/iet-its.2016.0006

    Article  Google Scholar 

  70. Sawalakhe H, Prakash R (2018) Development of roads pothole detection system using image processing. Lect Notes Electr Eng 492:187–195. https://doi.org/10.1007/978-981-10-8575-8_20

    Article  Google Scholar 

  71. Li C, Song D, Tong R, Tang M (2019) Illumination-aware faster R-CNN for robust multispectral pedestrian detection. Pattern Recognit 85:161–171. https://doi.org/10.1016/j.patcog.2018.08.005

    Article  Google Scholar 

  72. Murugan BS, Elhoseny M, Shankar K, Uthayakumar J (2019) Region-based scalable smart system for anomaly detection in pedestrian walkways. Comput Electr Eng 75:146–160. https://doi.org/10.1016/j.compeleceng.2019.02.017

    Article  Google Scholar 

  73. Kim JH, Batchuluun G, Park KR (2018) Pedestrian detection based on faster R-CNN in nighttime by fusing deep convolutional features of successive images. Expert Syst Appl 114:15–33. https://doi.org/10.1016/j.eswa.2018.07.020

    Article  Google Scholar 

  74. Mateus A, Ribeiro D, Miraldo P, Nascimento JC (2019) Efficient and robust pedestrian detection using deep learning for human-aware navigation. Rob Auton Syst 113:23–37. https://doi.org/10.1016/j.robot.2018.12.007

    Article  Google Scholar 

  75. Dominguez-Sanchez A, Cazorla M, Orts-Escolano S (2017) Pedestrian movement direction recognition using convolutional neural networks. IEEE Trans Intell Transp Syst 18(12):3540–3548. https://doi.org/10.1109/TITS.2017.2726140

    Article  Google Scholar 

  76. Yoshihashi R, Trinh TT, Kawakami R, You S, Iida M, Naemura T (2018) Pedestrian detection with motion features via two-stream ConvNets. IPSJ Trans Comput Vis Appl. https://doi.org/10.1186/s41074-018-0048-5

    Article  Google Scholar 

  77. Yang D, Zhang J, Xu S, Ge S, Kumar GH, Zhang X (2018) Real-time pedestrian detection via hierarchical convolutional feature. Multimed Tools Appl 77(19):25841–25860. https://doi.org/10.1007/s11042-018-5819-6

    Article  Google Scholar 

  78. Ouyang W, Zeng X, Wang X (2016) Partial occlusion handling in pedestrian detection with a deep model. IEEE Trans Circ Syst Video Technol 26(11):2123–2137. https://doi.org/10.1109/TCSVT.2015.2501940

    Article  Google Scholar 

  79. Tomè STD, Monti F, Baroffion L, Bondi L, Tagliasacchi M (2016) Deep Convolutional Neural Networks for pedestrian detection. Signal Process Image Commun 47:482–489

    Article  Google Scholar 

  80. Suhr JK, Jung HG (2018) Rearview camera-based backover warning system exploiting a combination of pose-specific pedestrian recognitions. IEEE Trans Intell Transp Syst 19(4):1122–1129. https://doi.org/10.1109/TITS.2017.2709797

    Article  Google Scholar 

  81. Jiang Y, Wang J, Liang Y, Xia J (2019) Combining static and dynamic features for real-time moving pedestrian detection. Multimed Tools Appl 78(3):3781–3795. https://doi.org/10.1007/s11042-018-6057-7

    Article  Google Scholar 

  82. Kim J, Mesmakhosroshahi M (2013) Stereo-based region of interest generation for real-time pedestrian detection. Peer-to-Peer Netw Appl 8(2):181–188. https://doi.org/10.1007/s12083-013-0234-2

    Article  Google Scholar 

  83. Cao J, Pang Y, Li X (2016) Pedestrian detection inspired by appearance constancy and shape symmetry. IEEE Trans Image Process 25(12):5538–5551. https://doi.org/10.1109/TIP.2016.2609807

    Article  MathSciNet  MATH  Google Scholar 

  84. Biswas SK, Milanfar P (2017) Linear support tensor machine with LSK channels: pedestrian detection in thermal infrared images. IEEE Trans Image Process 26(9):4229–4242. https://doi.org/10.1109/TIP.2017.2705426

    Article  MathSciNet  MATH  Google Scholar 

  85. Hua C, Makihara Y, Yagi Y, Iwasaki S, Miyagawa K, Li B (2015) Onboard monocular pedestrian detection by combining spatio-temporal hog with structure from motion algorithm. Mach Vis Appl 26(2–3):161–183. https://doi.org/10.1007/s00138-014-0653-y

    Article  Google Scholar 

  86. Li H, Liu Y, Xiong S, Wang L (2015) Pedestrian detection algorithm based on video sequences and laser point cloud. Front Comput Sci 9(3):402–414. https://doi.org/10.1007/s11704-014-3413-2

    Article  Google Scholar 

  87. Zhang S, Bauckhage C, Cremers AB (2015) Efficient pedestrian detection via rectangular features based on a statistical shape model. IEEE Trans Intell Transp Syst 16(2):763–775. https://doi.org/10.1109/TITS.2014.2341042

    Article  Google Scholar 

  88. Hu X et al (2019) SINet: a scale-insensitive convolutional neural network for fast vehicle detection. IEEE Trans Intell Transp Syst 20(3):1010–1019. https://doi.org/10.1109/TITS.2018.2838132

    Article  Google Scholar 

  89. Cai Y, Wang H, Zheng Z, Sun X (2017) Scene-adaptive vehicle detection algorithm based on a composite deep structure. IEEE Access 5:22804–22811. https://doi.org/10.1109/ACCESS.2017.2756081

    Article  Google Scholar 

  90. Zhang Q, Wan C, Han W (2018) A modified faster region-based convolutional neural network approach for improved vehicle detection performance. Multimed Tools Appl. https://doi.org/10.1007/s11042-018-6769-8

    Article  Google Scholar 

  91. Wang X, Zhang W, Wu X, Xiao L, Qian Y, Fang Z (2019) Real-time vehicle type classification with deep convolutional neural networks. J Real-Time Image Process 16(1):5–14. https://doi.org/10.1007/s11554-017-0712-5

    Article  Google Scholar 

  92. Dai X (2019) HybridNet: a fast vehicle detection system for autonomous driving. Signal Process Image Commun 70:79–88. https://doi.org/10.1016/j.image.2018.09.002

    Article  Google Scholar 

  93. Li Y, Er MJ, Shen D (2015) A novel approach for vehicle detection using an AND-OR-graph-based multiscale model. IEEE Trans Intell Transp Syst 16(4):2284–2289. https://doi.org/10.1109/TITS.2014.2359493

    Article  Google Scholar 

  94. Tao H, Lu X (2019) Automatic smoky vehicle detection from traffic surveillance video based on vehicle rear detection and multi-feature fusion. IET Intell Transp Syst 13(2):293–302. https://doi.org/10.1049/iet-its.2018.5039

    Article  Google Scholar 

  95. Kuang H, Chen L, Chan LLH, Cheung RCC, Yan H (2019) “Feature selection based on tensor decomposition and object proposal for night-time multiclass vehicle detection. IEEE Trans Syst Man Cybern Syst 49(1):71–80. https://doi.org/10.1109/TSMC.2018.2872891

    Article  Google Scholar 

  96. Wang X, Xu L, Sun H, Xin J, Zheng N (2016) On-road vehicle detection and tracking using MMW radar and monovision fusion. IEEE Trans Intell Transp Syst 17(7):2075–2084. https://doi.org/10.1109/TITS.2016.2533542

    Article  Google Scholar 

  97. Dooley D, McGinley B, Hughes C, Kilmartin L, Jones E, Glavin M (2016) A blind-zone detection method using a rear-mounted fisheye camera with combination of vehicle detection methods. IEEE Trans Intell Transp Syst 17(1):264–278. https://doi.org/10.1109/TITS.2015.2467357

    Article  Google Scholar 

  98. Wang C, Fang Y, Zhao H, Guo C, Mita S, Zha H (2016) Probabilistic inference for occluded and multiview on-road vehicle detection. IEEE Trans Intell Transp Syst 17(1):215–229. https://doi.org/10.1109/TITS.2015.2466109

    Article  Google Scholar 

  99. Murugan V, Vijaykumar VR (2018) Automatic moving vehicle detection and classification based on artificial neural fuzzy inference system. Wirel Pers Commun 100(3):745–766. https://doi.org/10.1007/s11277-018-5347-8

    Article  Google Scholar 

  100. Tang Y, Zhang C, Gu R, Li P, Yang B (2017) Vehicle detection and recognition for intelligent traffic surveillance system. Multimed Tools Appl 76(4):5817–5832. https://doi.org/10.1007/s11042-015-2520-x

    Article  Google Scholar 

  101. Zhang D (2018) Vehicle target detection methods based on color fusion deformable part model. Eurasip J Wirel Commun Netw. https://doi.org/10.1186/s13638-018-1111-8

    Article  Google Scholar 

  102. Wei Y, Tian Q, Guo J, Huang W, Cao J (2019) Multi-vehicle detection algorithm through combining Harr and HOG features. Math Comput Simul 155:130–145. https://doi.org/10.1016/j.matcom.2017.12.011

    Article  MathSciNet  MATH  Google Scholar 

  103. Anandhalli M, Baligar VP (2018) A novel approach in real-time vehicle detection and tracking using Raspberry Pi. Alexandria Eng J 57(3):1597–1607. https://doi.org/10.1016/j.aej.2017.06.008

    Article  Google Scholar 

  104. Ra M, Jung HG, Suhr JK, Kim WY (2018) Part-based vehicle detection in side-rectilinear images for blind-spot detection. Expert Syst Appl 101:116–128. https://doi.org/10.1016/j.eswa.2018.02.005

    Article  Google Scholar 

  105. Fang Song J (2018) Vehicle detection using spatial relationship gmm for complex urban surveillance in daytime and nighttime. Int J Parallel Prog 46(5):859–872. https://doi.org/10.1007/s10766-017-0543-9

    Article  Google Scholar 

  106. Nam Y, Nam YC (2018) Vehicle classification based on images from visible light and thermal cameras. Eurasip J Image Video Process 2018(1):2–10. https://doi.org/10.1186/s13640-018-0245-2

    Article  Google Scholar 

  107. Zhou Y, Liu L, Shao L, Mellor M (2018) Fast automatic vehicle annotation for urban traffic surveillance. IEEE Trans Intell Transp Syst 19(6):1973–1984. https://doi.org/10.1109/TITS.2017.2740303

    Article  Google Scholar 

  108. Wang JG, Zhou LB (2019) Traffic light recognition with high dynamic range imaging and deep learning. IEEE Trans Intell Transp Syst 20(4):1341–1352. https://doi.org/10.1109/TITS.2018.2849505

    Article  Google Scholar 

  109. Hirabayashi M, Sujiwo A, Monrroy A, Kato S, Edahiro M (2019) Traffic light recognition using high-definition map features. Rob Auton Syst 111:62–72. https://doi.org/10.1016/j.robot.2018.10.004

    Article  Google Scholar 

  110. Lu Y, Lu J, Zhang S, Hall P (2018) Traffic signal detection and classification in street views using an attention model. Comput Vis Media 4(3):253–266. https://doi.org/10.1007/s41095-018-0116-x

    Article  Google Scholar 

  111. Lee E, Kim D (2019) Accurate traffic light detection using deep neural network with focal regression loss. Image Vis Comput 87:24–36. https://doi.org/10.1016/j.imavis.2019.04.003

    Article  Google Scholar 

  112. Luo H, Yang Y, Tong B, Wu F, Fan B (2018) Traffic sign recognition using a multi-task convolutional neural network. IEEE Trans Intell Transp Syst 19(4):1100–1111. https://doi.org/10.1109/TITS.2017.2714691

    Article  Google Scholar 

  113. Lee HS, Kim K (2018) Simultaneous traffic sign detection and boundary estimation using convolutional neural network. IEEE Trans Intell Transp Syst 19(5):1652–1663. https://doi.org/10.1109/TITS.2018.2801560

    Article  Google Scholar 

  114. Zhu Y, Liao M, Yang M, Liu W (2018) Cascaded segmentation-detection networks for text-based traffic sign detection. IEEE Trans Intell Transp Syst 19(1):209–219. https://doi.org/10.1109/TITS.2017.2768827

    Article  Google Scholar 

  115. Li J, Wang Z (2019) Real-time traffic sign recognition based on efficient CNNs in the wild. IEEE Trans Intell Transp Syst 20(3):975–984. https://doi.org/10.1109/TITS.2018.2843815

    Article  Google Scholar 

  116. Yuan Y, Xiong Z, Wang Q (2019) VSSA-NET: vertical spatial sequence attention network for traffic sign detection. IEEE Trans Image Process 28(7):3423–3434. https://doi.org/10.1109/TIP.2019.2896952

    Article  MathSciNet  MATH  Google Scholar 

  117. Liu Z, Du J, Tian F, Wen J (2019) MR-CNN: a multi-scale region-based convolutional neural network for small traffic sign recognition. IEEE Access 7:57120–57128. https://doi.org/10.1109/ACCESS.2019.2913882

    Article  Google Scholar 

  118. Wang G, and Ma X (2018) Traffic police gesture recognition using RGB-D and faster R-CNN. In: 2018 Int. Conf. Intell. Informatics Biomed. Sci. ICIIBMS 2018, vol. 3, pp 78–81. https://doi.org/10.1109/ICIIBMS.2018.8549975

  119. Wang H, Zhang Q, Du Y, and Wang S (2019) Traffic police pose estimation based on multi-branch network. In: Proc. 2018 Chinese Autom. Congr. CAC 2018, no. 1, pp 1769–1773. https://doi.org/10.1109/CAC.2018.8623106

  120. Y C, Yin J (2018) Hand gesture recognition using in intelligent transportation. Cogn Syst Signal Process 1006:52–64

    Google Scholar 

  121. Alam A, Jaffery ZA (2019) Indian traffic sign detection and recognition. Int J Intell Transp Syst Res. https://doi.org/10.1007/s13177-019-00178-1

    Article  Google Scholar 

  122. Guo F, Cai Z and Tang J (2011) Chinese traffic police gesture recognition in complex scene. In: Proc. 10th IEEE Int. Conf. Trust. Secur. Priv. Comput. Commun. Trust. 2011, 8th IEEE Int. Conf. Embed. Softw. Syst. ICESS 2011, 6th Int. Conf. FCST 2011, pp 1505–1511. https://doi.org/10.1109/TrustCom.2011.208

  123. Jang C, Cho S, Jeong S, Suhr JK, Jung HG, Sunwoo M (2017) Traffic light recognition exploiting map and localization at every stage. Expert Syst Appl 88:290–304. https://doi.org/10.1016/j.eswa.2017.07.003

    Article  Google Scholar 

  124. Hou YL, Hao X, Chen H (2017) A cognitively motivated method for classification of occluded traffic signs. IEEE Trans Syst Man Cybern Syst 47(2):255–262. https://doi.org/10.1109/TSMC.2016.2560126

    Article  Google Scholar 

  125. Yuan Y, Xiong Z, Wang Q (2017) An incremental framework for video-based traffic sign detection, tracking, and recognition. IEEE Trans Intell Transp Syst 18(7):1918–1929. https://doi.org/10.1109/TITS.2016.2614548

    Article  Google Scholar 

  126. Zhu Z, Lu J, Martin RR, Hu S (2017) An optimization approach for localization refinement of candidate traffic signs. IEEE Trans Intell Transp Syst 18(11):3006–3016. https://doi.org/10.1109/TITS.2017.2665647

    Article  Google Scholar 

  127. Sathya R, Kalaiselvi Geetha M (2015) Framework for traffic personnel gesture recognition. Procedia Comput Sci 46:1700–1707. https://doi.org/10.1016/j.procs.2015.02.113

    Article  Google Scholar 

  128. Prakash A, Swathi R, Kumar S, Ashwin TS and Reddy GRM (2017) Kinect based real time gesture recognition tool for air marshallers and traffic policemen. In: Proc. - IEEE 8th Int. Conf. Technol. Educ. T4E 2016, pp 34–37, https://doi.org/10.1109/T4E.2016.015

  129. Guo F, Tang J, Wang X (2017) Gesture recognition of traffic police based on static and dynamic descriptor fusion. Multimed Tools Appl 76(6):8915–8936. https://doi.org/10.1007/s11042-016-3497-9

    Article  Google Scholar 

  130. Everingham M, Van Gool L, Williams CKI, Winn J, Zisserman A (2010) The pascal visual object classes (VOC) challenge. Int J Comput Vis 88(2):303–338. https://doi.org/10.1007/s11263-009-0275-4

    Article  Google Scholar 

  131. Zhou B, Zhao H, Puig X, Fidler S, Barriuso A, and Torralba A (2017) Scene parsing through ADE20K dataset. In: Proc. - 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017, vol. 2017-Janua, pp 5122–5130, https://doi.org/10.1109/CVPR.2017.544

  132. Zhou B et al (2019) Semantic understanding of scenes through the ADE20K Dataset. Int J Comput Vis 127(3):302–321. https://doi.org/10.1007/s11263-018-1140-0

    Article  Google Scholar 

  133. Gould S, Fulton R, Koller D (2009) Decomposing a scene into geometric and semantically consistent regions. Proc IEEE Int Conf Comput Vis. https://doi.org/10.1109/ICCV.2009.5459211

    Article  Google Scholar 

  134. Song S, Lichtenberg SP and Xiao J (2015) SUN RGB-D: a RGB-D scene understanding benchmark suite. In: Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 07–12-June, pp 567–576, https://doi.org/10.1109/CVPR.2015.7298655

  135. Aly M (2008) Real time detection of lane markers in urban streets. IEEE Intell Veh Symp Proc. https://doi.org/10.1109/IVS.2008.4621152

    Article  Google Scholar 

  136. Maddern W, Pascoe G, Linegar C and Newman P (2015) 1 Year, 1000 km : The Oxford RobotCar Dataset,” vol. 3

  137. Pan X, Shi J, Luo P, Wang X, Tang X (2018) Spatial as deep: spatial CNN for traffic scene understanding. 32nd AAAI Conf Artif Intell AAAI 2018:7276–7283

    Google Scholar 

  138. Veit T, Tarel JP, Nicolle P, Charbonnier P (2008) Evaluation of road marking feature extraction. IEEE Conf Intell Transp Syst Proc. https://doi.org/10.1109/ITSC.2008.4732564

    Article  Google Scholar 

  139. Image D and Korea S (2017) DIML/CVL RGB-D dataset: 2M RGB-D images of natural indoor and outdoor scenes. no. Diml, pp 1–7, Available: https://dimlrgbd.github.io/

  140. Hwang S, Park J, Kim N, Choi Y, Kweon IS (2015) Multispectral pedestrian detection: aenchmark dataset and baseline. Proc IEEE Comput Soc Conf Comput Vis Pattern Recogn. https://doi.org/10.1109/CVPR.2015.7298706

    Article  Google Scholar 

  141. Chan AB, Vasconcelos N (2008) Modeling, clustering, and segmenting video with mixtures of dynamic textures. IEEE Trans Pattern Anal Mach Intell 30(5):909–926. https://doi.org/10.1109/TPAMI.2007.70738

    Article  Google Scholar 

  142. Surasak T, Takahiro I, Cheng CH, Wang CE and Sheng PY (2018) Histogram of oriented gradients for human detection in video. In: Proc. 2018 5th Int. Conf. Bus. Ind. Res. Smart Technol. Next Gener. Information, Eng. Bus. Soc. Sci. ICBIR 2018, pp. 172–176, https://doi.org/10.1109/ICBIR.2018.8391187

  143. Cao J, Pang Y, Li X (2017) Learning multilayer channel features for pedestrian detection. IEEE Trans Image Process 26(7):3210–3220. https://doi.org/10.1109/TIP.2017.2694224

    Article  MathSciNet  MATH  Google Scholar 

  144. Enzweiler M, Eigenstetter A, Schiele B, Gavrila DM (2010) Multi-cue pedestrian classification with partial occlusion handling. Proc IEEE Comput Soc Conf Comput Vis Pattern Recogn. https://doi.org/10.1109/CVPR.2010.5540111

    Article  Google Scholar 

  145. Dollar P, Wojek C, Schiele B and Perona P (2010) Pedestrian detection: a benchmark. pp 304–311. https://doi.org/10.1109/cvpr.2009.5206631

  146. Ouyang X, Cheng Y, Jiang Y, Li C-L and Zhou P (2018) Pedestrian-synthesis-GAN: generating pedestrian data in real scene and beyond. Available: http://arxiv.org/abs/1804.02047

  147. Li X et al (2017) A unified framework for concurrent pedestrian and cyclist detection. IEEE Trans Intell Transp Syst 18(2):269–281. https://doi.org/10.1109/TITS.2016.2567418

    Article  Google Scholar 

  148. Olmeda D, Premebida C, Nunes U, Armingol JM, De La Escalera A (2013) Pedestrian detection in far infrared images. Integr Comput Aided Eng 20(4):347–360. https://doi.org/10.3233/ICA-130441

    Article  Google Scholar 

  149. Wu Z, Fuller N, Theriault D, Betke M (2014) A thermal infrared video benchmark for visual analysis. IEEE Comput Soc Conf Comput Vis Pattern Recogn Work. https://doi.org/10.1109/CVPRW.2014.39

    Article  Google Scholar 

  150. Sivaraman S, Trivedi MM (2010) A general active-learning framework for on-road vehicle recognition and tracking. IEEE Trans Intell Transp Syst 11(2):267–276. https://doi.org/10.1109/TITS.2010.2040177

    Article  Google Scholar 

  151. Zhu Z, Liang D, Zhang S, Huang X, Li B, Hu S (2016) Traffic-sign detection and classification in the wild. Proc IEEE Comput Soc Conf Comput Vis Pattern Recogn 2016:2110–2118. https://doi.org/10.1109/CVPR.2016.232

    Article  Google Scholar 

  152. Behrendt K, Novak L, Botros R (2017) A deep learning approach to traffic lights: detection, tracking, and classification. Proc IEEE Int Conf Robot Autom. https://doi.org/10.1109/ICRA.2017.7989163

    Article  Google Scholar 

  153. Møgelmose A, Trivedi MM, Moeslund TB (2012) Vision-based traffic sign detection and analysis for intelligent driver assistance systems: perspectives and survey. IEEE Trans Intell Transp Syst 13(4):1484–1497. https://doi.org/10.1109/TITS.2012.2209421

    Article  Google Scholar 

  154. Stallkamp J, Schlipsing M, Salmen J, Igel C (2012) Man vs. computer: benchmarking machine learning algorithms for traffic sign recognition. Neural Netw 32:323–332. https://doi.org/10.1016/j.neunet.2012.02.016

    Article  Google Scholar 

  155. Larsson F, Felsberg M (2011) Using Fourier descriptors and spatial models for traffic sign recognition. Lect Notes Comput Sci 6688:238–249. https://doi.org/10.1007/978-3-642-21227-7_23 (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics)

    Article  Google Scholar 

  156. Bonaci I, Kusalic I and Kovacek I (2011) Addressing false alarms and localization inaccuracy in traffic sign detection and recognition. In: 16Th Comput. Vis. Available: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.225.5845&rep=rep1&type=pdf

  157. Huang L Chinese traffic sign database. http://www.nlpr.ia.ac.cn/pal/trafficdata/index.html. Available: http://www.nlpr.ia.ac.cn/pal/trafficdata/index.html

  158. Stallkamp J, Schlipsing M, Salmen J, Igel C (2011) The German Traffic Sign recognition benchmark: a multi-class classification competition. Proc Int Jt Conf Neural Netw 1:1453–1460. https://doi.org/10.1109/IJCNN.2011.6033395

    Article  Google Scholar 

  159. Dosovitskiy A, Ros G, Codevilla F, Lopez A and Koltun V (2017) CARLA: an open urban driving simulator. CoRL, pp 1–16. Available: http://arxiv.org/abs/1711.03938

  160. DRIVE Constellation—Simulation for Self Driving Cars. https://www.nvidia.com/en-in/self-driving-cars/simulation/ (accessed Jan. 14, 2022)

  161. Deep Drive—Self Driving AI Simulator. https://deepdrive.io/ (accessed Jan. 14, 2022)

  162. Woods JO, Christian JA (2016) GLIDAR: An open GL-based, real-time, and open source 3d sensor simulator for testing computer vision algorithms. J Imaging. https://doi.org/10.3390/jimaging2010005

    Article  Google Scholar 

  163. Bechtold S and Höfle B (2016) Helios: a multi-purpose lidar simulation framework for research, planning and training of laser scanning operations with airborne, ground-based mobile and stationary platforms. In: ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci., vol. III–3, no. July, pp 161–168. https://doi.org/10.5194/isprsannals-iii-3-161-2016.

  164. Radar Simulator (RADSim). https://www.riversideresearch.org/what-we-do/tools/radar-simulator-radsim (accessed Jan 14, 2022)

  165. Méndez V, Catalán H, Rosell JR, Arnó J, Sanz R, Tarquis A (2012) SIMLIDAR—simulation of LIDAR performance in artificially simulated orchards. Biosyst Eng 111(1):72–82. https://doi.org/10.1016/j.biosystemseng.2011.10.010

    Article  Google Scholar 

  166. Udacity Open Sources—Self-Driving Car Simulator. https://techcrunch.com/2017/02/08/udacity-open-sources-its-self-driving-car-simulator-for-anyone-to-use/ (accessed Jan 14, 2022)

  167. Waymos Simulation City for Autonomous Driving. https://blog.waymo.com/2021/06/SimulationCity.html (accessed Jan 14, 2022)

  168. CarSim-ADAS and Automated Driving. https://www.carsim.com/ (accessed Dec 15, 2021)

  169. Gazebo—Robot Simulation. http://gazebosim.org/ (accessed Jan 14, 2022)

  170. SVL SIMULATOR—An end-to-end autonomous vehicle simulation platform. https://www.svlsimulator.com/ (accessed Jan 14, 2022)

  171. Automated Driving Toolbox—MATLAB. https://in.mathworks.com/products/automated-driving.html (accessed Jan 14, 2022)

  172. PreScan-Simulation of ADAS and active safety. https://in.mathworks.com/products/connections/product_detail/prescan.html (accessed Jan 14, 2022)

  173. New Autonomous Mileage Reports Are Out, but Is the Data Meaningful? https://www.ttnews.com/articles/new-autonomous-mileage-reports-are-out-data-meaningful (accessed Jan 14, 2022)

  174. Stilgoe J (2018) Machine learning, social learning and the governance of self-driving cars. Soc Stud Sci 48(1):25–56. https://doi.org/10.1177/0306312717741687

    Article  Google Scholar 

  175. Self-driving cars more likely to drive into black people, study claims. Independent Newspaper [website] 2019. https://www.independent.co.uk/life-style/gadgets-and-tech/news/self-driving-car-crash-racial-bias-black-people-study-a8810031.html

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Siksha O Anusandhan Deemed to be University, Bhubaneswar, Odisha, India

    Deepak Kumar Dewangan

  2. Department of Information Technology, National Institute of Technology, Raipur (C.G.), India

    Satya Prakash Sahu

Authors
  1. Deepak Kumar Dewangan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Satya Prakash Sahu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Deepak Kumar Dewangan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dewangan, D.K., Sahu, S.P. Towards the design of vision-based intelligent vehicle system: methodologies and challenges. Evol. Intel. 16, 759–800 (2023). https://doi.org/10.1007/s12065-022-00713-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12065-022-00713-2

Keywords

Search

Navigation