Skip to main content
SpringerLink
Log in

A Novel Convolutional Neural Network Based Architecture for Object Detection and Recognition with an Application to Traffic Sign Recognition from Road Scenes

  • PATTERN RECOGNITION AND IMAGE ANALYSIS AUTOMATED SYSTEMS, HARDWARE AND SOFTWARE
  • Published:
Pattern Recognition and Image Analysis Aims and scope Submit manuscript

Abstract

Object detection and recognition is a significant activity in computer vision applications. Advanced driver assistance systems (ADAS) uses computer vision predominantly as its tool. For improving the performance of ADAS, traffic sign is one of the important object that needs to be detected and recognized to assist the drivers for safe driving. Under real time conditions, this befits extremely challenging due to varying illumination, resolution of images, external weather conditions, position of sign board and occlusions. This article proposes an efficient algorithm that can detect, and classify (recognize) the traffic signs. This traffic sign processing has been done in two phases: sign detection and sign recognition through classification. In the first phase, the traffic signs are detected using YOLOv3 architecture by generating seven classes based on shape, color and background. In phase 2, traffic sign classification has been done using the newly proposed architecture based on convolutional neural networks, using the output generated from the first phase. The German Traffic Sign Detection Benchmark (GTSDB) and German Traffic Sign Recognition Benchmark (GTSRB) datasets have been used for experimentation. The proposed method gives a mean average precision of 89.56% for traffic sign detection with an accuracy of 86.6% for traffic sign recognition. This shows the efficacy of the proposed architecture.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.
Fig. 10.
Fig. 11.

Similar content being viewed by others

REFERENCES

  1. A. M. Geetha and S. K. Thangavel, “Deep learning for driver assistance using estimated trajectory complexity parameter,” J. Adv. Res. Dyn. Control Syst. 10 (9), 871–879 (2018).

    Google Scholar 

  2. Á. Arcos-García, J. A. Álvarez-García, and L. M. Soria-Morillo, “Deep neural network for traffic sign recognition systems: An analysis of spatial transformers and stochastic optimisation methods,” Neural Networks 99, 158-165 (2018).  https://doi.org/10.1016/j.neunet.2018.01.005

    Article  Google Scholar 

  3. Á. Arcos-García, J. A. Álvarez-García, and L. M. Soria-Morillo, “Evaluation of deep neural networks for traffic sign detection systems,” Neurocomputing 316, 332–344 (2018).  https://doi.org/10.1016/j.neucom.2018.08.009

    Article  Google Scholar 

  4. X. Changzhen, W. Cong, M. Weixin, and S. Yanmei, “A traffic sign detection algorithm based on deep convolutional neural network,” in IEEE Int. Conf. on Signal and Image Processing (ICSIP), Beijing, 2016 (IEEE, 2016), pp. 676–679.  https://doi.org/10.1109/SIPROCESS.2016.7888348

  5. J. Choi, D. Chun, H. Kim, and H.-J. Lee, “Gaussian YOLOv3: An accurate and fast object detector using localization uncertainty for autonomous driving,” in IEEE/CVF Int. Conf. on Computer Vision (ICCV), Seoul, 2019 (IEEE, 2019), pp. 502–511.  https://doi.org/10.1109/ICCV.2019.00059

  6. A. Ćorović, V. Ilić, S. Ðurić, M. Marijan, and B. Pavković, “The real-time detection of traffic participants using yolo algorithm,” in 26th Telecommunications Forum (TELFOR), Belgrade, 2018 (IEEE, 2018), pp. 1–4.  https://doi.org/10.1109/TELFOR.2018.8611986

  7. J. Dharneeshkar, V. Soban Dhakshana, S. A. Aniruthan, R. Karthika, and L. Parameswaran, “Deep Learning based Detection of potholes in Indian roads using YOLO,” in Int. Conf. on Inventive Computation Technologies (ICICT), Coimbatore, India, 2020 (IEEE, 2020), pp. 381–385.  https://doi.org/10.1109/ICICT48043.2020.9112424

  8. A. Dominguez-Sanchez, M. Cazorla, and S. Orts-Escolano, “A new dataset and performance evaluation of a region-based CNN for urban object detection,” Electronics 7, 301 (2018).  https://doi.org/10.3390/electronics7110301

    Article  Google Scholar 

  9. A. Ellahyani, M. El Ansari, and I. El Jaafari, “Traffic sign detection and recognition based on random forests,” Appl. Soft Comput. 46, 805–815 (2016).  https://doi.org/10.1016/j.asoc.2015.12.041

    Article  Google Scholar 

  10. R. Gavrilescu, C. Zet, C. Foșalău, M. Skoczylas, and D. Cotovanu, “Faster R-CNN: An approach to real-time object detection,” in Int. Conf. and Exposition on Electrical and Power Engineering (EPE), Iasi, Romania, 2018 (IEEE, 2018), pp. 0165-0168.  https://doi.org/10.1109/ICEPE.2018.8559776

  11. R. Gokul, A. Nirmal, K. M. Bharath, M. P. Pranesh, and R. Karthika, “A comparative study between state-of-the-art object detectors for traffic light detection,” in Int. Conf. on Emerging Trends in Information Technology and Engineering (ic-ETITE), Vellore, India, 2020 (IEEE, 2020), pp. 1–6.  https://doi.org/10.1109/ic-ETITE47903.2020.449

  12. C. Han, G. Gao, and Y. Zhang, “Real-time small traffic sign detection with revised faster-RCNN,” Multimedia Tools Appl. 78, 13263–13278 (2019).  https://doi.org/10.1007/s11042-018-6428-0

    Article  Google Scholar 

  13. H. Haritha and T. S. Kumar, “Survey on various traffic monitoring and reasoning techniques,” in Artificial Intelligence Trends in Intelligent Systems. CSOC 2017, Ed. by R. Silhavy, R. Senkerik, Z. Kominkova Oplatkova, Z. Prokopova, and P. Silhavy, Advances in Intelligent Systems and Computing, vol. 573 (Springer, Cham, 2017), pp. 507–516.  https://doi.org/10.1007/978-3-319-57261-1_50

  14. INI Benchmark Website. Institut für Neuroinformatik. http://benchmark.ini.rub.de/?section=gtsdb&subsection=news.

  15. K. I. Kiy, “A new method of global image analysis and its application in understanding road scenes,” Pattern Recognit. Image Anal. 28, 483–495 (2018).  https://doi.org/10.1134/S1054661818030100

    Article  Google Scholar 

  16. J. Li and Z. Wang, “Real-time traffic sign recognition based on efficient CNNs in the wild,” IEEE Trans. Intell. Transp. 20, 975–984 (2019).  https://doi.org/10.1109/TITS.2018.2843815

    Article  Google Scholar 

  17. D. Li, D. Zhao, Y. Chen, and Q. Zhang, “Deepsign: Deep learning based traffic sign recognition,” in Int. Joint Conf. on Neural Networks (IJCNN), Rio de Janeiro, 2018 (IEEE, 2018), pp. 1–6.  https://doi.org/10.1109/IJCNN.2018.8489623

  18. J. M. Lillo-Castellano, I. Mora-Jiménez, C. Figuera-Pozuelo, and J. L. Rojo-Álvarez, “Traffic sign segmentation and classification using statistical learning methods,” Neurocomputing 153, 286–299 (2015). doi https://doi.org/10.1016/j.neucom.2014.11.026

    Article  Google Scholar 

  19. H. Liu, Y. Liu, and F. Sun, “Traffic sign recognition using group sparse coding,” Inf. Sci. 266, 75–89 (2014).  https://doi.org/10.1016/j.ins.2014.01.010

    Article  Google Scholar 

  20. Z. Meng, X. Fan, X. Chen, M. Chen, and Y. Tong, “Detecting small signs from large images,” in IEEE Int. Conf. on Information Reuse and Integration (IRI), San Diego, 2017 (IEEE, 2017), pp. 217–224.  https://doi.org/10.1109/IRI.2017.57

  21. J. G. Park and K. J. Kim, “Design of a visual perception model with edge-adaptive Gabor filter and support vector machine for traffic sign detection,” Exp. Syst. Appl. 40, 3679–3687 (2013).  https://doi.org/10.1016/j.eswa.2012.12.072

    Article  Google Scholar 

  22. P. Pham, D. Nguyen, T. Do, T. D. Ngo, and D. D. Le, “Evaluation of deep models for real-time small object detection,” in Neural Information Processing, Ed. by D. Liu, S. Xie, Y. Li, D. Zhao, and E. S. El-Alfy, Lecture Notes in Computer Science, vol. 10636 (Springer, Cham, 2017), pp. 516–526.  https://doi.org/10.1007/978-3-319-70090-8_53

    Book  Google Scholar 

  23. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016 (IEEE, 2016), pp. 779–788.  https://doi.org/10.1109/CVPR.2016.91

  24. J. Redmon and A. Farhadi, “YOLO9000: better, faster, stronger,” in IEEE Conf. on Computer Vision and Pattern Recognition, Honolulu, Hawaii, 2017 (IEEE, 2017), pp. 7263–7271.  https://doi.org/10.1109/CVPR.2017.690

  25. J. Redmon and A. Farhadi, “Yolov3: An incremental improvement” (2018). arXiv:1804.02767

  26. S. M.Staroletov, M. A. Laptev, and D. V. Nekrasov, “Development and testing of algorithms for vehicle type recognition and car tracking with photo and video traffic enforcement cameras,” Pattern Recognit. Image Anal. 31, 323–333.  https://doi.org/10.1134/S1054661821020152

  27. Z. L. Sun, H. Wang, W.-S. Lau, G. Seet, and D. Wang, “Application of BW-ELM model on traffic sign recognition,” Neurocomputing 128, 153–159 (2014).  https://doi.org/10.1016/j.neucom.2012.11.057

    Article  Google Scholar 

  28. P. Y. Yakimov, “Preprocessing digital images for quickly and reliably detecting road signs,” Pattern Recognit. Image Anal. 25, 729–732 (2015).  https://doi.org/10.1134/S1054661815040264

    Article  Google Scholar 

  29. F. Zaklouta and B. Stanciulescu, “Real-time traffic sign recognition in three stages,” Rob. Autonom. Syst. 62, 16–24 (2014).  https://doi.org/10.1016/j.robot.2012.07.019

    Article  Google Scholar 

  30. J. Zhang, M. Huang, X. Jin, and X. Li, “A real-time Chinese traffic sign detection algorithm based on modified YOLOv2,” Algorithms 10, 127 (2017).  https://doi.org/10.3390/a10040127

    Article  MathSciNet  MATH  Google Scholar 

  31. Y. Zhu, C. Zhang, D. Zhou, X. Wang, X. Bai, and W. Liu, “Traffic sign detection and recognition using fully convolutional network guided proposals,” Neurocomputing 214, 758–766 (2016).  https://doi.org/10.1016/j.neucom.2016.07.009

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, Amrita School of Engineering, 641112, Coimbatore, Amrita Vishwa Vidyapeetham, India

    R. Karthika

  2. Department of Computer Science and Engineering, Amrita School of Engineering, 641112, Coimbatore, Amrita Vishwa Vidyapeetham, India

    Latha Parameswaran

Authors
  1. R. Karthika
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Latha Parameswaran
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to R. Karthika or Latha Parameswaran.

Ethics declarations

COMPLIANCE WITH ETHICAL STANDARDS

This article is a completely original work of its authors; it has not been published before and will not be sent to other publications until the PRIA Editorial Board decides not to accept it for publication.

Conflict of Interest

The authors declare that they have no conflicts of interest.

Additional information

R. Karthika is working as an Assistant Professor in the department of Electronics and Communication Engineering, Amrita Vishwa Vidyapeetham, Coimbatore. She completed her B. E. degree in Electronics and Communication from Madras University. She received her M.Tech. degree in Computer Vision and Image Processing from Amrita Vishwa Vidyapeetham. Her current research interests include pattern recognition, image processing and deep learning.

Dr. Latha Parameswaran serving as Honorary Distinguished Professor with Department of Computer science and Engineering, Amrita Vishwa Vidyapeetham. She completed her Master’s Degree from PSG College of Technology and her PhD from Bharathiar University. Her areas of research include image processing, information retrieval, image mining, information security and theoretical computer science.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Karthika, R., Parameswaran, L. A Novel Convolutional Neural Network Based Architecture for Object Detection and Recognition with an Application to Traffic Sign Recognition from Road Scenes. Pattern Recognit. Image Anal. 32, 351–362 (2022). https://doi.org/10.1134/S1054661822020110

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S1054661822020110

Keywords:

Search

Navigation