Skip to main content
SpringerLink
Log in

Traffic-light sign recognition using capsule network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Automated driving gradually emerges as a real reality, but it still has to face various challenges, including sophisticated and volatile traffic conditions, human operating faults, etc. Amongst them, accurate understanding of traffic signs by using computer vision and deep learning methods has great significance for driving safety. In recent years, the advent of deep learning has made this issue much effectively. In this article, our major goal is to use deep learning models for conducting traffic-light sign recognition related to autonomous vehicles. As far as we know, this is the first time that the capsule neural network is employed as a method for scene understanding so as to effectively identify a class of traffic-light signs. Compared with the well-known convolutional neural networks, capsule networks diminish the demands for training datasets and tackle the spatial relationship much precisely.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Bach M, Stumper D, Dietmayer K (2018) Deep convolutional traffic light recognition for automated driving. Intelligent Transportation Systems, 851–858

  2. Behrendt K, Novak L, Botros R (2017) A deep learning approach to traffic lights: Detection, tracking, and classification. IEEE International Conference on Robotics and Automation (ICRA), Singapore, 1370–1377

  3. Bernstein D (2010) Essentials of psychology. Cengage Learning, 123–124

  4. Brefczynski-Lewis JA, Lewis JW (2017) Auditory object perception: A neurobiological model and prospective review. Neuropsychologia, 223–242

  5. Han C, Gao G, Zhang Y (2018) Real-time small traffic sign detection with revised faster-RCNN. Springer Multimedia Tools and Applications. 78

  6. Husain F, Dellen B, Torras C (2017) Scene Understanding Using Deep Learning. Academic Press, 373–382

  7. Jeon HS, Kum DS, Jeong WY (2018) Traffic scene prediction via deep learning: Introduction of multichannel occupancy grid map as a scene representation. IEEE Intelligent Vehicles Symposium (IV), 1496–1501

  8. John V, Yoneda K, Qi B, Liu Z, Mita S (2014) Traffic light recognition in varying illumination using deep learning and saliency map. Intelligent Transportation Systems, 2286–2291

  9. Kheradpisheh S, Ghodrati M, Ganjtabesh M et al (2016) Deep networks can resemble human feed-forward vision in invariant object recognition. Nature Sci Report 6:32672

    Google Scholar 

  10. Kim M, Chi S (2019) Detection of centerline crossing in abnormal driving using CapsNet. Journal of Supercomputer, 189–196

  11. Kim H, Park J, Jung H (2018) An efficient color space for deep learning based traffic light recognition. Advanced Transportation, pp.1–12

  12. Kim Y, Wang P, Zhu Y, Mihaylova L (2019) A capsule network for traffic speed prediction in complex road networks. Sensor Data Fusion: Trends, Solutions, Applications (SDF), 1–6

  13. Kwabena M, Felix A, Abra A, Edward Y (2019) Capsule networks – A survey. Journal of King Saud University - Computer and Information Sciences, 1–16

  14. Lewicki MS, Olshausen BA, Surlykke A, Moss CF (2014) Scene analysis in the natural environment. Frontiers in Psychology, 199

  15. Malcolm GL, Groen I, Baker CI (2016) Making sense of real-world scenes. Trends in Cognitive Sciences, 843–856

  16. Morris T (2014) Computer vision and image processing. Palgrave Macmillan

  17. Mukhometzianov R, Carrillo J (2018) CapsNet comparative performance evaluation for image classification. Computational Intelligence and Security, 1–14

  18. Müller J, Dietmayer K (2018) Detecting traffic lights by single shot detection. Intelligent Transportation Systems, 266–273

  19. Nandi D, Saif AS, Prottoy P, Zubair KM, Shubho SA (2018) Traffic sign detection based on color segmentation of obscure image candidates: a comprehensive study. Int J Mod Educ Comput Sci 10(6):35–46

    Article  Google Scholar 

  20. Park H, Jang S, Jeong H, Ha Y (2019) Roadway image preprocessing for deep learning-based driving scene understanding. IEEE International Conference on Big Data and Smart Computing (BigComp), pp. 1–4

  21. Peixinho AZ, Benato BC, Nonato LG, Falcão AX (2018) Delaunay triangulation data augmentation guided by visual analytics for deep learning. SIBGRAPI Conference on Graphics, Patterns and Images, 384–391

  22. Qiao K, Chen J, Wang L, Zhang C, Zeng L, Tong L, Yan B (2019) Category decoding of visual stimuli from human brain activity using a bidirectional recurrent neural network to simulate bidirectional information flows in human visual cortices. Frontiers in Neuroscience, 692

  23. Qu H, Zhang L, Wu X, He X, Hu X, Wen X (2019) Multiscale object detection in infrared streetscape images based on deep learning and instance level data augmentation. Applied Sciences, 553–565

  24. Saadna Y, Behloul A (2017) An overview of traffic sign detection and classification methods. Multimedia Information Retrieval, 1–18

  25. Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. International Conference on Neural Information Processing Systems, pp.3859–386

  26. Sermanet P, LeCun Y (2011) Traffic sign recognition with multi-scale convolutional networks. International Joint Conference, 2809–2813

  27. Stivaktakis R, Tsagkatakis G, Tsakalides P (2019) Deep learning for multilabel land cover scene categorization using data augmentation. IEEE Geosci Remote Sens Lett 16(7):1031–1035

    Article  Google Scholar 

  28. Tampubolon H, Yang C, Chan S, Sutrisno H, Hua K-L (2019) Optimized CapsNet for traffic jam speed prediction using mobile sensor data under urban swarming transportation. Sensors, 5277

  29. Tran T, Pham C, Phuoc N, Duong T, Jeon J (2016) Real-time traffic light detection using color density. IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia) 1–4

  30. Tsoi T, Wheelus C (2020) Traffic signal classification with cost-sensitive deep learning models. IEEE International Conference on Knowledge Graph (ICKG), 586–592

  31. Wali SB, Abdullah MA, Hannan MA, Hussain A, Samad SA, Ker PJ, Mansor MB (2019) Vision-based traffic sign detection and recognition systems: Current trends and challenges. Sensors (Basel), 2093

  32. Wu N, Fang H (2017) A novel traffic light recognition method for traffic monitoring systems. Asia-Pacific Conference on Intelligent Robot Systems, 141–145

  33. Zhang F, Wang Y, Ye M (2018) Network traffic classification method based on improved capsule neural network. International Conference on Computational Intelligence and Security, 174–178

  34. Zhang Z, Zhang D, Wei H (2019) Vehicle type recognition using capsule network. Chinese Control and Decision Conference, 2944–2948

  35. Zhang Y, Li J, Guo Y, Xu C, Bao C, Song Y (2019) Vehicle driving behavior recognition based on multi-view convolutional neural network with joint data augmentation. IEEE Trans Veh Technol 68(5):4223–4234

    Article  Google Scholar 

  36. Zhao Z, Zheng P, Xu S, Wu X (2019) Object detection with deep learning: A review. Neural Networks and Learning Systems, 3212–3232

Download references

Author information

Authors and Affiliations

  1. Auckland University of Technology, Auckland, 1010, New Zealand

    Xiaoxu Liu & Wei Qi Yan

Authors
  1. Xiaoxu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Qi Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Qi Yan.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, X., Yan, W.Q. Traffic-light sign recognition using capsule network. Multimed Tools Appl 80, 15161–15171 (2021). https://doi.org/10.1007/s11042-020-10455-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10455-x

Keywords

Search

Navigation