Skip to main content
SpringerLink
Log in

Traffic Sign Recognition Robustness in Autonomous Vehicles Under Physical Adversarial Attacks

  • Chapter
  • First Online:
Cutting Edge Applications of Computational Intelligence Tools and Techniques

Part of the book series: Studies in Computational Intelligence ((SCI,volume 1118))

  • 238 Accesses

Abstract

Nowadays we are all witnesses of the technological development of the so-called 4th industrial revolution (Industry 4.0). In this context, a daily living environment of smart cities is formed in which artificial intelligence applications play a dominant role. Autonomous (pilotless) vehicles are a shining example of the application of artificial intelligence, based on which vehicles are allowed to move autonomously in both residential and rural areas. The proposed article examines the robustness, in adversarial attacks in the physical layer, of the deep learning models used in autonomous vehicles for the recognition of road traffic signals. As a case study the roads of Greece, having traffic signs highly contaminated not on purpose, is considered. Towards investigating this direction, a novel dataset with clear and attacked images of traffic signs is proposed and used in the evaluation of popular deep learning models. This study investigates the level of readiness of autonomous vehicles to perform in noisy environments that affect their ability to recognize road signs. This work highlights the need for more robust deep learning models in order to make the use of autonomous vehicles a reality with maximum safety for citizens.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    The dataset is provided via the GitHub account of the MLV Research Group. (https://github.com/MachineLearningVisionRG/GRATS).

References

  1. A. Geiger, P. Lenz, C. Stiller, and R. Urtasun, “Vision meets robotics: The KITTI dataset,” The International Journal of Robotics Research, vol. 32, no. 11, pp. 1231–1237, Sep. 2013, doi: https://doi.org/10.1177/0278364913491297.

    Article  Google Scholar 

  2. K. Apostolidis, P. Amanatidis, and G. Papakostas, “Performance Evaluation of Convolutional Neural Networks for Gait Recognition,” in 24th Pan-Hellenic Conference on Informatics, Athens Greece, Nov. 2020, pp. 61–63. doi: https://doi.org/10.1145/3437120.3437276.

  3. Z. Zou, Z. Shi, Y. Guo, and J. Ye, “Object detection in 20 years: A survey,” arXiv preprint arXiv:1905.05055, 2019.

  4. J. Janai, F. Güney, A. Behl, and A. Geiger, “Computer vision for autonomous vehicles: Problems, datasets and state of the art,” Foundations and Trends® in Computer Graphics and Vision, vol. 12, no. 1–3, pp. 1–308, 2020.

    Google Scholar 

  5. C. Liu, S. Li, F. Chang, and Y. Wang, “Machine Vision Based Traffic Sign Detection Methods: Review, Analyses and Perspectives,” Machine Vision, vol. 7, p. 19, 2019.

    Google Scholar 

  6. D. Tabernik and D. Skočaj, “Deep Learning for Large-Scale Traffic-Sign Detection and Recognition,” arXiv:1904.00649 [cs], Apr. 2019, Accessed: Apr. 28, 2022. [Online]. Available: http://arxiv.org/abs/1904.00649.

  7. K. Bayoudh, F. Hamdaoui, and A. Mtibaa, “Transfer learning based hybrid 2D-3D CNN for traffic sign recognition and semantic road detection applied in advanced driver assistance systems,” Appl Intell, vol. 51, no. 1, pp. 124–142, Jan. 2021, doi: https://doi.org/10.1007/s10489-020-01801-5.

    Article  Google Scholar 

  8. Z. Liu, J. Du, F. Tian, and J. Wen, “MR-CNN: A Multi-Scale Region-Based Convolutional Neural Network for Small Traffic Sign Recognition,” IEEE Access, vol. 7, pp. 57120–57128, 2019, doi: https://doi.org/10.1109/ACCESS.2019.2913882.

    Article  Google Scholar 

  9. Y. Yuan et al., “VSSA-NET: Vertical Spatial Sequence Attention Network for Traffic Sign Detection,” IEEE Trans. on Image Process., vol. 28, no. 7, pp. 3423–3434, Jul. 2019, doi: https://doi.org/10.1109/TIP.2019.2896952.

    Article  MathSciNet  MATH  Google Scholar 

  10. A. Vennelakanti, S. Shreya, R. Rajendran, D. Sarkar, D. Muddegowda, and P. Hanagal, “Traffic Sign Detection and Recognition using a CNN Ensemble,” in 2019 IEEE International Conference on Consumer Electronics (ICCE), Jan. 2019, pp. 1–4. doi: https://doi.org/10.1109/ICCE.2019.8662019.

  11. C. Szegedy et al., “Intriguing properties of neural networks,” arXiv:1312.6199 [cs], Feb. 2014, Accessed: Jun. 04, 2021. [Online]. Available: http://arxiv.org/abs/1312.6199.

  12. I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and Harnessing Adversarial Examples,” arXiv:1412.6572 [cs, stat], Mar. 2015, Accessed: Jun. 04, 2021. [Online]. Available: http://arxiv.org/abs/1412.6572.

  13. K. D. Apostolidis and G. A. Papakostas, “A Survey on Adversarial Deep Learning Robustness in Medical Image Analysis,” Electronics, vol. 10, no. 17, p. 2132, Sep. 2021, doi: https://doi.org/10.3390/electronics10172132.

    Article  Google Scholar 

  14. M. Costa, A. Simone, V. Vignali, C. Lantieri, and N. Palena, “Fixation distance and fixation duration to vertical road signs,” Applied Ergonomics, vol. 69, pp. 48–57, May 2018, doi: https://doi.org/10.1016/j.apergo.2017.12.017.

    Article  Google Scholar 

  15. S. B. Wali et al., “Vision-Based Traffic Sign Detection and Recognition Systems: Current Trends and Challenges,” Sensors, vol. 19, no. 9, 2019, doi: https://doi.org/10.3390/s19092093.

  16. Y. Zhu, “Traffic sign recognition based on deep learning,” Multimedia Tools and Applications, p. 13.

    Google Scholar 

  17. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection,” arXiv:1506.02640 [cs], May 2016, Accessed: Apr. 28, 2022. [Online]. Available: http://arxiv.org/abs/1506.02640.

  18. C. Dewi, R.-C. Chen, and H. Yu, “Weight analysis for various prohibitory sign detection and recognition using deep learning,” Multimedia Tools and Applications, vol. 79, no. 43–44, pp. 32897–32915, 2020.

    Article  Google Scholar 

  19. W. Liu et al., “SSD: Single Shot MultiBox Detector,” arXiv:1512.02325 [cs], vol. 9905, pp. 21–37, 2016, doi: https://doi.org/10.1007/978-3-319-46448-0_2.

  20. S. You, Q. Bi, Y. Ji, S. Liu, Y. Feng, and F. Wu, “Traffic sign detection method based on improved SSD,” Information, vol. 11, no. 10, p. 475, 2020.

    Article  Google Scholar 

  21. S. Houben, J. Stallkamp, J. Salmen, M. Schlipsing, and C. Igel, “Detection of traffic signs in real-world images: The German traffic sign detection benchmark,” in The 2013 International Joint Conference on Neural Networks (IJCNN), Dallas, TX, USA, Aug. 2013, pp. 1–8. doi: https://doi.org/10.1109/IJCNN.2013.6706807.

  22. J. Stallkamp, M. Schlipsing, J. Salmen, and C. Igel, “Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition,” Neural Networks, vol. 32, pp. 323–332, Aug. 2012, doi: https://doi.org/10.1016/j.neunet.2012.02.016.

    Article  Google Scholar 

  23. M. Mathias, R. Timofte, R. Benenson, and L. Van Gool, “Traffic sign recognition — How far are we from the solution?,” in The 2013 International Joint Conference on Neural Networks (IJCNN), Dallas, TX, USA, Aug. 2013, pp. 1–8. doi: https://doi.org/10.1109/IJCNN.2013.6707049.

  24. Z. Zhu, D. Liang, S. Zhang, X. Huang, B. Li, and S. Hu, “Traffic-Sign Detection and Classification in the Wild,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, Jun. 2016, pp. 2110–2118. doi: https://doi.org/10.1109/CVPR.2016.232.

  25. A. Mogelmose, D. Liu, and M. M. Trivedi, “Detection of U.S. Traffic Signs,” IEEE Trans. Intell. Transport. Syst., vol. 16, no. 6, pp. 3116–3125, Dec. 2015, doi: https://doi.org/10.1109/TITS.2015.2433019.

    Article  Google Scholar 

  26. F. Larsson and M. Felsberg, “Using Fourier Descriptors and Spatial Models for Traffic Sign Recognition,” in Image Analysis, vol. 6688, A. Heyden and F. Kahl, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011, pp. 238–249. doi: https://doi.org/10.1007/978-3-642-21227-7_23.

  27. C. Grigorescu and N. Petkov, “Distance sets for shape filters and shape recognition,” IEEE Transactions on Image Processing, vol. 12, no. 10, pp. 1274–1286, Oct. 2003, doi: https://doi.org/10.1109/TIP.2003.816010.

    Article  MathSciNet  MATH  Google Scholar 

  28. R. Belaroussi, P. Foucher, J.-P. Tarel, B. Soheilian, P. Charbonnier, and N. Paparoditis, “Road Sign Detection in Images: A Case Study,” in 2010 20th International Conference on Pattern Recognition, Istanbul, Turkey, Aug. 2010, pp. 484–488. doi: https://doi.org/10.1109/ICPR.2010.1125.

  29. H. Fleyeh, “Traffic and Road Sign Recognition,” p. 255.

    Google Scholar 

  30. S. Segvic et al., “A computer vision assisted geoinformation inventory for traffic infrastructure,” in 13th International IEEE Conference on Intelligent Transportation Systems, Funchal, Madeira Island, Portugal, Sep. 2010, pp. 66–73. doi: https://doi.org/10.1109/ITSC.2010.5624979.

  31. C. Gamez Serna and Y. Ruichek, “Classification of Traffic Signs: The European Dataset,” IEEE Access, vol. 6, pp. 78136–78148, 2018, doi: https://doi.org/10.1109/ACCESS.2018.2884826.

  32. N. Akhtar, A. Mian, N. Kardan, and M. Shah, “Advances in Adversarial Attacks and Defenses in Computer Vision: A Survey,” vol. 9, p. 36, 2021.

    Google Scholar 

  33. T. Maliamanis and G. Papakostas, “Adversarial computer vision: a current snapshot,” in Twelfth International Conference on Machine Vision (ICMV 2019), Amsterdam, Netherlands, Jan. 2020, p. 121. doi: https://doi.org/10.1117/12.2559582.

  34. A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards Deep Learning Models Resistant to Adversarial Attacks,” arXiv:1706.06083 [cs, stat], Sep. 2019, Accessed: Jun. 04, 2021. [Online]. Available: http://arxiv.org/abs/1706.06083.

  35. N. Papernot, P. McDaniel, S. Jha, M. Fredrikson, Z. B. Celik, and A. Swami, “The Limitations of Deep Learning in Adversarial Settings,” arXiv:1511.07528 [cs, stat], Nov. 2015, Accessed: Jun. 04, 2021. [Online]. Available: http://arxiv.org/abs/1511.07528.

  36. N. Carlini and D. Wagner, “Towards Evaluating the Robustness of Neural Networks,” arXiv:1608.04644 [cs], Mar. 2017, Accessed: Jun. 04, 2021. [Online]. Available: http://arxiv.org/abs/1608.04644.

  37. N. Papernot, P. McDaniel, X. Wu, S. Jha, and A. Swami, “Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks,” arXiv:1511.04508 [cs, stat], Mar. 2016, Accessed: Feb. 27, 2022. [Online]. Available: http://arxiv.org/abs/1511.04508.

  38. S.-M. Moosavi-Dezfooli, A. Fawzi, and P. Frossard, “DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, Jun. 2016, pp. 2574–2582. doi: https://doi.org/10.1109/CVPR.2016.282.

  39. T. B. Brown, D. Mané, A. Roy, M. Abadi, and J. Gilmer, “Adversarial Patch.” arXiv, May 16, 2018. Accessed: Mar. 19, 2023. [Online]. Available: http://arxiv.org/abs/1712.09665.

  40. S.-M. Moosavi-Dezfooli, A. Fawzi, O. Fawzi, and P. Frossard, “Universal Adversarial Perturbations,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, Jul. 2017, pp. 86–94. doi: https://doi.org/10.1109/CVPR.2017.17.

  41. H. Yakura, Y. Akimoto, and J. Sakuma, “Generate (non-software) Bugs to Fool Classifiers.” arXiv, Nov. 19, 2019. Accessed: Mar. 19, 2023. [Online]. Available: http://arxiv.org/abs/1911.08644.

  42. N. Akhtar and A. Mian, “Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey,” IEEE Access, vol. 6, pp. 14410–14430, 2018, doi: https://doi.org/10.1109/ACCESS.2018.2807385.

    Article  Google Scholar 

  43. A. Kurakin, I. Goodfellow, and S. Bengio, “Adversarial examples in the physical world,” arXiv:1607.02533 [cs, stat], Feb. 2017, Accessed: Jun. 04, 2021. [Online]. Available: http://arxiv.org/abs/1607.02533.

  44. H. Ren, T. Huang, and H. Yan, “Adversarial examples: attacks and defenses in the physical world,” Int. J. Mach. Learn. & Cyber., vol. 12, no. 11, pp. 3325–3336, Nov. 2021, doi: https://doi.org/10.1007/s13042-020-01242-z.

    Article  Google Scholar 

  45. M. Sharif, S. Bhagavatula, and L. Bauer, “Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition,” p. 13.

    Google Scholar 

  46. K. N. Kumar, C. Vishnu, R. Mitra, and C. K. Mohan, “Black-box Adversarial Attacks in Autonomous Vehicle Technology,” arXiv:2101.06092 [cs], Jan. 2021, Accessed: Apr. 28, 2022. [Online]. Available: http://arxiv.org/abs/2101.06092.

  47. H. Lengyel, V. Remeli, and Z. Szalay, “EASILY DEPLOYED STICKERS COULD DISRUPT TRAFFIC SIGN RECOGNITION,” p. 9.

    Google Scholar 

  48. Y. Zhong, X. Liu, D. Zhai, J. Jiang, and X. Ji, “Shadows can be Dangerous: Stealthy and Effective Physical-world Adversarial Attack by Natural Phenomenon,” arXiv:2203.03818 [cs], Mar. 2022, Accessed: Apr. 28, 2022. [Online]. Available: http://arxiv.org/abs/2203.03818.

  49. K. Eykholt et al., “Robust Physical-World Attacks on Deep Learning Visual Classification,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, Jun. 2018, pp. 1625–1634. doi: https://doi.org/10.1109/CVPR.2018.00175.

  50. K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” arXiv:1409.1556 [cs], Apr. 2015, Accessed: Jun. 04, 2021. [Online]. Available: http://arxiv.org/abs/1409.1556.

  51. K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, Jun. 2016, pp. 770–778. doi: https://doi.org/10.1109/CVPR.2016.90.

  52. C. Szegedy et al., “Going Deeper with Convolutions,” arXiv:1409.4842 [cs], Sep. 2014, Accessed: Apr. 28, 2022. [Online]. Available: http://arxiv.org/abs/1409.4842.

  53. G. Huang, Z. Liu, L. van der Maaten, and K. Q. Weinberger, “Densely Connected Convolutional Networks,” arXiv:1608.06993 [cs], Jan. 2018, Accessed: Sep. 14, 2021. [Online]. Available: http://arxiv.org/abs/1608.06993.

Download references

Author information

Authors and Affiliations

  1. MLV Research Group, Department of Computer Science, International Hellenic University, Kavala, Greece

    Kyriakos D. Apostolidis, Emmanouil V. Gkouvrikos, Eleni Vrochidou & George A. Papakostas

Authors
  1. Kyriakos D. Apostolidis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Emmanouil V. Gkouvrikos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eleni Vrochidou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. George A. Papakostas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to George A. Papakostas .

Editor information

Editors and Affiliations

  1. University of Detroit Mercy, Detroit, MI, USA

    Kevin Daimi

  2. Asia Pacific International College (APIC), Sydney, NSW, Australia

    Abeer Alsadoon

  3. Polytechnic Institute of Porto, Porto, Portugal

    Luis Coelho

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Apostolidis, K.D., Gkouvrikos, E.V., Vrochidou, E., Papakostas, G.A. (2023). Traffic Sign Recognition Robustness in Autonomous Vehicles Under Physical Adversarial Attacks. In: Daimi, K., Alsadoon, A., Coelho, L. (eds) Cutting Edge Applications of Computational Intelligence Tools and Techniques. Studies in Computational Intelligence, vol 1118. Springer, Cham. https://doi.org/10.1007/978-3-031-44127-1_13

Download citation

Publish with us

Policies and ethics

Search

Navigation