Skip to main content
SpringerLink
Log in

Detection and classification of road signs in raining condition with limited dataset

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

The road sign recognition (RSR) system is used to complete two tasks: localizing the traffic sign in an image and then classifying it according to the image features. Some of the applications of such system are incorporated into the Advanced Driver Assistance System (ADAS) and autonomous vehicles. However, the accuracy of the model decreases when changes in lighting or weather occurs, and the lack of training samples taken in rainy condition causes the model to be sub-optimal. The research was conducted with the focus on solving two problems, the accuracy of an object detection model decreases when changes in lighting or weather occurs, and the lack of training data, especially images taken under adverse condition. In this work, we compare and analyze three methods; automatic white balance (AWB), policy augmentations and Image-to-Image-translation (I2IT) technique on their performance to detect traffic signs in raining conditions. All methods were built upon the pre-trained SSD-MobileNetV2 model and using the TensorFlow2 framework. The images from the Malaysia Traffic Sign Dataset (MTSD) are used to train all models. Finally based on the result of the three model a final combination model is proposed that achieved the best performance in rainy condition. Experimental results showed that AWB was not that effective in detecting road sign in raining condition, while the other two techniques were highly effective. The final proposed model was implemented by combining policy augmentation and I2IT, obtained an mAP of 0.7967 in clear images and mAP of 0.7160 when rainy images were added to the testing dataset. These corresponds to mAP at 50% IoU of 0.8921mAP@0.5 during clear weather and 0.8340mAP@0.5 in raining images, which outperformed other models. Thus, a road sign detection and classification system that can perform well in rainy condition with limited training dataset has been successfully developed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Availability of data and materials

The image database and algorithms used in this research are listed in the reference list.

References

  1. Guofeng, T., Huairong, C., Yong, L., Kai, Z.: Traffic sign recognition based on SVM and convolutional neural network. In: 2017 12th IEEE Conference on Industrial Electronics and Applications (ICIEA), pp. 2066–2071 (2017). https://doi.org/10.1109/ICIEA.2017.8283178

  2. Palak, Sangal AL: Traffic signs classification using convolutional neural networks: a review. In: 2021 2nd International Conference on Secure Cyber Computing and Communications (ICSCCC), pp. 450–455 (2021). https://doi.org/10.1109/ICSCCC51823.2021.9478172

  3. Yang, W., Zhang, W.: Real-time traffic signs detection based on YOLO network model. In: 2020 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC), pp. 354–357 (2020). https://doi.org/10.1109/CyberC49757.2020.00066

  4. Islam, Kh.T., Raj, R.G.: Real-time (vision-based) road sign recognition using an artificial neural network. Sensors (Basel, Switzerland) 17(4), 853 (2017). https://doi.org/10.3390/s17040853

    Article  Google Scholar 

  5. Zheng, Z., Zhang, H., Wang, B., Gao, Z.: Robust traffic sign recognition and tracking for Advanced Driver Assistance Systems. In: 2012 15th International IEEE Conference on Intelligent Transportation Systems, pp. 704–709 (2012). https://doi.org/10.1109/ITSC.2012.6338799.

  6. Hnewa, M., Radha, H.: Object detection under rainy conditions for autonomous vehicles: a review of state-of-the-art and emerging techniques. IEEE Signal Process. Mag. 38(1), 53–67 (2021). https://doi.org/10.1109/MSP.2020.2984801

    Article  Google Scholar 

  7. Madam, A., Yusof, R.: Malaysian traffic sign dataset for traffic sign detection and recognition systems. J. Telecommun. Electron. Comput. Eng. 8, 137–143 (2016)

    Google Scholar 

  8. Zoph, B., Cubuk, E.D., Ghiasi, G., Lin, T.-Y., Shlens, J., Le, Q.V.: Learning data augmentation strategies for object detection. In: ECCV (2020). https://doi.org/10.48550/arXiv.1906.11172

  9. Zhu, Z., Liang, D., Zhang, S., Huang, X., Li, B., Hu, S.: Traffic-sign detection and classification in the wild. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2110–2118 (2016). https://doi.org/10.1109/CVPR.2016.232

  10. Cubuk, E.D., Zoph, B., Mané, D., Vasudevan, V., Le, Q.V.: AutoAugment: learning augmentation strategies from data. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 113–123 (2019). https://doi.org/10.1109/CVPR.2019.00020

  11. DeepVoltaire. AutoAugment. Retrieved from GitHub (2021). https://github.com/DeepVoltaire/AutoAugment/blob/master/AutoAugment_Exploration.ipynb

  12. Sahota, H. BBAug. Retrieved from GitHub (2020). https://github.com/harpalsahota/bbaug

  13. Soviany, P., Ionescu, R.T.: Optimizing the trade-off between single-stage and two-stage deep object detectors using image difficulty prediction. In: 2018 20th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), pp. 209–214 (2018). https://doi.org/10.1109/SYNASC.2018.00041

  14. Mohd-Isa, W.-N., Abdullah, M.-S., Sarzil, M., Abdullah, J., Ali, A., Hashim, N.: Detection of Malaysian traffic signs via modified YOLOv3 algorithm. In: 2020 International Conference on Data Analytics for Business and Industry: Way Towards a Sustainable Economy (ICDABI), pp. 1–5 (2020). https://doi.org/10.1109/ICDABI51230.2020.9325690

Download references

Funding

This research was funded by UM International Collaboration Grant ST085-2022.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Faculty of Engineering, Universiti Malaya, Kuala Lumpur, Malaysia

    Mohamed Jamil Khan Bin Mohamed Bysul Khan, Noraisyah Mohamed Shah & Norrima Mokhtar

Authors
  1. Mohamed Jamil Khan Bin Mohamed Bysul Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Noraisyah Mohamed Shah
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Norrima Mokhtar
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

NMS and MJKBMBK wrote the manuscript and NM prepared the figures.

Corresponding author

Correspondence to Noraisyah Mohamed Shah.

Ethics declarations

Competing interests

The authors declare no competing interests.

Conflict of interest

The authors declare that they have no competing interests.

Ethical approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mohamed Bysul Khan, M.J.K.B., Shah, N.M. & Mokhtar, N. Detection and classification of road signs in raining condition with limited dataset. SIViP 17, 2015–2023 (2023). https://doi.org/10.1007/s11760-022-02414-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-022-02414-w

Keywords

Search

Navigation