Skip to main content
SpringerLink
Log in

Effective deep neural networks for license plate detection and recognition

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Recently, automatic license plate recognition (ALPR) has drawn much of attention from researchers due to the impressive performance of deep learning (DL) techniques. While a large number of methods for ALPR have been investigated, there are a few attempts emphasizing efficient yet accurate models for facilitating the deployment on traditional CPU boxes or low resource devices. In the present work, we propose lightweight and effective deep convolutional neuron networks to address the problems of license plate detection and recognition. Differing from traditional DL methods, the proposed models discard the use of max-pooling modules, belong to a single-phase object detector, and consist of alternating convolutional layers and Inception residual networks. Different strategies for character prediction are also studied and deeply discussed. To have more insight of convolutional features, we provide a sequence of visual snapshots or slices of feature maps learned by the network at different layers for a given input image. These data are useful to explain the model’s behavior for the task of character recognition. Extensive experiments conducted on two public datasets, CCPD and AOLP, showed a promising improvement of LP detection and recognition accuracy. Interestingly, the full system can work on low-resource CPU machines with a real-time speed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. https://github.com/TropComplique/FaceBoxes-tensorflow.

  2. https://github.com/AlexeyAB/darknet.

References

  1. Bochkovskiy, A.: Darknet: open source neural networks in python (2020). https://github.com/AlexeyAB/darknet

  2. Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: Yolov4: Optimal speed and accuracy of object detection (2020). arXiv:2004.10934v1 [cs.CV]

  3. Dong, M., He, D., Luo, C., Liu, D., Zeng, W.: A CNN-based approach for automatic license plate recognition in the wild. In: British Machine Vision Conference (BMVC), pp. 1–12 (2017)

  4. Gonçalves, G.R., da Silva, S.P.G., Menotti, D., Schwartz, W.R.: Benchmark for license plate character segmentation. J. Electron. Imaging 25(5), 1–5 (2016)

    Article  Google Scholar 

  5. Graves, A., Liwicki, M., Fernández, S., Bertolami, R., Bunke, H., Schmidhuber, J.: A novel connectionist system for unconstrained handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31(5), 855–868 (2009). https://doi.org/10.1109/TPAMI.2008.137

    Article  Google Scholar 

  6. Hsu, G.S., Chen, J.C., Chung, Y.Z.: Application-oriented license plate recognition. IEEE Trans. Veh. Technol. 62(2), 552–561 (2013). https://doi.org/10.1109/TVT.2012.2226218

    Article  Google Scholar 

  7. Izidio, D.M.F., Ferreira, A.P.A., Barros, E.N.S.: An embedded automatic license plate recognition system using deep learning. Des. Autom. Embed. Syst. 24, 23–43 (2020). https://doi.org/10.1007/s10617-019-09230-5

    Article  Google Scholar 

  8. Laroca, R., Severo, E., Zanlorensi, L.A., Oliveira, L.S., Gonçalves, G.R., Schwartz, W.R., Menotti, D.: A robust real-time automatic license plate recognition based on the yolo detector. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1–10 (2018). https://doi.org/10.1109/IJCNN.2018.8489629

  9. Laroca, R., Zanlorensi, L.A., Gonçalves, G.R., Todt, E., Schwartz, W.R., Menotti, D.: An efficient and layout-independent automatic license plate recognition system based on the YOLO detector. IET Intel. Transp. Syst. 15(4), 483–503 (2021). https://doi.org/10.1049/itr2.12030

    Article  Google Scholar 

  10. Li, H., Wang, P., Shen, C.: Toward end-to-end car license plate detection and recognition with deep neural networks. IEEE Trans. Intell. Transp. Syst. 20(3), 1126–1136 (2019). https://doi.org/10.1109/TITS.2018.2847291

    Article  Google Scholar 

  11. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., Berg, A.C.: SSD: single shot multibox detector. In: European Conference on Computer Vision (ECCV), pp. 21–37 (2016)

  12. Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th International Conference on International Conference on Machine Learning, pp. 807–814. Omni Press, Madison (2010)

  13. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779–788 (2016). https://doi.org/10.1109/CVPR.2016.91

  14. Redmon, J., Farhadi, A.: Yolo9000: better, faster, stronger. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6517–6525 (2017). https://doi.org/10.1109/CVPR.2017.690

  15. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(06), 1137–1149 (2017)

    Article  Google Scholar 

  16. Selmi, Z., Ben Halima, M., Alimi, A.M.: Deep learning system for automatic license plate detection and recognition. In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), vol. 01, pp. 1132–1138 (2017). https://doi.org/10.1109/ICDAR.2017.187

  17. Selmi, Z., Halima, M.B., Pal, U., Alimi, M.A.: Delp-dar system for license plate detection and recognition. Pattern Recogn. Lett. 129, 213–223 (2020)

    Article  Google Scholar 

  18. Silva, S.M., Jung, C.R.: Real-time Brazilian license plate detection and recognition using deep convolutional neural networks. In: 2017 30th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 55–62 (2017). https://doi.org/10.1109/SIBGRAPI.2017.14

  19. Silva, S.M., Jung, C.R.: License plate detection and recognition in unconstrained scenarios. In: 2018 European Conference on Computer Vision (ECCV), Lecture Notes in Computer Science, vol. 11216, pp. 580–596 (2018)

  20. Silva, S.M., Jung, C.R.: Real-time license plate detection and recognition using deep convolutional neural networks. J. Vis. Commun. Image Represent. 71, 102773–102781 (2020). https://doi.org/10.1016/j.jvcir.2020.102773

    Article  Google Scholar 

  21. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7–9, 2015, Conference Track Proceedings (2015). arXiv:org/abs/1409.1556

  22. Spanhel, J., Sochor, J., Juranek, R., Herout, A., Marsik, L., Zemcik, P.: Holistic recognition of low quality license plates by CNN using track annotated data. In: 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1–6 (2017)

  23. Su, H., Liu, F., Xie, Y., Xing, F., Meyyappan, S., Yang, L.: Region segmentation in histopathological breast cancer images using deep convolutional neural network. In: 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI), pp. 55–58 (2015). https://doi.org/10.1109/ISBI.2015.7163815

  24. Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence, AAAI’17, pp. 4278–4284. AAAI Press (2017)

  25. Tan, M., Pang, R., Le, Q.V.: Efficientdet: scalable and efficient object detection (2020). arXiv:1911.09070 [cs.CV]

  26. Xu, Z., Yang, W., Meng, A., Lu, N., Huang, H., Ying, C., Huang, L.: Towards end-to-end license plate detection and recognition: a large dataset and baseline. In: European Conference on Computer Vision (ECCV), Lecture Notes in Computer Science, vol. 11217, pp. 261–277 (2018)

  27. Yang, Y., Li, D., Duan, Z.: Chinese vehicle license plate recognition using kernel-based extreme learning machine with deep convolutional features. IET Intell. Transp. Syst. (2017). https://doi.org/10.1049/iet-its.2017.0136

  28. Zhang, S., Wang, X., Lei, Z., Li, S.Z.: Faceboxes: A CPU real-time and accurate unconstrained face detector. Neurocomputing 364, 297–309 (2019)

    Article  Google Scholar 

  29. Zhang, X., Chen, F., Yu, T., An, J., Huang, Z., Liu, J., Hu, W., Wang, L., Duan, H., Si, J.: Real-time gastric polyp detection using convolutional neural networks. PLoS One 14(3), 1–16 (2019). https://doi.org/10.1371/journal.pone.0214133

    Article  Google Scholar 

  30. Zhuang, J., Hou, S., Wang, Z., Zha, Z.J.: Towards human-level license plate recognition. In: European Conference on Computer Vision (ECCV), Lecture Notes in Computer Science, vol. 11207, pp. 314–329 (2018)

Download references

Acknowledgements

This research is funded by Vietnam National Foundation for Science and Technology Development (NAFOSTED) Under Grant Number 102.05-2020.02.

Author information

Authors and Affiliations

  1. Hong Duc University (HDU), Thanh Hoa City, Vietnam

    The-Anh Pham

Authors
  1. The-Anh Pham
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to The-Anh Pham.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pham, TA. Effective deep neural networks for license plate detection and recognition. Vis Comput 39, 927–941 (2023). https://doi.org/10.1007/s00371-021-02375-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02375-0

Keywords

Search

Navigation