An embedded automatic license plate recognition system using deep learning

Abstract

A system to automatically recognize vehicle license plates is a growing need to improve safety and traffic control, specifically in major urban centers. However, the license plate recognition task is generally computationally intensive, where the entire input image frame is scanned, the found plates are segmented, and character recognition is then performed for each segmented character. This paper presents a methodology for engineering a system to detect and recognize Brazilian license plates using convolutional neural networks (CNN) that is suitable for embedded systems. The resulting system detects license plates in the captured image using Tiny YOLOv3 architecture and identifies its characters using a second convolutional network trained on synthetic images and fine-tuned with real license plate images. The proposed architecture has demonstrated to be robust to angle, lightning, and noise variations while requiring a single forward pass for each network, therefore allowing faster processing compared to other deep learning approaches. Our methodology was validated using real license plate images under different environmental conditions reached a detection rate of 99.37% and an overall recognition rate of 98.43% while showing an average time of 2.70 s to process \(1024 \times 768\) images with a single license plate in a Raspberry Pi3 (ARM Cortex-A53 CPU). To improve the recognition accuracy, an ensemble of CNN models was tested instead of a single CNN model, which resulted in an increase in the average processing time to 4.88 s for each image while increasing the recognition rate to 99.53%. Finally, we discuss the impact of using an ensemble of CNNs considering the accuracy-performance trade-off when engineering embedded systems for license plate recognition.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

References

  1. 1.

    Yung N, Au K, Lai A (1999) Recognition of vehicle registration mark on moving vehicles in an outdoor environment. In: IEEE conference on intelligent transportation systems, proceedings, ITSC. IEEE

  2. 2.

    Kocer HE, Cevik KK (2011) Artificial neural networks based vehicle license plate recognition. Procedia Comput Sci 3:1033–1037

    Article  Google Scholar 

  3. 3.

    Chen Z-X, Liu C-Y, Chang F-L, Wang G-Y (2009) Automatic license-plate location and recognition based on feature salience. IEEE Trans Veh Technol 58(7):3781

    Article  Google Scholar 

  4. 4.

    Du S, Ibrahim M, Shehata M, Badawy W (2013) Automatic license plate recognition (ALPR): a state-of-the-art review. IEEE Trans Circuits Syst Video Technol 23(2):311–325

    Article  Google Scholar 

  5. 5.

    Patel C, Shah D, Patel A (2013) Automatic number plate recognition system (ANPR): a survey. Int J Comput Appl 69(9):21–33

    Google Scholar 

  6. 6.

    Li H, Shen C (2016) Reading car license plates using deep convolutional neural networks and lstms, arXiv preprint arXiv:1601.05610

  7. 7.

    LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436

    Article  Google Scholar 

  8. 8.

    Nejati M, Majidi A, Jalalat M (2015) License plate recognition based on edge histogram analysis and classifier ensemble. In: 2015 Signal processing and intelligent systems conference (SPIS) IEEE, 2015, pp 48–52

  9. 9.

    Redmon J, Farhadi A (2018) Yolov3: an incremental improvement. arXiv preprint arXiv:1804.02767

  10. 10.

    Sarfraz M, Ahmed MJ, Ghazi SA (2003) Saudi Arabian license plate recognition system. In: 2003 International conference on geometric modeling and graphics, 2003. Proceedings. IEEE, 2003, pp 36–41

  11. 11.

    Björklund T, Fiandrotti A, Annarumma M, Francini G, Magli E (2017) Automatic license plate recognition with convolutional neural networks trained on synthetic data. In: 2017 IEEE 19th international workshop on multimedia signal processing (MMSP). IEEE, 2017, pp 1–6

  12. 12.

    Rizvi S, Patti D, Björklund T, Cabodi G, Francini G (2017) Deep classifiers-based license plate detection, localization and recognition on gpu-powered mobile platform. Fut Internet 9(4):66

    Article  Google Scholar 

  13. 13.

    Hsu G-S, Chen J-C, Chung Y-Z (2012) Application-oriented license plate recognition. IEEE Trans Veh Technol 62(2):552–561

    Article  Google Scholar 

  14. 14.

    Rezk NM, Purnaprajna M, Nordström T, Ul-Abdin Z (2019) Recurrent neural networks: an embedded computing perspective, arXiv preprint arXiv:1908.07062

  15. 15.

    Redmon J, Farhadi A (2016) YOLO9000: better, faster, stronger, ArXiv e-prints

  16. 16.

    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  17. 17.

    Earl M (2016) Number plate recognition with tensorflow. https://github.com/matthewearl/deep-anpr

  18. 18.

    Xiao J, Hays J, Ehinger KA, Oliva A, Torralba A (2010) Sun database: large-scale scene recognition from abbey to zoo. In: 2010 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, 2010, pp 3485–3492

  19. 19.

    Stark F, Hazırbas C, Triebel R, Cremers D (2015) Captcha recognition with active deep learning. In: GCPR workshop on new challenges in neural computation

  20. 20.

    Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

  21. 21.

    Dietterich TG (2000) Ensemble methods in machine learning. In: International workshop on multiple classifier systems. Springer, pp 1–15

  22. 22.

    Brown G, Kuncheva LI (2010) “Good” and “bad” diversity in majority vote ensembles. In: International workshop on multiple classifier systems. Springer, Berlin, pp 124–133

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Diogo M. F. Izidio.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Izidio, D.M.F., Ferreira, A.P.A., Medeiros, H.R. et al. An embedded automatic license plate recognition system using deep learning. Des Autom Embed Syst 24, 23–43 (2020). https://doi.org/10.1007/s10617-019-09230-5

Download citation

Keywords

  • Embedded systems
  • Automatic license plate recognition (ALPR)
  • Image processing
  • Deep learning
  • Neural networks