Multi-task Learning for Low-Resolution License Plate Recognition

  • Gabriel Resende GonçalvesEmail author
  • Matheus Alves Diniz
  • Rayson Laroca
  • David Menotti
  • William Robson Schwartz
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11896)


License plate recognition is an important task applied to a myriad of important scenarios. Even though there are several methods for performing license plate recognition, our approach is designed to work not only on high resolution license plates but also when the license plate characters are not recognizable by humans. Early approaches divided the task into several subtasks that are executed in sequence. However, since each task has its own accuracy, the errors of each are propagated to the next step. This is critical in the last two steps of the pipeline known as segmentation and recognition of the characters. Thus, we employ a technique to perform these two steps at once. The approach is based on a multi-task network where each task represents the recognition of an entire license plate character. We do not address the license plate detection problem in this paper. We also propose the use of a so called generative model for data augmentation of low-resolution images simulating images as if they were acquired farther away from where they actually are. We are able to achieve very promising results with improvements of more than 30% points of accuracy on images with multiple resolutions and a character recognition accuracy on low-resolution images higher than 87%.


Multi-task learning Low-resolution Deep learning CNN 



The authors would like to thank the Brazilian National Research Council – CNPq (Grants #311053/2016-5, #428333/2016-8, #313423/2017-2 and #438629/2018-3), the Minas Gerais Research Foundation – FAPEMIG (Grants APQ-00567-14 and PPM-00540-17), the Coordination for the Improvement of Higher Education Personnel – CAPES (DeepEyes Project), Maxtrack Industrial LTDA and Empresa Brasileira de Pesquisa e Inovacao Industrial – EMBRAPII.


  1. 1.
    Dong, M., He, D., Luo, C., Liu, D., Zeng, W.: A CNN-based approach for automatic license plate recognition in the wild. In: BMVC (2017)Google Scholar
  2. 2.
    Du, S., Ibrahim, M., Shehata, M., Badawy, W.: Automatic license plate recognition (ALPR): A state-of-the-art review. TCSVT (2013)Google Scholar
  3. 3.
    Gonçalves, G., Diniz, M.A., Laroca, R., Menotti, D., Schwartz, W.R.: Real-time automatic license plate recognition through deep multi-task networks. In: SIBGRAPI. IEEE (2018)Google Scholar
  4. 4.
    Gonçalves, G.R., Menotti, D., Schwartz, W.R.: License plate recognition based on temporal redundancy. In: ITSC (2016)Google Scholar
  5. 5.
    Gonçalves, G.R., da Silva, S.P.G., Menotti, D., Schwartz, W.R.: Benchmark for license plate character segmentation. JEI 25, 053034 (2016)Google Scholar
  6. 6.
    Goodfellow, I., et al.: Generative adversarial nets. In: NIPS (2014)Google Scholar
  7. 7.
    Hu, C., Bai, X., Qi, L., Chen, P., Xue, G., Mei, L.: Vehicle color recognition with spatial pyramid deep learning. T-ITS 16, 2925–2934 (2015)Google Scholar
  8. 8.
    Krizhevsky, A., Sutskever, I., Hinton, G.: Imagenet classification with deep convolutional neural networks. In: NIPS (2012)Google Scholar
  9. 9.
    Laroca, R., Barroso, V., Diniz, M.A., Gonçalves, G.R., Schwartz, W.R., Menotti, D.: Convolutional neural networks for automatic meter reading. JEI 28, 013023 (2019)Google Scholar
  10. 10.
    Laroca, R., et al.: A robust real-time automatic license plate recognition based on the YOLO detector. In: IJCNN (2018)Google Scholar
  11. 11.
    Li, H., Wang, P., Shen, C.: Towards end-to-end car license plates detection and recognition with deep neural networks. arXiv preprint arXiv:1709.08828 (2017)
  12. 12.
    Moeskops, P., et al.: Deep learning for multi-task medical image segmentation in multiple modalities. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 478–486. Springer, Cham (2016). Scholar
  13. 13.
    Nomura, S., Yamanaka, K., Shiose, T., Kawakami, H., Katai, O.: Morphological preprocessing method to thresholding degraded word images. PRL 30, 729–744 (2009)CrossRefGoogle Scholar
  14. 14.
    Ranjan, R., Patel, V.M., Chellappa, R.: Hyperface: a deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. PAMI 41, 121–135 (2017)CrossRefGoogle Scholar
  15. 15.
    Rao, Y.: Automatic vehicle recognition in multiple cameras for videosurveillance. Vis. Comput. 31, 271–280 (2015)CrossRefGoogle Scholar
  16. 16.
    Rizvi, S.T.H., Patti, D., Björklund, T., Cabodi, G., Francini, G.: Deep classifiers-based license plate detection, localization and recognition on gpu-powered mobile platform. Fut. Internet 9, 66 (2017)CrossRefGoogle Scholar
  17. 17.
    Shuang-Tong, T., Wen-Ju, L.: Number and letter character recognition of vehicle license plate based on edge Hausdorff distance. In: PDCAT (2005)Google Scholar
  18. 18.
    Silva, S.M., Jung, C.R.: Real-time Brazilian license plate detection and recognition using deep convolutional neural networks. In: SIBGRAPI. IEEE (2017)Google Scholar
  19. 19.
    Soumya, K.R., Babu, A., Therattil, L.: License plate detection and character recognition using contour analysis. IJATCA (2014)Google Scholar
  20. 20.
    Špaňhel, J., Sochor, J., Juránek, R., Herout, A., Maršík, L., Zemčík, P.: Holistic recognition of low quality license plates by CNN using track annotated data. In: AVSS (2017)Google Scholar
  21. 21.
    Walker, J., Doersch, C., Gupta, A., Hebert, M.: An uncertain future: forecasting from static images using variational autoencoders. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 835–851. Springer, Cham (2016). Scholar
  22. 22.
    Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for text classification. In: NIPS (2015)Google Scholar
  23. 23.
    Zhang, Y., Yang, Q.: A survey on multi-task learning. arXiv preprint arXiv:1707.08114 (2017)
  24. 24.
    Zhang, Z., Luo, P., Loy, C.C., Tang, X.: Facial landmark detection by deep multi-task learning. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 94–108. Springer, Cham (2014). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Gabriel Resende Gonçalves
    • 1
    Email author
  • Matheus Alves Diniz
    • 1
  • Rayson Laroca
    • 2
  • David Menotti
    • 2
  • William Robson Schwartz
    • 1
  1. 1.Smart Sense LaboratoryUniversidade Federal de Minas GeraisBelo HorizonteBrazil
  2. 2.Department of InformaticsUniversidade Federal do ParanáCuritibaBrazil

Personalised recommendations