Towards Human-Level License Plate Recognition

  • Jiafan Zhuang
  • Saihui Hou
  • Zilei WangEmail author
  • Zheng-Jun Zha
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11207)


License plate recognition (LPR) is a fundamental component of various intelligent transport systems, which is always expected to be accurate and efficient enough. In this paper, we propose a novel LPR framework consisting of semantic segmentation and character counting, towards achieving human-level performance. Benefiting from innovative structure, our method can recognize a whole license plate once rather than conducting character detection or sliding window followed by per-character recognition. Moreover, our method can achieve higher recognition accuracy due to more effectively exploiting global information and avoiding sensitive character detection, and is time-saving due to eliminating one-by-one character recognition. Finally, we experimentally verify the effectiveness of the proposed method on two public datasets (AOLP and Media Lab) and our License Plate Dataset. The results demonstrate our method significantly outperforms the previous state-of-the-art methods, and achieves the accuracies of more than 99% for almost all settings.


License Plate Recognition (LPR) Semantic segmentation Convolutional Neural Networks (CNN) Character counting 



This work is supported partially by the NSFC under Grant 61673362, Youth Innovation Promotion Association CAS, and the Fundamental Research Funds for the Central Universities.


  1. 1.
    Adorni, G., Bergenti, F., Cagnoni, S.: Vehicle license plate recognition by means of cellular automata. In: IV (1998)Google Scholar
  2. 2.
    Anagnostopoulos, C.N.E., Anagnostopoulos, I.E., Loumos, V., Kayafas, E.: A license plate-recognition algorithm for intelligent transportation system applications. IEEE Trans. Intell. Transp. Syst. 7(3), 377–392 (2006)CrossRefGoogle Scholar
  3. 3.
    Anagnostopoulos, C.N.E., Anagnostopoulos, I.E., Psoroulas, I.D., Loumos, V., Kayafas, E.: License plate recognition from still images and video sequences: a survey. IEEE Trans. Intell. Transp. Syst. 9(3), 377–391 (2008)CrossRefGoogle Scholar
  4. 4.
    Arteta, C., Lempitsky, V., Noble, J.A., Zisserman, A.: Interactive object counting. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8691, pp. 504–518. Springer, Cham (2014). Scholar
  5. 5.
    Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)CrossRefGoogle Scholar
  6. 6.
    Bulan, O., Kozitsky, V., Ramesh, P., Shreve, M.: Segmentation-and annotation-free license plate recognition with deep localization and failure identification. IEEE Trans. Intell. Transp. Syst. 18(9), 2351–2363 (2017)CrossRefGoogle Scholar
  7. 7.
    Cheang, T.K., Chong, Y.S., Tay, Y.H.: Segmentation-free vehicle license plate recognition using convnet-RNN. arXiv preprint arXiv:1701.06439 (2017)
  8. 8.
    Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2018)CrossRefGoogle Scholar
  9. 9.
    Cherng, S., Fang, C.Y., Chen, C.P., Chen, S.W.: Critical motion detection of nearby moving vehicles in a vision-based driver-assistance system. IEEE Tran. Intell. Transp. Syst. 10(1), 70–82 (2009)CrossRefGoogle Scholar
  10. 10.
    Davies, P., Emmott, N., Ayland, N.: License plate recognition technology for toll violation enforcement. In: Image Analysis for Transport Applications (1990)Google Scholar
  11. 11.
    Du, S., Ibrahim, M., Shehata, M., Badawy, W.: Automatic license plate recognition (ALPR): a state-of-the-art review. IEEE Trans. Circuits Syst. Video Technol. 23(2), 311–325 (2013)CrossRefGoogle Scholar
  12. 12.
    Duan, T.D., Du, T.H., Phuoc, T.V., Hoang, N.V.: Building an automatic vehicle license plate recognition system. In: RIVF (2005)Google Scholar
  13. 13.
    Hegt, H.A., De La Haye, R.J., Khan, N.A.: A high performance license plate recognition system (1998)Google Scholar
  14. 14.
    Hsu, G.S., Alexandra, P., Chen, J.C., Yeh, F., Chen, M.H.: License plate recognition for categorized applications. In: ICVES (2011)Google Scholar
  15. 15.
    Hsu, G.S., Chen, J.C., Chung, Y.Z.: Application-oriented license plate recognition. IEEE Trans. Veh. Technol. 62(2), 552–561 (2013)CrossRefGoogle Scholar
  16. 16.
    Huang, Y.S., Weng, Y.S., Zhou, M.: Critical scenarios and their identification in parallel railroad level crossing traffic control systems. IEEE Trans. Intell. Transp. Syst. 11(4), 968–977 (2010)CrossRefGoogle Scholar
  17. 17.
    Jia, Y., et al.: Caffe: convolutional architecture for fast feature embedding. In: ACM MM (2014)Google Scholar
  18. 18.
    Kim, K.K., Kim, K., Kim, J., Kim, H.J.: Learning-based approach for license plate recognition. In: Neural Networks for Signal Processing (2000)Google Scholar
  19. 19.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: NIPS (2012)Google Scholar
  20. 20.
    Lempitsky, V., Zisserman, A.: Learning to count objects in images. In: NIPS (2010)Google Scholar
  21. 21.
    Li, H., Shen, C.: Reading car license plates using deep convolutional neural networks and LSTMs. arXiv preprint arXiv:1601.05610 (2016)
  22. 22.
    Liu, W., et al.: SSD: single shot MultiBox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). Scholar
  23. 23.
    Liu, X., Wang, Z., Feng, J., Xi, H.: Highway vehicle counting in compressed domain. In: CVPR (2016)Google Scholar
  24. 24.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: CVPR (2015)Google Scholar
  25. 25.
    Masood, S.Z., Shu, G., Dehghan, A., Ortiz, E.G.: License plate detection and recognition using deeply learned convolutional neural networks. arXiv preprint arXiv:1703.07330 (2017)
  26. 26.
    Naito, T., Tsukada, T., Yamada, K., Kozuka, K., Yamamoto, S.: Robust license-plate recognition method for passing vehicles under outside environment. IEEE Trans. Veh. Technol. 49(6), 2309–2319 (2000)CrossRefGoogle Scholar
  27. 27.
    Nijhuis, J., et al.: Car license plate recognition with neural networks and fuzzy logic (1995)Google Scholar
  28. 28.
    Omitaomu, O.A., Ganguly, A.R., Patton, B.W., Protopopescu, V.A.: Anomaly detection in radiation sensor data with application to transportation security. IEEE Trans. Intell. Transp. Syst. 10(2), 324–334 (2009)CrossRefGoogle Scholar
  29. 29.
    Parizi, S.N., Targhi, A.T., Aghazadeh, O., Eklundh, J.O.: Reading street signs using a generic structured object detection and signature recognition approach. In: VISAPP (2009)Google Scholar
  30. 30.
    Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: CVPR (2016)Google Scholar
  31. 31.
    Wang, F., Man, L., Wang, B., Xiao, Y., Pan, W., Lu, X.: Fuzzy-based algorithm for color recognition of license plates. Pattern Recogn. Lett. 29(7), 1007–1020 (2008)CrossRefGoogle Scholar
  32. 32.
    Wu, Y., Li, J.: License plate recognition using deep FCN. In: Sun, F., Liu, H., Hu, D. (eds.) ICCSIP 2016. CCIS, vol. 710, pp. 225–234. Springer, Singapore (2017). Scholar
  33. 33.
    Yamaguchi, K., Nagaya, Y., Ueda, K., Nemoto, H., Nakagawa, M.: A method for identifying specific vehicles using template matching (1999)Google Scholar
  34. 34.
    Yu, M., Kim, Y.D.: An approach to Korean license plate recognition based on vertical edge matching (2000)Google Scholar
  35. 35.
    Zhu, S., Dianat, S., Mestha, L.K.: End-to-end system of license plate localization and recognition. JEI 24(2), 023020 (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.University of Science and Technology of ChinaHefeiChina

Personalised recommendations