License plate detection and recognition using hierarchical feature layers from CNN

  • Qiang Lu
  • Yu Liu
  • Jing Huang
  • Xiaohui Yuan
  • Qingxin HuEmail author


In recent years, a variety of systems using deep convolutional neural network (CNN) approaches have achieved good performance on license plate detection and character recognition. However, most of these systems are not stable when the scenes changed, specification of each hierarchical layer to get the final detection result, which can detect multi-scale license plates from an input image. Meanwhile, at the stage of character recognition, data annotation is heavy and time-consuming, giving rise to a large burden on training a better model. We devise an algorithm to generate annotated training data automatically and approximate the data from the real scenes. Our system used for detecting license plate achieves 99.99% mean average precision (mAP) on OpenITS datasets. Character recognition also sees high accuracy, thus verifying the superiority of our method.


License plate detection Character recognition Hierarchical feature layers Generated training data 



This work is supported by the National Nature Science Foundation of China (No. 61672201), the Natural Science Foundation of Anhui Province of China (No. 1708085MF158) and the China Scholarship Council (CSC No.201706695044).


  1. 1.
    Abolghasemi V, Ahmadyfard A (2009) An edge-based color-aided method for license plate detection. Image Vis Comput 27(8):1134CrossRefGoogle Scholar
  2. 2.
    Alexe B, Deselaers T, Ferrari V (2012) Measuring the Objectness of Image Windows. IEEE Trans Pattern Anal Mach Intell 34(11):2189CrossRefGoogle Scholar
  3. 3.
    Arbelaez P, Ponttuset J, Barron J, Marques F, Malik J (2014) Multiscale Combinatorial Grouping, in Computer Vision and Pattern Recognition, 328–335Google Scholar
  4. 4.
    Cao G, Chen J, Jiang J (2003) An adaptive approach to vehicle license plate localization, in Industrial Electronics Society, 2003. IECON ‘03. the Conference of the IEEE, 1786–1791 Vol.2Google Scholar
  5. 5.
    Chen YN, Han CC, Wang CT, Jeng BS (2006) The Application of a Convolution Neural Network on Face and License Plate Detection, in International Conference on Pattern Recognition, 552–555Google Scholar
  6. 6.
    Chen Y, Nguyen TV, Kankanhalli M, Yuan J (2014) Audio Matters in Visual Attention, Circuits and Systems for Video Technology IEEE Transactions on 24(11), 1992Google Scholar
  7. 7.
    Chen Y, Zhang L, Liu X, Chen C (2016) Flickr Circles: Aesthetic Tendency Discovery by Multi-View Regularized Topic Modeling, Information Sciences 372(C), 148Google Scholar
  8. 8.
    Chen Y, Hu Y, Zhang L, Li P, Zhang C (2017) Engineering Deep Representations for Modeling Aesthetic Perception, IEEE Transactions on Cybernetics PP (99), 1Google Scholar
  9. 9.
    Dai J, Li Y, He K, Sun J (2016) R-fcn: Object detection via region-based fully convolutional networks, Advances in neural information processing systems, 379–387Google Scholar
  10. 10.
    Deb K, Jo K (2009) A Vehicle License Plate Detection Method for Intelligent Transportation System Applications 40(8), 689Google Scholar
  11. 11.
    Dlagnekov (2004) License plate detection using adaboost, Computer Science and Engineering Department pp. 42–53Google Scholar
  12. 12.
    Feng BY, Ren M, Zhang XY, Liu CL (2014) Effective license plate detection using fast candidate region selection and covariance feature based filtering, in IEEE International Conference on Advanced Video and Signal Based Surveillance, 1–60Google Scholar
  13. 13.
    Girshick R, Donahue J, Darrell T, Malik J (2013) Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation, in IEEE Conference on Computer Vision and Pattern Recognition, 580–587Google Scholar
  14. 14.
    He K, Zhang X, Ren S, Sun J (2015) Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence 37(9), 1904Google Scholar
  15. 15.
    Hongliang B, Changping L (2004) A hybrid license plate extraction method based on edge statistics and morphology, in International Conference on Pattern Recognition, 831–834 Vol. 2Google Scholar
  16. 16.
    Huang H, Gu M, Chao H (2008) An Efficient Method of License Plate Location in Natural-Scene Image, in International Conference on Fuzzy Systems and Knowledge Discovery, Fskd 2008, 18–20 October 2008, Jinan, Shandong, China, Proceedings, Volume, 15–19Google Scholar
  17. 17.
    Hung KM, Hsieh CT (2010) A Real-Time Mobile Vehicle License Plate Detection and Recognition. Tamkang J Sci Eng 13(4):197MathSciNetGoogle Scholar
  18. 18.
    Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) Ssde: Single shot multibox detector, European conference on computer vision, pp. 21–37Google Scholar
  19. 19.
    Long J, Shelhamer E, Darrell T (2014) Fully convolutional networks for semantic segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence PP (99), 640Google Scholar
  20. 20.
    Luo L, Sun H, Zhou W, Luo L (2009) An Efficient Method of License Plate Location, in International Conference on Information Science and Engineering, 770–773Google Scholar
  21. 21.
    Maglad KW (2012) A vehicle license plate detection and recognition system. J Comput Sci 8(3):310CrossRefGoogle Scholar
  22. 22.
    Parkhi OM, Vedaldi A, Zisserman A (2015) Deep Face Recognition, in British Machine Vision Conference, 41.1–41.12Google Scholar
  23. 23.
    Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks, IEEE Transactions on Pattern Analysis and Machine Intelligence PP (99), 1Google Scholar
  24. 24.
    Simonyan K, Zisserman A (2014) Very Deep Convolutional Networks for Large-Scale Image Recognition, Computer ScienceGoogle Scholar
  25. 25.
    Uijlings JR, Sande KE, Gevers T, Smeulders AW (2013) Selective Search for Object Recognition. Int J Comput Vis 104(2):154CrossRefGoogle Scholar
  26. 26.
    Wazalwar D, Oruklu E, Saniie J (2012) A Design Flow for Robust License Plate Localization and Recognition in Complex Scenes. J Trans Technol 02(1):13Google Scholar
  27. 27.
    Wu HHP, Chen HH, Wu RJ, Shen DF (2006) License Plate Extraction in Low Resolution Video, in International Conference on Pattern Recognition, 824–827Google Scholar
  28. 28.
    Xia Y, Liu Z, Yan Y, Chen Y, Zhang L, Zimmermann R (2017) Media Quality Assessment by Perceptual Gaze-Shift Patterns Mining, IEEE Transactions on Multimedia PP (99), 1Google Scholar
  29. 29.
    H. Zhang, W. Jia, X. He, Q. Wu (2006) Learning-Based License Plate Detection Using Global and Local Features, in International Conference on Pattern Recognition, 1102–1105Google Scholar
  30. 30.
    Zitnick CL, Dollár P (2014) Edge Boxes: Locating Object Proposals from Edges 8693, 391Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Qiang Lu
    • 1
  • Yu Liu
    • 2
  • Jing Huang
    • 2
  • Xiaohui Yuan
    • 3
  • Qingxin Hu
    • 2
    Email author
  1. 1.School of Computer and Information, Anhui Province Key Laboratory of Industry Safety and Emergency Technology, Hefei Engineering Research Center of Electric Power Data ApplicationHefei University of TechnologyHefeiChina
  2. 2.School of Computer and InformationHefei University of TechnologyHefeiChina
  3. 3.Department of Science and EngineeringUniversity of North TexasDentonUSA

Personalised recommendations