Advertisement

License Plate Detection and Recognition in Unconstrained Scenarios

  • Sérgio Montazzolli SilvaEmail author
  • Cláudio Rosito Jung
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11216)

Abstract

Despite the large number of both commercial and academic methods for Automatic License Plate Recognition (ALPR), most existing approaches are focused on a specific license plate (LP) region (e.g. European, US, Brazilian, Taiwanese, etc.), and frequently explore datasets containing approximately frontal images. This work proposes a complete ALPR system focusing on unconstrained capture scenarios, where the LP might be considerably distorted due to oblique views. Our main contribution is the introduction of a novel Convolutional Neural Network (CNN) capable of detecting and rectifying multiple distorted license plates in a single image, which are fed to an Optical Character Recognition (OCR) method to obtain the final result. As an additional contribution, we also present manual annotations for a challenging set of LP images from different regions and acquisition conditions. Our experimental results indicate that the proposed method, without any parameter adaptation or fine tuning for a specific scenario, performs similarly to state-of-the-art commercial systems in traditional scenarios, and outperforms both academic and commercial approaches in challenging ones.

Keywords

License plate Deep learning Convolutional neural networks 

Notes

Acknowledgements

The authors would like to thank the funding agencies CAPES and CNPq, as well as NVIDIA Corporation for donating a Titan X Pascal GPU.

References

  1. 1.
    Anagnostopoulos, C.N., Anagnostopoulos, I., Psoroulas, I., Loumos, V., Kayafas, E.: License plate recognition from still images and video sequences: a survey. IEEE Trans. Intell. Transp. Syst. 9(3), 377–391 (2008).  https://doi.org/10.1109/TITS.2008.922938. http://ieeexplore.ieee.org/document/4518951/CrossRefGoogle Scholar
  2. 2.
    Bulan, O., Kozitsky, V., Ramesh, P., Shreve, M.: Segmentation- and annotation-free license plate recognition with deep localization and failure identification. IEEE Trans. Intell. Transp. Syst. 18(9), 2351–2363 (2017).  https://doi.org/10.1109/TITS.2016.2639020CrossRefGoogle Scholar
  3. 3.
    Delmar Kurpiel, F., Minetto, R., Nassu, B.T.: Convolutional neural networks for license plate detection in images. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 3395–3399. IEEE (2017)  https://doi.org/10.1109/ICIP.2017.8296912. http://ieeexplore.ieee.org/document/8296912/
  4. 4.
    Du, S., Ibrahim, M., Shehata, M., Badawy, W.: Automatic license plate recognition (ALPR): a state-of-the-art review. IEEE Trans. Circuits Syst. Video Technol. 23(2), 311–325 (2013).  https://doi.org/10.1109/TCSVT.2012.2203741. http://ieeexplore.ieee.org/document/6213519/CrossRefGoogle Scholar
  5. 5.
    Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)CrossRefGoogle Scholar
  6. 6.
    Gonçalves, G.R., da Silva, S.P.G., Menotti, D., Schwartz, W.R.: Benchmark for license plate character segmentation. J. Electron. Imaging 25(5), 1–5 (2016). http://www.ssig.dcc.ufmg.br/wp-content/uploads/2016/11/JEI-2016-Benchmark.pdfCrossRefGoogle Scholar
  7. 7.
    Gupta, A., Vedaldi, A., Zisserman, A.: Synthetic data for text localisation in natural images. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2315–2324 (2016)Google Scholar
  8. 8.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 4, pp. 770–778. IEEE, June 2016.  https://doi.org/10.1109/CVPR.2016.90
  9. 9.
    Hsu, G.S., Ambikapathi, A., Chung, S.L., Su, C.P.: Robust license plate detection in the wild. In: 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1–6. IEEE, August 2017.  https://doi.org/10.1109/AVSS.2017.8078493. http://ieeexplore.ieee.org/document/8078493/
  10. 10.
    Hsu, G.S., Chen, J.C., Chung, Y.Z.: Application-oriented license plate recognition. IEEE Trans. Veh. Technol. 62(2), 552–561 (2013).  https://doi.org/10.1109/TVT.2012.2226218CrossRefGoogle Scholar
  11. 11.
    Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2261–2269. IEEE, July 2017.  https://doi.org/10.1109/CVPR.2017.243. http://arxiv.org/abs/1608.06993. http://ieeexplore.ieee.org/document/8099726/
  12. 12.
    Huang, J., et al.: Speed/accuracy trade-offs for modern convolutional object detectors. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3296–3297. IEEE, July 2017.  https://doi.org/10.1109/CVPR.2017.351. http://ieeexplore.ieee.org/document/8099834/
  13. 13.
    Jaderberg, M., Simonyan, K., Vedaldi, A., Zisserman, A.: Synthetic data and artificial neural networks for natural scene text recognition. In: NIPS, Conference on Neural Information Processing Systems, pp. 1–10 (2014)Google Scholar
  14. 14.
    Jaderberg, M., Simonyan, K., Zisserman, A., kavukcuoglu, k.: Spatial transformer networks. In: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems 28, pp. 2017–2025. Curran Associates, Inc. (2015)Google Scholar
  15. 15.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2014)Google Scholar
  16. 16.
    Krause, J., Stark, M., Deng, J., Fei-Fei, L.: 3D object representations for fine-grained categorization. In: 4th International IEEE Workshop on 3D Representation and Recognition (3dRR-2013), Sydney, Australia (2013)Google Scholar
  17. 17.
    Laroca, R., et al.: A robust real-time automatic license plate recognition based on the YOLO detector. CoRR abs/1802.09567 (2018). http://arxiv.org/abs/1802.09567
  18. 18.
    Li, H., Shen, C.: Reading Car License Plates Using Deep Convolutional Neural Networks and LSTMs. arXiv preprint arXiv:1601.05610, January 2016. http://arxiv.org/abs/1601.05610
  19. 19.
    Li, H., Wang, P., Shen, C.: Towards end-to-end car license plates detection and recognition with deep neural networks. CoRR abs/1709.08828 (2017). http://arxiv.org/abs/1709.08828
  20. 20.
    Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10602-1_48CrossRefGoogle Scholar
  21. 21.
    Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46448-0_2CrossRefGoogle Scholar
  22. 22.
    Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning. In: NIPS Workshop on Deep Learning and Unsupervised Feature Learning, vol. 2011, p. 5 (2011)Google Scholar
  23. 23.
    Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779–788. IEEE, June 2016.  https://doi.org/10.1109/CVPR.2016.91. http://ieeexplore.ieee.org/document/7780460/
  24. 24.
    Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6517–6525. IEEE, July 2017.  https://doi.org/10.1109/CVPR.2017.690. http://arxiv.org/abs/1612.08242. http://ieeexplore.ieee.org/document/8100173/
  25. 25.
    Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017).  https://doi.org/10.1109/TPAMI.2016.2577031CrossRefGoogle Scholar
  26. 26.
    Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. (IJCV) 115(3), 211–252 (2015).  https://doi.org/10.1007/s11263-015-0816-yMathSciNetCrossRefGoogle Scholar
  27. 27.
    Selmi, Z., Ben Halima, M., Alimi, A.M.: Deep learning system for automatic license plate detection and recognition. In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), pp. 1132–1138. IEEE, November 2017.  https://doi.org/10.1109/ICDAR.2017.187. http://ieeexplore.ieee.org/document/8270118/
  28. 28.
    Silva, S.M., Jung, C.R.: Real-time Brazilian license plate detection and recognition using deep convolutional neural networks. In: 2017 30th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 55–62, October 2017.  https://doi.org/10.1109/SIBGRAPI.2017.14
  29. 29.
    Wang, F., Zhao, L., Li, X., Wang, X., Tao, D.: Geometry-aware scene text detection with instance transformation network. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, pp. 1381–1389 (2018)Google Scholar
  30. 30.
    Weinman, J.J., Learned-Miller, E., Hanson, A.R.: Scene text recognition using similarity and a lexicon with sparse belief propagation. IEEE Trans. Pattern Anal. Mach. Intell. 31(10), 1733–1746 (2009)CrossRefGoogle Scholar
  31. 31.
    Xie, L., Ahmad, T., Jin, L., Liu, Y., Zhang, S.: A new CNN-based method for multi-directional car license plate detection. IEEE Trans. Intell. Transp. Syst. 19(2), 507–517 (2018).  https://doi.org/10.1109/TITS.2017.2784093CrossRefGoogle Scholar
  32. 32.
    Zhou, X., Zhu, M., Leonardos, S., Daniilidis, K.: Sparse representation for 3D shape estimation: a convex relaxation approach. IEEE Trans. Pattern Anal. Mach. Intell. 39(8), 1648–1661 (2017)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Institute of InformaticsFederal University of Rio Grande do SulPorto AlegreBrazil

Personalised recommendations