Advertisement

Towards End-to-End License Plate Detection and Recognition: A Large Dataset and Baseline

  • Zhenbo Xu
  • Wei Yang
  • Ajin Meng
  • Nanxue Lu
  • Huan Huang
  • Changchun Ying
  • Liusheng Huang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11217)

Abstract

Most current license plate (LP) detection and recognition approaches are evaluated on a small and usually unrepresentative dataset since there are no publicly available large diverse datasets. In this paper, we introduce CCPD, a large and comprehensive LP dataset. All images are taken manually by workers of a roadside parking management company and are annotated carefully. To our best knowledge, CCPD is the largest publicly available LP dataset to date with over 250k unique car images, and the only one provides vertices location annotations. With CCPD, we present a novel network model which can predict the bounding box and recognize the corresponding LP number simultaneously with high speed and accuracy. Through comparative experiments, we demonstrate our model outperforms current object detection and recognition approaches in both accuracy and speed. In real-world applications, our model recognizes LP numbers directly from relatively high-resolution images at over 61 fps and 98.5% accuracy.

Keywords

Object detection Object recognition Object segmentation Convolutional neural network 

Notes

Acknowledgments

This work was supported by the National Natural Science Foundation of China (No. 61572456) and the Anhui Province Guidance Funds for Quantum Communication and Quantum Computers.

Supplementary material

474201_1_En_16_MOESM1_ESM.pdf (11.1 mb)
Supplementary material 1 (pdf 11372 KB)

References

  1. 1.
    Xie, L., Ahmad, T., Jin, L., Liu, Y., Zhang, S.: A new CNN-based method for multi-directional car license plate detection. IEEE Trans. Intell. Transp. Syst. 19, 507–517 (2018)CrossRefGoogle Scholar
  2. 2.
    Al-Shemarry, M.S., Li, Y., Abdulla, S.: Ensemble of adaboost cascades of 3L-LBPs classifiers for license plates detection with low quality images. Expert Syst. Appl. 92, 216–235 (2018)CrossRefGoogle Scholar
  3. 3.
    Laroca, R., et al.: A robust real-time automatic license plate recognition based on the yolo detector (2018). arXiv preprint: arXiv:1802.09567
  4. 4.
    Montazzolli, S., Jung, C.: Real-time Brazilian license plate detection and recognition using deep convolutional neural networks. In: 2017 30th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 55–62 (2017)Google Scholar
  5. 5.
    Selmi, Z., Halima, M.B., Alimi, A.M.: Deep learning system for automatic license plate detection and recognition. In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), vol. 1, pp. 1132–1138. IEEE (2017)Google Scholar
  6. 6.
    Rizvi, S.T.H., Patti, D., Björklund, T., Cabodi, G., Francini, G.: Deep classifiers-based license plate detection, localization and recognition on GPU-powered mobile platform. Future Internet 9(4), 66 (2017)CrossRefGoogle Scholar
  7. 7.
    Masood, S.Z., Shu, G., Dehghan, A., Ortiz, E.G.: License plate detection and recognition using deeply learned convolutional neural networks (2017). arXiv preprint: arXiv:1703.07330
  8. 8.
    Yuan, Y., Zou, W., Zhao, Y., Wang, X., Hu, X., Komodakis, N.: A robust and efficient approach to license plate detection. IEEE Trans. Image Process. 26(3), 1102–1114 (2017)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Cheang, T.K., Chong, Y.S., Tay, Y.H.: Segmentation-free vehicle license plate recognition using convnet-RNN (2017). arXiv preprint: arXiv:1701.06439
  10. 10.
    Azam, S., Islam, M.M.: Automatic license plate detection in hazardous condition. J. Vis. Commun. Image Represent. 36, 172–186 (2016)CrossRefGoogle Scholar
  11. 11.
    Zhang, Z., Yin, S.: Hough transform and its application in vehicle license plate tilt correction. Comput. Inf. Sci. 1(3), 116 (2008)Google Scholar
  12. 12.
    Li, H., Wang, P., Shen, C.: Towards end-to-end car license plates detection and recognition with deep neural networks (2017). arXiv preprint: arXiv:1709.08828
  13. 13.
    Caltech: Caltech Licese Plate Dataset. http://www.vision.caltech.edu/html-files/archive.html
  14. 14.
  15. 15.
    Špaňhel, J., Sochor, J., Juránek, R., Herout, A., Maršík, L., Zemčík, P.: Holistic recognition of low quality license plates by CNN using track annotated data. In: 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1–6. IEEE (2017)Google Scholar
  16. 16.
    Hsu, G.S., Ambikapathi, A., Chung, S.L., Su, C.P.: Robust license plate detection in the wild. In: 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1–6. IEEE (2017)Google Scholar
  17. 17.
    Gonçalves, G.R., da Silva, S.P.G., Menotti, D., Schwartz, W.R.: Benchmark for license plate character segmentation. J. Electron. Imaging 25(5), 1–5 (2016)CrossRefGoogle Scholar
  18. 18.
    Haralick, R.M., Sternberg, S.R., Zhuang, X.: Image analysis using mathematical morphology. IEEE Trans. Pattern Anal. Mach. Intell. PAMI 9(4), 532–550 (1987)CrossRefGoogle Scholar
  19. 19.
    Hsieh, J.W., Yu, S.H., Chen, Y.S.: Morphology-based license plate detection from complex scenes. In: Proceedings of the 16th International Conference on Pattern Recognition, vol. 3, pp. 176–179. IEEE (2002)Google Scholar
  20. 20.
    Wu, H.H.P., Chen, H.H., Wu, R.J., Shen, D.F.: License plate extraction in low resolution video. In: 18th International Conference on Pattern Recognition, ICPR 2006, vol. 1, pp. 824–827. IEEE (2006)Google Scholar
  21. 21.
    Yu, S., Li, B., Zhang, Q., Liu, C., Meng, M.Q.H.: A novel license plate location method based on wavelet transform and emd analysis. Pattern Recogn. 48(1), 114–125 (2015)CrossRefGoogle Scholar
  22. 22.
    Saha, S., Basu, S., Nasipuri, M., Basu, D.K.: License plate localization from vehicle images: an edge based multi-stage approach. Int. J. Recent Trends Eng. 1(1), 284–288 (2009)Google Scholar
  23. 23.
    Wang, R., Sang, N., Huang, R., Wang, Y.: License plate detection using gradient information and cascade detectors. Optik Int. J. Light Electron Opt. 125(1), 186–190 (2014)CrossRefGoogle Scholar
  24. 24.
    Bachchan, A.K., Gorai, A., Gupta, P.: Automatic license plate recognition using local binary pattern and histogram matching. In: Huang, D.-S., Jo, K.-H., Figueroa-García, J.C. (eds.) ICIC 2017, Part II. LNCS, vol. 10362, pp. 22–34. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-63312-1_3CrossRefGoogle Scholar
  25. 25.
    Deb, K., Jo, K.H.: HSI color based vehicle license plate detection. In: International Conference on Control, Automation and Systems, ICCAS 2008, pp. 687–691. IEEE (2008)Google Scholar
  26. 26.
    Yao, Z., Yi, W.: License plate detection based on multistage information fusion. Inf. Fusion 18, 78–85 (2014)CrossRefGoogle Scholar
  27. 27.
    Lee, Y., Song, T., Ku, B., Jeon, S., Han, D.K., Ko, H.: License plate detection using local structure patterns. In: 2010 Seventh IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 574–579. IEEE (2010)Google Scholar
  28. 28.
    Girshick, R.: Fast R-CNN (2015). arXiv preprint: arXiv:1504.08083
  29. 29.
    Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)Google Scholar
  30. 30.
    Liu, W., et al.: SSD: single shot MultiBox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016, Part I. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46448-0_2CrossRefGoogle Scholar
  31. 31.
    Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)Google Scholar
  32. 32.
    Redmon, J., Farhadi, A.: Yolo9000: better, faster, stronger (2016). arXiv preprint: arXiv:1612.08242
  33. 33.
    Ho, W.T., Lim, H.W., Tay, Y.H.: Two-stage license plate detection using gentle adaboost and SIFT-SVM. In: First Asian Conference on Intelligent Information and Database Systems, ACIIDS 2009, pp. 109–114. IEEE (2009)Google Scholar
  34. 34.
    Duan, T.D., Du, T.H., Phuoc, T.V., Hoang, N.V.: Building an automatic vehicle license plate recognition system. In: Proceedings of the International Conference on Computer Science RIVF, pp. 59–63 (2005)Google Scholar
  35. 35.
    Yousef, K.M.A., Al-Tabanjah, M., Hudaib, E., Ikrai, M.: Sift based automatic number plate recognition. In: 2015 6th International Conference on Information and Communication Systems (ICICS), pp. 124–129. IEEE (2015)Google Scholar
  36. 36.
    Maglad, K.W.: A vehicle license plate detection and recognition system. J. Comput. Sci. 8(3), 310 (2012)CrossRefGoogle Scholar
  37. 37.
    Gou, C., Wang, K., Yao, Y., Li, Z.: Vehicle license plate recognition based on extremal regions and restricted Boltzmann machines. IEEE Trans. Intell. Transp. Syst. 17(4), 1096–1107 (2016)CrossRefGoogle Scholar
  38. 38.
    Ciregan, D., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3642–3649. IEEE (2012)Google Scholar
  39. 39.
    Abdel-Hakim, A.E., Farag, A.A.: CSIFT: a SIFT descriptor with color invariant characteristics. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), vol. 2, pp. 1978–1983 (2006)Google Scholar
  40. 40.
    Anagnostopoulos, C.N.E., Anagnostopoulos, I.E., Psoroulas, I.D., Loumos, V., Kayafas, E.: License plate recognition from still images and video sequences: a survey. IEEE Trans. Intell. Transp. Syst. 9(3), 377–391 (2008)CrossRefGoogle Scholar
  41. 41.
    Chorowski, J.K., Bahdanau, D., Serdyuk, D., Cho, K., Bengio, Y.: Attention-based models for speech recognition. In: Advances in Neural Information Processing Systems, pp. 577–585 (2015)Google Scholar
  42. 42.
    Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Object detectors emerge in deep scene CNNs (2014). arXiv preprint arXiv:1412.6856
  43. 43.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)Google Scholar
  44. 44.
    Russakovsky, O., et al.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  45. 45.
    Wang, S.Z., Lee, H.J.: A cascade framework for a real-time statistical plate recognition system. IEEE Trans. Inf. Forensics Secur. 2(2), 267–282 (2007)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.School of Computer Science and TechnologyUniversity of Science and Technology of ChinaHefeiChina
  2. 2.Xingtai Financial Holdings Group Co., Ltd.HefeiChina

Personalised recommendations