Skip to main content
SpringerLink
Log in

Representation learning in a deep network for license plate recognition

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The goal of license plate recognition (LPR) is to read the license plate characters. Due to image degradation, there are many difficulties in the way of achieving this goal. In this paper, the proposed method recognizes the license plate characters without employing the traditional segmentation and binarization techniques. This method uses a deep learning algorithm and tries to achieve better learning experience by engaging a multi-task learning algorithm based on sharing features. The features of license plate characters are extracted by a deep encoder-decoder network, and transferred to 8 parallel classifiers for recognition. To evaluate the current work, a database of 11,000 license plate images, collected from a currently working surveillance system installed on a dual carriageway, is employed. The proposed method achieved the correct character recognition rate of 96% for 4000 test images that is acceptable in comparison to the competing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Notes

  1. t-distributed stochastic neighbor embedding (t-SNE)

References

  1. Abolghasemi V, Ahmadyfard A (2009) An edge-based color-aided method for license plate detection. Image Vis Comput 27:1134–1142. https://doi.org/10.1016/j.imavis.2008.10.012

    Article  Google Scholar 

  2. Al-Rfou R, Alain G, Almahairi A, Angermueller C, Bahdanau D, Ballas N, Bastien F, Bayer J, Belikov A, Belopolsky A, Bengio Y, Bergeron A, Bergstra J, Bisson V, Bleecher Snyder J, Bouchard N, Boulanger-Lewandowski N, Bouthillier X, de Brébisson A, Breuleux O, Carrier P-L, Cho K, Chorowski J, Christiano P, Cooijmans T, Côté M-A, Côté M, Courville A, Dauphin YN, Delalleau O, Demouth J, Desjardins G, Dieleman S, Dinh L, Ducoffe M, Dumoulin V, Ebrahimi Kahou S, Erhan D, Fan Z, Firat O, Germain M, Glorot X, Goodfellow I, Graham M, Gulcehre C, Hamel P, Harlouchet I, Heng J-P, Hidasi B, Honari S, Jain A, Jean S, Jia K, Korobov M, Kulkarni V, Lamb A, Lamblin P, Larsen E, Laurent C, Lee S, Lefrancois S, Lemieux S, Léonard N, Lin Z, Livezey JA, Lorenz C, Lowin J, Ma Q, Manzagol P-A, Mastropietro O, McGibbon RT, Memisevic R, van Merriënboer B, Michalski V, Mirza M, Orlandi A, Pal C, Pascanu R, Pezeshki M, Raffel C, Renshaw D, Rocklin M, Romero A, Roth M, Sadowski P, Salvatier J, Savard F, Schlüter J, Schulman J, Schwartz G, Serban IV, Serdyuk D, Shabanian S, Simon É, Spieckermann S, Subramanyam SR, Sygnowski J, Tanguay J, van Tulder G, Turian J, Urban S, Vincent P, Visin F, de Vries H, Warde-Farley D, Webb DJ, Willson M, Xu K, Xue L, Yao L, Zhang S, Zhang Y (2016) Theano: A Python framework for fast computation of mathematical expressions, arXiv E-Prints. abs/1605.0. http://arxiv.org/abs/1605.02688. Accessed 2 Oct 2017

  3. Ashtari AH, Nordin MJ, Fathy M (2014) An Iranian license plate recognition system based on color features. IEEE Trans Intell Transp Syst 15:1690–1705. https://doi.org/10.1109/TITS.2014.2304515

    Article  Google Scholar 

  4. Bengio Y (2012) Deep learning of representations for unsupervised and transfer learning. JMLR Work Conf Proc 7:1–20

    Google Scholar 

  5. Bengio Y, Courville A, Vincent P (2013) Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Mach Intell 35:1798–1828. https://doi.org/10.1109/TPAMI.2013.50

    Article  Google Scholar 

  6. Björklund T, Fiandrotti A, Annarumma M, Francini G, Magli E (2019) Robust license plate recognition using neural networks trained on synthetic images. Pattern Recogn 93:134–146. https://doi.org/10.1016/J.PATCOG.2019.04.007

    Article  Google Scholar 

  7. Chang SL, Chen LS, Chung YC, Chen SW (2004) Automatic license plate recognition. IEEE Trans Intell Transp Syst 5:42–53. https://doi.org/10.1109/TITS.2004.825086

    Article  Google Scholar 

  8. Chen KN, Chen CH, Chang CC (2012) Efficient illumination compensation techniques for text images. Digit Signal Process A Rev J 22:726–733. https://doi.org/10.1016/j.dsp.2012.04.010

    Article  Google Scholar 

  9. Chollet F (2015) Others, Keras. https://github.com/fchollet/keras

  10. Cireşan D, Meier U, Schmidhuber J (2012) Multi-column deep neural networks for image classification. In: Comput. Vis. Pattern Recognit. (CVPR), 2012 IEEE Conf., IEEE. https://doi.org/10.1109/CVPR.2012.6248110, p 3642–3649

  11. CS231n convolutional neural networks for visual recognition - visualizing what ConvNets learn (n.d.). http://cs231n.github.io/understanding-cnn/

  12. Erhan D, Courville A, Vincent P (2010) Why does unsupervised pre-training help deep learning. J Mach Learn Res 11:625–660. https://doi.org/10.1145/1756006.1756025

    Article  MathSciNet  MATH  Google Scholar 

  13. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press http://www.deeplearningbook.org. Accessed 10 Aug 2017

  14. Havaei M, Davy A, Warde-Farley D, Biard A, Courville A, Bengio Y, Pal C, Jodoin PM, Larochelle H (2017) Brain tumor segmentation with deep neural networks. Med Image Anal 35:18–31. https://doi.org/10.1016/j.media.2016.05.004

    Article  Google Scholar 

  15. He K, Zhang X, Ren S, Sun J (2016) Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. Proc. IEEE Int. Conf. Comput. Vis. https://doi.org/10.1109/ICCV.2015.123, p 1026–1034

  16. Hinton GE, Srivastava N, Krizhevsky A, Sutskever N, Salakhutdinov RR (2012) Improving neural net- works by preventing co-adaptation of feature detectors, arXiv:1207.0580 [cs.NE]. https://arxiv.org/abs/1207.0580, 1–5

  17. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9:1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  18. Hsu G-S, Chen J-C, Chung Y-Z (2013) Application-oriented license plate recognition. IEEE Trans Veh Technol 62:552–561. https://doi.org/10.1109/TVT.2012.2226218

    Article  Google Scholar 

  19. Jaderberg M, Simonyan K, Vedaldi A, Zisserman A (2016) Reading text in the wild with convolutional neural networks. Int J Comput Vis 116:1–20. https://doi.org/10.1007/s11263-015-0823-z

    Article  MathSciNet  Google Scholar 

  20. Jiao J, Ye Q, Huang Q (2009) A configurable method for multi-style license plate recognition. Pattern Recogn 42:358–369. https://doi.org/10.1016/j.patcog.2008.08.016

    Article  MATH  Google Scholar 

  21. Kashef S, Nezamabadi-pour H, Rashedi E (2017) Adaptive enhancement and binarization techniques for degraded plate images. Multimed Tools Appl 77:1–19. https://doi.org/10.1007/s11042-017-5229-1

    Article  Google Scholar 

  22. Ko M-A, Kim Y-M (2003) License plate surveillance system using weighted template matching, 32nd Appl. Imag. Pattern Recognit. Work. 2003. Proceedings. https://doi.org/10.1109/AIPR.2003.1284283, p 269–274

  23. Ko MA, Kim YM (2004) A simple OCR method from strong perspective view(b). In: Proc. - Appl. Imag. Pattern Recognit. Work., IEEE. https://doi.org/10.1109/AIPR.2004.8, p. 235–240

  24. Kobchaisawat T, Chalidabhongse TH (2016) A method for multi-oriented Thai text localization in natural scene images using Convolutional Neural Network. In: IEEE 2015 Int. Conf. Signal Image Process. Appl. ICSIPA 2015 - Proc. https://doi.org/10.1109/ICSIPA.2015.7412193, p. 220–225

  25. Kocer HE, Cevik KK (2011) Artificial neural networks based vehicle license plate recognition. Procedia Comput Sci 3:1033–1037. https://doi.org/10.1016/j.procs.2010.12.169

    Article  Google Scholar 

  26. Krizhevsky A, Hinton GE, Sutskever I (2012) ImageNet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 1–9. https://doi.org/10.1016/j.protcy.2014.09.007

    Article  Google Scholar 

  27. Li H, Shen C (2016) Reading car license plates using deep convolutional neural networks and lSTMs, arXiv:1601.05610 [cs.CV]. 1–17. http://arxiv.org/abs/1601.05610

  28. Li H, Shen C (2016) Reading car license plates using deep convolutional neural networks and LSTMs. http://arxiv.org/abs/1601.05610. Accessed 27 Aug 2019

  29. Lin S-P, Wu B-F, Chiu C-C (2007) Extracting characters from real vehicle licence plates out-of-doors. IET Comput Vis 1:2–10. https://doi.org/10.1049/iet-cvi:20050132

    Article  MathSciNet  Google Scholar 

  30. Liu X, Kawanishi T, Wu X, Kashino K (2016) Scene text recognition with high performance CNN classifier and efficient word inference. Icassp 2016:1322–1326

    Google Scholar 

  31. Llorens D, Marzal A, Palazón V, Vilar J (2005) Car license plates extraction and recognition based on connected components analysis and HMM decoding. Springer-Verlag, Berlin Heidelb, pp 571–578. https://doi.org/10.1007/11492429_69

    Book  Google Scholar 

  32. Naito T, Tsukada T, Yamada K, Kozuka K, Yamamoto S (2000) Robust license-plate recognition method for passing vehicles under outside environment. IEEE Trans Veh Technol 49:2309–2319. https://doi.org/10.1109/25.901900

    Article  Google Scholar 

  33. Nejati M, Majidi A, Jalalat M (2015) License plate recognition based on edge histogram analysis and classifier ensemble. Conf Signal Process Intell Syst 48–52. https://doi.org/10.1109/SPIS.2015.7422310

  34. Nukano T, Fukumi M, Khalid M (2004) Vehicle license plate character recognition by neural networks. In: Proc. 2004 Int. Symp. Intell. Signal Process. Commun. Syst. 2004. ISPACS 2004, IEEE. https://doi.org/10.1109/ISPACS.2004.1439164, p. 771–775

  35. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830 http://scikit-learn.org/stable/documentation.html

    MathSciNet  MATH  Google Scholar 

  36. Qian Y, Bi M, Tan T, Yu K (2016) Very deep convolutional neural networks for noise robust speech recognition. IEEE/ACM Trans. Audio, Speech, Lang. Process 24: 2263–2276. https://doi.org/10.1109/TASLP.2016.2602884

    Article  Google Scholar 

  37. Rashedi E, Nezamabadi-pour H (2017) A hierarchical algorithm for vehicle license plate localization. Multimed Tools Appl 77:1–20. https://doi.org/10.1007/s11042-017-4429-z

    Article  Google Scholar 

  38. Salahshoor M, Broumandnia A, Rastgarpour M (2014) Application of intelligent systems for Iranian license plate recognition, Iran. Conf. Intell. Syst. High. Educ. Complex, Bam, Iran. 1–6. https://doi.org/10.1109/IranianCIS.2014.6802578

  39. Sedighi A, Vafadust M (2011) A new and robust method for character segmentation and recognition in license plate images. Expert Syst Appl 38:13497–13504. https://doi.org/10.1016/j.eswa.2011.02.030

    Article  Google Scholar 

  40. Tunisia data set on Kaggle (visited 8/15/2019) (2019). https://www.kaggle.com/achrafkhazri/anpr-dataset-tunisian-plates-and-digits. Accessed 10 Aug 2017

  41. van der Maaten L, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9:2579–2605 http://www.jmlr.org/papers/v9/vandermaaten08a.html. Accessed 10 Aug 2017

  42. Vázquez N, Nakano M, Pérez-Meana H (2002) Automatic system for localization and recognition of vehicle plate numbers. J Appl Res Technol 1:63–77 http://www.scielo.org.mx/pdf/jart/v1n1/v1n1a6.pdf. Accessed June 1, 2017

    Google Scholar 

  43. Vincent P, Larochelle H (2010) Stacked Denoising autoencoders: learning useful representations in a deep network with a local Denoising criterion. J Mach Learn Res 11:3371–3408. https://doi.org/10.1111/1467-8535.00290

    Article  MathSciNet  MATH  Google Scholar 

  44. Wang R, Sang N, Huang R, Wang Y (2014) License plate detection using gradient information and cascade detectors. Optik (Stuttg) 125:186–190. https://doi.org/10.1016/j.ijleo.2013.06.008

    Article  Google Scholar 

  45. Xu J-F, Li S-F, Yu M-S (n.d.) Car license plate extraction using color and edge information, in: Proc. 2004 Int. Conf. Mach. Learn. Cybern. (IEEE Cat. No.04EX826), IEEE. https://doi.org/10.1109/ICMLC.2004.1380528, p. 3904–3907

  46. Yang Y, Gao X, Yang G (2011) Study the method of vehicle license locating based on color segmentation. In: Procedia Eng., Elsevier B.V. https://doi.org/10.1016/j.proeng.2011.08.245, p 1324–1329

    Article  Google Scholar 

  47. Yu S, Li B, Zhang Q, Liu C, Meng MQH (2015) A novel license plate location method based on wavelet transform and EMD analysis. Pattern Recogn 48:114–125. https://doi.org/10.1016/j.patcog.2014.07.027

    Article  Google Scholar 

  48. Zheng L, He X, Samali B, Yang LT (2013) An algorithm for accuracy enhancement of license plate recognition. J Comput Syst Sci 79:245–255. https://doi.org/10.1016/j.jcss.2012.05.006

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

We express our especial gratitude to Mr. Milad Noorani and Taraddod Rahnama Company, Kerman, Iran for their supports and helps in providing the database.

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Graduate University of Advanced Technology, P.O. Box 76315-117, Kerman, Iran

    Sajed Rakhshani & Esmat Rashedi

  2. Intelligent Data Processing Laboratory (IDPL), Department of Electrical Engineering, Shahid Bahonar University of Kerman, Kerman, Iran

    Hossein Nezamabadi-pour

Authors
  1. Sajed Rakhshani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Esmat Rashedi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hossein Nezamabadi-pour
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Esmat Rashedi.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rakhshani, S., Rashedi, E. & Nezamabadi-pour, H. Representation learning in a deep network for license plate recognition. Multimed Tools Appl 79, 13267–13289 (2020). https://doi.org/10.1007/s11042-019-08416-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08416-0

Keywords

Search

Navigation