Font Recognition in Natural Images via Transfer Learning

  • Yizhi Wang
  • Zhouhui LianEmail author
  • Yingmin Tang
  • Jianguo Xiao
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10704)


Font recognition is an important and challenging problem in areas of Document Analysis, Pattern Recognition and Computer Vision. In this paper, we try to handle a tougher task that aims to accurately recognize the font styles of texts in natural images by proposing a novel method based on deep learning and transfer learning. Major contributions of this paper are threefold: First, we develop a fast and scalable system to synthesize huge amounts of natural images containing texts in various fonts and styles, which are then utilized to train the deep neural network for font recognition. Second, we design a transfer learning scheme to alleviate the domain mismatch between synthetic and real-world text images. Thus, large numbers of unlabeled text images can be adopted to markedly enhance the discrimination and robustness of our font classifier. Third, we build a benchmarking database which consists of numerous labeled natural images containing Chinese characters in 48 fonts. As far as we know, it is the first publicly-available dataset for font recognition of Chinese characters in natural images.



This work was supported by National Natural Science Foundation of China (Grant No.: 61472015, 61672043 and 61672056), Beijing Natural Science Foundation (Grant No.: 4152022), National Language Committee of China (Grant No.: ZDI135-9), and Key Laboratory of Science, Technology and Standard in Press Industry (Key Laboratory of Intelligent Press Media Technology).


  1. 1.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  2. 2.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  3. 3.
    Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587 (2014)Google Scholar
  4. 4.
    Girshick, R.: Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1440–1448 (2015)Google Scholar
  5. 5.
    Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)Google Scholar
  6. 6.
    Kimura, F., Takashina, K., Tsuruoka, S., Miyake, Y.: Modified quadratic discriminant functions and the application to Chinese character recognition. IEEE Trans. Pattern Anal. Mach. Intell. 1, 149–153 (1987)CrossRefGoogle Scholar
  7. 7.
    Ding, X., Chen, L., Wu, T.: Character independent font recognition on a single Chinese character. IEEE Trans. Pattern Anal. Mach. Intell. 29(2), 195–204 (2007)CrossRefGoogle Scholar
  8. 8.
    Moussa, S.B., Zahour, A., Benabdelhafid, A., Alimi, A.M.: New features using fractal multi-dimensions for generalized Arabic font recognition. Pattern Recogn. Lett. 31(5), 361–371 (2010)CrossRefGoogle Scholar
  9. 9.
    Slimane, F., Kanoun, S., Hennebert, J., Alimi, A.M., Ingold, R.: A study on font-family and font-size recognition applied to Arabic word images at ultra-low resolution. Pattern Recogn. Lett. 34(2), 209–218 (2013)CrossRefGoogle Scholar
  10. 10.
    Tao, D., Jin, L., Zhang, S., Yang, Z., Wang, Y.: Sparse discriminative information preservation for Chinese character font categorization. Neurocomputing 129, 159–167 (2014)CrossRefGoogle Scholar
  11. 11.
    Chen, G., Yang, J., Jin, H., Brandt, J., Shechtman, E., Agarwala, A., Han, T.X.: Large-scale visual font recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3598–3605 (2014)Google Scholar
  12. 12.
    Song, W., Lian, Z., Tang, Y., Xiao, J.: Content-independent font recognition on a single Chinese character using sparse representation. In: 2015 13th International Conference on Document Analysis and Recognition (ICDAR), pp. 376–380. IEEE (2015)Google Scholar
  13. 13.
    Wang, Z., Yang, J., Jin, H., Shechtman, E., Agarwala, A., Brandt, J., Huang, T.S.: Deepfont: identify your font from an image. In: Proceedings of the 23rd ACM International Conference on Multimedia, pp. 451–459. ACM (2015)Google Scholar
  14. 14.
    Tao, D., Lin, X., Jin, L., Li, X.: Principal component 2-D long short-term memory for font recognition on single Chinese characters. IEEE Trans. Cybern. 46(3), 756–765 (2016)CrossRefGoogle Scholar
  15. 15.
    Gupta, A., Vedaldi, A., Zisserman, A.: Synthetic data for text localisation in natural images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2315–2324 (2016)Google Scholar
  16. 16.
    Arbeláez, P., Pont-Tuset, J., Barron, J.T., Marques, F., Malik, J.: Multiscale combinatorial grouping. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 328–335 (2014)Google Scholar
  17. 17.
    Liu, F., Shen, C., Lin, G.: Deep convolutional neural fields for depth estimation from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5162–5170 (2015)Google Scholar
  18. 18.
    Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. ACM Trans. Graph. (TOG) 22, 313–318 (2003). ACMGoogle Scholar
  19. 19.
    Jaderberg, M., Simonyan, K., Vedaldi, A., Zisserman, A.: Reading text in the wild with convolutional neural networks. Int. J. Comput. Vis. 116(1), 1–20 (2016)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Tian, Z., Huang, W., He, T., He, P., Qiao, Y.: Detecting text in natural image with connectionist text proposal network. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 56–72. Springer, Cham (2016). CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Yizhi Wang
    • 1
  • Zhouhui Lian
    • 1
    Email author
  • Yingmin Tang
    • 1
  • Jianguo Xiao
    • 1
  1. 1.Institute of Computer Science and TechnologyPeking UniversityBeijingPeople’s Republic of China

Personalised recommendations