Adaptively Transfer Category-Classifier for Handwritten Chinese Character Recognition

  • Yongchun Zhu
  • Fuzhen ZhuangEmail author
  • Jingyuan Yang
  • Xi Yang
  • Qing He
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11439)


Handwritten character recognition (HCR) plays an important role in real-world applications, such as bank check recognition, automatic sorting of postal mail, the digitization of old documents, intelligence education and so on. Last decades have witnessed the vast amount of interest and research on handwritten character recognition, especially in the competition of HCR tasks on the specific data sets. However, the HCR task in real-world applications is much more complex than the one in HCR competition, since everyone has their own handwriting style, e.g., the HCR task on middle school students is much harder than the one on adults. Therefore, state-of-the-art methods proposed by the competitors may fail. Moreover, there is not enough labeled data to train a good model, since manually labelling data is usually tedious and expensive. So one question arises, is it possible to transfer the knowledge from related domain data to train a good recognition model for the target domain, e.g., from the handwritten character data of adults to the one of students? To this end, we propose a new neural network structure for handwritten Chinese character recognition (HCCR), in which we try to make full use of a large amount of labeled source domain data and a small number of target domain data to learn the model parameters. Furthermore, we make a transfer on the category-classifier level, and adaptively assign different weights to category-classifiers according to the usefulness of source domain data. Finally, experiments constructed from three data sets demonstrate the effectiveness of our model compared with several state-of-the-art baselines.



The research work is supported by the National Key Research and Development Program of China under Grant No. 2018YFB1004300, the National Natural Science Foundation of China under Grant Nos. U1836206, U1811461, 61773361, the Project of Youth Innovation Promotion Association CAS under Grant No. 2017146.


  1. 1.
    Chen, L., Wang, S., Fan, W., Sun, J., Naoi, S.: Beyond human recognition: a CNN-based framework for handwritten character recognition. In: ACPR, pp. 695–699 (2015)Google Scholar
  2. 2.
    Ciresan, D., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: CVPR, pp. 3642–3649 (2012)Google Scholar
  3. 3.
    Cireşan, D.C., Meier, U., Schmidhuber, J.: Transfer learning for Latin and Chinese characters with deep neural networks. In: IJCNN, pp. 1–6 (2012)Google Scholar
  4. 4.
    Ciresan, D.C., Meier, U., Gambardella, L.M., Schmidhuber, J.: Convolutional neural network committees for handwritten character classification. In: ICDAR, pp. 1135–1139 (2011)Google Scholar
  5. 5.
    Dai, W., Xue, G.R., Yang, Q., Yu, Y.: Co-clustering based classification for out-of-domain documents. In: ACM SIGKDD, pp. 210–219 (2007)Google Scholar
  6. 6.
    Gao, J., Fan, W., Jiang, J., Han, J.: Knowledge transfer via multiple model local structure mapping. In: SIGKDD, pp. 283–291 (2008)Google Scholar
  7. 7.
    Grosicki, E., El-Abed, H.: ICDAR 2011-French handwriting recognition competition. In: ICDAR, pp. 1459–1463 (2011)Google Scholar
  8. 8.
    Jiang, J., Zhai, C.: Instance weighting for domain adaptation in NLP. In: ACL, pp. 264–271 (2007)Google Scholar
  9. 9.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS, pp. 1097–1105 (2012)Google Scholar
  10. 10.
    LeCun, Y., et al.: Handwritten digit recognition with a back-propagation network. In: NIPS, pp. 396–404 (1990)Google Scholar
  11. 11.
    Lin, Z., Wan, L.: Style-preserving English handwriting synthesis. Pattern Recognit. 40, 2097–2109 (2007)CrossRefGoogle Scholar
  12. 12.
    Liu, C.L., Jaeger, S., Nakagawa, M.: ‘Online recognition of Chinese characters: the state-of-the-art. IEEE TPAMI 26, 198–213 (2004)CrossRefGoogle Scholar
  13. 13.
    Liu, C.L., Yin, F., Wang, D.H., Wang, Q.F.: Online and offline handwritten Chinese character recognition: benchmarking on new databases. Pattern Recognit. 46, 155–162 (2013)CrossRefGoogle Scholar
  14. 14.
    Liu, C.L., Yin, F., Wang, Q.F., Wang, D.H.: ICDAR 2011 Chinese handwriting recognition competition. In: ICDAR (2011)Google Scholar
  15. 15.
    Pan, S.J., Tsang, I.W., Kwok, J.T., Yang, Q.: Domain adaptation via transfer component analysis. IEEE TNN 22, 199–210 (2011)Google Scholar
  16. 16.
    Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE TKDE 22, 1345–1359 (2010)Google Scholar
  17. 17.
    Shao, L., Zhu, F., Li, X.: Transfer learning for visual categorization: a survey. IEEE TNNLS 26, 1019–1034 (2015)MathSciNetGoogle Scholar
  18. 18.
    Wu, C., Fan, W., He, Y., Sun, J., Naoi, S.: Handwritten character recognition by alternately trained relaxation convolutional neural network. In: Proceedings of 14th ICFHR, pp. 291–296 (2014)Google Scholar
  19. 19.
    Yang, W., Jin, L., Tao, D., Xie, Z., Feng, Z.: DropSample: a new training method to enhance deep convolutional neural networks for large-scale unconstrained handwritten Chinese character recognition. Pattern Recognit. 58, 190–203 (2016)CrossRefGoogle Scholar
  20. 20.
    Yin, F., Wang, Q.F., Zhang, X.Y., Liu, C.L.: ICDAR 2013 Chinese handwriting recognition competition. In: ICDAR, pp. 1464–1470 (2013)Google Scholar
  21. 21.
    Zhang, H., Guo, J., Chen, G., Li, C.: HCL2000-a large-scale handwritten Chinese character database for handwritten character recognition. In: ICDAR, pp. 286–290 (2009)Google Scholar
  22. 22.
    Zhong, Z., Jin, L., Xie, Z.: High performance offline handwritten Chinese character recognition using googlenet and directional feature maps. In: Proceedings of 13th ICDAR, pp. 846–850 (2015)Google Scholar
  23. 23.
    Zhu, Y., et al.: Heterogeneous transfer learning for image classification. In: AAAI (2011)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Yongchun Zhu
    • 1
    • 2
  • Fuzhen Zhuang
    • 1
    • 2
    Email author
  • Jingyuan Yang
    • 3
  • Xi Yang
    • 4
  • Qing He
    • 1
    • 2
  1. 1.Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing TechnologyCASBeijingChina
  2. 2.University of Chinese Academy of SciencesBeijingChina
  3. 3.George Mason UniversityFairfaxUSA
  4. 4.Sunny Education Inc.BeijingChina

Personalised recommendations