A high-performance CNN method for offline handwritten Chinese character recognition and visualization

  • Pavlo Melnyk
  • Zhiqiang YouEmail author
  • Keqin Li


Recent researches introduced fast, compact and efficient convolutional neural networks (CNNs) for offline handwritten Chinese character recognition (HCCR). However, many of them did not address the problem of network interpretability. We propose a new architecture of a deep CNN with high recognition performance which is capable of learning deep features for visualization. A special characteristic of our model is the bottleneck layers which enable us to retain its expressiveness while reducing the number of multiply-accumulate operations and the required storage. We introduce a modification of global weighted average pooling (GWAP)—global weighted output average pooling (GWOAP). This paper demonstrates how they allow us to calculate class activation maps (CAMs) in order to indicate the most relevant input character image regions used by our CNN to identify a certain class. Evaluating on the ICDAR-2013 offline HCCR competition dataset, we show that our model enables a relative 0.83% error reduction while having 49% fewer parameters and the same computational cost compared to the current state-of-the-art single-network method trained only on handwritten data. Our solution outperforms even recent residual learning approaches.


Handwritten Chinese character recognition Convolutional neural network Global average pooling Class activation maps 



This work is supported by National Natural Science Foundation of China under Grant No. 61472123 and Hunan Provincial Natural Science Foundation under Grant No. 2018JJ2064. We would like to express our gratitude to the China Scholarship Council for giving the first author an opportunity to obtain master’s degree at Hunan University under Chinese Government Scholarship.

Compliance with ethical standards

Conflict of interest

All authors declare that they have no conflict of interest regarding the publication of this paper.


  1. Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M et al (2016) Tensorflow: a system for large-scale machine learning. OSDI 16:265–283Google Scholar
  2. Al-Janabi S (2018) Smart system to create an optimal higher education environment using ida and iots. Int J Comput Appl. Google Scholar
  3. Al-Janabi S, Abaid Mahdi M (2019) Evaluation prediction techniques to achievement an optimal biomedical analysis. Int J Grid Utility Comput. Google Scholar
  4. Al-Janabi S, Alkaim AF (2019) A nifty collaborative analysis to predicting a novel tool (DRFLLS) for missing values estimation. Soft Comput 1–15Google Scholar
  5. Al-Janabi S, Salman M, Fanfakh A (2018a) Recommendation system to improve time management for people in education environments. J Eng Appl Sci 13:10182–10193Google Scholar
  6. Al-Janabi S, Salman MA, Mohammad M (2018b) Multi-level network construction based on intelligent big data analysis. In: International conference on bigdata and smart digital environment. Springer, pp 102–118Google Scholar
  7. Ali SH (2012) A novel tool (FP-KC) for handle the three main dimensions reduction and association rule mining. In: 2012 6th International conference on sciences of electronics. Technologies of information and telecommunications (SETIT). IEEE, pp 951–961Google Scholar
  8. Arqub OA, Mohammed AS, Momani S, Hayat T (2016) Numerical solutions of fuzzy differential equations using reproducing kernel Hilbert space method. Soft Comput 20(8):3283–3302CrossRefzbMATHGoogle Scholar
  9. Arqub OA, Al-Smadi M, Momani S, Hayat T (2017) Application of reproducing kernel algorithm for solving second-order, two-point fuzzy boundary value problems. Soft Comput 21(23):7191–7206CrossRefzbMATHGoogle Scholar
  10. Chen TQ, Rubanova Y, Bettencourt J, Duvenaud DK (2018) Neural ordinary differential equations. In: Advances in neural information processing systems, pp 6572–6583Google Scholar
  11. Cheng C, Zhang XY, Shao XH, Zhou XD (2016) Handwritten Chinese character recognition by joint classification and similarity ranking. In: 2016 15th International conference on frontiers in handwriting recognition (ICFHR). IEEE, pp 507–511Google Scholar
  12. Chollet F et al (2015) Keras.
  13. Cireşan D, Meier U, Schmidhuber J (2012) Multi-column deep neural networks for image classification. arXiv preprint arXiv:1202.2745
  14. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp 1026–1034Google Scholar
  15. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778Google Scholar
  16. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167
  17. Kenan K, Ali SH, Patel A (2015) Rapid lossless compression of short text messages. Comput Stand Interfaces 37:53–59.
  18. Kimura F, Takashina K, Tsuruoka S, Miyake Y (1987) Modified quadratic discriminant functions and the application to Chinese character recognition. IEEE Trans Pattern Anal Mach Intell 1:149–153CrossRefGoogle Scholar
  19. Li F, Shen Q, Li Y, Mac Parthaláin N (2016) Handwritten Chinese character recognition using fuzzy image alignment. Soft Comput 20(8):2939–2949CrossRefGoogle Scholar
  20. Li Z, Teng N, Jin M, Lu H (2018) Building efficient CNN architecture for offline handwritten Chinese character recognition. Int J Doc Anal Recognit (IJDAR) 21(4):233–240CrossRefGoogle Scholar
  21. Lin M, Chen Q, Yan S (2013) Network in network. arXiv preprint arXiv:1312.4400
  22. Liu CL, Yin F, Wang DH, Wang QF (2011) CASIA online and offline Chinese handwriting databases. In: 2011 International conference on document analysis and recognition (ICDAR). IEEE, pp 37–41Google Scholar
  23. Liu CL, Yin F, Wang DH, Wang QF (2013) Online and offline handwritten Chinese character recognition: benchmarking on new databases. Pattern Recognit 46(1):155–162CrossRefGoogle Scholar
  24. Lu S, Wei X, Lu Y (2015) Cost-sensitive MQDF classifier for handwritten Chinese address recognition. In: 2015 13th International conference on document analysis and recognition (ICDAR). IEEE, pp 76–80Google Scholar
  25. Qin Z, Yu F, Liu C, Chen X (2018) How convolutional neural networks see the world—a survey of convolutional neural network visualization methods. Math Found Comput 1(2):149–180CrossRefGoogle Scholar
  26. Saravanan B, Mohanraj V, Senthilkumar J (2019) A fuzzy entropy technique for dimensionality reduction in recommender systems using deep learning. Soft Comput 23(8):2575–2583CrossRefGoogle Scholar
  27. Silver D, Huang A, Maddison CJ, Guez A, Sifre L, van den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M, Dieleman S, Grewe D, Nham J, Kalchbrenner N, Sutskever I, Lillicrap T, Leach M, Kavukcuoglu K, Graepel T, Hassabis D (2016) Mastering the game of Go with deep neural networks and tree search. Nature 529:484–503CrossRefGoogle Scholar
  28. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958MathSciNetzbMATHGoogle Scholar
  29. Xiao X, Jin L, Yang Y, Yang W, Sun J, Chang T (2017) Building fast and compact convolutional neural networks for offline handwritten Chinese character recognition. Pattern Recognit 72:72–81CrossRefGoogle Scholar
  30. Yang X, He D, Zhou Z, Kifer D, Giles CL (2017) Improving offline handwritten Chinese character recognition by iterative refinement. In: 2017 14th IAPR international conference on document analysis and recognition (ICDAR). IEEE, pp 5–10Google Scholar
  31. Yin F, Wang QF, Zhang XY, Liu CL (2013) ICDAR 2013 Chinese handwriting recognition competition. In: 2013 12th International conference on document analysis and recognition (ICDAR). IEEE, pp 1464–1470Google Scholar
  32. Zhang Y (2015) Deep convolutional network for handwritten Chinese character recognition. Computer Science Department, Stanford UniversityGoogle Scholar
  33. Zhang XY, Bengio Y, Liu CL (2017) Online and offline handwritten Chinese character recognition: a comprehensive study and new benchmark. Pattern Recognit 61:348–360CrossRefGoogle Scholar
  34. Zhang Y, Liang S, Nie S, Liu W, Peng S (2018) Robust offline handwritten character recognition through exploring writer-independent features under the guidance of printed data. Pattern Recognit Lett 106:20–26CrossRefGoogle Scholar
  35. Zhong Z, Jin L, Xie Z (2015) High performance offline handwritten Chinese character recognition using googlenet and directional feature maps. In: 2015 13th International conference on document analysis and recognition (ICDAR). IEEE, pp 846–850Google Scholar
  36. Zhong Z, Zhang XY, Yin F, Liu CL (2016) Handwritten Chinese character recognition with spatial transformer and deep residual networks. In: 2016 23rd International conference on pattern recognition (ICPR). IEEE, pp 3440–3445Google Scholar
  37. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2921–2929Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Key Laboratory for Embedded and Network Computing of Hunan ProvinceHunan UniversityChangshaPeople’s Republic of China
  2. 2.College of Computer Science and Electronic EngineeringHunan UniversityChangshaPeople’s Republic of China
  3. 3.Department of Computer ScienceState University of New York at New PaltzNew PaltzUSA

Personalised recommendations