Skip to main content
Log in

Multi-scale multi-class conditional generative adversarial network for handwritten character generation

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Handwritten character generation is a popular research topic with various applications. Various methods have been proposed in the literatures which are based on methods such as pattern recognition, machine learning, deep learning or others. However, seldom method could generate realistic and natural handwritten characters with a built-in determination mechanism to enhance the quality of generated image and make the observers unable to tell whether they are written by a person. To address these problems, in this paper, we proposed a novel generative adversarial network, multi-scale multi-class generative adversarial network (MSMC-CGAN). It is a neural network based on conditional generative adversarial network (CGAN), and it is designed for realistic multi-scale character generation. MSMC-CGAN combines the global and partial image information as condition, and the condition can also help us to generate multi-class handwritten characters. Our model is designed with unique neural network structures, image features and training method. To validate the performance of our model, we utilized it in Chinese handwriting generation, and an evaluation method called mean opinion score (MOS) was used. The MOS results show that MSMC-CGAN achieved good performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp 2672–2680

  2. Mirza M, Osindero S (2014) Conditional generative adversarial nets. Comput Sci 2672–2680. arXiv:1411.1784

  3. Lake BM, Salakhutdinov R, Tenenbaum JB (2015) Human-level concept learning through probabilistic program induction. Science 350(6266):1332–1338

    Article  MathSciNet  MATH  Google Scholar 

  4. Haines TSF, Mac Aodha O, Brostow GJ (2016) My text in your handwriting. ACM Trans Gr 35(3):26

    Article  Google Scholar 

  5. Xu S, Jin T, Jiang H, Lau FCM (2009) Automatic generation of personal chinese handwriting by capturing the characteristics of personal handwriting. In: Conference on Innovative Applications of Artificial Intelligence, Pasadena, California, USA. DBLP, 14–16 July 2009

  6. Lin JW, Hong CY, Chang R, Wang YC, Lin SY, Ho JM (2015) Complete font generation of Chinese characters in personal handwriting style. In: IEEE Computing and Communications Conference, pp 1–5

  7. Kuroiwa T, Shin J (2011) Discovery of efficient chinese characters for handwritten-style font generation. Int J Digit Content Technol Appl 5(12):1–10

    Article  Google Scholar 

  8. Xiao J, Xiao J, Xiao J (2016) Automatic generation of large-scale handwriting fonts via style learning. In: SIGGRAPH ASIA 2016 Technical Briefs, ACM, New York, p 12

  9. Myronenko A, Song X (2010) Point set registration: coherent point drift. IEEE Trans Pattern Anal Mach Intell 32(12):2262–2275

    Article  Google Scholar 

  10. Zhang XY, Yin F, Zhang YM, Liu CL, Bengio Y (2017) Drawing and recognizing chinese characters with recurrent neural network. In: IEEE Transactions on Pattern Analysis and Machine Intelligence

  11. Ha D, Eck D (2017) A neural representation of sketch drawings. arXiv preprint arXiv:1704.03477

  12. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp 1097–1105

  13. Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434

  14. Zeiler MD, Krishnan D, Taylor GW, Fergus R (2010) Deconvolutional networks. In: IEEE Computer Vision and Pattern Recognition, vol 238, pp 2528–2535

  15. Huang R, Zhang S, Li T, He R (2017) Beyond face rotation: global and local perception gan for photorealistic and identity preserving frontal view synthesis. arXiv preprint arXiv:1704.04086

  16. Reed S, Akata Z, Yan X, Logeswaran L, Schiele B, Lee H (2016) Generative adversarial text to image synthesis. arXiv preprint arXiv:1605.05396

  17. Wang X, Gupta A (2016, October) Generative image modeling using style and structure adversarial networks. In: European Conference on Computer Vision, Springer, Berlin, pp 318–335

  18. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, Shi W (2016) Photo-realistic single image super-resolution using a generative adversarial network. arXiv preprint arXiv:1609.04802

  19. Chidambaram M, Qi Y (2017) Style transfer generative adversarial networks: learning to play chess differently. arXiv preprint arXiv:1702.06762

  20. Jetchev N, Bergmann U, Vollgraf R (2016) Texture synthesis with spatial generative adversarial networks. arXiv preprint arXiv:1611.08207

  21. Li C, Xu K, Zhu J, Zhang B (2017) Triple generative adversarial nets. arXiv preprint arXiv:1703.02291

  22. Arjovsky M, Chintala S, Bottou L (2017) Wasserstein gan. arXiv preprint arXiv:1701.07875

  23. Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. arXiv preprint arXiv:1703.10593

  24. Antipov G, Baccouche M, Dugelay JL (2017) Face aging with conditional generative adversarial networks. arXiv preprint arXiv:1702.01983

  25. Zhang H, Sindagi, V, Patel VM (2017) Image de-raining using a conditional generative adversarial network. arXiv preprint arXiv:1701.05957

  26. Isola P, Zhu JY, Zhou T, Efros AA (2016) Image-to-image translation with conditional adversarial networks. arXiv preprint arXiv:1611.07004

  27. Denton E, Gross S, Fergus R (2016) Semi-supervised learning with context-conditional generative adversarial networks. arXiv preprint arXiv:1611.06430

  28. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp 448–456

  29. Zeiler MD, Krishnan D, Taylor GW, Fergus R (2010) Deconvolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010, pp 2528–2535

  30. Kingma D, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

  31. Nair V, Hinton GE (2010) Rectified linear units improve restricted Boltzmann machines. In: International Conference on International Conference on Machine Learning, Omnipress, Madison, pp 807–814

  32. Li Q, Xu Q, Xiao J, Liu Q, Zhang J (2017) A structure and style model for chinese character dynamic generation. Beijing Daxue Xuebao (Ziran Kexue Ban)/Acta Scientiarum Naturalium Universitatis Pekinensis, vol 53, pp 219–229. https://doi.org/10.13209/j.0479-8023.2017.048

  33. Wang Y, Che W, Xu B (2017) Encoder–decoder recurrent network model for interactive character animation generation. Vis Comput 33(6–8):971–980

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the State Oceanic Administration China research fund Project (201305026), the NSFC (61772454), and by the open research fund of the Key Lab of Broadband Wireless Communication and Sensor Network Technology (Nanjing University of Posts and Telecommunications), Ministry of Education. Prof. Jeong-Uk Kim is the corresponding author.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jeong-Uk Kim.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, J., Gu, C., Wang, J. et al. Multi-scale multi-class conditional generative adversarial network for handwritten character generation. J Supercomput 75, 1922–1940 (2019). https://doi.org/10.1007/s11227-017-2218-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-017-2218-0

Keywords

Navigation