Skip to main content
SpringerLink
Log in

Chinese character style transfer based on multi-scale GAN

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Chinese calligraphy has a strong artistry and appreciation. Through deep learning methods such as image translation methods, Chinese characters lacking in a set of calligraphy fonts can be quickly generated. Because of the clear structure of Chinese characters, the quality of the generated images is required to be high. However, it is difficult for a single generator to generate Chinese characters with clear texture, so we propose a multi-scale generative adversarial network (GAN), which contains two sub-GANs. The small-scale GAN generates low-resolution Chinese character outline, while the large-scale GAN supplements the details of the characters based on it. In addition to quantitatively evaluating the generated characters, we pre-train a vgg-19 network to extract stylistic characteristics of Chinese characters and compare the feature discrepancy between generated and real Chinese characters. From the experiment, using our method, the definition of the generated Chinese characters has indeed improved.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Goodfellow, J.I., Pouget-Abadie, J., Mirza, M., et al.: Generative adversarial networks. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2018)

  2. Baluja, S.: Learning typographic style: from discrimination to synthesis. Mach. Vis. Appl. 28(5), 551–568 (2017)

    Article  Google Scholar 

  3. Yang, S., Liu, J., Lian, Z., et al.: Awesome typography: statistics-based text effects transfer. In: Procedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2886–2895 (2017)

  4. Xu, S., Jin, T., Jiang, H., et al.: Automatic generation of personal Chinese handwriting by capturing the characteristics of personal handwriting. In: Procedings of the Innovative Applications of Artificial Intelligence Conference, pp. 191–196 (2009)

  5. Lian, Z., Xiao, J.: Automatic shape morphing for Chinese characters. In: Procedings of the SIGGRAPH Asia 2012 Technical Briefs, pp. 1–4 (2012)

  6. Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: Procedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2414–2423 (2016)

  7. Atarsaikhan, G., Iwana, B.K., Narusawa, A., et al.: Neural font style transfer. In: Procedings of the 2017 14th IAPR International Conference on Document Analysis and Recognition, pp. 51–56 (2017)

  8. Johnson, J., Alahi, A., Li, F.: Perceptual losses for real-time style transfer and super-resolution. In: Procedings of the European Conference on Computer Vision, pp. 694–711 (2016)

  9. Dosovitskiy, A., Brox, T.: Generating images with perceptual similarity metrics based on deep networks. In: Advances in neural information processing systems, pp. 658–666 (2016)

  10. Lyu, P., Bai, X., Yao, C., et al.: Auto-encoder guided GAN for Chinese calligraphy synthesis. In: Procedings of the 2017 14th IAPR International Conference on Document Analysis and Recognition, pp. 1095–1100 (2017)

  11. Isola, P., Zhu, J., Zhou, T., et al: Image-to-image translation with conditional adversarial networks. In: Procedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5967–5976(2017)

  12. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. Méd Image Comput. Comput. Assist. Interv. Soc. 9351, 234–241 (2015)

    Google Scholar 

  13. Qin, M., Chen, X.: Restore the incomplete calligraphy based on style transfer. In: Proceedings of 2019 Chinese Control Conference, pp. 8812–8817 (2019)

  14. Huang, G., Liu, Z., Maaten, L., et al.: Densely connected convolutional networks. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2261–2269 (2017)

  15. Tian, Y.: Zi2zi: master Chinese calligraphy with conditional adversarial networks. https://github.com/kaonashi-tyc/zi2zi. (2017)

  16. Zhou, X., Zhang, Z., Chen, X., et al.: Chinese calligraphy character generating via CGAN with a multi subnet parallel and cascade generator. In: Proceedings of the 39th Chinese Control Conference, pp. 7446–7451 (2020)

  17. Chang, B., Zhang, Q., Pan, S., et al.: Generating handwritten Chinese characters using cycleGAN. In: Procedings of the IEEE Winter Conference on Applications of Computer Vision, pp. 199–207 (2018)

  18. Zhu, J., Park, T., Isola, P., et al.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Procedings of the IEEE International Conference on Computer Vision, pp. 2242–2251 (2017)

  19. Zhang, J., Chen, D., Han, G., et al.: SSNet: structure-semantic net for Chinese typography generation based on image translation. Neurocomputing 371, 15–26 (2020)

    Article  Google Scholar 

  20. Mirza, M., Osindero, S.: Conditional generative adversarial nets. arXiv: 1411.1784 (2014)

  21. Wang, T., Liu, M., Zhu, J., et al.: High-resolution image synthesis and semantic manipulation with conditional GANs. In: Procedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8798–8807 (2018)

  22. Li, Y., Wang, N., Liu, J., et al: Demystifying neural style transfer. In: Proceedings of the international joint conference on artificial intelligence, pp. 2230–2236 (2017)

  23. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (2014)

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China under Grants 61873248, the Hubei Provincial Natural Science Foundation of China under Grant 2017CFA030, and the 111 project under Grant B17040.

Author information

Authors and Affiliations

  1. School of Arts and Communication, China University of Geosciences, Wuhan, 430074, China

    Ziying Zhang

  2. School of Automation, China University of Geosciences, Wuhan, 430074, China

    Xiaoxue Zhou, Mengxi Qin & Xin Chen

  3. Hubei Key Laboratory of Advanced Control and Intelligent Automation for Complex Systems, Wuhan, 430074, China

    Xiaoxue Zhou, Mengxi Qin & Xin Chen

  4. Engineering Research Center of Intelligent Technology for Geo-Exploration, Ministry of Education, Wuhan, 430074, China

    Xiaoxue Zhou, Mengxi Qin & Xin Chen

Authors
  1. Ziying Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoxue Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mengxi Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xin Chen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Z., Zhou, X., Qin, M. et al. Chinese character style transfer based on multi-scale GAN. SIViP 16, 559–567 (2022). https://doi.org/10.1007/s11760-021-02000-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-021-02000-6

Keywords

Search

Navigation