The Visual Computer

, Volume 35, Issue 6–8, pp 1193–1205 | Cite as

Intelligent Chinese calligraphy beautification from handwritten characters for robotic writing

  • Xinyue Zhang
  • Yuanhao Li
  • Zhiyi Zhang
  • Kouichi Konno
  • Shaojun HuEmail author
Original Article


Chinese calligraphy is the artistic expression of character writing and is highly valued in East Asia. However, it is a challenge for non-expert users to write visually pleasing calligraphy with his or her own unique style. In this paper, we develop an intelligent system that beautifies Chinese handwriting characters and physically writes them in a certain calligraphy style using a robotic arm. First, we sketch the handwriting characters using a mouse or a touch pad. Then, we employ a convolutional neural network to identify each stroke from the skeletons, and the corresponding standard stroke is retrieved from a pre-built calligraphy stroke library for robotic arm writing. To output aesthetically beautiful calligraphy with the user’s style, we propose a global optimization approach to solve the minimization problem between the handwritten strokes and standard calligraphy strokes, in which a shape character vector is presented to describe the shape of standard strokes. Unlike existing systems that focus on the generation of digital calligraphy from handwritten characters, our system has the advantage of converting the user-input handwriting into physical calligraphy written by a robotic arm. We take the regular script (Kai) style as an example and perform a user study to evaluate the effectiveness of the system. The writing results show that our system can achieve visually pleasing calligraphy from various input handwriting while retaining the user’s style.


Calligraphy Beautification Robotic arm Optimization Handwritten characters 



We thank the CGI2019 reviewers for their thoughtful comments. The work is supported by the NSFC (61303124), NSBR Plan of Shaanxi (2019JM370) and the Fundamental Research Funds for the Central Universities (2452017343).

Compliance with ethical standards

Conflict of interest

All authors declare that they have no conflict of interest.

Supplementary material

371_2019_1675_MOESM1_ESM.mp4 (22 mb)
Supplementary material 1 (mp4 22575 KB)


  1. 1.
    Chao, F., Lv, J., Zhou, D., Yang, L., Lin, C., Shang, C., Zhou, C.: Generative adversarial nets in robotic Chinese calligraphy. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 1104–1110 (2018)Google Scholar
  2. 2.
    Chen, X., Lian, Z., Tang, Y., Xiao, J.: A benchmark for stroke extraction of Chinese character. Acta Sci. Nat. Univ. Pekin. 52(1), 49–57 (2016)MathSciNetGoogle Scholar
  3. 3.
    Dobot Robotic Arms: Accessed on Feb 10, 2019
  4. 4.
    Furrer, F., Wermelinger, M., Yoshida, H., Gramazio, F., Kohler, M., Siegwart, R., Hutter, M.: Autonomous robotic stone stacking with online next best object target pose planning. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 2350–2356 (2017)Google Scholar
  5. 5.
    Gupta, A., Srivastava, M., Mahanta, C.: Offline handwritten character recognition using neural network. In: 2011 IEEE International Conference on Computer Applications and Industrial Electronics (ICCAIE), pp. 102–107 (2011)Google Scholar
  6. 6.
    Hashiguchi, H., Arimoto, S., Ozawa, R.: Control of a handwriting robot with DOF-redundancy based on feedback in task-coordinates. J. Robot. Mechatron. 16, 381–387 (2004)CrossRefGoogle Scholar
  7. 7.
    Igarashi, T., Inami, M.: Exploration of alternative interaction techniques for robotic systems. IEEE Comput. Graph. Appl. 35(3), 33–41 (2015)CrossRefGoogle Scholar
  8. 8.
    Igarashi, T., Matsuoka, S., Tanaka, H.: Teddy: A sketching interface for 3D freeform design. In: SIGGRAPH’99, pp. 409–416, New York (1999)Google Scholar
  9. 9.
    Kim, B., Wang, O., Öztireli, A.C., Gross, M.: Semantic segmentation for line drawing vectorization using neural networks. Comput. Graph. Forum 37(2), 329–338 (2018)CrossRefGoogle Scholar
  10. 10.
    Kim, S., Jo, J., Oh, Y., Oh, S., Srinivasa, S., Likhachev, M.: Robotic handwriting: multi-contact manipulation based on reactional internal contact hypothesis. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 877–884 (2014)Google Scholar
  11. 11.
    Krizek, M., Neittaanmki, P., Glowinski, R., Korotov, S.: Conjugate Gradient Algorithms and Finite Element Methods. Springer, Berlin (2004)CrossRefzbMATHGoogle Scholar
  12. 12.
    Li, H., Liu, P., Xu, S., Lin, S.: Calligraphy beautification method for Chinese handwritings. In: 2012 4th International Conference on Digital Home, pp. 122–127 (2012)Google Scholar
  13. 13.
    Liu, L., Xia, W., Jin, L., Mao, H., Tian, F.: A Kai style contour beautification method for Chinese handwriting characters. In: 2010 IEEE International Conference on Systems, Man and Cybernetics, pp. 3644–3649 (2010)Google Scholar
  14. 14.
    Lo, K.W., Kwok, K.W., Wong, S.M., Yam, Y.: Brush footprint acquisition and preliminary analysis for Chinese calligraphy using a robot drawing platform. In: 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5183–5188 (2006)Google Scholar
  15. 15.
    Ma, Z., Su, J.: Aesthetics evaluation for robotic Chinese calligraphy. IEEE Trans. Cogn. Dev. Syst. 9(1), 80–90 (2017)CrossRefGoogle Scholar
  16. 16.
    Mueller, S., Huebel, N., Waibel, M., D’Andrea, R.: Robotic calligraphy-learning how to write single strokes of Chinese and Japanese characters. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1734–1739 (2013)Google Scholar
  17. 17.
    Okutomi, M.: Digital Image Processing. The Computer Graphic Arts Society (CG-ARTS), Tokyo (2015)Google Scholar
  18. 18.
    Online Chinese Calligraphy Generator: Accessed on Dec. 4, 2018
  19. 19.
    Sun, Y., Ding, N., Qian, H., Xu, Y.: A robot for classifying Chinese calligraphic types and styles. In: 2013 IEEE International Conference on Robotics and Automation, pp. 4279–4284 (2013)Google Scholar
  20. 20.
    Sun, Y., Qian, H., Xu, Y.: A geometric approach to stroke extraction for the Chinese calligraphy robot. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 3207–3212 (2014)Google Scholar
  21. 21.
    Sun, Y., Qian, H., Xu, Y.: Robot learns Chinese calligraphy from demonstrations. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4408–4413. IEEE, New York (2014)Google Scholar
  22. 22.
    Syamlan, A.T., Nurhadi, H., Pramujati, B.: Character recognition for writing robot control using ANFIS. In: 2015 International Conference on Advanced Mechatronics, Intelligent Manufacture, and Industrial Automation (ICAMIMIA), pp. 46–48 (2015)Google Scholar
  23. 23.
    Tian, X., Tian, Y.: The Elaboration of Ancient and Modern Famous Kai Calligraphy. Jiangxi Fine Arts Press, Jiangxi (2007)Google Scholar
  24. 24.
    Xiao, X., Jin, L., Yang, Y., Yang, W., Sun, J., Chang, T.: Building fast and compact convolutional neural networks for offline handwritten Chinese character recognition. Pattern Recognit. 72, 72–81 (2017)CrossRefGoogle Scholar
  25. 25.
    Yao, F., Shao, G., Yi, J.: Trajectory generation of the writing-brush for a robot arm to inherit block-style Chinese character calligraphy techniques. Adv. Robot. 18(3), 331–356 (2004)CrossRefGoogle Scholar
  26. 26.
    Yi, T., Lian, Z., Tang, Y., Xiao, J.: A data-driven personalized digital ink for Chinese characters. In: Gurrin, C., Hopfgartner, F., Hurst, W., Johansen, H., Lee, H., O’Connor, N. (eds.) MultiMedia Modeling, pp. 254–265. Springer, New York (2014).
  27. 27.
    Zeng, H., Huang, Y., Chao, F., Zhou, C.: Survey of robotic calligraphy research. CAAI Trans. Intell. Syst. 11(1), 15–26 (2016)Google Scholar
  28. 28.
    Zhang, Z., Wu, J., Yu, K.: Chinese calligraphy specific style rendering system. In: Proceedings of the ACM International Conference on Digital Libraries, pp. 99–108 (2010)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  • Xinyue Zhang
    • 1
  • Yuanhao Li
    • 1
  • Zhiyi Zhang
    • 1
    • 2
  • Kouichi Konno
    • 3
  • Shaojun Hu
    • 1
    • 2
    Email author
  1. 1.College of Information EngineeringNorthwest A&F UniversityYanglingChina
  2. 2.Key Laboratory of Agricultural Internet of ThingsMinistry of AgricultureYanglingChina
  3. 3.Faculty of Science and EngineeringIwate UniversityMoriokaJapan

Personalised recommendations