The Visual Computer

, Volume 32, Issue 6–8, pp 717–727 | Cite as

3D cartoon face generation by local deformation mapping

Original Article
  • 410 Downloads

Abstract

We present a data-driven method for automatically generating a 3D cartoon of a real 3D face. Given a sparse set of 3D real faces and their corresponding cartoon faces modeled by an artist, our method models the face in each subspace as the deformation of its nearby exemplars and learn a mapping between the deformations defined by the real faces and their cartoon counterparts. To reduce the exemplars needed for learning, we regress a collection of linear mappings defined locally in both face geometry and identity spaces and develop a progressive scheme for users to gradually add new exemplars for training. At runtime, our method first finds the nearby exemplars of an input real face and then constructs the result cartoon face from the corresponding cartoon faces of the nearby real face exemplars and the local deformations mapped from the real face subspace. Our method greatly simplifies the cartoon generation process by learning artistic styles from a sparse set of exemplars. We validate the efficiency and effectiveness of our method by applying it to faces of different facial features. Results demonstrate that our method not only preserves the artistic style of the exemplars, but also keeps the unique facial geometric features of different identities.

Keywords

3D cartoon generation Local deformation mapping Data-driven method 

Supplementary material

371_2016_1265_MOESM1_ESM.pdf (5 mb)
Supplementary material 1 (pdf 5145 KB)

References

  1. 1.
    Akleman, E.: Making caricatures with morphing. In: ACM SIGGRAPH 97 Visual Proceedings: the art and interdisciplinary programs of SIGGRAPH’97, p. 145. ACM (1997)Google Scholar
  2. 2.
    Akleman, E., Reisch, J.: Modeling expressive 3d caricatures. In: ACM SIGGRAPH 2004 Sketches, p. 61. ACM (2004)Google Scholar
  3. 3.
    Baran, I., Vlasic, D., Grinspun, E., Popović, J.: Semantic deformation transfer. In: ACM Transactions on Graphics (TOG), vol. 28, p. 36. ACM (2009)Google Scholar
  4. 4.
    Bradley, D., Heidrich, W., Popa, T., Sheffer, A.: High resolution passive facial performance capture. ACM Trans. Gr. (Proc. SIGGRAPH) 29(3) (2010)Google Scholar
  5. 5.
    Cao, C., Weng, Y., Zhou, S., Tong, Y., Zhou, K.: Facewarehouse: A 3d facial expression database for visual computing. Vis. Comput. Gr. IEEE Trans. 20(3), 413–425 (2014)CrossRefGoogle Scholar
  6. 6.
    Chen, H., Xu, Y.Q., Shum, H.Y., Zhu, S.C., Zheng, N.N.: Example-based facial sketch generation with non-parametric sampling. In: Computer Vision, 2001. ICCV 2001. Proceedings. Eighth IEEE International Conference on, vol. 2, pp. 433–438. IEEE (2001)Google Scholar
  7. 7.
    Chen, Y.L., Liao, W.H., Chiang, P.Y.: Generation of 3d caricature by fusing caricature images. In: Systems, Man and Cybernetics. SMC’06. IEEE International Conference on, vol. 1, pp. 866–871. IEEE (2006)Google Scholar
  8. 8.
    Clarke, L., Chen, M., Mora, B.: Automatic generation of 3d caricatures based on artistic deformation styles. Vis. Comput. Gr. IEEE Trans. 17(6), 808–821 (2011)CrossRefGoogle Scholar
  9. 9.
    Fujiwara, T., Koshimizu, H., Fujimura, K., Kihara, H., Noguchi, Y., Ishikawa, N.: 3d modeling system of human face and full 3d facial caricaturing. In: 3-D Digital Imaging and Modeling. Proceedings. Third International Conference on, pp. 385–392. IEEE (2001)Google Scholar
  10. 10.
    Fujiwara, T., Nishihara, T., Tominaga, M., Kato, K., Murakami, K., Koshirnizu, H.: On the detection of feature points of 3d facial image and its application to 3d facial caricature. In: 3-D Digital Imaging and Modeling, 1999. Proceedings. Second International Conference on, pp. 490–496. IEEE (1999)Google Scholar
  11. 11.
    Kamada, T., Kawai, S.: An algorithm for drawing general undirected graphs. Inf. Process. Lett. 31(1), 7–15 (1989)MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Kondo, T., Murakami, K., Koshimizu, H.: From coarse to fine correspondence of 3-d facial images and its application to 3-d facial caricaturing. In: 3-D Digital Imaging and Modeling. Proceedings. International Conference on Recent Advances in, pp. 283–288. IEEE (1997)Google Scholar
  13. 13.
    Li, P., Chen, Y., Liu, J., Fu, G.: 3d caricature generation by manifold learning. In: Multimedia and Expo. IEEE International Conference on, pp. 941–944. IEEE (2008)Google Scholar
  14. 14.
    Liang, L., Chen, H., Xu, Y.Q., Shum, H.Y.: Example-based caricature generation with exaggeration. In: Computer Graphics and Applications. Proceedings. 10th Pacific Conference on, pp. 386–393. IEEE (2002)Google Scholar
  15. 15.
    Liu, J., Chen, Y., Miao, C., Xie, J., Ling, C.X., Gao, X., Gao, W.: Semi-supervised learning in reconstructed manifold space for 3d caricature generation. In: Computer Graphics Forum, vol. 28, pp. 2104–2116. Wiley Online Library (2009)Google Scholar
  16. 16.
    Liu, S., Wang, J., Zhang, M., Wang, Z.: Three-dimensional cartoon facial animation based on art rules. Vis. Comput. 29(11), 1135–1149 (2013)CrossRefGoogle Scholar
  17. 17.
    Mingming, Z., Shoukuai, L., Jiajun, W., Huaqing, S., Zhigeng, P.: The 3d caricature face modeling based on aesthetic formulae. In: Proceedings of the 9th ACM SIGGRAPH Conference on Virtual-Reality Continuum and its Applications in Industry, pp. 191–198. ACM (2010)Google Scholar
  18. 18.
    Mo, Z., Lewis, J., Neumann, U.: Improved automatic caricature by feature normalization and exaggeration. In: ACM SIGGRAPH 2004 Sketches, p. 57. ACM (2004)Google Scholar
  19. 19.
    Sadimon, S.B., Sunar, M.S., Mohamad, D., Haron, H.: Computer generated caricature: a survey. In: Cyberworlds (CW), 2010 International Conference on, pp. 383–390. IEEE (2010)Google Scholar
  20. 20.
    Sucontphunt, T.: 3d artistic face transformation with identity preservation. In: Smart Graphics, pp. 154–165. Springer, New York (2014)Google Scholar
  21. 21.
    Sucontphunt, T.: A practical approach for identity-embodied 3d artistic face modeling. Int. J. Comput. Games Technol. 2014, 7 (2014)CrossRefGoogle Scholar
  22. 22.
    Sumner, R.W., Popović, J.: Deformation transfer for triangle meshes. In: ACM Transactions on Graphics (TOG), vol. 23, pp. 399–405. ACM (2004)Google Scholar
  23. 23.
    Wang, S.F., Lai, S.H.: Manifold-based 3d face caricature generation with individualized facial feature extraction. In: Computer Graphics Forum, vol. 29, pp. 2161–2168. Wiley Online Library (2010)Google Scholar
  24. 24.
    Xie, J., Chen, Y., Liu, J., Miao, C., Gao, X.: Interactive 3d caricature generation based on double sampling. In: Proceedings of the 17th ACM international conference on Multimedia, pp. 745–748. ACM (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Jingyong Zhou
    • 1
  • Xin Tong
    • 2
  • Zicheng Liu
    • 3
  • Baining Guo
    • 1
    • 2
  1. 1.Tsinghua UniversityBeijingChina
  2. 2.Microsoft ResearchBeijingChina
  3. 3.Microsoft ResearchRedmondUSA

Personalised recommendations