Advertisement

3D cartoon face rigging from sparse examples

  • 337 Accesses

  • 1 Citations

Abstract

We present a data-driven method for automatically constructing cartoonized 3D blendshapes of a subject’s face. Given a pre-defined blendshape template of the real facial expressions and corresponding cartoonized blendshape template created by an artist, we represent the blendshapes of an identity in the real and cartoon face spaces with the deformations of the blendshape template in each space and learn a mapping between the deformations in the two spaces. To this end, our method decomposes the deformations in each space into two parts: an identity-independent part that is represented with the deformation gradient of the blendshape template, and an identity-dependent part that is modeled by a low-rank linear model. We regress the linear model for the real expressions from a 3D facial expression dataset. An algorithm is then introduced to regress the mapping between the linear models in the two spaces from a small set of real expressions and their cartoonized counterparts. At run time, given the blendshapes of a subject’s real face and her 3D cartoon neutral face, our method automatically constructs the cartoonized blendshapes of the subject with the help of the cartoonized blendshape template and the learned mapping. Our method is user-independent and only requires a small set of 3D cartoonized expressions modeled by the artist for cartoon face rigging. We evaluate our method by creating cartoonized 3D facial animations for variant identities in two different artistic styles. The rigging results demonstrate that our method successfully preserves both artistic styles and personalized expressions of different identities.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 199

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

  1. 1.

    Akleman, E.: Making caricatures with morphing. In: ACM SIGGRAPH 97 Visual Proceedings: The Art and Interdisciplinary Programs of SIGGRAPH’97, p. 145. ACM (1997)

  2. 2.

    Akleman, E., Reisch, J.: Modeling expressive 3D caricatures. In: ACM SIGGRAPH 2004 Sketches, p. 61. ACM (2004)

  3. 3.

    Alexander, O., Rogers, M., Lambeth, W., Chiang, M., Debevec, P.: The digital emily project: photoreal facial modeling and animation. In: Acm Siggraph 2009 Courses, p. 12. ACM (2009)

  4. 4.

    Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, pp. 187–194. ACM Press/Addison-Wesley Publishing Co. (1999)

  5. 5.

    Bouaziz, S., Wang, Y., Pauly, M.: Online modeling for realtime facial animation. ACM Trans. Graph. (TOG) 32(4), 40 (2013)

  6. 6.

    Cao, C., Bradley, D., Zhou, K., Beeler, T.: Real-time high-fidelity facial performance capture. ACM Trans. Graph. (ToG) 34(4), 46 (2015)

  7. 7.

    Cao, C., Hou, Q., Zhou, K.: Displaced dynamic expression regression for real-time facial tracking and animation. ACM Trans. Graph. (TOG) 33(4), 43 (2014)

  8. 8.

    Cao, C., Weng, Y., Lin, S., Zhou, K.: 3D shape regression for real-time facial animation. ACM Trans. Graph. (TOG) 32(4), 41 (2013)

  9. 9.

    Cao, C., Weng, Y., Zhou, S., Tong, Y., Zhou, K.: Facewarehouse: a 3D facial expression database for visual computing. IEEE Trans. Vis. Comput. Graph. 20(3), 413–425 (2014)

  10. 10.

    Casas, D., Feng, A., Alexander, O., Fyffe, G., Debevec, P., Ichikari, R., Li, H., Olszewski, K., Suma, E., Shapiro, A.: Rapid photorealistic blendshape modeling from RGB-D sensors. In: Proceedings of the 29th International Conference on Computer Animation and Social Agents, pp. 121–129 (2016)

  11. 11.

    Cosker, D., Krumhuber, E., Hilton, A.: A FACS valid 3D dynamic action unit database with applications to 3D dynamic morphable facial modeling. In: IEEE International Conference on Computer Vision (ICCV), 2011, pp. 2296–2303. IEEE (2011)

  12. 12.

    Fujiwara, T., Koshimizu, H., Fujimura, K., Kihara, H., Noguchi, Y., Ishikawa, N.: 3D modeling system of human face and full 3D facial caricaturing. In: Proceedings of Third International Conference on 3-D Digital Imaging and Modeling, 2001, pp. 385–392. IEEE (2001)

  13. 13.

    Fujiwara, T., Nishihara, T., Tominaga, M., Kato, K., Murakami, K., Koshirnizu, H.: On the detection of feature points of 3D facial image and its application to 3D facial caricature. In: Proceedings of Second International Conference on 3-D Digital Imaging and Modeling, 1999, pp. 490–496. IEEE (1999)

  14. 14.

    Garrido, P., Zollhöfer, M., Casas, D., Valgaerts, L., Varanasi, K., Pérez, P., Theobalt, C.: Reconstruction of personalized 3D face rigs from monocular video. ACM Trans. Graph. 35(3), 28:1–28:15 (2016)

  15. 15.

    Ichim, A.E., Bouaziz, S., Pauly, M.: Dynamic 3D avatar creation from hand-held video input. ACM Trans. Graph. (SIGGRAPH) 34(4), 45:1–45:14 (2015)

  16. 16.

    Lewis, J.P., Anjyo, K., Rhee, T., Zhang, M., Pighin, F., Deng, Z.: Practice and theory of blendshape facial models. In: EuroGraphics (2014)

  17. 17.

    Li, H., Weise, T., Pauly, M.: Example-based facial rigging. ACM Trans. Graph. (ToG) 29, 32 (2010)

  18. 18.

    Li, P., Chen, Y., Liu, J., Fu, G.: 3D caricature generation by manifold learning. In: IEEE International Conference on Multimedia and Expo, 2008, pp. 941–944. IEEE (2008)

  19. 19.

    Li, T., Bolkart, T., Black, M.J., Li, H., Romero, J.: Learning a model of facial shape and expression from 4D scans. ACM Trans. Graph. 36(6), 194 (2017). https://doi.org/10.1145/3130800.3130813. (Proc. SIGGRAPH Asia)

  20. 20.

    Liu, J., Chen, Y., Miao, C., Xie, J., Ling, C.X., Gao, X., Gao, W.: Semi-supervised learning in reconstructed manifold space for 3D caricature generation. In: Computer Graphics Forum, vol. 28, pp. 2104–2116. Wiley Online Library (2009)

  21. 21.

    Sadimon, S.B., Sunar, M.S., Mohamad, D., Haron, H.: Computer generated caricature: a survey. In: International Conference on Cyberworlds (CW), 2010, pp. 383–390. IEEE (2010)

  22. 22.

    Sumner, R.W., Popović, J.: Deformation transfer for triangle meshes. ACM Trans. Graph. (TOG) 23, 399–405 (2004)

  23. 23.

    Vlasic, D., Brand, M., Pfister, H., Popović, J.: Face transfer with multilinear models. ACM Trans. Graph. (TOG) 24, 426–433 (2005)

  24. 24.

    Waltz, R.A., Morales, J.L., Nocedal, J., Orban, D.: An interior algorithm for nonlinear optimization that combines line search and trust region steps. Math. Program. 107(3), 391–408 (2006)

  25. 25.

    Weise, T., Bouaziz, S., Li, H., Pauly, M.: Realtime performance-based facial animation. ACM Trans. Graph. (TOG) 30, 77 (2011)

  26. 26.

    Xie, J., Chen, Y., Liu, J., Miao, C., Gao, X.: Interactive 3D caricature generation based on double sampling. In: Proceedings of the 17th ACM International Conference on Multimedia, pp. 745–748. ACM (2009)

  27. 27.

    Zhou, J., Tong, X., Liu, Z., Guo, B.: 3D cartoon face generation by local deformation mapping. Vis. Comput. 32(6–8), 717–727 (2016)

Download references

Acknowledgements

We thank the anonymous reviewers for their constructive comments and suggestions. We also thank the graphics and parallel processing laboratory of Zhejiang University for sharing the FaceWarehouse dataset for our research. All cartoon exemplars used in our system are created by Xing Zhao and Shuitian Yan.

Author information

Correspondence to Jingyong Zhou.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interests.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 39774 KB)

Supplementary material 1 (mp4 39774 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhou, J., Wu, H., Liu, Z. et al. 3D cartoon face rigging from sparse examples. Vis Comput 34, 1177–1187 (2018). https://doi.org/10.1007/s00371-018-1553-3

Download citation

Keywords

  • Cartoon face animation
  • Data-driven method
  • Deformation gradient
  • Blendshape model