Skip to main content

Generating animation from natural language texts and semantic analysis for motion search and scheduling

Abstract

This paper presents an animation system that generates an animation from natural language texts such as movie scripts or stories. It also proposes a framework for a motion database that stores numerous motion clips for various characters. We have developed semantic analysis methods to extract information for motion search and scheduling from script-like input texts. Given an input text, the system searches for an appropriate motion clip in the database for each verb in the input text. Temporal constraints between verbs are also extracted from the input text and are used to schedule the motion clips found. In addition, when necessary, certain automatic motions such as locomotion, taking an instrument, changing posture, and cooperative motions are searched for in the database. An animation is then generated using an external motion synthesis system. With our system, users can make use of existing motion clips. Moreover, because it takes natural language text as input, even novice users can use our system.

This is a preview of subscription content, access via your institution.

References

  1. Oshita, M.: Generating animation from natural language texts and framework of motion database. In: Proc. of International Conference on Cyberworlds 2009, pp. 146–153, Bradford, UK (2009)

  2. Winograd, T.: Understanding Natural Language. Academic Press, San Diego (1972)

    Google Scholar 

  3. Badler, N., Bindiganavale, R., Allbeck, J., Schuler, W., Zhao, L., Palmer, M.: Parameterized action representation for virtual human agents. In: Embodied Conversational Agents, pp. 256–284 (2000)

  4. Bindiganavale, R., Schuler, W., Allbeck, J., Badler, N., Joshi, A., Palmer, M.: Dynamically altering agent behaviors using natural language instructions. In: Proc. of Autonomous Agents 2000, pp. 293–300 (2000)

  5. Tokunaga, T., Funakoshi, K., Tanaka, H.: K2: animated agents that understand speech commands and perform actions. In: Proc. of 8th Pacific Rim International Conference on Artificial Intelligence 2004, pp. 635–643 (2004)

  6. Fillmore, C.J.: The case for case. In: Universals in Linguistic Theory, pp. 1–88 (1968)

  7. Lu, R., Zhan, S.: Automatic Generation of Computer Animation: Using AI for Movie Animation. Springer, Berlin (2002)

    Google Scholar 

  8. Sumi, K., Nagata, M.: Animated storytelling system via text. In: Proc. of International Conference on Advances in Computer Entertainment Technology (2006)

  9. Baba, H., Noma, T., Okada, N.: Visualization of temporal and spatial information in natural language descriptions. Trans. Inf. Syst. E79-D(5), 591–599 (1996)

    Google Scholar 

  10. Coyne, B., Sproat, R.: Wordseye: an automatic text-to-scene conversion system. In: Proc. of SIGGRAPH 2001, pp. 487–496 (2000)

  11. Levine, S., Theobalt, C., Koltun, V.: Real-time prosody-driven synthesis of body language. ACM Trans. Graph. 28(5), 172 (2009) (In: Proc. of ACM SIGGRAPH Asia 2009)

    Article  Google Scholar 

  12. Perlin, K., Goldberg, A.: Improv: a system for scripting interactive actors in virtual worlds. In: Proc. of SIGGRAPH ’96 Proceedings, pp. 205–216 (1996)

  13. Conway, M.J.: Alice: easy-to-learn 3D scripting for novices. PhD Dissertation, University of Virginia (1997)

  14. Hayashi, M., Ueda, H., Kurihara, T., Yasumura, M.: TVML (TV program Making Language)—automatic TV program generation from text-based script. In: Proc. of Imagina ’99, pp. 84–89 (1999)

  15. Shim, H., Kang, B.G.: CAMEO—camera, audio and motion with emotion orchestration for immersive cinematography. In: Proc. of International Conference on Advances in Computer Entertainment Technology (ACE) 2008, pp. 115–118 (2008)

  16. Park, S.I., Shin, H.J., Shin, S.Y.: On-line locomotion generation based on motion blending. In: Proc. of ACM SIGGRAPH Symposium on Computer Animation 2002, pp. 105–111 (2002)

  17. Rose, C., Cohen, M.F., Bodenheimer, B.: Verbs and adverbs: Multidimensional motion interpolation. IEEE Comput. Graph. Appl. 18(5), 32–40 (1998)

    Article  Google Scholar 

  18. Kovar, L., Gleicher, M.: Automated extraction and parameterization of motions in large data sets. ACM Trans. Graph. 23(3), 559–568 (2004)

    Article  Google Scholar 

  19. Klein, D., Manning, C.D.: Fast exact inference with a factored model for natural language parsing. In: Advances in Neural Information Processing Systems 15 (NIPS 2002), pp. 3–10 (2003)

  20. Oshita, M.: Smart motion synthesis. Comput. Graph. Forum 27(7), 1909–1918 (2008)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Masaki Oshita.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Oshita, M. Generating animation from natural language texts and semantic analysis for motion search and scheduling. Vis Comput 26, 339–352 (2010). https://doi.org/10.1007/s00371-010-0423-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-010-0423-4

Keywords

  • Computer animation
  • Motion database
  • Natural language processing