Towards Animated Visualization of Actors and Actions in a Learning Environment

  • Oleksandr KolomiyetsEmail author
  • Marie-Francine Moens
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 292)


This paper describes ongoing research focused on natural language understanding and visualization of actors and actions extracted from narrative text. The technique employs a natural language processing pipeline for sophisticated syntactic and semantic analysis of text, and extracts information about events, actors and their roles in events, as well as temporal ordering of the events and spatial roles. This kind of information is traditionally considered indicative for text comprehension skill tests with novel readers. The visualization is implemented in the 3D graphics prototyping environment Alice, which provides a set of visual primitives and instructions for interactions and spatial manipulations of primitives.


Natural language processing and understanding semantic analysis text-to-scene translation visualization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Coyne, B., Sproat, R.: WordsEye: an automatic text-to-scene conversion system. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, pp. 487–496. ACM (2001)Google Scholar
  2. 2.
    Johansson, R., Berglund, A., Danielsson, M., Nugues, P.: Automatic text-to-scene conversion in the traffic accident domain. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 1073–1078 (2005)Google Scholar
  3. 3.
    Kayser, D., Nouioua, F.: From the textual description of an accident to its causes. Artificial Intelligence 173(12), 1154–1193 (2009)CrossRefGoogle Scholar
  4. 4.
    Joshi, D., Wang, J.Z., Li, J.: The story picturing engine—a system for automatic text illustration. ACM Transactions on Multimedia Computing, Communications, and Applications 2(1), 68–89 (2006)CrossRefGoogle Scholar
  5. 5.
    Mihalcea, R., Leong, C.W.: Toward communicating simple sentences using pictorial representations. Machine Translation 22(3), 153–173 (2008)CrossRefGoogle Scholar
  6. 6.
    Zhu, X., Goldberg, A.B., Eldawy, M., Dyer, C.R., Strock, B.: A text-to-picture synthesis system for augmenting communication. In: Association for the Advancement of Artificial Intelligence, pp. 1590–1595 (2007)Google Scholar
  7. 7.
    Bui, D., Nakamura, C., Bray, B.E., Zeng-Treitler, Q.: Automated illustration of patients instructions. In: Proceedings of the Annual Symposium of the American Medical Informatics Association, AMIA, pp. 1158–1167 (2012)Google Scholar
  8. 8.
    Filatova, E., Hatzivassiloglou, V.: Domain-independent detection, extraction, and labeling of atomic events. In: Proceedings of the Conference on Recent Advances in Natural Language Processing (2003)Google Scholar
  9. 9.
    Pustejovsky, J., Castaño, J., Ingria, R., Saurí, R., Gaizauskas, R., Setzer, A., Katz, G.: TimeML: Robust specification of event and temporal expressions in text. In: Proceedings of the International Workshop on Computational Semantics (2003)Google Scholar
  10. 10.
    Bethard, S., Kolomiyets, O., Moens, M.F.: Annotating narrative timelines as temporal dependency structures. In: Proceedings of the International Conference on Linguistic Resources and Evaluation (2012)Google Scholar
  11. 11.
    Kolomiyets, O., Bethard, S., Moens, M.F.: Extracting narrative timelines as temporal dependency structures. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, pp. 88–97. ACL (2012)Google Scholar
  12. 12.
    Chinchor, N., Robinson, P.: MUC-7 Named entity task definition. In: Proceedings of the 7th Conference on Message Understanding (1997)Google Scholar
  13. 13.
    Marcus, M.P., Marcinkiewicz, M.A., Santorini, B.: Building a large annotated corpus of English: The Penn Treebank. Computational linguistics 19(2) (1993)Google Scholar
  14. 14.
    Baker, C.F., Fillmore, C.J., Lowe, J.B.: The Berkeley FrameNet project. In: Proceedings of the 17th International Conference on Computational Linguistics, pp. 86–90. ACL (1998)Google Scholar
  15. 15.
    Palmer, M., Gildea, D., Kingsbury, P.: The proposition bank: An annotated corpus of semantic roles. Computational Linguistics 31(1), 71–106 (2005)CrossRefGoogle Scholar
  16. 16.
    Nivre, J., Hall, J., Nilsson, J.: Maltparser: A data-driven parser-generator for dependency parsing. In: Proceedings of LREC, pp. 2216–2219 (2006)Google Scholar
  17. 17.
    De Marneffe, M.C., Manning, C.D.: The Stanford typed dependencies representation. In: Proceedings of the Workshop on Cross-Framework and Cross-Domain Parser Evaluation, pp. 1–8. ACL (2008)Google Scholar
  18. 18.
    Lee, H., Peirsman, Y., Chang, A., Chambers, N., Surdeanu, M., Jurafsky, D.: Stanford’s multi-pass sieve coreference resolution system at the CoNLL-2011 shared task. In: Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task, pp. 28–34. ACL (2011)Google Scholar
  19. 19.
    Verhagen, M., Saurí, R., Caselli, T., Pustejovsky, J.: SemEval-2010 Task 13: TempEval-2. In: Proceedings of the 5th International Workshop on Semantic Evaluation, pp. 57–62 (2010)Google Scholar
  20. 20.
    Pustejovsky, J., Moszkowicz, J.L.: The qualitative spatial dynamics of motion in language. Spatial Cognition & Computation 11(1), 15–44 (2011)CrossRefGoogle Scholar
  21. 21.
    Kolomiyets, O., Moens, M.F.: MotionML: A shallow approach for annotating motions in text. In: Proceedings of Corpus Linguistics (2013)Google Scholar
  22. 22.
    Kolomiyets, O., Kordjamshidi, P., Moens, M.F., Bethard, S.: SemEval-2013 Task 3: Spatial Role Labeling. In: Proceedings of the Seventh International Workshop on Semantic Evaluations, pp. 255–262. ACL (2013)Google Scholar
  23. 23.
    Miller, G.A.: WordNet: a lexical database for English. Communications of the ACM 38, 39–41 (1995)CrossRefGoogle Scholar
  24. 24.
    Miller, S.A., Schubert, L.K.: Time revisited. Computational Intelligence 6(2), 108–118 (1990)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.Department of Computer ScienceKU LeuvenLeuvenBelgium

Personalised recommendations