Extraction of Visual Material and Spatial Information from Text Description for Scene Visualization

Conference paper
Part of the Advances in Intelligent and Soft Computing book series (AINSC, volume 104)


The generation of 3D virtual scene could be greatly simplified by integration of natural language interface. This involves complex task of extracting of visual object, material and their spatial information from language description. The knowledge of visual perception, spatial language, natural language processing and graphic representation can be combined with one another to accomplish the task of generating a virtual scene. This paper first mentions a variety of research in developing language based scene application. This is followed by a brief introduction of natural language processing technology. Some related theories of human visual perception and spatial language are investigated in section 3. A representation formalism based on word-concept-visual information has been proposed by linking the levels of meaning and visualization in section 4. Finally, we draw conclusion and mentioned future work.


Virtual Reality CAD Material and Spatial Representation Text Visualization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Winograd, T.: Understanding Natural Language. Academic Press, London (1972)Google Scholar
  2. 2.
    Yamada, A., Yamamoto, T., Ikeda, H.: Reconstructing Spatial Image from Natural Language Texts. In: Proceedings of COLING 1992, Nantes, pp. 23–28 (1992)Google Scholar
  3. 3.
    Clay, R., Wilhelms, J.: Put: language-based interactive manipulation of objects. IEEE Computer Graphics and Applications 16(2), 31–39 (1996)CrossRefGoogle Scholar
  4. 4.
    Coyne, B., Sproat, R.: WordsEye: An automatic text-to-scene conversion system. In: Proceedings of 28th SIGGRAPH Annual Conf. Computer Graphics and Interactive Techniques, Los Angeles, pp. 487–496 (2001)Google Scholar
  5. 5.
    Akazawa, Y., Okada, Y., Niijima, K.: Automatic 3D object placement for 3D scene generation. In: The European Simulation and Modeling Conference, pp. 316–318 (2003)Google Scholar
  6. 6.
    Kahn, K.M.: Creation of Computer Animation from Story Descriptions. AI Technical report 540 MIT. Artificial Intelligence Laboratory, Cambridge (1979)Google Scholar
  7. 7.
    Mukerjee, A., Singh, M., Mishra, N.: Conceptual description of visual scenes from linguistic models. Journal of Image and Vision Computing. Special Issue on Conceptual Descriptions 18 (2000)Google Scholar
  8. 8.
    Nugues, P.: Verbal and written interaction in virtual worlds, some application examples. In: Proceedings of the Fifteenth Twente Workshop on Language Technology, Universiteit Twente, Enschede, pp. 137–145 (1999)Google Scholar
  9. 9.
    Glass, K., Bangay, S.: Constraint-based conversion of fiction text to a time-based graphical representation. In: Proceedings of the South African institute of computer scientists and information technologists on IT research in developing countries, New York, pp. 19–28 (2007)Google Scholar
  10. 10.
    Samad, T.: A Natural Language Interface for Computer-Aided Design. Kluwer Academic Pulishers, Boston (1986)CrossRefGoogle Scholar
  11. 11.
    Allen, J.: Natural Language Understanding. Benjamin/Cummings, Menlo Park (1987)Google Scholar
  12. 12.
    McKevitt, P.: Integration of natural language and vision processing. In: AAAI-1994. Workshop, Seattle, WA (1994)Google Scholar
  13. 13.
    Maaß, W.: How spatial information connects visual perception and natural language generation in dynamic environments: Towards a computational model. In: Proceedings of the International Conference, COSIT 1995, pp. 223–240. Springer, Heidelberg (1995)Google Scholar
  14. 14.
    Marr, D.: Vision: a Computational Investigation into the Human Representation and Processing of Visual Information. Freemann, San Francisco (1982)Google Scholar
  15. 15.
    Jackendoff, R.: On beyond zebra: the relation of linguistic and visual information. Cognition 26(2), 89–114 (1987)CrossRefGoogle Scholar
  16. 16.
    Talmy, L.: How language structures space. In: Herbert, P., Linda, A. (eds.) Spatial Orientation: Theory, Research, and Application, pp. 225–282. Plenum Press, New York (1983)Google Scholar
  17. 17.
    Miller, G.A., Johnson-Laird, P.N.: Language and perception. Belknap Press, Cambridge (1976)Google Scholar
  18. 18.
    Herskovits, A.: Language and Spatial Cognition: An interdisciplinary Study of the prepositions in English. Cambridge University Press, Cambridge (1986)Google Scholar
  19. 19.
    Retz-Schmidt, G.: Various views on spatial prepositions. AI Magazine 9(2), 95–105 (1988)Google Scholar
  20. 20.
    McDermott, V.: Spatial reasoning. In: Encyclopedia of Artificial Intelligence, vol. 2. John Wiley & Sons, Chichester (1987)Google Scholar
  21. 21.
    Laudau, B., Jackendoff, R.: What and Where in spatial language and spatial cognition. Behaviour and Brain Sciences 16, 217–265 (1993)CrossRefGoogle Scholar
  22. 22.
    Fludernik, M.: Towards a Natural Narratology. Routledge, London (1996)CrossRefGoogle Scholar
  23. 23.
    Zeng, X., Tan, M.L.: The Application of Text to Visualization Technology in Virtual Environment. Journal of Computer Engineer and Science (2) (2010)Google Scholar
  24. 24.
    Levinson, S.C., et al.: Returning the tables: language affects spatial reasoning. Cognition 84, 155–188 (2002)CrossRefGoogle Scholar
  25. 25.
    Logan, G., Sadler, D.: A computational analysis of the apprehension of spatial relations. In: Bloom, P., Peterson, M., Nadel, L., Garrett, M. (eds.) Language and Space, pp. 493–529. MIT Press, Cambridge (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  1. 1.School of Architecture and ArtCentral South UniversityChina

Personalised recommendations