Skip to main content

Collecting Semantic Information for Locations in the Scenario-Based Lexical Knowledge Resource of a Text-to-Scene Conversion System

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNAI,volume 6884)

Abstract

WordsEye is a system for automatically converting a text description of a scene into a 3D image. In converting a text description into a corresponding 3D scene, it is necessary to map objects and locations specified in the text into the actual 3D objects. Individual objects typically correspond to single 3D models, but locations (e.g. a living room) are typically an ensemble of objects. Prototypical mappings from locations to objects and their relations are called location vignettes, which are not present in existing lexical resources. In this paper we propose a new methodology using Amazon’s Mechanical Turk to collect semantic information for location vignettes. Our preliminary results show that this is a promising approach.

Keywords

  • Text-to-Scene Systems
  • Amazon’s Mechanical Turk
  • Lexical Resources
  • Location Information
  • Dependency Parsing

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-642-23866-6_40
  • Chapter length: 10 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   74.99
Price excludes VAT (USA)
  • ISBN: 978-3-642-23866-6
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   99.00
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Adorni, G., Di Manzo, M., Giunchiglia, F.: Natural language driven image generation. In: COLING, pp. 495–500 (1984)

    Google Scholar 

  2. Baker, C., Fillmore, C., Lowe, J.: The Berkeley FrameNet Project. In: COLING-ACL (1998)

    Google Scholar 

  3. Badler, N., Bindiganavale, R., Bourne, J., Palmer, M., Shi, J., Schule, W.: A parameterized action representation for virtual human agents. In: Workshop on Embodied Conversational Characters, Lake Tahoe (1998)

    Google Scholar 

  4. Boberg, R.: Generating Line Drawings from Abstract Scene Descriptions. Masters thesis, Dept. of Elec. Eng, MIT, Cambridge, MA (1972)

    Google Scholar 

  5. Callison-Burch, C., Dredze, M.: Creating speech and language data with Amazons mechanical turk. In: NAACL 2010 Workshop on Creating Speech and Language Data with Amazons Mechanical Turk, Los Angeles, USA, pp. 1–12 (2010)

    Google Scholar 

  6. Clay, S.R., Wilhelms, J.: Put: Language-based interactive manipulation of objects. IEEE Computer Graphics and Applications, 31–39 (1996)

    Google Scholar 

  7. Coyne, B., Sproat, R.: Wordseye: An automatic text-to-scene conversion system. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, Los Angeles, CA, USA, pp. 487–496 (2001)

    Google Scholar 

  8. Coyne, B., Rambow, O., Hirschberg, J., Sproat, R.: Frame Semantics in Text-to-Scene Generation. In: Setchi, R., Jordanov, I., Howlett, R., Jain, L. (eds.) KES 2010. LNCS, vol. 6279, pp. 375–384. Springer, Heidelberg (2010)

    CrossRef  Google Scholar 

  9. Dupuy, S., Egges, A., Legendre, V., Nugues, P.: Generating a 3d simulation of a car accident from a written description in natural language: The CarSim system. In: Proceedings of ACL Workshop on Temporal and Spatial Information Processing, pp. 1–8 (2001)

    Google Scholar 

  10. Fellbaum, C.: WordNet: An Electronic Lexical Database. Bradford Books (1998)

    Google Scholar 

  11. Girju, R., Beamer, B., Rozovskaya, A., Fister, A., Bhat, S.: A knowledge-rich approach to identifying semantic relations between nominals. Information Processing and Management 46(5), 589–610 (2010)

    Google Scholar 

  12. Glass, K.R.: Automating the conversion of natural language fiction to multi-modal 3D animated virtual environments. PhD thesis, Rhodes University (2009)

    Google Scholar 

  13. Hanser, E., Mc Kevitt, P., Lunney, T., Condell, J., Ma, M.: SceneMaker: Multimodal Visualisation of Natural Language Film Scripts. In: Setchi, R., Jordanov, I., Howlett, R., Jain, L. (eds.) KES 2010. LNCS, vol. 6279, pp. 430–439. Springer, Heidelberg (2010)

    CrossRef  Google Scholar 

  14. Johansson, R., Williams, D., Berglund, A., Nugues, P.: Carsim: A System to Visualize Written Road Accident Reports as Animated 3D Scenes. In: Proceedings of the 2nd Workshop on Text Meaning and Interpretation, pp. 57–64. Association for Computational Linguistics, Stroudsburg (2004)

    Google Scholar 

  15. Kahn, K.: Creation of Computer Animation from Story Descriptions. Ph.D. thesis, MIT, AI Lab, Cambridge, MA (1979)

    Google Scholar 

  16. Ma, M.: Automatic Conversion of Natural Language to 3D Animation. Ph.D. thesis, University of Ulster (2006)

    Google Scholar 

  17. Rouhizadeh, M., Bowler, M., Sproat, R., Coyne, B.: Data Collection and Normalization for Building the Scenario-Based Lexical Knowledge Resource of a Text-to-Scene Conversion System. In: SMAP 2010: 5th International Workshop on Semantic Media Adaptation and Personalization, Limassol, Cyprus (2010)

    Google Scholar 

  18. Rouhizadeh, M., Bowler, M., Sproat, R., Coyne, B.: Collecting Semantic Data from Amazon’s Mechanical Turk for a Lexical Knowledge Resource in a Text to Picture Generating System. In: International Conference on Computational Semantics (IWCS 2011), Oxford (2011)

    Google Scholar 

  19. Schwarz, K., Rojtberg, P., Caspar, J., Gurevych, I., Goesele, M., Lensch, H.P.A.: Text-to-Video: Story Illustration from Online Photo Collections. In: Setchi, R., Jordanov, I., Howlett, R., Jain, L. (eds.) KES 2010. LNCS, vol. 6279, pp. 402–409. Springer, Heidelberg (2010)

    CrossRef  Google Scholar 

  20. Seversky, L.: Real-time Automatic 3D Scene Generation from Natural Language Voice and Text Descriptions. In: Proceedings of The 14Th Annual ACM International Conference on Multimedia (2006)

    Google Scholar 

  21. Simmons, R.: The clowns microworld. In: Proceedings of TINLAP, pp. 17–19 (1998)

    Google Scholar 

  22. Sproat, R.: Inferring the environment in a text-to-scene conversion system. In: First International Conference on Knowledge Capture, Victoria, BC (2001)

    Google Scholar 

  23. Turney, P., Littman, M.: Corpus-based Learning of Analogies and Semantic Relations. Machine Learning Journal 60(1-3), 251–278 (2005)

    CrossRef  Google Scholar 

  24. Turney, P.: Expressing implicit semantic relations without supervision. In: Proceedings of COLING-ACL, Australia (2006)

    Google Scholar 

  25. Winograd, T.: Understanding Natural Language. Ph.D. thesis, Massachusetts Institute of Technology (1972)

    Google Scholar 

  26. Ye, P., Baldwin, T.: Towards automatic animated storyboarding. In: Proceedings of the 23rd National Conference on Artificial Intelligence, vol. 1, pp. 578–583 (2008)

    Google Scholar 

  27. Zhu, X., Goldberg, A., Eldawy, M., Dyer, C., Strock, B.: A text-to-picture synthesis system for augmenting communication. In: Proceedings of the 22nd National Conference on Artificial Intelligence, vol. 2, pp. 1590–1595 (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Rouhizadeh, M., Coyne, B., Sproat, R. (2011). Collecting Semantic Information for Locations in the Scenario-Based Lexical Knowledge Resource of a Text-to-Scene Conversion System. In: König, A., Dengel, A., Hinkelmann, K., Kise, K., Howlett, R.J., Jain, L.C. (eds) Knowledge-Based and Intelligent Information and Engineering Systems. KES 2011. Lecture Notes in Computer Science(), vol 6884. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23866-6_40

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-23866-6_40

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-23865-9

  • Online ISBN: 978-3-642-23866-6

  • eBook Packages: Computer ScienceComputer Science (R0)