A Synthetic 3D Multimedia Environment

  • Ronald Genswaider
  • Helmut Berger
  • Michael Dittenbach
  • Andreas Pesenhofer
  • Dieter Merkl
  • Andreas Rauber
  • Thomas Lidy
Part of the Studies in Computational Intelligence book series (SCI, volume 96)

In this chapter we present The MediaSquare, a synthetic 3D multimedia environment we are currently developing. The MediaSquare enables users, impersonated as avatars, to browse and experience multimedia content by literally walking through it. Users may engage in conversations with other users, exchange experiences as well as collectively explore and enjoy the featured content. The combination of algorithms from the area of artificial intelligence with state-of-the-art 3D virtual environments creates an intuitive interface that provides access to manually as well as automatically structured multimedia data while allowing to take advantage of spatial metaphors.


Virtual Environment Digital Library Game Engine Music Information Retrieval Rhythm Pattern 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    H. Ahonen, O. Heinonen, M. Klemettinen, and A.I. Verkamo. Applying data mining techniques for descriptive phrase extraction in digital documents. In Proceedings of the Advances in Digital Libraries Conference (ADL98), page 2, Santa Barbara, CA, 1998. IEEE Computer Society.Google Scholar
  2. 2.
    R.B. Allen, P. Obry, and M. Littman. An interface for navigating clustered document sets returned by queries. In Proceedings of the Conference on Organizational Computing Systems (COCS93), pages 166–171, Milpitas, CA, 1993. ACM.CrossRefGoogle Scholar
  3. 3.
    M.Q. Baldonado and T. Winograd. Sensemaker: An information-exploration interface supporting the contextual evolution of a user’s interests. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 11–18, Atlanta, GA, 1997. ACM.CrossRefGoogle Scholar
  4. 4.
    E. Castronova. Synthetic Worlds: The Business and Culture of Online Games. University of Chicago Press, Chicago, IL, 2005.Google Scholar
  5. 5.
    H. Chen, C. Schuffels, and R. Orwig. Internet categorization and search: A self-organizing approach. Journal of Visual Communication and Image Representation, 7(1):88–102, 1996.CrossRefGoogle Scholar
  6. 6.
    M. Christoffel and B. Schmitt. Accessing libraries as easy as a game. In JCDL 2002 Workshop: Visual Interfaces to Digital Libraries, pages 25–38, London, UK, 2002. Springer.CrossRefGoogle Scholar
  7. 7.
    M.F. Costabile, F. Esposito, G. Semeraro, N. Fanizzi, and S. Ferilli. Interacting with idl: The adaptive visual interface. In Proceedings of the Second European Conference on Research and Advanced Technology for Digital Libraries, pages 515–534, Heraklion, Greece, 1998. Springer.Google Scholar
  8. 8.
    P. Cubaud, J. Dupire, and A. Topol. Fluid interaction for the document in context. In Proceedings of the 2007 Conference on Digital Libraries (JCDL’07), page 504, Vancouver, Canada, 2007. ACM.CrossRefGoogle Scholar
  9. 9.
    P. Cubaud, C. Thiria, and A. Topol. Experimenting a 3D interface for the access to a digital library. In Proceedings of the Third ACM Conference on Digital Libraries (DL’98), pages 281–382, Pittsburgh, PA, 1998. ACM.CrossRefGoogle Scholar
  10. 10.
    M.J. Dovey. A technique for “regular expression” style searching in polyphonic music. In Proceedings of the International Symposium on Music Information Retrieval (ISMIR 2001), 2001.Google Scholar
  11. 11.
    T.L. Friedman. The World is Flat: A Brief History of the Twenty-First Century. Farrar, Straus and Giroux, New York, 2005.Google Scholar
  12. 12.
    A. Ghias, J. Logan, D. Chamberlin, and B. C. Smith. Query by humming: musical information retrieval in an audio database. In Proceedings of the Third ACM International Conference on Multimedia, pages 231–236, New York, NY, USA, 1995. ACM.Google Scholar
  13. 13.
    S. Greenberg and M. Roseman. Using a room metaphor to ease transitions in groupware. In M. Ackerman, V. Pipek, and V. Wulf, editors, Sharing Expertise: Beyond Knowledge Management, pages 203–256. MIT, Cambridge, MA, January 2003.Google Scholar
  14. 14.
    M. Hearst and C. Karadi. Cat-a-cone: an interactive interface for specifying searches and viewing retrieval results using a large category hierarchy. SIGIR Forum, 31(SI):246–255, 1997.CrossRefGoogle Scholar
  15. 15.
    T. Honkela, S. Kaski, K. Lagus, and T. Kohonen. WEBSOM – Self-organizing maps of document collections. In Proceedings of the Workshop on Self-Organizing Maps (WSOM97), Espoo, Finland, 1997.Google Scholar
  16. 16.
    P. Knees, M. Schedl, T. Pohle, and G. Widmer. An innovative three-dimensional user interface for exploring music collections enriched with meta-information from the web. In Proceedings of the 14th Annual ACM International Conference on Multimedia, pages 17–24. ACM, 2006.Google Scholar
  17. 17.
    T. Kohonen. Self-organized formation of topologically correct feature maps. Biological Cybernetics, 43, 1982.Google Scholar
  18. 18.
    T. Kohonen. Self-Organizing Maps. Springer, Berlin Heidelberg New York, 1995.Google Scholar
  19. 19.
    T. Kohonen, S. Kaski, K. Lagus, J. Salojärvi, J. Honkela, V. Paatero, and A. Saarela. Self-organization of a massive document collection. IEEE Transactions on Neural Networks, 11(3):574–585, May 2000.CrossRefGoogle Scholar
  20. 20.
    T. Lidy and A. Rauber. Evaluation of feature extractors and psycho-acoustic transformations for music genre classification. In Proceedings of the International Conference on Music Information Retrieval (ISMIR), pages 34–41, London, UK, September 11–15, 2005.Google Scholar
  21. 21.
    Y. Liu, P. Dantzig, M. Sachs, J. Corey, M. Hinnebusch, T. Sullivan, M. Damashek, and J. Cohen. Visualizing document classification: A search aid for the digital library. In Proceedings of the European Conference on Digital Libraries, Heraklion, Greece, 1998.Google Scholar
  22. 22.
    D. Lübbers. SoniXplorer: Combining visualization and auralization for content-based exploration of music collections. In Proceedings of the 6th International Conference on Music Information Retrieval (ISMIR 2005), pages 590–593, 2005.Google Scholar
  23. 23.
    R. Mayer, T. Lidy, and A. Rauber. The map of mozart. In Proceedings of the International Conference on Music Information Retrieval (ISMIR), Victoria, Canada, October 8–12, 2006.Google Scholar
  24. 24.
    R. Mayer, D. Merkl, and A. Rauber. Mnemonic SOMs: Recognizable shapes for self-organizing maps. In M. Cottrell, editor, Proceedings of the Fifth Workshop on Self-Organizing Maps (WSOM’05), pages 131–138, Paris, France, September 5–8 2005.Google Scholar
  25. 25.
    R. J. McNab, L. A. Smith, I. H. Witten, C. L. Henderson, and S. J. Cunningham. Towards the digital music library: tune retrieval from acoustic input. In Proceedings of the First ACM International Conference on Digital Libraries (DL ’96), pages 11–18, New York, NY, USA, 1996. ACM.CrossRefGoogle Scholar
  26. 26.
    D. Merkl and A. Rauber. Automatic labeling of self-organizing maps for information retrieval. In Proceedings of the International Conference on Neural Information Processing (ICONIP’99), Perth, WA, 1999.Google Scholar
  27. 27.
    D. Merkl and A. Rauber. Document classification with unsupervised neural networks. In F. Crestani and G. Pasi, editors, Soft Computing in Information Retrieval, pages 102–121. Physica, 2000.Google Scholar
  28. 28.
    R. Neumayer, M. Dittenbach, and A. Rauber. PlaySOM and PocketSOMPlayer, Alternative interfaces to large music collections. In Proceedings of the 6th International Conference on Music Information Retrieval (ISMIR 2005), pages 618–623, 2005.Google Scholar
  29. 29.
    A. Pejtersen. A library system for information retrieval based on cognitive task analysis and supported by an icon-based interface. In Proceedings of the Annual ACM SIGIR Conference on Research and Developement in Information Retrieval (SIGIR’89), 1989.Google Scholar
  30. 30.
    A. Rauber and M. Frühwirth. Automatically analyzing and organizing music archives. In Proceedings of the 5th European Conference on Research and Advanced Technology for Digital Libraries (ECDL 2001), Springer Lecture Notes in Computer Science, Darmstadt, Germany, September 4–8, 2001. Springer.Google Scholar
  31. 31.
    A. Rauber and D. Merkl. Text mining in the SOMLib digital library system: The representation of topics and genres. Applied Intelligence, 18(3):271–293, 2003.zbMATHCrossRefGoogle Scholar
  32. 32.
    A. Rauber, E. Pampalk, and D. Merkl. Using psycho-acoustic models and self-organizing maps to create a hierarchical structuring of music by musical styles. In Proceedings of the 3rd International Symposium on Music Information Retrieval (ISMIR 2002), pages 71–80, Paris, France, October 13–17 2002.Google Scholar
  33. 33.
    G. Robertson, S. Card, and J. Mackinlay. Information visualization using 3D interactive animation. Communications of the ACM, 36(4):57–71, 1993.CrossRefGoogle Scholar
  34. 34.
    G. Salton. Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer. Addison-Wesley, Reading, MA, 1989.Google Scholar
  35. 35.
    I. Stavness, J. Gluck, L. Vilhan, and S. Fels. The MUSICtable: A map-based ubiquitous system for social interaction with a digital music collection. In Proceedings of the 4th International Conference on Entertainment Computing (ICEC 2005), 2005.Google Scholar
  36. 36.
    M. Torrens, P. Hertzog, and J.-L. Arcos. Visualizing and exploring personal music libraries. In Proceedings of the 5th International Conference on Music Information Retrieval (ISMIR 2004), 2004.Google Scholar
  37. 37.
    W.-H. Tsai. A query-by-example technique for retrieving cover versions of popular songs with similar melodies. In Proceedings of the 6th International Conference on Music Information Retrieval (ISMIR 2005), pages 183–190, 2005.Google Scholar
  38. 38.
    G. Tzanetakis and P. Cook. Marsyas3D: A prototype audio browser-editor using a large scale immersive visual and audio display. In Proceedings of the International Conference on Auditory Display, 2001.Google Scholar
  39. 39.
    B.S. Woodcock. An analysis of mmog subscription growth.
  40. 40.
    N. Yee. The psychology of massively multi-user online role-playing games: Emotional investment, motivations, relationship formation, and problematic usage. In R. Schroeder and A. Axelsson, editors, Avatars at Work and Play: Collaboration and Interaction in Shared Virtual Environments, volume 34 of Computer Supported Cooperative Work. Springer, Heidelberg, Germany, 2005.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Ronald Genswaider
    • 1
  • Helmut Berger
    • 1
  • Michael Dittenbach
    • 1
  • Andreas Pesenhofer
    • 1
  • Dieter Merkl
    • 2
  • Andreas Rauber
    • 1
    • 2
  • Thomas Lidy
    • 2
  1. 1.E-Commerce Competence Center – EC3iSpaces Research GroupWienAustria
  2. 2.Department of Software Technology and Interactive SystemsVienna University of TechnologyViennaAustria

Personalised recommendations