Visualisation Techniques for Analysis and Exploration of Multimedia Data

  • Vedran Sabol
  • Wolfgang Kienreich
  • Michael Granitzer
Part of the Studies in Computational Intelligence book series (SCI, volume 101)

Information technology has developed past its traditional focus on text-based data. The much-cited rapid growth of available information has been accompanied by a diversification of information types. Multimedia data is rapidly becoming the predominant form of information created, processed and distributed in many application domains.Multimedia data sets are characterized by their heterogeneous nature and complex structure. Documents often combine of different modalities, for example video streams, audio streams and textual information. Document content often features a pronounced temporal component, as in the case of audio and video data. Multimedia documents frequently include rich semantic descriptors and complex structures of cross-modal, inter- and intra-document references.

It is often not feasible to manually annotate such complex semantics, especially in the context of very large data sets. Various automated methods for the extraction of semantic metadata have been proposed and evaluated. However, the capabilities of automatic extraction are limited in terms of accuracy, performance and diversity of results. Visualisation techniques employ the vast processing power of the human visual apparatus to quickly identify complex patterns in large amounts of data. When combined with machine processing capabilities, such techniques provide unparalleled means for gaining insight into large data sets in general, and into multimedia data sets in particular.

The multi-faceted nature of multimedia documents has led to a variety of visual representations for navigating, analysing and understanding of multimedia data sets. As each representation is specifically designed to address different aspects of the data, innovative approaches combining several visualisations in a single coordinated interface had to be introduced. This chapter presents a comparative discussion of selected multimedia visual representations and tools.


Visualisation Technique Multimedia Content Multimedia Data Information Visualisation Multimedia Document 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Adamczyk, P. D., “Seeing Sounds: Exploring Musical Social Networks”, Poster, In Proceedings of the 12th annual ACM international conference on Multimedia, 2004, pp. 512 - 515.Google Scholar
  2. [2]
    AllMusicGuide. http://www. allmusic. com/, last accessed 06/2007
  3. [3]
    Andrews, K., Wolte, J., Pichler, M., “Information Pyramids”, IEEE Visualiza- tion, USA, 1997.Google Scholar
  4. [4]
    Basalaj, W., Incremental multidimensional scaling method for database visu- alization. In Visual Data Exploration and Analysis VI (Proc. SPIE, volume 3643), January 1999.Google Scholar
  5. [5]
    Bederson, B. B., “Quantum Treemaps and Bubblemaps for a Zoomable Image Browser”, In Proc. User Interface Systems and Technology, pp. 71--80, 2001.Google Scholar
  6. [6]
    BubbleShare, 2007, http://www. bubbleshare. com/, last accessed 06/2007
  7. [7]
    Burkhard, R., Eppler, M. “Knowledge Visualisation”, in: T. Keller and S. O. Tergan, “Knowledge and Information Visualisation”, Springer, Germany, 2005Google Scholar
  8. [8]
    Card, S. K., Mackinlay, J. D., Shneiderman. B., “Readings in information visu- alization: using vision to think”, San Diego: Academic Press, pp. 1-34.Google Scholar
  9. [9]
    Casares, J. P., “SILVER: An Intelligent Video Editor”, Poster at CHI ’01, In extended abstracts on Human factors in computing systems, pp. 425 - 426, 2001.Google Scholar
  10. [10]
    Chiu, P., Girgensohn, A., Lertsithichai, S., Polak, W., Shipman, F., “Media- Metro: Browsing Multimedia Document Collections with a 3D City Meta- phor”, Proceedings of the 13th annual ACM international conference on Multimedia, 2005, pp. 213 - 214.Google Scholar
  11. [11]
    Christel, M., Martin, D., “Information Visualization within a Digital Video Library”, J. Intelligent Info. Systems 11(3), pp. 235-257, 1998.CrossRefGoogle Scholar
  12. [12]
    Cugini, J., Laskowski, S., “Design of 3-D Visualization of Search Results - Evolution and Evaluation”, Proceedings of IST/SPIE’s 12th Annual Inter- national Symposium: Electronic Imaging 2000: Visual Data Exploration and Analysis (SPIE 2000), San Jose, CA, 23-28 January 2000.Google Scholar
  13. [13]
    Flickr, 2007, www flickr. com, last accessed 06/2007Google Scholar
  14. [14]
    Hearst, M. A., TileBars: Visualization of terms distribution information in full text information access. In Proc. of the ACM SIGCHI Conference on Human Factors in Computing Systems, p. 59-66, Denver, CO, May 1995.Google Scholar
  15. [15]
    Herman, I., Melançon, G., Marshall, M. S., “Graph Visualization and Naviga- tion in Information Visualization - A Survey”, IEEE Transactions on Visuali- zation and Computer Graphics, Vol. 6, No. 1, 2000.CrossRefGoogle Scholar
  16. [16]
    Hewett, T., Baecker, R., Card, S., Carey, T., Gasen, J., Mantei, M., Perlman, G., Strong, G., Verplank, W., “ACM SIGCHI Curricula for Human-Computer Interaction”, Technical Report of the ACM SIGCHI Curriculum Development Group, 1992. http://sigchi. org/cdg/
  17. [17]
    IDC, “The Expanding Digital Universe - A Forecast of Worldwide Infor- mation Growth Through 2010”, Study available at http://www. emc. com/ about/destination/digital universe/pdf/Expanding Digital Universe IDC WhitePaper 022507. pdf
  18. [18]
    Inxight Software, Inc., http://www. inxight. com, last accessed 06/2007
  19. [19]
    SO 9241-11: Guidance on Usability (1998), http://www. usabilitynet. org/tools/r international htm#9241-11, last accessed 06/2007
  20. [20]
    Jaffe, A., Naaman, M., Tassa, T., Davis, M., “Generating Summaries and Visualization for Large Collections of GeoReferenced Photographs”, Proceedings of the 8th ACM international workshop on Multimedia information retrieval, pp. 89 - 98, 2006.Google Scholar
  21. [21]
    Kapler, T. and Wright, W., “GeoTime Information Visualization”, Informa- tion Visualization, 4(2): pp. 136-146, 2005.CrossRefGoogle Scholar
  22. [22]
    Kienreich, W., Granitzer, M., “Visualising Knowledge Webs for Encyclo- pedia Articles”, 9th International Conference on Information Visualization, UK, 2005.Google Scholar
  23. [23]
    Konqueror, 2007, http://www. konqueror. org/, last accessed 06/2007
  24. [24]
    Kuchinsky, A., Pering, C., Creech, M. L., Freeze, D., Serra, B., Gwizdka, J., “FotoFile: A Consumer Multimedia Organization and Retrieval System”, Proc. ACM CHI99 Conference on Human Factors in Computing Systems, pp. 496-503, May 1999.Google Scholar
  25. [25]
    Lalanne, D., Lisowska, A., Bruno, E., Flynn, M., Georgescul, M., Guillemot, M., Janvier, B., Marchand-Maillet, S., Melichar, M., Moenne-Loccoz, N., Popescu-Belis, A., Rajman, M., Rigamonti, M., von Rotz1, D., Wellner, P., “The IM2 Multimodal Meeting Browser Family”, Joint IM2 Technical Report, March 2005.Google Scholar
  26. [26]
    Lamping, J., Rao, R., Pirolli, P., “A focus+context technique based on hyper- bolic geometry for visualizing large hierarchies”, In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (May I995), ACM.Google Scholar
  27. [27]
    Long, A. C., Myers, B. A., Casares, J., Stevens, S. M., Corbett, A., “Video Editing Using Lenses and Semantic Zooming”, Technical Report, http://www. cs. cmu. edu/∼silver/silver2. pdf
  28. [28]
    Lux, M., “Magick - Ein Werkzeug für Cross-Media Clustering und Visualis- ierung”, Master’s Thesis, Graz University of Technology, 2004.Google Scholar
  29. [29]
    Lyman, P., Varian, H. R., How much information 2003, http://www2. sims. berkeley. edu/research/projects/how-much-info-2003/
  30. [30]
    M4 Project, http://www mistral-project. at, last accessed 06/2007
  31. [31]
    Mills, M., Cohen, J., Wong, Y., “A Magnifier Tool for Video Data” CHI ’92 Conference Proceedings, ACM Press, pp. 93-98, 1992.Google Scholar
  32. [32]
    Mistral Project, www. mistral-project. at, 2006
  33. [33]
    Müller, W., Schumann, H., “Visualisation Methods for Time-dependent Data - an Overview”, Proceedings of the 2003 Winter Simulation Conference, Vol. 1, pp. 737- 745Google Scholar
  34. [34]
    Nautilus, http://www. gnome. org/projects/nautilus/, last accessed 06/2007
  35. [35]
    Picasa, http://picasa. google. com/, last accessed 06/2007
  36. [36]
    Pingali, G., Opalach, A., Carlbom, I., “Multimedia Retrieval Through Spatio- temporal Activity Maps”, Proceedings of the ninth ACM international confer- ence on Multimedia, pp. 129 - 136, 2001.Google Scholar
  37. [37]
    Pingali, G., Opalach, A., Jean, Y., Carlbom, I., “Visualization of Sports using Motion Trajectories: Providing Insights into Performance, Style, and Strat- egy”, Proceedings of the conference on Visualization ’01, Pp: 75 - 82, 2001.Google Scholar
  38. [38]
    Ponceleon, D., Dieberger, A., “Hierarchical Brushing in a Collection of Video Data”, Proc. 34th Hawaii International Conference on System Sciences 2001, pp. 1654-1661.Google Scholar
  39. [39]
    Rennison, E., "Galaxy of News: An Approach to Visualizing and Understanding Expansive News Landscapes”, ACM Symposium on User Interface Software and Technology, USA, 1994.Google Scholar
  40. [40]
    Roberts, J. C., Wright, M. A. E., “Towards Ubiquitous Brushing for Informa- tion Visualization”, Proceedings of the conference on Information Visualiza- tion 2006, pp: 151 - 156.Google Scholar
  41. [41]
    Rodden, K., Basalaj, W., Sinclair, D., Wood, K., “Does Organisation by Similarity Assist Image Browsing?”, International ACM SIGCHI conference, pp. 190-197, 2001.Google Scholar
  42. [42]
    Rodden, K., Basalaj, W., Sinclair, D., Wood, K., “Evaluating a Visualisation of Image Similarity as a Tool for Image Browsing”, Proceedings of the 1999 IEEE Symposium on Information Visualization, page 36.Google Scholar
  43. [43]
    Sabol, V., Granitzer, M. and Kienreich, W., “Fused Exploration of Temporal Developments and Topical Relationships in Heterogeneous Data Sets”, 3rd International Symposium of Knowledge and Argument Visualization, 11th International Conference Information Visualisation. London, UK: IEEE Computer Society, 2007.Google Scholar
  44. [44]
    Sabol, V., Gütl, C., Neidhart, T., Juffinger, A., Klieber, W., Granitzer, M., “Visualization Metaphors for Multi-modal Meeting Data”, Workshop Multi- media Semantics - The Role of Metadata (WMSRM 07), Proceedings Band “Aachener Informatik Berichte”, Aachen 2007.Google Scholar
  45. [45]
    Shneiderman, B, Plaisant, C., “Designing the User Interface: Strategies for Effective Human-Computer Interactions”, Addison-Wesley, Reading, USA, 2004.Google Scholar
  46. [46]
    Tanaka, Y., Okada, Y., Niijima, K., “Interactive Interfaces of Treecube for Browsing 3D Multimedia Data”, Proceedings of the working conference on Advanced visual interfaces, 2004, pp. 298 - 302.Google Scholar
  47. [47]
    Thomas, J. J., Cook, K. A., “Illuminating the Path: The Research and Devel- opment Agenda for Visual Analytics”, IEEE CS Press, USA, 2005.Google Scholar
  48. [48]
    TouchGraph, 2007, http://www. touchgraph. com/, last accessed 06/2007
  49. [49]
    Wactlar, H. D., Kanade, T., Smith, M. A., Stevens, S. M., “Intelligent Access to Digital Video: Informedia Project”, Computer, 29(5), 46-52, 1996.CrossRefGoogle Scholar
  50. [50]
    Wactlar, H. D., “Multi-Document Summarization and Visualization in the Informedia Digital Video Library”, in New Information Technology, 2001.Google Scholar
  51. [51]
    Wactlar, H. D., “Extracting and Visualizing Knowledge from Film and Video Archives”, Proceedings of I-Know’02 International Conference on Knowledge Management, 2002.Google Scholar
  52. [52]
    Wactlar, H. D., “Extracting and Visualizing Knowledge from Film and Video Archives”, Presentation at the I-know’02, available at http://i-know know- center. tugraz. at/previous/i-know02/downloads/hwactlar. pdf
  53. [53]
    Wellner, P., Flynn, M., Guillemot, M., “Browsing Recorded Meetings with Ferret”, Machine Learning for Multimodal Interaction, MLMI 2004, Martigny, Switzerland, June 21-23, 2004.Google Scholar
  54. [54]
    Wellner, P., Flynn, M., Tucker, S., Whittaker, S., “A Meeting Browser Evaluation Test”, Conference on Human factors in Computing System (CHI), Portland, Oregon, 2005.Google Scholar
  55. [55]
    YouTube, 2007, www. youtube. com, last accessed 06/2007
  56. [56]
    Zooomr, 2007, http://www. zooomr. com/, last accessed 06/2007

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Vedran Sabol
    • 1
  • Wolfgang Kienreich
    • 1
  • Michael Granitzer
    • 1
  1. 1.Know-Center GrazGrazAustria

Personalised recommendations