Skip to main content
Book cover

Multimedia Analysis, Processing and Communications pp 3–57Cite as

Visualisation and Browsing of Image Databases

  • Chapter

Part of the Studies in Computational Intelligence book series (SCI,volume 346)

Abstract

In this chapter we provide a comprehensive overview of the emerging field of visualising and browsing image databases. We start with a brief introduction to content-based image retrieval and the traditional query-by-example search paradigm that many retrieval systems employ. We specify the problems associated with this type of interface, such as users not being able to formulate a query due to not having a target image or concept in mind. The idea of browsing systems is then introduced as a means to combat these issues, harnessing the cognitive power of the human mind in order to speed up image retrieval.We detail common methods in which the often high-dimensional feature data extracted from images can be used to visualise image databases in an intuitive way. Systems using dimensionality reduction techniques, such as multi-dimensional scaling, are reviewed along with those that cluster images using either divisive or agglomerative techniques as well as graph-based visualisations. While visualisation of an image collection is useful for providing an overview of the contained images, it forms only part of an image database navigation system. We therefore also present various methods provided by these systems to allow for interactive browsing of these datasets. A further area we explore are user studies of systems and visualisations where we look at the different evaluations undertaken in order to test usability and compare systems, and highlight the key findings from these studies. We conclude the chapter with several recommendations for future work in this area.

Keywords

  • Image Retrieval
  • User Study
  • Image Database
  • Representative Image
  • Relevance Feedback

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-642-19551-8_1
  • Chapter length: 55 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   299.00
Price excludes VAT (USA)
  • ISBN: 978-3-642-19551-8
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   379.99
Price excludes VAT (USA)
Hardcover Book
USD   379.99
Price excludes VAT (USA)
Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abdel-Mottaleb, M., Krischnamachari, S., Mankovich, N.J.: Performance Evaluation of Clustering Algorithms for Scalable Image Retrieval. In: IEEE Computer Society Workshop on Empirical Evaluation of Computer Vision Algorithms (1998)

    Google Scholar 

  2. Assfalg, J., Del-Bimbo, A., Pala, P.: Virtual Reality for Image Retrieval. Journal of Visual Languages and Computing 11(2), 105–124 (2000)

    CrossRef  Google Scholar 

  3. Atkinson, K.E.: An Introduction to Numerical Analysis. John Wiley and Sons, Chichester (1989)

    MATH  Google Scholar 

  4. Bederson, B.: Quantum Treemaps and Bubblemaps for a Zoomable Image Browser. In: ACM Symposium on User Interface Software and Technology, pp. 71–80 (2001)

    Google Scholar 

  5. Borth, D., Schulze, C., Ulges, A., Breuel, T.: Navidgator - Similarity Based Browsing for Image and Video Databases. In: German Conference on Advances in Artificial Intelligence, pp. 22–29 (2008)

    Google Scholar 

  6. Chen, C.: Information Visualization. Springer, Heidelberg (2004)

    Google Scholar 

  7. Chen, C., Gagaudakis, G., Rosin, P.: Similarity-Based Image Browsing. In: International Conference on Intelligent Information Processing, pp. 206–213 (2000)

    Google Scholar 

  8. Chen, Y., Butz, A.: Photosim: Tightly Integrating Image Analysis into a Photo Browsing UI. In: International Symposium on Smart Graphics (2008)

    Google Scholar 

  9. Combs, T., Bederson, B.: Does Zooming Improve Image Browsing? In: ACM Conference on Digital Libraries, pp. 130–137 (1999)

    Google Scholar 

  10. Cruz-Neira, C., Sandin, D., DeFanti, T.: Surround-screen Projection-based Virtual Reality: The Design and Implementation of the CAVE. In: 20th Annual Conference on Computer Graphics and Interactive Techniques, pp. 135–142 (1993)

    Google Scholar 

  11. Datta, R., Joshi, D., Li, J., Wang, J.Z.: Image Retrieval: Ideas, Influences, and Trends of the New Age. ACM Computing Surveys 40(2), 1–60 (2008)

    CrossRef  Google Scholar 

  12. Deng, D., Zhang, J., Purvis, M.: Visualisation and Comparison of Image Collections based on Self-organised Maps. In: Workshop on Australasian Information Security, Data Mining and Web Intelligence, and Software Internationalisation, pp. 97–102 (2004)

    Google Scholar 

  13. Dontcheva, M., Agrawala, M., Cohen, M.: Metadata Visualization for Image Browsing. In: ACM Symposium on User Interface Software and Technology (2005)

    Google Scholar 

  14. Eidenberger, H.: A Video Browsing Application Based on Visual MPEG-7 Descriptors and Self-organising Maps. International Journal of Fuzzy Systems 6(3) (2004)

    Google Scholar 

  15. Faloutsos, C., Equitz, W., Flickner, M., Niblack, W., Petkovic, D., Barber, R.: Efficient and Effective Querying by Image Content. Journal of Intelligent Information Systems 3, 231–262 (1994)

    CrossRef  Google Scholar 

  16. Faloutsos, C., Lin, K.: FastMap: A Fast Algorithm for Indexing, Datamining and Visualization of Traditional and Multimedia Datasets. In: ACM SIGMOD International Conference on Management of Data, pp. 163–174 (1995)

    Google Scholar 

  17. Flickr (2009), http://www.flickr.com/

  18. Gomi, A., Miyazaki, R., Itoh, T., Li, J.: CAT: A Hierarchical Image Browser Using a Rectangle Packing Technique. In: International Conference on Information Visualization, pp. 82–87 (2008)

    Google Scholar 

  19. Graham, A., Garcia-Molina, H., Paepcke, A., Winograd, T.: Time as Essence for Photo Browsing through Personal Digital Libraries. In: ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 326–335 (2002)

    Google Scholar 

  20. Gupta, A., Jain, R.: Visual Information Retrieval. Communications of the ACM 40(5), 70–79 (1997)

    CrossRef  Google Scholar 

  21. Hare, J.S., Lewis, P.H.: Content-based Image Retrieval Using a Mobile Device as a Novel Interface. In: SPIE Storage and Retrieval Methods and Applications for Multimedia, pp. 64–75 (2005)

    Google Scholar 

  22. Heesch, D., Rüger, S.M.: NNk Networks for Content-Based Image Retrieval. In: McDonald, S., Tait, J.I. (eds.) ECIR 2004. LNCS, vol. 2997, pp. 253–266. Springer, Heidelberg (2004)

    CrossRef  Google Scholar 

  23. Heesch, D., Rüger, S.: Three Interfaces for Content-Based Access to Image Collections. In: International Conference on Image and Video Retrieval, pp. 491–499 (2004)

    Google Scholar 

  24. Hilliges, O., Baur, D., Butz, A.: Photohelix: Browsing, Sorting and Sharing Digital Photo Collections. In: IEEE Tabletop Workshop on Horizontal Interactive Human-Computer Systems, pp. 87–94 (2007)

    Google Scholar 

  25. Hilliges, O., Kunath, P., Pryakhin, A., Butz, A., Kriegel, H.P.: Browsing and Sorting Digital Pictures using Automatic Image Classification and Quality Analysis. In: International Conference on Human-Computer Interaction, pp. 882–891 (2007)

    Google Scholar 

  26. Hinton, G., Roweis, S.: Stochastic Neighbor Embedding. In: Advances in Neural Information Processing Systems, vol. 15, pp. 833–840 (2002)

    Google Scholar 

  27. Apple iPhoto (2009), http://www.apple.com/ilife/iphoto/

  28. Jacobs, C.E., Finkelstein, A., Salesin, D.H.: Fast Multiresolution Image Querying. In: Conference on Computer Graphics and Interactive Techniques, pp. 277–286 (1995)

    Google Scholar 

  29. Jain, A.K., Dubes, R.C.: Algorithms for Clustering Data. Prentice-Hall, Englewood Cliffs (1988)

    MATH  Google Scholar 

  30. Jambu, M.: Exploratory and Multivariate Data Analysis. Academic Press, London (1991)

    MATH  Google Scholar 

  31. Keller, I., Meiers, T., Ellerbrock, T., Sikora, T.: Image Browsing with PCA-Assisted User-Interaction. In: IEEE Workshop on Content-Based Access of Image and Video Libraries, pp. 102–108 (2001)

    Google Scholar 

  32. Khella, A., Bederson, B.: Pocket PhotoMesa: A Zooming Image Browser for PDA’s. In: International Conference on Mobile and Ubiquitous Multimedia, pp. 19–24 (2004)

    Google Scholar 

  33. Kohonen, T.: Self-organizing Maps. Springer, Heidelberg (1997)

    MATH  Google Scholar 

  34. Koikkalainen, P., Oja, E.: Self-organizing Hierarchical Feature Maps. In: International Joint Conference on Neural Networks, vol. 2, pp. 279–285 (1990)

    Google Scholar 

  35. Krischnamachari, S., Abdel-Mottaleb, M.: Image Browsing using Hierarchical Clustering. In: IEEE Symposium Computers and Communications, pp. 301–307 (1999)

    Google Scholar 

  36. Kruskal, J.B., Wish, M.: Multidimensional Scaling. Sage, Thousand Oaks (1978)

    Google Scholar 

  37. Laaksonen, J., Koskela, M., Oja, E.: PicSOM – Self-organizing Image Retrieval with MPEG-7 Content Descriptors. IEEE Transactions on Neural Networks: Special Issue on Multimedia Processing 13(4), 841–853 (2002)

    Google Scholar 

  38. Lew, M.S., Sebe, N.: Visual Websearching Using Iconic Queries. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 788–789 (2000)

    Google Scholar 

  39. Li, J., Wang, J.Z.: Real-Time Computerized Annotation of Pictures. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(6), 985–1002 (2008)

    CrossRef  Google Scholar 

  40. Lim, S., Chen, L., Lu, G., Smith, R.: Browsing Texture Image Databases. In: International Conference on Multimedia Modelling, pp. 328–333 (2005)

    Google Scholar 

  41. Linde, Y., Buzo, A., Gray, R.: An Algorithm for Vector Quantizer Design. IEEE Transactions on Communications 28, 84–94 (1980)

    CrossRef  Google Scholar 

  42. Liu, H., Xie, X., Tang, X., Li, Z.W., Ma, W.Y.: Effective Browsing of Web Image Search Results. In: ACM International Workshop on Multimedia Information Retrieval, pp. 84–90 (2004)

    Google Scholar 

  43. Ma, W.Y., Manjunath, B.S.: NeTra: A Toolbox for Navigating Large Image Databases. Multimedia Systems 7(3), 184–198 (1999)

    CrossRef  Google Scholar 

  44. Milanese, R., Squire, D., Pun, T.: Correspondence Analysis and Hierarchical Indexing for Content-Based Image Retrieval. In: IEEE International Conference on Image Processing, pp. 859–862 (1996)

    Google Scholar 

  45. Moghaddam, B., Tian, Q., Lesh, N., Shen, C., Huang, T.: Visualization and User-Modeling for Browsing Personal Photo Libraries. International Journal of Computer Vision 56(1/2), 109–130 (2004)

    CrossRef  Google Scholar 

  46. Müller, H., Müller, W., Squire, D.M., Marchand-Maillet, S., Pun, T.: Performance Evaluation in Content-Based Image Retrieval: Overview and Proposals. Pattern Recognition Letters 22(5), 593–601 (2001)

    CrossRef  MATH  Google Scholar 

  47. Nakazato, M., Huang, T.: 3D MARS: Immersive Virtual Reality for Content-Based Image Retrieval. In: IEEE International Conference on Multimedia and Expo., pp. 44–47 (2001)

    Google Scholar 

  48. Nakazato, M., Manola, L., Huang, T.: ImageGrouper: A Group-Oriented User Interface for Content-Based Image Retrieval and Digital Image Arrangement. Journal of Visual Language and Computing 14(4), 363–386 (2003)

    CrossRef  Google Scholar 

  49. Nguyen, G.P., Worring, M.: Interactive Access to Large Image Collections using Similarity Based Visualization. Journal of Visual Languages and Computing 19, 203–224 (2008)

    CrossRef  Google Scholar 

  50. Osman, T., Thakker, D., Schaefer, G., Lakin, P.: An Integrative Semantic Framework for Image Annotation and Retrieval. In: IEEE/WIC/ACM International Conference on Web Intelligence, pp. 366–373 (2007)

    Google Scholar 

  51. Pecenovic, Z., Do, M.N., Vetterli, M., Pu, P.: Integrated Browsing and Searching of Large Image Collections. In: Laurini, R. (ed.) VISUAL 2000. LNCS, vol. 1929, pp. 279–289. Springer, Heidelberg (2000)

    CrossRef  Google Scholar 

  52. Pentland, A., Picard, W.R., Sclaroff, S.: Photobook: Content-Based Manipulation of Image Databases. International Journal of Computer Vision 18(3), 233–254 (1996)

    CrossRef  Google Scholar 

  53. Google Picasa (2009), http://picasa.google.com/

  54. Plaisant, C., Carr, D., Shneiderman, B.: Image Browsers: Taxonomy, Guidelines, and Informal Specifications. IEEE Software 12, 21–32 (1995)

    CrossRef  Google Scholar 

  55. Platt, J., Czerwinski, M., Field, B.: PhotoTOC: Automatic Clustering for Browsing Personal Photographs. Technical report, Microsoft Research (2002)

    Google Scholar 

  56. Platt, J.C.: AutoAlbum: Clustering Digital Photographs using Probalistic Model Merging. In: IEEE Workshop on Content-Based Access of Image and Video Libraries, pp. 96–100 (2000)

    Google Scholar 

  57. Porta, M.: New Visualization Modes for Effective Image Presentation. International Journal of Image and Graphics 9(1), 27–49 (2009)

    CrossRef  Google Scholar 

  58. Rodden, K.: Evaluating Similarity-Based Visualisations as Interfaces for Image Browsing. PhD thesis, University of Cambridge Computer Laboratory (2001)

    Google Scholar 

  59. Rodden, K., Basalaj, W., Sinclair, D., Wood, K.: Evaluating a Visualisation of Image Similarity as a Tool for Image Browsing. In: IEEE Symposium on Information Visualisation, pp. 36–43 (1999)

    Google Scholar 

  60. Rodden, K., Basalaj, W., Sinclair, D., Wood, K.: A Comparison of Measures for Visualising Image Similarity. In: The Challenge of Image Retrieval (2000)

    Google Scholar 

  61. Rodden, K., Basalaj, W., Sinclair, D., Wood, K.: Does Organisation by Similarity Assist Image Browsing? In: SIGCHI Conference on Human Factors in Computing Systems, pp. 190–197 (2001)

    Google Scholar 

  62. Rodden, K., Wood, K.: How Do People Manage Their Digital Photographs? In: SIGCHI Conference on Human Factors in Computing Systems, pp. 409–416 (2003)

    Google Scholar 

  63. Roweis, S., Saul, L.: Nonlinear Dimensionality Reduction by Locally Linear Embedding. Science 290(5500), 2323–2326 (2000)

    CrossRef  Google Scholar 

  64. Rubner, Y., Guibas, L.J., Tomasi, C.: The Earth Movers Distance, Multi-dimensional Scaling, and Color-based Image Retrieval. In: APRA Image Understanding Workshop, pp. 661–668 (1997)

    Google Scholar 

  65. Rui, Y., Huang, T.S., Ortega, M., Mehrotra, M.: Relevance Feedback: A Power Tool for Interactive Content-based Image Retrieval. IEEE Transaction on Circuits and Systems for Video Technology 8(5), 644–655 (1998)

    CrossRef  Google Scholar 

  66. Ruszala, S., Schaefer, G.: Visualisation Models for Image Databases: A Comparison of Six Approaches. In: Irish Machine Vision and Image Processing Conference, pp. 186–191 (2004)

    Google Scholar 

  67. Sammon, J.W.: A Nonlinear Mapping for Data Structure Analysis. IEEE Transactions on Computers 18(5), 401–409 (1969)

    CrossRef  Google Scholar 

  68. Santini, S., Jain, R.: Integrated Browsing and Querying for Image Databases. IEEE Multimedia 7, 26–39 (2000)

    CrossRef  Google Scholar 

  69. Sarkar, M., Brown, M.: Graphical Fisheye Views. Communications of the ACM 37(12), 73–83 (1994)

    CrossRef  Google Scholar 

  70. Sarvas, R., Herrarte, E., Wilhelm, A., Davis, M.: Metadata Creation System for Mobile Images. In: International Conference on Mobile Systems, Applications, and Services, pp. 36–48 (2004)

    Google Scholar 

  71. Schaefer, G., Ruszala, S.: Image database navigation: A globe-al approach. In: Bebis, G., Boyle, R., Koracin, D., Parvin, B. (eds.) ISVC 2005. LNCS, vol. 3804, pp. 279–286. Springer, Heidelberg (2005)

    CrossRef  Google Scholar 

  72. Schaefer, G., Ruszala, S.: Image Database Navigation on a Hierarchical Hue Sphere. In: International Symposium on Visual Computing, pp. 814–823 (2006)

    Google Scholar 

  73. Schaefer, G., Ruszala, S.: Image Database Navigation on a Hierarchical MDS Grid. In: 28th Pattern Recognition Symposium, pp. 304–313 (2006)

    Google Scholar 

  74. Schaefer, G., Stich, M.: UCID – An Uncompressed Colour Image Database. In: Storage and Retrieval Methods and Applications for Multimedia, pp. 472–480 (2004)

    Google Scholar 

  75. Smeulders, A., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-Based Image Retrieval at the End of the Early Years. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(12), 1349–1380 (2000)

    CrossRef  Google Scholar 

  76. Swain, M., Ballard, D.: Color Indexing. International Journal of Computer Vision 7(1), 11–32 (1991)

    CrossRef  Google Scholar 

  77. Tao, D., Tang, X., Li, X., Rui, Y.: Direct Kernal Biased Discriminant Analysis: A New Content-Based Image Retrieval Relevance Feedback Algorithm. IEEE Transactions on Multimedia 8(4), 716–727 (2006)

    CrossRef  Google Scholar 

  78. Tao, D., Tang, X., Li, X., Wu, X.: Asymmetric Bagging and Random Subspace for Support Vector Machines-Based Relevance Feedback in Image Retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(7),1088–1099 (2006)

    CrossRef  Google Scholar 

  79. Tenenbaum, J., Silva, V., Langford, J.: A Global Geometric Framework for Nonlinear Dimensionality Reduction. Science 290(5500), 2319–2322 (2000)

    CrossRef  Google Scholar 

  80. Tian, G.Y., Taylor, D.: Colour Image Retrieval Using Virtual Reality. In: IEEE International Conference on Information Visualization, pp. 221–225 (2000)

    Google Scholar 

  81. Urban, J., Jose, J.M.: EGO: A Personalized Multimedia Management and Retrieval Tool. International Journal of Intelligent Systems 21(7), 725–745 (2006)

    CrossRef  MATH  Google Scholar 

  82. van Liere, R., de Leeuw, W.: Exploration of Large Image Collections Using Virtual Reality Devices. In: Workshop on New Paradigms in Information Visualization and Manipulation, held in conjunction with the 8th ACM International Conference on Information and Knowledge Management, pp. 83–86 (1999)

    Google Scholar 

  83. Worring, M., de Rooij, O., van Rijn, T.: Browsing Visual Collections Using Graphs. In: International Workshop on Multimedia Information Retrieval, pp. 307–312 (2007)

    Google Scholar 

  84. Yeh, T., Tollmar, K., Darrell, T.: Searching the Web with Mobile Images for Location Recognition. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 76–81 (2004)

    Google Scholar 

  85. Zhang, H., Zhong, D.: A Scheme for Visual Feature Based Image Indexing. In: SPIE/IS&T Conference on Storage and Retrieval for Image and Video Databases, pp. 36–46 (1995)

    Google Scholar 

  86. Zhou, X., Huang, T.: Relevance Feedback in Image Retrieval: A Comprehensive Review. Multimedia Systems 8(6), 536–544 (2003)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Engineering and Applied Science, Aston University, Birmingham, U.K.

    William Plant

  2. Department of Computer Science, Loughborough University, Loughborough, U.K.

    Gerald Schaefer

Authors
  1. William Plant
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gerald Schaefer
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Computer Engineering , Nanyang Technological University, 639798, Singapore

    Weisi Lin & Dacheng Tao & 

  2. Intelligent Systems Laboratory Systems Research Institute , Polish Academy of Sciences, Poland

    Janusz Kacprzyk

  3. Department of Computing , Hong Kong Polytechnic University, Hung Hom, Hong Kong

    Zhu Li

  4. School of Electronic Engineering and Computer Science, Queen Mary, University of London, London, U.K.

    Ebroul Izquierdo

  5. TCL-Thomson Electronics , Santa Clara, California

    Haohong Wang

Rights and permissions

Reprints and Permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Plant, W., Schaefer, G. (2011). Visualisation and Browsing of Image Databases. In: Lin, W., Tao, D., Kacprzyk, J., Li, Z., Izquierdo, E., Wang, H. (eds) Multimedia Analysis, Processing and Communications. Studies in Computational Intelligence, vol 346. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-19551-8_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-19551-8_1

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-19550-1

  • Online ISBN: 978-3-642-19551-8

  • eBook Packages: EngineeringEngineering (R0)