Skip to main content
SpringerLink
Log in

A survey of browsing models for content based image retrieval

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The problem of content based image retrieval (CBIR) has traditionally been investigated within a framework that emphasises the explicit formulation of a query: users initiate an automated search for relevant images by submitting an image or draw a sketch that exemplifies their information need. Often, relevance feedback is incorporated as a post-retrieval step for optimising the way evidence from different visual features is combined. While this sustained methodological focus has helped CBIR to mature, it has also brought out its limitations more clearly: There is often little support for exploratory search and scaling to very large collections is problematic. Moreover, the assumption that users are always able to formulate an appropriate query is questionable. An effective, albeit much less studied, method of accessing image collections based on visual content is that of browsing. The aim of this survey paper is to provide a structured overview of the different models that have been explored over the last one to two decades, to highlight the particular challenges of the browsing approach and to focus attention on a few interesting issues that warrant more intense research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Anderson J (1983) A spreading activation theory of memory. J Verbal Learn Verbal Behav 22:261–295

    Article  Google Scholar 

  2. Barnard K, Forsyth D (2001) Learning the semantics of words and pictures. In: Proc IEEE int’l conf computer vision, vol 2. IEEE, Piscataway, pp 408–415

    Google Scholar 

  3. Beckmann N, Kriegel H-P, Schneider R, Seeger B (1990) The R*-tree: an efficient and robust access method for points and rectangles. In: Proc int’l conf management of data, Atlantic City, 23–26 May 1990, pp 322–331

  4. Bentley J (1975) Multidimensional binary search trees used for associative searching. Commun ACM 18(9):509–517

    Article  MATH  MathSciNet  Google Scholar 

  5. Berkhin P (2002) Survey of clustering data mining techniques. Technical report, Accrue Software, San Jose, CA

  6. Beyer K, Goldstein J, Ramakrishnan R, Shaft U (1999) When is ‘nearest neighbour’ meaningful? In: Proc int’l conf data theory, Jerusalem, 10–12 January 1999, pp 217–235

  7. Boucheron L, Creusere C (2005) Lossless wavelet-based compression of digital elevation maps for fast and efficient search and retrieval. IEEE Trans Geosci Remote Sens 43(5):1210–1214

    Article  Google Scholar 

  8. Browne P, Smeaton A (2004) Video information retrieval using objects and ostensive relevance feedback. In: ACM symp applied computing. ACM, New York, pp 1084–1090

    Chapter  Google Scholar 

  9. Campbell I (2000) The ostensive model of developing information-needs. PhD thesis, University of Glasgow

  10. Carmel E, Crawford S, Chen H (1992) Browsing in hypertext: a cognitive study. IEEE Trans Syst Man Cybern 22:865–884

    Article  Google Scholar 

  11. Chen C, Kuljis J (2003) The rising landscape: a visual exploration of superstring revolutions in physics. J Am Soc Inf Sci Technol 54(5):435–446

    Article  Google Scholar 

  12. Chen C, Morris S (2003) Visualizing evolving networks: minimum spanning trees versus pathfinder networks. In: IEEE symp information visualization. IEEE, Piscataway, pp 67–74

    Google Scholar 

  13. Chen C, Gagaudakis G, Rosin P (2000) Similarity-based image browsing. In: Proc int’l conf intelligent information processing, Beijing, 22 August 2000, pp 206–213

  14. Chen J, Bouman C, Dalton J (1998) Similarity pyramids for browsing and organization of large image databases. In: Proc SPIE conf human vision and electronic imaging III, vol 3299. SPIE, Bellingham, pp 563–575

    Google Scholar 

  15. Chen J, Bouman C, Dalton J (2000) Hierarchical browsing and search of large image databases. IEEE Trans Image Process 9(3):442–455

    Article  Google Scholar 

  16. Cheung S, Zakhor A (2005) Fast similarity search and clustering of video sequences on the World-Wide-Web. IEEE Trans Multimedia 7(3):524–537

    Article  Google Scholar 

  17. Clough P, Joho H, Sanderson M (2005) Automatically organising images using concept hierarchies. In: Proc ACM multimedia workshop (SIGIR), Singapore, 6–11 November 2005

  18. Cox K (1992) Information retrieval by browsing. In: Proc int’l conf new information technology, Hong Kong, 30 November–2 December 1992

  19. Cox K (1995) Searching through browsing. PhD thesis, University of Canberra

  20. Croft B, Parenty T (1985) Comparison of a network structure and a database system used for document retrieval. Inf Syst 10:377–390

    Article  Google Scholar 

  21. Crucianu M, Ferecatu M, Boujemaa N (2004) Relevance feedback for image retrieval: a short review. In: State of the art in audiovisual content-based retrieval, information universal access and interaction including datamodels and languages (DELOS2 report)

  22. Datta R, Joshi D, Li J, Wang J (2008) Image retrieval: ideas, influences and trends of the new age. ACM Trans Comput Surv (in press)

  23. Descampe A, Vleeschouwer C, Iregui M, Macq N, Marqués F (2007) Prefetching and caching strategies for remote and interactive browsing of JPEG2000 images. IEEE Trans Image Process 16(5):1339–1354

    Article  Google Scholar 

  24. Duda R, Hart P, Stork D (2001) Pattern recognition. Wiley, New York

    Google Scholar 

  25. Fauqueur J, Boujemaa N (2006) Mental image search by boolean composition of region categories. Multimed Tools Appl 31(1):95–117

    Article  Google Scholar 

  26. Feng S, Manmatha R, Lavrenko V (2004) Multiple Bernoulli relevance models for image and video annotation. In: Proc int’l conf computer vision and pattern recognition. IEEE, Piscataway, pp 1002–1009

    Google Scholar 

  27. Forsyth D (2001) Benchmarks for storage and retrieval in multimedia databases. In: Proc SPIE conf storage and retrieval for media databases, vol 4676. SPIE, Bellingham, pp 240–247

    Google Scholar 

  28. Fowler R, Wilson B, Fowler W (1992) Information navigator: an information system using associative networks for display and retrieval. Technical report, Department of Computer Science, University of Texas, No. 92-1

  29. Fukunaga K, Narendra P (1975) A branch and bound algorithm for computing k-nearest neighbors. IEEE Trans Comput 24(7):750–753

    Article  MATH  MathSciNet  Google Scholar 

  30. Furnas G (1986) Generalized fisheye views. In: Proc SIGCHI conf human factors in computing systems, Boston, 13–17 April 1986, pp 16–23

  31. Gevers T, Smeulders A (2004) Content-based image retrieval: an overview. In: Medioni G, Kang S (eds) Emerging topics in computer vision. Prentice Hall, Englewood Cliffs

    Google Scholar 

  32. Goldberger J, Gordon S, Greenspan H (2006) Unsupervised image set clustering using an information theoretic framework. IEEE Trans Image Process 15(2):449–458

    Article  Google Scholar 

  33. Gupta A, Jain R (1997) Visual information retrieval. Commun ACM 40(5):71–79

    Article  Google Scholar 

  34. Guttmann A (1984) R-trees: a dynamic index structure for spatial searching. In: Proc ACM int’l conf management of data (SIGMOD), ACM, New York, pp 47–57

    Google Scholar 

  35. Heesch D (2005) The NNk technique for image searching and browsing. PhD thesis, Imperial College London

  36. Heesch D, Rüger S (2004) NNk networks for content-based image retrieval. In: Proc European conf information retrieval, LNCS 2997. Springer, Berlin Heidelberg New York, pp 253–266

    Google Scholar 

  37. Heesch D, Rüger S (2006) Interaction models and relevance feedback in content-based image retrieval. In: Zhang Y-J (ed) Semantic-based visual information retrieval. Idea-Group, Harrisburg, pp 160–186

    Google Scholar 

  38. Heesch D, Pickering M, Yavlinsky A, Rüger S (2004) Video retrieval within a browsing framework using keyframes. In: Proc TREC video. NIST, Gaithersburg

    Google Scholar 

  39. Heesch D, Yavlinsky A, Rüger S (2006) NNk networks and automated annotation for browsing large image collections from the World Wide Web. In: Proc ACM int’l conf multimedia (SIGMM). ACM, New York, pp 220–224

    Google Scholar 

  40. Hinton G, Roweis S (2002) Stochastic neighbour embedding. In: Advances in neural information processing systems, vol 15. MIT, Cambridge, pp 833–840

    Google Scholar 

  41. Hiroike T, Musha Y, Sugimoto A, Mori Y (1999) Visualization of information spaces to retrieve and browse image data. In: Visual information systems. Morgan Kaufmann, San Francisco, pp 155–162

    Google Scholar 

  42. Jacobs C, Finkelstein A, Salesin D (1995) Fast multiresolution image querying. Technical report, University of Washington, US

  43. Katayama N, Satoh S (1997) SR-tree: an index structure for high-dimensional nearest neighbour queries. In: Proc ACM int’l conf management of data (SIGMOD), ACM, New York, pp 369–380

    Chapter  Google Scholar 

  44. Keller I, Meiers T, Ellerbrock T, Sikora T (2001) Image browsing with PCA-assisted user-interaction. In: IEEE workshop content-based access of image and video libraries. IEEE, Piscataway, pp 102–108

    Chapter  Google Scholar 

  45. Kohonen T (2001) Self-organizing maps, volume 30 of Springer series in information sciences. Springer, Berlin Heidelberg New York

    Google Scholar 

  46. Krishnamachari S, Abdel-Mottaleb M (1999) Image browsing using hierarchical clustering. In: IEEE symp computers and communications. IEEE, Piscataway, pp 301–307

    Google Scholar 

  47. Kurniawati R, Jin J, Shepherd J (1997) Techniques for supporting efficient content-based retrieval in multimedia databases. Aust Comput J 29(4):122–130

    Google Scholar 

  48. Laaksonen J, Oja E, Koskela M, Brandt S (2000) Analyzing low-level visual features using content-based image retrieval. In: Proc int’l conf neural information processing, Taejon, 14–18 November 2000

  49. Lavrenko V, Manmatha R, Jeon J (2003) A model for learning the semantics of pictures. In: Advances in neural information processing systems, vol 16. MIT, Cambridge

    Google Scholar 

  50. Lim S, Chen L, Lu G, Smith R (2005) Browsing texture image databases. In: Proc int’l conf multimedia modelling. IEEE, Piscataway, pp 328–333

    Google Scholar 

  51. Liu T, Joung Y (2004) Multi-dimension browse. In: Proc IEEE int’l conf computer software and applications. IEEE, Piscataway, pp 480–485

    Google Scholar 

  52. MacCuish J, McPherson A, Barros J, Kelly P (1996) Interactive layout mechanisms for image database retrieval. In: Proc SPIE conf visual data exploration and analysis III, vol 2656. SPIE, Bellingham, pp 104–115

    Google Scholar 

  53. Milanese R, Squire D, Pun T (1996) Correspondence analysis and hierarchical indexing for content-based image retrieval. In: Proc IEEE int’l conf image processing. IEEE, Piscataway, pp 859–862

    Chapter  Google Scholar 

  54. Minka T, Picard R (1996) Interactive learning using a society of models. In: Proc IEEE conf computer vision and pattern recognition. IEEE, Piscataway, pp 447–452

    Google Scholar 

  55. Moghaddam B, Tian Q, Lesh N, Shen C, Huang T (2004) Visualization and user-modeling for browsing personal photo libraries. Int J Comput Vis 56(1–2):109–130

    Article  Google Scholar 

  56. Mukhopadhyay R, Ma A, Sethi I (2004) Pathfinder networks for content based image retrieval based on automated shape feature discovery. In: Proc IEEE int’l symp multimedia software engineering. IEEE, Piscataway, pp 522–528

    Chapter  Google Scholar 

  57. Musha Y, Hiroike A, Mori Y, Sugimoto A (1998) An interface for visualizing feature space in image retrieval. In: Proc IAPR workshop machine vision applications, Chiba, 17–19 November 1998, pp 447–450

  58. Newell A (1990) Unified theories of cognition. Harvard University Press, Cambridge

    Google Scholar 

  59. Nguyen G, Worring M (2008) Interactive access to large image collections using similarity-based visualization. J Vis Lang Comput (in press)

  60. Obdržálek S, Matas J (2005) Sub-linear indexing for large scale object recognition. In: Proc conf British machine vision, Versailles, 5–8 September 2005, pp 1–10

  61. Pečenović Z, Do M, Vetterli M, Pu P (2000) Integrated browsing and searching of large image collections. In: Proc int’l conf advances in visual information systems, LNCS 1929. Springer, Berlin Heidelberg New York, pp 279–289

    Google Scholar 

  62. Platt J, Czerwinski M, Field B (2002) PhotoTOC: automatic clustering for browsing personal photographs. Technical report, Microsoft Research

  63. Rodden K, Basalaj W, Sinclair D, Wood K (2001) Does organization by similarity assist image browsing? In: Proc int’l conf computer human interaction, New Orleans, 5–10 August 2001, pp 190–197

  64. Rogers T, McClelland J (2006) Semantic cognition: a parallel distributed processing approach. MIT, Cambridge

    Google Scholar 

  65. Roussinov D, Chen H (1998) A scalable self-organizing map algorithm for textual classification: a neural network approach to thesaurus generation. Commun Cogn 15(1–2):81–112

    Google Scholar 

  66. Roussopoulos N, Kelley S, Vincent F (1995) Nearest neighbor queries. In: Proc ACM int’l conf management of data (SIGMOD). ACM, New York

    Google Scholar 

  67. Roweis S, Saul L (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Article  Google Scholar 

  68. Rubner Y, Guibas L, Tomasi C (1997) The earth mover’s distance, multi-dimensional scaling, and color-based image retrieval. In: Proc ARPA image understanding workshop, New Orleans, May 1997, pp 661–668

  69. Rubner Y, Tomasi C, Guibas L (1998) A metric for distributions with applications to image databases. In: Proc IEEE int’l conf computer vision. IEEE, Piscataway, pp 59–66

    Google Scholar 

  70. Salton G, Buckley C (1988) On the use of spreading activation methods in automatic information. In: Proc ACM int’l conf information retrieval (SIGIR). ACM, New York, pp 147–160

    Google Scholar 

  71. Sammon J (1969) A nonlinear mapping for data structure analysis. IEEE Trans Comput C-18(5):401–409

    Article  Google Scholar 

  72. Santini S, Jain R (2000) Integrated browsing and querying for image databases. IEEE Multimed Mag 7(3):26–39

    Article  Google Scholar 

  73. Santini S, Gupta A, Jain R (2001) Emergent semantics through interaction in image databases. IEEE Trans Knowl Data Eng 13(3):337–351

    Article  Google Scholar 

  74. Schvanefeldt R (1990) Pathfinder associative networks: studies in knowledge organization. In: Ablex series in computational sciences. Ablex, Norwood

    Google Scholar 

  75. Schvaneveldt R, Durso F, Dearholt D (1989) Network structures in proximity data. In: Bower GH (ed) The psychology of learning and motivation. Academic, London, pp 249–284

    Google Scholar 

  76. Sclaroff S, Taycher L, La Cascia M (1997) ImageRover: a content-based image browser for the World Wide Web. Technical report, Boston University

  77. Sloutsky V (2003) The role of similarity in the development of categorization. Trends Cogn Sci 7(6):246–252

    Article  Google Scholar 

  78. Smeulders A, Worring M, Santini S, Gupta A, Jain R (2000) Content based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22(12):1349–1380

    Article  Google Scholar 

  79. Smith J, Chang S-F (1996) VisualSEEk: a fully automated content-based image query system. In: Proc ACM int’l conf multimedia (SIGMM). ACM, New York

    Google Scholar 

  80. Spence R (1999) A framework for navigation. Int J Hum Comput Stud 51:919–945

    Article  Google Scholar 

  81. Tenenbaum J (2006) Theory-based bayesian models of inductive learning and reasoning. Trends Cogn Sci 10(7):309–317

    Article  Google Scholar 

  82. Tenenbaum J, de Silva V, Langford J (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(5500):2319–2323

    Article  Google Scholar 

  83. Tishby N, Pereira F, Bialek W (1999) The information bottleneck method. In: Proc allerton conf communication, control and computing, Monticello, September 1999, pp 368–377

  84. Urban J, Jose J, van Rijsbergen C (2003) An adaptive approach towards content-based image retrieval. In: Proc int’l workshop content-based multimedia indexing, Rennes, 22–24 September, pp 119–126

  85. Vendrig J, Worring M, Smeulders A (1999) Filter image browsing: exploiting interaction in image retrieval. In: Visual information and information systems, Amsterdam, 2–4 June 1999, pp 147–154

  86. Wang Q, You S (2006) Fast similarity search for high-dimensional datasets. In: Proc IEEE int’l symp multimedia. IEEE, Piscataway

    Google Scholar 

  87. Weber R, Blott S (1997) An approximation based data structure for similarity search. Technical Report 24, ETH Zurich, Switzerland

  88. Weber R, Schek J-J, Blott S (1998) A quantitative analysis and performance study for similarity-search methods in high-dimensional space. In: Proc int’l conf very large databases, New York, 24–27 August 1998, pp 194–205

  89. White D, Jain R (1996) Similarity indexing with the SS-tree. In: Proc IEEE int’l conf data engineering. IEEE, Piscataway, pp 516–523

    Google Scholar 

  90. Wolfram S (2004) A new kind of science. Wolfram, Champaign

    Google Scholar 

  91. Yang C (2004) Content-based image retrieval: a comparison between query by example and image browsing map approaches. J Inf Sci 30(3):254–267

    Article  Google Scholar 

  92. Yang J, Fan J, Hubball D, Gao Y, Luo H, Ribarsky W, Ward M (2006) Semantic image browser: bridging information visualization with automated intelligent image analysis. In: IEEE symp visual analytics science and technology. IEEE, Piscataway, pp 191–198

    Chapter  Google Scholar 

  93. Yavlinsky A, Heesch D (2007) An online system for gathering image similarity judgements. In: Proc ACM int’l conf multimedia (SIGMM). ACM, New York, pp 565–568

    Google Scholar 

  94. Yavlinsky A, Schofield E, Rüger S (2005) Automated image annotation using global features and robust nonparametric density estimation. In: Proc int’l conf video and image retrieval, LNCS 3568. Springer, Berlin Heidelberg New York, pp 507–517

    Google Scholar 

  95. Yeung M, Liu B (1995) Efficient matching and clustering of video shots. In: Proc IEEE int’l conf image processing. IEEE, Piscataway, pp 338–341

    Chapter  Google Scholar 

  96. Yeung M, Yeo B-L (1997) Video visualization for compact presentation and fast browsing of pictorial content. IEEE Trans Circuits Syst Video Technol 7(5):771–785

    Article  Google Scholar 

  97. Zass R, Shashua A (2005) A unified treatment of hard and probabilistic clustering methods. In: Proc int’l conf computer vision. IEEE, Piscataway

    Google Scholar 

  98. Zhang H, Zhong D (1995) A scheme for visual feature based image indexing. In: Proc SPIE/IS&T conf storage and retrieval for image and video databases III, vol 2420. SPIE, Bellingham, pp 36–46

    Google Scholar 

  99. Zhang R, Zhang Z, Li M, Ma W-Y, Zhang H-J (2005) A probabilistic semantic model for image annotation and multi-modal image retrieval. In: Proc int’l conf computer vision. IEEE, Piscataway, pp 846–851

    Google Scholar 

  100. Zhou X, Huang T (2003) Relevance feedback in image retrieval: a comprehensive review. ACM Multimed Syst 8(6):536–544

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Electronic Engineering, Imperial College London, SW7 2AZ, London, UK

    Daniel Heesch

Authors
  1. Daniel Heesch
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniel Heesch.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Heesch, D. A survey of browsing models for content based image retrieval. Multimed Tools Appl 40, 261–284 (2008). https://doi.org/10.1007/s11042-008-0207-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-008-0207-2

Keywords

Search

Navigation