Skip to main content
SpringerLink
Log in

Relevance Feedback and Learning in Content-Based Image Search

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

A major bottleneck in content-based image retrieval (CBIR) systems or search engines is the large gap between low-level image features used to index images and high-level semantic contents of images. One solution to this bottleneck is to apply relevance feedback to refine the query or similarity measures in image search process. In this paper, we first address the key issues involved in relevance feedback of CBIR systems and present a brief overview of a set of commonly used relevance feedback algorithms. Almost all of the previously proposed methods fall well into such framework. We present a framework of relevance feedback and semantic learning in CBIR. In this framework, low-level features and keyword annotations are integrated in image retrieval and in feedback processes to improve the retrieval performance. We have also extended framework to a content-based web image search engine in which hosting web pages are used to collect relevant annotations for images and users' feedback logs are used to refine annotations. A prototype system has developed to evaluate our proposed schemes, and our experimental results indicated that our approach outperforms traditional CBIR system and relevance feedback approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. C. Buckley and G. Salton, “Optimization of relevance feedback weights,” in Proceedings of SIGIR'95, 1995.

  2. S. K. Chang, C. W. Yan, D. C. Dimitroff, and T. Arndt, “An intelligent image database system,” IEEE Transactions on Software Engineering 14(5), 1988.

  3. Z. Chen, W. Liu, C. Hu, M. Li, and H. J. Zhang, “iFind: A web image search engine,” in Proceedings of SIGIR2001, 2001.

  4. Z. Chen, W. Liu, F. Zhang, M. Li, and H. J. Zhang, “Web mining for web image retrieval,” Journal of the American Society for Information Science and Technology 52(10), August 2001, 831-839.

    Google Scholar 

  5. I. J. Cox, T. P. Minka, T. V. Papathomas, and P. N. Yianilos, “The Bayesian image retrieval system, PicHunter: Theory, implementation, and psychophysical experiments,” IEEE Transactions on Image Processing, Special Issue on Digital Libraries, 2000.

  6. M. Flickner, H. Sawhney, W. Niblack et al., “Query by image and video content: The QBIC system,” IEEE Computer Magazine 28, 1995, 23-32.

    Google Scholar 

  7. J. Huang, S. R. Kumar, and M. Metra, “Combining supervised learning with color correlograms for contentbased image retrieval,” in Proceedings of ACM Multimedia'95, November 1997, pp. 325-334.

  8. Y. Ishikawa, R. Subramanya, and C. Faloutsos, “Mindreader: Query databases through multiple examples,” in Proceedings of the 24th VLDB Conference, New York, 1998.

  9. F. Jing, M. Li, H. J. Zhang, and B. Zhang, “An effective region-based image retrieval framework,” in Proceedings of ACM Multimedia 2002, Juan-les-Pins, France, December 1-6, 2002.

  10. J. Laaksonen, M. Koskela, and E. Oja, “PicSOM: Self-organizing maps for content-based image retrieval,” in Proceedings of International Joint Conference on NN, July 1999.

  11. C. Lee, W. Y. Ma, and H. J. Zhang, “Information embedding based on user's relevance feedback for image retrieval,” in Proceedings of SPIE International Conference on Multimedia Storage and Archiving Systems IV, Boston, 19-22 September 1999.

  12. Y. Lu et al., “A unified framework for semantics and feature based relevance feedback in image retrieval systems,” in Proceedings of ACM MM2000, 2000.

  13. S. D. MacArthur, C. E. Brodley, and C.-R. Shyu, “Relevance feedback decision trees in content-based image retrieval,” in IEEE Workshop on Content-Based Access of Image and Video Libraries, 2000, pp. 68-72.

  14. T. Minka and R. Picard, “Interactive learning using a 'Society of Models',” Pattern Recognition 30(4), 1997.

  15. T. Mitchell, Machine Learning, McGraw-Hill, 1997.

  16. J. J. Rocchio Jr., “Relevance feedback in information retrieval,” in The SMART Retrieval System: Experiments in Automatic Document Processing, ed. G. Salton, Prentice-Hall, 1971, pp. 313-323.

  17. Y. Rui and T. S. Huang, “A novel relevance feedback technique in image retrieval,” in Proceedings of 7th ACM Conference on Multimedia, 1999.

  18. Y. Rui, T. S. Huang, and S. Mehrotra, “Content-based image retrieval with relevance feedback in MARS,” in Proceedings of IEEE International Conference on Image Processing, 1997.

  19. G. Salton, Automatic Text Processing, Addison-Wesley, Reading, MA, 1989.

    Google Scholar 

  20. G. Salton and M. McGill, Introduction to Modern Information Retrieval, McGraw-Hill, 1983.

  21. S. Sclaroff, L. Taycher, and M. L. Cascia, “ImageRover: a content-based image browser for theWorldWide Web,” Technical Report 97-005, Boston University CS Dept., 1997.

  22. H. T. Shen, B. C. Ooi, and K. L. Tan, “Giving meanings to WWW images,” in Proceedings of ACM MM2000, 2000, pp. 39-48.

  23. Z. Su, S. Li, and H. J. Zhang, “Extraction of feature subspaces for content-based retrieval using relevance feedback,” in ACM Multimedia 2001, Ottawa, Canada, 2001.

    Google Scholar 

  24. Z. Su, H. J. Zhang, and S. Ma, “Relevant feedback using a Bayesian classifier in content-based image retrieval,” in SPIE Electronic Imaging 2001, San Jose, CA, January 2001.

    Google Scholar 

  25. K. Tieu and P. Viola, “Boosting image retrieval,” in IEEE Conference on Computer Vision and Pattern Recognition, 2000.

  26. S. Tong and E. Chang, “Support vector machine active leaning for image retrieval,” in ACM Multimedia 2001, Ottawa, Canada, 2001.

    Google Scholar 

  27. N. Vasconcelos and A. Lippman, “Learning from user feedback in image retrieval systems,” in NIPS'99, Denver, CO, 1999.

  28. P. Wu and B. S. Manjunath, “Adaptive nearest neighbour search for relevance feedback in large image database,” in ACM Multimedia Conference, Ottawa, Canada, 2001.

  29. Y. Wu, Q. Tian, and T. S. Huang, “Discriminant EM algorithm with application to image retrieval,” in IEEE CVPR, South Carolina, 2000.

  30. H. J. Zhang and D. Zhong, “A scheme for visual feature based image indexing,” in Proceedings of IS&T/SPIE Conference on Storage and Retrieval for Image and Video Databases III, 1995, pp. 36-46.

Download references

Author information

Authors and Affiliations

  1. Microsoft Research Asia, 49 Zhichun Road, Beijing, 100080, China

    Hongjiang Zhang, Zheng Chen, Mingjing Li & Zhong Su

Authors
  1. Hongjiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zheng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mingjing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhong Su
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhang, H., Chen, Z., Li, M. et al. Relevance Feedback and Learning in Content-Based Image Search. World Wide Web 6, 131–155 (2003). https://doi.org/10.1023/A:1023618504691

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1023618504691

Search

Navigation