Fuzzy SVM Ensembles for Relevance Feedback in Image Retrieval

  • Yong Rao
  • Padma Mundur
  • Yelena Yesha
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4071)


Relevance feedback has been integrated into content-based retrieval systems to overcome the semantic gap problem. Recently, Support Vector Machines (SVMs) have been widely used to learn the users’ semantic query concept from users’ feedback. The feedback is either ‘relevant’ or ‘irrelevant’ which forces the users to make a binary decision during each retrieval iteration. However, human’s perception of visual content is quite subjective and therefore, the notion of whether or not an image is relevant is rather vague and hard to define. Part of the small training samples problem faced by traditional SVMs can be thought of as the result of strict binary decision-making. In this paper, we propose a Fuzzy SVM technique to overcome the small sample problem. Using Fuzzy SVM, each sample can be assigned a fuzzy membership to model users’ feedback gradually from ‘irrelevant’ to ‘relevant’ instead of strict binary labeling. We also propose to use Fuzzy SVM ensembles to further improve the classification results. We conduct extensive experiments to evaluate the performance of our proposed algorithm. Compared to the experimental results using traditional SVMs, we demonstrate that our proposed approach can significantly improve the retrieval performance of semantic image retrieval.


Image Retrieval Average Precision Fuzzy Membership Relevance Feedback Fuzzy Membership Function 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Sebe, N., Lew, M.S., Zhou, X.S., Huang, T.S., Bakker, E.M.: The State of the Art in Image and Video Retrieval. In: Bakker, E.M., Lew, M., Huang, T.S., Sebe, N., Zhou, X.S. (eds.) CIVR 2003. LNCS, vol. 2728, pp. 24–25. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  2. 2.
    Zhou, X.S., Huang, T.S.: Relevance Feedback in Image retrieval: A comprehensive review. Multimedia Systems 8(6), 536–544 (2003)CrossRefGoogle Scholar
  3. 3.
    Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1995)MATHGoogle Scholar
  4. 4.
    He, X., King, O., Ma, W.-Y., Li, M., Zhang, H.J.: Learning a semantic space from user’s relevance feedback for image retrieval. IEEE Transactions on Circuits and Systems for Video Technology 13(1), 39–48 (2003)CrossRefGoogle Scholar
  5. 5.
    Yoshizawa, T., Schweitzer, H.: Long-term learning of semantic grouping from relevance-feedback. Multimedia Information Retrieval, 165–172 (2004)Google Scholar
  6. 6.
    Carson, C., Belongie, S., Greenspan, H., Malik, J.: Blobworld: Image segmentation using Expectation-Maximization and its application to image querying, EECS department technical report, UC Berkeley (1999)Google Scholar
  7. 7.
    Rui, Y., Huang, T.S., Ortega, M., Mehrotra, S.: Relevance feedback: A power tool in interactive content-based image retrieval. IEEE Transaction on Circuits and Systems for Video Tech., Special Issue on Interactive Multimedia Systems for the Internet 8(5), 644–655 (1998)Google Scholar
  8. 8.
    Frigui, H.: Adaptive Image Retrieval Using the Fuzzy Integral. In: Proceedings of North American Fuzzy Information Processing Society Conference, pp. 575–578 (1999)Google Scholar
  9. 9.
    Krishnapuram, R., Medasani, S., Jung, S.H., Choi, R.Y.S.: Content-based image retrieval based on a fuzzy approach. IEEE Transactions on Knowledge and Data Engineering 16(10), 1185–1199 (2004)CrossRefGoogle Scholar
  10. 10.
    Hoi, C.-H., Lyu, M.R.: A Novel Log-based Relevance Feedback Technique in Content-based Image Retrieval. In: Proceedings of the 12th ACM International Conference on Multimedia (MM 2004), New York, USA, pp. 24–31 (2004)Google Scholar
  11. 11.
    Tao, D., Tang, X.: Random Sampling Based SVM for Relevance Feedback Image Retrieval. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2004), July 2007, pp. 647–652 (2004)Google Scholar
  12. 12.
    Huang, H., Liu, Y.: Fuzzy Support Vector Machines for Pattern Recognition and Data Mining. International Journal of Fuzzy Systems 4(3), 826–925 (2002)MathSciNetGoogle Scholar
  13. 13.
    Lin, C., Wang, S.: Fuzzy Support Vector Machines. IEEE Transaction on Neural Networks 13(2), 464–471 (2002)CrossRefGoogle Scholar
  14. 14.
    Abe, S., Inoue, T.: Fuzzy Support Vector Machines for Multi-class Problems. In: Proceedings of European Symposium on Artificial Neural Networks, Bruges (Belgium), 24-26 April, pp. 113–118 (2002)Google Scholar
  15. 15.
    Kiranyaz, S., Caglar, K., Guldogan, E., Guldogan, O., Gabbouj, M.: MUVIS: a content-based multimedia indexing and retrieval framework. In: Proc. of the 7th Intl. Symposium on Signal Processing and its Applications, ISSPA 2003, Paris, France, July 2003, pp. 1–8 (July 2003)Google Scholar
  16. 16.
    Wang, J.Z., Li, J., Wiederhold, G.: SIMPLIcity: Semantics-sensitive Integrated Matching for Picture LIbraries. IEEE Transaction on Pattern Analysis and Machine Intelligence 23(9), 947–963 (2001)CrossRefGoogle Scholar
  17. 17.
    Sonka, M., Hlavac, V., Boyle, R.: Image Processing, Analysis, and Machine Vision, 2nd edn. Thomson-Engineering (1998)Google Scholar
  18. 18.
    Chang, C.-C., Lin, C.-J.: LIBSVM: A library for support vector machines (2001), Software available at:

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Yong Rao
    • 1
  • Padma Mundur
    • 1
  • Yelena Yesha
    • 1
  1. 1.Department of Computer Science and Electrical EngineeringUniversity of MarylandBaltimoreUSA

Personalised recommendations