Combining Gaussian Mixture Models and Support Vector Machines for Relevance Feedback in Content Based Image Retrieval

  • Apostolos Marakakis
  • Nikolaos Galatsanos
  • Aristidis Likas
  • Andreas Stafylopatis
Part of the IFIP International Federation for Information Processing book series (IFIPAICT, volume 296)


A relevance feedback (RF) approach for content based image retrieval (CBIR) is proposed, which combines Support Vector Machines (SVMs) with Gaussian Mixture (GM) models. Specifically, it constructs GM models of the image features distribution to describe the image content and trains an SVM classifier to distinguish between the relevant and irrelevant images according to the preferences of the user. The method is based on distance measures between probability density functions (pdfs), which can be computed in closed form for GM models. In particular, these distance measures are used to define a new SVM kernel function expressing the similarity between the corresponding images modeled as GMs. Using this kernel function and the user provided feedback examples, an SVM classifier is trained in each RF round, resulting in an updated ranking of the database images. Numerical experiments are presented that demonstrate the merits of the proposed relevance feedback methodology and the advantages of using GMs for image modeling in the RF framework.


Support Vector Machine Gaussian Mixture Model Image Retrieval Relevance Feedback Content Base Image Retrieval 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Y. Ishikawa, R. Subramanya, and C. Faloutsos, “MindReader: Querying databases through multiple examples”, Proceedings International Conference on Very large Data Bases (VLDB), 1998.Google Scholar
  2. 2.
    N. Vasconcelos, “Minimum Probability of Error Image Retrieval”, IEEE Transactions on Signal Processing, vol. 52, no. 8, pp. 2322–2336, Aug. 2004.MathSciNetCrossRefGoogle Scholar
  3. 3.
    Ritendra Datta, Jia Li and James Ze Wang, “Content-based image retrieval: approaches and trends of the new age”, Multimedia Information Retrieval, pp. 253–262, 2005.Google Scholar
  4. 4.
    G. D. Guo, A. K. Jain, W. Y. Ma, and H. J. Zhang, “Learning similarity measure for natural image retrieval with relevance feedback”, IEEE Transactions on Neural Networks., vol. 13, no. 4, pp. 811–820, Jul. 2002.CrossRefGoogle Scholar
  5. 5.
    C.T. Hsu, and C. Y. Li, “Relevance Feedback Using Generalized Bayesian Framework With Region-Based Optimization Learning”, IEEE Transactions on Image Processing., Vol. 14, No. 10, pp. 1617–1631, October 2005.CrossRefGoogle Scholar
  6. 6.
    F. Qian, M. Li, L. Zhang, H. J. Zhang, and B. Zhang, “Gaussian mixture model for relevance feedback in image retrieval”, Proceedings IEEE ICME, Aug. 2002.Google Scholar
  7. 7.
    F. Jing, M. Li, H-J. Zhang, and B. Zhang, “Relevance Feedback in Region-Based Image Retrieval”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 14, no. 5, pp. 672–681, May 2004.CrossRefGoogle Scholar
  8. 8.
    Wei Jiang, Guihua Er, Qionghai Dai and Jinwei Gu, “Similarity-Based Online Feature Selection in Content-Based Image Retrieval”, IEEE Transactions on Image Processing, vol. 15, no. 3, pp. 702–712, March 2006.CrossRefGoogle Scholar
  9. 9.
    C. Carson, S. Belongie, H. Greenspan, and J. Malik, “Blobworld: Image segmentation using expectation-maximization and its application to image querying”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 8, pp. 1026–1038, Aug. 2002.CrossRefGoogle Scholar
  10. 10.
    A. Marakakis, N. Galatsanos, A. Likas and A. Stafylopatis, “A Relevance Feedback Approach for Content Based Image Retrieval Using Gaussian Mixture Models”, Proceedings International Conference Artificial Neural Networks (ICANN), Athens, Greece, September 2006.Google Scholar
  11. 11.
    N. Vlassis and A. Likas, “A greedy EM algorithm for Gaussian mixture learning”, Neural Processing Letters, vol. 15, pp. 77–87, 2002.CrossRefMATHGoogle Scholar
  12. 12.
    N. Vasconcelos, “On the Efficient Evaluation of Probabilistic Similarity Functions for Image Retrieval”, IEEE Transactions on Information Theory, vol. 50, no. 7, pp. 1482– 1496, July 2004.MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    N. Vasconcelos and A. Lippman, “Learning from user feedback in image retrieval systems”, Advances in Neural Information Processing Systems, 1999.Google Scholar
  14. 14.
    S. Tong and E. Chang, “Support vector machine active learning for image retrieval”, ACM Multimedia, 2001.Google Scholar
  15. 15.
    Dacheng Tao, Xiaoou Tang and Xuelong Li, “Which Components Are Important for Interactive Image Searching?”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 18, no. 1, pp. 3–11, Jan. 2008.CrossRefGoogle Scholar
  16. 16.
    C. M. Bishop, Neural Networks for Pattern Recognition, Oxford Univ. Press Inc., New York, 1995.MATHGoogle Scholar
  17. 17.
    C.M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006.Google Scholar
  18. 18.
    Jacob Goldberger and Sam Roweis, “Hierarchical Clustering of a Mixture Model”, Neural Information Processing Systems 17 (NIPS'04), pp 505–512, 2004.Google Scholar
  19. 19.
    Microsoft Research Cambridge Object Recognition Image Database, version 1.0.
  20. 20.
    LIBSVM — A Library for Support Vector Machines.

Copyright information

© IFIP International Federation for Information Processing 2009

Authors and Affiliations

  • Apostolos Marakakis
    • 1
  • Nikolaos Galatsanos
    • 2
  • Aristidis Likas
    • 3
  • Andreas Stafylopatis
    • 1
  1. 1.School of Electrical and Computer EngineeringNational Technical University of AthensAthensGreece
  2. 2.Department of Electrical and Computer EngineeringUniversity of PatrasPatrasGreece
  3. 3.Department of Computer ScienceUniversity of IoanninaIoanninaGreece

Personalised recommendations