A Multimedia Retrieval Framework Based on Automatic Graded Relevance Judgments

  • Miriam Redi
  • Bernard Merialdo
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7131)


Traditional Content Based Multimedia Retrieval (CBMR) systems measure the relevance of visual samples using a binary scale (Relevant/Non Relevant). However, a picture can be relevant to a semantic category with different degrees, depending on the way such concept is represented in the image. In this paper, we build a CBMR framework that supports graded relevance judgments. In order to quickly build graded ground truths, we propose a measure to reassess binary-labeled databases without involving manual effort: we automatically assign a reliable relevance degree (Non, Weakly, Average, Very Relevant) to each sample, based on its position with respect to the hyperplane drawn by support vector machines in the feature space. We test the effectiveness of our system on two large-scale databases, and we show that our approach outperforms the traditional binary relevance-based frameworks in both scene recognition and video retrieval.


Semantic Concept Mean Average Precision Video Retrieval Relevance Judgment Scene Recognition 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ayache, S., Quénot, G.: Trecvid 2007 collaborative annotation using active learning. In: Proceedings of the TRECVID 2007 Workshop (2007)Google Scholar
  2. 2.
    Boser, B.E., Guyon, I.M., Vapnik, V.N.: A training algorithm for optimal margin classifiers. In: Proceedings of the Fifth Annual Workshop on Computational Learning Theory, pp. 144–152. ACM (1992)Google Scholar
  3. 3.
    Elleuch, N., Zarka, M., Feki, I., Ammar, A.B., Alimi, A.: Regimvid at trecvid 2010: Semantic indexing. In: Proceedings of the TRECVID 2010 Workshop (2010)Google Scholar
  4. 4.
    Freund, Y., Iyer, R., Schapire, R.E., Singer, Y.: An efficient boosting algorithm for combining preferences. J. Mach. Learn. Res. 4, 933–969 (2003)zbMATHGoogle Scholar
  5. 5.
    Ji, Z., Lu, B.-L.: Gender Classification Based on Support Vector Machine with Automatic Confidence. In: Leung, C.S., Lee, M., Chan, J.H. (eds.) ICONIP 2009, Part I. LNCS, vol. 5863, pp. 685–692. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  6. 6.
    Kekäläinen, J.: Binary and graded relevance in ir evaluations–comparison of the effects on ranking of ir systems. Information processing & management 41(5), 1019–1033 (2005)CrossRefGoogle Scholar
  7. 7.
    Kelly, D., Teevan, J.: Implicit feedback for inferring user preference: a bibliography. SIGIR Forum 37, 18–28 (2003)CrossRefGoogle Scholar
  8. 8.
    Lin, C., Wang, S.: Training algorithms for fuzzy support vector machines with noisy data. Pattern Recognition Letters 25(14), 1647–1656 (2004)CrossRefGoogle Scholar
  9. 9.
    Lin, C.F., Wang, S.D.: Fuzzy support vector machines. IEEE Transactions on Neural Networks 13(2), 464–471 (2002)CrossRefGoogle Scholar
  10. 10.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  11. 11.
    Papageorgiou, C.P., Oren, M., Poggio, T.: A General Framework for Object Detection. In: Proceedings of the Sixth International Conference on Computer Vision, p. 555. IEEE Computer Society (1998)Google Scholar
  12. 12.
    Paterno, M.C.S., Lim, F.S., Leow, W.K.: Fuzzy semantic labeling for image retrieval. In: 2004 IEEE International Conference on Multimedia and Expo, ICME 2004, vol. 2, pp. 767–770. IEEE (2004)Google Scholar
  13. 13.
    Platt, J.: Probabilistic outputs for support vector machines. In: Bartlett, P., Schoelkopf, B., Schurmans, D., Smola, A.J. (eds.) Advances in Large Margin Classifiers, pp. 61–74Google Scholar
  14. 14.
    Redi, M., Merialdo, B., Wang, F.: Eurecom and ecnu at trecvid 2010: The semantic indexing task. In: Proceedings of the TRECVID 2010 Workshop (2010)Google Scholar
  15. 15.
    Redi, M., Merialdo, B.: Saliency moments for image categorization. In: Proceedings of the 1st ACM International Conference on Multimedia Retrieval, ICMR 2011, pp. 39:1–39:8. ACM, New York (2011)Google Scholar
  16. 16.
    Saracevic, T.: Relevance: A review of the literature and a framework for thinking on the notion in information science. part iii: Behavior and effects of relevance. Journal of the American Society for Information Science and Technology 58(13), 2126–2144 (2007)CrossRefGoogle Scholar
  17. 17.
    Smeaton, A.F., Over, P., Kraaij, W.: Evaluation campaigns and trecvid. In: MIR 2006: Proceedings of the 8th ACM International Workshop on Multimedia Information Retrieval, pp. 321–330. ACM Press, New York (2006)Google Scholar
  18. 18.
    Snow, R., O’Connor, B., Jurafsky, D., Ng, A.Y.: Cheap and fast—but is it good?: evaluating non-expert annotations for natural language tasks. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 254–263. Association for Computational Linguistics (2008)Google Scholar
  19. 19.
    Sormunen, E.: Liberal relevance criteria of trec-: counting on negligible documents? In: Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 324–330. ACM (2002)Google Scholar
  20. 20.
    Stricker, M.A., Orengo, M.: Similarity of color images. In: Proceedings of SPIE, vol. 2420, p. 381 (1995)Google Scholar
  21. 21.
    Svore, K., Vanderwende, L., Burges, C.: Enhancing single-document summarization by combining ranknet and third-party sources. In: Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pp. 448–457 (2007)Google Scholar
  22. 22.
    Won, C.S., Park, D.K., Park, S.J.: Efficient use of MPEG-7 edge histogram descriptor. Etri Journal 24(1), 23–30 (2002)CrossRefGoogle Scholar
  23. 23.
    Zheng, Z., Chen, K., Sun, G., Zha, H.: A regression framework for learning ranking functions using relative relevance judgments. In: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 287–294. ACM (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Miriam Redi
    • 1
  • Bernard Merialdo
    • 1
  1. 1.EurecomSophia AntipolisFrance

Personalised recommendations