Skip to main content
SpringerLink
Log in
Book cover

International Conference on Multimedia Modeling

MMM 2017: MultiMedia Modeling pp 318–330Cite as

Rocchio-Based Relevance Feedback in Video Event Retrieval

Part of the Lecture Notes in Computer Science book series (LNISA,volume 10133)

Abstract

This paper investigates methods for user and pseudo relevance feedback in video event retrieval. Existing feedback methods achieve strong performance but adjust the ranking based on few individual examples. We propose a relevance feedback algorithm (ARF) derived from the Rocchio method, which is a theoretically founded algorithm in textual retrieval. ARF updates the weights in the ranking function based on the centroids of the relevant and non-relevant examples. Additionally, relevance feedback algorithms are often only evaluated by a single feedback mode (user feedback or pseudo feedback). Hence, a minor contribution of this paper is to evaluate feedback algorithms using a larger number of feedback modes. Our experiments use TRECVID Multimedia Event Detection collections. We show that ARF performs significantly better in terms of Mean Average Precision, robustness, subjective user evaluation, and run time compared to the state-of-the-art.

Keywords

  • Information retrieval
  • Relevance feedback
  • Video search
  • Rocchio
  • ARF

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Cochran, W.G., Cox, G.M.: Experimental designs (1957)

    Google Scholar 

  2. Crucianu, M., Ferecatu, M., Boujemaa, N.: Relevance feedback for image retrieval: a short survey. Report of the DELOS2 European Network of Excellence (FP6) (2004)

    Google Scholar 

  3. Dalton, J., Allan, J., Mirajkar, P.: Zero-shot video retrieval using content and concepts. In: Proceedings of the 22nd ACM International Conference on Information and Knowledge Management, pp. 1857–1860. ACM (2013)

    Google Scholar 

  4. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: CVPR 2009, pp. 248–255. IEEE (2009)

    Google Scholar 

  5. Deselaers, T., Paredes, R., Vidal, E., Ney, H.: Learning weighted distances for relevance feedback in image retrieval. In: 19th International Conference on Pattern Recognition, ICPR 2008, pp. 1–4. IEEE (2008)

    Google Scholar 

  6. Gia, G., Roli, F., et al.: Instance-based relevance feedback for image retrieval. In: Advances in Neural Information Processing Systems, pp. 489–496 (2004)

    Google Scholar 

  7. Jiang, L., Meng, D., Mitamura, T., Hauptmann, A.G.: Easy samples first: self-paced reranking for zero-example multimedia search. In: Proceedings of the ACM International Conference on Multimedia, pp. 547–556. ACM (2014)

    Google Scholar 

  8. Jiang, L., Mitamura, T., Yu, S.I., Hauptmann, A.G.: Zero-example event search using multimodal pseudo relevance feedback. In: Proceedings of the International Conference on Multimedia Retrieval, p. 297. ACM (2014)

    Google Scholar 

  9. Jiang, L., Yu, S.I., Meng, D., Mitamura, T., Hauptmann, A.G.: Bridging the ultimate semantic gap: a semantic search engine for internet videos. In: ACM International Conference on Multimedia Retrieval, pp. 27–34 (2015)

    Google Scholar 

  10. Jiang, Y.G., Wu, Z., Wang, J., Xue, X., Chang, S.F.: Exploiting feature and class relationships in video categorization with regularized deep neural networks. arXiv preprint arXiv:1502.07209 (2015)

  11. Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., Fei-Fei, L.: Large-scale video classification with convolutional neural networks. In: CVPR (2014)

    Google Scholar 

  12. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111–3119 (2013)

    Google Scholar 

  13. Over, P., Awad, G., Michel, M., Fiscus, J., Sanders, G., Kraaij, W., Smeaton, A.F., Quéenot, G., Ordelman, R.: TRECVID 2015 - an overview of the goals, tasks, data, evaluation mechanisms and metrics. In: Proceedings of the TRECVID 2015, p. 52. NIST, USA (2015)

    Google Scholar 

  14. Patil, S.: A comprehensive review of recent relevance feedback techniques in CBIR. Int. J. Eng. Res. Technol. (IJERT) 1(6) (2012)

    Google Scholar 

  15. Rocchio, J.J.: Relevance feedback in information retrieval (1971)

    Google Scholar 

  16. Sakai, T., Manabe, T., Koyama, M.: Flexible pseudo-relevance feedback via selective sampling. ACM Trans. Asian Lang. Inf. Process. (TALIP) 4(2), 111–135 (2005)

    CrossRef  Google Scholar 

  17. Tao, D., Tang, X., Li, X.: Which components are important for interactive image searching? IEEE Trans. Circuits Syst. Video Technol. 18(1), 3–11 (2008)

    CrossRef  Google Scholar 

  18. Tong, S., Chang, E.: Support vector machine active learning for image retrieval. In: Proceedings of the 9th ACM International Conference on Multimedia, pp. 107–118. ACM (2001)

    Google Scholar 

  19. Wang, X.Y., Liang, L.L., Li, W.Y., Li, D.M., Yang, H.Y.: A new SVM-based relevance feedback image retrieval using probabilistic feature and weighted kernel function. J. Vis. Commun. Image Represent. 38, 256–275 (2016)

    CrossRef  Google Scholar 

  20. Xu, S., Li, H., Chang, X., Yu, S.I., Du, X., Li, X., Jiang, L., Mao, Z., Lan, Z., Burger, S., et al.: Incremental multimodal query construction for video search. In: Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, pp. 675–678. ACM (2015)

    Google Scholar 

  21. Yang, L., Hanjalic, A.: Supervised reranking for web image search. In: Proceedings of the International Conference on Multimedia, pp. 183–192. ACM (2010)

    Google Scholar 

  22. Ye, G., Liu, D., Chang, S.F., Saleemi, I., Shah, M., Ng, Y., White, B., Davis, L., Gupta, A., Haritaoglu, I.: BBN VISER TRECVID 2012 multimedia event detection and multimedia event recounting systems

    Google Scholar 

  23. Zhang, H., Lu, Y.J., de Boer, M., ter Haar, F., Qiu, Z., Schutte, K., Kraaij, W., Ngo, C.W.: VIREO-TNO@ TRECVID 2015: multimedia event detection

    Google Scholar 

  24. Zhou, B., Lapedriza, A., Xiao, J., Torralba, A., Oliva, A.: Learning deep features for scene recognition using places database. In: Advances in Neural Information Processing Systems, pp. 487–495 (2014)

    Google Scholar 

  25. Zhou, X.S., Huang, T.S.: Relevance feedback in image retrieval: a comprehensive review. Multimed. Syst. 8(6), 536–544 (2003)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Twente, P.O. Box 217, 7500 AE, Enschede, The Netherlands

    G. L. J. Pingen & R. B. N. Aly

  2. TNO, P.O. Box 96864, 2509 JG, The Hague, The Netherlands

    G. L. J. Pingen & M. H. T. de Boer

  3. University of Nijmegen, P.O. Box 9010, 6500 GL, Nijmegen, The Netherlands

    M. H. T. de Boer

Authors
  1. G. L. J. Pingen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. H. T. de Boer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. R. B. N. Aly
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to G. L. J. Pingen .

Editor information

Editors and Affiliations

  1. CNRS–IRISA, Rennes, France

    Laurent Amsaleg

  2. Reykjavík University, Reykjavik, Iceland

    Gylfi Þór Guðmundsson

  3. Dublin City University, Dublin, Ireland

    Cathal Gurrin

  4. Reykjavik University, Reykjavik, Ireland

    Björn Þór Jónsson

  5. National Institute of Informatics, Tokyo, Japan

    Shin’ichi Satoh

Rights and permissions

Reprints and Permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Pingen, G.L.J., de Boer, M.H.T., Aly, R.B.N. (2017). Rocchio-Based Relevance Feedback in Video Event Retrieval. In: Amsaleg, L., Guðmundsson, G., Gurrin, C., Jónsson, B., Satoh, S. (eds) MultiMedia Modeling. MMM 2017. Lecture Notes in Computer Science(), vol 10133. Springer, Cham. https://doi.org/10.1007/978-3-319-51814-5_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-51814-5_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-51813-8

  • Online ISBN: 978-3-319-51814-5

  • eBook Packages: Computer ScienceComputer Science (R0)

search

Navigation