An User-Driven Tool for Interactive Retrieval of Non Annotated Videos

  • M. Angeles Mendoza
  • Tomás Arnau
  • Isabel Gracia
  • Filiberto Pla
  • Nicolás Pérez de la Blanca
Part of the Intelligent Systems Reference Library book series (ISRL, volume 48)


A prototype to retrieve videos from non-annotated video databases is proposed. We focus on the problem of retrieving relevant videos from the audiovisual signal when the query is unknown for the system, since it is assumed that most of the available annotations are useless, as it is the case for most of the videos from common users in Internet. The approach presented is defined inside of the on-line learning paradigm where user and system collaborate to improve alternative rankings of the items dataset. The user guides the system in the semantic level and the system tries to adapt the low-level similarity distance between items according to the user preferences. The user interacts with the system until a prefixed number of relevant items is retrieved. The video database is represented as a dense graph where a semi-supervised algorithm is used to propagate the user feeedback.


Rapid Serial Visual Presentation Relevant Item Video Retrieval Video Database Initial Query 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lscom lexicon definitions and annotations version 1.0. Technical Report 217-2006-3, DTO Challenge Workshop on Large Scale Concept Ontology for Multimedia, Columbia University ADVENT Technical ReportGoogle Scholar
  2. 2.
  3. 3.
    Multimedia Analysis and Retrieval SystemGoogle Scholar
  4. 4.
    Chapelle, O., Scholkopf, B., Zien, A.: Semi-Supervised Learning. MIT Press (2006)Google Scholar
  5. 5.
    Chechik, G., Sharma, V., Shalit, U., Bengio, S.: Large Scale Online Learning of Image Similarity through Ranking. In: Araujo, H., Mendonça, A.M., Pinho, A.J., Torres, M.I. (eds.) IbPRIA 2009. LNCS, vol. 5524, pp. 11–14. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  6. 6.
    Cotton, C., Ellis, D.: Audio fingerprinting to identify multiple videos of an event. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (2010)Google Scholar
  7. 7.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection, vol. 1, pp. 886–893. IEEE Computer Society, Washington, DC (2005)Google Scholar
  8. 8.
    de Rooij, O., Worring, M.: Browsing video along multiple threads. IEEE Transactions on Multimedia 12(2), 121–130 (2010)CrossRefGoogle Scholar
  9. 9.
    Fergus, R., Weiss, Y., Torralba, A.: Semi-supervised learning in gigantic image collections. In: NIPS (2009)Google Scholar
  10. 10.
    Hastie, T., Tibshirani, R., Friedman, J.: The Elemensts of Statistical Learning: Data Mining Inference and prediction. Springer (2009)Google Scholar
  11. 11.
    Hauptmann, A.G., Lin, W.H., Yan, R., Yang, J., Chen, M.Y.: Extreme video retrieval: joint maximization of human and computer performance. In: 14th Annual ACM International Conference on Multimedia, pp. 385–394. ACM Press, New York (2006)CrossRefGoogle Scholar
  12. 12.
    Hearst, M.A.: Search user interfaces. Cambridge University Press (2009)Google Scholar
  13. 13.
  14. 14.
    Jiang, Y.-G., Ye, G., Chang, S.-F., Ellis, D., Loui, A.C.: Consumer video understanding: A benchmark database and an evaluation of human and machine performance. In: Proceedings of ACM International Conference on Multimedia Retrieval (ICMR), Oral Session (2011)Google Scholar
  15. 15.
    Kumar, S., Mohri, M., Talwalkar, A.: Sampling techniques for the nystrom method. In: AISTATS (2009)Google Scholar
  16. 16.
    Kushki, A., Androutsos, P., Plataniotis, K.N., Venetsanopoulos, A.N.: Query feedback for interactive image retrieval. IEEE Transactions on Circuits and Systems for Video Technology 14(5), 644–655 (2004)CrossRefGoogle Scholar
  17. 17.
    Laptev, I., Pérez, P.: Retrieving actions in movies, pp. 1–8 (October 2007)Google Scholar
  18. 18.
    Le, Q.V., Ranzato, M.A., Mong, R., Devin, M., Chen, K., Corrado, G.S., Dean, J., Ng, A.Y.: Building high-level features using large scale unsupervised learning. In: Twenty-Ninth International Conference on Machine Learning (2012)Google Scholar
  19. 19.
    Lee, H., Grosse, R., Ranganath, R., Ng, A.Y.: Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: Twenty-Sixth International Conference on Machine Learning (2009)Google Scholar
  20. 20.
  21. 21.
    Lowe, D.C.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  22. 22.
    Müller, H., Clough, P., Deselaers, T., Caputo, B. (eds.): ImageCLEF, Experimental Evaluation in Visual Information Retrieval. The Information Retrieval Series, vol. 32. Springer (2010)Google Scholar
  23. 23.
    Rui, Y., Huang, T.S., Mehrotra, S.: Content-based image retrieval with relevance feedback in mars. In: Proc. IEEE Int. Conf. on Image Proc., pp. 815–818 (1997)Google Scholar
  24. 24.
    Schmid, C., Mohr, R., Bauckhage, C.: Evaluation of interest point detectors. IJCV 37(2), 151–172 (2000)zbMATHCrossRefGoogle Scholar
  25. 25.
    Schoelkopf, B., Smola, A.: Learning with Kernels Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press (2002)Google Scholar
  26. 26.
    Smeaton, A.F., Over, P., Kraaij, W.: Trecvid: evaluating the effectiveness of information retrieval tasks on digital video. In: 12th Annual ACM International Conference on Multimedia, pp. 652–655. ACM, New York (2004)CrossRefGoogle Scholar
  27. 27.
    Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R.: Content based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(12), 1349–1380 (2000)CrossRefGoogle Scholar
  28. 28.
    Snoek, C.G.M., Worring, M.: Concept-based video retrieval. Foundations and Trends in Information Retrieval 4(2), 215–322 (2009)Google Scholar
  29. 29.
    Snoek, C.G.M., Worring, M., van Gemert, J.C., Geusebroek, J.-M., Smeulders, A.W.M.: The challenge problem for automated detection of 101 semantic concepts in multimedia. In: Proceedings of ACM Multimedia, Santa Barbara, USA, pp. 421–430 (2006)Google Scholar
  30. 30.
    Snoek, C.G.M., Everts, I., van Gemert, J.C., Geusebroek, J.M., Huurnink, B., Koelma, D.C., van Liempt, M., de Rooij, O., van de Sande, K.E.A., Smeulders, A.W.M., et al.: The mediamill trecvid 2007 semantic video search engine. In: 5th TRECVID Workshop (2007)Google Scholar
  31. 31.
    Talwalkar, A., Kumar, S., Rowley, H.: Large-scale manifold learning. In: CVPR (2008)Google Scholar
  32. 32.
    TRECVID Multimedia Event Detection Track,
  33. 33.
  34. 34.
  35. 35.
  36. 36.
    Wactlar, H.D., Kanade, T., Smith, M.A., Stevens, S.M.: Intelligent access to digital video: Informedia project. IEEE Computer (1996)Google Scholar
  37. 37.
    Wang, J., Jebara, T., Chang, S.-F.: Graph transduction via alternating minimization. In: Proceedings of the 25th International Conference on Machine Learning (2008)Google Scholar
  38. 38.
    Yang, Y., Nie, F., Xu, D., Luo, J., Zhuang, Y., Pan, Y.: A multimedia retrieval framework based on semi-supervised ranking and relevance feedback. IEEE Transactions on Pattern Analysis and Machine Intelligence 34(4), 723–742 (2012)CrossRefGoogle Scholar
  39. 39.
    Zheng, Y.-T., Neo, S.-Y., Chen, X., Chua, T.-S.: Visiongo: towards true interactivity. In: Proceedings of the ACM International Conference on Image and Video Retrieval, CIVR 2009, pp. 51:1. ACM, New York (2009)Google Scholar
  40. 40.
    Zhou, X.S., Huang, T.S.: Relevance feedback in image retrieval: A comprehensive review. Multimedia Syst. 8(6), 536–544 (2003)CrossRefGoogle Scholar
  41. 41.
    Zhu, X., Laffertty, J.: Harmonic mixtures: combining mixture models and graph-based methods for inductive and scalable semi-supervised learning. In: ICML (2005)Google Scholar
  42. 42.
    Zhu, X.: Semi-supervised learning literature survey. Technical Report Computer Sciences TR 1530, University of Wisconsin – Madison (Last modified on July 19, 2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • M. Angeles Mendoza
    • 1
  • Tomás Arnau
    • 2
  • Isabel Gracia
    • 2
  • Filiberto Pla
    • 2
  • Nicolás Pérez de la Blanca
    • 1
  1. 1.Department of Computer Science and A.I.University of GranadaGranadaSpain
  2. 2.Institute of New Imaging TechnologiesUniversity Jaume ICastellónSpain

Personalised recommendations