What Is the Role of Similarity for Known-Item Search at Video Browser Showdown?

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11223)


Across many domains, machine learning approaches start to compete with human experts in tasks originally considered as very difficult for automation. However, effective retrieval of general video shots still represents an issue due to their variability, complexity and insufficiency of training sets. In addition, users can face problems trying to formulate their search intents in a given query interface. Hence, many systems still rely also on interactive human-machine cooperation to boost effectiveness of the retrieval process. In this paper, we present our experience with known-item search tasks in the Video Browser Showdown competition, where participating interactive video retrieval systems mostly rely on various similarity models. We discuss the observed difficulty of known-item search tasks, categorize employed interaction components (relying on similarity models) and inspect successful interactive known-item searches from the recent iteration of the competition. Finally, open similarity search challenges for known-item search in video are presented.


Interactive video retrieval Known-item search Similarity search 



This paper has been supported by Czech Science Foundation (GAČR) project no. 17-22224S and by grant SVV-260451. This work is also supported by Universität Klagenfurt and Lakeside Labs GmbH, Klagenfurt, Austria and funding from the European Regional Development Fund and the Carinthian Economic Promotion Fund (KWF) under grant KWF 20214 u. 3520/26336/38165.


  1. 1.
    Awad, G., et al.: TRECVID 2017: evaluating ad-hoc and instance video search, events detection, video captioning and hyperlinking. In: Proceedings of TRECVID 2017. NIST, Gaithersburg (2017)Google Scholar
  2. 2.
    Bailer, W., Thallinger, G.: A framework for multimedia content abstraction and its application to rushes exploration. In: Proceedings of the 6th ACM International Conference on Image and Video Retrieval, pp. 146–153 (2007)Google Scholar
  3. 3.
    Barthel, K.U., Hezel, N., Jung, K.: Fusing keyword search and visual exploration for untagged videos. In: Schoeffmann, K., et al. (eds.) MMM 2018. LNCS, vol. 10705, pp. 413–418. Springer, Cham (2018). Scholar
  4. 4.
    Chávez, E., Navarro, G., Baeza-Yates, R.A., Marroquín, J.L.: Searching in metric spaces. ACM Comput. Surv. 33(3), 273–321 (2001)CrossRefGoogle Scholar
  5. 5.
    Cobârzan, C., et al.: Interactive video search tools: a detailed analysis of the video browser showdown 2015. Multimed. Tools Appl. 76(4), 5539–5571 (2017)CrossRefGoogle Scholar
  6. 6.
    Gurrin, C., Smeaton, A.F., Doherty, A.R.: LifeLogging: personal big data. Found. Trends Inf. Retr. 8(1), 1–125 (2014)CrossRefGoogle Scholar
  7. 7.
    Keim, D.A.: Information visualization and visual data mining. IEEE Trans. Vis. Comput. Graph. 8(1), 1–8 (2002)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  9. 9.
    Lokoc, J., Bailer, W., Schoeffmann, K., Muenzer, B., Awad, G.: On influential trends in interactive video retrieval: video browser showdown 2015–2017. IEEE Trans. Multimed., 1 (2018).
  10. 10.
    Lokoč, J., Kovalčík, G., Souček, T.: Revisiting SIRET video retrieval tool. In: Schoeffmann, K., et al. (eds.) MMM 2018. LNCS, vol. 10705, pp. 419–424. Springer, Cham (2018). Scholar
  11. 11.
    Moumtzidou, A., et al.: VERGE in VBS 2018. In: Schoeffmann, K. (ed.) MMM 2018. LNCS, vol. 10705, pp. 444–450. Springer, Cham (2018). Scholar
  12. 12.
    Nguyen, P.A., Lu, Y.J., Zhang, H., Ngo, C.W.: Enhanced VIREO KIS at VBS 2018. In: Schoeffmann, K., et al. (eds.) MMM 2018. LNCS, vol. 10705, pp. 407–412. Springer, Cham (2018). Scholar
  13. 13.
    Primus, M.J., Münzer, B., Leibetseder, A., Schoeffmann, K.: The ITEC collaborative video search system at the video browser showdown 2018. In: Schoeffmann, K., et al. (eds.) MMM 2018. LNCS, vol. 10705, pp. 438–443. Springer, Cham (2018). Scholar
  14. 14.
    Rossetto, L., Giangreco, I., Gasser, R., Schuldt, H.: Competitive video retrieval with vitrivr. In: Schoeffmann, K., et al. (eds.) MMM 2018. LNCS, vol. 10705, pp. 403–406. Springer, Cham (2018). Scholar
  15. 15.
    Rui, Y., Huang, T.: A unified framework for video browsing and retrieval. In: Image and Video Processing Handbook, pp. 705–715 (2000)Google Scholar
  16. 16.
    Rujikietgumjorn, S., Watcharapinchai, N., Marukatat, S.: Sloth search system. In: Schoeffmann, K., et al. (eds.) MMM 2018. LNCS, vol. 10705, pp. 431–437. Springer, Cham (2018). Scholar
  17. 17.
    Schoeffmann, K.: A user-centric media retrieval competition: the video browser showdown 2012–2014. IEEE MultiMed. 21(4), 8–13 (2014)CrossRefGoogle Scholar
  18. 18.
    Szegedy, C., et al.: Going deeper with convolutions. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, 7–12 June 2015, pp. 1–9 (2015)Google Scholar
  19. 19.
    Truong, T.D., et al.: Video search based on semantic extraction and locally regional object proposal. In: Schoeffmann, K., et al. (eds.) MMM 2018. LNCS, vol. 10705, pp. 451–456. Springer, Cham (2018). Scholar
  20. 20.
    Zezula, P., Amato, G., Dohnal, V., Batko, M.: Similarity Search - The Metric Space Approach. Advances in Database Systems, vol. 32. Springer, Boston (2006). Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Faculty of Mathematics and PhysicsCharles UniversityPragueCzech Republic
  2. 2.DIGITAL–Institute for ICTJOANNEUM RESEARCHGrazAustria
  3. 3.Institute of Information TechnologyAlpen-Adria-Universität KlagenfurtKlagenfurtAustria

Personalised recommendations