Collaborative Feature Maps for Interactive Video Search

  • Klaus Schoeffmann
  • Manfred Jürgen Primus
  • Bernd Muenzer
  • Stefan Petscharnig
  • Christof Karisch
  • Qing Xu
  • Wolfgang Huerst
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10133)

Abstract

This extended demo paper summarizes our interface used for the Video Browser Showdown (VBS) 2017 competition, where visual and textual known-item search (KIS) tasks, as well as ad-hoc video search (AVS) tasks in a 600-h video archive need to be solved interactively. To this end, we propose a very flexible distributed video search system that combines many ideas of related work in a novel and collaborative way, such that several users can work together and explore the video archive in a complementary manner. The main interface is a perspective Feature Map, which shows keyframes of shots arranged according to a selected content similarity feature (e.g., color, motion, semantic concepts, etc.). This Feature Map is accompanied by additional views, which allow users to search and filter according to a particular content feature. For collaboration of several users we provide a cooperative heatmap that shows a synchronized view of inspection actions of all users. Moreover, we use collaborative re-ranking of shots (in specific views) based on retrieved results of other users.

Keywords

Video retrieval Interactive search Collaboration 

References

  1. 1.
    Barthel, K.U., Hezel, N., Mackowiak, R.: Navigating a graph of scenes for exploring large video collections. In: Tian, Q., Sebe, N., Qi, G.-J., Huet, B., Hong, R., Liu, X. (eds.) MMM 2016. LNCS, vol. 9517, pp. 418–423. Springer, Heidelberg (2016). doi:10.1007/978-3-319-27674-8_43 CrossRefGoogle Scholar
  2. 2.
    Cobârzan, C., Fabro, M., Schoeffmann, K.: Collaborative browsing and search in video archives with mobile clients. In: He, X., Luo, S., Tao, D., Xu, C., Yang, J., Hasan, M.A. (eds.) MMM 2015. LNCS, vol. 8936, pp. 266–271. Springer, Heidelberg (2015). doi:10.1007/978-3-319-14442-9_26 Google Scholar
  3. 3.
    Hudelist, M.A., Cobârzan, C., Beecks, C., Werken, R., Kletz, S., Hürst, W., Schoeffmann, K.: Collaborative video search combining video retrieval with human-based visual inspection. In: Tian, Q., Sebe, N., Qi, G.-J., Huet, B., Hong, R., Liu, X. (eds.) MMM 2016. LNCS, vol. 9517, pp. 400–405. Springer, Heidelberg (2016). doi:10.1007/978-3-319-27674-8_40 CrossRefGoogle Scholar
  4. 4.
    Hürst, W., Werken, R., Hoet, M.: A storyboard-based interface for mobile video browsing. In: He, X., Luo, S., Tao, D., Xu, C., Yang, J., Hasan, M.A. (eds.) MMM 2015. LNCS, vol. 8936, pp. 261–265. Springer, Heidelberg (2015). doi:10.1007/978-3-319-14442-9_25 Google Scholar
  5. 5.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) Advances in Neural Information Processing Systems 25, pp. 1097–1105. Curran Associates Inc. (2012)Google Scholar
  6. 6.
    Over, P., Awad, G., Michel, M., Fiscus, J., Sanders, G., Shaw, B., Kraaij, W., Smeaton, A.F., Quénot, G.: TRECVID 2012 – an overview of the goals, tasks, data, evaluation mechanisms and metrics. In: Proceedings of TRECVID 2012 (2012)Google Scholar
  7. 7.
    Schoeffmann, K.: A user-centric media retrieval competition: the video browser showdown 2012–2014. IEEE MultiMed. 21(4), 8–13 (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Klaus Schoeffmann
    • 1
  • Manfred Jürgen Primus
    • 1
  • Bernd Muenzer
    • 1
  • Stefan Petscharnig
    • 1
  • Christof Karisch
    • 1
  • Qing Xu
    • 2
  • Wolfgang Huerst
    • 3
  1. 1.Institute of Information TechnologyKlagenfurt UniversityKlagenfurtAustria
  2. 2.School of Computer Science and TechnologyTianjin UniversityTianjinChina
  3. 3.Information and Computer SciencesUtrecht UniversityUtrechtThe Netherlands

Personalised recommendations