Collaborative Video Search Combining Video Retrieval with Human-Based Visual Inspection

  • Marco A. HudelistEmail author
  • Claudiu Cobârzan
  • Christian Beecks
  • Rob van de Werken
  • Sabrina Kletz
  • Wolfgang Hürst
  • Klaus Schoeffmann
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9517)


We propose a novel video browsing approach that aims at optimally integrating traditional, machine-based retrieval methods with an interface design optimized for human browsing performance. Advanced video retrieval and filtering (e.g., via color and motion signatures, and visual concepts) on a desktop is combined with a storyboard-based interface design on a tablet optimized for quick, brute-force visual inspection. Both modules run independently but exchange information to significantly minimize the data for visual inspection and compensate mistakes made by the search algorithms.


Video retrieval Interactive search Interaction design Feature signatures 



The work was funded by the Federal Ministry for Transport, Innovation and Technology (bmvit) and Austrian Science Fund (FWF): TRP 273-N15, supported by Lakeside Labs GmbH, Klagenfurt, Austria and funded by the European Regional Development Fund and the Carinthian Economic Promotion Fund (KWF) under grant 20214/26336/38165.


  1. 1.
    Beecks, C.: Distance-based similarity models for content-based multimedia retrieval. Ph.D. thesis, RWTH Aachen University (2013)Google Scholar
  2. 2.
    Beecks, C., Hassani, M., Hinnell, J., Schüller, D., Brenger, B., Mittelberg, I., Seidl, T.: Spatiotemporal similarity search in 3D motion capture gesture streams. In: Claramunt, C., Schneider, M., Wong, R.C.-W., Xiong, L., Loh, W.-K., Shahabi, C., Li, K.-J. (eds.) SSTD 2015. LNCS, vol. 9239, pp. 355–372. Springer, Heidelberg (2015) CrossRefGoogle Scholar
  3. 3.
    Beecks, C., Kirchhoff, S., Seidl, T.: Signature matching distance for content-based image retrieval. In: ICMR, pp. 41–48 (2013)Google Scholar
  4. 4.
    Beecks, C., Kirchhoff, S., Seidl, T.: On stability of signature-based similarity measures for content-based image retrieval. MTAP 71(1), 349–362 (2014)Google Scholar
  5. 5.
    Blažek, A., Lokoč, J., Matzner, F., Skopal, T.: Enhanced signature-based video browser. In: He, X., Luo, S., Tao, D., Xu, C., Yang, J., Hasan, M.A. (eds.) MMM 2015, Part II. LNCS, vol. 8936, pp. 243–248. Springer, Heidelberg (2015) Google Scholar
  6. 6.
    Cobârzan, C., Del Fabro, M., Schoeffmann, K.: Collaborative browsing and search in video archives with mobile clients. In: He, X., Luo, S., Tao, D., Xu, C., Yang, J., Hasan, M.A. (eds.) MMM 2015, Part II. LNCS, vol. 8936, pp. 266–271. Springer, Heidelberg (2015) Google Scholar
  7. 7.
    Cobârzan, C., Hudelist, M.A., Del Fabro, M.: Content-based video browsing with collaborating mobile clients. In: Gurrin, C., Hopfgartner, F., Hurst, W., Johansen, H., Lee, H., O’Connor, N. (eds.) MMM 2014, Part II. LNCS, vol. 8326, pp. 402–406. Springer, Heidelberg (2014) CrossRefGoogle Scholar
  8. 8.
    Hudelist, M.A., Schoeffmann, K., Boeszoermenyi, L.: Mobile video browsing with the thumbbrowser. In: Proceedings of the 21st ACM International Conference on Multimedia, MM 2013, pp. 405–406. ACM, New York (2013)Google Scholar
  9. 9.
    Hudelist, M.A., Schoeffmann, K., Xu, Q.: Improving interactive known-item search in video with the keyframe navigation tree. In: He, X., Luo, S., Tao, D., Xu, C., Yang, J., Hasan, M.A. (eds.) MMM 2015, Part I. LNCS, vol. 8935, pp. 306–317. Springer, Heidelberg (2015) Google Scholar
  10. 10.
    Hürst, W., Snoek, C.G.M., Spoel, W.-J., Tomin, M.: Size matters! how thumbnail number, size, and motion influence mobile video retrieval. In: Lee, K.-T., Tsai, W.-H., Liao, H.-Y.M., Chen, T., Hsieh, J.-W., Tseng, C.-C. (eds.) MMM 2011 Part II. LNCS, vol. 6524, pp. 230–240. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  11. 11.
    Hürst, W., Snoek, C.G., Spoel, W.-J., Tomin, M.: Keep moving!: revisiting thumbnails for mobile video retrieval. In: Proceedings of the International Conference on Multimedia, MM 2010, pp. 963–966. ACM, New York (2010)Google Scholar
  12. 12.
    Hürst, W., van de Werken, R.: Human-based video browsing - invesitgating interface design for fast video browsing. In: IEEE ISM 2015 (2015, to appear)Google Scholar
  13. 13.
    Hürst, W., van de Werken, R., Hoet, M.: A storyboard-based interface for mobile video browsing. In: He, X., Luo, S., Tao, D., Xu, C., Yang, J., Hasan, M.A. (eds.) MMM 2015, Part II. LNCS, vol. 8936, pp. 261–265. Springer, Heidelberg (2015) Google Scholar
  14. 14.
    Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: Convolutional architecture for fast feature embedding. In: Proceedings of the ACM International Conference on Multimedia, MM 2014, pp. 675–678. ACM, New York (2014)Google Scholar
  15. 15.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) Advances in Neural Information Processing Systems 25, pp. 1097–1105. Curran Associates Inc. (2012)Google Scholar
  16. 16.
    Schoeffmann, K.: A user-centric media retrieval competition: the video browser showdown 2012–2014. IEEE MultiMedia 21(4), 8–13 (2014)CrossRefGoogle Scholar
  17. 17.
    Schoeffmann, K., Ahlström, D., Bailer, W., Cobarzan, C., Hopfgartner, F., McGuinness, K., Gurrin, C., Frisson, C., Le, D.-D., Fabro, M., Bai, H., Weiss, W.: The video browser showdown: a live evaluation of interactive video search tools. Int. J. Multimedia Inf. Retrieval 3, 113–127 (2014)Google Scholar
  18. 18.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556 (2014)
  19. 19.
    Uysal, M.S., Beecks, C., Seidl, T.: On efficient content-based near-duplicate video detection. In: CBMI, pp. 1–6 (2015)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Marco A. Hudelist
    • 1
    Email author
  • Claudiu Cobârzan
    • 1
  • Christian Beecks
    • 3
  • Rob van de Werken
    • 2
  • Sabrina Kletz
    • 1
  • Wolfgang Hürst
    • 2
  • Klaus Schoeffmann
    • 1
  1. 1.Klagenfurt UniversityKlagenfurtAustria
  2. 2.Utrecht UniversityUtrechtThe Netherlands
  3. 3.RWTH Aachen UniversityAachenGermany

Personalised recommendations