Skip to main content
SpringerLink
Log in

The Video Browser Showdown: a live evaluation of interactive video search tools

  • Regular Paper
  • Published:
International Journal of Multimedia Information Retrieval Aims and scope Submit manuscript

Abstract

The Video Browser Showdown evaluates the performance of exploratory video search tools on a common data set in a common environment and in presence of the audience. The main goal of this competition is to enable researchers in the field of interactive video search to directly compare their tools at work. In this paper, we present results from the second Video Browser Showdown (VBS2013) and describe and evaluate the tools of all participating teams in detail. The evaluation results give insights on how exploratory video search tools are used and how they perform in direct comparison. Moreover, we compare the achieved performance to results from another user study where 16 participants employed a standard video player to complete the same tasks as performed in VBS2013. This comparison shows that the sophisticated tools enable better performance in general, but for some tasks common video players provide similar performance and could even outperform the expert tools. Our results highlight the need for further improvement of professional tools for interactive search in videos.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. http://www.videobrowsershowdown.org/.

References

  1. Adams B, Greenhill S, Venkatesh S (2012) Towards a video browser for the digital native. In: Proceedings of IEEE International Conference on Multimedia and Expo Workshops, pp 127–132. doi:10.1109/ICMEW.2012.29

  2. Ahlström D, Hudelist MA, Schoeffmann K, Schaefer G (2012) A user study on image browsing on touchscreens. In: Proceedings of the 20th ACM International Conference on Multimedia, MM ’12, pp 925–928. ACM, New York. doi:10.1145/2393347.2396348

  3. Bailer W, Schoeffmann K, Ahlström D, Weiss W, del Fabro M (2013) Interactive evaluation of video browsing tools. In: Proceedings of the Multimedia Modeling Conference, pp 81–91

  4. Bailer W, Weiss W, Kienast G, Thallinger G, Haas W (2010) A video browsing tool for content management in post-production. Int J Digit Multimed Broadcasting 2010:1–17. doi:10.1155/2010/856761

    Article  Google Scholar 

  5. Barnes C, Goldman DB, Shechtman E, Finkelstein A (2010) Video tapestries with continuous temporal zoom. In: ACM SIGGRAPH 2010 Papers, SIGGRAPH ’10, pp 89:1–89:9. ACM, New York. doi:10.1145/1833349.1778826

  6. Borgo R, Chen M, Daubney B, Grundy E, Heidemann G, H-ferlin B, H-ferlin M, Leitte H, Weiskopf D, Xie X (2012) State of the art report on video-based graphics and video visualization. Comput Graph Forum 31(8):2450–2477. doi:10.1111/j.1467-8659.2012.03158.x

    Article  Google Scholar 

  7. Christel M, Stevens S, Kanade T, Mauldin M, Reddy R, Wactlar H (1996) Multimedia tools and applications. In: Furht B (ed) Techniques for the creation and exploration of digital video libraries., The Kluwer International series in engineering and computer scienceSpringer, US, pp 283–327. doi:10.1007/978-1-4613-1387-8_8

    Google Scholar 

  8. Christel MG, Smith MA, Taylor CR, Winkler DB (1998) Evolving video skims into useful multimedia abstractions. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’98, pp 171–178. ACM Press/Addison-Wesley Publishing Co., New York. doi:10.1145/274644.274670

  9. del Fabro M, Münzer B, Böszörmenyi L (2013) AAU video browser with augmented navigation bars. In: Proceedings of the Multimedia Modeling Conference, pp 544–546

  10. del Fabro M, Schoeffmann K, Böszörmenyi L (2010) Instant video browsing: a tool for fast non-sequential hierarchical video browsing. In: Proceedings of the 6th Symposium of the Workgroup Human-Computer Interaction and Usability, Engineering, pp 443–446. doi:10.1007/978-3-642-16607-5_30

  11. Frisson C, Dupont S, Moinet A, Picard-Limpens C, Ravet T, Siebert X, Dutoit T(2013) Videocycle: user-friendly navigation by similarity in video databases. In: Proceedings of the Multimedia Modeling Conference, pp 550–553

  12. Girgensohn A, Shipman F, Wilcox L (2011) Adaptive clustering and interactive visualizations to support the selection of video clips. In: Proceedings of the 1st ACM International Conference on Multimedia Retrieval, pp 34:1–34:8. doi:10.1145/1991996.1992030

  13. Huber J, Steimle J, Lissermann R, Olberding S, Mühlhäuser M (2010) Wipe‘n’watch: spatial interaction techniques for interrelated video collections on mobile devices. In: Proceedings of the 24th BCS Interaction Specialist Group Conference, pp 423–427

  14. Hürst W, Götz G, Welte M (2007) Interactive video browsing on mobile devices. In: Proceedings of the 15th international conference on Multimedia, pp 247–256. doi:10.1145/1291233.1291284

  15. Jansen M, Heeren W, van Dijk B (2008) Videotrees: improving video surrogate presentation using hierarchy. In: Content-based multimedia indexing. International Workshop on CBMI 2008, pp 560–567. doi:10.1109/CBMI.2008.4564997

  16. Jégou H, Douze M, Schmid C (2011) Product quantization for nearest neighbor search. IEEE Trans Pattern Anal Mach Intell 33:117–128

    Article  Google Scholar 

  17. Jégou H, Douze M, Schmid C, Perez P (2010) Aggregating local descriptors into a compact image representation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 3304–3311

  18. Jiang J, Zhang XP (2011) A smart video player with content-based fast-forward playback. In: Proceedings of the 19th ACM international conference on Multimedia, pp 1061–1064. doi:10.1145/2072298.2071938

  19. Le DD, Lam V, Ngo TD, Tran VQ, Nguyen VH, Duong DA, Satoh S (2013) NII-UIT-VBS: A video browsing tool for known item search. In: Proceedings of the Multimedia Modeling Conference, pp 547–549

  20. Lowe D (1999) Object recognition from local scale-invariant features. In: IEEE International Conference on Computer Vision, pp 1150–1157

  21. Marchand-Maillet S, Morrison D, Szekely E, Bruno E (2010) Interactive representations of multimodal databases. In: Thiran J-P, Marques F, Bourlard H (eds) Multimodal signal processing, theory and applications for human-computer interaction. Elsevier, Amsterdam, ISBN: 9780123748256

  22. Meixner B, Köstler J, Kosch H (2011) A mobile player for interactive non-linear video. In: Proceedings of the 19th ACM international conference on Multimedia, pp 779–780. doi:10.1145/2072298.2072453

  23. Mueller C, Smole M, Schöffmann K (2012) A demonstration of a hierarchical multi-layout 3D video browser. In: Proceedings of the IEEE International Conference on Multimedia and Expo Workshops. doi:10.1109/ICMEW.2012.121

  24. Over P, Awad G, Michel M, Fiscus J, Sanders G, Shaw B, Kraaij W, Smeaton AF, Quénot G (2012) Trecvid 2012: an overview of the goals, tasks, data, evaluation mechanisms and metrics. In: Proceedings of TRECVID 2012

  25. de Rooij O, Snoek CG, Worring M (2008) Balancing thread based navigation for targeted video search. In: Proceedings of the 2008 international conference on content-based image and video retrieval, pp 485–494. doi:10.1145/1386352.1386414

  26. Schoeffmann K, Boeszoermenyi L (2011) Image and video browsing with a cylindrical 3D storyboard. In: Proceedings of the 1st ACM International Conference on Multimedia Retrieval, ICMR ’11, pp 63:1–63:2. ACM, New York. doi:10.1145/1991996.1992059

  27. Schoeffmann K, Cobarzan C (2013) An evaluation of interactive search with modern video players. In: IEEE International Conference on Multimedia and Expo Workshops (ICMEW), 2013, pp 1–4. doi:10.1109/ICMEW.2013.6618282

  28. Schoeffmann K, Fabro MD (2011) Hierarchical video browsing with a 3D carousel. In: Proceedings of the 19th ACM international conference on Multimedia, pp 827–828. doi:10.1145/2072298.2072479

  29. Schoeffmann K, Hopfgartner F, Marques O, Böszörmenyi L, Jose JM (2010) Video browsing interfaces and applications: a review. SPIE Rev 1(1):018004. doi:10.1117/6.0000005

    Google Scholar 

  30. Schoeffmann K, Taschwer M, Böszörmenyi L (2010) The video explorer: a tool for navigation and searching within a single video based on fast content analysis. In: Proceedings of the first annual ACM SIGMM conference on Multimedia systems, pp 247–258. doi:10.1145/1730836.1730867

  31. Scott D, Guo J, Gurrin C, Hopfgartner F, McGuinness K, O’Connor N, Smeaton A, Yang Y, Zhang Z (2013) DCU at MMM 2013 Video Browser Showdown. In: Proceedings of the Multimedia Modeling Conference, pp 541–543

  32. Scott D, Hopfgartner F, Guo J, Gurrin C (2013) Evaluating novice and expert users on handheld video retrieval systems. In: Proceedings of MMM, pp 69–78

  33. Smeaton AF, Over P, Kraaij W (2006) Evaluation campaigns and TRECVid. In: Proceedings of the 8th ACM International Workshop on Multimedia Information Retrieval, pp 321–330. http://doi.acm.org/10.1145/1178677.1178722

  34. Snoek C, Worring M, de Rooij O, van de Sande K, Yan R, Hauptmann A (2008) Videolympics: real-time evaluation of multimedia retrieval systems. IEEE Multimed 15(1):86–91. doi:10.1109/MMUL.2008.21

    Article  Google Scholar 

  35. Tao K, Dong Y, Bian Y, Chang X, Bai H (2012) The France Telecom Orange Labs (Beijing) video semantic indexing systems. In: TRECVID 2012

  36. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 511–518 (2001)

  37. Wilkins P, Byrne D, Jones GJF, Lee H, Keenan G, McGuinness K, O’Connor NE, O’Hare N, Smeaton AF, Adamek T, Troncy R, Amin A, Benmokhtar R, Dumont E, Huet B, Mérialdo B, Tolias G, Spyrou E, Avrithis YS, Papadopoulos G, Mezaris V, Kompatsiaris I, Mörzinger R, Schallauer P, Bailer W, Chandramouli K, Izquierdo E, Goldmann L, Haller M, Samour A, Cobet A, Sikora T, Praks P, Hannah D, Halvey M, Hopfgartner F, Villa R, Punitha P, Goyal A, Jose JM (2008) K-space at trecvid 2008. In: TRECVID

  38. Worring M, Sajda P, Santini S, Shamma DA, Smeaton AF, Yang Q (2012) Where is the user in multimedia retrieval? IEEE MultiMed 19(4):6–10

    Article  Google Scholar 

Download references

Acknowledgments

This work was funded by the Austrian Federal Ministry for Transport, Innovation and Technology (bmvit) and Austrian Science Fund (FWF): TRP 273-N15 and the European Regional Development Fund and the Carinthian Economic Promotion Fund (KWF), supported by Lakeside Labs GmbH, Klagenfurt, Austria. The research leading to these results has received funding from the European Union’s Seventh Framework Programme. (FP7/2007-2013) under grant agreement no. 287532 entitled “TOSCA-MP”. Christian Frisson has been funded by the numediart research program in digital arts technologies, grant from the Walloon Region of Belgium, from 2010 to 2013. He would like to acknowledge his colleagues part of the VideoCycle team and doctoral jury, Thierry Dutoit (advisor) and Stéphane Dupont, that lead to the participation of the team to VBS 2013. The content used in the experiments has been made available for the EBU MIM/SCAIE project by the Flemish public broadcasting organisation VRT (http://www.vrt.be/), Belgium.

Author information

Authors and Affiliations

  1. Alpen-Adria-Universität Klagenfurt, Klagenfurt, Austria

    Klaus Schoeffmann, David Ahlström, Claudiu Cobârzan & Manfred Del Fabro

  2. Joanneum Research, Graz, Austria

    Werner Bailer & Wolfgang Weiss

  3. Technische Universität Berlin, Berlin, Germany

    Frank Hopfgartner

  4. Dublin City University, Dublin, Ireland

    Kevin McGuinness & Cathal Gurrin

  5. Université de Mons, Mons, Belgium

    Christian Frisson

  6. National Institute of Informatics, Tokyo, Japan

    Duy-Dinh Le

  7. Orange Labs International Centers, Beijing, China

    Hongliang Bai

Authors
  1. Klaus Schoeffmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Ahlström
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Werner Bailer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Claudiu Cobârzan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Frank Hopfgartner
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kevin McGuinness
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Cathal Gurrin
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Christian Frisson
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Duy-Dinh Le
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Manfred Del Fabro
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Hongliang Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Wolfgang Weiss
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Klaus Schoeffmann.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Schoeffmann, K., Ahlström, D., Bailer, W. et al. The Video Browser Showdown: a live evaluation of interactive video search tools. Int J Multimed Info Retr 3, 113–127 (2014). https://doi.org/10.1007/s13735-013-0050-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13735-013-0050-8

Keywords

Search

Navigation