Advertisement

Improving Interactive Known-Item Search in Video with the Keyframe Navigation Tree

  • Marco A. Hudelist
  • Klaus Schoeffmann
  • Qing Xu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8935)

Abstract

In this paper we propose the Keyframe Navigation Tree (KNT) as navigational aid in video for interactive search. The KNT is a hierarchical visualization of keyframes that can compactly represent the content of a video with different levels of details. It can be used as an alternative, or in addition, to a common seeker-bar of a video player. Through a user study with 20 participants we show that the proposed navigation approach not only allows significantly faster interactive search in video than a common video player, but also requires significantly less effort (also less mental and physical load) and is much more enjoyable to use.

Keywords

User Study Video Content Shot Boundary Video Player Interactive Search 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Ahlström, D., Schoeffmann, K.: A visual search user study on the influences of aspect ratio distortion of preview thumbnails. In: Zhang, J., Schonfeld, D., Feng, D.D., Nanyang, J.C., Hanjalic, A., Magli, E., Pickering, M., Friedland, G., Hua, X.-S. (eds.) Proceedings of the 2012 IEEE International Conference on Multimedia and Expo Workshops, Los Alamitos, CA, USA, pp. 546–551. IEEE Computing Society (July 2012)Google Scholar
  2. 2.
    Bailer, W., Schoeffmann, K., Ahlström, D., Weiss, W., Del Fabro, M.: Interactive evaluation of video browsing tools. In: Li, S., El Saddik, A., Wang, M., Mei, T., Sebe, N., Yan, S., Hong, R., Gurrin, C. (eds.) MMM 2013, Part I. LNCS, vol. 7732, pp. 81–91. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  3. 3.
    Cobârzan, C., Hudelist, M.A., Del Fabro, M.: Content-based video browsing with collaborating mobile clients. In: Gurrin, C., Hopfgartner, F., Hurst, W., Johansen, H., Lee, H., O’Connor, N. (eds.) MMM 2014, Part II. LNCS, vol. 8326, pp. 402–406. Springer, Heidelberg (2014), http://dx.doi.org/10.1007/978-3-319-04117-9_46 CrossRefGoogle Scholar
  4. 4.
    Rooij, O.d., Snoek, C.G., Worring, M.: Query on demand video browsing. In: Proceedings of the 15th International Conference on Multimedia, pp. 811–814. ACM (2007)Google Scholar
  5. 5.
    de Rooij, O., Snoek, C.G.M., Worring, M.: Mediamill: semantic video search using the rotorbrowser. In: Proceedings of the 6th ACM International Conference on Image and Video Retrieval, pp. 649–649. ACM Press (2007)Google Scholar
  6. 6.
    Dragicevic, P., Ramos, G., Bibliowitcz, J., Nowrouzezahrai, D., Balakrishnan, R., Singh, K.: Video browsing by direct manipulation. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2008, pp. 237–246. ACM, New York (2008)Google Scholar
  7. 7.
    Del Fabro, M., Böszörmenyi, L.: AAU video browser: Non-sequential hierarchical video browsing without content analysis. In: Schoeffmann, K., Merialdo, B., Hauptmann, A.G., Ngo, C.-W., Andreopoulos, Y., Breiteneder, C. (eds.) MMM 2012. LNCS, vol. 7131, pp. 639–641. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  8. 8.
    Girgensohn, A., Shipman, F., Wilcox, L.: Adaptive clustering and interactive visualizations to support the selection of video clips. In: Proceedings of the 1st ACM International Conference on Multimedia Retrieval, p. 34. ACM (2011)Google Scholar
  9. 9.
    Goeau, H., Thièvre, J., Viaud, M.-L., Pellerin, D.: Interactive visualization tool with graphic table of video contents. In: 2007 IEEE International Conference on Multimedia and Expo, pp. 807–810. IEEE (2007)Google Scholar
  10. 10.
    Hanjalic, A.: Shot-boundary detection: unraveled and resolved? IEEE Transactions on Circuits, Systems, and Video Technology 12(2), 90–105 (2002)CrossRefGoogle Scholar
  11. 11.
    Hart, S.G., Staveland, L.: Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In: Hancock, P.A., Meshkati, N. (eds.) Human Mental Workload, pp. 139–183. Elsevier, Amsterdam (1988)CrossRefGoogle Scholar
  12. 12.
    Huber, J., Steimle, J., Lissermann, R., Olberding, S., Mühlhäuser, M.: Wipe’n’watch: spatial interaction techniques for interrelated video collections on mobile devices. In: Proceedings of the 24th BCS Interaction Specialist Group Conference, BCS 2010, pp. 423–427. British Computer Society, Swinton (2010)Google Scholar
  13. 13.
    Hürst, W., Götz, G., Welte, M.: Interactive video browsing on mobile devices. In: Proceedings of the 15th International Conference on Multimedia, MULTIMEDIA 2007, pp. 247–256. ACM, New York (2007)Google Scholar
  14. 14.
    Hürst, W., Meier, K.: Interfaces for timeline-based mobile video browsing. In: Proceedings of the 16th ACM International Conference on Multimedia, pp. 469–478. ACM (2008)Google Scholar
  15. 15.
    Hürst, W., Snoek, C.G.M., Spoel, W.-J., Tomin, M.: Size matters! how thumbnail number, size, and motion influence mobile video retrieval. In: Lee, K.-T., Tsai, W.-H., Liao, H.-Y.M., Chen, T., Hsieh, J.-W., Tseng, C.-C. (eds.) MMM 2011 Part II. LNCS, vol. 6524, pp. 230–240. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  16. 16.
    Jansen, M., Heeren, W., van Dijk, B.: Videotrees: Improving video surrogate presentation using hierarchy. In: International Workshop on Content-Based Multimedia Indexing, CBMI 2008, pp. 560–567. IEEE (2008)Google Scholar
  17. 17.
    Karrer, T., Wittenhagen, M., Borchers, J.: Pocketdragon: a direct manipulation video navigation interface for mobile devices. In: Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2009, pp. 47:1–47:3. ACM, New York (2009)Google Scholar
  18. 18.
    Lienhart, R., Pfeiffer, S., Effelsberg, W.: Video abstracting. Commun. ACM 40(12), 54–62 (1997)CrossRefGoogle Scholar
  19. 19.
    Luo, X., Xu, Q., Sbert, M., Schoeffmann, K.: F-divergences driven video key frame extraction. In: 2014 IEEE International Conference on Multimedia & Expo (ICME 2014). IEEE (2014)Google Scholar
  20. 20.
    Matejka, J., Grossman, T., Fitzmaurice, G.: Swifter: Improved online video scrubbing. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2013, pp. 1159–1168. ACM, New York (2013)CrossRefGoogle Scholar
  21. 21.
    Mueller-Seelich, H., Tan, E.: Visualizing the semantic structure of film and video (2000)Google Scholar
  22. 22.
    Münzer, B., Schoeffmann, K., Böszörmenyi, L.: Relevance segmentation of laparoscopic videos. In: Proceedings of the 2013 IEEE International Symposium on Multimedia (ISM 2013), pp. 1–8 (2013)Google Scholar
  23. 23.
    Over, P., Awad, G., Michel, M., Fiscus, J., Sanders, G., Shaw, B., Kraaij, W., Smeaton, A.F., Quénot, G.: Trecvid 2012 – an overview of the goals, tasks, data, evaluation mechanisms and metrics. In: Proceedings of TRECVID 2012 (2012)Google Scholar
  24. 24.
    Schoeffmann, K.: A user-centric media retrieval competition: The video browser showdown 2012-2014. IEEE Multimedia Magazine, 1–5 (to appear, 2014)Google Scholar
  25. 25.
    Schoeffmann, K., Ahlstrom, D., Bailer, W., Cobarzan, C., Hopfgartner, F., McGuinness, K., Gurrin, C., Frisson, C., Le, D.-D., Del Fabro, M., Bai, H., Weiss, W.: The video browser showdown: a live evaluation of interactive video search tools. International Journal of Multimedia Information Retrieval 3, 113–127 (2014)Google Scholar
  26. 26.
    Schoeffmann, K., Ahlström, D., Böszörmenyi, L.: A user study of visual search performance with interactive 2D and 3D storyboards. In: Detyniecki, M., García-Serrano, A., Nürnberger, A., Stober, S. (eds.) AMR 2011. LNCS, vol. 7836, pp. 18–32. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  27. 27.
    Schoeffmann, K., Bailer, W.: Video browser showdown. ACM SIGMultimedia Records 4(2), 1–2 (2012)CrossRefGoogle Scholar
  28. 28.
    Schoeffmann, K., Boeszoermenyi, L.: Video browsing using interactive navigation summaries. In: Proceedings of the 7th International Workshop on Content-Based Multimedia Indexing, pp. 243–248. IEEE, Chania (2009)Google Scholar
  29. 29.
    Schoeffmann, K., Cobârzan, C.: An evaluation of interactive search with modern video players. In: IEEE International Conference on Multimedia and Expo Workshops (ICMEW), pp. 1–4 (July 2013)Google Scholar
  30. 30.
    Schoeffmann, K., Hopfgartner, F., Marques, O., Boeszoermenyi, L., Jose, J.M.: Video browsing interfaces and applications: a review. SPIE Reviews 1(1), 018004 (2010)Google Scholar
  31. 31.
    Schoeffmann, K., Taschwer, M., Boeszoermenyi, L.: The video explorer: A tool for navigation and searching within a single video based on fast content analysis. In: Proceedings of the First Annual ACM SIGMM Conference on Multimedia Systems, MMSys 2010, pp. 247–258. ACM, New York (2010)CrossRefGoogle Scholar
  32. 32.
    Xu, Q., Li, X., Yang, Z., Wang, J., Sbert, M., Li, J.: Key frame selection based on jensen-rényi divergence. In: 2012 21st International Conference on Pattern Recognition (ICPR), pp. 1892–1895. IEEE (2012)Google Scholar
  33. 33.
    Xu, Q., Liu, Y., Li, X., Yang, Z., Wang, J., Sbert, M., Scopigno, R.: Browsing and exploration of video sequences: A new scheme for key frame extraction and 3d visualization using entropy based jensen divergence. Information Sciences 278, 736–756 (2014)CrossRefMathSciNetGoogle Scholar
  34. 34.
    Xu, Q., Wang, P.-C., Long, B., Sbert, M., Feixas, M., Scopigno, R.: Selection and 3d visualization of video key frames. In: Proceedings of IEEE International Conference on Systems Man and Cybernetics (SMC), pp. 52–59 (2010)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Marco A. Hudelist
    • 1
  • Klaus Schoeffmann
    • 1
  • Qing Xu
    • 2
  1. 1.Klagenfurt UniversityKlagenfurtAustria
  2. 2.School of Computer Science and TechnologyTianjin UniversityTianjinChina

Personalised recommendations

Cite paper

Buy options