Navigating a Graph of Scenes for Exploring Large Video Collections

  • Kai Uwe BarthelEmail author
  • Nico Hezel
  • Radek Mackowiak
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9517)


We present a novel approach to browse huge sets of video scenes using a hierarchical graph and visually sorted image maps allowing the user to explore the graph similar to navigation services. In a previous paper [1] we proposed a scheme to generate such a graph of video scenes and investigated several browsing and visualization concepts. In this paper we extend our work by adding semantic features learned from a convolutional neural network. In combination with visual features we constructed an improved graph where related images (video scenes) are connected with each other. Different images or areas in the graph may be reached by following the most promising path of edges. For efficient navigation we propose a method which projects images onto a 2D plane preserving their complex inter-image relationships. To start a search process, the user may either choose from a selection of typical videos scenes or use tools such as search by sketch or category. The retrieved video frames are arranged on a canvas and the view of the graph is directed to a location where matching frames can be found.


Content-based video retrieval Exploration Image browsing Visualization Navigation Convolutional neural networks 


  1. 1.
    Barthel, K.U., Hezel, N., Mackowiak, R.: Graph-based browsing for large video collections. In: He, X., Luo, S., Tao, D., Xu, C., Yang, J., Hasan, M.A. (eds.) MMM 2015, Part II. LNCS, vol. 8936, pp. 237–242. Springer, Heidelberg (2015)Google Scholar
  2. 2.
    Schoeffmann, K., et al.: The video browser showdown: a live evaluation of interactive video search tools. Int. J. Multimed. Inf. Retr. (MMIR) 3(2), 113–127 (2014)Google Scholar
  3. 3.
    Barthel, K.U., Hezel, N., Mackowiak, R.: ImageMap - visually browsing millions of images. In: He, X., Luo, S., Tao, D., Xu, C., Yang, J., Hasan, M.A. (eds.) MMM 2015, Part II. LNCS, vol. 8936, pp. 287–290. Springer, Heidelberg (2015)Google Scholar
  4. 4. Accessed 21 September 2015
  5. 5.
    Krizhevsky, A., Sutskever, I. Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: NIPS 2012, Neural Information Processing Systems, Lake Tahoe, Nevada (2012)Google Scholar
  6. 6.
    Razavian, A.S., Azizpour, H., Sullivan, J., Carlsson, S.: CNN features off-the-shelf: an astounding baseline for recognition. In: CVPR Workshops 2014, pp. 512–519 (2014)Google Scholar
  7. 7.
    Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet large scale visual recognition challenge. IJCV 115(3), 211–252 (2015)CrossRefGoogle Scholar
  8. 8.
    Coates, A., Lee, H., Ng, A.Y.: An analysis of single layer networks in unsupervised feature learning. In: AISTATS (2011)Google Scholar
  9. 9.
    Linde, Y., Buzo, A., Gray, R.: An algorithm for vector quantizer design. IEEE Trans. Commun. 28, 84 (1980)CrossRefGoogle Scholar
  10. 10.
    Donald, S.: A two-dimensional interpolation function for irregularly-spaced data. In: Proceedings of the 1968 ACM National Conference, pp. 517–524 (1968)Google Scholar
  11. 11.
    Lokoč, J., Blažek, A., Skopal, T.: Signature-based video browser. In: Hopfgartner, F., Hurst, W., Johansen, H., Lee, H., O’Connor, N., Gurrin, C. (eds.) MMM 2014, Part II. LNCS, vol. 8326, pp. 415–418. Springer, Heidelberg (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Visual Computing GroupHTW Berlin, University of Applied SciencesBerlinGermany

Personalised recommendations