Computational Visual Media

, Volume 2, Issue 3, pp 291–304 | Cite as

VideoMap: An interactive and scalable visualization for exploring video content

  • Cui-Xia Ma
  • Yang Guo
  • Hong-An Wang
Open Access
Research Article


Large-scale dynamic relational data visualization has attracted considerable research attention recently. We introduce dynamic data visualization into the multimedia domain, and present an interactive and scalable system, VideoMap, for exploring large-scale video content. A long video or movie has much content; the associations between the content are complicated. VideoMap uses new visual representations to extract meaningful information from video content. Map-based visualization naturally and easily summarizes and reveals important features and events in video. Multi-scale descriptions are used to describe the layout and distribution of temporal information, spatial information, and associations between video content. Firstly, semantic associations are used in which map elements correspond to video contents. Secondly, video contents are visualized hierarchically from a large scale to a fine-detailed scale. VideoMap uses a small set of sketch gestures to invoke analysis, and automatically completes charts by synthesizing visual representations from the map and binding them to the underlying data. Furthermore, VideoMap allows users to use gestures to move and resize the view, as when using a map, facilitating interactive exploration. Our experimental evaluation of VideoMap demonstrates how the system can assist in exploring video content as well as significantly reducing browsing time when trying to understand and find events of interest.


map metaphor video content visualization sketch-based interaction association analysis 


  1. [1]
    Mashima, D.; Kobourov, S.; Hu, Y. Visualizing dynamic data with maps. IEEE Transactions on Visualization and Computer Graphics Vol. 18, No. 9, 1424–1437, 2012.CrossRefGoogle Scholar
  2. [2]
    Gansner, E. R.; Hu, Y.; North, S. Visualizing streaming text data with dynamic graphs and maps. Lecture Notes in Computer Science, Vol. 7704. Didimo, W.; Patrignani, M. Eds. Springer Berlin Heidelberg, 439–450, 2013.MathSciNetCrossRefzbMATHGoogle Scholar
  3. [3]
    Information on Scholar
  4. [4]
    Fels, S.; Mase, K. Interactive video cubism. In: Proceedings of the 1999 Workshop on New Paradigms in Information Visualization and Manipulation in Conjunction with the 8th ACM International Conference on Information and Knowledge Management, 78–82, 1999.Google Scholar
  5. [5]
    Daniel, G.; Chen, M. Video visualization. In: Proceedings of IEEE Visualization, 409–416, 2003.Google Scholar
  6. [6]
    Yeung, M. M.; Yeo, B.-L. Video visualization for compact presentation and fast browsing of pictorial content. IEEE Transactions on Circuits and Systems for Video Technology Vol. 7, No. 5, 771–785, 1997.CrossRefGoogle Scholar
  7. [7]
    Taniguchi, Y.; Akutsu, A.; Tonomura, Y. Panorama Excerpts: Extracting and packing panoramas for video browsing. In: Proceedings of the 5th ACM International Conference on Multimedia, 427–436, 1997.Google Scholar
  8. [8]
    Hua, X.-S.; Li, S.; Zhang, H.-J. Video booklet. 2010. Available at Scholar
  9. [9]
    Goldman, D. B.; Curless, B.; Salesin, D.; Seitz, S. M. Schematic storyboarding for video visualization and editing. ACM Transactions on Graphics Vol. 25, No. 3, 862–871, 2006.CrossRefGoogle Scholar
  10. [10]
    Nguyen, C.; Niu, Y.; Liu, F. Video summagator: An interface for video summarization and navigation. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 647–650, 2012.Google Scholar
  11. [11]
    Shah, R.; Narayanan, P. J. Interactive video manipulation using object trajectories and scene backgrounds. IEEE Transactions on Circuits and Systems for Video Technology Vol. 23, No. 9, 1565–1576, 2013.CrossRefGoogle Scholar
  12. [12]
    Ma, C.-X.; Liu, Y.-J.; Wang, H.-A.; Teng, D.-X.; Dai, G.-Z. Sketch-based annotation and visualization in video authoring. IEEE Transactions on Multimedia Vol. 14, No. 4, 1153–1165, 2012.CrossRefGoogle Scholar
  13. [13]
    Truong, B. T.; Venkatesh, S. Video abstraction: A systematic review and classification. ACM Transactions on Multimedia Computing, Communications, and Applications Vol. 3, No. 1, Article No. 3, 2007.Google Scholar
  14. [14]
    Viaud, M.-l.; Buisson, O.; Saulnier, A.; Guenais, C. Video exploration: From multimedia content analysis to interactive visualization. In: Proceedings of the 18th ACM International Conference on Multimedia, 1311–1314, 2010.Google Scholar
  15. [15]
    Höferlin, M.; Höferlin, B.; Weiskopf, D. Video visual analytics of tracked moving objects. 2012. Available at vispublications/Hoeferlin2009b.pdf.Google Scholar
  16. [16]
    Meghdadi, A. H.; Irani, P. Interactive exploration of surveillance video through action shot summarization and trajectory visualization. IEEE Transactions on Visualization and Computer Graphics Vol. 19, No. 12, 2119–2128, 2013.CrossRefGoogle Scholar
  17. [17]
    Walton, S.; Berger, K.; Ebert, D.; Chen, M. Vehicle object retargeting from dynamic traffic videos for realtime visualisation. The Visual Computer Vol. 30, No. 5, 493–505, 2014.CrossRefGoogle Scholar
  18. [18]
    Höferlin, B.; Höferlin, M.; Heidemann, G.; Weiskopf, D. Scalable video visual analytics. Information Visualization Vol. 14, No. 1, 10–26, 2013.CrossRefGoogle Scholar
  19. [19]
    Caspi, Y.; Axelrod, A.; Matsushita, Y.; Gamliel, A. Dynamic stills and clip trailers. The Visual Computer Vol. 22, No. 9, 642–652, 2006.CrossRefGoogle Scholar
  20. [20]
    Correa, C. D.; Ma, K.-L. Dynamic video narratives. ACM Transactions on Graphics Vol. 29, No. 4, Article No. 88, 2010.Google Scholar
  21. [21]
    Tang, A.; Greenberg, S.; Fels, S. Exploring video streams using slit-tear visualizations. In: Proceedings of Extended Abstracts on Human Factors in Computing Systems, 3509–3510, 2009.Google Scholar
  22. [22]
    Tanahashi, Y.; Ma, K.-L. Design considerations for optimizing storyline visualizations. IEEE Transactions on Visualization and Computer Graphics Vol. 18, No. 12, 2679–2688, 2012.CrossRefGoogle Scholar
  23. [23]
    Li, F. C.; Gupta, A.; Sanocki, E.; He, L.-w.; Rui, Y. Browsing digital video. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 169–176, 2000.CrossRefGoogle Scholar
  24. [24]
    Liu, Y.-J.; Ma, C.-X.; Fu, Q.; Fu, X.; Qin, S.-F.; Xie, L. A sketch-based approach for interactive organization of video clips. ACM Transactions on Multimedia Computing, Communications, and Applications Vol. 11, No. 1, Article No. 2, 2014.Google Scholar
  25. [25]
    Jawaheer, G.; Weller, P.; Kostkova, P. Modeling user preferences in recommender systems: A classification framework for explicit and implicit user feedback. ACM Transactions on Interactive Intelligent Systems Vol. 4, No. 2, Article No. 8, 2014.Google Scholar
  26. [26]
    Zhang, X.; Furnas, G. W. mCVEs: Using crossscale collaboration to support user interaction with multiscale structures. Presence Vol. 14, No. 1, 31–46, 2005.CrossRefGoogle Scholar
  27. [27]
    McCarthy, J. F.; Meidel, E. S. ActiveMap: A visualization tool for location awareness to support informal interactions. Lecture Notes in Computer Science, Vol. 1707. Gellersen, H.-W. Ed. Springer Berlin Heidelberg, 158–170, 2000.CrossRefGoogle Scholar
  28. [28]
    Nesbitt, K. V. Getting to more abstract places using the metro map metaphor. In: Proceedings of the 8th International Conference on Information Visualisation, 488–493, 2004.Google Scholar
  29. [29]
    Gansner, E. R.; Hu, Y.; Kobourov, S. GMap: Visualizing graphs and clusters as maps. In: Proceedings of IEEE Pacific Visualization Symposium, 201–208, 2010.Google Scholar
  30. [30]
    Fu, X. L.; Cai, L. H.; Liu, Y.; Jia, J.; Chen, W. F.; Yi, Z.; Zhao, G. Z.; Liu, Y. J.; Wu, C. X. A computational cognition model of perception, memory, and judgment. Science China Information Sciences Vol. 57, No. 3, 1–15, 2014.CrossRefGoogle Scholar
  31. [31]
    Solway, A.; Botvinick, M. M. Goal-directed decision making as probabilistic inference: A computational framework and potential neural correlates. Psychological Review Vol. 119, No. 1, 120–154, 2012.CrossRefGoogle Scholar

Copyright information

© The Author(s) 2016

Authors and Affiliations

  1. 1.State Key Lab of Computer Science, Institute of SoftwareChinese Academy of SciencesBeijingChina
  2. 2.School of Computer and Control EngineeringUniversity of Chinese Academy of SciencesBeijingChina
  3. 3.Beijing Key Lab of Human–Computer Interaction, Institute of SoftwareChinese Academy of SciencesBeijingChina

Personalised recommendations