Multimedia Tools and Applications

, Volume 55, Issue 1, pp 127–150 | Cite as

Information-based adaptive fast-forward for visual surveillance

  • Benjamin Höferlin
  • Markus Höferlin
  • Daniel Weiskopf
  • Gunther Heidemann


Automated video analysis lacks reliability when searching for unknown events in video data. The practical approach is to watch all the recorded video data, if applicable in fast-forward mode. In this paper we present a method to adapt the playback velocity of the video to the temporal information density, so that the users can explore the video under controlled cognitive load. The proposed approach can cope with static changes and is robust to video noise. First, we formulate temporal information as symmetrized Rényi divergence, deriving this measure from signal coding theory. Further, we discuss the animated visualization of accelerated video sequences and propose a physiologically motivated blending approach to cope with arbitrary playback velocities. Finally, we compare the proposed method with the current approaches in this field by experiments and a qualitative user study, and show its advantages over motion-based measures.


Information theory Adaptive fast-forward Video browsing  Video summarization Visual surveillance 



We’d like to thank Michael Wörner for proofreading this paper. This work was funded by German Research Foundation (DFG) as part of the Priority Program “Scalable Visual Analytics” (SPP 1335).


  1. 1.
    Basseville M (1988) Distance measures for signal processing and pattern recognition. Technical Report 899, Institut de Recherche En Informatique Et Systemes Aleatoires (IRISA)Google Scholar
  2. 2.
    Blyth S (1994) Local divergence and association. Biometrika 81(3):579–584MathSciNetMATHCrossRefGoogle Scholar
  3. 3.
    Cheng K, Luo S, Chen B, Chu H (2009) Smartplayer: user-centric video fast-forwarding. In: Proceedings of the 27th international conference on human factors in computing systems (CHI). ACM, New York, pp 789–798CrossRefGoogle Scholar
  4. 4.
    Cover T, Thomas J (2006) Elements of information theory. Wiley-InterscienceGoogle Scholar
  5. 5.
    Fairchild MD (2005) Color appearance models. Wiley-IS&T, ChichesterGoogle Scholar
  6. 6.
    Ghazal M, Amer A, Ghrayeb A (2007) A real-time technique for spatio-temporal video noise estimation. IEEE Trans Circuits Syst Video Technol 17(12):1690–1699CrossRefGoogle Scholar
  7. 7.
    Gill M, Spriggs A (2005) Assessing the impact of CCTV. Home office research, development and statistics directorate. Home Office Research Study 292Google Scholar
  8. 8.
    Hero A, Ma B, Michel O, Gorman J (2002) Alpha-divergence for classification, indexing and retrieval. Technical Report CSPL-328, Communication and Signal Processing Laboratory, The University of MichiganGoogle Scholar
  9. 9.
    Horn B, Schunck B (1981) Determining optical flow. Computer Vision 17:185–203Google Scholar
  10. 10.
    Kang H, Chen X, Matsushita Y, Tang X (2006) Space-time video montage. In: 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR), vol 2, pp 1331–1338Google Scholar
  11. 11.
    Kolb H, Fernandez E, Nelson R (2007) Webvision: the organization of the retina and visual system. Part IX. Psychophysics of vision. National Library of Medicine (US), NCBI. Available from:
  12. 12.
    Kullback S (1959) Information theory and statistics. Wiley Publication in Mathematical StatisticsGoogle Scholar
  13. 13.
    Lin J (1991) Divergence measures based on the Shannon entropy. IEEE Trans Inf Theory 37(1):145–151MATHCrossRefGoogle Scholar
  14. 14.
    Neemuchwala H, Hero A, Carson P (2005) Image matching using alpha-entropy measures and entropic graphs. Signal Process 85(2):277–296MATHCrossRefGoogle Scholar
  15. 15.
    Peker K, Divakaran A (2004) Adaptive fast playback-based video skimming using a compressed-domain visual complexity measure. In: 2004 IEEE international conference on multimedia and expo, 2004. ICME’04, vol 3, pp 2055–2058Google Scholar
  16. 16.
    Peker K, Divakaran A, Sun H (2001) Constant pace skimming and temporal sub-sampling of video using motion activity. In: Proc. IEEE international conference on image processing (ICIP), vol 3. Thessaloniki, pp 414–417Google Scholar
  17. 17.
    Petrovic N, Jojic N, Huang T (2005) Adaptive video fast forward. Multimed Tools Appl 26(3):327–344CrossRefGoogle Scholar
  18. 18.
    Poynton C (2003) Digital video and HDTV: algorithms and interfaces. Morgan Kaufmann, San FranciscoGoogle Scholar
  19. 19.
    Pritch Y, Rav-Acha A, Gutman A, Peleg S (2007) Webcam synopsis: peeking around the world. In: Proc. ICCV, pp 1–8Google Scholar
  20. 20.
    Pritch Y, Rav-Acha A, Peleg S (2008) Nonchronological video synopsis and indexing. IEEE Trans Pattern Anal Mach Intell 30(11):1971–1984CrossRefGoogle Scholar
  21. 21.
    Punchihewa A, Bailey D (2002) Artefacts in image and video systems; classification and mitigation. In: Proceedings of image and vision computing New Zealand, pp 197–202Google Scholar
  22. 22.
    Ramos G, Balakrishnan R (2003) Fluid interaction techniques for the control and annotation of digital video. In: Proceedings of the 16th annual ACM symposium on user interface software and technology (UIST). ACM, New York, pp 105–114CrossRefGoogle Scholar
  23. 23.
    Rav-Acha A, Pritch Y, Peleg S (2006) Making a long video short: dynamic video synopsis. In: IEEE computer society conference on computer vision and pattern recognition (CVPR), vol 1, pp 435–441Google Scholar
  24. 24.
    Renyi A (1961) On measures of entropy and information. In: Proceedings of the 4th Berkeley symposium on mathematical statistics and probability, vol 1, pp 547–561Google Scholar
  25. 25.
    Schoeffmann K, Boeszoermenyi L (2009) Video browsing using interactive navigation summaries. In: International workshop on content-based multimedia indexing, vol 7, pp 243–248Google Scholar
  26. 26.
    Scott-Brown K, Cronin P (2007) An instinct for detection: Psychological perspectives on CCTV surveillance. Police J 80(4):287–305CrossRefGoogle Scholar
  27. 27.
    Song B, Chun K (2003) Motion-compensated noise estimation for efficient pre-filtering in a video encoder. In: IEEE international conference on image processing (ICIP), vol 2, pp 211–214Google Scholar
  28. 28.
    Truong BT, Venkatesh S (2007) Video abstraction: a systematic review and classification. ACM Trans Multimed Comput Comm Appl 3(1):3CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • Benjamin Höferlin
    • 1
  • Markus Höferlin
    • 2
  • Daniel Weiskopf
    • 2
  • Gunther Heidemann
    • 1
  1. 1.Intelligent Systems GroupUniversität StuttgartStuttgartGermany
  2. 2.VISUSUniversität StuttgartStuttgartGermany

Personalised recommendations