Skip to main content
SpringerLink
Log in

Smart video summarization using mealy machine-based trajectory modelling for surveillance applications

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this paper, we propose a smart video summarization technique that compiles a synopsis of event(s)-of-interest occurring within a segment of image frames in a video. The proposed solution space consists of extracting appropriate features that represent the dynamics of targets in surveillance environments using their motion trajectories combined with a finite state automaton model for analyzing state changes of such features to detect and localize event(s)-of-interest. We introduce the cumulative moving average (CMA) and the preceding segment average (PSA) statistical metric as features that indicate gradual and sudden changes in the instantaneous velocity of moving targets. In order to support both on-line and off-line summarization, a finite state machine, that is often referred to as Mealy Machine, has been proposed to model the trajectory of a moving target and used for detecting transitions that represents a change from one state to another when initiated by a triggering event or condition. We conduct several systematic experiments on different scenario-specific in-house videos and other publicly available datasets to demonstrate the effectiveness of our proposed approach and benchmark its performance against chosen baseline strategies. The results of our experiments highlight the superiority of our proposed method in accurately localizing the start and end of event(s)-of-interest in videos within the chosen dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

Notes

  1. “Big Brother is DEFINITELY watching you: Shocking study reveals UK has one CCTV for every 32 people.”, Read more: http://www.dailymail.co.uk/news/article-1362493/One-CCTV-camera-32-people-Big-Brother-Britain.html

  2. http://homepages.inf.ed.ac.uk/rbf/CAVIAR/

  3. http://www.openvisor.org

  4. http://www.ee.cuhk.edu.hk/%7Ejshao/CUHKcrowd_files/cuhk_crowd_dataset.htm

  5. http://www.mathworks.com/help/vision/examples/scene-change-detection.html

References

  1. Ajmal M, Ashraf M, Shakir M, Abbas Y, Shah F (2012) Video summarization: techniques and classification. In: Proceedings of international conference on computer vision and graphics (ICCVG), vol 7594, pp 1–13

  2. Besiris D, Makedonas A, Economou G, Fotopoulos S (2009) Combining graph connectivity & dominant set clustering for video summarization. Springer J Multimed Tools Appl 44(2):161–186

    Article  Google Scholar 

  3. Bucak SS, Gunsel B (2011) Online Video Scene Clustering by Competitive Incremental NMF. Springer J Signal Image Video Process:1–17

  4. Chen F, De Vleeschouwer C, Cavallaro A (2014) Resource allocation for personalized video summarization. IEEE Trans Multimed 16(2):455–469

    Article  Google Scholar 

  5. Damnjanovic U, Arguedas V, Izquierdo E, Martnez J (2008) Event detection and clustering for surveillance video summarization. In: Proceedings of international workshop on image analysis for multimedia interactive services (WIAMIS), pp 63–66

  6. Dinh T, Vo N, Medioni G (2011) Context tracker: exploring supporters and distracters in unconstrained environments. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition (CVPR), pp 1177–1184

  7. Doulamis N, Doulamis A, Ntalianis K (2002) An optimal interpolation-based scheme for video summarization. In: Proceedings of IEEE international conference on multimedia and expo (ICME), pp 297–300

  8. Fujimura K, Honda K, Uehara K (2002) Automatic video summarization by using color and utterance information. In: Proceedings of IEEE international conference on multimedia and expo (ICME), pp 49–52

  9. Furini M, Ghini V (2006) An audio-video summarization scheme based on audio and video analysis. In: Proceedings of IEEE consumer communications & networking, pp 1209–1213

  10. Habibian A, Mensink T, Snoek CGM (2014) VideoStory: a new multimedia embedding for few-example recognition and translation of events. In: Proceedings of the ACM international conference on multimedia (MM), pp 17–26

  11. Irani M, Anandan P (1998) Video indexing based on mosaic representations. In: Proceedings of the IEEE, pp 905–921

  12. Jiang R, Sadka A, Crookes D (2009) Advances in video summarization and skimming. Springer Series Studies Comput Intell Recent Adv Multimed Signal Process Commun 231:27–50

    Google Scholar 

  13. Kang H, Chen X, Matsushita Y, Tang X (2006) Space-time video montage. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition (CVPR), pp 1331–1338

  14. Khosla A, Hamid R, Lin C, Sundaresan N (2013) Large-scale video summarization using web-image priors. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition (CVPR), pp 2698–2705

  15. Lee Y, Ghosh J, Grauman K (2012) Discovering important people and objects for egocentric video summarization. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition (CVPR), pp 1346–1353

  16. Liu D, Wang M, Hua X-S, Zhang H-J (2011) Semi-automatic tagging of photo albums via exemplar selection and tag inference. IEEE Trans Multimed 13(1):82-91

    Article  Google Scholar 

  17. Lu Z, Grauman K (2013) Story-driven summarization for egocentric video. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition (CVPR), pp 2714–2721

  18. Ma X, Chen X, Khokhar A, Schonfeld D (2010) Motion trajectory-based video retrieval, classification, and summarization. Springer Series Studies Comput Intell Video Search Mining 287:53–82

    Google Scholar 

  19. Meghdadi A, Irani P (2013) Interactive exploration of surveillance video through action shot summarization and trajectory visualization. IEEE Trans Vis Comput Graph 19(12):2119-2128

    Article  Google Scholar 

  20. Nam J, Tewfik A (1999) Dynamic video summarization and visualization. In: Proceedings of the seventh ACM international conference on multimedia (Part 2), pp 53–56

  21. Panagiotakis C, Ovsepian N, Michael E (2013) Video synopsis based on a sequential distortion minimization method. In: Wilson R, Hancock E, Bors A, Smith W (eds) Computer analysis of images and patterns. Llecture notes in computer science, vol 8047. pp 94–101

  22. Peng W-T, Chu W-T, Chang C-H, Chou C-N, Huang W-J, Chang W-Y, Hung Y-P (2011) Editing by viewing: automatic home video summarization by viewing behavior analysis. IEEE Trans Multimed 13(3):539–550

    Article  Google Scholar 

  23. Rodriguez M (2010) Cram: compact representation of actions in movies. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition (CVPR), pp 3328–3335

  24. Shafeian H, Bhanu B (2012) Integrated personalized video summarization and retrieval. In: Proceedings of international conference on pattern recognition (ICPR), pp 996–999

  25. Vezzani R, Cucchiara R (2010) Video surveillance online repository (visor): an integrated framework. Springer J of Multimed Tools and Appl 50(2):359-380

    Article  Google Scholar 

  26. Wang F, Ngo C (2007) Rushes video summarization by object and event understanding. In: Proceedings of the international workshop on TRECVID video summarization, pp 25–29

  27. Wang M, Hong R, Li G, Zha Z, Yan S, Chua T (2012) Event driven web video summarization by tag localization and key-shot identification. IEEE Trans Multimed 14(4):975–985

    Article  Google Scholar 

  28. Wang M, Li G, Lu Z, Gao Y, Chua T-S (2013) When Amazon meets Google: product visualization by exploring multiple web sources. ACM Trans Internet Technol 12(4)

  29. Wang X, Tieu K, Grimson E (2006) Learning semantic scene models by trajectory analysis. In: Proceedings of the European conference on computer vision (ECCV), pp 110–123

  30. Xiang X, Kankanhalli MS (2011) Affect-based adaptive presentation of home videos. In: Proceedings of the 19th ACM international conference on multimedia (MM), pp 553–562

  31. Zhao S, Yao H, Sun X, Jiang X, Xu P (2013) Flexible presentation of videos based on affective content analysis. In: Advances in multimedia modeling, lecture notes in computer science, vol 7732, pp 368–379

  32. Zhao S, Yao H, Sun X, Xu P, Liu X, Ji R (2011) Video indexing and recommendation based on affective analysis of viewers. In: Proceedings of the 19th ACM international conference on Multimedia (MM), pp 1473–1476

  33. Zhao S, Yao H, Sun X (2013) Video classification and recommendation based on affective analysis of viewers. Elsevier J Neurocomputing 119:101–110

    Article  Google Scholar 

  34. Zhang Y, Huang Q, Qin L, Zhao S, Yao H, Xu P (2014) Representing dense crowd patterns using bag of trajectory graphs. Springer J Signal Image Video Process 8(1):173–181

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electrical Sciences, IIT Bhubaneswar, Bhubaneswar, India

    Debi Prosad Dogra

  2. Department of Computer Science and Engineering, Haldia Institute of Technology, Haldia, India

    Arif Ahmed

  3. Department of Electrical and Computer Engineering, Khalifa University of Science Technology and Research, Abu Dhabi, 127788, United Arab Emirates

    Harish Bhaskar

  4. Bristol Vision Institute, University of Bristol, Bristol, UK

    Harish Bhaskar

Authors
  1. Debi Prosad Dogra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arif Ahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Harish Bhaskar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Debi Prosad Dogra.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dogra, D., Ahmed, A. & Bhaskar, H. Smart video summarization using mealy machine-based trajectory modelling for surveillance applications. Multimed Tools Appl 75, 6373–6401 (2016). https://doi.org/10.1007/s11042-015-2576-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-2576-7

Keywords

Search

Navigation