Skip to main content
SpringerLink
Log in

An improved surveillance video synopsis framework: a HSATLBO optimization approach

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Video surveillance cameras capture huge amount of data 24 hours a day. However, most of these videos contain redundant data which make the process difficult for browsing and analysis. A significant amount of research findings have been made in summarization of recorded video, but such schemes do not have much impact on video surveillance applications. On the contrary, video synopsis is a smart technology that preserves all the activities of every single object and projects them concurrently in a condensed time. The energy minimization module in video synopsis framework plays a vital role, which in turn minimizes the activity loss, number of collision and temporal consistency cost. In most of the reported schemes, Simulated Annealing (SA) algorithm is employed to solve the energy minimization problem. However, it suffers from slow convergence rate resulting in a high computational load to the system. In order to mitigate this issue, this article presents an improved energy minimization scheme using hybridization of SA and Teaching Learning based Optimization (TLBO) algorithms. The suggested framework for static surveillance video synopsis generation consists of four computational modules, namely, Object detection and segmentation, Tube formation, Optimization, and finally Stitching and the central focus is on the optimization module. Thus, the present work deals with an improved hybrid energy minimization problem to achieve global optimal solution with reduced computational time. The motivation behind hybridization (HSATLBO) is that TLBO algorithm has the ability to search rigorously, ensuring to reach the optimum solution with less computation. On the contrary, SA reaches the global optimum solution, but it may get disarrayed and miss some critical search points. Exhaustive experiments are carried out and results compared with that of benchmark schemes in terms of minimizing the activity, collision and temporal consistency costs. All the experiments are conducted on five widely used videos taken from standard surveillance video data set (PETS 2001, MIT Surveillance Dataset, ChangeDetection.Net, PETS 2006 and UMN Dataset) as well as one real generated surveillance video from the IIIT Bhubaneswar Surveillance Dataset. To make a fair comparison, additionally, performance of the proposed hybrid scheme to solve video synopsis optimization problem is also compared with that of the other benchmark functions. Experimental evaluation and analysis confirm that the proposed scheme outperforms other state-of-the-art approaches. Finally, the suggested scheme can be easily and reliably deployed in the off-line video synopsis generation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Bishop G, Welch G, et al. (2001) An introduction to the Kalman filter. Tech. Rep. 27599-23175 Chapel Hill, NC, USA

  2. Boykov Y, Veksler O, Zabih R (2001) Fast approximate energy minimization via graph cuts. IEEE Trans Pattern Anal Mach Intell 23(11):1222–1239. https://doi.org/10.1109/34.969114

    Article  Google Scholar 

  3. Chou CL, Lin CH, Chiang TH, Chen HT, Lee SY (2015) Coherent event-based surveillance video synopsis using trajectory clustering. In: 2015 IEEE international conference on multimedia & expo workshops (ICMEW). IEEE, pp 1–6

  4. Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6(2):182–197. https://doi.org/10.1109/4235.996017

    Article  Google Scholar 

  5. Ferryman JM (2001) Second IEEE international workshop on performance evaluation of tracking and surveillance (pets 2001). IEEE Computer Society

  6. Ferryman JM (2006) Ninth IEEE international workshop on performance evaluation of tracking and surveillance (pets 2006). IEEE Computer Society

  7. He Y, Gao C, Sang N, Qu Z, Han J (2017) Graph coloring based surveillance video synopsis. Neurocomputing 225:64–79

    Article  Google Scholar 

  8. Hofmann M, Tiefenbacher P, Rigoll G (2012) Background segmentation with feedback: the pixel-based adaptive segmenter. In: 2012 IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW). IEEE, pp 38–43

  9. Huang CR, Chung PCJ, Yang DK, Chen HC, Huang GJ (2014) Maximum a posteriori probability estimation for online surveillance video synopsis. IEEE Trans Circ Syst Vid Technol 24(8):1417–1429. https://doi.org/10.1109/TCSVT.2014.2308603

    Article  Google Scholar 

  10. Jamil M, Yang XS (2013) A literature survey of benchmark functions for global optimization problems. arXiv:13084008

  11. Khadanga RK, Kumar A (2016) Hybrid adaptive ‘gbest’-guided gravitational search and pattern search algorithm for automatic generation control of multi-area power system. IET Gen Transmiss Distrib 11(13):3257–3267

    Article  Google Scholar 

  12. Kirkpatrick S, Gelatt CD, Vecchi MP (1983) Optimization by simulated annealing. Science 220(4598):671–680

    Article  MathSciNet  Google Scholar 

  13. Kolmogorov V, Zabin R (2004) What energy functions can be minimized via graph cuts? IEEE Trans Pattern Anal Mach Intell 26(2):147–159. https://doi.org/10.1109/TPAMI.2004.1262177

    Article  Google Scholar 

  14. Lee DS (2005) Effective gaussian mixture learning for video background subtraction. IEEE Trans Pattern Anal Machne Intell 5:827–832

    Google Scholar 

  15. Li K, Yan B, Wang W, Gharavi H (2016) An effective video synopsis approach with seam carving. IEEE Signal Process Lett 23(1):11–14. https://doi.org/10.1109/LSP.2015.2496558

    Article  Google Scholar 

  16. Li X, Wang Z, Lu X (2016) Surveillance video synopsis via scaling down objects. IEEE Trans Image Process 25(2):740–755. https://doi.org/10.1109/TIP.2015.2507942

    Article  MathSciNet  MATH  Google Scholar 

  17. Liao W, Tu Z, Wang S, Li Y, Zhong R, Zhong H (2017) Compressed-domain video synopsis via 3d graph cut and blank frame deletion. In: Proceedings of the on thematic workshops of ACM multimedia 2017. ACM, pp 253–261

  18. Lin W, Zhang Y, Lu J, Zhou B, Wang J, Zhou Y (2015) Summarizing surveillance videos with local-patch-learning-based abnormality detection, blob sequence optimization, and type-based synopsis. Neurocomputing 155:84–98. https://doi.org/10.1016/j.neucom.2014.12.044

    Article  Google Scholar 

  19. Liu Y, Li X, Wang L, Niu Y (2014) Interpolation-tuned salient region detection. Sci China Inf Sci 57(1):1–9

    Article  Google Scholar 

  20. Lu M, Wang Y, Pan G (2013) Generating fluent tubes in video synopsis. In: 2013 IEEE international conference on acoustics, speech and signal processing, pp 2292–2296. https://doi.org/10.1109/ICASSP.2013.6638063

  21. Metropolis N, Rosenbluth AW, Rosenbluth MN, Teller AH, Teller E (1953) Equation of state calculations by fast computing machines. J Chem Phys 21 (6):1087–1092

    Article  Google Scholar 

  22. Mirjalili S, Mirjalili SM, Lewis A (2014) Grey wolf optimizer. Adv Eng Softw 69:46–61

    Article  Google Scholar 

  23. Mishra SK (2006) Some new test functions for global optimization and performance of repulsive particle swarm method. MPRA Paper

  24. Nie Y, Xiao C, Sun H, Li P (2013) Compact video synopsis via global spatiotemporal optimization. IEEE Trans Vis Comput Graph 19(10):1664–1676. https://doi.org/10.1109/TVCG.2012.176

    Article  Google Scholar 

  25. Pérez P, Gangnet M, Blake A (2003) Poisson image editing. ACM Trans Graph (TOG) 22(3):313–318

    Article  Google Scholar 

  26. Pritch Y, Rav-Acha A, Gutman A, Peleg S (2007) Webcam synopsis: peeking around the world. In: IEEE 11th international conference on computer vision, 2007. ICCV 2007. IEEE, pp 1–8

  27. Pritch Y, Rav-Acha A, Peleg S (2008) Nonchronological video synopsis and indexing. IEEE Trans Pattern Anal Mach Intell 30(11):1971–1984. https://doi.org/10.1109/TPAMI.2008.29

    Article  Google Scholar 

  28. Pritch Y, Ratovitch S, Hendel A, Peleg S (2009) Clustered synopsis of surveillance video. In: 2009 advanced video and signal based surveillance. IEEE, pp 195–200

  29. Rao R (2016) Jaya: a simple and new optimization algorithm for solving constrained and unconstrained optimization problems. Int J Ind Eng Comput 7(1):19–34

    Google Scholar 

  30. Rao R (2016) Review of applications of tlbo algorithm and a tutorial for beginners to solve the unconstrained and constrained optimization problems. Decis Sci Lett 5 (1):1–30

    Google Scholar 

  31. Rao RV, Savsani VJ, Vakharia D (2011) Teaching–learning-based optimization: a novel method for constrained mechanical design optimization problems. Comput Aided Des 43(3):303–315

    Article  Google Scholar 

  32. Rav-Acha A, Pritch Y, Peleg S (2006) Making a long video short: dynamic video synopsis. In: 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR’06), vol 1, pp 435–441. https://doi.org/10.1109/CVPR.2006.179

  33. Simionescu PA (2014) Computer-aided graphing and simulation tools for AutoCAD users. Chapman and Hall/CRC

  34. St-Charles PL, Bilodeau GA (2014) Improving background subtraction using local binary similarity patterns. In: 2014 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 509–515

  35. St-Charles PL, Bilodeau GA, Bergevin R (2014) Flexible background subtraction with self-balanced local sensitivity. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 408–413

  36. Sun L, Xing J, Ai H, Lao S (2012) A tracking based fast online complete video synopsis approach. In: Proceedings of the 21st international conference on pattern recognition (ICPR2012), pp 1956–1959

  37. Tian Y, Zheng H, Chen Q, Wang D, Lin R (2016) Surveillance video synopsis generation method via keeping important relationship among objects. IET Comput Vis 10(8):868–872

    Article  Google Scholar 

  38. UMN (2006) Umn: unusual crowd activity dataset. http://mha.cs.umn.edu/Movies/Crowd-Activity-All.avi

  39. Vural U, Akgul YS (2009) Eye-gaze based real-time surveillance video synopsis. Pattern Recogn Lett 30(12):1151–1159. https://doi.org/10.1016/j.patrec.2009.03.002

    Article  Google Scholar 

  40. Wang S, Yang J, Zhao Y, Cai A, Li SZ, et al. (2011) A surveillance video analysis and storage scheme for scalable synopsis browsing. In: 2011 IEEE international conference on computer vision workshops (ICCV Workshops). IEEE, pp 1947–1954

  41. Wang Sz, Wang Zy, Hu Rm (2013) Surveillance video synopsis in the compressed domain for fast video browsing. J Vis Commun Image Represent 24(8):1431–1442

    Article  Google Scholar 

  42. Wang Y, Jodoin PM, Porikli F, Konrad J, Benezeth Y, Ishwar P (2014) Cdnet 2014: an expanded change detection benchmark dataset. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 387–394

  43. Wang W, Shen J, Li X, Porikli F (2015) Robust video object cosegmentation. IEEE Trans Image Processing 24(10):3137–3148

    Article  MathSciNet  Google Scholar 

  44. Wang WC, Chung PC, Huang CR, Huang WY (2017) Event based surveillance video synopsis using trajectory kinematics descriptors. In: 2017 fifteenth IAPR international conference on machine vision applications (MVA), pp 250–253. https://doi.org/10.23919/MVA.2017.7986848

  45. Xu M, Li SZ, Li B, Yuan XT, Xiang SM (2008) A set theoretical method for video synopsis. In: Proceedings of the 1st ACM international conference on multimedia information retrieval. ACM, pp 366–370

  46. Yao J, Odobez JM (2007) Multi-layer background subtraction based on color and texture. In: 2007 IEEE conference on computer vision and pattern recognition, pp 1–8. https://doi.org/10.1109/CVPR.2007.383497

  47. Yao T, Xiao M, Ma C, Shen C, Li P (2014) Object based video synopsis. In: 2014 IEEE workshop on advanced research and technology in industry applications (WARTIA), pp 1138–1141. https://doi.org/10.1109/WARTIA.2014.6976479

  48. Yildiz A, Ozgur A, Akgul YS (2008) Fast non-linear video synopsis. In: 23rd international symposium on computer and information sciences, 2008. ISCIS’08. IEEE, pp 1–6

  49. Yuan Y, Lu X, Chen X (2015) Multi-spectral pedestrian detection. Signal Process 110:94–100. https://doi.org/10.1016/j.sigpro.2014.08.003. http://www.sciencedirect.com/science/article/pii/S0165168414003661, machine learning and signal processing for human pose recovery and behavior analysis

    Article  Google Scholar 

  50. Zhong R, Hu R, Wang Z, Wang S (2014) Fast synopsis for moving objects using compressed video. IEEE Signal Process Lett 21(7):834–838. https://doi.org/10.1109/LSP.2014.2317754

    Article  Google Scholar 

  51. Zhu X, Liu J, Wang J, Lu H (2012) Key observation selection for effective video synopsis. In: 2012 21st international conference on pattern recognition (ICPR). IEEE, pp 2528–2531

Download references

Author information

Authors and Affiliations

  1. Image and Video Processing Laboratory, Department of Computer Science and Engineering, International Institute of Information Technology, Bhubaneswar, 751003, India

    Subhankar Ghatak & Suvendu Rup

  2. School of Computer and Electrical Engineering, IIITDM Kancheepuram, Chennai, India

    Banshidhar Majhi

  3. Department of Electrical and Computer Engineering, Concordia University, Montreal, Canada

    M. N. S. Swamy

Authors
  1. Subhankar Ghatak
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Suvendu Rup
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Banshidhar Majhi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. M. N. S. Swamy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Subhankar Ghatak.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ghatak, S., Rup, S., Majhi, B. et al. An improved surveillance video synopsis framework: a HSATLBO optimization approach. Multimed Tools Appl 79, 4429–4461 (2020). https://doi.org/10.1007/s11042-019-7389-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-7389-7

Keywords

Search

Navigation