Skip to main content
SpringerLink
Log in

Moving human detection and tracking from thermal video through intelligent surveillance system for smart applications

  • 1216: Intelligent and Sustainable Techniques for Multimedia Big Data Management for Smart Cities Services
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In real-time based smart video surveillance system, the moving human detection in thermal video is a critical task that filters out redundant information and extracts exigent information. The thermal imaging-based system is used to extract the motion-based object in an unseen or dark environment because it captures heat generated from the human or manmade objects. It also penetrates challenging problems due to cluttered nature, low light, illumination variation, dust, mist, or haze available in the background. So, there is huge demand for identification and monitoring of unwanted activities, minimization of crime or trespassing, etc for safety and security. The state-of-the-art methods worked for various problems raised due to cluttered or illumination variation type of behavior of the background. This paper provides a performance analysis of state-of-the-art literature and also focus on the challenging issues involved. Here, the proposed work developed an adaptive method for the maintenance of the background model and adaptive threshold generation during testing phase. This threshold is applied to classify the moving and non-moving pixels by avoiding the external involvement for threshold selection at run-time. To evaluate the efficacy, the performance of the proposed work is analyzed through numerous parameters that achieved higher accuracy with minimum false alarm rate and impressive detection results. The qualitative and quantitative experimental results demonstrate better real-time performance and usability against considered peer methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Ahmad J, Akula A, Mulaveesala R, Sardana HK (2019) An independent component analysis based approach for frequency modulated thermal wave imaging for subsurface defect detection in steel sample. Inf Phys Technol, Elsevier 98:45–54

    Article  Google Scholar 

  2. Akula A, Khanna N, Ghosh R, Kumar S, Das A, Sardana HK (2013) Adaptive contour based statistical background subtraction method for moving target detection in infrared video sequences. J Infrared Phys Technol Elsevier 63:103–109

    Article  Google Scholar 

  3. Applications of Thermal Imaging: https://www.techimaging.com/applications/infrared-thermal-imaging-applications

  4. Bandarupalli, S 2009 “Vehicle detection and tracking using wireless sensors and video cameras”, University of New Orleans Theses and Dissertations, (https://scholarworks.uno.edu/td/989).

  5. Bouwmans T, Porikli F, Höferlin B, Vacavant A (2014) Background modeling and foreground detection for video surveillance. Chapman & Hall, London, U.K.

    Book  MATH  Google Scholar 

  6. Bouwmans T, Sobral A, Javed S, Jung S, Zahzah E (2017) Decomposition into low-rank plus additive matrices for background/foreground separation: a review for a comparative evaluation with a large-scale dataset. Comp Sci Rev 23:1–71

    Article  MATH  Google Scholar 

  7. Chen P, Dang Y, Liang R, Zhu W, He X (2018) Real-time object tracking on a drone with multi-inertial sensing data. IEEE Trans Intell Transp Syst 19(1):131–139

    Article  Google Scholar 

  8. Demir B, Ergunay S, Nurlu G et al (2020) Real-time high-resolution omnidirectional imaging platform for drone detection and tracking. J Real-Time Image Proc Springer 17:1625–1635

    Article  Google Scholar 

  9. Dollár P, Appel R, Blondie S, Perona P (2014) Fast feature pyramids for object detection. IEEE Trans Pattern Anal Mach Intell 36(8):1532–1545

    Article  Google Scholar 

  10. Goyette N, Jodoin PM, Porikli F, Ishwar P (2012) changedetection.net. A new change detection benchmark database. Proc IEEE Workshop on Change Detect at CVPR:1–8

  11. Gupta H, Verma OP(2021) “Monitoring and surveillance of urban road traffic using low altitude drone images: a deep learning approach”, J Multimed Tools App, Springer

  12. Haines T, Xiang T (Apr. 2014) Background subtraction with Dirichlet process mixture models. IEEE Trans Pattern Anal Mach Intell 36(4):670–683

    Article  Google Scholar 

  13. Haque M, Murshed M, Paul M 2008 On Stable Dynamic Background Generation Technique using Gaussian Mixture Models for Robust Object Detection. 5th International Conference on Advanced Video and Signal Based Surveillance, IEEE, pp. 41–48. https://doi.org/10.1109/AVSS.2008.12.

  14. Hashemi M, Hall M (2019) Detecting and classifying online dark visual propaganda. J Image Vision Comp Elsevier 89:95–105

    Article  Google Scholar 

  15. Hu W, Tan T, Wang L, Maybank S (2004) A survey on visual surveillance of object motion and behaviours. IEEE transac on syst. Man and Cybernetics – Part C 34:334–352. https://doi.org/10.1109/TSMCC.2004.829274

    Article  Google Scholar 

  16. Infrared detector in thermal imaging: https://www.atncorp.com/howthermalimagingworks

  17. Jung CR 2009 “Efficient Background Subtraction and Shadow Removal for Monochromatic Video Sequences”, IEEE Transactions on Multimedia, vol. 11, no. 3.

  18. Lee S, Lee C Low complexity background subtraction based on spatial similarity. Eurasip J Image and Video Proc, Springer 35:2–16

  19. Mandal M, Dhar V, Mishra A, Vipparthi SK, Abdel-Mottaleb M (2021) 3DCD: scene independent end-to-end spatiotemporal feature learning framework for change detection in unseen videos. IEEE Trans Image Process 30:546–558

    Article  Google Scholar 

  20. Rai M, Husain AA, Maity T, Yadav RK (2018) Advance intelligent video surveillance system (AIVSS): a future aspect, intelligent video surveillance. Publisher IntechOpen, 5th. https://doi.org/10.5772/intechopen.76444

    Book  Google Scholar 

  21. Reddy V, Sanderson C, Lovell B (2013) Improved foreground detection via block based classifier cascade with probabilistic decision integration. IEEE Transac Circuit Syst Video Technol 23(1):175–181

    Google Scholar 

  22. Saboo S, Singha J (May 2021) Vision based two-level hand tracking system for dynamic hand gestures in indoor environment. J Multimedia Tools App Springer 80:20579–20598

    Article  Google Scholar 

  23. Sanin A, Sanderson C, Lovell BC (2012) Shadow detection: a survey and comparative evaluation of recent methods. Elsevier, Pattern Recog 45(4):1684–1695

    Article  Google Scholar 

  24. Security for Infiltration: https://www.git-security.com/topstories/security/thermal-imaging-technology-ready-further-conquer-world

  25. Shahbaz A, Jo K (2020) Improved Change Detector using Dual-Camera Sensors for Intelligent Surveillance Systems. IEEE Sensors J:1–8

  26. Sharma L, Yadav DK (2017) Histogram based Adaptive Learning Rate for Background Modelling and Moving Object Detection in Video Surveillance. Intern J Telemed Clinical Prac, Indersci 2(1):74–92

    Google Scholar 

  27. Sharma L, Yadav DK, Singh A (2016) Fisher’s Linear Discriminant Ratio based Threshold for Moving Human Detection in Thermal Video. Infrared Phys Technol, Elsevier 78:118–128

    Article  Google Scholar 

  28. Song J, Gao B, Woob WL, Tian GY (2020) Ensemble tensor decomposition for infrared thermography 568 cracks detection system. Infr Phys Technol Elsevier 105:1–9 https://www.sciencedirect.com/science/article/abs/pii/S1350449519308618?via%3Dihub https://doi.org/10.1016/j.infrared.2020.103203

  29. Stauffer C, Grimson W (1999) Adaptive background mixture models for real-time tracking. Int Conf On Comp Vision Pattern Recog IEEE Comp Soc 2:252–256. https://doi.org/10.1109/CVPR.1999.784637

    Article  Google Scholar 

  30. St-Charles PL, Bilodeau GA, Bergevin R (2015) SuBSENSE: a universal change detection method with local adaptive sensitivity. IEEE Trans Image Process 24(1):359–373. https://doi.org/10.1109/TIP.2014.2378053

    Article  MathSciNet  MATH  Google Scholar 

  31. Tezcan MO, Ishwar P, Konrad J (2021) BSUV-net 2.0: Spatio-temporal data augmentations for video-agnostic supervised background subtraction. IEEE Access 9:53849–53860. https://doi.org/10.1109/ACCESS.2021.3071163

    Article  Google Scholar 

  32. Tokmakov P, Alahari K, Schmid C (2017) “Learning Motion Patterns in Videos”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 531–539.

  33. Wang Y, Jodoin PM, Porikli F, Konrad J, Benezeth Y, Ishwar P (2014) CDNET 2014: an expanded change detection benchmark dataset. IEEE Conf Compr Vision and Pattern Recog Workshops:393–400

  34. Xua X, Yanga P, Xiana H, Liu Y (2019) Robust moving objects detection in long-distance imaging through turbulent medium. Inf Phys Technol. Elsevier 100:87–98. https://doi.org/10.1016/j.infrared.2019.02.014

    Article  Google Scholar 

  35. Yadav DK (2019) Chapter-12: detection of moving human in vision based smart surveillance under cluttered background: an application for IoT. Book- Visual Surveil Int Things: Technol App, Taylor & Francis, March:1–296

  36. Yadav DK, Singh K (2016) A Combined Approach of Kullback-Leibler Divergence Method and Background Subtraction for Moving Object Detection in Thermal Video. Infra Phys Technol, Elsevier 76:21–31

    Article  Google Scholar 

  37. Yadav DK, Singh K (Jan, 2019) Adaptive background modeling technique for moving object detection in video under dynamic environment. Intern J Spatio-Temp Data Sci, Indersci 1(1):4–21

    Google Scholar 

  38. Yazdi M, Bouwmans T (March, 2018) New trends on moving object detection in video images captured by a moving camera: a survey. Comp Sci Rev, Elsevier 28:1–66

    MathSciNet  Google Scholar 

  39. Zeng Q, Adu J, Liu J, Yang J, Xu Y, Gong M (2020) Real-time adaptive visible and infrared image registration based on morphological gradient and C_SIFT. J Real-Time Image Proc Springer 17:1103–1115

    Article  Google Scholar 

  40. Zhou X, Yang C, Yu W (2014) Moving object detection by detecting contiguous outliers in the low-rank representation”, IEEE transactions on pattern analysis and machine intelligence, vol. 35, issue-3, pp. 597-610, march, 2013.

Download references

Acknowledgements

The authors are very grateful to the CDNET dataset [10, 33] community for providing frames along with ground truth of thermal frame sequences.

Author information

Authors and Affiliations

  1. Manav Rachna University, Faridabad, Haryana, India

    Manoj Kumar & Susmita Ray

  2. Galgotias University, Greater Noida, UP, India

    Dileep Kumar Yadav

Authors
  1. Manoj Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Susmita Ray
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dileep Kumar Yadav
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manoj Kumar.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kumar, M., Ray, S. & Yadav, D.K. Moving human detection and tracking from thermal video through intelligent surveillance system for smart applications. Multimed Tools Appl 82, 39551–39570 (2023). https://doi.org/10.1007/s11042-022-13515-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13515-6

Keywords

Search

Navigation