Abstract
The importance of infrared or thermal imaging can be seen in various applications based on Machine Vision. The real-time based application in surveillance systems like detection and tracking of moving objects has shown strong potential. According to available research, there exist some significant issues in colored video frames due to sudden environmental change or other external effects like varying backgrounds. In contrast, the thermal video frames are less affected by sudden light intensity and varying backgrounds. The suggested work utilizes a combined approach of Pearson's correlation and Background Subtraction (BGS) technique over thermal video frame sequences for detection of moving objects. The automatically generated threshold value boosts the efficacy of the algorithm in differentiating between background and foreground frames. The quantitative analysis shows the higher value of different performance parameters like Accuracy, F-measure, Recall, etc. Moreover, the qualitative analysis of obtained results strongly indicates that the proposed method outperforms when compared with the existing methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Akula A, Khanna N, Ghosh R, Kumar S, Das A, Sardana HK (2014) Adaptive contour-based statistical background subtraction method for moving target detection in infrared video sequences. Infrared Phys Technol 63:103–109. https://doi.org/10.1016/j.infrared.2013.12.012
Babaee M, Dinh DT, Rigoll G (2018) A deep convolutional neural network for video sequence background subtraction. Pattern Recogn 76:635–649. https://doi.org/10.1016/j.patcog.2017.09.040
Barnich O, Van Droogenbroeck M (2011) ViBe: a universal background subtraction algorithm for video sequences. IEEE Trans Image Process 20(6):1709–1724. https://doi.org/10.1109/tip.2010.2101613
Bouwmans T (2012) Background subtraction for visual surveillance: a fuzzy approach. In: Handbook on soft computing for video surveillance, pp 103–138. https://doi.org/10.1201/b11631-6.
Bouwmans T, El Baf F, Vachon B (2010) Statistical background modeling for foreground detection: a survey. In: Handbook of pattern recognition and computer vision, pp 181–199. https://doi.org/10.1142/9789814273398_0008
Bouwmans T, Porikli F, Höferlin B, Vacavant A (2014) Background modeling and foreground detection for video surveillance: traditional and recent approaches. In: Implementations, benchmarking and evaluation. CRC Press, Taylor and Francis Group
Bouwmans T, Javed S, Sultana M, Jung SK (2019) Deep neural network concepts for background subtraction: a systematic review and comparative evaluation. Neural Netw. https://doi.org/10.1016/j.neunet.2019.04.024
Chu K-Y, Kuo Y-H, Hsu WH (2013) Real-time privacy-preserving moving object detection in the cloud. In: Proceedings of the 21st ACM international conference on multimedia - MM ’13, pp 597–600, https://doi.org/10.1145/2502081.2502157.
Cui Y, Zeng Z, Fu B, Cui W (2012) Moving object detection based on the histograms of oriented gradients and cloud model. Res J Appl Sci Eng Technol 4(16):2639–2644
Darwich A, Hébert P-A, Bigand A, Mohanna Y (2018) Background subtraction based on a new fuzzy mixture of gaussians for moving object detection. J Imaging 4(7):92. https://doi.org/10.3390/jimaging4070092
Davis JW, Sharma V (2006) Background-subtraction in thermal imagery using contour saliency. Int J Comput vis 71(2):161–181. https://doi.org/10.1007/s11263-006-4121-7
Dong X, Huang X, Zheng Y, Bai S, Xu W (2014) A novel infrared small moving target detection method based on tracking interest points under complicated background. Infrared Phys Technol 65:36–42. https://doi.org/10.1016/j.infrared.2014.03.007
El Baf F, Bouwmans T, Vachon B (2009) Fuzzy statistical modeling of dynamic backgrounds for moving object detection in infrared videos. In: IEEE computer society conference on computer vision and pattern recognition workshops. https://doi.org/10.1109/cvprw.2009.5204109
Elgammal A, Harwood D, Davis L (2004) Non-parametric model for background subtraction. In: Proc. European conference on computer vision, pp 751–767. https://doi.org/10.1007/3-540-45053-X_48.
Giraldo JH, Javed S, Sultana M, Bouwmans T (2020) Graph moving object segmentation. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/ICIVC50857.2020.9177480
Giraldo JH, Javed S, Sultana M, Jung SK, Bouwmans T (2021) The emerging field of graph signal processing for moving object segmentation. In: Jeong H., Sumi K (eds) Frontiers of computer vision. IW-FCV 2021. Communications in computer and information science, vol 1405. Springer, Cham. https://doi.org/10.1007/978-3-030-81638-4_3.
Haines TSF, Xiang T (2014) Background subtraction with dirichlet process mixture models. IEEE Trans Pattern Anal Mach Intell 36(4):670–683. https://doi.org/10.1109/tpami.2013.239
Hao J, Li C, Kim Z, Xiong Z (2013) Spatio-temporal traffic scene modeling for object motion detection. IEEE Trans Intell Transp Syst 14(1):295–302. https://doi.org/10.1109/tits.2012.2212432
Haque M, Murshed M, Paul M (2008) On stable dynamic background, generation technique using gaussian mixture models for robust object detection. In: 2008 IEEE fifth international conference on advanced video and signal based surveillance, pp. 41–48.https://doi.org/10.1109/avss.2008.12
Haritaoglu I, Harwood D, Davis LS (1998) W/sup 4/: a real time system for detecting and tracking people. In: Proceedings in IEEE computer society conference on computer vision and pattern recognition (Cat. No.98CB36231), pp. 962–962. https://doi.org/10.1109/CVPR.1998.698720.
Harville M (2002) A framework for high-level feedback to adaptive, per-pixel, mixture-of-Gaussian background models. In: Proc. European conference on computer vision, pp. 543–560. https://doi.org/10.1007/3-540-47977-5_36.
Huang S-C, Do B-H (2014) Radial basis function based neural network for motion detection in dynamic scenes. IEEE Transactions on Cybernetics 44(1):114–125. https://doi.org/10.1109/tcyb.2013.2248057
Javed S, Narayanamurthy P, Bouwmans T, Vaswani N (2018) Robust PCA and robust subspace tracking: a comparative evaluation. In: IEEE statistical signal processing workshop (SSP). https://doi.org/10.1109/ssp.2018.8450718
Jung CR (2009) Efficient background subtraction and shadow removal for monochromatic video sequences. IEEE Trans Multimedia 11(3):571–577. https://doi.org/10.1109/tmm.2009.2012924
Kim J-Y, Ha J-E (2020) Foreground objects detection using a fully convolutional network with a background model image and multiple original images. IEEE Access 8:159864–159878. https://doi.org/10.1109/access.2020.3020818
Kotu V, Deshpande B (2019) Classification. Data Sci. https://doi.org/10.1016/b978-0-12-814761-0.00004-6
Lee D-S (2005) Effective gaussian mixture learning for video background subtraction. IEEE Trans Pattern Anal Mach Intell 27(5):827–832. https://doi.org/10.1109/tpami.2005.102
Li C, Wang X, Zhang L, Tang J, Wu H, Lin L (2016) WELD: weighted low-rank decomposition for robust grayscale-thermal foreground detection. IEEE Trans Circuits Syst Video Technol. https://doi.org/10.1109/tcsvt.2016.2556586
Lu Y, Liu J, Liu W, Ma S, Xiu X, Liu W, Chen H (2020) Detecting moving objects from dynamic background combining subspace learning with mixed norm approach. Multimedia Tools Appl 79(25–26):18747–18766. https://doi.org/10.1007/s11042-020-08779-9
Makantasis K, Nikitakis A, Doulamis A, Doulamis N, Papaefstathiou Y (2017) Data-driven background subtraction algorithm for in-camera acceleration in thermal imagery. IEEE Trans Circuits Syst Video Technol. https://doi.org/10.1109/tcsvt.2017.2711259
Mandal M, Vipparthi SK (2021) An empirical review of deep learning frameworks for change detection: model design, experimental frameworks, challenges and research needs. IEEE Trans Intell Transp Syst. https://doi.org/10.1109/tits.2021.3077883
Minematsu T, Shimada A, Uchiyama H, Taniguchi R (2018) Analytics of deep neural network-based background subtraction. J Imaging 4(6):78. https://doi.org/10.3390/jimaging4060078
Minematsu T, Shimada A, Taniguchi R (2020) Rethinking background and foreground in deep neural network-based background subtraction. In: IEEE international conference on image processing (ICIP). https://doi.org/10.1109/icip40778.2020.919115
Mittal A, Paragios N (2004) Motion-based background subtraction using adaptive kernel density estimation. In: Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition. CVPR 2004, pp 302–309. https://doi.org/10.1109/cvpr.2004.1315179.
Monnet A, Mittal A, Paragios N, Ramesh V (2003) Background modeling and subtraction of dynamic scenes. In: Proceedings ninth IEEE international conference on computer vision, vol 2(13–16), pp 1305–1312. https://doi.org/10.1109/iccv.2003.1238641.
Narayanamurthy P, Vaswani N (2018) A fast and memory-efficient algorithm for robust PCA (MEROP). In: IEEE international conference on acoustics, speech and signal processing (ICASSP). https://doi.org/10.1109/icassp.2018.8461540
Nebili W, Farou B, Seridi H (2020) Background subtraction using artificial immune recognition system and single gaussian (AIRS-SG). Multimedia Tools Appl 79(35–36):26099–26121. https://doi.org/10.1007/s11042-020-089351
Qin H, Han J, Yan X, Zeng Q, Zhou H, Li J, Chen Z (2016) Infrared small moving target detection using sparse representation-based image decomposition. Infrared Phys Technol 76:148–156
Qiu S, Luo J, Yang S, Zhang M, Zhang W (2019) A moving target extraction algorithm based on the fusion of infrared and visible images. Infrared Phys Technol 98:285–291. https://doi.org/10.1016/j.infrared.2019.03.022
Rai M, Yadav RK (2016) A novel method for detection and extraction of human face for video surveillance applications. Int J Signal Imaging SystEng 9(3):165. https://doi.org/10.1504/ijsise.2016.076226
Rai M, Maity T, Yadav RK (2017) Thermal imaging system, and its real time applications: a survey. J Eng Technol 6(2):290–303
Rodriguez P, Wohlberg B (2016) Incremental principal component pursuit for video background modeling. J Math Imaging vis 55(1):1–18. https://doi.org/10.1007/s10851-015-0610-z
Sharma L, Lohan N (2019) Performance analysis of moving object detection using BGS techniques in visual surveillance. Int J Spatio-Temporal Data Sci 1(1):22–53. https://doi.org/10.1504/ijstds.2019.097607
Sharma L, Yadav DK, Singh A (2016) Fisher’s linear discriminant ratio based threshold for moving human detection in thermal video. Infrared Phys Technol 78:118–128. https://doi.org/10.1016/j.infrared.2016.07.012
Shi D, Yin K, Huang J, Yuan K, Zhu W, Xie C, Wang Y (2019) Fast tracking of moving objects using single-pixel imaging. Opt Commun 440:155–162. https://doi.org/10.1016/j.optcom.2019.02.006
ShiQiu, Du Y, Yang S (2018) The infrared moving target extraction and fast video reconstruction algorithm. Infrared Phys Technol. https://doi.org/10.1016/j.infrared.2018.11.02
Stauffer C, Grimson WEL (1999) Adaptive background mixture models for real-time tracking. In: Proceedings. IEEE computer society conference on computer vision and pattern recognition (Cat. No PR00149), vol 2, pp 246–252. https://doi.org/10.1109/cvpr.1999.784637.
Stauffer C, Grimson WEL (2000) Learning patterns of activity using real-time tracking. IEEE Trans Pattern Anal Mach Intell 22(8):747–757. https://doi.org/10.1109/34.868677
St-Charles P-L, Bilodeau G-A, Bergevin R (2015) SuBSENSE: a universal change detection method with local adaptive sensitivity. IEEE Trans Image Process 24(1):359–373. https://doi.org/10.1109/tip.2014.2378053
Supreeth HSG, Patil CM (2018) Efficient multiple moving object detection and tracking using combined background subtraction and clustering. SIViP 12(6):1097–1105. https://doi.org/10.1007/s11760-018-1259-z
Tan Y, Guo Y, Gao C (2013) Background subtraction based level sets for human segmentation in thermal infrared surveillance systems. Infrared Phys Technol 61:230–240. https://doi.org/10.1016/j.infrared.2013.08.014
Tattersall GJ, Danner RM, Chaves JA, Levesque DL (2020) Activity analysis of thermal imaging videos using a difference imaging approach. J Therm Biol 91:102611. https://doi.org/10.1016/j.jtherbio.2020.102611
Teju V, Bhavana D (2020) An efficient object detection using OFSA for thermal imaging. Int J Electr Eng Educ. https://doi.org/10.1177/0020720920944434
Wren C, Azarbayejani A, Darrell T, Pentland A (1997) Pfinder: real-time tracking of the human body. IEEE Trans Pattern Anal Mach Intell 19(7):780–785. https://doi.org/10.1109/34.598236
Xu Y, Dong J, Zhang B, Xu D (2016) Background modeling methods in video analysis: a review and comparative evaluation. CAAI Trans Intell Technol 1(1):43–60. https://doi.org/10.1016/j.trit.2016.03.005
Yadav DK, Singh K (2015) Moving object detection for visual surveillance using quasi-Euclidian distance. Adv Intell Syst Comput. https://doi.org/10.1007/978-81-322-2526-3_25
Yadav DK, Singh K (2016) A combined approach of Kullback-Leibler divergence and background subtraction for moving object detection in thermal video. Infrared Phys Technol 76:21–31. https://doi.org/10.1016/j.infrared.2015.12.027
Yadav DK, Sharma L, Bharti SK (2014) Moving object detection in real-time visual surveillance using background subtraction technique. In: 14th international conference on hybrid intelligent systems, IEEE, Kuwait, pp 79–84.
Yang S, Luo B, Li C, Wang G, Tang J (2017) Fast grayscale-thermal foreground detection with collaborative low-rank decomposition. IEEE Trans Circuits Syst Video Technol. https://doi.org/10.1109/tcsvt.2017.2721460
Zhao C, Basu A (2020) Dynamic deep pixel distribution learning for background subtraction. IEEE Trans Circuits Syst Video Technol 30(11):4192–4206. https://doi.org/10.1109/tcsvt.2019.2951778
Zhong J (2003) Segmenting foreground objects from a dynamic textured background via a robust kalman filter. In: Proceedings ninth IEEE international conference on computer vision, vol 1(13–16), pp 44–50. https://doi.org/10.1109/iccv.2003.1238312.
Zhou X, Yang C, Weichuan Yu (2013a) Moving object detection by detecting contiguous outliers in the low rank representation. IEEE Trans Pattern Anal Mach Intell 35(3):597–610. https://doi.org/10.1109/tpami.2012.132
Zhou H, Chen Y, Feng R (2013b) A novel background subtraction method based on color invariants. Comput vis Image Underst 117(11):1589–1597. https://doi.org/10.1016/j.cviu.2013.07.008
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Rai, M., Maity, T., Husain, A.A. et al. Pearson's correlation and background subtraction (BGS) based approach for object's motion detection in infrared video frame sequences. Stat Papers 64, 449–475 (2023). https://doi.org/10.1007/s00362-022-01323-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00362-022-01323-x