Skip to main content
SpringerLink
Log in

A real-time surveillance system with multi-object tracking

  • Published:
Multidimensional Systems and Signal Processing Aims and scope Submit manuscript

Abstract

Multi-object detection and occlusion tracking in the computer vision field is an important research topic on the modern surveillance system. These studies largely rely on computer vision work. However, most algorithms are too complex and not practical to be used as real-time systems. This paper proposes a real-time surveillance system. The proposed method mainly improves the foreground detection to get low-complexity and high-quality effects of tracking. A novel occlusion-adaptive tracking method is also applied. It can immediately track multi-objects in successive positions without color cues and an appearance model. To track moving objects, the proposed method uses labeling information to eliminate noises and group moving objects. Additionally, we are also concerned about several cases of occlusions to increase the tracking efficiency. The comparison has been performed with other algorithms of background subtraction. Experimental results show that the proposed method has better performance than other foreground detection methods in terms of both computation speed and detection rate.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5:
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  • Athanesious, J. J. and Suresh, P. “Systematic survey on object tracking methods in video,” Int. J. Adv. Res. Comput. Eng. Technol., vol. 1, no. 8, pp. 242, 2012.

  • Barnich, O., & VanDroogenbroeck, M. (2011). ViBe: A universal background subtraction algorithm for video sequences. IEEE Transactions on Image Processing, 20, 1709–1724.

    Article  MathSciNet  MATH  Google Scholar 

  • Bernardin, K. & Stiefelhagen, R.: ‘Evaluating multiple object tracking performance: The CLEAR MOT

  • Bloisi, D. and Iocchi, L. “Independent multimodal background subtraction.,” in CompIMAGE, 2012, pp. 39–44.

  • C. R.Wren, A.Azarbayejani, T.Darrell, andA. P.Pentland, “P finder: Real-time tracking of the human body,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 780–785, 1997.

  • Cao, W., Wang, Y., Sun, J., Meng, D., Yang, C., Cichocki, A., & Xu, Z. (2016). Total variation regularized tensor RPCA for background subtraction from compressive measurements. IEEE Transactions on Image Processing, 25(9), 4075–4090.

    Article  MathSciNet  MATH  Google Scholar 

  • Chen, L., Shen, J., Wang, W., & Ni, B. (2015). Video object segmentation via dense trajectories. IEEE Transactions on Multimedia, 17(12), 2225–2234.

    Article  Google Scholar 

  • Chen, L., Wang, W., Panin, G., & Knoll, A. (2015). Hierarchical grid-based multi-people tracking-by-detection with global optimization. IEEE Transactions on Image Processing, 24(11), 4197–4212.

    Article  MathSciNet  MATH  Google Scholar 

  • Cheng, D., Gong, Y., Zhou, S., Wang, J. and Zheng, N. PersonRe-identification by multi-channel parts-based CNN with improved triplet loss function. In CVPR, 2016.

  • Comaniciu, D., Ramesh, V., & Meer, P. (2003). Kernel-based object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25, 564–577.

    Article  Google Scholar 

  • Elgammal, A., Harwood, D., and Davis, L. “Non-parametric model for background subtraction,” in European conference on computer vision, 2000, pp. 751–767.

  • Elhamod, M., & Levine, M. D. (2018). Automated real-time detection of potentially suspicious behavior in public transport areas. IEEE Transactions on Intelligent Transportation Systems, 14(2), 688–699.

    Article  Google Scholar 

  • Ferryman, J., & Shahrokni, A. (2009). “Pets2009: Dataset and challenge”, in Performance Evaluation of Tracking and Surveillance (PETS-Winter). Twelfth IEEE International Workshop on, 2009, 1–6.

    Google Scholar 

  • Funk, N. “A study of the Kalman filter applied to visual tracking,” Univ. Alberta, Proj. C., vol. 652, no. 6, 2003.

  • Heikkila, M., & Pietikainen, M. (2006). A texture-based method for modeling the background and detecting moving objects. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28, 657–662.

    Article  Google Scholar 

  • Hofmann, M., Tiefenbacher, P., and Rigoll, G. “Background segmentation with feedback: The pixel-based adaptive segmenter,” in Computer Vision and Pattern Recognition Workshops (CVPRW), 2012 IEEE Computer Society Conference on, 2012, pp. 38–43.

  • Hou, L., Wan, W., Lee, K.-H., Hwang, J.-N., Okopal, G., Pitton, J. “Deformable multiple-kernel based human tracking using a moving camera,” in Acoustics, Speech and Signal Processing (ICASSP), 2015 IEEE International Conference on, 2015, pp. 2249–2253.

  • J. Fan, W. Xu, Wu, Y. and Gong, Y. “Human tracking using convolutional neural networks,” In IEEE Transactions on neural networks, vol. 21, no. 10, 2010.

  • Jiang, H., Deng, W., & Shen, Z. (2012). Surveillance video processing using compressive sensing. Inverse Problem Imag., 6(2), 201–214.

    Article  MathSciNet  MATH  Google Scholar 

  • Jung, C. R. (2009). Efficient background subtraction and shadow removal for monochromatic video sequences. IEEE Transactions on Multimedia, 11(3), 571–577.

    Article  Google Scholar 

  • Kim, J.-Y., & Ha, J.-E. (1807). Foreground objects detection by U-net with multiple difference images. Applied Sciences, 11(4), 2021.

    Google Scholar 

  • Kim, I., Khan, M. M., Awan, T. W., & Soh, Y. (2014). Multi-target tracking using color information. Int. J. Comput. Commun. Eng., 3, 11.

    Article  Google Scholar 

  • Kothiya, S.V., Mistree, K. B. “A review on real time object tracking in video sequences,” in Electrical, Electronics, Signals, Communication and Optimization (EESCO), 2015 International Conference on, 2015, pp. 1–4.

  • Leal-Taixé, L., Milan, A., Reid, I., Roth, S. & Schindler, K. MOTChallenge 2015: Towards a benchmark for multi-target tracking. arXiv:1504.01942 [cs], 2015.

  • Lee, H., Kim, H. S., & Kim, J. I. (2016). Background subtraction using background sets with image- and color-space reduction. IEEE Transactions on Multimedia, 18(10), 2093–2103.

    Article  MathSciNet  Google Scholar 

  • Li, X., Wang, K., Wang, W. and Li, Y. “A multiple object tracking method using Kalman filter,” in Information and Automation (ICIA), 2010 IEEE International Conference on, 2010, pp. 1862–1866.

  • Li, B., Yang, C. and Xu, G. "Multi-pedestrian tracking based on feature learning method with lateral inhibition," In: IEEE International Conference on Information and Automation, Lijiang, 2015, pp. 524–529

  • Lin, T.Y., Yeh, J.S., Wu, F.C., Chuang, Y.Y. and Dellinger, A. “An experiment for background subtraction in a dynamic scene,” Advances in Artificial Intelligence. Canadian AI 2019. Lecture Notes in Computer Science, vol. 11489. Springer, Cham, 2019.

  • Long, J., Shelhamer E., Darrell, T. “Fully convolutional networks for semantic segmentation,” In CVPR, pp. 3431–3440, 2015.

  • Lu, X. and Xu, C. "Novel Gaussian mixture model background subtraction method for detecting moving objects," In: 2018 IEEE International Conference of Safety Produce Informatization (IICSPI), 2018.

  • Maddalena, L., & Petrosino, A. (2018). Background subtraction for moving object detection in RGBD data: A survey. Journal of Imaging, 4(5), 71.

    Article  Google Scholar 

  • Magadán, A., Martin, I., Conde, C., & Cabello, E. (2016). Evaluation of keypoint descriptors applied in the pedestrian detection in low quality images. IEEE Transactions on Intelligent Transportation Systems, 14, 1401–1407.

    Google Scholar 

  • Manzanera, A., & Richefeu, J. C. (2007). A new motion detection algorithm based on Σ-Δ background estimation. Pattern Recognition Letter, 28(3), 320–328.

    Article  Google Scholar 

  • Moeslund, T. B., Hilton, A., & Krüger, V. (2006). A survey of advances in vision-based human motion capture and analysis. Computer Vision and Image Understanding, 104(2–3), 90–126.

    Article  Google Scholar 

  • Rao, A. S., Gubbi, J., Marusic, S., & Palaniswami, M. (2016). Crowd event detection on optical flow manifolds. IEEE Transactions on Cybernetics, 46(7), 1524–1537.

    Article  Google Scholar 

  • Rawitas Krungkaew, Worapan Kusakunniran "Foreground segmentation in a video by using a novel dynamic codebook", 13th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON), 2016

  • Rekha, V., Natarajan, K., & Innila, R. J. (2020). Foreground algorithms for detection and extraction of an object in multimedia. International Journal of Electrical and Computer Engineering (IJECE), 10(2), 1849–1858.

    Article  Google Scholar 

  • Savaş, M., Demirel, H., & Erkal, B. (2018). Moving object detection using an adaptive background subtraction method based on block-based structure in dynamic scene. Optik, 168, 605–618.

    Article  Google Scholar 

  • Sehairi, K., Chouireb, F., & Meunier, J. (2017). Comparative study of motion detection methods for video surveillance systems. Journal of Electronic Imaging, 26, 023025–023025.

    Article  Google Scholar 

  • Stauffer, C. and Grimson, W. E. L. “Adaptive background mixture models for real-time tracking,” in Computer Vision and Pattern Recognition, 1999. IEEE Computer Society Conference on., 1999, vol. 2, pp. 246–252.

  • Suwannatat, T., Chinnasarn, K. and Indra-Payoong, N. “Multi-features particle PHD filtering for multiple humans tracking,” in Computer Science and Engineering Conference (ICSEC), 2015 International, 2015, pp. 1–6.

  • Tian, Y., Senior, A., & Lu, M. (2012). Robust and efficient foreground analysis in complex surveillance videos. Machine Vision and Applications, 23, 967–983.

    Article  Google Scholar 

  • Tsai, T.-H., Peng, D.-Z., Lin, C.-Y. and Sheu, W.-T. “A low cost foreground object detection architecture design with multi-model background maintenance algorithm,” In VLSI vol. 16. 2009.

  • Vacavant, A., Chateau, T., Wilhelm, A., and Lequievre, L. “A benchmark dataset for foreground/background extraction,” in: ACCV 2012, Workshop: Background Models Challenge, 2012.

  • Van Droogenbroeck, M. and Paquot, O. “Background subtraction: Experiments and improvements for ViBe,” in Computer Vision and Pattern Recognition Workshops (CVPRW), 2012 IEEE Computer Society Conference on, 2012, pp. 32–37.

  • Wahyono, Filonenko, A. and Jo, K. H. “Detecting abandoned objects in crowded scenes of surveillance videos using adaptive dual background model,” in Proceedings - 2015 8th International Conference on Human System Interaction, HSI 2015, 2015, vol. 55, no. 2, pp. 224–227.

  • Wang, L., Ouyang, W., Wang, X. and Lu, H. “Visual tracking with fully convolutional networks,” In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2015, pp. 3119–3127.

  • Wang, Y., Jodoin, P.-M., Porikli, F., Konrad, J., Benezeth, Y., and Ishwar, P. “CDnet 2014: An expanded change detection benchmark dataset,” in: Computer Vision and Pattern Recognition Workshops (CVPRW), 2014 IEEE Conference on, 2014, pp. 393–400.

  • Wei, H., & Peng, Q. (2018). A block-wise frame difference method for real-time video motion detection. International Journal of Advanced Robotic Systems, 15, 1729881418783633.

    Article  Google Scholar 

  • Xie, Q., Remil, O., Guo, Y., Wang, M., Wei, M., & Wang, J. (2017). Object detection and tracking under occlusion for object-level RGB-D video segmentation. IEEE Transactions on Multimedia, 20(3), 580–592.

    Article  Google Scholar 

  • Yang, C. J., Chou, T., Chang, F. A., Ssu-Yuan, C. and Guo, J. I. “A smart surveillance system with multiple people detection, tracking, and behavior analysis,” in 2016 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2016, 2016, pp. 1–4.

  • Yao, J. and Odobez, J.-M. “Multi-layer background subtraction based on color and texture,” in Computer Vision and Pattern Recognition, 2007. CVPR’07. IEEE Conference on, 2007, pp. 1–8

  • Zaharin, M. S., Ibrahim, N., & Dir, T. M. A. T. (2020). Comparison of human detection using background subtraction and frame difference. Bulletin of Electrical Engineering and Informatics, 9(1), 345–353.

    Article  Google Scholar 

  • Zhang, X., Zhu, C., Hong-gang, W., Liu, Z., & Yuan-yuan, X. (2017). “An imbalance compensation framework for background subtraction. IEEE Transactions on Multimedia, 19(11), 2425–2438.

    Article  Google Scholar 

  • Zhang, H. and Xu, D. “Fusing color and texture features for background model,” in Fuzzy Systems and Knowledge Discovery: Third International Conference, FSKD 2006, Xi’an, China, 2006. Proceedings 3, 2006, pp. 887–893.

  • Zhong, Z., Zhang, B., Lu, G., Zhao, Y., & Xu, Y. (2017). An adaptive background modeling method for foreground segmentation. IEEE Transactions on Intelligent Transportation Systems, 18(5), 1109–1121.

    Article  Google Scholar 

  • Zhou, Y., Hu, B., and Zhang, J. “Occlusion detection and tracking method based on bayesian decision theory,” in Pacific-Rim Symposium on Image and Video Technology, 2006, pp. 474–482.

  • Zhu, Q., Xiong, H. and Jiang, X. “Shape-oriented segmentation with graph matching corroboration for silhouette tracking,” in Visual Communications and Image Processing (VCIP), 2012 IEEE, 2012, pp. 1–6.

  • Zivkovic, Z., & van derHeijden, F. (2006). Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recognition Letter, 27(7), 773–780.

    Article  Google Scholar 

  • “PETS 2001 dataset.” [Online]. Available: http://ftp.pets.rdg.ac.uk/pub/PETS2001/.

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, National Central University, Taoyüan, 320, Taiwan

    Tsung-Han Tsai & Ching-Chin Yang

Authors
  1. Tsung-Han Tsai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ching-Chin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tsung-Han Tsai.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tsai, TH., Yang, CC. A real-time surveillance system with multi-object tracking. Multidim Syst Sign Process 34, 767–791 (2023). https://doi.org/10.1007/s11045-023-00883-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11045-023-00883-x

Keywords

Search

Navigation