Journal of Real-Time Image Processing

, Volume 13, Issue 2, pp 397–414 | Cite as

Embedded architecture for noise-adaptive video object detection using parameter-compressed background modeling

  • Kumara Ratnayake
  • Aishy Amer
Original Research Paper


Video processing algorithms are computationally intensive and place stringent requirements on performance and efficiency of memory bandwidth and capacity. As such, efficient hardware accelerations are inevitable for fast video processing systems. In this paper, we propose resource- and power-optimized FPGA-based configurable architecture for video object detection by integrating noise estimation, Mixture-of-Gaussian background modeling, motion detection, and thresholding. Due to large amount of background modeling parameters, we propose a novel Gaussian parameter compression technique suitable for resource- and power-constraint embedded video systems. The proposed architecture is simulated, synthesized and verified for its functionality, accuracy and performance on a Virtex-5 FPGA-based embedded platform by directly interfacing to a digital video input. Intentional exploitation of heterogeneous resources in FPGAs, and advanced design techniques such as heavy pipelining and data parallelism yield real-time processing of HD-1080p video streams at 30 frames per second. Objective and subjective evaluations to existing hardware-based methods show that the proposed architecture obtains orders of magnitude performance improvements, while utilizing minimal hardware resources. This work is an early attempt to devise a complete video surveillance system onto a stand-alone resource-constraint FPGA-based smart camera.


Field-programmable gate arrays FPGA Video signal processing Noise Moving objects Motion detection Thresholding Gaussian background update Gaussian parameter compression 



This work was supported, in part, by the Fonds de la recherche sur la nature et les technologies du Quebec (NATEQ).


  1. 1.
    Oliver, N., Rosario, B., Pentland, A.: A Bayesian computer vision system for modeling human interactions. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 831–843 (2000)CrossRefGoogle Scholar
  2. 2.
    Li, L., Huang, W., Gu, I.Y.H., Tian, Q.: Statistical modeling of complex backgrounds for foreground object detection. IEEE Trans Image Process. 13(11), 1459–1472 (2004)CrossRefGoogle Scholar
  3. 3.
    Cheung, S.C.S., Kamath, C.: Robust techniques for background subtraction in urban traffic video. Proc. SPIE 5308, 881–892 (2004)CrossRefGoogle Scholar
  4. 4.
    Happe, M., Lübbers, E., Platzner, M.: A self-adaptive heterogeneous multi-core architecture for embedded real-time video object tracking. J. Real Time Image Process. (2011) (Published online)Google Scholar
  5. 5.
    Chakraborty, D., Shankar, B.U., Pal, S.K.: Granulation, Rough Entropy and Spatiotemporal Moving Object Detection. Applied Soft Computing (2012)Google Scholar
  6. 6.
    Gao, H., Peng, Y., Dai, Z., Xie, F.: A new detection algorithm of moving objects based on human morphology. In: IEEE International Conference on Intelligent Information Hiding and Multimedia, Signal Processing, pp. 411–414 (2012)Google Scholar
  7. 7.
    Wang, Y.T., Chen, K.W., Chiou, M.J.: Moving object detection using monocular vision. Intell. Auton. Syst., pp. 183–192 (2013)Google Scholar
  8. 8.
    Stauffer, C., Grimson, W.E.: Learning patterns of activity using real-time tracking. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 747–757 (2000)CrossRefGoogle Scholar
  9. 9.
    Piccardi, M.: Background subtraction techniques: a review. IEEE Int. Conf. Syst. Man Cybern. 4, 3099–3104 (2004)Google Scholar
  10. 10.
    Cope, B., Cheung, P.Y.K., Luk, W., Howes, L.: Performance comparison of graphics processors to reconfigurable logic: a case study. IEEE Trans. Comput. 59(4), 433–448 (2010)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Papakonstantinou, A., Gururaj, K., Stratton, J.A., Chen, D., Cong, J., Hwu, W.M.W.: FCUDA: Enabling efficient compilation of CUDA kernels onto FPGAs. In: IEEE Symposium on Application Specific Processors, pp. 35–42 (2009)Google Scholar
  12. 12.
    Amer, A., Dubois, E.: Fast and reliable structure-oriented video noise estimation. IEEE Trans. Circuits Syst. Video Technol. 15, 113–118 (2005)CrossRefGoogle Scholar
  13. 13.
    Achkar, F., Amer, A.: Hysteresis-based selective Gaussian mixture models for real-time background maintenance. IS T/SPIE Symp. Electron. Imaging 6508(2), 65082J.1–65082J.11 (2007)Google Scholar
  14. 14.
    Amer, A.: Memory-based spatio-temporal real-time object segmentation. In: SPIE International Symposium on Electronic Imaging, Conference on Real-Time Imaging, vol. 5012, pp. 10–21 (2003)Google Scholar
  15. 15.
    Ratnayake, K., Amer, A.: An FPGA-based implementation of spatio-temporal object segmentation. In: IEEE International Conference on Image Processing, pp. 3265–3268 (2006)Google Scholar
  16. 16.
    Zhang, D., Lu, G.: Segmentation of moving objects in image sequence: a review. Circuits Syst. Signal Process. 20(2), 143–183 (2001)CrossRefzbMATHGoogle Scholar
  17. 17.
    Karman, K.P., von Brandt, A.: Moving object recognition using an adaptive background memory. Time Varying Image Process. Movi. Object Recogn., pp. 297–307 (1990)Google Scholar
  18. 18.
    Toyama, K., Krumm, J., Brumitt, B., Meyers, B.: Wallflower: Principles and practice of background maintenance. IEEE Int. Conf. Comput. Vis. 1, 255–261 (1999)Google Scholar
  19. 19.
    Zivkovic, Z.: Improved adaptive Gaussian mixture model for background subtraction. In: IEEE International Conference on the, Pattern Recognition, pp. 28–31 (2004)Google Scholar
  20. 20.
    Lee, D.S.: Effective Gaussian mixture learning for video background subtraction. IEEE Trans. Pattern Anal. Mach. Intell. 27(5), 827–832 (2005)CrossRefGoogle Scholar
  21. 21.
    Cucchiara, R., Grana, C., Piccardi, M., Prati, A.: Detecting moving objects, ghosts, and shadows in video streams. IEEE Trans. Pattern Anal. Mach. Intell. 25(10), 1337–1342 (2003)CrossRefGoogle Scholar
  22. 22.
    Elgammal, A.M., Duraiswami, R., Davis, L.S.: Efficient kernel density estimation using the fast gauss transform with applications to color modeling and tracking. IEEE Trans. Pattern Anal. Mach. Intell. 25(11), 1499–1504 (2003)CrossRefGoogle Scholar
  23. 23.
    Appiah, K., Hunter, A.: A Single-Chip FPGA implementation of real-time adaptive background model. In: EEE International Conference on Field-Programmable Technolog, pp. 95–102 (2005)Google Scholar
  24. 24.
    Oliveira, J., Printes, A., Freire, R.C.S., Melcher, E., Silva, I.S.S.: FPGA architecture for static background subtraction in real time. In: Annual Symposium on Integrated Circuits and Systems Design, pp. 26–31 (2006)Google Scholar
  25. 25.
    Schlessman, J., Lodato, M., Ozer, B., Wolf, W.: Heterogeneous MPSoC architectures for embedded computer vision. In IEEE International Conference on Multimedia and Expo, pp. 1870–1873 (2007)Google Scholar
  26. 26.
    Kristensen, F., Hedberg, H., Jiang, H., Nilsson, P., Öwall, V.: An embedded real-time surveillance system: implementation and evaluation. J. Signal Process. Syst. 52(1), 75–94 (2008)CrossRefGoogle Scholar
  27. 27.
    Jiang, H., Ardo, H., Öwall, V.: A hardware architecture for real-time video segmentation utilizing memory reduction techniques. IEEE Trans. Circuits Syst. Video Technol. 19(2), 226–236 (2009)CrossRefGoogle Scholar
  28. 28.
    Genovese, M., Napoli, E.: FPGA-based architecture for real time segmentation and denoising of HD video. J. Real Time Image Process. (2011) (Published online)Google Scholar
  29. 29.
    Teuhola, J.: A compression method for clustered bit-vectors. Inf. Process. Lett. 7, 308–311 (1978)CrossRefzbMATHGoogle Scholar
  30. 30.
    Ratnayake, K., Amer, A.: An FPGA architecture of stable-sorting on a large data volume: application to video signals. In: IEEE Conference on Information Sciences and Systems, pp. 431–436 (2007)Google Scholar
  31. 31.
    Wang, T.C., Fang, H.C., Chao, W.M., Chen, H.H., Chen, L.G.: An UVLC encoder architecture for H. 26L. In: IEEE International Symposium on Circuits and Systems, vol. 2, pp. 308–311 (2002)Google Scholar
  32. 32.
    Osman, H., Mahjoup, W., Nabih, A., Aly, G.M.: JPEG encoder for low-cost FPGAs. In: International Conference on Computer Engineering Systems, pp. 406–411 (2007)Google Scholar
  33. 33.
    Yu, G., Vladimirova, T., Wu, X., Sweeting, M.N.: A new high-level reconfigurable lossless image compression system for space applications. In: NASA/ESA Conference on Adaptive Hardware and Systems, pp. 183–190 (2008)Google Scholar
  34. 34.
    Mahapatra, S., Singh, K.: An FPGA-based implementation of multi-alphabet arithmetic coding. IEEE Trans. Circuits Syst. I Regul. Papers 54(8), 1678–1686 (2007)CrossRefGoogle Scholar
  35. 35.
    Rosin, P.L.: Thresholding for change detection. Comput. Vis. Image Underst. 86, 79–95 (2002)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Department of Electrical and Computer EngineeringConcordia UniversityMontréalCanada

Personalised recommendations