Skip to main content
SpringerLink
Log in

Incremental Principal Component Pursuit for Video Background Modeling

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

Video background modeling is an important preprocessing step in many video analysis systems. Principal component pursuit (PCP), which is currently considered to be the state-of-the-art method for this problem, has a high computational cost, and processes a large number of video frames at a time, resulting in high memory usage and constraining the applicability of this method to streaming video. In this paper, we propose a novel fully incremental PCP algorithm for video background modeling. It processes one frame at a time, obtaining similar results to standard batch PCP algorithms, while being able to adapt to changes in the background. It has an extremely low memory footprint, and a computational complexity that allows real-time processing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Notes

  1. The GRASTA and GOSUS algorithms can perform the initial background estimation in a non-batch fashion, however the resulting performance is not as good as when the default batch procedure is used, as shown in Sect. 6

  2. The code is publicly available [33].

  3. These are the default values found the in GRASTA [14] and GOSUS [47] implementations

  4. As will be explained next, when GRASTA’s “rand” variant is used, its background estimate is not stable for several frames (usually about 100, but varying with each case). The ground-truth frames for “Fountain” start at frame 158 and for “Lobby” at 350; also, for the latter case, see Fig. 11.

  5. TP, FN and FP variables in (41) are accumulated for all frames before computing the precision and recall measures that lead to the F-measure.

References

  1. Aybat, N., Goldfarb, G., Iyengar, G.: Fast first-order methods for stable principal component pursuit (2011). http://arxiv.org/abs/1105.2126

  2. Baker, C., Gallivan, K., Dooren, P.V.: Low-rank incremental methods for computing dominant singular subspaces. Linear Algebra Appl. 436(8), 2866–2888 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bezdek, J., Hathaway, R.: Some notes on alternating optimization. In: Advances in Soft Computing—AFSS 2002. Lecture Notes in Computer Science, vol. 2275, pp. 288–300. Springer, Heidelberg (2002)

  4. Bouwmans, T., Baf, F., Vachon, B.: Statistical background modeling for foreground detection: a survey. Handbook of Pattern Recognition and Computer Vision, pp. 181–199. World Scientific Publishing, Singapore (2010)

    Google Scholar 

  5. Bouwmans, T., Zahzah, E.: Robust PCA via principal component pursuit: a review for a comparative evaluation in video surveillance. Comput. Vis. Image Underst. 122, 22–34 (2014)

    Article  Google Scholar 

  6. Brand, M.: Fast low-rank modifications of the thin singular value decomposition. Linear Algebra Appl. 415(1), 20–30 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  7. Bunch, J., Nielsen, C.: Updating the singular value decomposition. Numer. Math. 31(2), 111–129 (1978). doi:10.1007/BF01397471

    Article  MathSciNet  MATH  Google Scholar 

  8. Candès, E., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM 58(3), 11 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  9. Chahlaoui, Y., Gallivan, K., Van Dooren, P.: Computational information retrieval. In: In Computational Information Retrieval, chap. An Incremental Method for Computing Dominant Singular Spaces, pp. 53–62. SIAM (2001)

  10. Guo, H.: Practical ReProCS code. http://www.ece.iastate.edu/~hanguo/PracReProCS.html

  11. Guo, H., Qiu, C., Vaswani, N.: An online algorithm for separating sparse and low-dimensional signal sequences from their sum. IEEE Trans. Signal Process. 62(16), 4284–4297 (2014)

    Article  MathSciNet  Google Scholar 

  12. Guyon, C., Bouwmans, T., Zahzah, E.: Robust principal component analysis for background subtraction: Systematic evaluation and comparative analysis. In: Principal Component Analysis. ch. 12. InTech (2012). doi:10.5772/38267

  13. He, J., Balzano, L., Lui, J.: Online robust subspace tracking from partial information. CoRR abs/1109.3827 (2011). http://arxiv.org/pdf/1109.3827v2

  14. He, J., Balzano, L., Szlam, A.: GRASTA code. https://sites.google.com/site/hejunzz/grasta

  15. He, J., Balzano, L., Szlam, A.: Incremental gradient on the grassmannian for online foreground and background separation in subsampled video. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1568–1575 (2012). doi:10.1109/CVPR.2012.6247848

  16. He, J., Zhang, D., Balzano, L., Tao, T.: Iterative Grassmannian optimization for robust image alignment. Image Vis. Comput. 32(10), 800–813 (2014)

    Article  Google Scholar 

  17. Hu, Y., Sirlantzis, K., Howells, G., Ragot, N., Rodriguez, P.: An online background subtraction algorithm using a contiguously weighted linear regression model. In: 23rd European Signal Processing Conference (EUSIPCO). Nice, France (2015)

  18. Jodoin, P., Konrad, J.: Cdnet 2014 dataset. http://wordpress-jodoin.dmi.usherb.ca/dataset2014/

  19. Lankershim boulevard dataset (2007). U.S. Department of Transportation Publication FHWA-HRT-07-029. http://ngsim-community.org/

  20. Li, L., Huang, W., Gu, I.Y.H., Tian, Q.: Background model test data. http://perception.i2r.a-star.edu.sg/bk_model/bk_index.html

  21. Li, L., Huang, W., Gu, I.Y.H., Tian, Q.: Foreground object detection from videos containing complex background. In: Proceedings of the Eleventh ACM International Conference on Multimedia, MULTIMEDIA ’03, pp. 2–10. ACM, New York (2003). doi:10.1145/957013.957017

  22. Lin, Z., Chen, M., Ma, Y.: The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices (2011). http://arxiv.org/abs/1009.5055v2

  23. Liu, G., Lin, Z., Yu, Y.: Robust subspace segmentation by low-rank representation. In: International Conference on Machine Learning (ICML), pp. 663–670 (2010)

  24. Mateos, G., Giannakis, G.: Robust pca as bilinear decomposition with outlier-sparsity regularization. IEEE Trans. Signal Process. 60(10), 5176–5190 (2012)

    Article  MathSciNet  Google Scholar 

  25. Min, K., Zhang, Z., Wright, J., Ma, Y.: Decomposing background topics from keywords by principal component pursuit. In: Proceedings of the 19th ACM International Conference on Information and Knowledge Management (CIKM), pp. 269–278. ACM, New York (2010)

  26. Peng, Y., Ganesh, A., Wright, J., Xu, W., Ma, Y.: RASL: robust alignment by sparse and low-rank decomposition for linearly correlated images. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2233–2246 (2012)

    Article  Google Scholar 

  27. Pope, G., Baumann, M., Studer, C., Durisi, G.: Real-time principal component pursuit. In: 45th Asilomar Conference on Signals, Systems and Computers, pp. 1433–1437 (2011). doi:10.1109/ACSSC.2011.6190254

  28. Qiu, C., Vaswani, N.: Real-time robust principal components’ pursuit. CoRR abs/1010.0608 (2010). http://arxiv.org/abs/1010.0608

  29. Qiu, C., Vaswani, N.: Reprocs: A missing link between recursive robust PCA and recursive sparse recovery in large but correlated noise. CoRR abs/1106.3286 (2011)

  30. Qiu, C., Vaswani, N.: Support predicted modified-cs for recursive robust principal components pursuit. In: IEEE International Symposium on Information Theory (ISIT), pp. 668–672 (2011)

  31. Rodríguez, P.: Jitter invariant incremental principal component pursuit for video background modeling on the tk1. In: IEEE Asilomar Conference on Signal, Systems and Computers (ACSSC) (2015), Accepted

  32. Rodriguez, P.: Real-time incremental principal component pursuit for video background modeling on the TK1 (2015). In: GPU Technical Conference

  33. Rodríguez, P., Wohlberg, B.: incremental PCP simulations. http://sites.google.com/a/istec.net/prodrig/Home/en/pubs/incpcp

  34. Rodríguez, P., Wohlberg, B.: Fast principal component pursuit via alternating minimization. In: Proceedings of IEEE International Conference on Image Processing (ICIP), pp. 69–73. Melbourne, Australia (2013)

  35. Rodríguez, P., Wohlberg, B.: Video background modeling under impulse noise. In: Proceedings of IEEE International Conference on Image Processing (ICIP). Paris, France (2014)

  36. Rodríguez, P., Wohlberg, B.: Translational and rotational jitter invariant incremental principal component pursuit for video background modeling. In: IEEE International Conference on Image Processing (ICIP) (2015), Accepted

  37. Rosin, P.: Unimodal thresholding. Pattern Recognit. 34, 2083–2096 (2001)

    Article  MATH  Google Scholar 

  38. Seidel, F., Hage, C., Kleinsteuber, M.: pROST: a smoothed lp-norm robust online subspace tracking method for background subtraction in video. Mach. Vis. Appl. 25(5), 1227–1240 (2014)

    Article  Google Scholar 

  39. Shah, M., Deng, J., Woodford, B.: Video background modeling: recent approaches, issues and our proposed techniques. Mach. Vis. Appl. (2013). doi:10.1007/s00138-013-0552-7

  40. USC Neovision2 Project. DARPA Neovision2. http://ilab.usc.edu/neo2/

  41. Srebro, N., Rennie, J., Jaakola, T.: Maximum-margin matrix factorization. In: Advances in Neural Information Processing Systems 17, pp. 1329–1336. MIT Press, London (2005)

  42. Wang, N.: PRMF code. http://winsty.net/prmf.html

  43. Wang, N., Yao, T., Wang, J., Yeung, D.Y.: A probabilistic approach to robust matrix factorization. In: A. Fitzgibbon, S. Lazebnik, P. Perona, Y. Sato, C. Schmid (eds.) Computer Vision—ECCV 2012, Lecture Notes in Computer Science, vol. 7578, pp. 126–139. Springer, Heidelberg (2012). http://dx.doi.org/10.1007/978-3-642-33786-4_10

  44. Wang, Y., Jodoin, P., Porikli, F., Konrad, J., Benezeth, Y., Ishwar, P.: Cdnet 2014: an expanded change detection benchmark dataset. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 393–400 (2014). doi:10.1109/CVPRW.2014.126

  45. Wright, J., Ganesh, A., Rao, S., Peng, Y., Ma, Y.: Robust principal component analysis: exact recovery of corrupted low-rank matrices via convex optimization. Adv. Neural Inf. Process. Syst. (NIPS) 22, 2080–2088 (2009)

    Google Scholar 

  46. Xiong, L., Chen, X., Schneider, J.: Direct robust matrix factorization for anomaly detection. In: IEEE International Conference on Data Mining, pp. 844–853 (2011)

  47. Xu, J., Ithapu, V.K., Mukherjee, L., Rehg, J.M., Singh, V.: GOSUS code. http://pages.cs.wisc.edu/~jiaxu/projects/gosus/

  48. Xu, J., Ithapu, V.K., Mukherjee, L., Rehg, J.M., Singh, V.: GOSUS: Grassmannian online subspace updates with structured-sparsity. In: IEEE International Conference on Computer Vision (ICCV), pp. 3376–3383 (2013)

  49. Zhou, T., Tao, D.: GoDec: Randomized low-rank & sparse matrix decomposition in noisy case. In: ACM International Conference on Machine Learning, ICML ’11, pp. 33–40 (2011)

  50. Zhou, X., Yang, C., Yu, W.: Moving object detection by detecting contiguous outliers in the low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell. 35(3), 597–610 (2013). doi:10.1109/TPAMI.2012.132

    Article  Google Scholar 

  51. Zhou, Z., Li, X., Wright, J., Candes, E., Ma, Y.: Stable principal component pursuit. In: IEEE International Symposium on Information Theory Proceedings (ISIT), pp. 1518–1522 (2010)

Download references

Author information

Authors and Affiliations

  1. Electrical Engineering Department, Pontificia Universidad Católica del Perú, Lima, Peru

    Paul Rodriguez

  2. Theoretical Division, Los Alamos National Laboratory, Los Alamos, NM, 87545, USA

    Brendt Wohlberg

Authors
  1. Paul Rodriguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Brendt Wohlberg
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paul Rodriguez.

Ethics declarations

Funding

This research was supported by the “Fondo para la Innovación, la Ciencia y la Tecnología” (Fincyt) Program for author Paul Rodriguez. This research was supported by the U.S. Department of Energy through the LANL/LDRD Program and by UC Lab Fees Research grant 12-LR-236660 for author Brendt Wohlberg.

Additional information

There is a patent application number 14/722,651 that covers the incremental PCP method described in this document.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rodriguez, P., Wohlberg, B. Incremental Principal Component Pursuit for Video Background Modeling. J Math Imaging Vis 55, 1–18 (2016). https://doi.org/10.1007/s10851-015-0610-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-015-0610-z

Keywords

Search

Navigation