Incremental Principal Component Pursuit for Video Background Modeling

  • Paul Rodriguez
  • Brendt Wohlberg


Video background modeling is an important preprocessing step in many video analysis systems. Principal component pursuit (PCP), which is currently considered to be the state-of-the-art method for this problem, has a high computational cost, and processes a large number of video frames at a time, resulting in high memory usage and constraining the applicability of this method to streaming video. In this paper, we propose a novel fully incremental PCP algorithm for video background modeling. It processes one frame at a time, obtaining similar results to standard batch PCP algorithms, while being able to adapt to changes in the background. It has an extremely low memory footprint, and a computational complexity that allows real-time processing.


Principal component pursuit Video background modeling  Incremental singular value decomposition 


Compliance with ethical standards


This research was supported by the “Fondo para la Innovación, la Ciencia y la Tecnología” (Fincyt) Program for author Paul Rodriguez. This research was supported by the U.S. Department of Energy through the LANL/LDRD Program and by UC Lab Fees Research grant 12-LR-236660 for author Brendt Wohlberg.


  1. 1.
    Aybat, N., Goldfarb, G., Iyengar, G.: Fast first-order methods for stable principal component pursuit (2011).
  2. 2.
    Baker, C., Gallivan, K., Dooren, P.V.: Low-rank incremental methods for computing dominant singular subspaces. Linear Algebra Appl. 436(8), 2866–2888 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Bezdek, J., Hathaway, R.: Some notes on alternating optimization. In: Advances in Soft Computing—AFSS 2002. Lecture Notes in Computer Science, vol. 2275, pp. 288–300. Springer, Heidelberg (2002)Google Scholar
  4. 4.
    Bouwmans, T., Baf, F., Vachon, B.: Statistical background modeling for foreground detection: a survey. Handbook of Pattern Recognition and Computer Vision, pp. 181–199. World Scientific Publishing, Singapore (2010)Google Scholar
  5. 5.
    Bouwmans, T., Zahzah, E.: Robust PCA via principal component pursuit: a review for a comparative evaluation in video surveillance. Comput. Vis. Image Underst. 122, 22–34 (2014)CrossRefGoogle Scholar
  6. 6.
    Brand, M.: Fast low-rank modifications of the thin singular value decomposition. Linear Algebra Appl. 415(1), 20–30 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Bunch, J., Nielsen, C.: Updating the singular value decomposition. Numer. Math. 31(2), 111–129 (1978). doi: 10.1007/BF01397471 MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Candès, E., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM 58(3), 11 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Chahlaoui, Y., Gallivan, K., Van Dooren, P.: Computational information retrieval. In: In Computational Information Retrieval, chap. An Incremental Method for Computing Dominant Singular Spaces, pp. 53–62. SIAM (2001)Google Scholar
  10. 10.
    Guo, H.: Practical ReProCS code.
  11. 11.
    Guo, H., Qiu, C., Vaswani, N.: An online algorithm for separating sparse and low-dimensional signal sequences from their sum. IEEE Trans. Signal Process. 62(16), 4284–4297 (2014)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Guyon, C., Bouwmans, T., Zahzah, E.: Robust principal component analysis for background subtraction: Systematic evaluation and comparative analysis. In: Principal Component Analysis. ch. 12. InTech (2012). doi: 10.5772/38267
  13. 13.
    He, J., Balzano, L., Lui, J.: Online robust subspace tracking from partial information. CoRR abs/1109.3827 (2011).
  14. 14.
    He, J., Balzano, L., Szlam, A.: GRASTA code.
  15. 15.
    He, J., Balzano, L., Szlam, A.: Incremental gradient on the grassmannian for online foreground and background separation in subsampled video. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1568–1575 (2012). doi: 10.1109/CVPR.2012.6247848
  16. 16.
    He, J., Zhang, D., Balzano, L., Tao, T.: Iterative Grassmannian optimization for robust image alignment. Image Vis. Comput. 32(10), 800–813 (2014)CrossRefGoogle Scholar
  17. 17.
    Hu, Y., Sirlantzis, K., Howells, G., Ragot, N., Rodriguez, P.: An online background subtraction algorithm using a contiguously weighted linear regression model. In: 23rd European Signal Processing Conference (EUSIPCO). Nice, France (2015)Google Scholar
  18. 18.
    Jodoin, P., Konrad, J.: Cdnet 2014 dataset.
  19. 19.
    Lankershim boulevard dataset (2007). U.S. Department of Transportation Publication FHWA-HRT-07-029.
  20. 20.
    Li, L., Huang, W., Gu, I.Y.H., Tian, Q.: Background model test data.
  21. 21.
    Li, L., Huang, W., Gu, I.Y.H., Tian, Q.: Foreground object detection from videos containing complex background. In: Proceedings of the Eleventh ACM International Conference on Multimedia, MULTIMEDIA ’03, pp. 2–10. ACM, New York (2003). doi: 10.1145/957013.957017
  22. 22.
    Lin, Z., Chen, M., Ma, Y.: The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices (2011).
  23. 23.
    Liu, G., Lin, Z., Yu, Y.: Robust subspace segmentation by low-rank representation. In: International Conference on Machine Learning (ICML), pp. 663–670 (2010)Google Scholar
  24. 24.
    Mateos, G., Giannakis, G.: Robust pca as bilinear decomposition with outlier-sparsity regularization. IEEE Trans. Signal Process. 60(10), 5176–5190 (2012)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Min, K., Zhang, Z., Wright, J., Ma, Y.: Decomposing background topics from keywords by principal component pursuit. In: Proceedings of the 19th ACM International Conference on Information and Knowledge Management (CIKM), pp. 269–278. ACM, New York (2010)Google Scholar
  26. 26.
    Peng, Y., Ganesh, A., Wright, J., Xu, W., Ma, Y.: RASL: robust alignment by sparse and low-rank decomposition for linearly correlated images. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2233–2246 (2012)CrossRefGoogle Scholar
  27. 27.
    Pope, G., Baumann, M., Studer, C., Durisi, G.: Real-time principal component pursuit. In: 45th Asilomar Conference on Signals, Systems and Computers, pp. 1433–1437 (2011). doi: 10.1109/ACSSC.2011.6190254
  28. 28.
    Qiu, C., Vaswani, N.: Real-time robust principal components’ pursuit. CoRR abs/1010.0608 (2010).
  29. 29.
    Qiu, C., Vaswani, N.: Reprocs: A missing link between recursive robust PCA and recursive sparse recovery in large but correlated noise. CoRR abs/1106.3286 (2011)Google Scholar
  30. 30.
    Qiu, C., Vaswani, N.: Support predicted modified-cs for recursive robust principal components pursuit. In: IEEE International Symposium on Information Theory (ISIT), pp. 668–672 (2011)Google Scholar
  31. 31.
    Rodríguez, P.: Jitter invariant incremental principal component pursuit for video background modeling on the tk1. In: IEEE Asilomar Conference on Signal, Systems and Computers (ACSSC) (2015), AcceptedGoogle Scholar
  32. 32.
    Rodriguez, P.: Real-time incremental principal component pursuit for video background modeling on the TK1 (2015). In: GPU Technical ConferenceGoogle Scholar
  33. 33.
    Rodríguez, P., Wohlberg, B.: incremental PCP simulations.
  34. 34.
    Rodríguez, P., Wohlberg, B.: Fast principal component pursuit via alternating minimization. In: Proceedings of IEEE International Conference on Image Processing (ICIP), pp. 69–73. Melbourne, Australia (2013)Google Scholar
  35. 35.
    Rodríguez, P., Wohlberg, B.: Video background modeling under impulse noise. In: Proceedings of IEEE International Conference on Image Processing (ICIP). Paris, France (2014)Google Scholar
  36. 36.
    Rodríguez, P., Wohlberg, B.: Translational and rotational jitter invariant incremental principal component pursuit for video background modeling. In: IEEE International Conference on Image Processing (ICIP) (2015), AcceptedGoogle Scholar
  37. 37.
    Rosin, P.: Unimodal thresholding. Pattern Recognit. 34, 2083–2096 (2001)CrossRefzbMATHGoogle Scholar
  38. 38.
    Seidel, F., Hage, C., Kleinsteuber, M.: pROST: a smoothed lp-norm robust online subspace tracking method for background subtraction in video. Mach. Vis. Appl. 25(5), 1227–1240 (2014)CrossRefGoogle Scholar
  39. 39.
    Shah, M., Deng, J., Woodford, B.: Video background modeling: recent approaches, issues and our proposed techniques. Mach. Vis. Appl. (2013). doi: 10.1007/s00138-013-0552-7
  40. 40.
    USC Neovision2 Project. DARPA Neovision2.
  41. 41.
    Srebro, N., Rennie, J., Jaakola, T.: Maximum-margin matrix factorization. In: Advances in Neural Information Processing Systems 17, pp. 1329–1336. MIT Press, London (2005)Google Scholar
  42. 42.
    Wang, N.: PRMF code.
  43. 43.
    Wang, N., Yao, T., Wang, J., Yeung, D.Y.: A probabilistic approach to robust matrix factorization. In: A. Fitzgibbon, S. Lazebnik, P. Perona, Y. Sato, C. Schmid (eds.) Computer Vision—ECCV 2012, Lecture Notes in Computer Science, vol. 7578, pp. 126–139. Springer, Heidelberg (2012).
  44. 44.
    Wang, Y., Jodoin, P., Porikli, F., Konrad, J., Benezeth, Y., Ishwar, P.: Cdnet 2014: an expanded change detection benchmark dataset. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 393–400 (2014). doi: 10.1109/CVPRW.2014.126
  45. 45.
    Wright, J., Ganesh, A., Rao, S., Peng, Y., Ma, Y.: Robust principal component analysis: exact recovery of corrupted low-rank matrices via convex optimization. Adv. Neural Inf. Process. Syst. (NIPS) 22, 2080–2088 (2009)Google Scholar
  46. 46.
    Xiong, L., Chen, X., Schneider, J.: Direct robust matrix factorization for anomaly detection. In: IEEE International Conference on Data Mining, pp. 844–853 (2011)Google Scholar
  47. 47.
    Xu, J., Ithapu, V.K., Mukherjee, L., Rehg, J.M., Singh, V.: GOSUS code.
  48. 48.
    Xu, J., Ithapu, V.K., Mukherjee, L., Rehg, J.M., Singh, V.: GOSUS: Grassmannian online subspace updates with structured-sparsity. In: IEEE International Conference on Computer Vision (ICCV), pp. 3376–3383 (2013)Google Scholar
  49. 49.
    Zhou, T., Tao, D.: GoDec: Randomized low-rank & sparse matrix decomposition in noisy case. In: ACM International Conference on Machine Learning, ICML ’11, pp. 33–40 (2011)Google Scholar
  50. 50.
    Zhou, X., Yang, C., Yu, W.: Moving object detection by detecting contiguous outliers in the low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell. 35(3), 597–610 (2013). doi: 10.1109/TPAMI.2012.132 CrossRefGoogle Scholar
  51. 51.
    Zhou, Z., Li, X., Wright, J., Candes, E., Ma, Y.: Stable principal component pursuit. In: IEEE International Symposium on Information Theory Proceedings (ISIT), pp. 1518–1522 (2010)Google Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Electrical Engineering DepartmentPontificia Universidad Católica del PerúLimaPeru
  2. 2.Theoretical DivisionLos Alamos National LaboratoryLos AlamosUSA

Personalised recommendations