Skip to main content

Simultaneous Motion Detection and Background Reconstruction with a Conditional Mixed-State Markov Random Field

Abstract

In this work we present a new way of simultaneously solving the problems of motion detection and background image reconstruction. An accurate estimation of the background is only possible if we locate the moving objects. Meanwhile, a correct motion detection is achieved if we have a good available background model. The key of our joint approach is to define a single random process that can take two types of values, instead of defining two different processes, one symbolic (motion detection) and one numeric (background intensity estimation). It thus allows to exploit the (spatio-temporal) interaction between a decision (motion detection) and an estimation (intensity reconstruction) problem. Consequently, the meaning of solving both tasks jointly, is to obtain a single optimal estimate of such a process. The intrinsic interaction and simultaneity between both problems is shown to be better modeled within the so-called mixed-state statistical framework, which is extended here to account for symbolic states and conditional random fields. Experiments on real sequences and comparisons with existing motion detection methods support our proposal. Further implications for video sequence inpainting will be also discussed.

This is a preview of subscription content, access via your institution.

References

  1. Benboudjema, D., & Pieczynski, W. (2007). Unsupervised statistical segmentation of nonstationary images using triplet Markov fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29, 1367–1378.

    Article  Google Scholar 

  2. Benedek, C., Sziranyi, T., Kato, Z., & Zerubia, J. (2007). A multi-layer Mrf model for object-motion detection in unregistered airborne image-pairs. In IEEE international conference on image processing, 2007 (ICIP07), 16 2007–Oct. 19 2007 (Vol. 6, pp. 141–144).

    Google Scholar 

  3. Besag, J. (1974). Spatial interaction and the statistical analysis of lattice systems. Journal of the Royal Statistical Society. Series B, 36, 192–236.

    MathSciNet  MATH  Google Scholar 

  4. Besag, J. (1986). On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society. Series B, 48(3), 259–302.

    MathSciNet  MATH  Google Scholar 

  5. Black, M. J., & Rangarajan, A. (1996). On the unification of line processes, outlier rejection, and robust statistics with applications in early vision. International Journal of Computer Vision, 19(1), 57–91.

    Article  Google Scholar 

  6. Blanchet, J., & Forbes, F. (2008). Triplet Markov fields for the classification of complex structure data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(6), 1055–1067.

    Article  Google Scholar 

  7. Bouthemy, P., & Lalande, P. (1993). Recovery of moving object masks in an image sequence using local spatiotemporal contextual information. Optical Engineering, 32(6), 1205–1212.

    Article  Google Scholar 

  8. Bouthemy, P., Hardouin, Ch., Piriou, G., & Yao, J.-F. (2006). Mixed-state auto-models and motion texture modeling. Journal of Mathematical Imaging and Vision, 25(3), 387–402.

    MathSciNet  Article  Google Scholar 

  9. Bugeau, A., & Pérez, P. (2007). Detection and segmentation of moving objects in highly dynamic scenes. In CVPR ’07: proc. of the 2007 IEEE conf. on computer vision and pattern recognition, Minneapolis, MI.

    Google Scholar 

  10. Caillol, H., Hillion, A., & Pieczynski, W. (1993). Fuzzy random fields and unsupervised image segmentation. IEEE Transactions on Geoscience and Remote Sensing, 31, 801–810.

    Article  Google Scholar 

  11. Carincotte, C., Derrode, S., Sicot, G., & Boucher, J. M. (2004). Unsupervised image segmentation based on a new fuzzy hmc model. In ICASSP’04 (pp. 17–21).

    Google Scholar 

  12. Carincotte, C., Derrode, S., & Bourennane, S. (2006). Unsupervised change detection on sar images using fuzzy hidden Markov chains. IEEE Transactions on Geoscience and Remote Sensing, 44(2), 432–441.

    Article  Google Scholar 

  13. Cernuschi-Frias, B. (2007). Mixed states Markov random fields with symbolic labels and multidimensional real values. Technical Report 6255, INRIA, July 2007.

  14. Chellappa, R. (1985). Two-dimensional discrete Gaussian Markov random field models for image processing. In Progress in pattern recognition 2 (Vol. 85, pp. 79–112).

    Google Scholar 

  15. Chen, J., & Tang, C. (2007). Spatio-temporal Markov random field for video denoising. In Proc. IEEE conf. on comp. vision and pattern recognition (CVPR’07), June (pp. 1–8).

    Google Scholar 

  16. Collet, C., & Murtagh, F. (2004). Segmentation based on a hierarchical Markov model. Pattern Recognition, 37(12), 2337–2347.

    Article  Google Scholar 

  17. Criminisi, A., Cross, G., Blake, A., & Kolmogorov, V. (2006). Bilayer segmentation of live video. In CVPR ’06: proceedings of the IEEE computer society conference on computer vision and pattern recognition (pp. 53–60). Washington: IEEE Computer Society.

    Google Scholar 

  18. Crivelli, T., Cernuschi-Frias, B., Bouthemy, P., & Yao, J.-F. (2006). Mixed-state Markov random fields for motion texture modeling and segmentation. In Proc. IEEE int. conf. on image processing (ICIP’06), Atlanta, USA (pp. 1857–1860).

    Google Scholar 

  19. Crivelli, T., Piriou, G., Bouthemy, P., Cernuschi-Frías, B., & Yao, J.-F. (2008). Simultaneous motion detection and background reconstruction with a mixed-state conditional Markov random field. In ECCV ’08: proceedings of the 10th European conference on computer vision, Marseille, France (pp. 113–126).

    Google Scholar 

  20. Crivelli, T., Bouthemy, P., Cernuschi-Frías, B., & Yao, J.-F. (2009). Learning mixed-state Markov models for statistical motion texture tracking. In MLVMA’09: 2nd IEEE int. workshop on machine learning for vision-based motion analysis, Kyoto, Japan.

    Google Scholar 

  21. Elfadel, I., & Picard, R. (1994). Gibbs random fields, cooccurrences, and texture modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(1), 24–37.

    Article  Google Scholar 

  22. Elgammal, A. M., Harwood, D., & Davis, L. S. (2000). Non-parametric model for background subtraction. In ECCV ’00: proc. of the 6th European conf. on comp. vision-part II, London, UK (pp. 751–767).

    Google Scholar 

  23. Fablet, R., & Bouthemy, P. (2003). Motion recognition using non-parametric image motion models estimated from temporal and multiscale co-occurrence statistics. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(12), 1619–1624.

    Article  Google Scholar 

  24. Geman, S., & Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721–741.

    MATH  Article  Google Scholar 

  25. Guyon, X. (1995). Random fields on a network: modeling, statistics and applications. New York: Springer.

    MATH  Google Scholar 

  26. Hardouin, C., & Yao, J. (2008). Multi-parameter auto-models and their applications. Biometrika, 95(2), 335–349.

    MathSciNet  MATH  Article  Google Scholar 

  27. Heitz, F., & Bouthemy, P. (1993). Multimodal estimation of discontinuous optical flow using Markov random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(12), 1217–1232.

    Article  Google Scholar 

  28. Jojic, N., & Frey, B. J. (2001). Learning flexible sprites in video layers. In IEEE int. conference on computer vision and pattern recognition 2001 (pp. 199–206).

    Google Scholar 

  29. Kasetkasem, T., & Varshney, P. K. (2002). An image change detection algorithm based on Markov random field models. IEEE Transactions on Geoscience and Remote Sensing, 40(8), 1815–1823.

    Article  Google Scholar 

  30. Ko, T., Soatto, S., & Estrin, D. (2008). Background subtraction on distributions. In ECCV08 (pp. 276–289).

    Google Scholar 

  31. Koller, D., Lerner, U., & Angelov, D. (1999). A general algorithm for approximate inference and its application to hybrid Bayes nets. In Proc. of the fifteenth conference on uncertainty in artificial intelligence, Stockholm, Sweden (pp. 324–333).

    Google Scholar 

  32. Kumar, S., & Hebert, M. (2006). Discriminative random fields. International Journal of Computer Vision, 68(2), 179–201.

    Article  Google Scholar 

  33. Lafferty, J., McCallum, A., & Pereira, F. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proc. 18th international conf. on machine learning, Williamstown, MA, USA (pp. 282–289).

    Google Scholar 

  34. Li, Y., & Huttenlocher, D. P. (2008). Learning for optical flow using stochastic optimization. In ECCV ’08: proceedings of the 10th European conference on computer vision, Marseille, France (pp. 379–391).

    Google Scholar 

  35. Lorette, A., Descombes, X., & Zerubia, J. (2000). Texture analysis through a Markovian modelling and fuzzy classification: Application to urban area extraction from satellite images. International Journal of Computer Vision, 36(3), 221–236.

    Article  Google Scholar 

  36. Lu, W. L., Murphy, K. P., Little, J. J., Sheffer, A., & Fu, H. B. (2009). A hybrid conditional random field for estimating the underlying ground surface from airborne lidar data. IEEE Transactions on Geoscience and Remote Sensing, 47(8), 2913–2922.

    Article  Google Scholar 

  37. Migdal, J., & Grimson, W. E. (2005). Background subtraction using Markov thresholds. In WACV-MOTION ’05: proceedings of the IEEE workshop on motion and video computing (pp. 58–65). Washington: IEEE Computer Society.

    Chapter  Google Scholar 

  38. Mittal, A., & Paragios, N. (2004). Motion-based background subtraction using adaptive kernel density estimation. In CVPR ’04: proc. of the 2004 IEEE conf. on computer vision and pattern recognition (Vol. 2, pp. 302–309).

    Chapter  Google Scholar 

  39. Monnet, A., Mittal, A., Paragios, N., & Visvanathan, R. (2003). Background modeling and subtraction of dynamic scenes. In Proc. of the ninth IEEE int. conf. on computer vision (Vol. 2, pp. 1305–1312).

    Chapter  Google Scholar 

  40. Murphy, K. P. (1999). A variational approximation for Bayesian networks with discrete and continuous latent variables. In Uncertainty in artificial intelligence (Vol. 15, pp. 457–466). San Mateo: Morgan Kaufmann.

    Google Scholar 

  41. Parag, T., Elgammal, A., & Mittal, A. (2006). A framework for feature selection for background subtraction. In CVPR ’06: proc. of the IEEE conf. on computer vision and pattern recognition, Washington, DC, USA (pp. 1916–1923).

    Google Scholar 

  42. Pieczynski, W., & Tebbache, A. (2000). Pairwise Markov random fields and segmentation of textured images. In Machine graphics and vision (pp. 705–718).

    Google Scholar 

  43. Salzenstein, F., & Collet, C. (2006). Fuzzy Markov random fields versus chains for multispectral image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(11), 1753–1767.

    Article  Google Scholar 

  44. Salzenstein, F., & Pieczynski, W. (1997). Parameter estimation in hidden fuzzy Markov random fields and image segmentation. Graphical Models and Image Processing, 59(4), 205–220.

    Article  Google Scholar 

  45. Sheikh, Y., & Shah, M. (2005). Bayesian modeling of dynamic scenes for object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(11), 1778–1792.

    Article  Google Scholar 

  46. Stauffer, C., & Grimson, W. E. L. (2000). Learning patterns of activity using real-time tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8), 747–757.

    Article  Google Scholar 

  47. Sun, J., Zhang, W., Tang, X., & Shum, H. (2006). Background cut. In Proc. European conf. comp. vision, ECCV 2006 (pp. 628–641).

    Chapter  Google Scholar 

  48. Veit, Th., Cao, F., & Bouthemy, P. (2006). An a contrario decision framework for region-based motion detection. International Journal of Computer Vision, 68(2), 163–178.

    Article  Google Scholar 

  49. Wang, T., Li, J., Diao, Q., Hu, W., Zhang, Y., & Dulong, C. (2006). Semantic event detection using conditional random fields. In CVPRW ’06: proceedings of the conference on computer vision and pattern recognition workshop, Washington, DC, USA.

    Google Scholar 

  50. Wren, C. R., Azarbayejani, A., Darrell, T., & Pentland, A. P. (1997). Pfinder: real-time tracking of the human body. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7), 780–785.

    Article  Google Scholar 

  51. Wright, J., Ganesh, A., Rao, S., & Ma, Y. (2009). Robust principal component analysis: exact recovery of corrupted low-rank matrices. In NIPS 2009. 0905.0233.

    Google Scholar 

  52. Wu, J., & Chung, A. C. S. (2007). A segmentation model using compound Markov random fields based on a boundary model. IEEE Transactions on Image Processing, 16(1), 241–252.

    MathSciNet  Article  Google Scholar 

  53. Zivkovic, Z., & van der Heijden, F. (2004). Recursive unsupervised learning of finite mixture models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(5), 651–656.

    Article  Google Scholar 

  54. Zivkovic, Z., & van der Heijden, F. (2006). Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recognition Letters, 27(7), 773–780.

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Tomás Crivelli.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Crivelli, T., Bouthemy, P., Cernuschi-Frías, B. et al. Simultaneous Motion Detection and Background Reconstruction with a Conditional Mixed-State Markov Random Field. Int J Comput Vis 94, 295–316 (2011). https://doi.org/10.1007/s11263-011-0429-z

Download citation

Keywords

  • Motion detection
  • Background reconstruction
  • Mixed-state Markov models
  • Conditional random fields