Exact Bayesian inference for off-line change-point detection in tree-structured graphical models

Abstract

We consider the problem of change-point detection in multivariate time-series. The multivariate distribution of the observations is supposed to follow a graphical model, whose graph and parameters are affected by abrupt changes throughout time. We demonstrate that it is possible to perform exact Bayesian inference whenever one considers a simple class of undirected graphs called spanning trees as possible structures. We are then able to integrate on the graph and segmentation spaces at the same time by combining classical dynamic programming with algebraic results pertaining to spanning trees. In particular, we show that quantities such as posterior distributions for change-points or posterior edge probabilities over time can efficiently be obtained. We illustrate our results on both synthetic and experimental data arising from biology and neuroscience.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

References

  1. Arbeitman, M.N., Furlong, E.E.M., Imam, F., Johnson, E., Null, B.H., Baker, B.S., Krasnow, Ma., Scott, M.P., Davis, R.W., White, K.P.: Gene expression during the life cycle of Drosophila melanogaster. Science 297(5590), 2270–2275 (2002). doi:10.1126/science.1072152. (ISSN: 1095-9203)

    Article  Google Scholar 

  2. Auger, I.E., Lawrence, C.E.: Algorithms for the optimal identification of segment neighborhoods. Bull. Math. Biol. 51(1), 39–54 (1989). doi:10.1007/BF02458835. (ISSN: 00928240)

    MathSciNet  Article  MATH  Google Scholar 

  3. Barber, D., Cemgil, A.: Graphical models for time-series. IEEE Signal Process. Mag. (2010). doi:10.1109/MSP.2010.938028 (ISSN: 1053-5888)

  4. Barry, D., Hartigan, J.A.: Product partition models for change point problems. Ann. Stat. 20(1), 260–279 (1992). doi:10.1214/aos/1176348521. (ISSN: 0090-5364)

    MathSciNet  Article  MATH  Google Scholar 

  5. Caron, F., Doucet, A., Gottardo, R.: On-line changepoint detection and parameter estimation with application to genomic data. Stat. Comput. 22(2), 579–595 (2012). doi:10.1007/s11222-011-9248-x. (ISSN: 09603174)

    MathSciNet  Article  MATH  Google Scholar 

  6. Cribben, I., Haraldsdottir, R., Atlas, L.Y., Wager, T.D., Lindquist, M.A.: Dynamic connectivity regression: determining state-related changes in brain connectivity. NeuroImage 61(4), 907–920 (2012). doi:10.1016/j.neuroimage.2012.03.070. (ISSN: 10538119)

    Article  Google Scholar 

  7. Dondelinger, F., Lèbre, S., Husmeier, D.: Non-homogeneous dynamic Bayesian networks with Bayesian regularization for inferring gene regulatory networks with gradually time-varying structure. Mach. Learn. 90(2), 191–230 (2013). (ISSN: 08856125)

    MathSciNet  Article  MATH  Google Scholar 

  8. Fearnhead, P.: Exact and efficient Bayesian inference for multiple changepoint problems. Stat. Comput. 16(2), 203–213 (2006). doi:10.1007/s11222-006-8450-8. (ISSN: 09603174)

    MathSciNet  Article  Google Scholar 

  9. Fearnhead, P., Liu, Z.: On-line inference for multiple changepoint problems. J. R. Stat. Soc. Ser B Stat. Methodol. 69(4), 589–605 (2007). doi:10.1111/j.1467-9868.2007.00601.x. (ISSN: 13697412)

    MathSciNet  Article  Google Scholar 

  10. Fox, E., Sudderth, E., Jordan, M., Willsky, A.: Nonparametric Bayesian learning of switching linear dynamical systems. Adv. Neural Inf. Process. Syst. 21, 457–464 (2009)

    Google Scholar 

  11. Friedman, N., Murphy, K., Russell, S.: Learning the structure of dynamic probabilistic networks. In: Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, pp. 139–147, (1998). doi:10.1111/j.1469-7580.2008.00962.x

  12. Grzegorczyk, M., Husmeier, D.: Improvements in the reconstruction of time-varying gene regulatory networks: dynamic programming and regularization by information sharing among genes. Bioinformatics 27(5), 693–699 (2011). doi:10.1093/bioinformatics/btq711. (ISSN: 1367-4803)

    Article  Google Scholar 

  13. Kolar, M., Song, L., Ahmed, A., Xing, E.P.: Estimating time-varying networks, 4. (2010). doi:10.1214/09-AOAS308 (ISBN: 0001409107)

  14. Lauritzen, S.L.: Graphical Models. Oxford University Press, Oxford (1996). (ISBN: 0-19-852219-3)

  15. Lèbre, S., Becq, J., Devaux, F., Stumpf, M.P.H., Lelandais, G.: Statistical inference of the time-varying structure of gene-regulation networks. BMC Syst. Biol. 4, 130 (2010). doi:10.1186/1752-0509-4-130. (ISSN: 1752-0509)

    Article  Google Scholar 

  16. Meilă, M., Jaakkola, T.: Tractable bayesian learning of tree belief networks. Stat. Comput. 16(1), 77–92 (2006)

    MathSciNet  Article  Google Scholar 

  17. Murphy, K., Mian, S.: Modelling Gene Expression Data Using Dynamic Bayesian Networks. Technical report (1999)

  18. Rigaill, G., Lebarbier, E., Robin, S.: Exact posterior distributions and model selection criteria for multiple change-point detection problems. Stat. Comput. 22(4), 917–929 (2012). doi:10.1007/s11222-011-9258-8. (ISSN: 0960-3174)

    MathSciNet  Article  MATH  Google Scholar 

  19. Robinson, J., Hartemink, A.: Learning non-stationary dynamic Bayesian networks. J. Mach. Learn. Res. 11, 3647–3680 (2010). (ISSN: 1532-4435)

    MathSciNet  MATH  Google Scholar 

  20. Schwaller, L., Robin, S., Stumpf, M.: Bayesian Inference of Graphical Model Structures Using Trees. http://arxiv.org/abs/1504.02723 (2015)

  21. Siracusa, M.R.: Tractable Bayesian inference of time-series dependence structure. In: Proceedings of the 23th International Conference on Artificial Intelligence and Statistics, vol. 5, pp. 528–535. (2009)

  22. Xuan, X., Murphy, K.: Modeling changing dependency structure in multivariate time series. In: Proceedings of the 24th International Conference on Machine Learning (2007), vol. 227, issue id m, pp. 1055–1062, (2007). doi:10.1145/1273496.1273629 (ISSN: 1595937935)

  23. Zhao, W., Serpedin, E., Dougherty, E.R.: Inferring gene regulatory networks from time series data using the minimum description length principle. Bioinformatics 22(17), 2129–2135 (2006). doi:10.1093/bioinformatics/btl364. (ISSN: 13674803)

    Article  Google Scholar 

  24. Zhou, S., Lafferty, J., Wasserman, L.: Time varying undirected graphs. Mach. Learn. 80, 295–319 (2010)

    MathSciNet  Article  Google Scholar 

Download references

Acknowledgments

The authors thank Ivor Cribben (Alberta School of Business, Canada) for kindly providing the fMRI data. They also thank Sarah Ouadah (AgroParisTech, INRA, Paris, France) for fruitful discussions.

Author information

Affiliations

Authors

Corresponding author

Correspondence to L. Schwaller.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 722 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Schwaller, L., Robin, S. Exact Bayesian inference for off-line change-point detection in tree-structured graphical models. Stat Comput 27, 1331–1345 (2017). https://doi.org/10.1007/s11222-016-9689-3

Download citation

Keywords

  • Change-point detection
  • Exact Bayesian inference
  • Graphical model
  • Multivariate time-series
  • Spanning tree