Statistics and Computing

, Volume 16, Issue 2, pp 203–213 | Cite as

Exact and efficient Bayesian inference for multiple changepoint problems

  • Paul Fearnhead
Article

Abstract

We demonstrate how to perform direct simulation from the posterior distribution of a class of multiple changepoint models where the number of changepoints is unknown. The class of models assumes independence between the posterior distribution of the parameters associated with segments of data between successive changepoints. This approach is based on the use of recursions, and is related to work on product partition models. The computational complexity of the approach is quadratic in the number of observations, but an approximate version, which introduces negligible error, and whose computational cost is roughly linear in the number of observations, is also possible. Our approach can be useful, for example within an MCMC algorithm, even when the independence assumptions do not hold. We demonstrate our approach on coal-mining disaster data and on well-log data. Our method can cope with a range of models, and exact simulation from the posterior distribution is possible in a matter of minutes.

Keywords

Bayes factor Forward-backward algorithm Model choice Perfect simulation Reversible jump MCMC Well-log data 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Albert J. H. and Chib S. 1993. Bayes inference via Gibbs sampling of autoregressive time series subject to Markov mean and variance shifts. Journal of Business and Economic Statistics 11: 1–15.Google Scholar
  2. Barry D. and Hartigan J. A. 1992. Product partition models for change point problems. The Annals of Statistics 20: 260–279.MathSciNetMATHGoogle Scholar
  3. Barry D. and Hartigan J. A. 1993. A Bayesian analysis for change point problems. Journal of the American Statistical Society 88: 309–319.MathSciNetMATHGoogle Scholar
  4. Braun J. V. and Muller H. G. 1998. Statistical methods for DNA sequence segmentation. Statistical Science 13: 142–162.CrossRefMATHGoogle Scholar
  5. Braun J. V., Braun R. K., and Muller H. G. 2000. Multiple changepoint fitting via quasilikelihood, with application to DNA sequence segmentation. Biometrika 87: 301–314.CrossRefMathSciNetMATHGoogle Scholar
  6. Brooks S. P., Giudici P., and Roberts G. O. 2003. Efficient construction of reversible jump Markov chain Monte Carlo proposal distributions. Journal of the Royal Statistical Society, series B 65: 3–39.CrossRefMathSciNetMATHGoogle Scholar
  7. Carlin B. P., Gelfand A. E., and Smith A. F. M. 1992. Hierarchical Bayesian analysis of changepoint problems. Applied Statistics 41: 389–405.MATHGoogle Scholar
  8. Carpenter J., Clifford P., and Fearnhead P. 1999. An improved particle filter for non-linear problems. IEE proceedings-Radar, Sonar and Navigation 146: 2–7.CrossRefGoogle Scholar
  9. Chen J. and Gupta A. K. 1997. Testing and locating changepoints with application to stock prices. Journal of the American Statistical Association 92: 739–747.MathSciNetMATHGoogle Scholar
  10. Chib S. 1995. Marginal likelihood from the Gibbs output. Journal of the American Statistical Association 90: 1313–1321.MATHMathSciNetGoogle Scholar
  11. Chib S. 1996. Calculating posterior distributions and modal estimates in Markov mixture models. Journal of Econometrics 75: 79–98.CrossRefMATHMathSciNetGoogle Scholar
  12. Chib S. 1998. Estimation and comparison of multiple change-point models. Journal of Econometrics 86: 221–241.CrossRefMATHMathSciNetGoogle Scholar
  13. Fearnhead P. 2005a. Direct simulation for discrete mixture distributions. Statistics and Computing 15: 125–133.CrossRefMathSciNetGoogle Scholar
  14. Fearnhead P. 2005b. Exact Bayesian curve fitting and signal segmentation. IEEE Transactions on Signal Processing 53: 2160–2166.CrossRefMathSciNetGoogle Scholar
  15. Fearnhead P. and Clifford P. 2003. Online inference for hidden Markov models. Journal of the Royal Statistical Society, Series B 65: 887–899.CrossRefMathSciNetMATHGoogle Scholar
  16. Fearnhead P. and Liu Z. 2005. Online inference for multiple changepoint problems. Submitted Available from http://www.maths.lancs.ac.uk/~fearnhea/publications.
  17. Fearnhead P. and Meligkotsidou L. 2004. Exact filtering for partially-observed continuous-time Markov models. Journal of the Royal Statistical Society, series B 66: 771–789.CrossRefMathSciNetMATHGoogle Scholar
  18. Green P. 1995. Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82: 711–732.CrossRefMATHMathSciNetGoogle Scholar
  19. Green P. J. 2003. Transdimensional Markov chain Monte Carlo. In: Highly Structured Stochastic Systems (eds. Green P. J., Hjort N. L., and Richardson S.), Oxford University Press.Google Scholar
  20. Hartigan J. A. 1990. Partition models. Communications in Statistics 19: 2745–2756.MathSciNetCrossRefGoogle Scholar
  21. Harvey A. C. 1989. Forecasting, stuctural time series and the Kalman filter. Cambridge University Press, Cambridge, UK.Google Scholar
  22. Jarrett R. G. 1979. A note on the intervals between coal-mining disasters. Biometrika 66: 191–3.CrossRefGoogle Scholar
  23. Johnson T. D., Elashoff R. M., and Harkema S. J. 2003. A Bayesian change-point analysis of electromyographic data: detecting muscle activation patterns and associated applications. Biostatistics 4: 143–164.CrossRefMATHGoogle Scholar
  24. Lavielle M. and Lebarbier E. 2001. An application of MCMC methods for the multiple change-points problem. Signal Processing 81: 39–53.CrossRefMATHGoogle Scholar
  25. Liu J. S. 2001. Monte Carlo strategies in scientific computing. New York: Springer.MATHGoogle Scholar
  26. Liu J. S. and Lawrence C. E. 1999. Bayesian inference on biopolymer models. Bioinformatics 15: 38–52.CrossRefGoogle Scholar
  27. Lund R. and Reeves J. 2002. Detection of undocumented changepoints: A revision of the two-phase regression model. Journal of Climate 15: 2547–2554.CrossRefGoogle Scholar
  28. ó Ruanaidh J. J. K. and Fitzgerald W. J. 1996. Numerical Bayesion Methods Applied to Signal Processing. New York: Springer.Google Scholar
  29. Pievatolo A. and Green P. J. 1998. Boundary detection through dynamic polygons. Journal of the Royal Statistical Society, Series B 60: 609–626.CrossRefMathSciNetMATHGoogle Scholar
  30. Propp J. G. and Wilson D. B. 1996. Exact sampling with coupled Markov chains and applications to statistical mechanics. Random Structures and Algorithms 9: 223–252.CrossRefMathSciNetMATHGoogle Scholar
  31. Punskaya E., Andrieu C., Doucet A., and Fitzgerald W. J. 2002. Bayesian curve fitting using MCMC with applications to signal segmentation. IEEE Transactions on Signal Processing 50: 747–758.CrossRefGoogle Scholar
  32. Raftery A. E. and Akman V. E. 1986. Bayesian analysis of a Poisson process with a change-point. Biometrika 73: 85–89.CrossRefMathSciNetGoogle Scholar
  33. Ritov Y., Raz A., and Bergman H. 2002. Detection of onset of neuronal activity by allowing for heterogeneity in the change points. Journal of Neuroscience Methods 122: 25–42.CrossRefGoogle Scholar
  34. Scott S. L. 2002. Bayesian methods for hidden Markov models: Recursive computing in the 21st century. Journal of the American Statistical Association 97: 337–351.CrossRefMATHMathSciNetGoogle Scholar
  35. Stephens D. A. 1994. Bayesian retrospective multiple-changepoint identification. Applied Statistics 43: 159–178.MATHGoogle Scholar
  36. Worsley K. J. 1979. On the likelihood ratio test for a shift in location of normal populations. Journal of the American Statistical Association 74: 363–367.MathSciNetGoogle Scholar
  37. Yang T. Y. and Kuo L. 2001. Bayesian binary segmentation procedure for a Poisson process with multiple changepoints. Journal of Computational and Graphical Statistics 10: 772–785.CrossRefMathSciNetGoogle Scholar
  38. Yao Y. 1984. Estimation of a noisy discrete-time step function: Bayes and empirical Bayes approaches. The Annals of Statistics 12: 1434–1447.MATHMathSciNetGoogle Scholar

Copyright information

© Springer Science + Business Media, LLC 2006

Authors and Affiliations

  • Paul Fearnhead
    • 1
  1. 1.Department of Mathematics and StatisticsLancaster UniversityLancaster

Personalised recommendations