Skip to main content
SpringerLink
Log in

Markov Chains and Monte Carlo Markov Chains

  • Chapter
  • First Online:
Book cover Statistics and Analysis of Scientific Data

Part of the book series: Graduate Texts in Physics ((GTP))

  • 4953 Accesses

Abstract

The theory of Markov chains is rooted in the work of Russian mathematician Andrey Markov, and has an extensive body of literature to establish its mathematical foundations. The availability of computing resources has recently made it possible to use Markov chains to analyze a variety of scientific data, and Monte Carlo Markov chains are now one of the most popular methods of data analysis. The modern-day data analyst will find that Monte Carlo Markov chains are an essential tool that permits tasks that are simply not possible with other methods, such as the simultaneous estimate of parameters for multi-parametric models of virtually any level of complexity. This chapter starts with an introduction to the mathematical properties of Markov chains necessary to understand its implementation as a Monte Carlo Markov chains. The second part is devoted to a description of the implementation of Monte Carlo Markov chains, including the Metropolis-Hasting algorithm and a few tests of convergence.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abramowitz, M., Stegun, I.A.: Handbook of Mathematical Functions: With Formulas, Graphs, and Mathematical Tables. Dover, New York (1970)

    Google Scholar 

  2. Akritas, M.G., Bershady, M.A.: Linear regression for astronomical data with measurement errors and intrinsic scatter. Astrophys. J. 470, 706 (1996). doi:10.1086/177901

    Article  ADS  Google Scholar 

  3. Bayes, T., Price, R.: An essay towards solving a problem in the doctrine of chances. Philos. Trans. R. Soc. Lond. 53 (1763)

    Google Scholar 

  4. Bonamente, M., Swartz, D.A., Weisskopf, M.C., Murray, S.S.: Swift XRT observations of the possible dark galaxy VIRGOHI 21. Astrophys. J. Lett. 686, L71–L74 (2008). doi:10.1086/592819

    Article  ADS  Google Scholar 

  5. Bonamente, M., Hasler, N., Bulbul, E., Carlstrom, J.E., Culverhouse, T.L., Gralla, M., Greer, C., Hawkins, D., Hennessy, R., Joy, M., Kolodziejczak, J., Lamb, J.W., Landry, D., Leitch, E.M., Marrone, D.P., Miller, A., Mroczkowski, T., Muchovej, S., Plagge, T., Pryke, C., Sharp, M., Woody, D.: Comparison of pressure profiles of massive relaxed galaxy clusters using the Sunyaev–Zel’dovich and x-ray data. N. J. Phys. 14(2), 025010 (2012). doi:10.1088/1367-2630/14/2/025010

    Article  Google Scholar 

  6. Brooks, S.P., Gelman, A.: General methods for monitoring convergence of iterative simulations. J. Comput. Graph. Stat. 7, 434–455 (1998)

    MathSciNet  Google Scholar 

  7. Bulmer, M.G.: Principles of Statistics. Dover, New York (1967)

    MATH  Google Scholar 

  8. Carlin, B., Gelfand, A., Smith, A.: Hierarchical Bayesian analysis for changepoint problems. Appl. Stat. 41, 389–405 (1992)

    Article  MATH  Google Scholar 

  9. Cash, W.: Parameter estimation in astronomy through application of the likelihood ratio. Astrophys. J. 228, 939–947 (1979). doi:10.1086/156922

    Article  ADS  Google Scholar 

  10. Cramer, H.: Mathematical Methods of Statistics. Princeton University Press, Princeton (1946)

    MATH  Google Scholar 

  11. Cowan, G.: Statistical Data Analysis, Oxford University Press (1998)

    Google Scholar 

  12. Emslie, A.G., Massone, A.M.: Bayesian confidence limits ETC ETC. ArXiv e-prints (2012)

    Google Scholar 

  13. Fisher, R.A.: On a distribution yielding the error functions of several well known statistics. Proc. Int. Congr. Math. 2, 805–813 (1924)

    Google Scholar 

  14. Gamerman, D.: Markov Chain Monte Carlo. Chapman and Hall CRC, London/New York (1997)

    MATH  Google Scholar 

  15. Gehrels, N.: Confidence limits for small numbers of events in astrophysical data. Astrophys. J. 303, 336–346 (1986). doi:10.1086/164079

    Article  ADS  Google Scholar 

  16. Gelman, A., Rubin, D.: Inference from iterative simulation using multiple sequences. Stat. Sci. 7, 457–511 (1992)

    Article  Google Scholar 

  17. Gosset, W.S.: The probable error of a mean. Biometrika 6, 1–25 (1908)

    Google Scholar 

  18. Hastings, W.K.: Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57(1), 97–109 (1970). doi:10.1093/biomet/57.1.97. http://biomet.oxfordjournals.org/content/57/1/97.abstract

  19. Helmert, F.R.: Die genauigkeit der formel von peters zur berechnung des wahrscheinlichen fehlers director beobachtungen gleicher genauigkeit. Astron. Nachr. 88, 192–218 (1876)

    Article  Google Scholar 

  20. Hubble, E., Humason, M.: The velocity-distance relation among extra-galactic nebulae. Astrophys. J. 74, 43 (1931)

    Article  ADS  Google Scholar 

  21. Jeffreys, H.: Theory of Probability. Oxford University Press, London (1939)

    Google Scholar 

  22. Kelly, B.C.: Some aspects of measurement error in linear regression of astronomical data. Astrophys. J. 665, 1489–1506 (2007). doi:10.1086/519947

    Article  ADS  Google Scholar 

  23. Kolmogorov, A.: Sulla determinazione empirica di una legge di distribuzione. Giornale dell’ Istituto Italiano degli Attuari 4, 1–11 (1933)

    MATH  Google Scholar 

  24. Kolmogorov, A.N.: Foundations of the Theory of Probability. Chelsea, New York (1950)

    Google Scholar 

  25. Lampton, M., Margon, B., Bowyer, S.: Parameter estimation in X-ray astronomy. Astrophys. J. 208, 177–190 (1976). doi:10.1086/154592

    Article  ADS  Google Scholar 

  26. Lewis, S.: gibbsit. http://lib.stat.cmu.edu/S/gibbsit

  27. Marsaglia, G., Tsang, W., Wang, J.: Evaluating kolmogorov’s distribution. J. Stat. Softw. 8, 1–4 (2003)

    Google Scholar 

  28. Mendel, G.: Versuche über plflanzenhybriden (experiments in plant hybridization). Verhandlungen des naturforschenden Vereines in Brünn pp. 3–47 (1865)

    Google Scholar 

  29. Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21, 1087–1092 (1953). doi:10.1063/1.1699114

    Article  ADS  Google Scholar 

  30. Pearson, K., Lee, A.: On the laws on inheritance in men. Biometrika 2, 357–462 (1903)

    Google Scholar 

  31. Plummer, M., Best, N., Cowles, K., Vines, K.: CODA: convergence diagnosis and output analysis for MCMC. R News 6(1), 7–11 (2006). http://CRAN.R-project.org/doc/Rnews/

  32. Press, W., Teukolski, S., Vetterling, W., Flannery, B.: Numerical Recipes 3rd Edition: The Art of Scientific Computing. Cambridge University Press, Cambridge/New York (2007)

    Google Scholar 

  33. Protassov, R., van Dyk, D.A., Connors, A., Kashyap, V.L., Siemiginowska, A.: Statistics, handle with care: detecting multiple model components with the likelihood ratio test. Astrophys. J. 571, 545–559 (2002). doi:10.1086/339856

    Article  ADS  Google Scholar 

  34. Raftery, A., Lewis, S.: How many iterations in the gibbs sampler? Bayesian Stat. 4, 763–773 (1992)

    Google Scholar 

  35. Ross, S.M.: Introduction to Probability Models. Academic, San Diego (2003)

    MATH  Google Scholar 

  36. Thomson, J.J.: Cathode rays. Philos. Mag. 44, 293 (1897)

    Google Scholar 

  37. Tremaine, S., Gebhardt, K., Bender, R., Bower, G., Dressler, A., Faber, S.M., Filippenko, A.V., Green, R., Grillmair, C., Ho, L.C., Kormendy, J., Lauer, T.R., Magorrian, J., Pinkney, J., Richstone, D.: The slope of the black hole mass versus velocity dispersion correlation. Astrophys. J. 574, 740–753 (2002). doi:10.1086/341002

    Article  ADS  Google Scholar 

  38. Wilks, S.S.: Mathematical Statistics. Princeton University Press, Princeton (1943)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Physics, University of Alabama, Huntsville, AL, USA

    Massimiliano Bonamente

Authors
  1. Massimiliano Bonamente
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer Science+Busines Media New York

About this chapter

Cite this chapter

Bonamente, M. (2013). Markov Chains and Monte Carlo Markov Chains. In: Statistics and Analysis of Scientific Data. Graduate Texts in Physics. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-7984-0_10

Download citation

  • DOI: https://doi.org/10.1007/978-1-4614-7984-0_10

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4614-7983-3

  • Online ISBN: 978-1-4614-7984-0

  • eBook Packages: Physics and AstronomyPhysics and Astronomy (R0)

Publish with us

Policies and ethics

Search

Navigation