Prior specification of neighbourhood and interaction structure in binary Markov random fields

Abstract

We formulate a prior distribution for the energy function of stationary binary Markov random fields (MRFs) defined on a rectangular lattice. In the prior we assign distributions to all parts of the energy function. In particular we define priors for the neighbourhood structure of the MRF, what interactions to include in the model, and for potential values. We define a reversible jump Markov chain Monte Carlo (RJMCMC) procedure to simulate from the corresponding posterior distribution when conditioned to an observed scene. Thereby we are able to learn both the neighbourhood structure and the parametric form of the MRF from the observed scene. We circumvent evaluations of the intractable normalising constant of the MRF when running the RJMCMC algorithm by adopting a previously defined approximate auxiliary variable algorithm. We demonstrate the usefulness of our prior in two simulation examples and one real data example.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

References

  1. Arnesen, P., Tjelmeland, H.: Fully Bayesian binary Markov random field models: prior specification and posterior simulation. Scand. J. Stat. 42, 967–987 (2015)

    MathSciNet  Article  MATH  Google Scholar 

  2. Augustin, N.H., Mugglestone, M.A., Buckland, S.T.: An autologistic model for the spatial distribution of wildlife. J. Appl. Ecol. 33, 339–347 (1996)

    Article  Google Scholar 

  3. Austad, H.M.: Approximations of binary Markov random fields, Ph.D. thesis, Norwegian University of Science and Technology. Thesis number 292:2011. http://urn.kb.se/resolve?urn=urn:nbn:no:ntnu:diva-14922 (2011)

  4. Austad, H.M., Tjelmeland, H.: Approximate computations for binary Markov random fields and their use in Bayesian models. Technical report, Submitted (2016)

  5. Besag, J.: Spatial interaction and the statistical analysis of lattice systems. J. R. Stat. Soc. Ser. B 36, 192–236 (1974)

    MathSciNet  MATH  Google Scholar 

  6. Besag, J.: Statistical analysis of non-lattice data. The Statistician 24, 179–195 (1975)

    Article  Google Scholar 

  7. Besag, J.: On the statistical analysis of dirty pictures. J. R. Stat. Soc. Ser. B 48, 259–302 (1986)

    MathSciNet  MATH  Google Scholar 

  8. Buckland, S.T., Elston, D.A.: Empirical models for the spatial distribution of wildlife. J. Appl. Ecol. 30, 478–495 (1993)

    Article  Google Scholar 

  9. Caimo, A., Friel, N.: Bayesian inference for exponential random graph models. Soc. Netw. 33, 41–55 (2011)

    Article  Google Scholar 

  10. Celeux, G., Hurn, M.A., Robert, C.P.: Computational and inferential difficulties with mixture posterior distributions. J. Am. Stat. Assoc. 95, 957–970 (2000)

    MathSciNet  Article  MATH  Google Scholar 

  11. Clifford, P.: Markov random fields in statistics. In: Grimmett, G., Welsh, D.J. (eds.) Disorder in Physical Systems, A Volume in Honour of John M. Hammersley. Oxford University Press, New York (1990)

    Google Scholar 

  12. Cowles, M.K., Carlin, B.P.: Markov chain Monte Carlo convergence diagnostics: a comparative review. J. Am. Stat. Assoc. 91, 883–904 (1996)

    MathSciNet  Article  MATH  Google Scholar 

  13. Cressie, N.A.: Statistics for Spatial Data, 2nd edn. Wiley, New York (1993)

    Google Scholar 

  14. Cressie, N., Davidson, J.L.: Image analysis with partially ordered Markov models. Comput. Stat. Data Anal. 29, 1–26 (1998)

    MathSciNet  Article  MATH  Google Scholar 

  15. Descombes, X., Mangin, J., Pechersky, E. and Sigelle, M.: Fine structures preserving Markov model for image processing. In: Proceedings of 9th SCIA 95, Uppsala, pp. 349–356 (1995)

  16. Everitt, R.G.: Bayesian parameter estimation for latent Markov random fields and social networks. J. Comput. Graph. Stat. 21, 940–960 (2012)

    MathSciNet  Article  Google Scholar 

  17. Friel, N., Rue, H.: Recursive computing and simulation-free inference for general factorizable models. Biometrika 94(3), 661–672 (2007)

    MathSciNet  Article  MATH  Google Scholar 

  18. Friel, N., Pettitt, A.N., Reeves, R., Wit, E.: Bayesian inference in hidden Markov random fields for binary data defined on large lattices. J. Comput. Graph. Stat. 18(2), 243–261 (2009)

    MathSciNet  Article  Google Scholar 

  19. Geyer, C.J., Thompson, E.A.: Constrained Monte Carlo maximum likelihood for dependent data. J. R. Stat. Soc. Ser. B 54(3), 657–699 (1992)

    MathSciNet  Google Scholar 

  20. Grabisch, M., Marichal, L.-L., Roubens, M.: Equivalent representation of set function. Math. Oper. Res. 25, 157–178 (2000)

    MathSciNet  Article  MATH  Google Scholar 

  21. Graham, R.L., Knuth, D.E., Patashnik, O.: Concrete Mathematics, 2nd edn. Addison-Wesley, Reading, MA (1988)

    Google Scholar 

  22. Hammer, P., Holzman, R.: Approximations of pseudo-Boolean functions; application to game theory. Methods Models Oper. Res. 36, 3–21 (1992)

    MathSciNet  Article  MATH  Google Scholar 

  23. Heikkinen, J., Högmander, H.: Fully Bayesian approach to image restoration with an application in biogeography. Appl. Stat. 43, 569–582 (1994)

    Article  MATH  Google Scholar 

  24. Higdon, D.M., Bowsher, J.E., Johnsen, V.E., Turkington, T.G., Gilland, D.R., Jaszczak, R.J.: Fully Bayesian estimation of Gibbs hyperparameters for emission computed tomography data. IEEE Trans. Med. Imaging 16, 516–526 (1997)

    Article  Google Scholar 

  25. Hurn, M., Husby, O. and Rue, H.: .A tutorial on image analysis. In: Møller J. (ed.), Spatial Statistics and Computational Methods. Lecture Notes in Statistics, vol. 173, pp. 87–141, Springer, New York (2003)

  26. Kindermann, R., Snell, J.L.: Markov Random Fields and Their Applications. American Mathematical Society, Providence, Rhode Island (1980)

    Google Scholar 

  27. Lauritzen, S.L.: Graphical Models. Clarendon Press, Oxford (1996)

    Google Scholar 

  28. Lyne, A.M., Girolami, M., Atchade, Y., Strathmann, H., Simpson, D.: On Russian roulette estimates for Bayesian inference with doubly-intractable likelihoods. Stat. Sci. 30, 443–467 (2015)

    MathSciNet  Article  Google Scholar 

  29. McGrory, C.A., Pettitt, A.N., Reeves, R., Griffin, M., Dwyer, M.: Variational Bayes and the reduced dependence approximation for the autologistic model on an irregular grid with applications. J. Comput. Graph. Stat. 21, 781–796 (2012)

    MathSciNet  Article  Google Scholar 

  30. Møller, J., Pettitt, A.N., Reeves, R., Berthelsen, K.K.: An efficient Markov chain Monte Carlo method for distributions with intractable normalising constants. Biometrika 93, 451–458 (2006)

    MathSciNet  Article  MATH  Google Scholar 

  31. Murray, I., Ghahramani, Z. and MacKay, D.: MCMC for doubly-intractable distributions. In: Proceedings of the Twenty-Second Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-06), pp. 359–366, AUAI Press, Arlington, Virginia (2006)

  32. Propp, J.G., Wilson, D.B.: Exact sampling with coupled Markov chains and applications to statistical mechanics. Random Struct. Algorithms 9, 223–252 (1996)

    MathSciNet  Article  MATH  Google Scholar 

  33. Qian, W., Titterington, D.M.: Multidimensional Markov chain models for image textures. J. R. Stat. Soc. Ser. B 53(3), 661–674 (1991)

    MathSciNet  MATH  Google Scholar 

  34. Tjelmeland, H., Austad, H.M.: Exact and approximate recursive calculations for binary Markov random fields defined on graphs. J. Comput. Graph. Stat. 21, 758–780 (2012)

    MathSciNet  Article  Google Scholar 

  35. Tjelmeland, H., Besag, J.: Markov random fields with higher order interactions. Scand. J. Stat. 25, 415–433 (1998)

    MathSciNet  Article  MATH  Google Scholar 

  36. Toftaker, H.: Modelling and parameter estimation for discrete random fields and spatial point processes, PhD thesis, Norwegian University of Science and Technology. Thesis number 200:2013. http://ntnu.diva-portal.org/smash/record.jsf?pid=diva2:662513 (2013)

  37. Walker, S.: Posterior sampling when the normalizing constant is unknown. Commun. Stat. Simul. Comput. 40, 784–792 (2011)

    MathSciNet  Article  MATH  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Håkon Tjelmeland.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 428 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Arnesen, P., Tjelmeland, H. Prior specification of neighbourhood and interaction structure in binary Markov random fields. Stat Comput 27, 737–756 (2017). https://doi.org/10.1007/s11222-016-9650-5

Download citation

Keywords

  • Auxiliary variables
  • Fully Bayesian model
  • Ising model
  • Markov random fields
  • Reversible jump MCMC