TEST

pp 1–24 | Cite as

Modelling covariance matrices by the trigonometric separation strategy with application to hidden Markov models

Original Paper
  • 13 Downloads

Abstract

Bayesian inference on the covariance matrix is usually performed after placing an inverse-Wishart or a multivariate Jeffreys as a prior density, but both of them, for different reasons, present some drawbacks. As an alternative, the covariance matrix can be modelled by separating out the standard deviations and the correlations. This separation strategy takes advantage of the fact that usually it is more straightforward and flexible to set priors on the standard deviations and the correlations rather than on the covariance matrix. On the other hand, the priors must preserve the positive definiteness of the correlation matrix. This can be obtained by considering the Cholesky decomposition of the correlation matrix, whose entries are reparameterized using trigonometric functions. The efficiency of the trigonometric separation strategy (TSS) is shown through an application to hidden Markov models (HMMs), with conditional distributions multivariate normal. In the case of an unknown number of hidden states, estimation is conducted using a reversible jump Markov chain Monte Carlo algorithm based on the split-and-combine and birth-and-death moves whose design is straightforward because of the use of the TSS. Finally, an example in remote sensing is described, where a HMM containing the TSS is used for the segmentation of a multi-colour satellite image.

Keywords

Cholesky decomposition Image segmentation Induced priors Multi-colour satellite images Multivariate normal distribution Peano–Hilbert curve 

Mathematics Subject Classification

62-XX 62FXX 62MXX 

Notes

Acknowledgements

This research was funded by the Scottish Government’s Rural and Environment Science and Analytical Services Division. The images in Fig. 6 were kindly produced by Laura Origgi. The satellite image was provided by Carlos Padovani. A discussion with Laura Poggio was useful to understand a few problems related to multispectral sensors. Comments from Mark Brewer, Glenn Marion, and two anonymous referees improved the quality of the final paper.

References

  1. Barnard J, McCulloch R, Meng X-L (2000) Modeling covariance matrices in terms of standard deviations and correlations, with application to shrinkage. Stat Sin 10:1281–1311MathSciNetMATHGoogle Scholar
  2. Cappé O, Moulines E, Rydén T (2005) Inference in hidden Markov models. Springer, New YorkMATHGoogle Scholar
  3. Cappé O, Robert CP, Rydén T (2003) Reversible jump, birth-and-death and more general continuous Markov chain Monte Carlo samplers. J R Stat Soc Ser B 63:679–700MathSciNetCrossRefMATHGoogle Scholar
  4. Celeux G, Hurn M, Robert CP (2000) Computational and differential difficulties with mixture posterior distributions. J Am Stat Assoc 95:957–970CrossRefMATHGoogle Scholar
  5. Daniels MJ, Kass RE (1999) Nonconjugate Bayesian estimation of covariance matrices and its use in hierarchical models. J Am Stat Assoc 94:1254–1263MathSciNetCrossRefMATHGoogle Scholar
  6. Daniels MJ, Pourahmadi M (2002) Bayesian analysis of covariance matrices and dynamic models for longitudinal data. Biometrika 89:553–566MathSciNetCrossRefMATHGoogle Scholar
  7. Daniels MJ, Pourahmadi M (2009) Modeling covariance matrices via partial autocorrelations. J Multivariate Anal 100:2352–2363MathSciNetCrossRefMATHGoogle Scholar
  8. Dellaportas P, Papageorgiou I (2006) Multivariate mixtures of normals with unknown number of components. Stat Comput 16:57–68MathSciNetCrossRefGoogle Scholar
  9. Dellaportas P, Plataniotis A, Titsias MK (2015) Scalable inference for a full multivariate stochastic volatility model. arXiv:1510.05257v1. Accessed 25 Aug 2017
  10. Friel N, Pettitt AN, Reeves R, Wit E (2009) Bayesian inference in hidden Markov random fields for binary data defined on large lattices. J Comput Graph Stat 18:243–261MathSciNetCrossRefGoogle Scholar
  11. Frühwirth-Schnatter S (2001) Markov chain Monte Carlo estimation of classical and dynamic switching and mixture models. J Am Stat Assoc 96:194–209MathSciNetCrossRefMATHGoogle Scholar
  12. Gelman A, Meng X-L (1998) Simulating normalizing constants: from importance sampling to bridge sampling to path sampling. Stat Sci 13:163–185MathSciNetCrossRefMATHGoogle Scholar
  13. Giordana N, Pieczynski W (1997) Estimation of generalised multisensor hidden Markov chains and unsupervised image segmentation. IEEE Trans Pattern Anal Mach Intell 19:465–475CrossRefGoogle Scholar
  14. Green PJ, Richardson S (2002) Hidden Markov models and disease mapping. J Am Stat Assoc 97:1055–1070MathSciNetCrossRefMATHGoogle Scholar
  15. Hamilton JD (1994) Time series analysis. Princeton University Press, PrincetonMATHGoogle Scholar
  16. Hoff PD (2009) A hierarchical eigenmodel for pooled covariance estimation. J R Stat Soc Ser B 71:971–992MathSciNetCrossRefGoogle Scholar
  17. Kamary K, Robert CP (2014) Reflecting about selecting noninformative priors. arXiv:1402.6257v3. Accessed 25 Aug 2017
  18. Kim C-J (1993) Dynamic linear models with Markov-switching. J Econ 60:1–22MathSciNetCrossRefGoogle Scholar
  19. Krolzig H-M (1997) Markov-switching vector autoregressions: modelling, statistical inference and applications to business cycle analysis. Springer, BerlinCrossRefMATHGoogle Scholar
  20. Leonard T, Hsu JST (1992) Bayesian inference for a covariance matrix. Ann Stat 20:1669–1696MathSciNetCrossRefMATHGoogle Scholar
  21. Liechty JC, Liechty MW, Müller P (2004) Bayesian correlation estimation. Biometrika 91:1–14MathSciNetCrossRefMATHGoogle Scholar
  22. Marin JM, Mengersen KL, Robert CP (2005) Bayesian modelling and inference on mixture of distributions. In: Dey D, Rao CR (eds) Handbooks of statistics 25. Elsevier Science, Amsterdam, pp 459–507Google Scholar
  23. Møller J, Pettitt AN, Berthelsen KK, Reeves RW (2006) An efficient Markov chain Monte Carlo method for distributions with intractable normalising constants. Biometrika 93:451–458MathSciNetCrossRefMATHGoogle Scholar
  24. Murray I, Ghahramani Z, MacKay DJC (2006) MCMC for doubly-intractable distributions. In: Dechter R, Richardson T (eds) Proceedings of the twenty-second conference on uncertainty in artificial intelligence. AUAI Press, Arlington, pp 359–366Google Scholar
  25. Paroli R, Spezia L (2010) Reversible jump MCMC methods and segmentation algorithms in hidden Markov models. Aust N Z J Stat 52:151–166MathSciNetCrossRefMATHGoogle Scholar
  26. Pinheiro JC, Bates DM (1996) Unconstrained parameterizations for the variance-covariance matrix. Stat Comput 6:289–296CrossRefGoogle Scholar
  27. Qian W, Titterington DM (1991) Estimation of parameters in hidden Markov models. Philos Trans Roy Soc Lond Ser A 337:407–428CrossRefMATHGoogle Scholar
  28. Richardson S, Green PJ (1997) On Bayesian analysis of mixtures with an unknown number of components (with discussion). J R Stat Soc Ser B 59:731–792CrossRefMATHGoogle Scholar
  29. Scott SL, James GM, Sugar CA (2005) Hidden Markov models for longitudinal comparisons. J Am Stat Assoc 100:359–369MathSciNetCrossRefMATHGoogle Scholar
  30. Seaman JW III, Seaman JW Jr, Stamey JD (2012) Hidden dangers of specifying noninformative priors. Am Stat 66:77–84MathSciNetCrossRefGoogle Scholar
  31. Smith M, Kohn R (2002) Parsimonius covariance matrix estimation for longitudinal data. J Am Stat Assoc 97:1141–1153CrossRefMATHGoogle Scholar
  32. Spezia L (2010) Bayesian analysis of multivariate Gaussian hidden Markov models with an unknown number of regimes. J Time Ser Anal 31:1–11MathSciNetCrossRefMATHGoogle Scholar
  33. Spezia L, Friel N, Gimona A (2017) Spatial hidden Markov models and species distribution. J Appl Stat, published onlineGoogle Scholar
  34. Wang H, Pillai NS (2013) On a class of shrinkage priors for covariance matrix estimation. J Comput Graph Stat 22:689–707MathSciNetCrossRefGoogle Scholar
  35. Yang R, Berger JO (1994) Estimation of a covariance matrix using the reference prior. Ann Stat 22:1195–1211MathSciNetCrossRefMATHGoogle Scholar
  36. Zucchini W, MacDonald IA, Langrock R (2016) Hidden Markov models for time series: an introduction using R, 2nd edn. Chapman & Hall/CRC Press, Boca RatonMATHGoogle Scholar

Copyright information

© Sociedad de Estadística e Investigación Operativa 2018

Authors and Affiliations

  1. 1.Biomathematics & Statistics ScotlandAberdeenUK

Personalised recommendations