Advertisement

Statistics and Computing

, Volume 26, Issue 4, pp 899–916 | Cite as

Computation of Gaussian orthant probabilities in high dimension

  • James RidgwayEmail author
Article

Abstract

We study the computation of Gaussian orthant probabilities, i.e. the probability that a Gaussian variable falls inside a quadrant. The Geweke–Hajivassiliou–Keane (GHK) algorithm (Geweke, Comput Sci Stat 23:571–578 1991, Keane, Simulation estimation for panel data models with limited dependent variables, 1993, Hajivassiliou, J Econom 72:85–134, 1996, Genz, J Comput Graph Stat 1:141–149, 1992) is currently used for integrals of dimension greater than 10. In this paper, we show that for Markovian covariances GHK can be interpreted as the estimator of the normalizing constant of a state-space model using sequential importance sampling. We show for an AR(1) the variance of the GHK, properly normalized, diverges exponentially fast with the dimension. As an improvement we propose using a particle filter. We then generalize this idea to arbitrary covariance matrices using Sequential Monte Carlo with properly tailored MCMC moves. We show empirically that this can lead to drastic improvements on currently used algorithms. We also extend the framework to orthants of mixture of Gaussians (Student, Cauchy, etc.), and to the simulation of truncated Gaussians.

Keywords

GHK Orthant probability PF SMC 

Notes

Acknowledgments

This work is part of my PhD thesis under the supervision of Nicolas Chopin. I am grateful for his comments.

References

  1. Adler, S.L.: Over-relaxation method for the Monte Carlo evaluation of the partition function for multiquadratic actions. Phys. Rev. D 23(12), 2901 (1981)CrossRefGoogle Scholar
  2. Andrieu, C., Doucet, A., Holenstein, R.: Particle Markov chain Monte Carlo. J. R. Stat. Soc. B 72, 269–342 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  3. Bretz, F., Genz, A., Hothorn, L.A.: On the numerical availability of multiple comparison procedures. Biom. J. 5, 645–656 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  4. Borsch-Supan, A., Hajivassiliou, V., Kotlikoff, L.J.: Health, children and elderly arrangments: a multiperiod multinomial probit model with unobserved heterogeneity and autocorrelated errors. Top. Econ. Aging pp. 79–108 (1992)Google Scholar
  5. Cappé, O., Moulines, E., Ryden, T.: Inference in hidden Markov models. Springer Series in Statistics, New York (2005)zbMATHGoogle Scholar
  6. Carpenter, J., Clifford, P., Fearnhead, P.: Improved particle filter for nonlinear problems. IEE Proc Radar Sonar Navig 146(1), 2–7 (1999)CrossRefGoogle Scholar
  7. Chopin, N.: A sequential particle filter method for static models. Biometrika 89(3), 539–551 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  8. Chopin, N.: Fast simulation of truncated Gaussian distributions. Stat. Comput., 21(2):275–288, 2011. ISSN 0960–3174. doi: 10.1007/s11222-009-9168-1
  9. Chopin, N., Papaspiliopoulos, O., Jacob, P.E.: \(\text{ SMC }^2\): an efficient algorithm for sequential analysis of state space models. J. R. Stat. Soc. B 75(3), 397–426 (2013)MathSciNetCrossRefGoogle Scholar
  10. Christen, J. A., Fox, C., Pérez-Ruiz, D. A., Santana-Cibrian, M.: On optimal direction Gibbs sampling. arXiv preprint arXiv:1205.4062, (2012)
  11. Craig, P.: A new reconstruction of multivariate normal orthant probabilities. J. R. Stat. Soc. 70(1), 227–243 (2008)MathSciNetzbMATHGoogle Scholar
  12. Del Moral, P.: Non-linear filtering: interacting particle resolution. Markov Process. Relat. Fields 2(4), 555–581 (1996)zbMATHGoogle Scholar
  13. Del Moral, P., Doucet, A., Jasra, A.: Sequential Monte Carlo samplers. J. R. Stat. Soc. B 68(3):411–436, (2006). ISSN 1467–9868Google Scholar
  14. Douc, R., Cappe, O., Moulines, E.: Comparison of resampling schemes for particle filtering. In: Proceedings of 4th International Symposium Image and Signal Processing and Analysis ISPA 2005, pp. 64–69 (2005)Google Scholar
  15. Doucet, A., Godsill, S., Andrieu, C.: On sequential Monte Carlo sampling methods for bayesian filtering. Stat. Comput. 10(3), 197–208 (2000)CrossRefGoogle Scholar
  16. Doucet, A., Briers, M., Senecal, S.: Efiicient block sampling strategies for sequential Monte Carlo. J. Comput. Graph. Stat. 15(3), 693–711 (2006)MathSciNetCrossRefGoogle Scholar
  17. Dubarry, C., Douc, R.: Particle approximation improvement of the joint smoothing distribution with on the fly variance estimation. arXiv:1107.5524v1, pp. 1–19, June 2011 (2011)
  18. Flecher, C., Allard, D., Naveau, P.: Truncated skew-normal distributions: Estimation by weighted moments and application to climatic data. Technical Report 39, Institut National de la Recherche Agronomique (2009)Google Scholar
  19. Genz, A.: Numerical computation of multivariate normal probabilities. J. Comput. Graph. Stat. 1(2), 141–149 (1992)Google Scholar
  20. Genz, A., Bretz, F.: Computation of Multivariate Normal and t Probabilities, vol. 195 of Lecture Notes in Statistics. Springer, New York (2009)Google Scholar
  21. Geweke, J.: Efficient simulation from the multivariate normal and student \(t\) distributions subject to linear constraints. Comput. Sci. Stat. 23, 571–578 (1991)Google Scholar
  22. Gibson, G.J., Glasbey, C.A., Elston, D.A.: Monte-carlo evalution of multivariate normal integrals and sensitivity to variate ordering. Adv. Numer. Methods Appl. 14, 120–126 (1994)zbMATHGoogle Scholar
  23. Gordon, N.J., Salmond, D.J., Smith, F.M.: Novel approach to nonlinear/non-gaussian bayesian state estimation. In: IEE Proceedings F (Radar and Signal Processing), IET, vol. 140, pp. 107–113 (1993)Google Scholar
  24. Hajivassiliou, V., McFadden, D., Ruud, P.: Simulation of multivariate normal rectangle probabilities and their derivatives theoretical and computational results. Journal of Econometrics 72(1—-2), 85–134 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  25. Hochberg, Y., Tamhane, A.C.: Multiple Comparison Procedures. Wiley, New York (1987)CrossRefzbMATHGoogle Scholar
  26. Holmes, C., Held, L.: Bayesian auxiliary variable models for binary and multinomial regression. Bayesian Anal. 1(1), 145–168 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  27. Keane, M.: Simulation Estimation for Panel Data Models with Limited Dependent Variables. MPRA Paper 53029. University Library of Munich, Munich (1993)Google Scholar
  28. LeSage, J.P., Pace, R.K., Lam, N., Campanella, R., Liu, X.: New Orleans buisness recovery in the aftermath of huricane Katrina. App. Stat. 174(4), 1007–1027 (2011)MathSciNetGoogle Scholar
  29. Meyn, S., Tweedie, R.L.: Markov Chains and Stochastic Stability, 2nd edn. Cambridge University Press, Cambridge (2009)CrossRefzbMATHGoogle Scholar
  30. Minka, T.: Expectation Propagation for approximate Bayesian inference. In Proc. 17th Conf. Uncertainty Artificial Intelligence, UAI ’01, pp. 362–369. Morgan Kaufmann Publishers Inc., San Francisco (2001)Google Scholar
  31. Minwa, T., Hayter, A.J., Kuriki, S.: The evaluation of general non-centered orthant pro. J. R. Stat. Soc. B 65, 223–234 (2003)CrossRefGoogle Scholar
  32. Neal, M.: MCMC using Hamiltonian dynamics. Handbook of Markov Chain Monte Carlo, p. 51 (2010)Google Scholar
  33. Neal, R.: Annealed importance sampling. Stat. Comput. 11(2), 125–139 (2001)MathSciNetCrossRefGoogle Scholar
  34. Pakman, A., Paninski, L.: Exact Hamiltonian Monte Carlo for Truncated Multivariate Gaussians. arXiv:1208.4118, pp. 1–30 (2012)
  35. Pandey, M.D.: An effective approximation to evaluate multinormal integrals. Struct. Saf. 20(1), 51–67 (1998)CrossRefGoogle Scholar
  36. Pitt, M., Shephard, N.: Filtering via simulation: auxiliary particle filters. J. Am. Stat. Assoc. 94(446), 590–599 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  37. Prekopa, A.: On probabilistic constrained programming. In: Proceedings of the Princeton Symposium on Mathematical Programming, pp. 113–138. Princeton University Press, Princeton (1970)Google Scholar
  38. Robert, C.P.: Simulation of truncated normal variables. Stat. Comput. 5(2), 121–125 (1995)CrossRefGoogle Scholar
  39. Roberts, G., Rosenthal, J.: Optimal scaling of discrete approximations to Langevin diffusions. J. R. Stat. Soc. B 60(1), 255–268 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  40. Schäefer, C., Chopin, N.: Sequential Monte Carlo on large binary sampling spaces. Stat. Comput. 23(2):163–184, 2013. ISSN 0960–3174. doi: 10.1007/s11222-011-9299-z
  41. Train, K.E.: Discrete Choice methods with simulation. Cambridge University Press, Cambridge (2009)CrossRefzbMATHGoogle Scholar
  42. Van der Vaart, A.: Asymptotic Statistics. Cambridge University Press, Cambridge (1998)CrossRefzbMATHGoogle Scholar
  43. Yao, G., Bockenholt, U.: Bayesian estimation of Thurstonian ranking models based on the Gibbs sampler. Br. J. Math. Stat. Psychol. 52, 79–92 (1999)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.CREST-ENSAE and CEREMADE Université DauphineParisFrance

Personalised recommendations