Abstract
Sampling from a truncated multivariate distribution subject to multiple linear inequality constraints is a recurring problem in many areas in statistics and econometrics, such as the order-restricted regressions, censored data models, and shape-restricted non-parametric regressions. However, the sampling problem remains nontrivial due to the analytically intractable normalizing constant of the truncated multivariate distribution. We first develop an efficient rejection sampling method for the truncated univariate normal distribution, and analytically establish its superiority in terms of acceptance rates compared to some of the popular existing methods. We then extend our methodology to obtain samples from a truncated multivariate normal distribution subject to convex polytope restriction regions. Finally, we generalize the sampling method to truncated scale mixtures of multivariate normal distributions. Empirical results are presented to illustrate the superior performance of our proposed Gibbs sampler in terms of various criteria (e.g., mixing and integrated auto-correlation time).
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Breslaw, J. 1994. Random sampling from a truncated multivariate normal distribution. Appl. Math. Lett., 7(1), 1–6.
Damien, P., and S. G. Walker. 2001. Sampling truncated normal, beta, and gamma densities. J. Comput. Graph. Stat., 10(2), 206–215.
Gelfand, A. E., and A. F. M. Smith. 1990. Sampling-based approaches to calculating marginal densities. J. Am. Stat. Assoc., 85(410), 398–409.
Gelfand, A. E., A. F. M. Smith, and T.-M. Lee. 1992. Bayesian analysis of constrained parameter and truncated data problems using Gibbs sampling. J. Am. Stat. Assoc., 87(418), 523–532.
Geman, S., and D. Geman. 1984. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Machine Intell., PAMI-6, 721–741.
Geweke, J. 1991. Efficient simulation from the multivariate normal and Student-t distributions subject to linear constraints and the evaluation of constraint probabilities. In Computing science and statistics: Proceedings of the 23rd Symposium on the Interface, ed. E. M. Keramidas, 571–578. Fairfax Station, VA: Interface Foundation of North America, Inc.
Geweke, J. 1996. Bayesian inference for linear models subject to linear inequality constraints. In Modelling and prediction: Honoring Seymour Geisser, ed. J. C. Lee, W. O. Johnson, and A. Zellner, 248–263. New York, NY: Springer.
Gómez-Sánchez-Manzano, E., M. Gómez-Villegas, and J. Marín. 2008. Multivariate exponential power distributions as mixtures of normal distributions with bayesian applications. Commun. Stat. Theory Methods, 37(6), 972–985.
Hajivassiliou, V. A., and D. L. Mcfadden, 1990. The method of simulated scores for the estimation of ldv models: With an application to the problem of external debt crises. Cowles Foundation Discussion Paper no. 967. https://ideas.repec.org/p/cwl/cwldpp/967.html
Horrace, W. C. 2005. Some results on the multivariate truncated normal distribution. J. Multivariate Anal., 94(1), 209–221.
Liechty, M. W., and J. Lu. 2010. Multivariate normal slice sampling. J. Comput. Graph. Stat., 19(2), 281–294. With supplementary material available online.
Neal, R. M. 2003. Slice sampling. Ann. Stat., 31(3), 705–767. With discussions and a rejoinder by the author.
Robert, C. P. 1995. Simulation of truncated normal variables. Stat. Comput., 5, 121–125.
Robert, C. P., and G. Casella. 2004. Monte Carlo statistical methods. 2nd ed. Springer Texts in Statistics. New York, NY: Springer-Verlag.
Rodrigues-Yam, G., R. A. Davis, and L. L. Scharf. 2004. Efficient Gibbs sampling of truncated multivariate normal with application to constrained linear regression. Technical report, Colorado State University, Fort Collins, CO.
Tierney, L. 1994. Markov chains for exploring posterior distributions. Ann. Stat., 22(4), 1701–1762. With discussion and a rejoinder by the author.
Yu, J., and G. Tian. 2011. Efficient algorithms for generating truncated multivariate normal distributions. Acta Math. Appl. Sin. (English Ser.), 27, 601–612.
Author information
Authors and Affiliations
Corresponding author
Additional information
Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/ujsp.
Rights and permissions
About this article
Cite this article
Li, Y., Ghosh, S.K. Efficient sampling methods for truncated multivariate normal and student-t distributions subject to linear inequality constraints. J Stat Theory Pract 9, 712–732 (2015). https://doi.org/10.1080/15598608.2014.996690
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1080/15598608.2014.996690