Advertisement

Finite Gaussian Mixture Approximations to Analytically Intractable Density Kernels

  • Natalia Khorunzhina
  • Jean-François Richard
Article
  • 72 Downloads

Abstract

The objective of the paper is that of constructing finite Gaussian mixture approximations to analytically intractable density kernels. The proposed method is adaptive in that terms are added one at the time and the mixture is fully re-optimized at each step using a distance measure that approximates the corresponding importance sampling variance. All functions of interest are evaluated under Gaussian product rules. Since product rules suffer from an obvious curse of dimensionality, the proposed algorithm as presented is only applicable to models whose non-linear and/or non-Gaussian subspace is of dimension up to three. Extensions to higher-dimensional applications would require the use of sparse grids, as discussed in the paper. Examples include a sequential (filtering) evaluation of the likelihood function of a stochastic volatility model where all relevant densities (filtering, predictive and likelihood) are closely approximated by mixtures.

Keywords

Finite mixture Distance measure Gaussian quadrature Importance sampling Adaptive algorithm Stochastic volatility Density kernel 

Notes

Acknowledgements

The authors have benefited from discussions with Dave DeJong and Roman Liesenfeld.

References

  1. Adcock, C. J. (2004). Capital asset pricing in uk stocks under the multivariate skew-normal distribution. In M. G. Genton (Ed.), Skew-elliptical distributions and their applications (pp. 191–204). London: Chapman & Hall/CRC.Google Scholar
  2. Adcock, C. J. (2010). Asset pricing and portfolio selection based on the multivariate extended skew-student-\(t\) distribution. Annals of Operations Research, 176(1), 221–234.CrossRefGoogle Scholar
  3. Ardia, D., Hoogerheide, L., & van Dijk, H. (2009). Adaptive mixture of student-t distributions as a flexible candidate distribution for efficient simulation: The r package admit. Journal of Statistical Software, 29(1), 1–32.Google Scholar
  4. Azzalini, A., & Valle, A. D. (1996). The multivariate skew-normal distribution. Biometrika, 83(4), 715–726.CrossRefGoogle Scholar
  5. Basturk, N., Hoogerheide, L., Opschoor, A., & van Dijk, H. (2012). The R package MitISEM: Mixture of student-\(t\) distributions using importance sampling weighted expectation maximization for efficient and robust simulation. Tinbergen Institute Discussion Paper 12-096/III, Tinbergen InstituteGoogle Scholar
  6. Bornkamp, B. (2011). Approximating probability densities by iterated laplace approximations. Journal of Computational and Graphical Statistics, 20(3), 656–669.CrossRefGoogle Scholar
  7. Bradley, P. S., & Fayyad, U. M. (1998). Refining initial points for k-means clustering. In: J. W. Shavlik (Ed.) Proceedings of the fifteenth international conference on machine learning, Morgan Kaufmann, San Francisco, CA, USA, pp. 91–99Google Scholar
  8. Bungartz, H. J., & Griebel, M. (2004). Sparse grids. Acta Numerica, 13, 147–269.CrossRefGoogle Scholar
  9. Cameron, S. V., & Heckman, J. J. (2001). The dynamics of educational attainment for black, hispanic, and white males. Journal of Political Economy, 109(3), 455–499.CrossRefGoogle Scholar
  10. Cappé, O., Guillin, A., Marin, J. M., & Robert, C. P. (2004). Population monte carlo. Journal of Computational and Graphical Statistics, 13, 907–929.CrossRefGoogle Scholar
  11. Cappé, O., Godsill, S. J., & Moulines, E. (2007). An overview of existing methods and recent advances in sequential Monte Carlo. Proceedings of the IEEE, 95(5), 899–924.CrossRefGoogle Scholar
  12. Cappé, O., Douc, R., Guillin, A., Marin, J. M., & Robert, C. P. (2008). Adaptive importance sampling in general mixture classes. Statistics and Computing, 18(4), 447–459.CrossRefGoogle Scholar
  13. DeSarbo, W. S., Degeratu, A. M., Wedel, M., & Saxton, M. (2001). The spatial representation of market information. Marketing Science, 20(4), 426–441.CrossRefGoogle Scholar
  14. Domínguez-Molina, J., González-Farías, G., & Ramos-Quiroga, R. (2004). Skew-normality in stochastic frontier analysis. In M. G. Genton (Ed.), Skew-elliptical distributions and their applications (pp. 223–242). London: Chapman & Hall/CRC.Google Scholar
  15. Domínguez-Molina, J., González-Farías, G., Ramos-Quiroga, R., & Gupta, A. K. (2007). A matrix variate closed skew-normal distribution with applications to stochastic frontier analysis. Communications in Statistics: Theory and Methods, 36(9), 1691–1703.CrossRefGoogle Scholar
  16. Douc, R., Guillin, A., Marin, J. M., & Robert, C. P. (2007). Minimum variance importance sampling via population monte carlo. ESAIM: Probability and Statistics, 11, 427–447.CrossRefGoogle Scholar
  17. Doucet, A., de Freitas, N., & Gordon, N. (2001). Sequential Monte Carlo methods in practice. New York: Springer.CrossRefGoogle Scholar
  18. Duffie, D., & Pan, J. (1997). An overview of value at risk. The Journal of Derivatives, 4(3), 7–49.CrossRefGoogle Scholar
  19. Duong, T. (2007). ks: Kernel density estimation and kernel discriminant analysis for multivariate data in r. Journal of Statistical Software, 21(1), 1–16.Google Scholar
  20. Everitt, B. S., & Hand, D. J. (1981). Finite mixture distributions. London: Chapman and Hall.CrossRefGoogle Scholar
  21. Ferguson, T. S. (1973). A bayesian analysis of some nonparametric problems. The Annals of Statistics, 1(2), 209–230.CrossRefGoogle Scholar
  22. Ferreira, J. T., & Steel, M. F. (2007). Model comparison of coordinate-free multivariate skewed distributions with an application to stochastic frontiers. Journal of Econometrics, 137(2), 641–673.CrossRefGoogle Scholar
  23. Frühwirth-Schnatter, S. (2006). Finite mixture and Markov switching models. New York: Springer.Google Scholar
  24. Geweke, J. (1996). Monte carlo simulation and numerical integration. In H. M. Amman, D. A. Kendrick, & J. P. Rust (Eds.), Handbook of computational economics (pp. 731–800). Amsterdam: North Holland.Google Scholar
  25. Gilks, W. R., Roberts, G. O., & Sahu, S. K. (1998). Adaptive markov chain monte carlo through regeneration. Journal of the American Statistical Association, 93(443), 1045–1054.CrossRefGoogle Scholar
  26. Giordani, P., & Kohn, R. (2010). Adaptive independent metropolis-hastings by fast estimation of mixtures of normals. Journal of Computational and Graphical Statistics, 19(2), 243–259.CrossRefGoogle Scholar
  27. Hamerly, G., & Elkan, C. (2002). Alternatives to the k-means algorithm that find better clusterings. In C. Nicholas, D. Grossman, K. Kalpakis, S. Qureshi, H. van Dissel, L. Seligman (Eds.) Proceedings of the eleventh international conference on information and knowledge management, ACM, New York, NY, USA, pp. 600–607Google Scholar
  28. Hamilton, J. D. (1989). A new approach to the economic analysis of nonstationary time series and the business cycle. Econometrica, 57(2), 357–384.CrossRefGoogle Scholar
  29. Han, B., Comaniciu, D., Zhu, Y., & Davis, L. S. (2008). Sequential kernel density approximation and its application to real-time visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(7), 1186–1197.CrossRefGoogle Scholar
  30. Heiss, F., & Winschel, V. (2008). Likelihood approximation by numerical integration on sparse grids. Journal of Econometrics, 144(1), 62–80.CrossRefGoogle Scholar
  31. Holmes, G. K. (1892). Measures of distribution. Publications of the American Statistical Association, 3(18/19), 141–157.CrossRefGoogle Scholar
  32. Hoogerheide, L., Opschoor, A., & van Dijk, H. K. (2012). A class of adaptive importance sampling weighted EM algorithms for efficient and robust posterior and predictive simulation. Journal of Econometrics, 171(2), 101–120.CrossRefGoogle Scholar
  33. Hoogerheide, L. F., Kaashoek, J. F., & van Dijk, H. K. (2007). On the shape of posterior densities and credible sets in instrumental variable regression models with reduced rank: An application of flexible sampling methods using neural networks. Journal of Econometrics, 139(1), 154–180.CrossRefGoogle Scholar
  34. Hull, J., & White, A. (1998). Incorporating volatility updating into the historical simulation method for value-at-risk. Journal of Risk, 1, 5–19.CrossRefGoogle Scholar
  35. Kasahara, H., & Shimotsu, K. (2015). Testing the number of components in normal mixture regression models. Journal of the American Statistical Association, 110(512), 1632–1645.CrossRefGoogle Scholar
  36. Keane, M. P., & Wolpin, K. I. (1997). The career decisions of young men. Journal of Political Economy, 105(3), 473–522.CrossRefGoogle Scholar
  37. Kim, S., Shephard, N., & Chib, S. (1998). Stochastic volatility: Likelihood inference and comparison with arch models. Review of Economic Studies, 65(3), 361–393.CrossRefGoogle Scholar
  38. Kon, S. J. (1984). Models of stock returns-a comparison. Journal of Finance, 39(1), 147–165.Google Scholar
  39. Kurtz, N., & Song, J. (2013). Cross-entropy-based adaptive importance sampling using gaussian mixture. Structural Safety, 42, 35–44.CrossRefGoogle Scholar
  40. Liesenfeld, R., & Richard, J. F. (2006). Classical and Bayesian analysis of univariate and multivariate stochastic volatility models. Econometric Reviews, 25(2–3), 335–360.CrossRefGoogle Scholar
  41. Marchenko, Y. V., & Genton, M. G. (2012). A heckman selection-\(t\) model. Journal of the American Statistical Association, 107(497), 304–317.CrossRefGoogle Scholar
  42. Mazzuco, S., & Scarpa, B. (2015). Fitting age-specific fertility rates by a flexible generalized skew normal probability density function. Journal of the Royal Statistical Society Series A, 178(1), 187–203.CrossRefGoogle Scholar
  43. McLachlan, G. J., & Peel, D. (2000). Finite mixture models. New York: Wiley.CrossRefGoogle Scholar
  44. Moe, W. W., & Fader, P. S. (2002). Using advance purchase orders to forecast new product sales. Marketing Science, 21(3), 347–364.CrossRefGoogle Scholar
  45. Newcomb, S. (1886). A generalized theory of the combination of observations so as to obtain the best result. American Journal of Mathematics, 8(4), 343–366.CrossRefGoogle Scholar
  46. Ogundimu, E. O., & Hutton, J. L. (2016). A sample selection model with skew-normal distribution. Scandinavian Journal of Statistics, 43(1), 172–190.CrossRefGoogle Scholar
  47. Oh, M. S., & Berger, J. O. (1993). Integration of multimodal functions by monte carlo importance sampling. Journal of the American Statistical Association, 88(422), 450–456.CrossRefGoogle Scholar
  48. Omori, Y., Chib, S., Shephard, N., & Nakajima, J. (2007). Stochastic volatility with leverage: Fast and efficient likelihood inference. Journal of Econometrics, 140(2), 425–449.CrossRefGoogle Scholar
  49. Pearson, K. (1894). Contributions to the mathematical theory of evolution. Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, 185, 71–110.CrossRefGoogle Scholar
  50. Pitt, M. K., & Shephard, N. (1999). Filtering via simulation: Auxiliary particle filters. Journal of the American Statistical Association, 94(446), 590–599.CrossRefGoogle Scholar
  51. Richard, J. F., & Zhang, W. (2007). Efficient high-dimensional importance sampling. Journal of Econometrics, 141(2), 1385–1411.CrossRefGoogle Scholar
  52. Ristic, B., Arulampalam, S., & Gordon, N. (2004). Beyond the Kalman filter: Particle filters for tracking applications. Boston: Artech House.Google Scholar
  53. Scott, D. W. (1992). Multivariate density estimation. New Jersey: Wiley.CrossRefGoogle Scholar
  54. Scott, D. W., & Szewczyk, W. F. (2001). From kernels to mixtures. Technometrics, 43(3), 323–335.CrossRefGoogle Scholar
  55. Smolyak, S. A. (1963). Quadrature and interpolation formulas for tensor products of certain class of functions. Soviet Mathematics, Doklady, 148(5), 1042–1045.Google Scholar
  56. Stock, J. H., & Watson, M. W. (2007). Why has U.S. inflation become harder to forecast? Journal of Money, Credit and Banking, 39(1), 3–33.CrossRefGoogle Scholar
  57. Titterington, M., Smith, A. F., & Makov, U. E. (1985). Statistical analysis of finite mixture distributions. Chichester: Wiley.Google Scholar
  58. Tucker, A. L. (1992). A reexamination of finite- and infinite-variance distributions as models of daily stock returns. Journal of Business and Economic Statistics, 10(1), 73–81.Google Scholar
  59. Venkataraman, S. (1997). Value at risk for a mixture of normal distributions: The use of quasi- bayesian estimation techniques. Economic Perspectives, 21, 2–13.Google Scholar
  60. Wang, X., & Wang, Y. (2015). Nonparametric multivariate density estimation using mixtures. Statistics and Computing, 25(2), 349–364.CrossRefGoogle Scholar
  61. Weldon, W. F. (1892). Certain correlated variations in Crangon vulgaris. Proceedings of the Royal Society of London, 51, 1–21.Google Scholar
  62. Weldon, W. F. (1893). On certain correlated variations in Carcinus maenas. Proceedings of the Royal Society of London, 54, 318–329.CrossRefGoogle Scholar
  63. West, M. (1992). Modelling with mixtures. In J. M. Bernardo, J. O. Berger, M. H. DeGroot, & A. F. Smith (Eds.), Bayesian statistics 4 (pp. 503–524). Oxford: Oxford University Press.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2017

Authors and Affiliations

  1. 1.Department of EconomicsCopenhagen Business SchoolFrederiksbergDenmark
  2. 2.Department of EconomicsUniversity of PittsburghPittsburghUSA

Personalised recommendations