Quasi-Monte Carlo for Highly Structured Generalised Response Models

  • F. Y. Kuo
  • W. T. M. Dunsmuir
  • I. H. Sloan
  • M. P. Wand
  • R. S. Womersley
Article

Abstract

Highly structured generalised response models, such as generalised linear mixed models and generalised linear models for time series regression, have become an indispensable vehicle for data analysis and inference in many areas of application. However, their use in practice is hindered by high-dimensional intractable integrals. Quasi-Monte Carlo (QMC) is a dynamic research area in the general problem of high-dimensional numerical integration, although its potential for statistical applications is yet to be fully explored. We survey recent research in QMC, particularly lattice rules, and report on its application to highly structured generalised response models. New challenges for QMC are identified and new methodologies are developed. QMC methods are seen to provide significant improvements compared with ordinary Monte Carlo methods.

Keywords

Generalised linear mixed models High-dimensional integration Lattice rules Longitudinal data analysis Maximum likelihood Quasi-Monte Carlo Semiparametric regression Serial dependence Time series regression 

AMS 2000 Subject Classification

62J12 62M10 62M30 65D30 65D32 

References

  1. P. J. Acklam, “An algorithm for computing the inverse normal cumulative distribution function,” http://home.online.no/∼pjacklam/notes/invnorm/, 2007.
  2. E. Al-Eid, and J. Pan, “Estimation in generalized linear mixed models using SNTO approximation.” In A. R. Francis, K. M. Matawie, A. Oshlack, and G. K. Smyth (eds.), Proceedings of the 20th International Workshop on Statistical Modelling, pp. 77–84, Sydney, Australia, 2005.Google Scholar
  3. E. Anderson, Z. Bai, C. Bischof, S. Blackford, J. Demmel, J. Dongarra, J. Du Croz, A. Greenbaum, S. Hammarling, A. McKenney, and D. Sorensen, LAPACK Users’ Guide, Third Edition, SIAM: Philadelphia, 1999.Google Scholar
  4. D. Bernat, W. T. M. Dunsmuir, and A. C. Wagenaar, “Effects of lowering the BAC limit to .08 on fatal traffic crashes in 19 states,” Accident Analysis and Prevention vol. 36 pp. 1089–1097, 2004.CrossRefGoogle Scholar
  5. J. G. Booth, J. P. Hobert, and W. Jank, “A survey of Monte Carlo algorithms for maximizing the likelihood of a two-stage hierarchical model,” Statistical Modelling vol. 1 pp. 333–349, 2001.MATHCrossRefGoogle Scholar
  6. P. Bratley, and B. L. Fox, “Algorithm 659: Implementing Sobol’s quasirandom sequence generator,” ACM Transactions on Mathematical Software vol. 14 pp. 88–100, 1988.MATHCrossRefGoogle Scholar
  7. N. E. Breslow, and D. G. Clayton, “Approximate inference in generalized linear mixed models,” Journal of the American Statistical Association vol. 88 pp. 9–25, 1993.MATHCrossRefGoogle Scholar
  8. BUGS Project, “BUGS: Bayesian Inference Using Gibbs Sampling,” http://www.mrc-bsu.cam.ac.uk/bugs, 2007.
  9. R. E. Caflisch, W. Morokoff, and A. Owen, “Valuation of mortgage backed securities using Brownian bridges to reduce effective dimension,” Journal of Computational Finance vol. 1 pp. 27–46, 1997.Google Scholar
  10. B. S. Caffo, W. Jank, and G. L. Jones, “Ascent-based Monte Carlo expectation-maximization,” Journal of the Royal Statistical Society Series B vol. 67 pp. 235–251, 2005.MATHCrossRefMathSciNetGoogle Scholar
  11. D. Clayton, “Generalized linear mixed models.” In W. R. Gilks, S. Richardson, and D. J. Spiegelhalter (eds.), Markov Chain Monte Carlo in Practice, pp. 275–301, Chapman & Hall: London, 1996.Google Scholar
  12. R. Cools, F. Y. Kuo, and D. Nuyens, “Constructing embedded lattice rules for multivariate integration,” SIAM Journal on Scientific Computing vol. 28, pp. 2162–2188, 2006.MATHCrossRefMathSciNetGoogle Scholar
  13. C. Crainiceanu, D. Ruppert, and M. P. Wand, “Bayesian analysis for penalised spline regression using WinBUGS,” Journal of Statistical Software vol. 14(14), 2005.Google Scholar
  14. R. A. Davis, W. T. M. Dunsmuir, and Y. Wang, “Modelling time series of count data.” In S. Ghosh (ed.), Asymptotics, Nonparametrics and Time Series, pp. 63–114, Marcel-Dekker: New York, 1999.Google Scholar
  15. R. A. Davis, W. T. M. Dunsmuir, and Y. Wang, “On autocorrelation in a Poisson regression model,” Biometrika vol. 87 pp. 491–505, 2000.MATHCrossRefMathSciNetGoogle Scholar
  16. R. A. Davis, and G. Rodriguez-Yam, “Estimation for state-space models based on a likelihood approximation,” Statistica Sinica vol. 15 pp. 81–406, 2005.MathSciNetGoogle Scholar
  17. J. Dick, F. Pillichshammer, and B. J. Waterhouse, “The construction of good extensible rank-1 lattices,” Mathematics of Computation, (in press), 2007.Google Scholar
  18. P. Diggle, K.-L. Liang, and S. Zeger, Analysis of Longitudinal Data, Oxford University Press: Oxford, 1995.Google Scholar
  19. P. Diggle, P. Heagerty, K.-L. Liang, and S. Zeger, Analysis of Longitudinal Data, Second Edition, Oxford University Press: Oxford, 2002.Google Scholar
  20. M. Evans, and T. Swartz, “Methods for approximating integrals in statistics with special emphasis on Bayesian integration problems,” Statistical Science vol. 10 pp. 254–272, 1995.MATHCrossRefMathSciNetGoogle Scholar
  21. R. Fletcher, Practical Methods of Optimisation, Second Edition, Wiley: Chichester, 1987.Google Scholar
  22. J. Gonzalez, F. Tuerlinckx, P. De Boeck, and R. Cools, “Numerical integration in logistic-normal models,” Computational Statistics and Data Analysis vol. 51 pp. 1535–1548, 2006.CrossRefMathSciNetMATHGoogle Scholar
  23. L. C. Gurrin, K. J. Scurrah, and M. L. Hazelton, “Tutorial in biostatistics: Spline smoothing with linear mixed models,” Statistics in Medicine vol. 24 pp. 3361–3381, 2005.CrossRefMathSciNetGoogle Scholar
  24. F. J. Hickernell, and H. S. Hong, “Quasi-Monte Carlo methods and their randomisations.” In R. Chan, Y.-K. Kwok, D. Yao, and Q. Zhang (eds.), Applied Probability, AMS/IP Studies in Advanced Mathematics, vol. 26, pp. 59–77, American Mathematical Society: Providence, 2002.Google Scholar
  25. F. J. Hickernell, C. Lemieux, and A. B. Owen, “Control variates for quasi-monte carlo,” Statistical Science vol. 20 pp. 1–31, 2005.MATHCrossRefMathSciNetGoogle Scholar
  26. W. Jank, “Quasi-Monte Carlo sampling to improve the efficiency of Monte Carlo EM,” Computational Statistics and Data Analysis vol. 48 pp. 685–701, 2005.CrossRefMathSciNetMATHGoogle Scholar
  27. S. Joe, and F. Y. Kuo, “Remark on Algorithm 659: Implementing Sobol’s quasirandom sequence generator,” ACM Transactions on Mathematical Software vol. 29 pp. 49–57, 2003.MATHCrossRefMathSciNetGoogle Scholar
  28. A. Y. C. Kuk, “Laplace importance sampling for generalized linear mixed models,” Journal of Statistical Computation and Simulation vol. 63 pp. 143–158, 1999.MATHCrossRefMathSciNetGoogle Scholar
  29. F. Y. Kuo, “Component-by-component constructions achieve the optimal rate of convergence for multivariate integration in weighted Korobov and Sobolev spaces,” Journal of Complexity vol. 19 pp. 301–320, 2003.MATHCrossRefMathSciNetGoogle Scholar
  30. F. Y. Kuo, G. W. Wasilkowski, and B. J. Waterhouse, “Randomly-shifted lattice rules for unbounded integrands,” Journal of Complexity vol. 22 pp. 630–651, 2006.MATHCrossRefMathSciNetGoogle Scholar
  31. F. Y. Kuo, and I. H. Sloan, “Lifting the curse of dimensionality,” Notices of the American Mathematical Society vol. 52 pp. 1320–1328, 2005.MATHMathSciNetGoogle Scholar
  32. P. L’Ecuyer, and C. Lemieux, “Recent advances in randomized quasi-Monte Carlo methods.” In M. Dror, P. L’Ecuyer, and F. Szidarovszki (eds.), Modeling Uncertainty: An Examination of Stochastic Theory, Methods, and Applications, pp. 419–474, Kluwer Dordrecht, 2002.Google Scholar
  33. R. Liu, and A. Owen, “Estimating mean dimensionality of analysis of variance decompositions,” Journal of the American Statistical Association vol. 101 pp. 712–721, 2006.MATHCrossRefMathSciNetGoogle Scholar
  34. C. E. McCulloch, and S. R. Searle, Generalized, Linear, and Mixed Models, Wiley: New York, 2000.Google Scholar
  35. H. Niederreiter, Random Number Generation and Quasi-Monte Carlo Methods, SIAM: Philadelphia, 1992.MATHGoogle Scholar
  36. H. Niederreiter, “Construction of (t,m,s)-nets and (t,s)-sequences,” Finite Fields and Their Applications vol. 11, pp. 578–600, 2005.MATHCrossRefMathSciNetGoogle Scholar
  37. J. Nocedal, and S. J. Wright, Numerical Optimization, Springer, 1999.Google Scholar
  38. D. Nuyens, and R. Cools, “Fast algorithms for component-by-component construction of rank-1 lattice rules in shift-invariant reproducing kernel Hilbert spaces,” Mathematics of Computation vol. 75 pp. 903–920, 2006.MATHCrossRefMathSciNetGoogle Scholar
  39. A. R. Owen, and S. D. Tribble, “A quasi-Monte Carlo Metropolis algorithm,” Proceedings of the National Academy of Sciences vol. 102 pp. 8844–8849, 2005.CrossRefMathSciNetMATHGoogle Scholar
  40. J.-X. Pan, and R. Thompson, “Quasi-Monte Carlo EM algorithm for estimation in generalized linear mixed models.” In R. Payne, and P. Green (eds.), Proceedings in Computational Statistics, pp. 419–424, Physical-Verlag, 1998.Google Scholar
  41. J. Pan, and R. Thompson, “Quasi-Monte Carlo estimation in generalized linear mixed models.” In A. Biggeri, E. Dreassi, C. Lagazio, M. Marchi (eds.), Proceedings of the 19th International Workshop on Statistical Modelling, pp. 239–243, Firenze University Press: FLorence, 2004.Google Scholar
  42. J. Pan, and R. Thompson, “Quasi-Monte Carlo approximation for estimation in generalized linear mixed models,” Computational Statistics and Data Analysis vol. 51 pp. 5765–5775, 2007.CrossRefMATHMathSciNetGoogle Scholar
  43. S. H. Paskov, and J. F. Traub, “Faster valuation of financial derivatives,” Journal of Portfolio Management vol. 22 pp. 113–120, 1995.CrossRefGoogle Scholar
  44. W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Numerical Recipes in C, Second Edition, Cambridge University Press, 1995.Google Scholar
  45. D. Ruppert, M. P. Wand, and R. J. Carroll, Semiparametric Regression, Cambridge University Press: New York, 2003.MATHGoogle Scholar
  46. SAS Institute, Inc, http://www.sas.com, 2007.
  47. A. Skrondal, and S. Rabe-Hesketh, Generalized Latent Variable Modelling: Multilevel, Longitudinal and Structural Equation Models, Chapman and Hall: Boca Raton, Florida, 2004.MATHGoogle Scholar
  48. I. H. Sloan, and S. Joe, Lattice Methods for Multiple Integration, Oxford University Press: Oxford, 1994.MATHGoogle Scholar
  49. I. H. Sloan, F. Y. Kuo, and S. Joe, “Constructing randomly shifted lattice rules in weighted Sobolev spaces,” SIAM Journal on Numerical Analysis vol. 40 pp. 1650–1665, 2002.MATHCrossRefMathSciNetGoogle Scholar
  50. I. H. Sloan, X. Wang, and H. Woźniakowski, “Finite-order weights imply tractability of multivariate integration,” Journal of Complexity vol. 20 pp. 46–74, 2004.MATHCrossRefMathSciNetGoogle Scholar
  51. I. H. Sloan, and H. Woźniakowski, “When are quasi-Monte Carlo algorithms efficient for high dimensional integrals?” Journal of Complexity vol. 14 pp. 1–33, 1998.MATHCrossRefMathSciNetGoogle Scholar
  52. M. P. Wand, “Smoothing and mixed models,” Computational Statistics vol. 18 pp. 223–249, 2003.MATHGoogle Scholar
  53. X. Wang, and K. T. Fang, “Effective dimensions and quasi-Monte Carlo integration,” Journal of Complexity vol. 19 pp. 101–124, 2003.MATHCrossRefMathSciNetGoogle Scholar
  54. R. Wolfinger, and M. O’Connell, “Generalized linear mixed models: a pseudo-likelihood approach,” Journal of Statistical Computation and Simulation vol. 48 pp. 233–243, 1993.MATHCrossRefGoogle Scholar
  55. Y. Zhao, J. Staudenmayer, B. A. Coull, and M. P. Wand, “General design Bayesian generalized linear mixed models,” Statistical Science vol. 21 pp. 35–51, 2006.MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  • F. Y. Kuo
    • 1
  • W. T. M. Dunsmuir
    • 1
  • I. H. Sloan
    • 1
  • M. P. Wand
    • 1
  • R. S. Womersley
    • 1
  1. 1.School of Mathematics and StatisticsUniversity of New South WalesSydneyAustralia

Personalised recommendations