Efficient Metropolis-Hastings Sampling for Nonlinear Mixed Effects Models

  • Belhal KarimiEmail author
  • Marc Lavielle
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 296)


The ability to generate samples of the random effects from their conditional distributions is fundamental for inference in mixed effects models. Random walk Metropolis is widely used to conduct such sampling, but such a method can converge slowly for medium dimension problems, or when the joint structure of the distributions to sample is complex. We propose a Metropolis–Hastings (MH) algorithm based on a multidimensional Gaussian proposal that takes into account the joint conditional distribution of the random effects and does not require any tuning, in contrast with more sophisticated samplers such as the Metropolis Adjusted Langevin Algorithm or the No-U-Turn Sampler that involve costly tuning runs or intensive computation. Indeed, this distribution is automatically obtained thanks to a Laplace approximation of the original model. We show that such approximation is equivalent to linearizing the model in the case of continuous data. Numerical experiments based on real data highlight the very good performances of the proposed method for continuous data model.


Nonlinear MCMC Metropolis Mixed effects Sampling 


  1. 1.
    Allassonniere, S., Kuhn, E.: Convergent stochastic expectation maximization algorithm with efficient sampling in high dimension. Application to Deformable Template Model Estimation. Comput. Stat. Data Anal. 91, 4–19 (2015)Google Scholar
  2. 2.
    Beal, S., Sheiner, L.: The NONMEM system. Am. Stat. 34(2), 118–119 (1980)CrossRefGoogle Scholar
  3. 3.
    Betancourt, M.: A Conceptual Introduction to Hamiltonian Monte Carlo (2017). arXiv:1701.02434
  4. 4.
    Brooks, S., Gelman, A., Jones, G., Meng, X.-L.: Handbook of Markov Chain Monte Carlo. CRC Press (2011)Google Scholar
  5. 5.
    Chan, P.L.S., Jacqmin, P., Lavielle, M., McFadyen, L., Weatherley, B.: The use of the SAEM algorithm in MONOLIX software for estimation of population pharmacokinetic-pharmacodynamic-viral dynamics parameters of maraviroc in asymptomatic HIV subjects. J. Pharmacokinet. Pharmacodyn. 38(1), 41–61 (2011)CrossRefGoogle Scholar
  6. 6.
    Comets, E., Lavenu, A., Lavielle, M.: Parameter estimation in nonlinear mixed effect models using saemix, an R implementation of the SAEM algorithm. J. Stat. Softw. 80(3), 1–42 (2017)CrossRefGoogle Scholar
  7. 7.
    Delyon, B., Lavielle, M., Moulines, E.: Convergence of a stochastic approximation version of the EM algorithm. Ann. Stat. 27(1), 94–128 (1999)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Karimi, B., Lavielle, M., Moulines, E.: Non linear mixed effects models: bridging the gap between independent Metropolis-Hastings and variational inference. ICML 2017 Implicit Models Workshop (2017)Google Scholar
  9. 9.
    Lavielle, M.: Mixed Effects Models for The Population Approach: Models, Tasks. CRC Press, Methods and Tools (2014)Google Scholar
  10. 10.
    Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21(6), 1087–1092 (1953)CrossRefGoogle Scholar
  11. 11.
    Neal, R.M.: MCMC using Hamiltonian dynamics. Handbook of Markov Chain Monte Carlo, vol. 2(11) (2011)Google Scholar
  12. 12.
    O’Reilly, R.A., Aggeler, P.M.: Studies on Coumarin anticoagulant drugs initiation of Warfarin therapy without a lading dose. Circulation 38(1), 169–177 (1968)CrossRefGoogle Scholar
  13. 13.
    Pav, S.E.: Madness: A Package for Multivariate Automatic Differentiation (2016)Google Scholar
  14. 14.
    Robert, C.P., Casella, G.: Monte Carlo Statistical Methods. Springer Texts in Statistics (2004)Google Scholar
  15. 15.
    Roberts, G.O., Gelman, A., Gilks, W.R.: Weak convergence and optimal scaling of random walk Metropolis algorithms. Ann. Appl. Probab. 7(1), 110–120 (1997)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Roberts, G.O., Tweedie, R.L.: Exponential convergence of Langevin distributions and their discrete approximations. Bernoulli 2(4), 341–363 (1996)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Stan Development Team: RStan: the R interface to Stan. R Package Version 2.17.3 (2018)Google Scholar
  18. 18.
    Stramer, O., Tweedie, R.L.: Langevin-type models I: Diffusions with given stationary distributions and their discretizations. Methodol. Comput. Appl. Probab. 1(3), 283–306 (1999)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.InriaParisFrance

Personalised recommendations