Statistics and Computing

, Volume 29, Issue 1, pp 79–98 | Cite as

Fitting monotone polynomials in mixed effects models

  • Joshua J. BonEmail author
  • Kevin Murray
  • Berwin A. Turlach


We provide a method for fitting monotone polynomials to data with both fixed and random effects. In pursuit of such a method, a novel approach to least squares regression is proposed for models with functional constraints. The new method is able to fit models with constrained parameter spaces that are closed and convex, and is used in conjunction with an expectation–maximisation algorithm to fit monotone polynomials with mixed effects. The resulting mixed effects models have constrained mean curves and have the flexibility to include either unconstrained or constrained subject-specific curves. This new methodology is demonstrated on real-world repeated measures data with an application from sleep science. Code to fit the methods described in this paper is available online.


Monotone polynomials Monotone regression Mixed effects Random effects Shape constraints 

Supplementary material


  1. Afriat, S.: Theory of maxima and the method of Lagrange. SIAM J. Appl. Math. 20(3), 343–357 (1971)MathSciNetzbMATHGoogle Scholar
  2. Auspitz, R., Lieben, R.: Untersuchungen über die Theorie des Preises. Duncker & Humblot, Berlin (1889)zbMATHGoogle Scholar
  3. Barlow, R.E., Brunk, H.D.: The isotonic regression problem and its dual. J. Am. Stat. Assoc. 67(337), 140–147 (1972)MathSciNetzbMATHGoogle Scholar
  4. Barlow, R.E., Bartholomew, D.J., Bremner, J., Brunk, H.D.: Statistical Inference Under Order Restrictions: The Theory and Application of Isotonic Regression. Wiley, New York (1972)zbMATHGoogle Scholar
  5. Bates, D., Mächler, M., Bolker, B., Walker, S.: Fitting linear mixed-effects models using lme4. J. Stat. Softw. 67(1), 1–48 (2015)Google Scholar
  6. Belenky, G., Wesensten, N.J., Thorne, D.R., Thomas, M.L., Sing, H.C., Redmond, D.P., Russo, M.B., Balkin, T.J.: Patterns of performance degradation and restoration during sleep restriction and subsequent recovery: a sleep dose-response study. J. Sleep Res. 12(1), 1–12 (2003)Google Scholar
  7. Booth, J.G., Hobert, J.P.: Maximizing generalized linear mixed model likelihoods with an automated Monte Carlo EM algorithm. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 61(1), 265–285 (1999)zbMATHGoogle Scholar
  8. Bradley, R.A., Srivastava, S.S.: Correlation in polynomial regression. Am. Stat. 33(1), 11–14 (1979)zbMATHGoogle Scholar
  9. Cassioli, A., Lorenzo, D.D., Sciandrone, M.: On the convergence of inexact block coordinate descent methods for constrained optimization. Eur. J. Oper. Res. 231(2), 274–281 (2013)MathSciNetzbMATHGoogle Scholar
  10. Chen, J., Zhang, D., Davidian, M.: A Monte Carlo EM algorithm for generalized linear mixed models with flexible random effects distribution. Biostatistics 3(3), 347–360 (2002)zbMATHGoogle Scholar
  11. Damien, P., Walker, S.G.: Sampling truncated normal, beta, and gamma densities. J. Comput. Graph. Stat. 10(2), 206–215 (2001)MathSciNetGoogle Scholar
  12. De Boor, C.: A Practical Guide to Splines. Springer, New York (1978)zbMATHGoogle Scholar
  13. Dette, H., Neumeyer, N., Pilz, K.F., et al.: A simple nonparametric estimator of a strictly monotone regression function. Bernoulli 12(3), 469–490 (2006)MathSciNetzbMATHGoogle Scholar
  14. Dierckx, I.P.: An algorithm for cubic spline fitting with convexity constraints. Computing 24(4), 349–371 (1980)MathSciNetzbMATHGoogle Scholar
  15. Elphinstone, C.D.: A target distribution model for nonparametric density estimation. Commun. Stat. Theory Methods 12(2), 161–198 (1983)MathSciNetGoogle Scholar
  16. Emerson, P.L.: Numerical construction of orthogonal polynomials from a general recurrence formula. Biometrics 24(3), 695–701 (1968)MathSciNetGoogle Scholar
  17. Forsythe, G.E.: Generation and use of orthogonal polynomials for data-fitting with a digital computer. J. Soc. Ind. Appl. Math. 5(2), 74–88 (1957)MathSciNetzbMATHGoogle Scholar
  18. Friedman, J., Tibshirani, R.: The monotone smoothing of scatterplots. Technometrics 26(3), 243–250 (1984)Google Scholar
  19. Hastings, W.K.: Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57(1), 97–109 (1970)MathSciNetzbMATHGoogle Scholar
  20. Hawkins, D.M.: Fitting monotonic polynomials to data. Comput. Stat. 9(3), 233–247 (1994)zbMATHGoogle Scholar
  21. Hazelton, M.L., Turlach, B.A.: Semiparametric regression with shape-constrained penalized splines. Comput. Stat. Data Anal. 55(10), 2871–2879 (2011)MathSciNetzbMATHGoogle Scholar
  22. Hodges, J.S.: Richly Parameterized Linear Models: Additive, Time Series, and Spatial Models Using Random Effects. CRC Press, Boca Raton (2013)zbMATHGoogle Scholar
  23. Holland, P.W., Welsch, R.E.: Robust regression using iteratively reweighted least-squares. Commun. Stat. Theory Methods 6(9), 813–827 (1977)zbMATHGoogle Scholar
  24. Hornung, U.: Monotone spline interpolation. In: Collatz, L., Meinardus, G., Werner, H. (eds.) Numerische Methoden der Approximationstheorie, pp. 172–191. Birkhäuser, Basel (1978)Google Scholar
  25. Horrace, W.C.: Some results on the multivariate truncated normal distribution. J. Multivar. Anal. 94(1), 209–221 (2005)MathSciNetzbMATHGoogle Scholar
  26. Kelly, C., Rice, J.: Monotone smoothing with application to dose-response curves and the assessment of synergism. Biometrics 46(4), 1071–1085 (1990)Google Scholar
  27. Laird, N., Lange, N., Stram, D.: Maximum likelihood computations with repeated measures: application of the EM algorithm. J. Am. Stat. Assoc. 82(397), 97–105 (1987)MathSciNetzbMATHGoogle Scholar
  28. Lee, L.F.: On the first and second moments of the truncated multi-normal distribution and a simple estimator. Econ. Lett. 3(2), 165–169 (1979)Google Scholar
  29. Leppard, P., Tallis, G.M.: Algorithm AS 249: evaluation of the mean and covariance of the truncated multinormal distribution. J. R. Stat. Soc. Ser. C (Appl. Stat.) 38(3), 543–553 (1989)zbMATHGoogle Scholar
  30. Levine, R.A., Casella, G.: Implementations of the Monte Carlo EM algorithm. J. Comput. Graph. Stat. 10(3), 422–439 (2001)MathSciNetGoogle Scholar
  31. Lindstrom, M.J., Bates, D.M.: Newton-Raphson and EM algorithms for linear mixed-effects models for repeated-measures data. J. Am. Stat. Assoc. 83(404), 1014–1022 (1988)MathSciNetzbMATHGoogle Scholar
  32. Meng, X.L., Rubin, D.B.: Maximum likelihood estimation via the ECM algorithm: a general framework. Biometrika 80(2), 267–278 (1993)MathSciNetzbMATHGoogle Scholar
  33. Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21(6), 1087–1092 (1953)Google Scholar
  34. Milgrom, P., Segal, I.: Envelope theorems for arbitrary choice sets. Econometrica 70(2), 583–601 (2002)MathSciNetzbMATHGoogle Scholar
  35. Murray, K., Müller, S., Turlach, B.A.: Revisiting fitting monotone polynomials to data. Comput. Stat. 28(5), 1989–2005 (2013)MathSciNetzbMATHGoogle Scholar
  36. Murray, K., Müller, S., Turlach, B.A.: Fast and flexible methods for monotone polynomial fitting. Stat. Comput. Simul. 86(15), 2946–2966 (2016)MathSciNetGoogle Scholar
  37. Narula, S.C.: Orthogonal polynomial regression. Int. Stat. Rev. 47(1), 31–36 (1979)zbMATHGoogle Scholar
  38. Pinheiro, J.C., Bates, D.M.: Unconstrained parametrizations for variance-covariance matrices. Stat. Comput. 6(3), 289–296 (1996)Google Scholar
  39. R Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. (2016)
  40. Ramsay, J.O.: Estimating smooth monotone functions. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 60(2), 365–375 (1998)MathSciNetzbMATHGoogle Scholar
  41. Samuelson, P.A.: Foundations of Economic Analysis. Harvard University Press, Cambridge (1947)zbMATHGoogle Scholar
  42. Schmidt, T.: Really pushing the envelope: early use of the envelope theorem by Auspitz and Lieben. Hist. Polit. Econ. 36(1), 103–129 (2004)MathSciNetGoogle Scholar
  43. Tallis, G.M.: The moment generating function of the truncated multi-normal distribution. J. R. Stat. Soc. Ser. B (Methodol.) 23(1), 223–229 (1961)MathSciNetzbMATHGoogle Scholar
  44. Tuddenham, R.D., Snyder, M.M.: Physical growth of California boys and girls from birth to eighteen years. Publ. Child Dev. Univ. Calif. Berkeley 1(2), 183–364 (1954)Google Scholar
  45. Turlach, B.A.: Shape constrained smoothing using smoothing splines. Comput. Stat. 20(1), 81–104 (2005)MathSciNetzbMATHGoogle Scholar
  46. Turlach, B.A., Murray, K.: MonoPoly: Functions to Fit Monotone Polynomials. (2016). R package version 0.3-8
  47. Utreras, F.I.: Convergence rates for monotone cubic spline interpolation. J. Approx. Theory 36(1), 86–90 (1982)MathSciNetzbMATHGoogle Scholar
  48. Utreras, F.I.: Smoothing noisy data under monotonicity constraints existence, characterization and convergence rates. Numer. Math. 47(4), 611–625 (1985)MathSciNetzbMATHGoogle Scholar
  49. Wilhelm, S., Manjunath, B.G.: tmvtnorm: Truncated Multivariate Normal and Student t Distribution. (2015). R package version 1.4-10
  50. Wong, Y.: An application of orthogonalization process to the theory of least squares. Ann. Math. Stat. 6(2), 53–75 (1935)zbMATHGoogle Scholar
  51. Zadrozny, B., Elkan, C.: Transforming classifier scores into accurate multiclass probability estimates. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 694–699. ACM (2002)Google Scholar
  52. Zimmerman, D.L., Núñez-Antón, V.: Parametric modelling of growth curve data: an overview. Test 10(1), 1–73 (2001)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2017

Authors and Affiliations

  1. 1.Centre for Applied Statistics (M019)The University of Western AustraliaCrawleyAustralia
  2. 2.School of Population and Global Health (M431)The University of Western AustraliaCrawleyAustralia

Personalised recommendations