Journal of Statistical Theory and Practice

, Volume 7, Issue 2, pp 204–218 | Cite as

Effect on Prediction When Modeling Covariates in Bayesian Nonparametric Models

  • Alejandro Cruz-Marcelo
  • Gary L. RosnerEmail author
  • Peter Müller
  • Clinton F. Stewart


In biomedical research, it is often of interest to characterize biologic processes giving rise to observations and to make predictions of future observations. Bayesian nonparamric methods provide a means for carrying out Bayesian inference making as few assumptions about restrictive parametric models as possible. There are several proposals in the literature for extending Bayesian nonparametric models to include dependence on covariates. In this article, we examine the effect on fitting and predictive performance of incorporating covariates in a class of Bayesian nonparametric models by one of two primary ways: either in the weights or in the locations of a discrete random probability measure. We show that different strategies for incorporating continuous covariates in Bayesian nonparametric models can result in big differences when used for prediction, even though they lead to otherwise similar posterior inferences. When one needs the predictive density, as in optimal design, and this density is a mixture, it is better to make the weights depend on the covariates. We demonstrate these points via a simulated data example and in an application in which one wants to determine the optimal dose of an anticancer drug used in pediatric oncology.


Covariates modeling Dependent Dirichlet process Dirichlet process mixture Hierarchical model Nonparametric Bayes Predictive distribution 

AMS Subject Classification



Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Burr, D., and H. Doss. 2005. A Bayesian semiparametric model for random-effects meta-analysis. J. Am. Stat. Assoc., 100(469), 242–251.MathSciNetzbMATHCrossRefGoogle Scholar
  2. Caron, F., M. Davy, A. Doucet, E. Duflos, and P. Vanheeghe. 2006. Bayesian inference for dynamic models with Dirichlet process mixtures. IEEE Trans. Signal Process., 56(1), 71–84.MathSciNetzbMATHCrossRefGoogle Scholar
  3. Carota, C., and G. Parmigiani. 2002. Semiparametric regression for count data. Biometrika, 89(2), 265–285.MathSciNetzbMATHCrossRefGoogle Scholar
  4. Cifarelli, D., and E. Regazzini, 1978. Nonparametric statistical problems under partial exchangeability. the use of associative means. Ann. Inst. Mat. Finianz. Univ. Torino, 12, 1–36.Google Scholar
  5. De Iorio, M., W. Johnson, P. Müller, and G. Rosner. 2009. Bayesian nonparametric nonproportional hazards survival modeling. Biometrics, 65(3), 762–771.MathSciNetzbMATHCrossRefGoogle Scholar
  6. De Iorio, M., P. Müller, G. Rosner, and S. MacEachern. 2004. An ANOVA model for dependent random measures. J. Am. Stat. Assoc., 99(465), 205–215.MathSciNetzbMATHCrossRefGoogle Scholar
  7. Dunson, D. 2009. Bayesian nonparametric hierarchical modeling. Biometrical J., 51(2), 273–284.MathSciNetCrossRefGoogle Scholar
  8. Dunson, D., and J. Park. 2008. Kernel stick-breaking processes. Biometrika, 95(2), 307–323.MathSciNetzbMATHCrossRefGoogle Scholar
  9. Dunson, D., N. Pillai, and J. Park. 2007. Bayesian density regression. J. R. Stat. Soc. Ser. B (Stat. Methodol.), 69(2), 163–183.MathSciNetzbMATHCrossRefGoogle Scholar
  10. Ferguson, T. 1973. A Bayesian analysis of some nonparametric problems. Ann. Stat., 1(2), 209–230.MathSciNetzbMATHCrossRefGoogle Scholar
  11. Fuentes-García, R., R. Mena, and S. Walker. 2009. A nonparametric dependent process for Bayesian regression. Stat. Probability Lett., 79(8), 1112–1119.MathSciNetzbMATHCrossRefGoogle Scholar
  12. Gelfand, A., A. Kottas, and S. MacEachern. 2005. Bayesian nonparametric spatial modeling with Dirichlet process mixing. J. Am. Stat. Assoc., 100(471), 1021–1036.MathSciNetzbMATHCrossRefGoogle Scholar
  13. Gibaldi, M., and D. Perrier. 1982. Pharmacokinetics, 2nd ed. New York, NY, Marcel Dekker.Google Scholar
  14. Griffin, J., and M. Steel. 2004. Semiparametric Bayesian inference for stochastic frontier models. J. Econometrics, 123(1), 121–152.MathSciNetzbMATHCrossRefGoogle Scholar
  15. Griffin, J., and M. Steel. 2006. Order-based dependent Dirichlet processes. J. Am. Stat. Assoc., 101(473), 179–194.MathSciNetzbMATHCrossRefGoogle Scholar
  16. Haario, H., E. Saksman, and J. Tamminen. 2001. An adaptive Metropolis algorithm. Bernoulli, 7(2), 223–242.MathSciNetzbMATHCrossRefGoogle Scholar
  17. Kim, S., M. Tadesse, and M. Vannucci. 2006. Variable selection in clustering via Dirichlet process mixture models. Biometrika, 93(4), 877–893.MathSciNetzbMATHCrossRefGoogle Scholar
  18. MacEachern, S. 1999. Dependent nonparametric processes. In ASA Proceedings of the Section on Bayesian Statistical Science, 50–55. Alexandria, VA, American Statistical Association.Google Scholar
  19. MacEachern, S., and P. Müller. 1998. Estimating mixture of Dirichlet process models. J. Comput. Graph. Stat., 7(2), 223–238.Google Scholar
  20. Müller, P., A. Erkanli, and M. West. 1996. Bayesian curve fitting using multivariate normal mixtures. Biometrika, 83(1), 67–79.MathSciNetzbMATHCrossRefGoogle Scholar
  21. Müller, P., F. Quintana, and G.L. Rosner. 2004. A method for combining inference across related nonparametric Bayesian models. J. R. Stat. Soc. Ser. B (Stat. Methodol.), 66(3), 735–749.MathSciNetzbMATHCrossRefGoogle Scholar
  22. Müller, P., and G.L. Rosner. 1998. Semi-parametric PK/PD models. In Practical nonparametric and semiparametric Bayesian statistics, ed. D. Dey, P. Müller, and D. Sinha, (Eds.), 323–337. New York, NY, Springer-Verlag.zbMATHCrossRefGoogle Scholar
  23. Rodriguez, A., and E. ter Horst. 2008. Bayesian dynamic density estimation. Bayesian Anal., 3(2), 339–366.MathSciNetzbMATHCrossRefGoogle Scholar
  24. Rosner, G.L., and P. Müller. 1997. Bayesian population pharmacokinetic and pharmacodynamic analyses using mixture models. J. Pharmacokin. Pharmacodyn., 25(2), 209–233.CrossRefGoogle Scholar
  25. Schaiquevich, P., J.C. Panetta, L.C. Iacono, B.B. Freeman, V.M. Santana, A. Gajjar, and C.F. Stewart. 2007. Population pharmacokinetic analysis of topotecan in pediatric cancer patients. Clin. Cancer Res., 13(22), 6703–6711.CrossRefGoogle Scholar
  26. Sethuraman, J. 1994. A constructive definition of Dirichlet priors. Stat. Sin., 4(2), 639–650.MathSciNetzbMATHGoogle Scholar
  27. Wakefield, J. 1994. An expected loss approach to the design of dosage regimens via sampling-based methods. Statistician, 43(1), 13–29.CrossRefGoogle Scholar
  28. Xing, E., M. Jordan, and R. Sharan. 2007. Bayesian haplotype inference via the Dirichlet process. J. Comput. Biol., 14(3), 267–284.MathSciNetCrossRefGoogle Scholar
  29. Zeger, S., and M. Karim. 1991. Generalized linear models with random effects; A Gibbs sampling approach. J. Am. Stat. Assoc., 86(413), 79–86.MathSciNetCrossRefGoogle Scholar

Copyright information

© Grace Scientific Publishing 2013

Authors and Affiliations

  • Alejandro Cruz-Marcelo
    • 1
  • Gary L. Rosner
    • 2
    Email author
  • Peter Müller
    • 3
  • Clinton F. Stewart
    • 4
  1. 1.Credit Risk ManagementCapital One Financial CorporationMcLeanUSA
  2. 2.Division of Oncology Biostatistics & BioinformaticsSidney Kimmel Comprehensive Cancer Center at John HopkinsBaltimoreUSA
  3. 3.Department of MathematicsUniversity of TexasAustinUSA
  4. 4.Pharmaceutical SciencesSt. Jude Children’s Research HospitalMemphisUSA

Personalised recommendations