A general approach to heteroscedastic linear regression
Our article presents a general treatment of the linear regression model, in which the error distribution is modelled nonparametrically and the error variances may be heteroscedastic, thus eliminating the need to transform the dependent variable in many data sets. The mean and variance components of the model may be either parametric or nonparametric, with parsimony achieved through variable selection and model averaging. A Bayesian approach is used for inference with priors that are data-based so that estimation can be carried out automatically with minimal input by the user. A Dirichlet process mixture prior is used to model the error distribution nonparametrically; when there are no regressors in the model, the method reduces to Bayesian density estimation, and we show that in this case the estimator compares favourably with a well-regarded plug-in density estimator. We also consider a method for checking the fit of the full model. The methodology is applied to a number of simulated and real examples and is shown to work well.
KeywordsDensity estimation Dirichlet process mixture Heteroscedasticity Model checking Nonparametric regression Variable selection
Unable to display preview. Download preview PDF.
- Bartels R., Fiebig D.G., and Plumb M.H. 1996. Gas or electricity, which is cheaper?: An econometric approach with application to Australian expenditure data. The Energy Journal 17: 33–58.Google Scholar
- Carroll R.J. and Ruppert D. 1988. Transformation and Weighting in Regression. Monographs on Statistics and Applied Probability, Chapman and Hall, London.Google Scholar
- Chan D., Kohn R., Nott D.J., and Kirby C. 2005. Locally adaptive semiparametric estimation of the mean and variance functions in regression models. Forthcoming in Journal of Computational and Graphical Statistics,15: 915–936.Google Scholar
- Dahl D.B. 2003. An improved merge-split sampler for conjugate Dirichlet process mixture models. Technical Report 1086, Department of Statistics, University of Wisconsin-Madison.Google Scholar
- Kottas A. and Krnjajic M. 2005. Bayesian nonparametric modeling in quantile regression. Technical Report 2005-06, UCSC Department of Applied Math and Statistics.Google Scholar
- Bayesian semiparametric inference for the accelerated failure time model. Canadian Journal of Statistics 25: 457–472.Google Scholar
- Ruppert D., Wand M.P., and Carroll R.J. 2003. Semiparametric Regression. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press.Google Scholar
- West M. 1992. Hyperparameter estimation in Dirichlet process mixture models. ISDS Discussion paper 92-A03, Duke University.Google Scholar
- West M., Müller P., and Escobar M.D. 1994. Hierarchical priors and mixture models, with application in regression and density estimation. In: Smith A. and Freeman P. (Eds.), Aspects of Uncertainty: A tribute to D.V. Lindley, Wiley, New York, pp. 363–386.Google Scholar