Statistics and Computing

, Volume 13, Issue 3, pp 191–208 | Cite as

Estimation and variable selection in nonparametric heteroscedastic regression

  • Paul Yau
  • Robert Kohn


The article considers a Gaussian model with the mean and the variance modeled flexibly as functions of the independent variables. The estimation is carried out using a Bayesian approach that allows the identification of significant variables in the variance function, as well as averaging over all possible models in both the mean and the variance functions. The computation is carried out by a simulation method that is carefully constructed to ensure that it converges quickly and produces iterates from the posterior distribution that have low correlation. Real and simulated examples demonstrate that the proposed method works well. The method in this paper is important because (a) it produces more realistic prediction intervals than nonparametric regression estimators that assume a constant variance; (b) variable selection identifies the variables in the variance function that are important; (c) variable selection and model averaging produce more efficient prediction intervals than those obtained by regular nonparametric regression.

Bayesian analysis Markov chain Monte Carlo variance function estimation penalized splines 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Aït-Sahalia Y. 1996a. Testing continuous-time models of the spot interest rate. The Review of Financial Studies 9: 385–426.Google Scholar
  2. Aït-Sahalia Y. 1996b. Nonparametric pricing of interest rate derivative securities. Econometrica 64: 527–560.Google Scholar
  3. Carroll R.J. (1982). Adapting for heteroscedasticity in linear models. Ann. Statist. 10: 1224–1233.Google Scholar
  4. Carroll R.J. and Ruppert D. 1988. Transformation and Weighting in Regression. Chapman and Hall, London.Google Scholar
  5. Chan K.C., Karolyi G.A., Longstaff F.A., and Sanders A.B. 1992. An empirical comparison of alternative models of the short-term interest rate. Journal of Finance 47: 1209–1227.Google Scholar
  6. Cox J.C., Ingersoll J.E., and Ross S.A. 1985. A theory of the term structure of interest rates. Econometrica 53: 385–407.Google Scholar
  7. Denison D.G.T., Mallick B.K., and Smith A.F.M. 1998. Automatic Bayesian curve fitting. Journal of the Royal Statistical Society B 60: 333–350.Google Scholar
  8. Dette H., Munk A., and Wagner T. 1998. Estimating the variance in nonparametric regression?What is a reasonable choice? J. R. Statist. Soc. B 60: 751–764.Google Scholar
  9. Eilers P.H.C. and Marx B.D. 1996. Flexible smoothing with B-splines and penalties. Statistical Science 11: 89–121.Google Scholar
  10. Gamerman D. 1997. Sampling from the posterior distribution in generalized linear mixed models. Statistics and Computing 7(1): 57–68.Google Scholar
  11. Hastie T. and Tibshirani R. 1990. Generalized Additive Models. Chapman and Hall, London.Google Scholar
  12. Kass R.E. and Wasserman L. 1995. A reference Bayesian test for nested hypotheses and its relationship to the Schwarz criterion. Journal of the American Statistical Association 90: 928–934.Google Scholar
  13. Kaufman L. and Rousseeuw P.J. 1990. Finding Groups in Data. John Wiley & Sons, New York.Google Scholar
  14. Kohn R., Smith M., and Chan D. 2001. Nonparametric regression using linear combination of basis functions. Statistics and Computing 11: 301–310.Google Scholar
  15. Marx B.D. and Eilers P.H.C. 1998. Direct generalized additive modelling with penalized likelihood. Computational Statistics and Data Analysis 28: 193–209.Google Scholar
  16. Merton R.C. 1973. Theory of rational option pricing. Bell Journal of Economics and Management Science 4: 141–183.Google Scholar
  17. Müller H.G. and Stadtmülleri U. 1987. Estimation of heteroscedasticity in regression analysis. Ann. Statist. 15: 610–625.Google Scholar
  18. Powell M. 1987. Radial basis functions for multivariate interpolation: A review. In: Mason J. and Cox M. (Eds.), Algorithms of Approximation. Clarendon Press, Oxford, pp. 143–167.Google Scholar
  19. Rendleman R. and Bartter B. 1980. The pricing of options on debt securities. Journal of Financial and Quantitative Analysis 15: 11–24.Google Scholar
  20. Ruppert D. 2002. Selecting the number of knots for penalized splines. Journal of Computational and Graphical Statistics 11: 735–757.Google Scholar
  21. Ruppert D. and Carroll R.J. 2000. Spatially-adaptive penalties for spline fitting. Australian and New Zealand Journal of Statistics 45: 205–223.Google Scholar
  22. Ruppert D., Wand M.P., Holst U., and Hössjer O. 1997. Local polynomial variance-function estimation. Technometrics 39: 262–273.Google Scholar
  23. Shively T., Kohn R., and Wood S. 1999. Variable selection and function estimation in additive nonparametric regression models using a data-based prior. (with discussion). Journal of the American Statistical Association 94: 777–806.Google Scholar
  24. Silverman B.W. 1985. Some aspects of the spline smoothing approach to non-parametric regression curve fitting. J. R. Statist. Soc. B 47: 1–52.Google Scholar
  25. Smith M. and Kohn R. 1996. Nonparametric regression using Bayesian variable selection. Journal of Econometrics 75: 317–344.Google Scholar
  26. Smith M., Yau P., Shively T., and Kohn R. 2002. Estimating longterm trends in tropospheric ozone levels. International Statistical Review 70: 99–124.Google Scholar
  27. Stanton R. 1997. A nonparametric model of term structure dynamics and the market price of interest rate risk. Journal of Finance 52: 1973–2002.Google Scholar
  28. Struyf A., Hubert M., and Rousseeuw P. 1996. Clustering in an objectoriented environment. Journal of Statistical Software 1(4): 1–30.Google Scholar
  29. Ullah A. 1985. Specification analysis of econometric models. Journal Quant. Econ. 2: 187–209.Google Scholar
  30. Tierney L. 1994. Markov chains for exploring posterior distributions. The Annals of Statistics 22: 1703–1762.Google Scholar
  31. Vasicek O.A. 1977. An equilibrium characterization of the term structure. Journal of Financial Economics 5: 177–188.Google Scholar
  32. Wahba G. 1990. Spline Models for Observational Data. SIAM, Philadelphia.Google Scholar
  33. Xiang D. and Wahba G. 1998. Approximate smoothing spline methods for large data sets in the binary case. In: Proceedings of the 1997 ASA Joint Statistical Meetings, Biometric Section, pp. 94–98.Google Scholar
  34. Yau P., Kohn R., and Wood S. 2003. Bayesian variable selection and model averaging in high dimensional multinomial nonparametric regression. Journal of Computational and Graphical Statistics 11: 23–54.Google Scholar
  35. Zellner A. 1986. On assessing prior distributions and Bayesian regression analysis with g-prior distributions. In: Goel P.K. and Zellner A. (Eds.), Bayesian Inference and Decision Techniques-Essays in Honor of Bruno de Finetti. North-Holland, Amsterdam, pp. 233–243.Google Scholar

Copyright information

© Kluwer Academic Publishers 2003

Authors and Affiliations

  • Paul Yau
    • 1
  • Robert Kohn
    • 1
  1. 1.Australian Graduate School of ManagementUniversity of NSW, UNSWSydneyAustralia

Personalised recommendations