Computational Statistics

, Volume 29, Issue 3–4, pp 829–848 | Cite as

Bayesian bandwidth estimation for a semi-functional partial linear regression model with unknown error density

  • Han Lin ShangEmail author
Original Paper


In the context of semi-functional partial linear regression model, we study the problem of error density estimation. The unknown error density is approximated by a mixture of Gaussian densities with means being the individual residuals, and variance a constant parameter. This mixture error density has a form of a kernel density estimator of residuals, where the regression function, consisting of parametric and nonparametric components, is estimated by the ordinary least squares and functional Nadaraya–Watson estimators. The estimation accuracy of the ordinary least squares and functional Nadaraya–Watson estimators jointly depends on the same bandwidth parameter. A Bayesian approach is proposed to simultaneously estimate the bandwidths in the kernel-form error density and in the regression function. Under the kernel-form error density, we derive a kernel likelihood and posterior for the bandwidth parameters. For estimating the regression function and error density, a series of simulation studies show that the Bayesian approach yields better accuracy than the benchmark functional cross validation. Illustrated by a spectroscopy data set, we found that the Bayesian approach gives better point forecast accuracy of the regression function than the functional cross validation, and it is capable of producing prediction intervals nonparametrically.


Functional Nadaraya–Watson estimator Functional regression Gaussian kernel mixture Error density estimation Markov chain Monte Carlo 



The author thanks Professors Rob Hyndman and Donald Poskitt for introducing him to functional data analysis, and Professors Xibin Zhang and Maxwell King for introducing him to Bayesian bandwidth estimation. The author would like to acknowledge Monash Sun Grid for its excellent parallel computing facility. Special thanks go to an associate editor and a referee for their insightful comments and suggestions, which led to a much improved manuscript.

Supplementary material

180_2013_463_MOESM1_ESM.r (33 kb)
Supplementary material 1 (R 33 KB)
180_2013_463_MOESM2_ESM.r (18 kb)
Supplementary material 2 (R 17 KB)
180_2013_463_MOESM3_ESM.r (9 kb)
Supplementary material 3 (R 8 KB)
180_2013_463_MOESM4_ESM.r (10 kb)
Supplementary material 4 (R 10 KB)
180_2013_463_MOESM5_ESM.r (2 kb)
Supplementary material 5 (R 1 KB)
180_2013_463_MOESM6_ESM.txt (169 kb)
Supplementary material 6 (txt 168 KB)
180_2013_463_MOESM7_ESM.pdf (116 kb)
Supplementary material 7 (pdf 115 KB)


  1. Aneiros-Pérez G, Cao R, Vilar-Fernández JM, Muñoz-San-Roque A (2011) Functional prediction for the residual demand in electricity spot markets. In: Ferraty F (ed) Recent advances in functional data analysis and related topics. Physica-Verlag, BerlinGoogle Scholar
  2. Aneiros-Pérez G, Vieu P (2006) Semi-functional partial linear regression. Stat Probab Lett 76(11): 1102–1110CrossRefzbMATHGoogle Scholar
  3. Aneiros-Pérez G, Vieu P (2008) Nonparametric time series prediction: a semi-functional partial linear modeling. J Multivar Anal 99(5):834–857CrossRefzbMATHGoogle Scholar
  4. Aneiros-Pérez G, Vieu P (2011) Automatic estimation procedure in partial linear model with functional data. Stat Papers 52(4):751–771CrossRefzbMATHGoogle Scholar
  5. Anglin PM, Gencay R (1996) Semiparametric estimation of a hedonic price function. J Appl Econom 11(6):633–648CrossRefGoogle Scholar
  6. Ansley CF, Wecker WE (1983) Extensions and examples of the signal extraction approach to regression. In: Zellner A (ed) Applied time series analysis of economic data. Bureau of the Census, Washington, pp 181–192Google Scholar
  7. Barrientos-Marin J, Ferraty F, Vieu P (2010) Locally modelled regression and functional data. J Nonparametr Stat 22(5):617–632CrossRefzbMATHMathSciNetGoogle Scholar
  8. Benhenni K, Ferraty F, Rachdi M, Vieu P (2007) Local smoothing regression with functional data. Comput Stat 22(3):353–369CrossRefMathSciNetGoogle Scholar
  9. Berlinet A, Elamine A, Mas A (2011) Local linear regression for functional data. Ann Inst Stat Math 63(5):1047–1075CrossRefzbMATHMathSciNetGoogle Scholar
  10. Boj E, Delicado P, Fortiana J (2010) Distance-based local linear regression for functional predictors. Comput Stat Data Anal 54(2):429–437CrossRefzbMATHMathSciNetGoogle Scholar
  11. Bowman AW (1984) An alternative method of cross-validation for the smoothing of density estimates. Biometrika 71(2):353–360CrossRefMathSciNetGoogle Scholar
  12. Burba F, Ferraty F, Vieu P (2009) \(k\)-nearest neighbour method in functional nonparametric regression. J Nonparametr Stat 21(4):453–469Google Scholar
  13. Engle R, Granger C, Rice J, Weiss A (1986) Semiparametric estimates of the relation between weather and electricity sales. J Am Stat Assoc 81(394):310–320CrossRefGoogle Scholar
  14. Eubank RL, Whitney P (1989) Convergence rates for estimation in certain partially linear models. J Stat Plan Inf 23(1):33–43CrossRefzbMATHMathSciNetGoogle Scholar
  15. Ferraty F, Van Keilegom I, Vieu P (2010) On the validity of the bootstrap in non-parametric functional regression. Scand J Stat 37(2):286–306CrossRefzbMATHMathSciNetGoogle Scholar
  16. Ferraty F, Vieu P (2002) The functional nonparametric model and application to spectrometric data. Comput Stat 17(4):545–564CrossRefzbMATHMathSciNetGoogle Scholar
  17. Ferraty F, Vieu P (2006) Nonparametric functional data analysis: theory and practice. Springer, New YorkGoogle Scholar
  18. Ferraty F, Vieu P (2009) Additive prediction and boosting for functional data. Comput Stat Data Anal 53(4):1400–1413CrossRefzbMATHMathSciNetGoogle Scholar
  19. Gabrys R, Horváth L, Kokoszka P (2010) Tests for error correlation in the functional linear model. J Am Stat Assoc 105(491):1113–1125CrossRefGoogle Scholar
  20. Garthwaite PH, Fan Y, Sisson SA (2010) Adaptive optimal scaling of Metropolis-Hastings algorithms using the Robbins-Monro process, Working paper, University of New South Wales.
  21. Geweke J (1992) Evaluating the accuracy of sampling-based approaches to calculating posterior moments. In: Bernardo JM, Berger J (eds) Bayesian statistics. Clarendon Press, Oxford, pp 169–193Google Scholar
  22. Geweke J (1999) Using simulation methods for Bayesian econometric models: inference, development, and communication. Econom Rev 18(1):1–73CrossRefzbMATHMathSciNetGoogle Scholar
  23. Geweke J (2010) Complete and incomplete econometric models. Princeton University Press, PrincetonzbMATHGoogle Scholar
  24. Gilks WR, Richardson S, Spiegelhalter DJ (1996) Markov chain Monte Carlo in practice. Chapman & Hall, LondonCrossRefzbMATHGoogle Scholar
  25. Goutis C (1998) Second-derivative functional regression with applications to near infra-red spectroscopy. J R Stat Soc Ser B 60(1):103–114CrossRefzbMATHMathSciNetGoogle Scholar
  26. Hall P (1987) On Kullback-Leibler loss and density estimation. Ann Stat 15(4):1491–1519CrossRefzbMATHGoogle Scholar
  27. Härdle W, Liang H, Gao J (2000) Partially linear models. Physica-Verlag, New YorkCrossRefzbMATHGoogle Scholar
  28. Heckman N (1986) Spline smoothing in a partly linear model. J R Stat Soc Ser B 48(2):244–248zbMATHMathSciNetGoogle Scholar
  29. Heidelberger P, Welch PD (1983) Simulation run length control in the presence of an initial transient. Oper Res 31(6):1109–1144CrossRefzbMATHGoogle Scholar
  30. Horváth L, Kokoszka P (2012) Inference for functional data with applications. Springer, New YorkCrossRefzbMATHGoogle Scholar
  31. Jaki T, West RW (2008) Maximum kernel likelihood estimation. J Comput Graph Stat 17(4):976–993CrossRefMathSciNetGoogle Scholar
  32. Jaki T, West W (2011) Symmetric maximum kernel likelihood estimation. J Stat Comput Simul 81(2): 193–206CrossRefzbMATHMathSciNetGoogle Scholar
  33. Kim S, Shephard N, Chib S (1998) Stochastic volatility: likelihood inference and comparison with arch models. Rev Econ Stud 65(3):361–393CrossRefzbMATHGoogle Scholar
  34. Marron JS, Wand MP (1992) Exact mean integrated squared error. Ann Stat 20(2):712–736CrossRefzbMATHMathSciNetGoogle Scholar
  35. McLachlan G, Peel D (2000) Finite mixture models. Wiley, New YorkCrossRefzbMATHGoogle Scholar
  36. Meyer R, Yu J (2000) BUGS for a Bayesian analysis of stochastic volatility models. Econom J 3(2):198–215CrossRefzbMATHGoogle Scholar
  37. Plummer M, Best N, Cowles K, Vines K (2006) Coda: convergence diagnosis and output analysis for mcmc. R News 6(1):7–11Google Scholar
  38. R Core Team (2013) R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria.
  39. Rachdi M, Vieu P (2007) Nonparametric regression for functional data: automatic smoothing parameter selection. J Stat Plan Inf 137(9):2784–2801CrossRefzbMATHMathSciNetGoogle Scholar
  40. Ramsay JO, Silverman BW (2005) Functional data analysis, 2nd edn. Springer, New YorkGoogle Scholar
  41. Rice J (1986) Convergence rates for partially splined models. Stat Probab Lett 4(4):203–208CrossRefzbMATHMathSciNetGoogle Scholar
  42. Robbins H, Monro S (1951) A stochastic approximation method. Ann Math Stat 22(3):400–407CrossRefzbMATHMathSciNetGoogle Scholar
  43. Robert CP, Casella G (2010) Introducing Monte Carlo methods with R. Springer, New YorkCrossRefzbMATHGoogle Scholar
  44. Roberts GO, Rosenthal JS (2009) Examples of adaptive MCMC. J Comput Graph Stat 18(2):349–367CrossRefMathSciNetGoogle Scholar
  45. Robinson P (1988) Root-N-consistent semiparametric regression. Econometrica 56(4):931–954CrossRefzbMATHMathSciNetGoogle Scholar
  46. Samb R (2011) Nonparametric estimation of the density of regression errors. C R Acad Sci Paris Ser I 349(23–24), 1281–1285Google Scholar
  47. Schmalensee R, Stoker TM (1999) Household gasoline demand in the united states. Econometrica 67(3):645–662CrossRefGoogle Scholar
  48. Shang HL (2013a) Bayesian bandwidth estimation for a nonparametric functional regression model with unknown error density. Comput Stat Data Anal 67:185–198CrossRefGoogle Scholar
  49. Shang HL (2013b) Functional time series approach for forecasting very short-term electricity demand. J Appl Stat 40(1):152–168CrossRefMathSciNetGoogle Scholar
  50. Speckman P (1988) Kernel smoothing in partial linear models. J R Stat Soc Ser B 50(3):413–436zbMATHMathSciNetGoogle Scholar
  51. Tse YK, Zhang X, Yu J (2004) Estimation of hyperbolic diffusion using the Markov chain Monte Carlo method. Quant Financ 4(2):158–169CrossRefMathSciNetGoogle Scholar
  52. Yao F, Müller H-G (2010) Functional quadratic regression. Biometrika 97(1):49–64CrossRefzbMATHMathSciNetGoogle Scholar
  53. Zhang X, Brooks RD, King ML (2009) A Bayesian approach to bandwidth selection for multivariate kernel regression with an application to state-price density estimation. J Econom 153(1):21–32CrossRefMathSciNetGoogle Scholar
  54. Zhang X, King ML (2011) Bayesian semiparametric GARCH models, Working paper, Monash University.

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  1. 1.University of SouthamptonHighfield, SouthamptonUK
  2. 2.The Australian National UniversityCanberraAustralia

Personalised recommendations