Computational Statistics

, Volume 34, Issue 4, pp 1871–1887 | Cite as

Maximum likelihood method for bandwidth selection in kernel conditional density estimate

  • Kateřina KonečnáEmail author
  • Ivanka Horová
Original Paper


This paper discusses the kernel estimator of conditional density. A significant problem of kernel smoothing is bandwidth selection. The problem consists in the fact that optimal bandwidth depends on the unknown conditional and marginal density. This is the reason why some data-driven method needs to be applied. In this paper, we suggest a method for bandwidth selection based on a classical maximum likelihood approach. We consider a slight modification of the original method—the maximum likelihood method with one observation being left out. Applied to two types of conditional density estimators—to the Nadaraya–Watson and local linear estimator, the proposed method is compared with other known methods in a simulation study. Our aim is to compare the methods from different points of view, concentrating on the accuracy of the estimated bandwidths, on the final model quality measure, and on the computational time.


Kernel smoothing Conditional density Methods for bandwidth selection Leave-one-out maximum likelihood method 



The research was supported by Grant Agency of Masaryk University grant No. MUNI/A/1503/2018, and, moreover, by the project of specific university research at Brno University of Technology FAST-S-19-5878.


  1. Akaike H (1970) Statistical predictor identification. Ann Inst Stat Math 22(1):203–217MathSciNetCrossRefGoogle Scholar
  2. Bashtannyk DM, Hyndman RJ (2001) Bandwidth selection for kernel conditional density estimation. Comput Stat Data Anal 36(3):279–298. MathSciNetCrossRefzbMATHGoogle Scholar
  3. Bertin K, Lacour C, Rivoirard V (2016) Adaptive pointwise estimation of conditional density function. Annales de l’Institut Henri Poincaré, Probabilités et Statistiques, Institut Henri Poincaré 52:939–980MathSciNetCrossRefGoogle Scholar
  4. Chen X, Linton O, Robinson P (2001) The estimation of conditional densities. The suntory centre discussion paper no. Tech. rep., EM/01/415, LSE, LondonGoogle Scholar
  5. De Gooijer JG, Zerom D (2003) On conditional density estimation. Stat Neerl 57(2):159–176. MathSciNetCrossRefzbMATHGoogle Scholar
  6. Fan J, Yim TH (2004) A crossvalidation method for estimating conditional densities. Biometrika 91(4):819–834MathSciNetCrossRefGoogle Scholar
  7. Fan J, Yao Q, Tong H (1996) Estimation of conditional densities and sensitivity measures in nonlinear dynamical systems. Biometrika 83(1):189–206MathSciNetCrossRefGoogle Scholar
  8. Hall P (1982) Cross-validation in density estimation. Biometrika 69(2):383–390MathSciNetCrossRefGoogle Scholar
  9. Hall P, Marron JS, Park BU (1992) Smoothed cross-validation. Probab Theory Relat Fields 92(1):1–20MathSciNetCrossRefGoogle Scholar
  10. Hall P, Racine J, Li Q (2004) Cross-validation and the estimation of conditional probability densities. J Am Stat Assoc 99(468):1015–1026MathSciNetCrossRefGoogle Scholar
  11. Hansen BE (2004) Nonparametric conditional density estimation. Unpublished manuscript.
  12. Härdle W (1990) Applied nonparametric regression, vol 19. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  13. Holmes MP, Gray AG, Isbell CL (2012) Fast nonparametric conditional density estimation. ArXiv e-prints arXiv:1206.5278
  14. Horová I, Zelinka J (2007) Contribution to the bandwidth choice for kernel density estimates. Comput Stat 22(1):31–47. MathSciNetCrossRefzbMATHGoogle Scholar
  15. Horová I, Koláček J, Zelinka J (2012) Kernel smoothing in MATLAB: theory and practice of kernel smoothing. World Scientific Publishing Co. Pte. Ltd., SingaporeCrossRefGoogle Scholar
  16. Hyndman RJ, Bashtannyk DM, Grunwald GK (1996) Estimating and visualizing conditional densities. J Comput Graph Stat 5(4):315–336MathSciNetGoogle Scholar
  17. Köhler M, Schindler A, Sperlich S (2014) A review and comparison of bandwidth selection methods for kernel regression. Int Stat Rev 82(2):243–274MathSciNetCrossRefGoogle Scholar
  18. Koláček J, Horová I (2012) Iterative bandwidth method for kernel regression. J Stat Adv Theory Appl 8:93–101Google Scholar
  19. Konečná K, Horová I (2014) Conditional density estimations. International Society for the Advancement of Science and Technology (ISAST), Athens, pp 15–31Google Scholar
  20. Konečná K, Horová I (2017) Methods for bandwidth detection in kernel conditional density estimations. In: European Young Statisticians Meeting, pp 143–148Google Scholar
  21. Leiva-Murillo JM, Artés-Rodríguez A (2012) Algorithms for maximum-likelihood bandwidth selection in kernel density estimators. Pattern Recognit Lett 33(13):1717–1724CrossRefGoogle Scholar
  22. Rice J (1984) Bandwidth choice for nonparametric regression. Ann Stat 12(4):1215–1230MathSciNetCrossRefGoogle Scholar
  23. Řezáč M (2002) The reference density method for higher-order kernels. J Electr Eng 53(12):38–41zbMATHGoogle Scholar
  24. Schuster EF, Gregory GG (1981) On the nonconsistency of maximum likelihood nonparametric density estimators. In: Computer Science and Statistics: Proceedings of the 13th Symposium on the interface. Springer, pp 295–298Google Scholar
  25. Scott DW (1992) Multivariate density estimation: theory, practice, and visualization. Wiley, HobokenCrossRefGoogle Scholar
  26. Silverman BW (1982) Algorithm as 176: kernel density estimation using the fast Fourier transform. J R Stat Soc Ser C (Appl Stat) 31(1):93–99zbMATHGoogle Scholar
  27. Terrell GR (1990) The maximal smoothing principle in density estimation. J Am Stat Assoc 85(410):470–477MathSciNetCrossRefGoogle Scholar
  28. Terrell GR, Scott DW (1985) Oversmoothed nonparametric density estimates. J Am Stat Assoc 80(389):209–214MathSciNetCrossRefGoogle Scholar
  29. van der Walt CM, Barnard E (2017) Variable kernel density estimation in high-dimensional feature spaces. In: Thirty-first AAAI conference on artificial intelligenceGoogle Scholar
  30. Wand MP, Jones MC (1994) Kernel smoothing. CRC Press, Boca RatonCrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Brno University of Technology, Faculty of Civil Engineering, Institute of Mathematics and Descriptive GeometryBrnoCzech Republic
  2. 2.Masaryk University, Faculty of Science, Department of Mathematics and StatisticsBrnoCzech Republic

Personalised recommendations