Kernel Estimation of Regression Functions in the Boundary Regions

  • Tomasz Gałkowski
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7895)


The article refers to the problem of regression functions estimation in the points near the edges of their domain. We investigate the model \(y_i = R\left( {x_i } \right) + \epsilon _i ,\,i = 1,2, \ldots n\), where x i is assumed to be the set of deterministic inputs, x i  ∈ D, y i is the set of probabilistic outputs, and ε i is a measurement noise with zero mean and bounded variance. \(R\left( . \right)\) is a completely unknown function. The possible clue of finding unknown function is to apply the algorithms based on Parzen kernel [5], [12]. The commonly known inconvenience of these algorithms is that the error of estimation dramatically increases if the point of estimation x is coming up to the left or right bound of interval D.

The main result of this paper is a new, original algorithm (named NMS) based on integral version of Parzen methods for estimation of edge values of a function R. The cross-validation-like technique is used in this procedure. The results of numerical experiments are presented.


IEEE Transaction Regression Function Kernel Estimation Orthogonal Series Multiple Fourier Series 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Chen, S.X.: Beta kernel estimators for density functions. Journal of Statistical Planning and Inference 139, 2269–2283 (2009)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Gałkowski, T., Rutkowski, L.: Nonparametric recovery of multivariate functions with applications to system identification. Proceedings of the IEEE 73, 942–943 (1985)CrossRefGoogle Scholar
  3. 3.
    Gałkowski, T., Rutkowski, L.: Nonparametric fitting of multivariable functions. IEEE Transactions on Automatic Control AC-31, 785–787 (1986)CrossRefGoogle Scholar
  4. 4.
    Gałkowski, T.: Nonparametric estimation of boundary values of functions. Archives of Control Science 3(1-2), 85–93 (1994)zbMATHGoogle Scholar
  5. 5.
    Gasser, T., Muller, H.G.: Kernel estimation of regression functions. Lecture Notes in Mathematics, vol. 757, pp. 23–68. Springer, Heidelberg (1979)Google Scholar
  6. 6.
    Greblicki, W., Rutkowski, L.: Density-free Bayes risk consistency of nonparametric pattern recognition procedures. Proceedings of the IEEE 69(4), 482–483 (1981)CrossRefGoogle Scholar
  7. 7.
    Greblicki, W., Rutkowska, D., Rutkowski, L.: An orthogonal series estimate of time-varying regression. Annals of the Institute of Statistical Mathematics 35(1), 215–228 (1983)MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Hazelton, M.L., Marshall, J.C.: Linear boundary kernels for bivariate density estimation. Statistics and Probability Letters 79, 999–1003 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Karunamuni, R.J., Alberts, T.: On boundary correction in kernel density estimation. Statistical Methodology 2, 191–212 (2005)MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Karunamuni, R.J., Alberts, T.: A locally adaptive transformation method of boundary correction in kernel density estimation. Journal of Statistical Planning and Inference 136, 2936–2960 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Kyung-Joon, C., Schucany, W.R.: Nonparametric kernel regression estimation near endpoints. Journal of Statistical Planning and Inference 66, 289–304 (1998)MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Marshall, J.C., Hazelton, M.L.: Boundary kernels for adaptive density estimators on regions with irregular boundaries. Journal of Multivariate Analysis 101, 949–963 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Müller, H.G.: Smooth optimum kernel estimators near endpoints. Biometrika 78, 521–530 (1991)MathSciNetzbMATHGoogle Scholar
  14. 14.
    Parzen, E.: On estimation of a probability density function and mode. Analysis of Mathematical Statistics 33(3), 1065–1076 (1962)MathSciNetzbMATHCrossRefGoogle Scholar
  15. 15.
    Poměnková-Dluhá, J.: Edge Effects of Gasser-Müller Estimator, Mathematica 15, Brno, Masaryk University, pp. 307-314 (2004)Google Scholar
  16. 16.
    Rafajłowicz, E.: Nonparametric least squares estimation of a regression function Statistics. A Journal of Theoretical and Applied Statistics 19(3), 349–358 (1988)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Rafajłowicz, E., Schwabe, R.: Halton and Hammersley sequences in multivariate nonparametric regression. Statistics and Probability Letters 76(8), 803–812 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    Rutkowski, L.: Sequential estimates of probability densities by orthogonal series and their application in pattern classification. IEEE Transactions on Systems, Man, and Cybernetics SMC-10(12), 918–920 (1980)MathSciNetGoogle Scholar
  19. 19.
    Rutkowski, L.: On Bayes risk consistent pattern recognition procedures in a quasi-stationary environment. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-4(1), 84–87 (1982)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Rutkowski, L.: On nonparametric identification with prediction of time-varying systems. IEEE Transactions on Automatic Control AC-29, 58–60 (1984)CrossRefGoogle Scholar
  21. 21.
    Rutkowski, L.: Nonparametric identification of quasi-stationary systems. Systems and Control Letters 6, 33–35 (1985)MathSciNetzbMATHCrossRefGoogle Scholar
  22. 22.
    Rutkowski, L.: Real-time identification of time-varying systems by non-parametric algorithms based on Parzen kernels. International Journal of Systems Science 16, 1123–1130 (1985)zbMATHCrossRefGoogle Scholar
  23. 23.
    Rutkowski, L.: A general approach for nonparametric fitting of functions and their derivatives with applications to linear circuits identification. IEEE Transactions Circuits Systems CAS-33, 812–818 (1986)CrossRefGoogle Scholar
  24. 24.
    Rutkowski, L.: Sequential pattern recognition procedures derived from multiple Fourier series. Pattern Recognition Letters 8, 213–216 (1988)zbMATHCrossRefGoogle Scholar
  25. 25.
    Rutkowski, L.: Application of multiple Fourier series to identification of multivariable nonstationary systems. International Journal of Systems Science 20(10), 1993–2002 (1989)MathSciNetzbMATHCrossRefGoogle Scholar
  26. 26.
    Rutkowski, L.: Non-parametric learning algorithms in the time-varying environments. Signal Processing 18(2), 129–137 (1989)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Rutkowski, L., Rafajłowicz, E.: On global rate of convergence of some nonparametric identification procedures. IEEE Transaction on Automatic Control AC-34(10), 1089–1091 (1989)CrossRefGoogle Scholar
  28. 28.
    Rutkowski, L.: Identification of MISO nonlinear regressions in the presence of a wide class of disturbances. IEEE Transactions on Information Theory IT-37, 214–216 (1991)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Rutkowski, L.: Multiple Fourier series procedures for extraction of nonlinear regressions from noisy data. IEEE Transactions on Signal Processing 41(10), 3062–3065 (1993)zbMATHCrossRefGoogle Scholar
  30. 30.
    Schuster, E.F.: Incorporating Support Constraints Into Nonparametric Estimators of Densities. Communications in Statistics, Part A - Theory and Methods 14, 1123–1136 (1985)MathSciNetzbMATHCrossRefGoogle Scholar
  31. 31.
    Skubalska-Rafajłowicz, E.: Pattern recognition algorithms based on space-filling curves and orthogonal expansions. IEEE Transactions on Information Theory 47(5), 1915–1927 (2001)zbMATHCrossRefGoogle Scholar
  32. 32.
    Skubalska-Rafajłowicz, E.: Random projection RBF nets for multidimensional density estimation. International Journal of Applied Mathematics and Computer Science 18(4), 455–464 (2008)zbMATHGoogle Scholar
  33. 33.
    Zhang, S., Karunamuni, R.J.: On kernel density estimation near endpoints. Journal of Statistical Planning and Inference 70, 301–316 (1998)MathSciNetzbMATHCrossRefGoogle Scholar
  34. 34.
    Zhang, S., Karunamuni, R.J.: Deconvolution boundary kernel method in nonparametric density estimation. Journal of Statistical Planning and Inference 139, 2269–2283 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  35. 35.
    Zhang, S., Karunamuni, R.J.: Boundary performance of the beta kernel estimators. Nonparametric Statistics 22, 81–104 (2010)MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Tomasz Gałkowski
    • 1
  1. 1.Institute of Computational IntelligenceCzestochowa University of TechnologyCzestochowaPoland

Personalised recommendations