Advertisement

Nonparametric Extension of Regression Functions Outside Domain

  • Tomasz Galkowski
  • Miroslaw Pawlak
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8467)

Abstract

The article refers to the problem of regression functions estimation in the points situated near the edges but outside of function domain. We investigate the model \(y_i = R\left( {x_i } \right) + \epsilon _i ,\,i = 1,2, \ldots n\), where x i is assumed to be the set of deterministic inputs, x i  ∈ D, y i is the set of probabilistic outputs, and ε i is a measurement noise with zero mean and bounded variance. R(.) is a completely unknown function. In the literature the possible ways of finding unknown function are based on the algorithms derived from the Parzen kernel. These algorithms were also applied to estimation of the derivatives of unknown functions. The commonly known disadvantage of the kernel algorithms is that the error of estimation dramatically increases if the point of estimation x is approaching to the left or right bound of interval D. Algorithms on predicting values in the boundary region outside the function domain D are unknown for the author, so far.

The main result of this paper is a new algorithm based on integral version of Parzen methods for local prediction of values of the function R near boundaries in the region outside domain. The results of numerical experiments are presented.

Keywords

Regression Function Model Predictive Control Edge Point Generalize Regression Neural Network Concept Drift 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Chen, S.X.: Beta kernel estimators for density functions. Journal of Statistical Planning and Inference 139, 2269–2283 (2009)CrossRefMathSciNetGoogle Scholar
  2. 2.
    Galkowski, T., Rutkowski, L.: Nonparametric recovery of multivariate functions with applications to system identification. Proceedings of the IEEE 73, 942–943 (1985)CrossRefGoogle Scholar
  3. 3.
    Galkowski, T., Rutkowski, L.: Nonparametric fitting of multivariable functions. IEEE Transactions on Automatic Control AC-31, 785–787 (1986)CrossRefGoogle Scholar
  4. 4.
    Galkowski, T.: Nonparametric estimates of parameters in a model of microbial growth process. AMSE Review 8(1), 1–10 (1988)Google Scholar
  5. 5.
    Galkowski, T.: On nonparametric fitting of higher order function derivatives by the kernel method - a simulation study. In: Gutierrez, R., Valderrama, M.J. (eds.) Fifth Int. Symp. Applied Stochastic Models and Data Analysis, Granada, Spain, pp. 230–242 (1991)Google Scholar
  6. 6.
    Galkowski, T.: Nonparametric estimation of boundary values of functions. Archives of Control Science 3(1-2), 85–93 (1994)zbMATHMathSciNetGoogle Scholar
  7. 7.
    Gałkowski, T.: Kernel estimation of regression functions in the boundary regions. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013, Part II. LNCS (LNAI), vol. 7895, pp. 158–166. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  8. 8.
    Gasser, T., Muller, H.G.: Kernel estimation of regression functions. Lecture Notes in Mathematics, vol. 757, pp. 23–68. Springer, Heidelberg (1979)Google Scholar
  9. 9.
    Gasser, T., Muller, H.G.: Estimating Regression Functions and Their Derivatives by the Kernel Method. Scandinavian Journal of Statistics 11(3), 171–185 (1984)zbMATHMathSciNetGoogle Scholar
  10. 10.
    Grbic, R., Kurtagic, D., Sliškovic, D.: Stream water temperature prediction based on Gaussian process regression. Expert Systems with Applications 40, 7407–7414 (2013)CrossRefGoogle Scholar
  11. 11.
    Greblicki, W., Rutkowski, L.: Density-free Bayes risk consistency of nonparametric pattern recognition procedures. Proceedings of the IEEE 69(4), 482–483 (1981)CrossRefGoogle Scholar
  12. 12.
    Greblicki, W., Rutkowska, D., Rutkowski, L.: An orthogonal series estimate of time-varying regression. Annals of the Institute of Statistical Mathematics 35(2), 215–228 (1983)CrossRefzbMATHMathSciNetGoogle Scholar
  13. 13.
    Hazelton, M.L., Marshall, J.C.: Linear boundary kernels for bivariate density estimation. Statistics and Probability Letters 79, 999–1003 (2009)CrossRefzbMATHMathSciNetGoogle Scholar
  14. 14.
    Jaworski, M., Duda, P., Pietruczuk, L.: On fuzzy clustering of data streams with concept drift. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2012, Part II. LNCS (LNAI), vol. 7268, pp. 82–91. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  15. 15.
    Karunamuni, R.J., Alberts, T.: On boundary correction in kernel density estimation. Statistical Methodology 2, 191–212 (2005)CrossRefzbMATHMathSciNetGoogle Scholar
  16. 16.
    Karunamuni, R.J., Alberts, T.: A locally adaptive transformation method of boundary correction in kernel density estimation. Journal of Statistical Planning and Inference 136, 2936–2960 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  17. 17.
    Korytkowski, M., Rutkowski, L., Scherer, R.: On combining backpropagation with boosting. In: IEEE International Joint Conference on Neural Network (IJCNN) Proceedings Vancouver, July 16-21, vols. 1-10, pp. 1274–1277 (2006)Google Scholar
  18. 18.
    Korytkowski, M., Rutkowski, L., Scherer, R.: From Ensemble of Fuzzy Classifiers to Single Fuzzy Rule Base Classifier. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2008. LNCS (LNAI), vol. 5097, pp. 265–272. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  19. 19.
    Kyung-Joon, C., Schucany, W.R.: Nonparametric kernel regression estimation near endpoints. Journal of Statistical Planning and Inference 66, 289–304 (1998)CrossRefzbMATHMathSciNetGoogle Scholar
  20. 20.
    Landberg, L.: Short-term prediction of local wind conditions. Journal of Wind Engineering and Industrial Aerodynamics 89, 235–245 (2001)CrossRefGoogle Scholar
  21. 21.
    Marshall, J.C., Hazelton, M.L.: Boundary kernels for adaptive density estimators on regions with irregular boundaries. Journal of Multivariate Analysis 101, 949–963 (2010)CrossRefzbMATHMathSciNetGoogle Scholar
  22. 22.
    Min, W., Wynter, L.: Real-time road traffic prediction with spatio-temporal correlations. Transportation Research Part C 19, 606–616 (2011)CrossRefGoogle Scholar
  23. 23.
    Mosallam, A., Medjaher, K., Zerhouni, N.: Nonparametric time series modelling for industrial prognostics and health management. Int. J. Adv. Manuf. Technol. 69, 1685–1699 (2013)CrossRefGoogle Scholar
  24. 24.
    Müller, H.G.: Smooth optimum kernel estimators near endpoints. Biometrika 78, 521–530 (1991)CrossRefzbMATHMathSciNetGoogle Scholar
  25. 25.
    Prampero Paulo, S., Romis, A.: Magnetic particle swarm optimization. Journal of Artificial Intelligence and Soft Computing Research 2(1), 59–72 (2012)Google Scholar
  26. 26.
    Parzen, E.: On estimation of a probability density function and mode. Analysis of Mathematical Statistics 33(3), 1065–1076 (1962)CrossRefzbMATHMathSciNetGoogle Scholar
  27. 27.
    Pietruczuk, L., Duda, P., Jaworski, M.: A new fuzzy classifier for data streams. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2012, Part I. LNCS (LNAI), vol. 7267, pp. 318–324. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  28. 28.
    Pietruczuk, L., Duda, P., Jaworski, M.: Adaptation of decision trees for handling concept drift. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013, Part I. LNCS (LNAI), vol. 7894, pp. 459–473. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  29. 29.
    Poměnková-Dluhá, J.: Edge Effects of Gasser-Müller Estimator. Mathematica 15, 307–314 (2004)Google Scholar
  30. 30.
    Rafajlowicz, E.: Nonparametric orthogonal series estimators of regression - A class attaining the optimal convergence in L2. Statistics and Probability Letters 5(3), 219–224 (1987)CrossRefzbMATHMathSciNetGoogle Scholar
  31. 31.
    Rafajlowicz, E., Schwabe, R.: Halton and Hammersley sequences in multivariate nonparametric regression. Statistics and Probability Letters 76(8), 803–812 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  32. 32.
    Rutkowski, L.: Sequential estimates of probability densities by orthogonal series and their application in pattern classification. IEEE Transactions on Systems, Man, and Cybernetics SMC-10(12), 918–920 (1980)MathSciNetGoogle Scholar
  33. 33.
    Rutkowski, L.: On Bayes risk consistent pattern recognition procedures in a quasi-stationary environment. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-4(1), 84–87 (1982)CrossRefMathSciNetGoogle Scholar
  34. 34.
    Rutkowski, L.: On-line identification of time-varying systems by nonparametric techniques. IEEE Transactions on Automatic Control AC-27, 228–230 (1982)CrossRefGoogle Scholar
  35. 35.
    Rutkowski, L.: On nonparametric identification with prediction of time-varying systems. IEEE Transactions on Automatic Control AC-29, 58–60 (1984)CrossRefGoogle Scholar
  36. 36.
    Rutkowski, L.: Nonparametric identification of quasi-stationary systems. Systems and Control Letters 6, 33–35 (1985)CrossRefzbMATHMathSciNetGoogle Scholar
  37. 37.
    Rutkowski, L.: The real-time identification of time-varying systems by non-parametric algorithms based on Parzen kernels. International Journal of Systems Science 16, 1123–1130 (1985)CrossRefzbMATHGoogle Scholar
  38. 38.
    Rutkowski, L.: A general approach for nonparametric fitting of functions and their derivatives with applications to linear circuits identification. IEEE Transactions Circuits Systems CAS-33, 812–818 (1986)CrossRefGoogle Scholar
  39. 39.
    Rutkowski, L.: Sequential pattern recognition procedures derived from multiple Fourier series. Pattern Recognition Letters 8, 213–216 (1988)CrossRefzbMATHGoogle Scholar
  40. 40.
    Rutkowski, L.: An application of multiple Fourier series to identification of multivariable nonstationary systems. International Journal of Systems Science 20(10), 1993–2002 (1989)CrossRefzbMATHMathSciNetGoogle Scholar
  41. 41.
    Rutkowski, L.: Non-parametric learning algorithms in the time-varying environments. Signal Processing 18(2), 129–137 (1989)CrossRefMathSciNetGoogle Scholar
  42. 42.
    Rutkowski, L., Rafajlowicz, E.: On optimal global rate of convergence of some nonparametric identification procedures. IEEE Transaction on Automatic Control AC-34(10), 1089–1091 (1989)CrossRefMathSciNetGoogle Scholar
  43. 43.
    Rutkowski, L.: Identification of MISO nonlinear regressions in the presence of a wide class of disturbances. IEEE Transactions on Information Theory IT-37, 214–216 (1991)CrossRefMathSciNetGoogle Scholar
  44. 44.
    Rutkowski, L.: Multiple Fourier series procedures for extraction of nonlinear regressions from noisy data. IEEE Transactions on Signal Processing 41(10), 3062–3065 (1993)CrossRefzbMATHGoogle Scholar
  45. 45.
    Rutkowski, L.: Adaptive probabilistic neural-networks for pattern classification in time-varying environment. IEEE Trans. Neural Networks 15, 811–827 (2004)CrossRefGoogle Scholar
  46. 46.
    Rutkowski, L.: Generalized regression neural networks in time-varying environment. IEEE Trans. Neural Networks 15, 576–596 (2004)CrossRefGoogle Scholar
  47. 47.
    Rutkowski, L., Przybyl, A., Cpalka, K.: Novel on-line speed profile generation for industrial machine tool based on flexible neuro-fuzzy approximation. IEEE Transactions on Industrial Electronics 59, 1238–1247 (2012)CrossRefGoogle Scholar
  48. 48.
    Rutkowski, L., Pietruczuk, L., Duda, P., Jaworski, M.: Decision trees for mining data streams based on the McDiarmid’s bound. IEEE Transactions on Knowledge and Data Engineering 25(6), 1272–1279 (2013)CrossRefGoogle Scholar
  49. 49.
    Rutkowski, L., Jaworski, M., Pietruczuk, L., Duda, P.: Decision trees for mining data streams based on the gaussian approximation. IEEE Transactions on Knowledge and Data Engineering 26(1), 108–119 (2014)CrossRefGoogle Scholar
  50. 50.
    Rutkowski, L., Jaworski, M., Pietruczuk, L., Duda, P.: The CART decision tree for mining data streams. Information Sciences 266, 1–15 (2014)Google Scholar
  51. 51.
    Starczewski, J., Rutkowski, L.: Interval type 2 neuro-fuzzy systems based on interval consequents. In: Rutkowski, L., Kacprzyk, J. (eds.) Neural Networks and Soft Computing, pp. 570–577. Physica-Verlag, Springer-Verlag Company, Heidelberg, New York (2003)Google Scholar
  52. 52.
    Starczewski, J.T., Rutkowski, L.: Connectionist Structures of Type 2 Fuzzy Inference Systems. In: Wyrzykowski, R., Dongarra, J., Paprzycki, M., Waśniewski, J. (eds.) PPAM 2001. LNCS, vol. 2328, pp. 634–642. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  53. 53.
    Schuster, E.F.: Incorporating Support Constraints Into Nonparametric Estimators of Densities. Communications in Statistics, Part A - Theory and Methods 14, 1123–1136 (1985)CrossRefzbMATHMathSciNetGoogle Scholar
  54. 54.
    Strang, G.: Calculus. MIT book, Wellesley-Cambridge Press (1991)Google Scholar
  55. 55.
    Vu Nam, T., Brdys Mietek, A.: Optimizing control by robustly feasible model predictive control and application to drinking water distribution systems. Journal of Artificial Intelligence and Soft Computing Research 1(1), 43–57 (2011)Google Scholar
  56. 56.
    Vilar, J.M., Cao, R., Aneiros, G.: Forecasting next-day electricity demand and price using nonparametric functional methods. Electrical Power and Energy Systems 39, 48–55 (2012)CrossRefGoogle Scholar
  57. 57.
    Zhang, J., Tan, Z.: Day-ahead electricity price forecasting using WT, CLSSVM and EGARCH model. Electrical Power and Energy Systems 45, 362–368 (2013)CrossRefMathSciNetGoogle Scholar
  58. 58.
    Zhang, S., Karunamuni, R.J.: On kernel density estimation near endpoints. Journal of Statistical Planning and Inference 70, 301–316 (1998)CrossRefzbMATHMathSciNetGoogle Scholar
  59. 59.
    Zhang, S., Karunamuni, R.J.: Deconvolution boundary kernel method in nonparametric density estimation. Journal of Statistical Planning and Inference 139, 2269–2283 (2009)CrossRefzbMATHMathSciNetGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Tomasz Galkowski
    • 1
  • Miroslaw Pawlak
    • 2
    • 3
  1. 1.Institute of Computational IntelligenceCzestochowa University of TechnologyCzestochowaPoland
  2. 2.Information Technology InstituteUniversity of Social SciencesLodzPoland
  3. 3.Department of Electrical and Computer EngineeringUniversity of ManitobaWinnipegCanada

Personalised recommendations