Advertisement

Nonparametric Function Fitting in the Presence of Nonstationary Noise

  • Tomasz Galkowski
  • Miroslaw Pawlak
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8467)

Abstract

The article refers to the problem of regression functions estimation in the presence of nonstationary noise. We investigate the model \(y_i = R\left( {{\bf x _i}} \right) + \epsilon _i ,\,i = 1,2, \ldots n\), where x i is assumed to be the d-dimensional vector, set of deterministic inputs, x i  ∈ S d, y i is the scalar, set of probabilistic outputs, and ε i is a measurement noise with zero mean and variance depending on n. \(R\left( . \right)\) is a completely unknown function. One of the possible solutions of finding function \(R\left( . \right)\) is to apply non-parametric methodology - algorithms based on the Parzen kernel or algorithms derived from orthogonal series. The novel result of this article is the analysis of convergence for some class of nonstationarity. We present the conditions when the algorithm of estimation is convergent even when the variance of noise is divergent with number of observations tending to infinity. The results of numerical experiments are presented.

Keywords

IEEE Transaction Generalize Regression Neural Network Concept Drift Orthogonal Series Nonparametric Function 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Galkowski, T., Rutkowski, L.: Nonparametric recovery of multivariate functions with applications to system identification. Proceedings of the IEEE 73, 942–943 (1985)CrossRefGoogle Scholar
  2. 2.
    Galkowski, T., Rutkowski, L.: Nonparametric fitting of multivariable functions. IEEE Transactions on Automatic Control AC-31, 785–787 (1986)CrossRefGoogle Scholar
  3. 3.
    Galkowski, T.: Nonparametric estimation of boundary values of functions. Archives of Control Science 3(1-2), 85–93 (1994)zbMATHMathSciNetGoogle Scholar
  4. 4.
    Gałkowski, T.: Kernel estimation of regression functions in the boundary regions. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013, Part II. LNCS (LNAI), vol. 7895, pp. 158–166. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  5. 5.
    Gasser, T., Muller, H.G.: Kernel estimation of regression functions. Lecture Notes in Mathematics, vol. 757, pp. 23–68. Springer, Heidelberg (1979)Google Scholar
  6. 6.
    Greblicki, W., Rutkowski, L.: Density-free Bayes risk consistency of nonparametric pattern recognition procedures. Proceedings of the IEEE 69(4), 482–483 (1981)CrossRefGoogle Scholar
  7. 7.
    Greblicki, W., Rutkowska, D., Rutkowski, L.: An orthogonal series estimate of time-varying regression. Annals of the Institute of Statistical Mathematics 35(2), 215–228 (1983)CrossRefzbMATHMathSciNetGoogle Scholar
  8. 8.
    Jaworski, M., Duda, P., Pietruczuk, L.: On fuzzy clustering of data streams with concept drift. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2012, Part II. LNCS, vol. 7268, pp. 82–91. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  9. 9.
    Korytkowski, M., Rutkowski, L., Scherer, R.: On combining backpropagation with boosting. In: IEEE International Joint Conference on Neural Network (IJCNN) Proceedings, Vancouver, July 16-21, vol. 1-10, pp. 1274–1277 (2006)Google Scholar
  10. 10.
    Korytkowski, M., Rutkowski, L., Scherer, R.: From Ensemble of Fuzzy Classifiers to Single Fuzzy Rule Base Classifier. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2008. LNCS (LNAI), vol. 5097, pp. 265–272. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  11. 11.
    Kroll, A.: On choosing the fuzziness parameter for identifying TS models with multidimensional membership functions. Journal of Artificial Intelligence and Soft Computing Research 1(4), 283–300 (2011)Google Scholar
  12. 12.
    Lobato, F.S., Steffen Jr., V., Neto, A.J.S.: Solution of singular optimal control problems using the improved differential evolution algorithm. Journal of Artificial Intelligence and Soft Computing Research 1(3), 195–206 (2011)Google Scholar
  13. 13.
    Parzen, E.: On estimation of a probability density function and mode. Analysis of Mathematical Statistics 33(3), 1065–1076 (1962)CrossRefzbMATHMathSciNetGoogle Scholar
  14. 14.
    Pietruczuk, L., Duda, P., Jaworski, M.: A new fuzzy classifier for data streams. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2012, Part I. LNCS (LNAI), vol. 7267, pp. 318–324. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  15. 15.
    Pietruczuk, L., Duda, P., Jaworski, M.: Adaptation of decision trees for handling concept drift. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013, Part I. LNCS (LNAI), vol. 7894, pp. 459–473. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  16. 16.
    Rafajlowicz, E.: Nonparametric orthogonal series estimators of regression - A class attaining the optimal convergence in L2. Statistics and Probability Letters 5(3), 219–224 (1987)CrossRefzbMATHMathSciNetGoogle Scholar
  17. 17.
    Rafajlowicz, E., Schwabe, R.: Halton and Hammersley sequences in multivariate nonparametric regression. Statistics and Probability Letters 76(8), 803–812 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  18. 18.
    Rutkowski, L.: Sequential estimates of probability densities by orthogonal series and their application in pattern classification. IEEE Transactions on Systems, Man, and Cybernetics SMC-10(12), 918–920 (1980)MathSciNetGoogle Scholar
  19. 19.
    Rutkowski, L.: On Bayes risk consistent pattern recognition procedures in a quasi-stationary environment. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-4(1), 84–87 (1982)CrossRefMathSciNetGoogle Scholar
  20. 20.
    Rutkowski, L.: On-line identification of time-varying systems by nonparametric techniques. IEEE Transactions on Automatic Control AC-27, 228–230 (1982)CrossRefGoogle Scholar
  21. 21.
    Rutkowski, L.: On nonparametric identification with prediction of time-varying systems. IEEE Transactions on Automatic Control AC-29, 58–60 (1984)CrossRefGoogle Scholar
  22. 22.
    Rutkowski, L.: Nonparametric identification of quasi-stationary systems. Systems and Control Letters 6, 33–35 (1985)CrossRefzbMATHMathSciNetGoogle Scholar
  23. 23.
    Rutkowski, L.: The real-time identification of time-varying systems by non-parametric algorithms based on Parzen kernels. International Journal of Systems Science 16, 1123–1130 (1985)CrossRefzbMATHGoogle Scholar
  24. 24.
    Rutkowski, L.: A general approach for nonparametric fitting of functions and their derivatives with applications to linear circuits identification. IEEE Transactions on Circuits Systems CAS-33, 812–818 (1986)CrossRefGoogle Scholar
  25. 25.
    Rutkowski, L.: Sequential pattern recognition procedures derived from multiple Fourier series. Pattern Recognition Letters 8, 213–216 (1988)CrossRefzbMATHGoogle Scholar
  26. 26.
    Rutkowski, L.: An application of multiple Fourier series to identification of multivariable nonstationary systems. International Journal of Systems Science 20(10), 1993–2002 (1989)CrossRefzbMATHMathSciNetGoogle Scholar
  27. 27.
    Rutkowski, L.: Non-parametric learning algorithms in the time-varying environments. Signal Processing 18(2), 129–137 (1989)CrossRefMathSciNetGoogle Scholar
  28. 28.
    Rutkowski, L., Rafajlowicz, E.: On optimal global rate of convergence of some nonparametric identification procedures. IEEE Transaction on Automatic Control AC-34(10), 1089–1091 (1989)CrossRefMathSciNetGoogle Scholar
  29. 29.
    Rutkowski, L.: Identification of MISO nonlinear regressions in the presence of a wide class of disturbances. IEEE Transactions on Information Theory IT-37, 214–216 (1991)CrossRefMathSciNetGoogle Scholar
  30. 30.
    Rutkowski, L.: Multiple Fourier series procedures for extraction of nonlinear regressions from noisy data. IEEE Transactions on Signal Processing 41(10), 3062–3065 (1993)CrossRefzbMATHGoogle Scholar
  31. 31.
    Rutkowski, L.: Adaptive probabilistic neural-networks for pattern classification in time-varying environment. IEEE Trans. Neural Networks 15, 811–827 (2004)CrossRefGoogle Scholar
  32. 32.
    Rutkowski, L.: Generalized regression neural networks in time-varying environment. IEEE Trans. Neural Networks 15, 576–596 (2004)CrossRefGoogle Scholar
  33. 33.
    Rutkowski, L., Przybyl, A., Cpalka, K.: Novel on-line speed profile generation for industrial machine tool based on flexible neuro-fuzzy approximation. IEEE Transactions on Industrial Electronics 59, 1238–1247 (2012)CrossRefGoogle Scholar
  34. 34.
    Rutkowski, L., Pietruczuk, L., Duda, P., Jaworski, M.: Decision trees for mining data streams based on the McDiarmid’s bound. IEEE Transactions on Knowledge and Data Engineering 25(6), 1272–1279 (2013)CrossRefGoogle Scholar
  35. 35.
    Rutkowski, L., Jaworski, M., Pietruczuk, L., Duda, P.: Decision trees for mining data streams based on the gaussian approximation. IEEE Transactions on Knowledge and Data Engineering 26(1), 108–119 (2014)CrossRefGoogle Scholar
  36. 36.
    Rutkowski, L., Jaworski, M., Pietruczuk, L., Duda, P.: The CART decision tree for mining data streams. Information Sciences 266, 1–15 (2014)Google Scholar
  37. 37.
    Starczewski, J., Rutkowski, L.: Interval type 2 neuro-fuzzy systems based on interval consequents. In: Rutkowski, L., Kacprzyk, J. (eds.) Neural Networks and Soft Computing, pp. 570–577. Physica-Verlag, Springer-Verlag Company, Heidelberg, New York (2003)Google Scholar
  38. 38.
    Starczewski, J.T., Rutkowski, L.: Connectionist Structures of Type 2 Fuzzy Inference Systems. In: Wyrzykowski, R., Dongarra, J., Paprzycki, M., Waśniewski, J. (eds.) PPAM 2001. LNCS, vol. 2328, pp. 634–642. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  39. 39.
    Theodoridis, D.C., Boutalis, Y.S., Christodoulou, M.A.: Robustifying analysis of the direct adaptive control of unknown multivariable nonlinear systems based on a new neuro-fuzzy method. Journal of Artificial Intelligence and Soft Computing Research 1(1), 59–79 (2011)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Tomasz Galkowski
    • 1
  • Miroslaw Pawlak
    • 2
    • 3
  1. 1.Institute of Computational IntelligenceCzestochowa University of TechnologyCzestochowaPoland
  2. 2.Information Technology InstituteUniversity of Social SciencesLodzPoland
  3. 3.Department of Electrical and Computer EngineeringUniversity of ManitobaWinnipegCanada

Personalised recommendations