Skip to main content
SpringerLink
Log in

Orthogonal Series Estimation of Regression Functions in Nonstationary Conditions

  • Conference paper
Artificial Intelligence and Soft Computing (ICAISC 2015)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9119))

Included in the following conference series:

Abstract

The article concerns of the problem of regression functions estimation when the output is contaminated by additive nonstationary noise. We investigate the model \(y_i = R\left( {{\bf x _i}} \right) + Z _i ,\,i = 1,2, \ldots n\), where x i is assumed to be the set of deterministic inputs (d-dimensional vector), y i is the scalar, probabilistic outputs, and Z i is a measurement noise with zero mean and variance depending on n. \(R\left( . \right)\) is a completely unknown function. The problem of finding function \(R\left( . \right)\) may be solved by applying non-parametric methodology, for instance: algorithms based on the Parzen kernel or algorithms derived from orthogonal series. In this work we present the orthogonal series approach. The analysis has been made for some class of nonstationarity. We present the conditions of convergence of the estimation algorithm for the variance of noise growing up when number of observations is tending to infinity. The results of numerical simulations are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Apolloni, B., Bassis, S., Mesiano, C., Rinaudo, S., Ciccazzo, A., Marotta, A.: Statistical Parameter Identification of Analog Integrated Circuit Reverse Models. Journal of Artificial Intelligence and Soft Computing Research 1(2), 115–134 (2011)

    Google Scholar 

  2. Chaibakhsh, A., Chaibakhsh, N., Abbasi, M., Norouzi, A.: Orthonormal Basis Function Fuzzy Systems for Biological Wastewater Treatment Processes Modeling. Journal of Artificial Intelligence and Soft Computing Research 2(4), 343–356 (2012)

    Google Scholar 

  3. Chu, J.L., Krzyzak, A.: The recognition of partially Occluded Objects With Support Vector Machines, Convolutional Neural Networks and Deep Belief Networks. Journal of Artificial Intelligence and Soft Computing Research 4(1), 5–19 (2014)

    Article  Google Scholar 

  4. Cpałka, K., Rutkowski, L.: Flexible Takagi-Sugeno Fuzzy Systems. In: Proceedings of the International Joint Conference on Neural Networks 2005, Montreal, pp. 1764–1769 (2005)

    Google Scholar 

  5. Cpałka, K., Rutkowski, L.: A New Method for Designing and Reduction of Neuro-fuzzy Systems. In: Proceedings of the 2006 IEEE International Conference on Fuzzy Systems (IEEE World Congress on Computational Intelligence, WCCI 2006), Vancouver, BC, Canada, pp. 8510–8516 (2006)

    Google Scholar 

  6. Galkowski, T., Rutkowski, L.: Nonparametric recovery of multivariate functions with applications to system identification. Proceedings of the IEEE 73, 942–943 (1985)

    Article  Google Scholar 

  7. Galkowski, T., Rutkowski, L.: Nonparametric fitting of multivariable functions. IEEE Transactions on Automatic Control AC-31, 785–787 (1986)

    Article  Google Scholar 

  8. Galkowski, T.: Nonparametric estimation of boundary values of functions. Archives of Control Science 3(1-2), 85–93 (1994)

    MathSciNet  MATH  Google Scholar 

  9. Gałkowski, T.: Kernel estimation of regression functions in the boundary regions. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013, Part II. LNCS (LNAI), vol. 7895, pp. 158–166. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  10. Galkowski, T., Pawlak, M.: Nonparametric Function Fitting in the Presence of Nonstationary Noise. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2014, Part I. LNCS (LNAI), vol. 8467, pp. 531–538. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  11. Gasser, T., Muller, H.G.: Kernel estimation of regression functions. Lecture Notes in Mathematics, vol. 757, pp. 23–68. Springer, Heidelberg (1979)

    Google Scholar 

  12. Greblicki, W., Rutkowski, L.: Density-free Bayes risk consistency of nonparametric pattern recognition procedures. Proceedings of the IEEE 69(4), 482–483 (1981)

    Article  Google Scholar 

  13. Greblicki, W., Rutkowska, D., Rutkowski, L.: An orthogonal series estimate of time-varying regression. Annals of the Institute of Statistical Mathematics 35(2), 215–228 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  14. Greblicki, W., Pawlak, M.: Fourier and Hermite series estimates of regression functions. Ann. of the Inst. of Stat. Math. 37(3A), 443–453 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  15. Hashimoto, K., Doki, K., Doki, S.: Estimation of Next Human Action and Its Timing Based on The Human Action Model Considering Time Series Information of The Situation. Journal of Artificial Intelligence and Soft Computing Research 2(3), 223–233 (2012)

    Google Scholar 

  16. Jaworski, M., Duda, P., Pietruczuk, L.: On fuzzy clustering of data streams with concept drift. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2012, Part II. LNCS (LNAI), vol. 7268, pp. 82–91. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  17. Korytkowski, M., Rutkowski, L., Scherer, R.: On combining backpropagation with boosting. In: IEEE International Joint Conference on Neural Network (IJCNN) Proceedings, vols. 1-10, Vancouver, July 16-21, pp. 1274–1277 (2006)

    Google Scholar 

  18. Korytkowski, M., Rutkowski, L., Scherer, R.: From Ensemble of Fuzzy Classifiers to Single Fuzzy Rule Base Classifier. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2008. LNCS (LNAI), vol. 5097, pp. 265–272. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  19. Korytkowski, M., Nowicki, R., Rutkowski, L., Scherer, R.: AdaBoost Ensemble of DCOG Rough–Neuro–Fuzzy Systems. In: Jędrzejowicz, P., Nguyen, N.T., Hoang, K. (eds.) ICCCI 2011, Part I. LNCS, vol. 6922, pp. 62–71. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  20. Kroll, A.: On choosing the fuzziness parameter for identifying TS models with multidimensional membership functions. Journal of Artificial Intelligence and Soft Computing Research 1(4), 283–300 (2011)

    Google Scholar 

  21. Lobato, F.S., Steffen Jr., V., Neto, A.J.S.: Solution of singular optimal control problems using the improved differential evolution algorithm. Journal of Artificial Intelligence and Soft Computing Research 1(3), 195–206 (2011)

    Google Scholar 

  22. Nowicki, R.: Rough Neuro-Fuzzy Structures for Classification With Missing Data. IEEE Transactions on Systems Man and Cybernetics Part B-Cybernetics 39(6), 1334–1347 (2009)

    Article  Google Scholar 

  23. Parzen, E.: On estimation of a probability density function and mode. Analysis of Mathematical Statistics 33(3), 1065–1076 (1962)

    Article  MathSciNet  MATH  Google Scholar 

  24. Pietruczuk, L., Duda, P., Jaworski, M.: A new fuzzy classifier for data streams. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2012, Part I. LNCS (LNAI), vol. 7267, pp. 318–324. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  25. Rafajlowicz, E.: Nonparametric orthogonal series estimators of regression - A class attaining the optimal convergence in L2. Statistics and Probability Letters 5(3), 219–224 (1987)

    Article  MathSciNet  Google Scholar 

  26. Rafajlowicz, E., Schwabe, R.: Halton and Hammersley sequences in multivariate nonparametric regression. Statistics and Probability Letters 76(8), 803–812 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  27. Rutkowski, L.: Sequential estimates of probability densities by orthogonal series and their application in pattern classification. IEEE Transactions on Systems, Man, and Cybernetics SMC-10(12), 918–920 (1980)

    MathSciNet  Google Scholar 

  28. Rutkowski, L.: On Bayes risk consistent pattern recognition procedures in a quasi-stationary environment. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-4(1), 84–87 (1982)

    Article  MathSciNet  Google Scholar 

  29. Rutkowski, L.: On-line identification of time-varying systems by nonparametric techniques. IEEE Transactions on Automatic Control AC-27, 228–230 (1982)

    Article  Google Scholar 

  30. Rutkowski, L.: On system identification by nonparametric function fitting. IEEE Trans. on Autom. Control AC-27, 225–227 (1982)

    Article  Google Scholar 

  31. Rutkowski, L.: Orthogonal series estimates of a regression function with application in system identification. In: Probab. and Statist. Inference, pp. 343–347. D. Reidel Publ. Comp. (1982)

    Google Scholar 

  32. Rutkowski, L.: On nonparametric identification with prediction of time-varying systems. IEEE Transactions on Automatic Control AC-29, 58–60 (1984)

    Article  Google Scholar 

  33. Rutkowski, L.: Nonparametric identification of quasi-stationary systems. Systems and Control Letters 6, 33–35 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  34. Rutkowski, L.: The real-time identification of time-varying systems by non-parametric algorithms based on Parzen kernels. International Journal of Systems Science 16, 1123–1130 (1985)

    Article  MATH  Google Scholar 

  35. Rutkowski, L.: A general approach for nonparametric fitting of functions and their derivatives with applications to linear circuits identification. IEEE Transactions on Circuits Systems CAS-33, 812–818 (1986)

    Article  Google Scholar 

  36. Rutkowski, L.: Sequential pattern recognition procedures derived from multiple Fourier series. Pattern Recognition Letters 8, 213–216 (1988)

    Article  MATH  Google Scholar 

  37. Rutkowski, L.: An application of multiple Fourier series to identification of multivariable nonstationary systems. International Journal of Systems Science 20(10), 1993–2002 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  38. Rutkowski, L.: Non-parametric learning algorithms in the time-varying environments. Signal Processing 18(2), 129–137 (1989)

    Article  MathSciNet  Google Scholar 

  39. Rutkowski, L., Rafajlowicz, E.: On optimal global rate of convergence of some nonparametric identification procedures. IEEE Transaction on Automatic Control AC-34(10), 1089–1091 (1989)

    Article  MathSciNet  Google Scholar 

  40. Rutkowski, L.: Identification of MISO nonlinear regressions in the presence of a wide class of disturbances. IEEE Transactions on Information Theory IT-37, 214–216 (1991)

    Article  MathSciNet  Google Scholar 

  41. Rutkowski, L.: Multiple Fourier series procedures for extraction of nonlinear regressions from noisy data. IEEE Transactions on Signal Processing 41(10), 3062–3065 (1993)

    Article  MATH  Google Scholar 

  42. Rutkowski, L.: Adaptive probabilistic neural-networks for pattern classification in time-varying environment. IEEE Trans. Neural Networks 15, 811–827 (2004)

    Article  Google Scholar 

  43. Rutkowski, L.: Generalized regression neural networks in time-varying environment. IEEE Trans. Neural Networks 15, 576–596 (2004)

    Article  Google Scholar 

  44. Rutkowski, L., Przybył, A., Cpałka, K., Er, M.J.: Online Speed Profile Generation for Industrial Machine Tool Based on Neuro-fuzzy Approach. In: Rutkowski, L., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2010, Part II. LNCS (LNAI), vol. 6114, pp. 645–650. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  45. Rutkowski, L., Przybyl, A., Cpałka, K.: Novel on-line speed profile generation for industrial machine tool based on flexible neuro-fuzzy approximation. IEEE Transactions on Industrial Electronics 59, 1238–1247 (2012)

    Article  Google Scholar 

  46. Rutkowski, L., Pietruczuk, L., Duda, P., Jaworski, M.: Decision trees for mining data streams based on the McDiarmid’s bound. IEEE Transactions on Knowledge and Data Engineering 25(6), 1272–1279 (2013)

    Article  Google Scholar 

  47. Rutkowski, L., Jaworski, M., Pietruczuk, L., Duda, P.: Decision trees for mining data streams based on the gaussian approximation. IEEE Transactions on Knowledge and Data Engineering 26(1), 108–119 (2014)

    Article  Google Scholar 

  48. Rutkowski, L., Jaworski, M., Pietruczuk, L., Duda, P.: The CART decision tree for mining data streams. Information Sciences 266, 1–15 (2014)

    Article  Google Scholar 

  49. Starczewski, A.: A cluster validity index for hard clustering. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2012, Part II. LNCS (LNAI), vol. 7268, pp. 168–174. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  50. Starczewski, J., Rutkowski, L.: Interval type 2 neuro-fuzzy systems based on interval consequents. In: Rutkowski, L., Kacprzyk, J. (eds.) Neural Networks and Soft Computing, pp. 570–577. Physica-Verlag, Springer-Verlag Company, Heidelberg, New York (2003)

    Chapter  Google Scholar 

  51. Starczewski, J., Rutkowski, L.: Connectionist Structures of Type 2 Fuzzy Inference Systems. In: Wyrzykowski, R., Dongarra, J., Paprzycki, M., Waśniewski, J. (eds.) PPAM 2001. LNCS, vol. 2328, pp. 634–642. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  52. Starczewski, J., Scherer, R., Korytkowski, M., Nowicki, R.: Modular Type-2 Neuro-fuzzy Systems. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Wasniewski, J. (eds.) PPAM 2007. LNCS, vol. 4967, pp. 570–578. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  53. Theodoridis Dimitris, C., Boutalis Yiannis, S., Christodoulou Manolis, A.: Robustifying analysis of the direct adaptive control of unknown multivariable nonlinear systems based on a new neuro-fuzzy method. Journal of Artificial Intelligence and Soft Computing Research 1(1), 59–79 (2011)

    Google Scholar 

  54. Tran, V.N., Brdys, M.A.: Optimizing Control by Robustly Feasible Model Predictive Control and Application to Drinkwater Distribution Systems. Journal of Artificial Intelligence and Soft Computing Research 1(1), 43–57 (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Computational Intelligence, Czestochowa University of Technology, Czestochowa, Poland

    Tomasz Galkowski

  2. Information Technology Institute, University of Social Sciences, Lodz, Poland

    Miroslaw Pawlak

  3. Department of Electrical and Computer Engineering, University of Manitoba, Winnipeg, Canada

    Miroslaw Pawlak

Authors
  1. Tomasz Galkowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Miroslaw Pawlak
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tomasz Galkowski .

Editor information

Editors and Affiliations

  1. Częstochowa University of Technology, Częstochowa, Poland

    Leszek Rutkowski

  2. Częstochowa University of Technology, Częstochowa, Poland

    Marcin Korytkowski

  3. Częstochowa University of Technology, Częstochowa, Poland

    Rafal Scherer

  4. AGH University of Science and Technology, Krakow, Poland

    Ryszard Tadeusiewicz

  5. University of California, Berkeley, California, USA

    Lotfi A. Zadeh

  6. University of Louisville, Louisville, Kentucky, USA

    Jacek M. Zurada

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Galkowski, T., Pawlak, M. (2015). Orthogonal Series Estimation of Regression Functions in Nonstationary Conditions. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L., Zurada, J. (eds) Artificial Intelligence and Soft Computing. ICAISC 2015. Lecture Notes in Computer Science(), vol 9119. Springer, Cham. https://doi.org/10.1007/978-3-319-19324-3_39

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-19324-3_39

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-19323-6

  • Online ISBN: 978-3-319-19324-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation