Advertisement

On the Global Convergence of the Parzen-Based Generalized Regression Neural Networks Applied to Streaming Data

  • Jinde Cao
  • Leszek RutkowskiEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10841)

Abstract

In the paper we study global (integral) properties of the Parzen-type recursive algorithm dealing with streaming data in the presence of the time-varying noise. The mean integrated squared error of the regression estimate is shown to converge under several conditions. Simulations results illustrate asymptotic properties of the algorithm and its convergence for a wide spectrum of a time-varying noise.

Keywords

Stream data mining Parzen-type estimator Global convergence Generalized regression neural network 

Notes

Acknowledgments

This work was supported by the Polish National Science Center under Grant No. 2014/15/B/ST7/05264.

References

  1. 1.
    Bologna, G., Hayashi, Y.: Characterization of symbolic rules embedded in deep DIMLP networks: a challenge to transparency of deep learning. J. Artif. Intell. Soft Comput. Res. 7(4), 265–286 (2017)CrossRefGoogle Scholar
  2. 2.
    Cao, J., Wang, J.: Global asymptotic stability of a general class of recurrent neural networks with time-varying delays. IEEE Trans. Circ. Syst. I Fundam. Theory Appl. 50(1), 34–44 (2003)MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Cao, J., Wang, J.: Global asymptotic and robust stability of recurrent neural networks with time delays. IEEE Trans. Circ. Syst. I Regul. Pap. 52(2), 417–426 (2005)MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Chang, O., Constante, P., Gordon, A., Singana, M.: A novel deep neural network that uses space-time features for tracking and recognizing a moving object. J. Artif. Intell. Soft Comput. Res. 7(2), 125–136 (2017)CrossRefGoogle Scholar
  5. 5.
    Devi, V.S., Meena, L.: Parallel MCNN (pMCNN) with application to prototype selection on large and streaming data. J. Artif. Intell. Soft Comput. Res. 7(3), 155–169 (2017)CrossRefGoogle Scholar
  6. 6.
    Devroye, L., Krzyżak, A.: On the hilbert kernel density estimate. Stat. Probab. Lett. 44(3), 299–308 (1999)MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Devroye, L., Krzyżak, A.: New multivariate product density estimators. J. Multivar. Anal. 82(1), 88–110 (2002)MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Diam, A., Last, M., Kandel, A.: Knowledge discovery in data streams with regression tree methods. WIREs Data Min. Knowl. Discov. 2, 69–78 (2012).  https://doi.org/10.1002/widm.51CrossRefGoogle Scholar
  9. 9.
    Ditzler, G., Roveri, M., Alippi, C., Polikar, R.: Learning in nonstationary environments: a survey. IEEE Comput. Intell. Mag. 10(4), 12–25 (2015)CrossRefGoogle Scholar
  10. 10.
    Domingos, P., Hulten, G.: Mining high-speed data streams. In: Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 71–80 (2000)Google Scholar
  11. 11.
    Duda, P., Jaworski, M., Pietruczuk, L., Rutkowski, L.: A novel application of Hoeffding’s inequality to decision trees construction for data streams. In: 2014 International Joint Conference on Neural Networks (IJCNN), pp. 3324–3330. IEEE (2014)Google Scholar
  12. 12.
    Duda, P., Jaworski, M., Rutkowski, L.: Knowledge discovery in data streams with the orthogonal series-based generalized regression neural networks. Inf. Sci. (2017).  https://doi.org/10.1016/j.ins.2017.07.013MathSciNetCrossRefGoogle Scholar
  13. 13.
    Duda, P., Jaworski, M., Rutkowski, L.: Convergent time-varying regression models for data streams: tracking concept drift by the recursive parzen-based generalized regression neural networks. Int. J. Neural Syst. 28(02), 1750048 (2018)CrossRefGoogle Scholar
  14. 14.
    Duda, P., Pietruczuk, L., Jaworski, M., Krzyzak, A.: On the Cesàro-means-based orthogonal series approach to learning time-varying regression functions. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2016. LNCS (LNAI), vol. 9693, pp. 37–48. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-39384-1_4CrossRefGoogle Scholar
  15. 15.
    Ellis, P.: The time-dependent mean and variance of the non-stationary Markovian infinite server system. J. Math. Stat. 6, 68–71 (2010)zbMATHCrossRefGoogle Scholar
  16. 16.
    Greblicki, W., Pawlak, M.: Nonparametric System Identification. Cambridge University Press, Cambridge (2008)zbMATHCrossRefGoogle Scholar
  17. 17.
    Hulten, G., Spencer, L., Domingos, P.: Mining time-changing data streams. In: Proceedings of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 97–106 (2001)Google Scholar
  18. 18.
    Jaworski, M., Duda, P., Rutkowski, L., Najgebauer, P., Pawlak, M.: Heuristic regression function estimation methods for data streams with concept drift. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2017. LNCS (LNAI), vol. 10246, pp. 726–737. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-59060-8_65CrossRefGoogle Scholar
  19. 19.
    Jaworski, M., Duda, P., Rutkowski, L.: New splitting criteria for decision trees in stationary data streams. IEEE Trans. Neural Netw. Learn. Syst. PP(99), 1–14 (2017).  https://doi.org/10.1109/TNNLS.2017.2698204CrossRefGoogle Scholar
  20. 20.
    Krawczyk, B., Minku, L.L., Gama, J., Stefanowski, J., Wozniak, M.: Ensemble learning for data stream analysis: a survey. Inf. Fusion 37, 132–156 (2017)CrossRefGoogle Scholar
  21. 21.
    Li, R., Cao, J., Alsaedi, A., Alsaadi, F.: Exponential and fixed-time synchronization of cohen-grossberg neural networks with time-varying delays and reaction-diffusion terms. Appl. Math. Comput. 313, 37–51 (2017)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Manivannan, R., Samidurai, R., Cao, J., Alsaedi, A., Alsaadi, F.E.: Delay-dependent stability criteria for neutral-type neural networks with interval time-varying delay signals under the effects of leakage delay. Adv. Differ. Equ. 2018(1), 53 (2018)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Phillips, P.C.: Impulse response and forecast error variance asymptotics in nonstationary VARs. J. Econom. 83(1), 21–56 (1998)MathSciNetzbMATHCrossRefGoogle Scholar
  24. 24.
    Pietruczuk, L., Rutkowski, L., Jaworski, M., Duda, P.: The Parzen kernel approach to learning in non-stationary environment. In: 2014 International Joint Conference on Neural Networks (IJCNN), pp. 3319–3323. IEEE (2014)Google Scholar
  25. 25.
    Pietruczuk, L., Rutkowski, L., Jaworski, M., Duda, P.: How to adjust an ensemble size in stream data mining? Inf. Sci. 381, 46–54 (2017)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Riid, A., Preden, J.S.: Design of fuzzy rule-based classifiers through granulation and consolidation. J. Artif. Intell. Soft Comput. Res. 7(2), 137–147 (2017)CrossRefGoogle Scholar
  27. 27.
    Rutkowski, L.: New Soft Computing Techniques for System Modeling, Pattern Classication and Image Processing. Springer, Heidelberg (2004).  https://doi.org/10.1007/978-3-540-40046-2zbMATHCrossRefGoogle Scholar
  28. 28.
    Rutkowski, L., Jaworski, M., Pietruczuk, L., Duda, P.: The CART decision tree for mining data streams. Inf. Sci. 266, 1–15 (2014)zbMATHCrossRefGoogle Scholar
  29. 29.
    Rutkowski, L., Jaworski, M., Pietruczuk, L., Duda, P.: Decision trees for mining data streams based on the Gaussian approximation. IEEE Trans. Knowl. Data Eng. 26(1), 108–119 (2014)zbMATHCrossRefGoogle Scholar
  30. 30.
    Rutkowski, L., Jaworski, M., Pietruczuk, L., Duda, P.: A new method for data stream mining based on the misclassification error. IEEE Trans. Neural Netw. Learn. Syst, 26(5), 1048–1059 (2015)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Rutkowski, L., Pietruczuk, L., Duda, P., Jaworski, M.: Decision trees for mining data streams based on the McDiarmid’s bound. IEEE Trans. Knowl. Data Eng. 25(6), 1272–1279 (2013)CrossRefGoogle Scholar
  32. 32.
    Specht, D.F.: A general regression neural network. IEEE Trans. Neural Netw. 2(6), 568–576 (1991)CrossRefGoogle Scholar
  33. 33.
    Street, W.N., Kim, Y.: A streaming ensemble algorithm (SEA) for large-scale classification. In: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 377–382. ACM (2001)Google Scholar
  34. 34.
    Villmann, T., Bohnsack, A., Kaden, M.: Can learning vector quantization be an alternative to SVM and deep learning? - recent trends and advanced variants of learning vector quantization for classification learning. J. Artif. Intell. Soft Comput. Res. 7(1), 65–81 (2017).  https://doi.org/10.1515/jaiscr-2017-0005CrossRefGoogle Scholar
  35. 35.
    Wong, K.F.K., Galka, A., Yamashita, O., Ozaki, T.: Modelling non-stationary variance in EEG time series by state space garch model. Comput. Biol. Med. 36(12), 1327–1335 (2006)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of MathematicsSoutheast UniversityNanjingChina
  2. 2.Institute of Computational IntelligenceCzestochowa University of TechnologyCzestochowaPoland
  3. 3.Information Technology InstituteAcademy of Social SciencesŁódźPoland

Personalised recommendations