Advertisement

Improvement of RBF Training by Removing of Selected Pattern

  • Pawel RozyckiEmail author
  • Janusz Kolbusz
  • Oleksandr Lysenko
  • Bogdan M. Wilamowski
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10245)

Abstract

Number of training patterns has a huge impact on artificial neural networks training process, not only because of time-consuming aspects but also on network capacities. During training process the error for the most patterns reaches low error very fast and is hold to the end of training so can be safely removed without prejudice to further training process. Skilful removal of patterns during training allow to achieve better training results decreasing both training time and training error. The paper presents some implementations of this approach for Error Correction algorithm and RBF networks. The effectiveness of proposed methods has been confirmed by several experiments.

Keywords

ANN training improvement Error correction Training set reduction 

References

  1. 1.
    Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323, 533–6 (1986)CrossRefGoogle Scholar
  2. 2.
    Fahlman, S.E., Lebiere, C.: The cascade-correlation learning architecture. In: Advances in Neural Information Processing Systems 2. pp. 524–532. Morgan Kaufmann, San Mateo (1990)Google Scholar
  3. 3.
    Wilamowski, B.M., Bo, W., Korniak, J.: Big data and deep learning. In: 20th Jubilee IEEE International Conference on Intelligent Engineering Systems (INES 2016), 30 June–July 2, pp. 11–16 (2016)Google Scholar
  4. 4.
    Lang, K.L., Witbrock, M.J.: Learning to tell two spirals apart. In: Proceedings of the 1988 Connectionists Models Summer School. Morgan Kaufman (1998)Google Scholar
  5. 5.
    Wilamowski, B.M., Korniak, J.: Learning architectures with enhanced capabilities and easier training. In: 19th IEEE International Conference on Intelligent Engineering Systems (INES 2015), 03–05 September, pp. 21–29 (2015)Google Scholar
  6. 6.
    Bengio, Y.: Learning deep architectures for AI. Found. Trends in Mach. Learn. 2(1), 1127 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Ciresan, D.C., Meier, U., Gambardella, L.M., Schmidhuber, J.: Deep big simple neural nets excel on handwritten digit recognition. CoRR (2010)Google Scholar
  8. 8.
    Wilamowski, B.M., Yu, H.: Neural network learning without backpropagation. IEEE Trans. Neural Netw. 21(11), 1793–80 (2010)CrossRefGoogle Scholar
  9. 9.
    Werbo, P.J.: Back-propagation: past and future. In: Proceeding of International Conference on Neural Networks, San Diego, CA, vol. 1, pp. 343–354 (1988)Google Scholar
  10. 10.
    Yu, H., Reiner, P., Xie, T., Bartczak, T., Wilamowski, B.M.: An incremental design of radial basis function networks. IEEE Trans. Neural Netw. Learn. Syst. 25(10), 1793–80 (2014)CrossRefGoogle Scholar
  11. 11.
    Nguyen, G.H., Bouzerdoum, A., Phung, S.L.: Efficient supervised learning with reduced training exemplars. In: 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), Hong Kong, pp. 2981–2987 (2008)Google Scholar
  12. 12.
    Lozano, M.T.: Data reduction techniques in classification processes. Ph.D. Dissertation, Universitat Jaume I, Spain (2007)Google Scholar
  13. 13.
    Chouvatut, V., Jindaluang, W., Boonchieng, E.: Training set size reduction in large dataset problems. In: 2015 International Computer Science and Engineering Conference (ICSEC), Chiang Mai, pp. 1–5 (2015)Google Scholar
  14. 14.
    Kolbusz, J., Rozycki, P.: Outliers elimination for error correction algorithm improvement. In: CS&P Proceedings 24th International Workshop Concurrency, Specification & Programming, (CS & P 2015), vol. 2, pp. 120–129 (2015)Google Scholar
  15. 15.
    Xie, T.: Growing and learning algorithms of radial basis function networks. Ph.D. Dissertation, Auburn University, USA (2013)Google Scholar
  16. 16.
    Dieterich, J., Hartke, B.: Empirical review of standard benchmark functions using evolutionary global optimization. Appl. Math. 3(10A), 1552–64 (2012)CrossRefGoogle Scholar
  17. 17.
    Wilamowski, B.M., Jaeger, R.C.: Implementation of RBF type networks by MLP networks. In: IEEE International Conference on Neural Networks (ICNN 96), pp. 1670–1675 (1996)Google Scholar
  18. 18.
    Wu, X., Wilamowski, B.M.: Advantage analysis of sigmoid based RBF networks. In: Proceedings of the 17th IEEE International Conference on Intelligent Engineering Systems (INES 13), pp. 243–248 (2013)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Pawel Rozycki
    • 1
    Email author
  • Janusz Kolbusz
    • 1
  • Oleksandr Lysenko
    • 1
  • Bogdan M. Wilamowski
    • 2
  1. 1.University of Information Technology and Management in RzeszowRzeszowPoland
  2. 2.Auburn UniversityAuburnUSA

Personalised recommendations