Application of the Givens Rotations in the Neural Network Learning Algorithm

  • Jarosław Bilski
  • Bartosz Kowalczyk
  • Jacek M. Żurada
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9692)

Abstract

This paper presents application of Givens rotations in the process of learning feedforward artificial neural network. This approach is based on QR decomposition. The paper describes mathematical background that needs to be considered during the application of the Givens rotations. The paper concludes with results of example simulations.

Keywords

Neural network training algorithm QR decomposition Givens rotation 

References

  1. 1.
    Chu, L.J., Krzyżak, A.: The recognition of partially occluded objects with support vector machines, convolutional neural networks and deep belief networks. J. Artif. Intell. Soft Comput. Res. 4(1), 5–19 (2014)CrossRefGoogle Scholar
  2. 2.
    Kiełbasiński, A., Schwetlick, H.: Numeryczna Algebra Liniowa. Wydawnictwa Naukowo-Techniczne (1992). (in Polish)Google Scholar
  3. 3.
    Bilski, J.: Momentum modification of the RLS algorithms. In: Rutkowski, L., Siekmann, J.H., Tadeusiewicz, R., Zadeh, L.A. (eds.) ICAISC 2004. LNCS (LNAI), vol. 3070, pp. 151–157. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  4. 4.
    Bilski, J.: Struktury równoległe dla jednokierunkowych i dynamicznych sieci neuronowych. Akademicka Oficyna Wydawnicza EXIT (2013). (in Polish)Google Scholar
  5. 5.
    Bilski, J., Smoląg, J.: Parallel realisation of the recurrent multi layer perceptron learning. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2012, Part I. LNCS (LNAI), vol. 7267, pp. 12–20. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  6. 6.
    Bilski, J., Smoląg, J.: Parallel approach to learning of the recurrent jordan neural network. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013, Part I. LNCS (LNAI), vol. 7894, pp. 32–40. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  7. 7.
    Bilski, J., Smoląg, J.: Parallel architectures for learning the RTRN and elman dynamic neural networks. IEEE Trans. Parallel Distrib. Syst. 26(9), 2561–2570 (2015)CrossRefGoogle Scholar
  8. 8.
    Werbos, J.: Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. Harvard University, Cambridge (1974)Google Scholar
  9. 9.
    Nowicki, R.K., Nowak, B.A., Starczewski, J.T., Cpalka, K.: The learning of neuro-fuzzy approximator with fuzzy rough sets in case of missing features. In: 2014 International Joint Conference on Neural Networks (IJCNN), pp. 3759–3766, July 2014Google Scholar
  10. 10.
    Bilski, J., Rutkowski, L.: A fast training algorithm for neural networks. IEEE Trans. Circ. Syst. Part II 45, 749–753 (1998)CrossRefGoogle Scholar
  11. 11.
    Bilski, J., Rutkowski, L.: Numerically robust learning algorithms for feed forward neural networks. In: Rutkowski, L., Kacprzyk, J. (eds.) Neural Networks and Soft Computing. Advances in Soft Computing, vol. 19, pp. 149–154. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  12. 12.
    Korytkowski, M., Scherer, R., Rutkowski, L.: On combining backpropagation with boosting. In: International Joint Conference on Neural Networks (2006)Google Scholar
  13. 13.
    Sakurai, S., Nishizawa, M.: A new approach for discovering Top-K sequential patterns based on the variety of items. J. Artif. Intell. Soft Comput. Res. 5(2), 141–153 (2015)CrossRefGoogle Scholar
  14. 14.
    Rutkowski, L., Jaworski, M., Pietruczuk, L., Duda, P.: A new method for data stream mining based on the misclassification error. IEEE Trans. Neural Netw. Learn. Syst. 26(5), 1048–1059 (2015)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Kitajima, R., Kamimura, R.: Accumulative information enhancement in the self-organizing maps and its application to the analysis of mission statements. J. Artif. Intell. Soft Comput. Res. 5(3), 161–176 (2015)CrossRefGoogle Scholar
  16. 16.
    Korytkowski, M., Rutkowski, L., Scherer, R.: Fast image classification by boosting fuzzy classifiers. Inform. Sci. 327, 175–182 (2016)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Nowicki, R.K., Korytkowski, M., Nowak, B.A., Scherer, R.: Design methodology for rough neuro-fuzzy classification with missing data. In: 2015 IEEE Symposium Series on Computational Intelligence, pp. 1650–1657, December 2015Google Scholar
  18. 18.
    Tadeusiewicz, R.: Sieci Neuronowe. Akademicka Oficyna Wydawnicza (1993)Google Scholar
  19. 19.
    Mleczko, W.K., Kapuscinski, T., Nowicki, R.K.: Rough deep belief network - application to incomplete handwritten digits pattern classification. In: Dregvaite, G., Damasevicius, R. (eds.) ICIST 2014. CCIS, vol. 538, pp. 400–411. Springer, Heidelberg (2015)Google Scholar
  20. 20.
    Nowak, B.A., Nowicki, R.K.: Learning in rough-neuro-fuzzy system for data with missing values. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Waśniewski, J. (eds.) PPAM 2011, Part I. LNCS, vol. 7203, pp. 501–510. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  21. 21.
    Aghdam, H.M., Heidari, S.: Feature selection using particle swarm optimization in text categorization. J. Artif. Intell. Soft Comput. Res. 5(4), 231–238 (2015)CrossRefGoogle Scholar
  22. 22.
    Hayashi, Y., Tanaka, Y., Takagi, T., Saito, T., Iiduka, H., Kikuchi, H., Bologna, G., Mitra, S.: Recursive-rule extraction algorithm with J48graft and applications to generating credit scores. J. Artif. Intell. Soft Comput. Res. 6(1), 35–44 (2016)CrossRefGoogle Scholar
  23. 23.
    Lee, P., Hsiao, T.: Applying LCS to affective image classification in spatial-frequency domain. J. Artif. Intell. Soft Comput. Res. 4(2), 99–123 (2014)CrossRefGoogle Scholar
  24. 24.
    Chen, Q., Abercrombie, K.R., Sheldon, T.F.: Risk assessment for industrial control systems quantifying availability using mean failure cost (MFC). J. Artif. Intell. Soft Comput. Res. 5(3), 205–220 (2015)CrossRefGoogle Scholar
  25. 25.
    Bilski, J., Smoląg, J., Żurada, J.M.: Parallel approach to the Levenberg-Marquardt learning algorithm for feedforward neural networks. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) Artificial Intelligence and Soft Computing. LNCS, vol. 9119, pp. 3–14. Springer, Heidelberg (2015)CrossRefGoogle Scholar
  26. 26.
    El-Samak, F.A., Ashour, W.: Optimization of traveling salesman problem using affinity propagation clustering and genetic algorithm. J. Artif. Intell. Soft Comput. Res. 5(4), 239–245 (2015)CrossRefGoogle Scholar
  27. 27.
    Knop, M., Kapuscinski, T., Mleczko, W.K.: Video key frame detection based on the restricted Boltzmann machine. J. Appl. Math. Comput. Mech. 14(3), 49–58 (2015)CrossRefGoogle Scholar
  28. 28.
    Nowak, B.A., Nowicki, R.K., Mleczko, W.K.: A new method of improving classification accuracy of decision tree in case of incomplete samples. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013, Part I. LNCS, vol. 7894, pp. 448–458. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  29. 29.
    Serdah, A.M., Ashour, W.M.: Clustering large-scale data based on modified affinity propagation algorithm. J. Artif. Intell. Soft Comput. Res. 6(1), 23–33 (2016)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Jarosław Bilski
    • 1
  • Bartosz Kowalczyk
    • 1
  • Jacek M. Żurada
    • 2
  1. 1.Institute of Computational IntelligenceCzȩstochowa University of TechnologyCzȩstochowaPoland
  2. 2.Department Electrical and Computer EngineeringUniversity of LouisvilleLouisvilleUSA

Personalised recommendations