Skip to main content
SpringerLink
Log in

A New Variant of the GQR Algorithm for Feedforward Neural Networks Training

  • Conference paper
  • First Online:
Artificial Intelligence and Soft Computing (ICAISC 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12854))

Included in the following conference series:

Abstract

This paper presents an application of the scaled Givens rotations in the process of feedforward artificial neural networks training. This method bases on the QR decomposition. The paper describes mathematical background that needs to be considered during the application of the scaled Givens rotations in neural networks training. The paper concludes with sample simulation results.

This work has been supported by the Polish National Science Center under Grant 2017/27/B/ST6/02852.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bilski, J.: Momentum modification of the RLS algorithms. In: Rutkowski, L., Siekmann, J.H., Tadeusiewicz, R., Zadeh, L.A. (eds.) ICAISC 2004. LNCS (LNAI), vol. 3070, pp. 151–157. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24844-6_18

    Chapter  Google Scholar 

  2. Bilski, J.: Parallel structures for feedforward and dynamic neural networks. (In Polish) Akademicka Oficyna Wydawnicza EXIT (2013)

    Google Scholar 

  3. Bilski, J., Kowalczyk, B., Grzanek, K.: The parallel modification to the Levenberg-Marquardt algorithm. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J.M. (eds.) ICAISC 2018. LNCS (LNAI), vol. 10841, pp. 15–24. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91253-0_2

  4. Bilski,, J., Kowalczyk, B., Żurada, J.M.: Application of the Givens rotations in the neural network learning algorithm. In: Artificial Intelligence and Soft Computing, volume 9602 of Lecture Notes in Artificial Intelligence, pp. 46–56. Springer-Verlag, Berlin, Heidelberg (2016). https://doi.org/10.1007/978-3-319-39378-0_5

  5. Bilski, J., Kowalczyk, B., Żurada, J.M.: Parallel implementation of the givens rotations in the neural network learning algorithm. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2017. LNCS (LNAI), vol. 10245, pp. 14–24. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59063-9_2

  6. Bilski, J., Smoląg, J.: Parallel realisation of the recurrent multi layer perceptron learning. Artificial Intelligence and Soft Computing, pp. 12–20. Springer-Verlag, Berlin, Heidelberg, (LNAI 7267) (2012). https://doi.org/10.1007/978-3-642-13232-2_3

  7. Bilski, J., Smoląg, J.: Parallel approach to learning of the recurrent Jordan neural network. Artificial Intelligence and Soft Computing, pp. 32–40. Springer-Verlag, Berlin, Heidelberg (LNAI 7895) (2013)

    Google Scholar 

  8. Bilski, J., Smoląg, J.: Parallel architectures for learning the RTRN and Elman dynamic neural network. IEEE Trans. Parallel Distrib. Syst. 26(9), 2561–2570 (2015)

    Article  Google Scholar 

  9. Bilski, J., Smoląg, J., Galushkin, A.I.: The parallel approach to the conjugate gradient learning algorithm for the feedforward neural networks. In: Artificial Intelligence and Soft Computing, volume 8467 of Lecture Notes in Computer Science, pp. 12–21. Springer-Verlag, Berlin, Heidelberg (2014). https://doi.org/10.1007/978-3-319-07173-2_2

  10. Bilski, J., Smoląg, J., Żurada, J.M.: Parallel approach to the Levenberg-Marquardt learning algorithm for feedforward neural networks. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2015. LNCS (LNAI), vol. 9119, pp. 3–14. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-19324-3_1

    Chapter  MATH  Google Scholar 

  11. Bilski, J., Kowalczyk, B., Marchlewska, A., Zurada, J.M.: Local levenberg-marquardt algorithm for learning feedforwad neural networks. J. Artif. Intell. Soft Comput. Res. 10(4), 299–316 (2020)

    Article  Google Scholar 

  12. Cao, Y., Samidurai, R., Sriraman, R.: Stability and dissipativity analysis for neutral type stochastic markovian jump static neural networks with time delays. J. Artif. Intell. Soft Comput. Res. 9(3), 189–204 (2019)

    Google Scholar 

  13. Costa, M., Oliveira, D., Pinto, S., Tavares, A.: Detecting driver’s fatigue, distraction and activity using a non-intrusive AI-based monitoring system. J. Artif. Intell. Soft Comput. Rese. 9(4), 247–266 (2019)

    Article  Google Scholar 

  14. de Souza, G.B., da Silva Santos, D.F., Pires, R.G., Marananil, A.N., Papa, J.P.: Deep features extraction for robust fingerprint spoofing attack detection. J. Artif. Intell. Soft Comput. Res. 9(1), 41–49 (2019)

    Google Scholar 

  15. Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12, 2121–2159 (2011)

    Google Scholar 

  16. Duda, P., Jaworski, M., Cader, A., Wang, L.: On training deep neural networks using a streaming approach. J. Artif. Intell. Soft Computi. Res. 10(1), 15–26 (2020)

    Article  Google Scholar 

  17. Gabryel, M., Grzanek, K., Hayashi, Y.: Browser fingerprint coding methods increasing the effectiveness of user identification in the web traffic. J. Artif. Intell. Soft Computi. Res. 10(4), 243–253 (2020)

    Article  Google Scholar 

  18. Gentleman, W.M.: Least squares computations by givens transformations without square roots. IMA J. Appl. Math. 12(3), 329–336 (1973)

    Google Scholar 

  19. Givens, W.: Computation of plain unitary rotations transforming a general matrix to triangular form. J. Soc. Indust. Appl. Math. 6, 26–50 (1958)

    Article  MathSciNet  Google Scholar 

  20. Grycuk, R., Najgebauer, P., Kordos, M., Scherer, M.M., Marchlewska, A.: Fast image index for database management engines. J. Artif. Intell. Soft Computi. Res. 10(2), 113–123 (2020)

    Article  Google Scholar 

  21. Grycuk, R., Wojciechowski, A., Wei, W., Siwocha, A.: Detecting visual objects by edge crawling. J. Artif. Intell. Soft Computi. Res. 10(3), 223–237 (2020)

    Article  Google Scholar 

  22. Hagan, M.T., Menhaj, M.B.: Training feedforward networks with the marquardt algorithm. IEEE Trans. Neuralnetw. 5, 989–993 (1994)

    Google Scholar 

  23. Hou, Y., Holder, L.B.: On graph mining with deep learning: introducing model r for link weight prediction. J. Artif. Intell. Soft Computi. Res. 9(1), 21–40 (2019)

    Article  Google Scholar 

  24. Kamimura, R.: Supposed maximum mutual information for improving generalization and interpretation of multi-layered neural networks. J. Artif. Intell. Soft Computi. Res. 9(2), 123–147 (2019)

    Article  Google Scholar 

  25. Kiełbasiński, A., Schwetlick, H.: Numeryczna Algebra Liniowa: Wprowadzenie do Obliczeń Zautomatyzowanych. Wydawnictwa Naukowo-Techniczne, Warszawa (1992)

    Google Scholar 

  26. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization (2014)

    Google Scholar 

  27. Krell, E., Sheta, A., Balasubramanian, A.P.R., King, S.A.: Collision-free autonomous robot navigation in unknown environments utilizing pso for path planning. J. Artif. Intell. Soft Comput. Res. 9(4), 267–282 (2019)

    Article  Google Scholar 

  28. Kumarratneshk, R., Weilleweill, E., Aghdasi, F., Sriram, P.: A strong and efficient baseline for vehicle re-identification using deep triplet embedding. J. Artif. Intell. Soft Comput. Res. 10(1), 27–45 (2020)

    Article  Google Scholar 

  29. Łapa, K., Cpałka, K., Wang, L.: New method for design of fuzzy systems for nonlinear modelling using different criteria of interpretability. In: Rutkowski, L., Korytkowski, M., Rafał Scherer, R., Tadeusiewicz, L.A.Z., Zurada, J.M. (eds.) Artificial Intelligence and Soft Computing, pp. 217–232. Springer International Publishing, Cham (2014). https://doi.org/10.1007/978-3-319-07173-2_20

  30. Ludwig, S.A.: Applying a neural network ensemble to intrusion detection. J. Artif. Intell. Soft Comput. Res. 9(3), 177–188 (2019)

    Article  Google Scholar 

  31. Abbas, M. Javaid, M., Jia-Bao, L., Teh, W.C., Jinde, C.: Topological properties of four-layered neural networks. J. Artif. Intell. Soft Comput. Res. 9(2), 111–122 (2019)

    Google Scholar 

  32. Nobukawa, S., Nishimura, H., Yamanishi, T.: Pattern classification by spiking neural networks combining self-organized and reward-related spike-timing-dependent plasticity. J. Artif. Intell. Soft Comput. Res. 9(4), 283–291 (2019)

    Article  Google Scholar 

  33. Nowicki, R.K., Grzanek, K., Hayashi, Y.: Rough support vector machine for classification with interval and incomplete data. J. Artif. Intell. Soft Comput. Res. 10(1), 47–56 (2020)

    Article  Google Scholar 

  34. Shewalkar, A., Nyavanandi, D., Ludwig, S.A.: Performance evaluation of deep neural networks applied to speech recognition: RNN, LSTM and GRU. J. Artif. Intell. Soft Comput. Res. 9(4), 235–245 (2019)

    Article  Google Scholar 

  35. Simões, D., Lau, N., Reis, L.P.: Multi agent deep learning with cooperative communication. J. Artif. Intell. Soft Comput. Res. 10(3), 189–207 (2020)

    Google Scholar 

  36. Szczypta, J., Przybył, A., Cpałka, K.: Some aspects of evolutionary designing optimal controllers. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013. LNCS (LNAI), vol. 7895, pp. 91–100. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38610-7_9

    Chapter  Google Scholar 

  37. Wei, Y., Ying, Yu., Lifeng, X., Huang, W., Guo, J., Wan, Y., Cao, J.: Vehicle emission computation through microscopic traffic simulation calibrated using genetic algorithm. J. Artif. Intell. Soft Comput. Res. 9(1), 67–80 (2019)

    Article  Google Scholar 

  38. Werbos, J.: Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. Harvard University (1974)

    Google Scholar 

  39. Zalasiński, M., Cpałka, K., Er, M.J.: New method for dynamic signature verification using hybrid partitioning. In: Rutkowski, L., Korytkowski, M., Rafał Scherer, R., Tadeusiewicz, L.A.Z., Zurada, J.M. (eds.) Artificial Intelligence and Soft Computing, pp. 216–230. Springer International Publishing, Cham (2014). https://doi.org/10.1007/978-3-319-07176-3_20

  40. Zeiler, M.D.: An adaptive learning rate method, Adadelta (2012)

    Google Scholar 

  41. Zhao, X., Song, M., Liu, A., Wang, Y., Wang, T., Cao, J.: Data-driven temporal-spatial model for the prediction of AQI in nanjing. J. Artif. Intell. Soft Comput. Res. 10(4), 255–270 (2020)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computational Intelligence, Czȩstochowa University of Technology, Czȩstochowa, Poland

    Jarosław Bilski & Bartosz Kowalczyk

Authors
  1. Jarosław Bilski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bartosz Kowalczyk
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jarosław Bilski .

Editor information

Editors and Affiliations

  1. Czestochowa University of Technology, Częstochowa, Poland

    Leszek Rutkowski

  2. Częstochowa University of Technology, Częstochowa, Poland

    Rafał Scherer

  3. Częstochowa University of Technology, Częstochowa, Poland

    Marcin Korytkowski

  4. Edmonton, AB, Canada

    Witold Pedrycz

  5. AGH University of Science and Technology, Krakow, Poland

    Ryszard Tadeusiewicz

  6. Electrical and Computer Engineering, University of Louisville, Louisville, KY, USA

    Jacek M. Zurada

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bilski, J., Kowalczyk, B. (2021). A New Variant of the GQR Algorithm for Feedforward Neural Networks Training. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J.M. (eds) Artificial Intelligence and Soft Computing. ICAISC 2021. Lecture Notes in Computer Science(), vol 12854. Springer, Cham. https://doi.org/10.1007/978-3-030-87986-0_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-87986-0_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-87985-3

  • Online ISBN: 978-3-030-87986-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation