Abstract
Artificial neural networks (ANN) are commonly used in function approximation as well as classification problems. This paper shows a configurable architecture of a simple feed forward neural network trained by particle swarm optimization (PSO) algorithm. PSO and ANN have several hyperparameters that have impact on the results of approximation. ANN parameters are the number of layers, number of neurons in each layer, and neuron activation functions. The hyperparameters of the PSO are the population size, the number of informants per particle, and the acceleration coefficients. Herein, this work comes to spot the light on how the PSO hyperparameters affect the ability of the algorithm to optimize ANNs weights in the function approximation task. This was examined and tested by generating multiple experiments on different types of input functions such as: cubic, linear, XOR problem. The results of the proposed method show the superiority of PSO compared to backpropagation in terms of MSE.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Zainuddin, Z., Pauline, O.: Function approximation using artificial neural networks. WSEAS Trans. Math. 7(6), 333–338 (2008)
Elbrächter, D., Perekrestenko, D., Grohs, P., Bölcskei, H.: Deep neural network approximation theory. arXiv preprint arXiv:1901.02220 (2019)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323(6088), 533–536 (1986)
Örkcü, H.H., Bal, H.: Comparing performances of backpropagation and genetic algorithms in the data classification. Expert Syst. Appl. 38(4), 3703–3709 (2011)
Han, F., Zhu, J.S.: Improved particle swarm optimization combined with backpropagation for feedforward neural networks. Int. J. Intell. Syst. 28(3), 271–288 (2013)
Rahmani, M.: Particle swarm optimization of artificial neural networks for autonomous robots. Master of Science in Complex Adaptive Systems Thesis, Department of Applied Physics, Chalmers University of Technology, Sweden (2008)
Dhar, V.K., Tickoo, A.K., Koul, R., Dubey, B.P.: Comparative performance of some popular artificial neural network algorithms on benchmark and function approximation problems. Pramana 74(2), 307–324 (2010). https://doi.org/10.1007/s12043-010-0029-4
El-Shorbagy, M.A., Aboul, E.H.: Particle swarm optimization from theory to applications. Int. J. Rough Sets Data Anal. (IJRSDA) 5(2), 1–24 (2018)
Piotrowski, A.P., Napiorkowski, J.J., Piotrowska, A.E.: Population size in particle swarm optimization. Swarm Evol. Comput. 58, 100718 (2020)
Abhijit Suresh, K.V., Harish, N.: Particle swarm optimization over back propagation neural network for length of stay prediction. Procedia Comput. Sci. 46, 268–275 (2015)
Yu, H., Gao, Y., Wang, L., Meng, J.: A hybrid particle swarm optimization algorithm enhanced with nonlinear inertial weight and Gaussian mutation for job shop scheduling problems. Mathematics 8(8), 1355 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Zaghloul, L., Zaghloul, R., Hamdan, M. (2021). Optimizing Artificial Neural Network for Functions Approximation Using Particle Swarm Optimization. In: Tan, Y., Shi, Y. (eds) Advances in Swarm Intelligence. ICSI 2021. Lecture Notes in Computer Science(), vol 12689. Springer, Cham. https://doi.org/10.1007/978-3-030-78743-1_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-78743-1_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-78742-4
Online ISBN: 978-3-030-78743-1
eBook Packages: Computer ScienceComputer Science (R0)