Abstract
Metaheuristic techniques have been recently used to counter the problems like slow convergence to global minima and network stagnancy in back-propagation neural network (BPNN) algorithm. Previously, a meta-heuristic search algorithm called Bat was proposed to train BPNN to achieve fast convergence in the neural network. Although, Bat-BP algorithm achieved fast convergence but it had a problem of oscillations in the gradient path, which can lead to sub-optimal solutions. In-order to remove oscillations in the BAT-BP algorithm, this paper proposed the addition of momentum coefficient to the weights update in the Bat-BP algorithm. The performance of the modified Bat-BP algorithm is compared with simple Bat-BP algorithm on XOR and OR datasets. The simulation results show that the convergence rate to global minimum in modified Bat-BP is highly enhanced and the oscillations are greatly reduced in the gradient path.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Deng, W. J., Chen, W. C., and Pei, W.: Back-propagation neural network based importance-performance analysis for determining critical service attributes, J. Expert Systems with Applications, vol. 34 (2). 2008
Kosko, B.: Neural Network and Fuzzy Systems, 1st Edition, Prentice Hall, India. (1992)
Rumelhart, D. E., Hinton, G. E., and Williams, R. J.: Learning Internal Representations by error Propagation, J. Parallel Distributed Processing: Explorations in the Microstructure of Cognition. (1986)
Rehman, M.Z., Nawi, N. M., and Ghazali, R.: Studying the effect of adaptive momentum in improving the accuracy of gradient descent back propagation algorithm on classification problems, J. International Journal of Modern Physics (IJMPCS), vol.1 (1). (2012)
Nawi, N. M., Ransing, M. R., and Ransing, R. S.: An improved Conjugate Gradient based learning algorithm for back propagation neural networks, J. Computational Intelligence, vol. 4. (2007)
Nawi, N. M., Rehman, M. Z., and Ghazali, M. I.: Noise-Induced Hearing Loss Prediction in Malaysian Industrial Workers using Gradient Descent with Adaptive Momentum Algorithm, J.International Review on Computers and Software (IRECOS), vol. 6 (5). (2011)
Lee, K., Booth, D., and Alam, P. A.: Comparison of Supervised and Unsupervised Neural Networks in Predicting Bankruptcy of Korean Firms, J. Expert Systems with Applications, vol. 29. (2005)
Mendes, R., Cortez, P., Rocha, M., and Neves, J.: Particle swarm for feed forward neural network training. In: Proceedings of the International Joint Conference on Neural Networks, vol. 2, pp. 1895–1899. (2002)
Nandy, S., Sarkar, P. P., and Das, A.: Training a Feed-forward Neural Network with Artificial Bee Colony Based Backpropagation Method, J. International Journal of Computer Science & Information Technology (IJCSIT), vol. 4 (4), pp. 33–46. (2012)
Karaboga, D., Akay, B., and Ozturk, C.: Artificial Bee Colony (ABC) Optimization Algorithm for Training Feed-Forward Neural Networks, In: 4th International Conference on Modeling Decisions for Artificial Intelligence (MDAI 2007), Kitakyushu, Japan, August 16-18. (2007)
Yao, X.: Evolutionary artificial neural networks, J. International Journal of Neural Systems, vol. 4(3), pp. 203–222. (1993)
Montana, D. J., & Davis, L.: Training feedforward neural networks using genetic algorithms, In: Proceedings of the eleventh international joint conference on artificial Intelligence, vol. 1, pp. 762–767. (1989)
Nawi, N. M., Rehman, M. Z., and Khan, A.: A New Bat-Based Back-propagation (BAT-BP) Algorithm, In: ICSS 2013, Wrocław, Poland, September 10-12. (2013)
Yang, X. S.: A new metaheuristic bat-inspired algorithm, In: Nature Inspired Cooperative Strategies for Optimization (NICSO 2010), pp. 65–74. (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer Science+Business Media Singapore
About this paper
Cite this paper
Nawi, N.M., Rehman, M.Z., Khan, A. (2014). Countering the Problem of Oscillations in Bat-BP Gradient Trajectory by Using Momentum. In: Herawan, T., Deris, M., Abawajy, J. (eds) Proceedings of the First International Conference on Advanced Data and Information Engineering (DaEng-2013). Lecture Notes in Electrical Engineering, vol 285. Springer, Singapore. https://doi.org/10.1007/978-981-4585-18-7_12
Download citation
DOI: https://doi.org/10.1007/978-981-4585-18-7_12
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-4585-17-0
Online ISBN: 978-981-4585-18-7
eBook Packages: EngineeringEngineering (R0)