Abstract
This paper analyzes global convergence and learning parameters of the back-propagation algorithm for quadratic functions. Some global convergence conditions of the steepest descent algorithm are obtained by directly analyzing the exact momentum equations for quadratic cost functions. In addition, in order to guarantee the convergence for a given learning task, the method is obtained to choose the proper learning parameters. The results presented in this paper are the improvement and extension of the existed ones in some existing works.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Phansalkar, V.V., Sastry, P.S.: Analysis of the Back-propagation Algorithm with Momentum. IEEE Trans. Neural Networks 5, 505–506 (1994)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning Representations by Back-propagating Errors. Nature 323, 533–536 (1986)
Hagan, M.T., Demuth, H.B., Beale, M.H.: Neural Network Design. PWS, Boston, MA (1996)
Torii, M., Hagan, M.T.: Stability of Steepest Descent with Momentum for Quadratic Functions. IEEE Trans. Neural Networks 13, 752–756 (2002)
Hagiwara, M., Sato, A.: Analysis of Momentum Term in Back-propagation. IEICE Trans. Inform. Syst. 8, 1–6 (1995)
Sato, A.: Analytical Study of the Momentum Term in A Backpropagation Algorithm. In: Proc. ICANN91, pp. 617–622 (1991)
Qian, N.: On the Momentum Term in Gradient Descent Learning Algorithms. Neural Networks 12, 145–151 (1999)
Zeng, Z.G., Huang, D.S., Wang, Z.F.: Global Convergence of Steepest Descent for Quadratic Functions. In: Yang, Z.R., Yin, H., Everson, R.M. (eds.) IDEAL 2004. LNCS, vol. 3177, pp. 672–677. Springer, Heidelberg (2004)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zeng, Z. (2007). Analysis of Global Convergence and Learning Parameters of the Back-Propagation Algorithm for Quadratic Functions. In: Huang, DS., Heutte, L., Loog, M. (eds) Advanced Intelligent Computing Theories and Applications. With Aspects of Artificial Intelligence. ICIC 2007. Lecture Notes in Computer Science(), vol 4682. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74205-0_2
Download citation
DOI: https://doi.org/10.1007/978-3-540-74205-0_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74201-2
Online ISBN: 978-3-540-74205-0
eBook Packages: Computer ScienceComputer Science (R0)