Abstract
A novel two phases training algorithm for a multilayer perceptron with regularization is proposed to solve a local minima problem for training networks and to enhance the generalization property of networks trained. The first phase is a trust region-based local search for fast training of networks. The second phase is an regularized line search tunneling for escaping local minima and moving toward a weight vector of next descent. These two phases are repeated alternatively in the weight space to achieve a goal training error. Benchmark results demonstrate a significant performance improvement of the proposed algorithm compared to other existing training algorithms.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barhen, J., Protopopescu, V., Reister, D.: TRUST: A Deterministic Algorithm for Global Optimization. Science 276, 1094–1097 (1997)
Foresee, F.D., Hagan, M.T.: Gauss-Newton Approximation to Bayesian Regularization. In: Procedeengs, International Joint Conference on Neural Networks, pp. 1930–1935 (1997)
Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (1999)
Haykin, S.: Neural Networks: A Comprehensive Foundation. Prentice-Hall, New York (1999)
Singh, Y.P., Roychowdhury, P.: Dynamic Tunneling Based Regularization in Feedforward Neural Networks. Artificial Intelligence 131, 55–71 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lee, DW., Choi, HJ., Lee, J. (2004). A Regularized Line Search Tunneling for Efficient Neural Network Learning. In: Yin, FL., Wang, J., Guo, C. (eds) Advances in Neural Networks – ISNN 2004. ISNN 2004. Lecture Notes in Computer Science, vol 3173. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-28647-9_41
Download citation
DOI: https://doi.org/10.1007/978-3-540-28647-9_41
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22841-7
Online ISBN: 978-3-540-28647-9
eBook Packages: Springer Book Archive