Abstract
In this paper we present a novel method for pruning redundant weights of a trained multilayer Perceptron (MLP). The proposed method is based on the correlation analysis of the errors produced by the output neurons and the backpropagated errors associated with the hidden neurons. Repeated applications of it leads eventually to the complete elimination of all connections of a neuron. Simulations using real-world data indicate that, in terms of performance, the proposed method compares favorably with standard pruning techniques, such as the Optimal Brain Surgeon (OBS) and Weight Decay and Elimination (WDE), but with much lower computational costs.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)
Seghouane, A.-K., Amari, S.-I.: The AIC criterion and symmetrizing the Kullback-Leibler divergence. IEEE Transactions on Neural Networks 18(1), 97–106 (2007)
Curry, B., Morgan, P.H.: Model selection in neural networks: Some dificulties. European Journal of Operational Research 170(2), 567–577 (2006)
Nakamura, T., Judd, K., Mees, A.I., Small, M.: A comparative study of information criteria for model selection. International Journal of Bifurcation and Chaos 16(8), 2153–2175 (2006)
Xiang, C., Ding, S.Q., Lee, T.H.: Geometric interpretation and architecture selection of the MLP. IEEE Transactions on Neural Networks 16(1), 84–96 (2005)
Fahlman, S.E., Lebiere, C.: The cascade-correlation learning architecture. In: Touretzky, D.S. (ed.) Advances in Neural Information Processing Systems, vol. 2, pp. 524–532. Morgan Kaufmann, San Mateo (1990)
Hassibi, B., Stork, D.G.: Second order derivatives for network pruning: Optimal brain surgeon. In: Hanson, S.J., Cowan, J.D., Giles, C.L. (eds.) Advances in Neural Information Processing Systems, vol. 5, pp. 164–171. Morgan Kaufmann, San Mateo, CA (1993)
Principe, J.C., Euliano, N.R., Lefebvre, W.C.: Neural and Adaptive Systems. John Wiley & Sons, West Sussex, England (2000)
Berthonnaud, E., Dimnet, J., Roussouly, P., Labelle, H.: Analysis of the sagittal balance of the spine and pelvis using shape and orientation parameters. Journal of Spinal Disorders & Techniques 18(1), 40–47 (2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Medeiros, C.M.S., Barreto, G.A. (2007). An Efficient Method for Pruning the Multilayer Perceptron Based on the Correlation of Errors. In: de Sá, J.M., Alexandre, L.A., Duch, W., Mandic, D. (eds) Artificial Neural Networks – ICANN 2007. ICANN 2007. Lecture Notes in Computer Science, vol 4668. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74690-4_23
Download citation
DOI: https://doi.org/10.1007/978-3-540-74690-4_23
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74689-8
Online ISBN: 978-3-540-74690-4
eBook Packages: Computer ScienceComputer Science (R0)