Neural Computing and Applications

, Volume 16, Issue 1, pp 57–68

Removal of hidden neurons in multilayer perceptrons by orthogonal projection and weight crosswise propagation

Original Article

DOI: 10.1007/s00521-006-0057-7

Cite this article as:
Liang, X. Neural Comput & Applic (2007) 16: 57. doi:10.1007/s00521-006-0057-7

Abstract

A new method of pruning away hidden neurons in neural networks is presented in this paper. The hidden neuron is removed by analyzing the orthogonal projection correlations among the outputs of other hidden neurons. The method guarantees the least loss of weight information in terms of orthogonal projection. The remaining weights and thresholds are updated based on the weight crosswise propagation. A practical technique for penalizing the superfluous hidden neurons is explored. Retraining is needed after pruning. Extensive experiments are conducted, and the results demonstrate that the method gives better initial points for retraining and retraining costs less epochs.

Keywords

Hidden neurons Pruning Orthogonal projection Weight crosswise propagation 

Copyright information

© Springer-Verlag London Limited 2006

Authors and Affiliations

  1. 1.Institute of Computer Science and TechnologyPeking UniversityBeijingChina
  2. 2.Department of Economics and Operations ResearchStanford UniversityStanfordUSA

Personalised recommendations