A New Adaptive Strategy for Pruning and Adding Hidden Neurons during Training Artificial Neural Networks
This paper presents a new strategy in designing artificial neural networks. We call this strategy as adaptive merging and growing strategy (AMGS). Unlike most previous strategies on designing ANNs, AMGS puts emphasis on autonomous functioning in the design process. The new strategy reduces or increases an ANN size during training based on the learning ability of hidden neurons and the training progress of the ANN, respectively. It merges correlated hidden neurons to reduce the network size, while it splits existing hidden neuron to increase the network size. AMGS has been tested on designing ANNs for five benchmark classification problems, including Australian credit card assessment, diabetes, heart, iris, and thyroid problems. The experimental results show that the proposed strategy can design compact ANNs with good generalization ability.
KeywordsArtificial neural network design merging neuron splitting neuron generalization ability
Unable to display preview. Download preview PDF.
- 3.Schaffer, J.D., Whitely, D., Eshelman, L.J.: Combinations of genetic algorithms and neural networks- a survey of the state of the art. In: Whitely, D., Schaffer, J.D. (eds.) International Workshop of Genetic Algorithms and Neural Networks, pp. 1–37. IEEE Computer Society Press, Los Alamitos (1992)Google Scholar
- 5.LeCun, Y., Denker, J.S., Solla, S.A.: Optimal brain damage. In: Touretzky, D.S. (ed.) Advances in Neural Information Processing Systems, vol. 2, pp. 598–605. Morgan Kaufmann, San Francisco (1990)Google Scholar
- 6.Hassibi, B., Stork, D.G.: Second-order derivatives for network pruning: optimal brain surgeon. In: Lee, C., Hanson, S., Cowan, J. (eds.) Advances in Neural Information Processing Systems, vol. 5, pp. 164–171. Morgan Kaufmann, San Mateo (1993)Google Scholar