Boundedness of Weight Elimination for BP Neural Networks
Weight elimination can be usefully interpreted as an assumption about the prior distribution of the weights trained in the backpropagation neural networks (BPNN). Weight elimination based on different scaling of weight parameters is of a general form, with the weight decay and subset selection methods as special cases. The applications of this method have been well developed, however, only few references provides more comprehensive theoretical analysis. To address this issue, we investigate the uniform boundedness of the trained weights based on a descriptive proof.
Keywordsbackpropagation neural networks weight decay weight elimination boundedness
Unable to display preview. Download preview PDF.
- 1.Werbos, P.J.: Beyond regression: new tools for prediction and analysis in the behavioral sciences. Ph.D. thesis. Harvard University, Cambridge, MA (1974)Google Scholar
- 2.Parker, D.B.: Learning-logic, invention report. Stanford University, Stanford (1982)Google Scholar
- 4.Cherkassky, V., Mulier, F.M.: Learning from data: Concepts, Theory, and Methods. IEEE Press (2007)Google Scholar
- 7.Weigend, A.S., Rumelhart, D.E., Huberman, B.A.: Generalization by weight-elimination applied to currency exchange rate prediction. In: IJCNN 1991-Seattle International Joint Conference on Neural Networks, vol. 1, pp. 837–841 (1991)Google Scholar
- 8.Bebis, G., Georgiopoulos, M., Kaspalris, T.: Coupling weight elimination and genetic algorithms. In: IEEE International Conference on Neural Networks, vol. 2, pp. 1115–1120 (1996)Google Scholar
- 13.May, P., Zhou, E., Lee, C.W.: A Comprehensive Evaluation of Weight Growth and Weight Elimination Methods Using the Tangent Plane Algorithm. International Journal of Advanced Computer Sciences and Applications 4, 149–156 (2013)Google Scholar
- 16.Wang, J., Wu, W., Zurada, J.M.: Boundedness and convergence of MPN for cyclic and almost cyclic learning with penalty. In: The 2011 International Joint Conference on Neural Networks (IJCNN), pp. 125–132 (2011)Google Scholar