Analyzing Weight Distribution of Feedforward Neural Networks and Efficient Weight Initialization
In this paper, we investigate and analyze the weight distribution of feedforward two-layer neural networks in order to understand and improve the time-consuming training process of neural networks. Generally, it takes a long time to train neural networks. However, when a new problem is presented, neural networks have to be trained again without any benefit from previous training. In order to address this problem, we view training process as finding a solution weight point in a weight space and analyze the distribution of solution weight points in the weight space. Then, we propose a weight initialization method that uses the information on the distribution of the solution weight points. Experimental results show that the proposed weight initialization method provides a better performance than the conventional method that uses a random generator in terms of convergence speed.
KeywordsNeural Network Classification Accuracy Weight Distribution Hide Neuron Feedforward Neural Network
- 3.Chen, Z., Feng, T., Houkes, Z.: Incorporating a priori knowledge into initialized weights for neural classifier. Proc. Int. Joint Conf. Neural Networks 2, 291–296 (2000)Google Scholar
- 5.Bellido, I., Fiesler, E.: Do backpropagation trained neural networks have normal weight distributions? In: Proc. Int. Conf. Artificial Neural Networks, pp. 772–775 (1993)Google Scholar
- 6.Richards, J.A.: Remote Sensing Digital Image Analysis. Springer, Heidelberg (1993)Google Scholar
- 8.Biehl, L.L., et al.: A crops and soils data base for scene radiation research. In: Proc. Machine Process. of Remotely Sensed Data Symp., West Lafayette, Indiana, pp. 169–177 (1982)Google Scholar