A New Multilayer Perceptron Initialisation Method with Selection of Weights on the Basis of the Function Variability
Learning results of multilayer perceptrons highly depend on the initial weight values. The proper selection of weights may improve network performance and reduce time of the learning process. In the paper, a new multilayer perceptron weight selection algorithm based on determination of the variability of the approximated function, within various fragments of its domain, has been proposed. This algorithm has a low computational complexity. Results of numerical experiments have been presented for many learning sets. The comparison of cost function values for neural networks initialized with the applying of the proposed algorithm and for networks initialised by the popular Nguyen-Widrow algorithm has been shown. Independently of the epoch number, the use of the proposed algorithm made it possible to achieve better results for a vast majority of the learning sets.
Keywordsneural network multilayer perceptron initialisation
Unable to display preview. Download preview PDF.
- 4.Guijarro-Berdiñas, B., Fontenla-Romero, O., Pérez-Sánchez, B., Alonso-Betanzos, A.: A new initialization method for neural networks using sensitivity analysis. In: Proceedings of the International Conference on Mathematical and Statistical Modeling in Honor of Enrique Castillo, San Diego (2006)Google Scholar
- 5.Wei, X., Xiu-Tao, Y.: The application of optimal weights initialization algorithm based on information amount in multi-layer perceptron networks. In: Proceedings of the 3rd IEEE International Conference on Computer Science and Information Technology (ICCSIT), vol. 6, pp. 196–198 (2010)Google Scholar
- 7.Jin-Song, P., Mai, E.C., Wright, J.P., Smyth, A.W.: Neural network initialization with prototypes - function approximation in engineering mechanics applications. In: Proceedings of the International Joint Conference on Neural Networks, Orlando, pp. 2110–2116 (2007)Google Scholar
- 9.Nguyen, D., Widrow, B.: Improving the learning speed of 2-Layer neural networks by choosing initial values of the adaptive weights. In: Proceedings of the International Joint Conference on Neural Networks, vol. 3, pp. 21–26 (1990)Google Scholar
- 10.Pavelka, A., Procházka: Algorithms for Initialization of Neural Network Weights, Sbornik prispevku 11. In: Konference MATLAB, vol. 2, pp. 453–459 (2004)Google Scholar
- 12.Online documentation of GTOPO30 data (2012), http://eros.usgs.gov/#/Find_Data/Products_and_Data_Available/gtopo30/README
- 13.UCI Machine Learning Repository, http://archive.ics.uci.edu/ml
- 14.Cortez, P., Morais, A.: A data mining approach to predict forest fires using meteorological data. In: Neves, J., Santos, M.F., Machado, J. (eds.) New Trends in Artificial Intelligence, Proceedings of the 13th EPIA - Portuguese Conference on Artificial Intelligence, pp. 512–523. APPIA, Guimarães (2007)Google Scholar