Maximum of Marginal Likelihood Criterion instead of Cross-Validation for Designing of Artificial Neural Networks
The cross-validation method is commonly applied in the design of Artificial Neural Networks (ANNs). In the paper the design of ANN is related to searching for an optimal value of the regularization coefficient or the number of neurons in the hidden layer of network. Instead of the cross-validation procedure, the Maximum of Marginal Likelihood (MML) criterion, taken from Bayesian approach, can be used. The MML criterion, applied to searching for the optimal values of design parameters of neural networks, is illustrated on two examples. The obtained results enable us to formulate conclusions that the MML criterion can be used instead of the cross-validation method (especially for small data sets), since it permits the design of ANNs without formulation of a validation set of patterns.
Keywordsneural network design cross-validation marginal likelihood Bayesian inference
Unable to display preview. Download preview PDF.
- 1.Haykin, S.: Neural Networks: A Comprehensive Introduction. Prentice-Hall, Englewood Cliffs (1999)Google Scholar
- 2.Waszczyszyn, Z. (ed.): Neural Networks in the Analysis and Design of Structures. CISM Courses and Lectures, vol. 404. Springer, Wien-New York (1999)Google Scholar
- 3.Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)Google Scholar
- 4.Tipping, M.E.: Bayesian Inference: An Introduction to Principles and Practice in Machine Learning. In: Bousquet, O., von Luxburg, U., Rätsch, G. (eds.) Machine Learning 2003. LNCS (LNAI), vol. 3176, pp. 41–62. Springer, Heidelberg (2004)Google Scholar
- 6.Kuźniar, K.: Analysis of vibrations of medium height buildings subjected to mining tremors with application of neural networks (in Polish). Cracow University of Technology (2004)Google Scholar
- 7.Kuźniar, K., Waszczyszyn, Z.: Neural networks for the simulation and identification of building subjected to paraseismic excitations. In: Lagaros, N.D., Tsompanakis, Y. (eds.) Intelligent Computational Paradigms in Earthquake Engineering. Idea Group Publishing (2007)Google Scholar
- 8.Ciesielski, R., Kuźniar, K., Maciag, E., Tatara, T.: Empirical formulae for fundamental natural periods of buildings with load bearing walls. Archives of Civil Engineering 38, 199–291 (1992)Google Scholar