Abstract
Supervised neural networks belong to the class of parameterised non-linear models whose optimum values are determined by a best fit procedure to training data. The problem of over-fitting can occur when more parameters than needed are included in the model. David MacKay’s Occam factor deploys a Bayesian approach to the investigation of how model order can be rationally restrained. This paper uses a case study to show how the Occam factor might be used to discriminate on model order and how it compares with similar indices e.g. Schwarz’s Bayesian information criterion and Akaike’s information criterion.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ghahramani, Z.: Machine Learning 2006 Course Web Page, Department of Engineering. University of Cambridge (2006)
MacKay, D.: Information Theory, Inference, and Learning Algorithms. Cambridge University Press, Cambridge (2003)
Schwarz, G.: Estimating the Dimension of a Model. Source: Ann. Statist. 6(2), 461–464 (1978)
Akaike, H.: A new look at the statistical model identification. IEEE Transactions on Automatic Control 19(6), 716–723 (1974)
Bishop, C.M.: Pattern Recognition and Machine Learning. Springer Science and Media LLC (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Booth, D.J., Tye, R. (2009). An Application of the Occam Factor to Model Order Determination. In: Palmer-Brown, D., Draganova, C., Pimenidis, E., Mouratidis, H. (eds) Engineering Applications of Neural Networks. EANN 2009. Communications in Computer and Information Science, vol 43. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03969-0_41
Download citation
DOI: https://doi.org/10.1007/978-3-642-03969-0_41
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03968-3
Online ISBN: 978-3-642-03969-0
eBook Packages: Computer ScienceComputer Science (R0)