Abstract
Following general arguments on approximation Bayesian discriminant functions by neural networks, rigorously proved is that a three layered neural network, having rather a small number of hidden layer units, can approximate the Bayesian discriminant function for the two category classification if the log ratio of the a posteriori probability is a polynomial. The accuracy of approximation is measured by the Kullback-Leibler information. An extension to the multi-category case is also discussed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Duda, R.O. and Hart, P.E.: Pattern classification and scene analysis. John Wiley & Sons, New York (1973)
Funahashi, K.: Multilayer neural networks and Bayes decision theory. Neural Networks 11 (1998) 209–213
Hinton, G.E.: Connectionist learning procedures. Artificial intelligence 40 (1989) 185–234
Ito, Y.: Simultaneous L p-approximations of polynomials and derivatives on the whole space. Proceedings of ICANN99 (1999) 587–592
Ito, Y., Srinivasan, C.: Bayesian decision theory and three layered neural networks, Proceedings of ESANN2001 (2001) 377–382
Richard, M.D. and Lippmann, R.P.: Neural network classifiers estimate Bayesian a posteriori probabilities. Neural Compt. 3 (1991) 461–483
Ripley, B.D.:Statistical aspect of neural networks. Networks and chaos-Statistical and Probabilistic Aspects. ed. Barndorff-Nielsen, O.E., Jensen, J.L., Kendall, W.S., Chapman & Hall. London (1993) 40–123
Ruck, M.D., Rogers, S., Kabrisky, M., Oxley, H., Sutter, B.: The multilayer perceptron as an approximator to a Bayes optimal discriminant function. IEEE Transactions on Neural Networks. 1 (1990) 296–298
M.J. Schervish, Theory of statistics. Springer-Verlag, Berlin, New York(1995)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ito, Y., Srinivasan, C. (2001). Approximation of Bayesian Discriminant Function by Neural Networks in Terms of Kullback-Leibler Information. In: Dorffner, G., Bischof, H., Hornik, K. (eds) Artificial Neural Networks — ICANN 2001. ICANN 2001. Lecture Notes in Computer Science, vol 2130. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44668-0_19
Download citation
DOI: https://doi.org/10.1007/3-540-44668-0_19
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42486-4
Online ISBN: 978-3-540-44668-2
eBook Packages: Springer Book Archive