Abstract
There are many choices that should be made before a neural network can be used in an application. These include the network architecture, learning method, training data, feature representation, initial parameter values, training set order, cost function. A number of alternatives are generated and tested and the one that performs best on a separate test set is adopted. Instead of discarding the rest, we propose here to combine them by taking a vote. First the approach is advocated and the existing literature is surveyed. Then we put the approach into a Bayesian framework where the weights in votes are taken as Bayesian priors computed based on model complexities. Initial results on classification of handwritten numerals are promising. The voting approach can be generalized to regression where the function approximated is continuous as opposed to discrete 0/1 of classification.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Alpaydin, E. (1991) “GAL: Networks that grow when they learn and shrink when they forget,” ICSI, TR–91–032, Berkeley, CA
Alpaydin, E. (1993) “Multiple networks for function learning,” IEEE International Conference on Neural Networks, March, San Francisco, 9–14
Benediktsson, J.A., Swain, P.H. (1992) “Consensus Theoretic Classification Methods,” IEEE Transactions on Systems, Man, and Cybernetics, 22, 688–704
Guyon, I., Poujoud, I., Personnaz, L., Dreyfus, G., Denker, J., le Cun, Y. (1989) “Comparing different neural architectures for classifying handwritten digits,” International Joint Conference on Neural Networks, Washington DC
Hampshire, J., Waibel, A. (1989) “A novel objective function for improved phoneme recognition using time delay neural networks,” CMU, TR CS–89–118
Hansen, L.K., Salamon, P. (1990) “Neural Network Ensembles,” IEEE Pattern Analysis and Machine Intelligence) 12, 993–1001
Jacobs, R.A., Jordan, M. I., Nowlan, S.J., Hinton, G.E., (1991) “Adaptive Mixtures of Local Experts,” Neural Computation, 3, 79–87
Lincoln, W.P., Skrzypek, J. (1990) “Synergy of Clustering Multiple Back Propagation Networks,” in Advances in Neural Information Processing Systems 2, D. Touretzky (Ed.), Morgan Kaufmann, 650–657
MacKay, D.J.C. (1992) “Bayesian Interpolation,” Neural Computation 4, 415– 447
Mani, G. (1991) “Lowering Variance of Decisions by using Artificial Neural Network Portfolios,” Neural Computation, 3, 484–486
Pearlmutter, B.A., Rosenfeld, R. (1991) “Chaitin–Kolmogorov Complexity and Generalization in Neural Networks,” in Advances in Neural Information Processing Systems 3, R. Lippmann, J. Moody, D. Touretzky (Eds.), Morgan Kaufmann, 925–931
Perrone, M.P. (1993) “Improving Regression Estimation: Averaging Methods for Variance Reduction with Extensions to General Convex Measure Optimization,” PhD Thesis, Department of Physics, Brown University
Poggio, T., Torre, V., Koch, C. (1985) “Computational vision and regulariza– tion theory,” Nature, 317, 314–319
Rissanen, J. (1987) “Stochastic Complexity,” Journal of Royal Statistics Society Bt 49, 223–239, 252–265
Beyer, U., Smieja, F. (1993) “Learning from Examples, Agent Teams, and the Concept of Reflection,” GMD, TR–93–766, St Augustin, Germany
Wolpert, D.H. (1992) “Stacked Generalization,” Neural Networks, 5, 241–259
Xu, L., Kryzak, A., Suen, C.Y. (1992) “Methods of Combining Multiple Classifiers and Their Applications to Handwriting Recognition,” IEEE Transactions on Systems, Man, and Cybernetics) 22, 418–435
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1995 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Alpaydm, E. (1995). Multiple Neural Experts for Improved Decision Making. In: Steels, L. (eds) The Biology and Technology of Intelligent Autonomous Agents. NATO ASI Series, vol 144. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-79629-6_21
Download citation
DOI: https://doi.org/10.1007/978-3-642-79629-6_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-79631-9
Online ISBN: 978-3-642-79629-6
eBook Packages: Springer Book Archive