Bounds for the Average Generalization Error of the Mixture of Experts Neural Network
In this paper we derive an upper bound for the average-case generalization error of the mixture of experts modular neural network, based on an average-case generalization error bound for an isolated neural network. By doing this we also generalize a previous bound for this architecture that was restricted to special problems.
We also present a correction factor for the original average generalization error, that was empirically obtained, that yields more accurate error bounds for the 6 data sets used in the experiments. These experiments illustrate the validity of the derived error bound for the mixture of experts modular neural network and show how it can be used in practice.
Keywordsmodular neural networks mixture of experts generalization error bounds
- 2.Vapnik, V.: The Nature of Statistical Learning Theory. Springer, Heidelberg (1999)Google Scholar
- 4.Alexandre, L., Campilho, A., Kamel, M.: Average error bound for the mixture of experts MNN architecture. In: Proceedings of the 12th Portuguese Conference on Pattern Recognition, Aveiro, Portugal (2002)Google Scholar
- 5.Jacobs, R., Jordan, M., Nowlan, S., Hinton, G.: Adaptive mixtures of local experts. Neural Computation, 79–87 (1991)Google Scholar
- 6.Auda, G., Kamel, M.: Modular neural network classifiers: A comparative study. J. Intel. Robotic Systems, 117–129 (1998)Google Scholar
- 7.Jordan, M., Jacobs, R.: Hierarchical mixture of experts and the EM algorithm. Neural Computation, 181–214 (1994)Google Scholar
- 9.Marques de Sá, J.: Pattern Recognition: Concepts, Methods and Applications. Springer, Heidelberg (2001)Google Scholar
- 10.Blake, C., Keogh, E., Merz, C.: UCI repository of machine learning databases (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
- 12.Bishop, C.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)Google Scholar
- 13.Riedmiller, M., Braun, H.: A direct adaptive method for faster backpropagation learning: The RPROP algorithm. In: IEEE International Conference on Neural Networks, San Francisco, pp. 586–591 (1993)Google Scholar