Multilayer Feedforward Ensembles for Classification Problems
As shown in the bibliography, training an ensemble of networks is an interesting way to improve the performance with respect to a single network. However there are several methods to construct the ensemble and there are no complete results showing which one could be the most appropriate. In this paper we present a comparison of eleven different methods. We have trained ensembles of 3, 9, 20 and 40 networks to show results in a wide spectrum of values. The results show that the improvement in performance above 9 networks in the ensemble depends on the method but it is usually marginal. Also, the best method is called “Decorrelated” and uses a penalty term in the usual Backpropagation function to decorrelate the network outputs in the ensemble.
Unable to display preview. Download preview PDF.
- 6.Freund, Y., Schapire, R.: Experiments with a New Boosting Algorithm. In: Proceedings of the Thirteenth International Conference on Machine Learning, pp. 148–156 (1996)Google Scholar
- 8.Auda, G., Kamel, M.: EVOL: Ensembles Voting On-Line. In: Proc. of the World Congress on Computational Intelligence, pp. 1356–1360 (1998)Google Scholar
- 9.Liu, Y., Yao, X.: A Cooperative Ensemble Learning System. In: Proc. of the World Congress on Computational Intelligence, pp. 2202–2207 (1998)Google Scholar
- 10.Jang, M., Cho, S.: Ensemble Learning Using Observational Learning Theory. In: Proceedings of the International Joint Conference on Neural Networks, vol. 2, pp. 1281–1286 (1999)Google Scholar