Preliminary Experiments with Ensembles of Neurally Diverse Artificial Neural Networks for Pattern Recognition
Although there have been a few approaches to achieve the goal of fault tolerance by diversifying redundancy of the individual networks that make up a neural network ensemble, some of which include ensembles of neural networks of different sizes, and ensembles of different models of neural networks such as Radial Basis Function Networks and Multilayer Perceptron, there is yet to be an empirical study on hybrid neural networks that makes use of a diverse set of transfer functions, which we would expect to be able to exhibit diverse network architectures, and thus possibly more diverse error patterns. In this paper, we present an approach that uses transfer function diversity to achieve significant results on ensembles. The results show that even with relatively small networks having 5 hidden nodes, and a relatively small ensemble size of just 10 members, the ensemble is able to get competitive results on the Iris data set. It also capable of obtaining competitive results with 20 ensemble members of relatively small networks on other popular data sets such as the Diabetes, Sonar, Hepatitis, and Australian Credit Card problems. In addition to that, it is shown that these results can be achieved with a simple sorting and selection of the Top N solutions of the population, in contrast to other methods of selecting ensemble members that can be computationally expensive, such as selection of the Pareto-front, or hill climbing methods of selection.
KeywordsHybrid Neural Networks Artificial Neural Networks Transfer function Optimization Pattern Recognition
Unable to display preview. Download preview PDF.
- 1.Lower Your Risks: Age, Race, Gender & Family History (2013), http://www.diabetes.org/are-you-at-risk/lower-your-risk/nonmodifiables.html
- 3.Bache, K., Lichman, M.: UCI Machine Learning Repository (2013), http://archive.ics.uci.edu/ml
- 4.Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press (1995)Google Scholar
- 13.Jankowski, N., Duch, W.: Optimal transfer function neural networks. In: 9th European Symposium on Artificial Neural Networks, ESANN 2001, Bruges, vol. (I), pp. 101–106 (2001)Google Scholar
- 14.Krogh, A., Vedelsby, J.: Neural network ensembles, cross validation, and active learning. In: Advances in Neural Information Processing Systems, pp. 231–238 (1995)Google Scholar
- 17.Opitz, D., Shavlik, J.: Generating Accurate and Diverse Members of a Neural-Network Ensemble. In: Advances in Neural Information Processing Systems, vol. 8, pp. 535–541 (1996)Google Scholar
- 18.Perrone, M., Cooper, L.: When networks disagree: Ensemble methods for hybrid neural networks. In: Neural Networks for Speech and Image Processing, pp. 126–142 (1993)Google Scholar
- 20.Wu, Z., Chen, Y.: Genetic algorithm based selective neural network ensemble. In: Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence, IJCAI 2001, vol. 1, pp. 797–802 (2001)Google Scholar