Abstract
Committee machines are a set of experts that their outputs are combined to improve the performance of the whole system which tend to grow into unnecessarily large size in most of the time. This can lead to extra memory usage, computational costs, and occasional decreases in effectiveness. Expert pruning is an intermediate technique to search for a good subset of all members before combining them. In this paper we studied an expert pruning method based on genetic algorithm to prune regression members. The proposed algorithm searches to find a best subset of experts by creating a logical weight for each member and chooses which member that the related weight is equal to one. The final weights for selected experts are calculated by genetic algorithm method. The results showed that MSE and R-square for the pruned CM are 0.148 and 0.9032 respectively that are reasonable rather than all experts separately.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bakker, B., Heskes, T.: Clustering ensembles of neural network models. Neural Netw. 16(2), 261–269 (2003)
Zhou, Z.-H., Wu, J., Tang, W.: Ensembling neural networks: Many could be better than all. Artificial Intelligence 137(1-2), 239–263 (2002)
Rooney, N., Patterson, D., Nugent, C.: Reduced Ensemble Size Stacking. In: 6th IEEE International Conference on Tools with Artificial Intelligence, pp. 266–271 (2004)
Patel, R., Shrawankar, U.N., Raghuwanshi, M.M.: Genetic Algorithm with Histogram Construction Technique. In: Second International Conference on Emerging Trends in Engineering & Technology, pp. 615–618 (2009)
Huang, H.-C., Chen, Y.-H.: Genetic fingerprinting for copyright protection of multicast media. Soft Computing 13(4), 383–391 (2008)
Banfield, R.E., et al.: Ensemble diversity measures and their application to thinning. Information Fusion 6(1), 49–62 (2005)
Caruana, R., et al.: Ensemble selection from libraries of models. In: Proceedings of the Twenty-First International Conference on Machine Learning (2004)
Partalas, I., Tsoumakas, G., Vlahavas, I.: An ensemble uncertainty aware measure for directed hill climbing ensemble pruning. Machine Learning 81(3), 257–282 (2010)
Fan, W., et al.: Pruning and dynamic scheduling of cost-sensitive ensembles. In: Eighteenth National Conference on Artificial Intelligence, pp. 146–151 (2002)
Brown, G., Wyatt, J.L., Peter: Managing Diversity in Regression Ensembles. Machine Learning Research 6, 1621–1650 (2005)
Martinez-Munoz, G., Suarez, A.: Pruning in ordered bagging ensembles. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 609–616 (2006)
Caruana, R., Munson, A., Niculescu-Mizil, A.: Getting the most out of ensemble selection. In: International Conference on Data Mining, pp. 828–833 (2006)
Zhang, Y., Burer, S., Street, W.N.: Ensemble Pruning Via Semi-definite Programming. Jornal of Machin. Learnning Research 7, 1315–1338 (2006)
Martinez-Muoz, G., Hernandez-Lobato, D., Suarez, A.: An Analysis of Ensemble Pruning Techniques Based on Ordered Aggregation. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(2), 245–259 (2009)
Hernandez-Lobato, D., Martinez-Munoz, G., Suarez, A.: Pruning in Ordered Regression Bagging Ensembles. In: Proceedings of the IEEE World Congress on Computational Intelligence, pp. 1266–1273 (2006b)
MartÃnez-Muñoz, G., Suárez, A.: Using boosting to prune bagging ensembles. Pattern Recognition Letters 28(1), 156–165 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jafari, S.A., Mashohor, S., Ramli, A.R., Marhaban, M.H. (2012). Expert Pruning Based on Genetic Algorithm in Regression Problems. In: Pan, JS., Chen, SM., Nguyen, N.T. (eds) Intelligent Information and Database Systems. ACIIDS 2012. Lecture Notes in Computer Science(), vol 7198. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28493-9_10
Download citation
DOI: https://doi.org/10.1007/978-3-642-28493-9_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-28492-2
Online ISBN: 978-3-642-28493-9
eBook Packages: Computer ScienceComputer Science (R0)