Skip to main content

Expert Pruning Based on Genetic Algorithm in Regression Problems

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNAI,volume 7198)

Abstract

Committee machines are a set of experts that their outputs are combined to improve the performance of the whole system which tend to grow into unnecessarily large size in most of the time. This can lead to extra memory usage, computational costs, and occasional decreases in effectiveness. Expert pruning is an intermediate technique to search for a good subset of all members before combining them. In this paper we studied an expert pruning method based on genetic algorithm to prune regression members. The proposed algorithm searches to find a best subset of experts by creating a logical weight for each member and chooses which member that the related weight is equal to one. The final weights for selected experts are calculated by genetic algorithm method. The results showed that MSE and R-square for the pruned CM are 0.148 and 0.9032 respectively that are reasonable rather than all experts separately.

Keywords

  • expert pruning
  • committee machine
  • learning algorithms
  • genetic algorithm

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (Canada)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bakker, B., Heskes, T.: Clustering ensembles of neural network models. Neural Netw. 16(2), 261–269 (2003)

    CrossRef  Google Scholar 

  2. Zhou, Z.-H., Wu, J., Tang, W.: Ensembling neural networks: Many could be better than all. Artificial Intelligence 137(1-2), 239–263 (2002)

    CrossRef  MathSciNet  MATH  Google Scholar 

  3. Rooney, N., Patterson, D., Nugent, C.: Reduced Ensemble Size Stacking. In: 6th IEEE International Conference on Tools with Artificial Intelligence, pp. 266–271 (2004)

    Google Scholar 

  4. Patel, R., Shrawankar, U.N., Raghuwanshi, M.M.: Genetic Algorithm with Histogram Construction Technique. In: Second International Conference on Emerging Trends in Engineering & Technology, pp. 615–618 (2009)

    Google Scholar 

  5. Huang, H.-C., Chen, Y.-H.: Genetic fingerprinting for copyright protection of multicast media. Soft Computing 13(4), 383–391 (2008)

    CrossRef  Google Scholar 

  6. Banfield, R.E., et al.: Ensemble diversity measures and their application to thinning. Information Fusion 6(1), 49–62 (2005)

    CrossRef  Google Scholar 

  7. Caruana, R., et al.: Ensemble selection from libraries of models. In: Proceedings of the Twenty-First International Conference on Machine Learning (2004)

    Google Scholar 

  8. Partalas, I., Tsoumakas, G., Vlahavas, I.: An ensemble uncertainty aware measure for directed hill climbing ensemble pruning. Machine Learning 81(3), 257–282 (2010)

    CrossRef  MathSciNet  Google Scholar 

  9. Fan, W., et al.: Pruning and dynamic scheduling of cost-sensitive ensembles. In: Eighteenth National Conference on Artificial Intelligence, pp. 146–151 (2002)

    Google Scholar 

  10. Brown, G., Wyatt, J.L., Peter: Managing Diversity in Regression Ensembles. Machine Learning Research 6, 1621–1650 (2005)

    MathSciNet  MATH  Google Scholar 

  11. Martinez-Munoz, G., Suarez, A.: Pruning in ordered bagging ensembles. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 609–616 (2006)

    Google Scholar 

  12. Caruana, R., Munson, A., Niculescu-Mizil, A.: Getting the most out of ensemble selection. In: International Conference on Data Mining, pp. 828–833 (2006)

    Google Scholar 

  13. Zhang, Y., Burer, S., Street, W.N.: Ensemble Pruning Via Semi-definite Programming. Jornal of Machin. Learnning Research 7, 1315–1338 (2006)

    MathSciNet  MATH  Google Scholar 

  14. Martinez-Muoz, G., Hernandez-Lobato, D., Suarez, A.: An Analysis of Ensemble Pruning Techniques Based on Ordered Aggregation. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(2), 245–259 (2009)

    CrossRef  Google Scholar 

  15. Hernandez-Lobato, D., Martinez-Munoz, G., Suarez, A.: Pruning in Ordered Regression Bagging Ensembles. In: Proceedings of the IEEE World Congress on Computational Intelligence, pp. 1266–1273 (2006b)

    Google Scholar 

  16. Martínez-Muñoz, G., Suárez, A.: Using boosting to prune bagging ensembles. Pattern Recognition Letters 28(1), 156–165 (2007)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Jafari, S.A., Mashohor, S., Ramli, A.R., Marhaban, M.H. (2012). Expert Pruning Based on Genetic Algorithm in Regression Problems. In: Pan, JS., Chen, SM., Nguyen, N.T. (eds) Intelligent Information and Database Systems. ACIIDS 2012. Lecture Notes in Computer Science(), vol 7198. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28493-9_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-28493-9_10

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-28492-2

  • Online ISBN: 978-3-642-28493-9

  • eBook Packages: Computer ScienceComputer Science (R0)