Advertisement

Building Optimal Committees of Genetic Programs

  • Byoung-Tak Zhang
  • Je-Gun Joung
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1917)

Abstract

Committee machines are known to improve the performance of individual learners. Evolutionary algorithms generate multiple individuals that can be combined to build committee machines. However, it is not easy to decide how big the committee should be and what members constitute the best committee. In this paper, we present a probabilistic search method for determining the size and members of the committees of individuals that are evolved by a standard GP engine. Applied to a suite of benchmark learning tasks, the GP committees achieved significant improvement in prediction accuracy.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Opitz, W., Shavlik, J.W.: Actively Searching for an Effective Neural-Network Ensemble. Connection Science, 8 (1996) 337–353.CrossRefGoogle Scholar
  2. 2.
    Zhang, B.-T., Joung, J.-G.: Enhancing Robustness of Genetic Programming at the Species Level. Genetic Programming Conference (GP-97), Morgan Kaufmann, (1997) 336–342.Google Scholar
  3. 3.
    Yao, X., Liu, Y.: Making Use of Population Information in Evolutionary Artificial Neural Networks. IEEE Transactions on Systems, Man, and Cybernetics, 28B(2) (1998) 417–425.Google Scholar
  4. 4.
    Zhang, B.-T., Joung, J.-G.: Time Series Prediction Using Committee Machines of Evolutionary Neural Trees. Proceedings of the 1999 Congress on Evolutionary Computation, 1 (1999) 281–286.zbMATHCrossRefGoogle Scholar
  5. 5.
    Perron, M.P.: Improving Regression Estimation: Averaging Methods for Variance Reduction with Extensions to General Convex Measure Optimization. PhD thesis, Department of Physics, Brown University, (1993).Google Scholar
  6. 6.
    Littlestone, N., Warmuth, M.K.: The Weighted Majority Algorithm. Information and Computation, 108 (1994) 212–261.zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Haykin, S.: Neural Networks, a Comprehensive Foundation. Prentice Hall. (1994).Google Scholar
  8. 8.
    Hansen, L., Salamon, P.: Neural Network Ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12 (1990) 993–1001.CrossRefGoogle Scholar
  9. 9.
    Drucker, H., Cortes, C, Jackel, L.D., LeCun, Y., Vapnik, V.: Boosting and Other Ensemble Methods. Neural Computation, 6(6) (1994) 1289–1301.zbMATHCrossRefGoogle Scholar
  10. 10.
    Jacobs, R.A.: Bias/variance Analyses of Mixture-of-Experts Architectures. Neural computation, 9 (1997) 369–383.zbMATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Hashem, S.: Optimal Linear Combinations of Neural Networks. Neural Networks, 10(4) (1997) 599–614.CrossRefGoogle Scholar
  12. 12.
    Opitz, D., Maclin, R.: Popular Ensemble Methods: An Empirical Study. Journal of Artificial Intelligence Research, 11 (1999) 169–198.zbMATHGoogle Scholar
  13. 13.
    Lemm, J.C.: Mixtures of Gaussian Process Priors. The Ninth International Conference on Artificial Neural Networks (ICANN 99), (1999) 7–10.Google Scholar
  14. 14.
    Neri, F., Giordana, A.: A Parallel Genetic Algorithm for Concept Learning. The Sixth international Conference on Genetic Algorithms, (1995) 436–443.Google Scholar
  15. 15.
    Zhang, B.-T., Ohm, P. and Müehlenbein, H.: Evolutionary Induction of Sparse Neural Trees. Evolutionary Computation, 5(2) (1997) 213–236.zbMATHGoogle Scholar
  16. 16.
    Perron, M.P., Cooper, L.N.: When Networks Disagree: Ensemble Methods for Hybrid Neural Networks. Artificial Neural Networks for Speech and Vision, Chapman & Hall, (1994) 126–142.Google Scholar
  17. 17.
    Zhang, B.-T.: A Bayesian Framework for Evolutionary Computation, Proceedings of the 1999 Congress on Evolutionary Computation, 1 (1999) 722–728.CrossRefGoogle Scholar
  18. 18.
    Murphy, P.M., Aha, D.W.: UCI Repository of Machine Learning Datasets (machine-readable data repository). University of California-Irvine, Department of Information and Computer Science, (1994).Google Scholar
  19. 19.
    Walter, A.T.: Genetic Programming for Feature Discovery and Image Discrimination. In Proceedings of the Fifth Conference on Genetic Algorithms, (1993) 303–309.Google Scholar
  20. 20.
    Gama, J.: Local Cascade Generalization. In Proceedings of the Fifth International Conference (ICML’98), (1998) 206–214.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Byoung-Tak Zhang
    • 1
  • Je-Gun Joung
    • 1
  1. 1.Artificial Intelligence Lab (SCAI) School of Computer Science and EngineeringSeoul National UniversitySeoulKorea

Personalised recommendations