Reducing Bloat in GP with Multiple Objectives

  • Stefan Bleuler
  • Johannes Bader
  • Eckart Zitzler
Part of the Natural Computing Series book series (NCS)


This chapter investigates the use of multiobjective techniques in genetic programming (GP) in order to evolve compact programs and to reduce the effects caused by bloating. The underlying approach considers the program size as a second, independent objective besides program functionality, and several studies have found this concept to be successful in reducing bloat. Based on one specific algorithm, we demonstrate the principle of multiobjective GP and show how to apply Pareto-based strategies to GP. This approach outperforms four classical strategies to reduce bloat with regard to both convergence speed and size of the produced programs on an even-parity problem. Additionally, we investigate the question of why the Pareto-based strategies can be more effective in reducing bloat than alternative strategies on several test problems. The analysis falsifies the hypothesis that the small but less functional individuals that are kept in the population act as building blocks building blocks for larger correct solutions. This leads to the conclusion that the advantages are probably due to the increased diversity in the population.


Genetic Programming Multiobjective Optimization Correct Solution High Mutation Rate Nondominated Solution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    H. A. Abbass and K. Deb. Searching under Multi-evolutionary Pressures. In C. M. Fonseca et al., editors, Evolutionary Multi-Criterion Optimization. Second International Conference, EMO 2003, pages 391–404, Berlin, Germany, 2003. Springer. Lecture Notes in Computer Science. Volume 2632.Google Scholar
  2. [2]
    W. Banzhaf, F. D. Francone, R. E. Keller, and P. Nordin. Genetic Programming: An Introduction. Morgan Kaufmann, San Francisco, CA, 1998.zbMATHGoogle Scholar
  3. [3]
    Y. Bemstein, X. Li, V. Ciesielski, and A. Song. Multiobjective parsimony enforcement for superior generalisation performance. In IEEE, editor, CEC 04, pages 83–89, 2004.Google Scholar
  4. [4]
    S. Bleuler, M. Brack, L. Thiele, and E. Zitzler. Multiobjective Genetic Programming: Reducing Bloat by Using SPEA2. In Congress on Evolutionary Computation (CEC-2001), pages 536–543, Piscataway, NJ, 2001. IEEE.Google Scholar
  5. [5]
    T. Blickle. Evolving Compact Solutions in Genetic Programming: A Case Study. In H. M. Voigt et al., editors, PPSN IV, pages 564–573. Springer-Verlag, 1996.Google Scholar
  6. [6]
    T. Blickle and L. Thiele. Genetic programming and redundancy. In J. Hopf, editor, Genetic Algorithms within the Framework of Evolutionary Computation (Workshop at KI-94, Saarbrddotucken), pages 33–38, 1994.Google Scholar
  7. [7]
    E. D. De Jong, R. A. Watson, and J. B. Pollack. Reducing Bloat and Promoting Diversity using Multi-Objective Methods. In L. Spector et al., editors, Genetic and Evolutionary Computation Conference (GECCO 2001), pages 11–18. Morgan Kaufmann Publishers, 2001.Google Scholar
  8. [8]
    K. Deb, S. Agrawal, A. Pratap, and T. Meyarivan. A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II. In M. Schoenauer et al., editors, Parallel Problem Solving from Nature (PPSN VI), Lecture Notes in Computer Science Vol. 1917, pages 849–858. Springer, 2000.Google Scholar
  9. [9]
    A. Ekacuteart and S. Z. Nacuteemeth. Selection Based on the Pareto Nondomination Criterion for Controlling Code Growth in Genetic Programming. Genetic Programming and Evolvable Machines, 2:61–73, 2001.Google Scholar
  10. [10]
    A. Hunter. Expression Inference - Genetic Symbolic Classification Integrated with Non-linear Coefficient Optimisation. In AISC 02, LNCS. Springer, 2002.Google Scholar
  11. [11]
    E. D. D. Jong and J. B. Pollack. Multi-objective methods for tree size control. Genetic Programming and Evolvable Machines, 4:211–233, 2003.CrossRefGoogle Scholar
  12. [12]
    T. Kalganova and J. F. Miller. Evolving More Efficient Digital Circuits by Allowing Circuit Layout Evolution and Multi-Objective Fitness. In A. Stoica et al., editors, Proceedings of the 1st NASA/DoD Workshop on Evolvable Hardware (EH’99), pages 54–63, Piscataway, NJ, 1999, 1999. IEEE Computer Society Press.Google Scholar
  13. [13]
    J. D. Knowles, R. A. Watson, and D. W. Corne. Reducing Local Optima in Single-Objective Problems by Multi-objectivization. In E. Zitzler et al., editors, Evolutionary Multi-Criterion Optimization (EMO 2001), volume 1993 of Lecture Notes in Computer Science, pages 269–283, Berlin, 2001. Springer-Verlag.Google Scholar
  14. [14]
    A. Kordon, E. Jordaan, L. Chew, G. Smits, T. Bruck, K. Haney, and A. Jenings. Biomass Inferential Sensor Based on Ensemble of Models Generated by Genetic Programming. In GECCO 04, LNCS, pages 1078–1089. Springer, 2004.Google Scholar
  15. [15]
    M. Kotanchek, G. Smits, and E. Vladislavleva. Pursuing the Pareto Paradigm Tournaments, Algorithm Variations & Ordinal Optimization. In R. L. Riolo, T. Soule, and B. Worzel, editors, Genetic Programming Theory and Practice IV, volume 5 of Genetic and Evolutionary Computation, chapter 3. Springer, 2006.Google Scholar
  16. [16]
    J. R. Koza. Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge, MA, USA, 1992.zbMATHGoogle Scholar
  17. [17]
    J. R. Koza. Genetic Programming II: Automatic Discovery of Reusable Programs. MIT Press, Cambridge, Massachusetts, 1994.zbMATHGoogle Scholar
  18. [18]
    W. B. Langdon. Quadratic Bloat in Genetic Programming. In D. Whitley et al., editors, GECCO 2000, pages 451–458, Las Vegas, Nevada, USA, 10-12 2000. Morgan Kaufmann.Google Scholar
  19. [19]
    W. B. Langdon and R. Poli. Fitness Causes Bloat. In P. K. Chawdhry et al., editors, Soft Computing in Engineering Design and Manufacturing, pages 13–22, Godalming, GU7 3DJ, UK, 1997. Springer-Verlag.Google Scholar
  20. [20]
    X. Llorgravea, D. E. Goldberg, I. Traus, and E. Bernadacuteo. Accuracy, parsimony, and generality in evolutionary learning systems via multiobjective selection. In Learning Classifier Systems, pages 118–142. Springer. Lecture Notes in Artificial Intelligence Vol. 2661, 2002.Google Scholar
  21. [21]
    L. Panait and S. Luke. Alternative Bloat Control Methods. In GECCO 04, LNCS, pages 630–641. Springer, 2004.Google Scholar
  22. [22]
    D. Parrot, L. Xiandong, and V. Ciesielski. Multi-objective techniques in genetic programming for evolving classifiers. In CEC 05, pages 1141–1148. IEEE, 2005.Google Scholar
  23. [23]
    K. Rodracuteiguez-Vacuteazquez, C. M. Fonseca, and P. J. Fleming. Multiobjective genetic programming: A nonlinear system identification application. In J. R. Koza, editor, Late Breaking Papers at the 1997 Genetic Programming Conference, pages 207–212, Stanford University, CA, USA, 13–16 1997. Stanford Bookstore.Google Scholar
  24. [24]
    G. F. Smits and M. Kotanchek. Pareto-Front Exploitation in Symbolic Regression. In Genetic Programming Theory and Practice II, volume 8. Springer, 2005.Google Scholar
  25. [25]
    T. Soule and J. A. Foster. Removal Bias: a New Cause of Code Growth in Tree Based Evolutionary Programming. In 1998 IEEE International Conference on Evolutionary Computation, pages 781–186, Anchorage, Alaska, USA, 1998. IEEE Press.Google Scholar
  26. [26]
    T. Soule and J. A. Foster. Effects of Code Growth and Parsimony Pressure on Populations in Genetic Programming. Evolutionary Computation, 6(4):293–309, 1999.Google Scholar
  27. [27]
    M. Streeter and L. A. Becker. Automated Discovery of Numerical Approximation Formulae via Genetic Programming. Genetic Programming and Evolvable Machines, 4(3):255–286, 2003.CrossRefGoogle Scholar
  28. [28]
    B.-T. Zhang and H. Mühlenbein. Balancing Accuracy and Parsimony in Genetic Programming. Evolutionary Computation, 3(1):17–38, 1995.Google Scholar
  29. [29]
    Y. Zhang and P. I. Rockett. Evolving optimal feature extraction using multi-objective genetic programming: a methodology and preliminary study on edge detection. In GECCO 05, pages 795–802, New York, NY, USA, 2005. ACM Press.Google Scholar
  30. [30]
    E. Zitzler. Evolutionary Algorithms for Multiobjective Optimization: Methods and Applications. PhD thesis, Swiss Federal Institute of Technology (ETH) Zürich, Switzerland, 1999.Google Scholar
  31. [31]
    E. Zitzler, M. Laumanns, and L. Thiele. SPEA2: Improving the Strength Pareto Evolutionary Algorithm for Multiobjective Optimization. In K. Giannakoglou et al., editors, Evolutionary Methods for Design, Optimisation and Control with Application to Industrial Problems (EUROGEN 2001), pages 95–100. International Center for Numerical Methods in Engineering (CIMNE), 2002.Google Scholar
  32. [32]
    E. Zitzler and L. Thiele. Multiobjective Evolutionary Algorithms: A Comparative Case Study and the Strength Pareto Approach. IEEE Transactions on Evolutionary Computation, 3(4):257–271, 1999.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Stefan Bleuler
    • 1
  • Johannes Bader
    • 1
  • Eckart Zitzler
    • 1
  1. 1.Computer Engineering and Networks Laboratory (TIK)ETH ZurichSwitzerland

Personalised recommendations