Age-Fitness Pareto Optimization

  • Michael Schmidt
  • Hod Lipson
Chapter
Part of the Genetic and Evolutionary Computation book series (GEVO, volume 8)

Abstract

We propose a multi-objective method, inspired by the Age Layered Population Structure algorithm, for avoiding premature convergence in evolutionary algorithms, and demonstrate a three-fold performance improvement over comparable methods. Previous research has shown that partitioning an evolving population into age groups can greatly improve the ability to identify global optima and avoid converging to local optima. Here, we propose that treating age as an explicit optimization criterion can increase performance even further, with fewer algorithm implementation parameters. The proposed method evolves a population on the two-dimensional Pareto front comprising (a) how long the genotype has been in the population (age); and (b) its performance (fitness). We compare this approach with previous approaches on the Symbolic Regression problem, sweeping the problem difficulty over a range of solution complexities and number of variables. Our results indicate that the multi-objective approach identifies the exact target solution more often than the age-layered population and standard population methods. The multi-objective method also performs better on higher complexity problems and higher dimensional datasets - finding global optima with less computational effort.

Keywords

Symbolic Regression Age Fitness Multi-objective 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Auger, Anne and Hansen, Nikolaus (2005). A restart CMA evolution strategy with increasing population size. In IEEE Congress on Evolutionary Computation, pages 1769–1776. IEEE.Google Scholar
  2. Bongard, J. and Lipson, H. (in press). Automated reverse engineering of nonlinear dynamical systems. Proceedings of the National Academy of Sciences of the United States of America.Google Scholar
  3. de Jong, Edwin D. and Pollack, Jordan B. (2003). Multi-objective methods for tree size control. Genetic Programming and Evolvable Machines, 4(3):211–233.CrossRefGoogle Scholar
  4. Deb, Kalyanmoy (2001).Multi-Objective Optimization Using Evolutionary Algorithms. Wiley, Chichester, UK.MATHGoogle Scholar
  5. Doucette, John, Lichodzijewski, Peter, and Heywood, Malcolm (2009). Evolving coevolutionary classifiers under large attribute spaces. In Riolo, Rick L., O’Reilly, Una-May, and McConaghy, Trent, editors, Genetic Programming Theory and Practice VII, Genetic and Evolutionary Computation, chapter 3, pages 37–54. Springer, Ann Arbor.Google Scholar
  6. Grunwald, P. (1999). Model selection based on minimum description length.Google Scholar
  7. Hoai, N. X., McKay, R. I., Essam,D., and Chau, R. (2002). Solving the symbolic regression problemwith tree-adjunct grammar guided genetic programming: The comparative results. In Fogel,DavidB., El-Sharkawi, Mohamed A., Yao, Xin, Greenwood, Garry, Iba, Hitoshi, Marrow, Paul, and Shackleton, Mark, editors, Proceedings of the 2002 Congress on Evolutionary Computation CEC2002, pages 1326–1331. IEEE Press.Google Scholar
  8. Hornby, Gregory (2006). ALPS: the age-layered population structure for reducing the problem of premature convergence. In Cattolico, Mike, editor, Genetic and Evolutionary Computation Conference, GECCO 2006, Proceedings, Seattle, Washington, USA, July 8-12, 2006, pages 815–822. ACM.Google Scholar
  9. Hornby, Gregory S. (2009a). Steady-state ALPS for real-valued problems. In Rothlauf, Franz, editor, Genetic and Evolutionary Computation Conference, GECCO 2009, Proceedings, Montreal, Québec, Canada, July 8-12, 2009, pages 795–802. ACM.Google Scholar
  10. Hornby, Gregory S. (2009b). A steady-state version of the age-layered population structure EA. In Riolo, Rick L., O’Reilly, Una-May, and McConaghy, Trent, editors, Genetic Programming Theory and Practice VII, Genetic and Evolutionary Computation, chapter 6, pages 87–102. Springer, Ann Arbor.Google Scholar
  11. Hu, Jianjun, Goodman, Erik D., Seo, Kisung, Fan, Zhun, and Rosenberg, Rondal (2005). The hierarchical fair competition (HFC) framework for sustainable evolutionary algorithms. Evolutionary Computation, 13(2):241–277.CrossRefGoogle Scholar
  12. Hu, Jianjun, Goodman, Erik D., Seo, Kisung, and Pei, Min (2002). Adaptive hierarchical fair competition (AHFC) model for parallel evolutionary algorithms. In Langdon, W. B., Cantú-Paz, E., Mathias, K., Roy, R., Davis, D., Poli, R., Balakrishnan, K., Honavar, V., Rudolph, G., Wegener, J., Bull, L., Potter,M.A., Schultz, A. C.,Miller, J. F., Burke, E., and Jonoska, N., editors, GECCO 2002: Proceedings of the Genetic and Evolutionary Computation Conference, pages 772–779, New York. Morgan Kaufmann Publishers.Google Scholar
  13. Jansen, Thomas (2002). On the analysis of dynamic restart strategies for evolutionary algorithms. In Merelo Guervós, Juan Julián, Adamidis, Panagiotis Beyer, Hans-Georg, Fernández-Villacañas, José-Luis, and Schwefel, Hans-Paul, editors, Parallel Problem Solving from Nature - PPSN VII, pages 33–43, Berlin. Springer.Google Scholar
  14. Korns, Michael F. (2009). Symbolic regression of conditional target expressions. In Riolo, Rick L., O’Reilly, Una-May, and McConaghy, Trent, editors, Genetic Programming Theory and Practice VII, Genetic and Evolutionary Computation, chapter 13, pages 211–228. Springer, Ann Arbor.Google Scholar
  15. Kotanchek, Mark E., Vladislavleva, Ekaterina Y., and Smits, Guido F. (2009). Symbolic regression via GP as a discovery engine: Insights on outliers and prototypes. In Riolo, Rick L., O’Reilly, Una-May, and McConaghy, Trent, editors, Genetic Programming Theory and Practice VII, Genetic and Evolutionary Computation, chapter 4, pages 55–72. Springer, Ann Arbor.Google Scholar
  16. Koza, John R. (1992). Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge, MA, USA.MATHGoogle Scholar
  17. Louis, Sushil J. and Rawlins, Gregory J. E. (1993). Syntactic analysis of convergence in genetic algorithms. In Whitley, L. Darrell, editor, Foundations of Genetic Algorithms 2, pages 141–151, San Mateo. Morgan Kaufmann.Google Scholar
  18. Mahfoud, SamirW. (1995).Nichingmethods for genetic algorithms. PhD thesis, Champaign, IL, USA.Google Scholar
  19. McKay, Ben, Willis, Mark J., and Barton, Geoffrey W. (1995). Using a tree structured genetic algorithm to perform symbolic regression. In Zalzala, A. M. S., editor, First International Conference on Genetic Algorithms in Engineering Systems: Innovations and Applications, GALESIA, volume 414, pages 487–492, Sheffield, UK. IEE.CrossRefGoogle Scholar
  20. Murphy, Gearoid and Ryan, Conor (2007). Manipulation of convergence in evolutionary systems. In Riolo, Rick L., Soule, Terence, and Worzel, Bill, editors, Genetic Programming Theory and Practice V, Genetic and Evolutionary Computation, chapter 3, pages 33–52. Springer, Ann Arbor.Google Scholar
  21. Ryan, Conor (1996). Reducing Premature Convergence in Evolutionary Algorithms. PhD thesis, University College, Cork, Ireland.Google Scholar
  22. Schmidt, Michael and Lipson, Hod (2007). Comparison of tree and graph encodings as function of problem complexity. In Thierens, Dirk, Beyer, Hans-Georg, Bongard, Josh, Branke, Jurgen, Clark, John Andrew, Cliff, Dave, Congdon, Clare Bates, Deb, Kalyanmoy, Doerr, Benjamin, Kovacs, Tim, Kumar, Sanjeev, Miller, Julian F., Moore, Jason, Neumann, Frank, Pelikan, Martin, Poli, Riccardo, Sastry, Kumara, Stanley, Kenneth Owen, Stutzle, Thomas, Watson, Richard A, and Wegener, Ingo, editors, GECCO ’07: Proceedings of the 9th annual conference on Genetic and evolutionary computation, volume 2, pages 1674–1679, London. ACM Press.Google Scholar
  23. Schmidt, Michael and Lipson, Hod (2009). Distilling free-form natural laws from experimental data. Science, 324(5923):81–85.CrossRefGoogle Scholar
  24. Schmidt, Michael D. and Lipson, Hod (2006). Co-evolving fitness predictors for accelerating and reducing evaluations. In Riolo, Rick L., Soule, Terence, and Worzel, Bill, editors, Genetic Programming Theory and Practice IV, volume 5 of Genetic and Evolutionary Computation, chapter 17, pages –. Springer, Ann Arbor.Google Scholar
  25. Schmidt, Michael D. and Lipson, Hod (2008). Coevolution of fitness predictors. IEEE Transactions on Evolutionary Computation, 12(6):736–749.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Michael Schmidt
    • 1
  • Hod Lipson
    • 2
    • 3
  1. 1.Computational BiologyCornell UniversityIthacaUSA
  2. 2.School of Mechanical and Aerospace EngineeringCornell UniversityIthacaUSA
  3. 3.Computing and Information ScienceCornell UniversityIthacaUSA

Personalised recommendations