Advertisement

Genetic Programming and Evolvable Machines

, Volume 13, Issue 3, pp 305–337 | Cite as

Evolutionary dynamics on multiple scales: a quantitative analysis of the interplay between genotype, phenotype, and fitness in linear genetic programming

  • Ting Hu
  • Joshua L. PayneEmail author
  • Wolfgang Banzhaf
  • Jason H. Moore
Article

Abstract

Redundancy is a ubiquitous feature of genetic programming (GP), with many-to-one mappings commonly observed between genotype and phenotype, and between phenotype and fitness. If a representation is redundant, then neutral mutations are possible. A mutation is phenotypically-neutral if its application to a genotype does not lead to a change in phenotype. A mutation is fitness-neutral if its application to a genotype does not lead to a change in fitness. Whether such neutrality has any benefit for GP remains a contentious topic, with reported experimental results supporting both sides of the debate. Most existing studies use performance statistics, such as success rate or search efficiency, to investigate the utility of neutrality in GP. Here, we take a different tack and use a measure of robustness to quantify the neutrality associated with each genotype, phenotype, and fitness value. We argue that understanding the influence of neutrality on GP requires an understanding of the distributions of robustness at these three levels, and of the interplay between robustness, evolvability, and accessibility amongst genotypes, phenotypes, and fitness values. As a concrete example, we consider a simple linear genetic programming system that is amenable to exhaustive enumeration and allows for the full characterization of these quantities, which we then relate to the dynamical properties of simple mutation-based evolutionary processes. Our results demonstrate that it is not only the distribution of robustness amongst phenotypes that affects evolutionary search, but also (1) the distributions of robustness at the genotypic and fitness levels and (2) the mutational biases that exist amongst genotypes, phenotypes, and fitness values. Of crucial importance is the relationship between the robustness of a genotype and its mutational bias toward other phenotypes.

Keywords

Accessibility Coreness Evolvability Genotype-phenotype map Phenotype-fitness map Networks Neutrality Redundancy Robustness 

Notes

Acknowledgments

This work was partially supported by NIH grants R01-LM009012, R01-LM010098, and R01-AI59694. J.L.P. was supported by NIH grant R25-CA134286. W.B. acknowledges support from NSERC Discovery Grants, under RGPIN 283304-07. The authors would like to thank the three anonymous reviewers for their scrutiny, Davnah Urbach for her thoughtful comments on earlier versions of this manuscript, and Bill Langdon for fruitful discussions at Evo* 2011.

References

  1. 1.
    W. Banzhaf, in Proceedings of Parallel Problem Solving from Nature III. Genotype-phenotype mapping and neutral variation—a case study in genetic programming. Lecture Notes in Computer Science, vol. 866 (1994) pp. 322–332Google Scholar
  2. 2.
    W. Banzhaf, A. Leier, Genetic Programming Theory and Practice III, Chapter 14: Evolution on Neutral Networks in Genetic Programming. (Springer, Berlin, 2006), pp. 207–221Google Scholar
  3. 3.
    D. Barber, Bayesian Reasoning and Machine Learning. (Cambridge University Press, Cambridge, 2011)Google Scholar
  4. 4.
    J.D. Bloom, S.T. Labthavikul, C.R. Otey, F.H. Arnold, Protein stability promotes evolvability. Proc. Natl. Acad. Sci. 103, 5869–5874 (2006)CrossRefGoogle Scholar
  5. 5.
    S. Bullock, in Proceedings of the 6th European Conference on Artificial Life. Smooth operator? Understanding and visualising mutation bias. Lecture Notes in Computer Science, vol. 2159 (2001), pp. 602–612Google Scholar
  6. 6.
    S. Ciliberti, O.C. Martin, A. Wagner, Innovation and robustness in complex regulatory gene networks. Proc. Nat. Acad. Sci. 104, 13591–13596 (2007)CrossRefGoogle Scholar
  7. 7.
    M. Collins, Finding needles in haystacks is harder with neutrality. Genet. Prog. Evolvable Mach. 7, 131–144 (2006)CrossRefGoogle Scholar
  8. 8.
    M.C. Cowperthwaite, E.P. Economo, W.R. Harcombe, E.L. Miller, L.A. Meyers, The ascent of the abundant: How mutational networks constrain evolution. PLoS Comput. Biol. 4(7), e10000110 (2008)MathSciNetCrossRefGoogle Scholar
  9. 9.
    B. Dorronsoro, E. Alba, in IEEE Congress on Evolutionary Computation. A simple cellular genetic algorithm for continuous optimization (2006), pp. 2838–2844Google Scholar
  10. 10.
    J.A. Draghi, T.L. Parsons, G.P. Wagner, J.B. Plotkin, Mutational robustness can facilitate adaptation. Nature 463, 353–355 (2010)CrossRefGoogle Scholar
  11. 11.
    M. Ebner, P. Langguth, J. Albert, M. Shackleton, R. Shipman, in Proceedings of the IEEE Congress on Evolutionary Computation. On neutral networks and evolvability (2001), pp 1–8Google Scholar
  12. 12.
    M. Ebner, M. Shackleton, R. Shipman, How neutral networks influence evolvability. Complexity 7(2), 19–33 (2002)MathSciNetCrossRefGoogle Scholar
  13. 13.
    E. Ferrada, A. Wagner, Protein robustness promotes evolutionary innovations on large evolutionary time-scales. Proc. R. Soc. Lond. B 275, 1595–1602 (2008)CrossRefGoogle Scholar
  14. 14.
    W. Fontana, P. Schuster, Continuity in evolution: On the nature of transitions. Science 280, 1451–1455 (1998)CrossRefGoogle Scholar
  15. 15.
    E. Galván-López, R. Poli, in Proceedings of the Genetic and Evolutionary Computation Conference. An empirical investigation of how and why neutrality affects evolutionary search (2006), pp. 1149–1156Google Scholar
  16. 16.
    E. Galván-López, R. Poli, A. Kattan, M. O’Neill, A. Brabazon, Neutrality in evolutionary algorithms… What do we know?. Evol. Syst. 2, 145–163 (2011)CrossRefGoogle Scholar
  17. 17.
    C. Grinstead, J. Snell, Introduction to Probability. (American Mathematical Society, Providence, 1991)Google Scholar
  18. 18.
    I. Harvey, A. Thompson, in Proceedings of the First International Conference on Evolvable Systems: From Biology to Hardware. Through the labyrinth evolution finds a way: A silicon ridge (1996), pp. 406–422Google Scholar
  19. 19.
    E.J. Hayden, E. Ferrada, A. Wagner, Cryptic genetic variation promotes rapid evolutionary adaptation in an RNA enzyme. Nature 474, 92–95 (2011)CrossRefGoogle Scholar
  20. 20.
    T. Hu, W. Banzhaf, in Proceedings of the Genetic and Evolutionary Computation Conference. Neutrality and variability: two sides of evolvability in linear genetic programming (2009), pp. 963–970Google Scholar
  21. 21.
    T. Hu, J.L. Payne, W. Banzhaf, J.H. Moore, in Proceedings of the 14th European Conference on Genetic Programming. Robustness, evolvability, and accessibility in linear genetic programming. Lecture Notes in Computer Science, vol. 6621 (2011), pp. 13–24Google Scholar
  22. 22.
    M.A. Huynen, P.F. Stadler, W. Fontana, Smoothness within ruggedness: the role of neutrality in adaptation. Proc. Natl. Acad. Sci. 93, 397–401 (1996)CrossRefGoogle Scholar
  23. 23.
    M. Isalan, C. Lemerle, K. Michalodimitrakis, C. Horn, P. Beltrao, E. Raineri, M. Garriga-Canut, L. Serrano, Evolvability and hierarchy in rewired bacterial gene networks. Nature 452, 840–846 (2008)CrossRefGoogle Scholar
  24. 24.
    H. Ishibuchi, N. Tsukamoto, Y. Nojima, in Proceedings of Parallel Problem Solving from Nature X. Examining the effect of elitism in cellular genetic algorithms using two neighborhood structures. Lecture Notes in Computer Science, vol. 5199 (2008), pp. 458–467Google Scholar
  25. 25.
    T. Jörg, O.C. Martin, A. Wagner, Neutral network sizes of biological RNA molecules can be computed and are not atypically small. BMC Bioinform. 9, 464 (2008)CrossRefGoogle Scholar
  26. 26.
    M. Kirschner, J. Gerhart, Evolvability. Proc. Natl. Acad. Sci. 95, 8420–8427 (1998)CrossRefGoogle Scholar
  27. 27.
    P.K. Lehre, P.C. Haddow, in Proceedings of the IEEE Congress on Evolutionary Computation. Accessibility between neutral networks in indirect genotype-phenotype mappings (2005), pp. 419–426Google Scholar
  28. 28.
    P.K. Lehre, P.C. Haddow, in Simulated Evolution and Learning. Accessibility and runtime between convex neutral networks. Lecture Notes in Computer Science, vol. 4247 (2006), pp. 734–741Google Scholar
  29. 29.
    O.C. Martin, A. Wagner, Effects of recombination on complex regulatory circuits. Genetics 138, 673–684 (2009)CrossRefGoogle Scholar
  30. 30.
    T. Mihaljev, B. Drossel, Evolution of a population of random Boolean networks. Eur. Phys. J. B 67, 259–267 (2009)zbMATHCrossRefGoogle Scholar
  31. 31.
    E. Murphy, M. O’Neill, A. Brabazon, in Proceedings of the 14th European Conference on Genetic Programming. Examining mutation landscapes in grammar based genetic programming. Lecture Notes in Computer Science, vol. 6621 (2011), pp. 130–141Google Scholar
  32. 32.
    M. Newman, Mixing patterns in networks. Phys. Rev. E 67, 026126 (2003)MathSciNetCrossRefGoogle Scholar
  33. 33.
    M. Newman, Networks: An Introduction. (Oxford University Press, Oxford, 2010)zbMATHGoogle Scholar
  34. 34.
    M. Newman, R. Engelhardt, Effects of selective neutrality on the evolution of molecular species. Proc. R. Soc. Lond. B 265, 1333–1338 (1998)CrossRefGoogle Scholar
  35. 35.
    J.L. Payne, J.H. Moore, in Proceedings of the European Conference on Artificial Life. Robustness, evolvability, and accessibility in the signal-integration space of gene regulatory circuits (2011), pp. 638–645Google Scholar
  36. 36.
    D.A. Pechenick, J.L. Payne, J.H. Moore, The influence of assortativity on the robustness of signal-integration logic in gene regulatory networks. J. Theor. Biol. 296, 21–32 (2012)CrossRefGoogle Scholar
  37. 37.
    K. Raman, A. Wagner, The evolvability of programmable hardware. J. R. Soc. Interf. 8, 269–281 (2011)CrossRefGoogle Scholar
  38. 38.
    C. Reidys, P.F. Stadler, P. Schuster, Generic properties of combinatory maps: Neutral networks of RNA secondary structures. Bull. Math. Biol. 59, 339–397 (1997)zbMATHCrossRefGoogle Scholar
  39. 39.
    J. Rodrigues, A. Wagner, Genotype networks, innovation, and robustness in sulfur metabolism. BMC Syst. Biol. 5, 39 (2011)CrossRefGoogle Scholar
  40. 40.
    F. Rothlauf, D.E. Goldberg, Redundant representations in evolutionary computation. Evol. Comput. 11, 381–415 (2003)CrossRefGoogle Scholar
  41. 41.
    P. Schuster, W. Fontana, P.F. Stadler, I.L. Hofacker, From sequences to shapes and back: a case study in RNA secondary structures. Proc. R. Soc. Lond. B 255, 279–284 (1994)CrossRefGoogle Scholar
  42. 42.
    R. Shipman, M. Shackleton, M. Ebner, R. Watson, in Proceedings of the 7th International Conference on Artificial Life. Neutral search spaces for artificial evolution: a lesson from life (2001), pp 162–169Google Scholar
  43. 43.
    T. Smith, P. Husbands, P. Layzell, M. O’Shea, Fitness landscapes and evolvability. Evol. Comput. 10, 1–34 (2002)CrossRefGoogle Scholar
  44. 44.
    T. Smith, P. Husbands, M. O’Shea, in Proceedings of the 6th European Conference on Artificial Life. Neutral networks and evolvability with complex genotype-phenotype mapping. Lecture Notes in Computer Science, vol. 2159 (2001), pp. 272–282Google Scholar
  45. 45.
    T. Smith, P. Husbands, M. O’Shea, in Proceedings of the IEEE Congress on Evolutionary Computation. Neutral networks in an evolutionary robotics search space (2001), pp. 136–145Google Scholar
  46. 46.
    T. Soule, Resilient individuals improve evolutionary search. Artif. Life 12, 17–34 (2006)CrossRefGoogle Scholar
  47. 47.
    B.M.R. Stadler, P.F. Stadler, G.P. Wagner, W. Fontana, The topology of the possible: Formal spaces underlying patterns of evolutionary change. J. Theor. Biol. 213, 241–274 (2001)MathSciNetCrossRefGoogle Scholar
  48. 48.
    van E. Nimwegen, J. Crutchfield, M. Huynen, Neutral evolution of mutational robustness. Proc. Natl. Acad. Sci. 96, 9716–9720 (1999)CrossRefGoogle Scholar
  49. 49.
    L. Vanneschi, Y. Pirola, P. Collard, M. Tomassini, S. Verel, G. Mauri, in Proceedings of the Genetic and Evolutionary Computation Conference. A quantitative study of neutrality in GP Boolean landscapes (2006), pp. 895–902Google Scholar
  50. 50.
    A. Wagner, Robustness and Evolvability in Living Systems. (Princeton University Press, Princeton, 2005)Google Scholar
  51. 51.
    A. Wagner, Neutralism and selectionism: a network-based reconciliation. Nat. Rev. Genet. 9, 965–974 (2008)CrossRefGoogle Scholar
  52. 52.
    A. Wagner, Robustness and evolvability: a paradox resolved. Proc. R. Soc. Lond. B 275, 91–100 (2008)CrossRefGoogle Scholar
  53. 53.
    A. Wagner, The Origins of Evolutionary Innovations. (Oxford University Press, Oxford, 2011)Google Scholar
  54. 54.
    G. Wagner, L. Altenberg, Complex adaptation and the evolution of evolvability. Evolution 50, 967–976 (1996)CrossRefGoogle Scholar
  55. 55.
    J. Whitacre, A. Bender, Degeneracy: a design principle for achieving robustness and evolvability. J. Theor. Biol. 263, 143–153 (2010)CrossRefGoogle Scholar
  56. 56.
    C.O. Wilke, Adaptive evolution on neutral networks. Bull. Math. Biol. 63, 715–730 (2001)CrossRefGoogle Scholar
  57. 57.
    T. Yu, J.F. Miller, Through the interaction of neutral and adaptive mutations, evolutionary search finds a way. Artif. Life 12, 525–551 (2006)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Ting Hu
    • 1
  • Joshua L. Payne
    • 1
    Email author
  • Wolfgang Banzhaf
    • 2
  • Jason H. Moore
    • 1
  1. 1.Computational Genetics LaboratoryDartmouth Medical SchoolHanoverUSA
  2. 2.Department of Computer ScienceMemorial University of NewfoundlandSt. John’sCanada

Personalised recommendations