Advertisement

Exact Schema Theory and Markov Chain Models for Genetic Programming and Variable-length Genetic Algorithms with Homologous Crossover

  • Riccardo Poli
  • Nicholas Freitag McPhee
  • Jonathan E. Rowe
Article

Abstract

Genetic Programming (GP) homologous crossovers are a group of operators, including GP one-point crossover and GP uniform crossover, where the offspring are created preserving the position of the genetic material taken from the parents. In this paper we present an exact schema theory for GP and variable-length Genetic Algorithms (GAs) which is applicable to this class of operators. The theory is based on the concepts of GP crossover masks and GP recombination distributions that are generalisations of the corresponding notions used in GA theory and in population genetics, as well as the notions of hyperschema and node reference systems, which are specifically required when dealing with variable size representations.

In this paper we also present a Markov chain model for GP and variable-length GAs with homologous crossover. We obtain this result by using the core of Vose's model for GAs in conjunction with the GP schema theory just described. The model is then specialised for the case of GP operating on 0/1 trees: a tree-like generalisation of the concept of binary string. For these, symmetries exist that can be exploited to obtain further simplifications.

In the absence of mutation, the Markov chain model presented here generalises Vose's GA model to GP and variable-length GAs. Likewise, our schema theory generalises and refines a variety of previous results in GP and GA theory.

genetic programming variable-length genetic algorithms schema theory Markov chain Vose model mixing matrices. 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    L. Altenberg, "Emergent phenomena in genetic programming,” in Evolutionary Programming-Proceedings of the Third Annual Conference, A. V. Sebald and L. J. Fogel (eds.), World Scientific Publishing: San Diego, CA, USA, 24-26 Feb. 1994, pp. 233–241.Google Scholar
  2. 2.
    L. Altenberg, “The schema theorem and price's Theorem,” in Foundations of Genetic Algorithms 3, L. D. Whitley and M. D. Vose (eds.), Morgan Kaufmann: San Francisco, CA, USA, 1995, pp. 23–49.Google Scholar
  3. 3.
    L. B. Booker, “Recombination distributions for genetic algorithms,” in Foundations of Genetic Algorithms 2, L. D. Whitley (ed.), Morgan Kaufmann: San Francisco, CA, 1993, pp. 29–44.Google Scholar
  4. 4.
    T. E. Davis and J. C. Principe, “A Markov chain framework for the simple genetic algorithm,” Evolutionary Comp., vol. 1, no. 3, pp. 269–288, 1993.Google Scholar
  5. 5.
    H. Geiringer, “On the probability theory of linkage in Mendelian heredity,” Annals of Mathematical Statistics, vol. 15, no. 1, pp. 25–57, 1944.Google Scholar
  6. 6.
    D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Addison-Wesley: Reading, Massachusetts, 1989.Google Scholar
  7. 7.
    J. Holland, Adaptation in Natural and Artificial Systems, University of Michigan Press: Ann Arbor, USA, 1975.Google Scholar
  8. 8.
    J. R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection, MIT Press: Cambridge, MA, USA, 1992.Google Scholar
  9. 9.
    W. B. Langdon, “Size fair and homologous tree genetic programming crossovers,” Genetic Programming and Evolvable Machines, vol. 1, no. 1/2, pp. 95–119, 2000.Google Scholar
  10. 10.
    W. B. Langdon and R. Poli, Foundations of Genetic Programming, Springer-Verlag, 2002.Google Scholar
  11. 11.
    N. F. McPhee, R. Poli, and J. E. Rowe, "A schema theory analysis of mutation size biases in genetic programming with linear representations,” in Proceedings of the 2001 Congress on Evolutionary Computation CEC2001, COEX, World Trade Center, 159 Samseong-dong, Gangnam-gu, Seoul, Korea, 27-30 May 2001, IEEE Press, 2001, pp. 1078-1085.Google Scholar
  12. 12.
    M. Mitchell, An introduction to genetic algorithms, MIT Press: Cambridge, MA, 1996.Google Scholar
  13. 13.
    D. J. Montana, “Strongly typed genetic programming,” Evolutionary Compu., vol. 3, no. 2, pp. 199–230, 1995.Google Scholar
  14. 14.
    A. E. Nix and M. D. Vose, “Modeling genetic algorithms with Markov chains,” Annals of Mathematics and Artificial Intelligence, vol. 5, pp. 79–88, 1992.Google Scholar
  15. 15.
    U.-M. O'Reilly and F. Oppacher, "The troubling aspects of a building block hypothesis for genetic programming,” in Foundations of Genetic Algorithms 3, L. D. Whitley and M. D. Vose (eds.), Morgan Kaufmann: San Francisco, CA, 1995, pp. 73–88.Google Scholar
  16. 16.
    R. Poli, “Exact schema theory for genetic programming and variable-length genetic algorithms with one-point crossover,” Genetic Programming and Evolvable Machines, vol. 2, no. 2, pp. 123–163, June 2001.Google Scholar
  17. 17.
    R. Poli and W. B. Langdon, “On the search properties of different crossover operators in genetic programming,” in Genetic Programming 1998: Proceedings of the Third Annual Conference, University of Wisconsin, Madison, Wisconsin, USA, 22-25 July 1998, J. R. Koza, W. Banzhaf, K. Chellapilla, K. Deb, M. Dorigo, D. B. Fogel, M. H. Garzon, D. E. Goldberg, H. Iba, and R. Riolo (eds.), Morgan Kaufmann: University of Wisconsin, Madison, Wisconsin, USA, 22-25 July 1998, pp. 293–301.Google Scholar
  18. 18.
    R. Poli and W. B. Langdon, “Schema theory for genetic programming with one-point crossover and point mutation,” Evolutionary Computation, vol. 6, no. 3, pp. 231–252, 1998.Google Scholar
  19. 19.
    R. Poli and N. F. McPhee, “Exact GP schema theory for headless chicken crossover and subtree mutation,” in Proceedings of the 2001 Congress on Evolutionary Computation CEC2001, IEEE Press: COEX, World Trade Center, 159 Samseong-dong, Gangnam-gu, Seoul, Korea, 27-30 May 2001, pp. 1062–1069.Google Scholar
  20. 20.
    R. Poli and N. F. McPhee, “Exact schema theory for GP and variable-length GAs with homologous crossover,” in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), San Francisco, California, USA, 7-11 July 2001, L. Spector, E. D. Goodman, A. Wu, W. B. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. H. Garzon, and E. Burke (eds.), Morgan Kaufmann: San Francisco, California, USA, 7-11 July 2001, pp. 104–111.Google Scholar
  21. 21.
    R. Poli and N. F. McPhee, “General schema theory for genetic programming with subtree-swapping crossover: Part I,” Evolutionary Comp., vol. 11, no. 1, pp. 53–66, 2003.Google Scholar
  22. 22.
    R. Poli and N. F. McPhee, “General schema theory for genetic programming with subtree-swapping crossover: Part II,” Evolutionary Comp., vol. 11, no. 2, pp. 169–206, 2003.Google Scholar
  23. 23.
    R. Poli, J. E. Rowe, and N. F. McPhee, “Markov chain models for GP and variable-length GAs with homologous crossover,” in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), L. Spector, E. D. Goodman, A. Wu, W. B. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. H. Garzon, and E. Burke (eds.), Morgan Kaufmann: San Francisco, California, USA, 7-11 July 2001, pp. 112–119.Google Scholar
  24. 24.
    R. Poli, C. R. Stephens, A. H. Wright, and J. E. Rowe, “On the search biases of homologuous crossover in linear genetic programming and variable-length genetic algorithms,” in GECCO 2002: Proceedings of the Genetic and Evolutionary Computation Conference, W. B. Langdon, E. Cantú -Paz, K. Mathias, R. Roy, D. Davis, R. Poli, K. Balakrishnan, V. Honavar, G. Rudolph, J. Wegener, L. Bull, M. A. Potter, A. C. Schultz, J. F. Miller, E. Burke, and N. Jonoska (eds.), Morgan Kaufmann Publishers, New York, 9-13 July 2002, pp. 868–876.Google Scholar
  25. 25.
    R. Poli, C. R. Stephens, A. H. Wright, and A. H. Rowe, “A schema-theory-based extension of Geiringer's theorem for linear GP and variable-length GAs under homologous crossover,” in Foundations of Genetic Algorithm 7, K. D. Jong, R. Poli, and J. Rowe (eds.), Morgan Kaufmann: San Francisco, CA, 2003, pp. 45–62.Google Scholar
  26. 26.
    A. Prügel-Bennett and J. L. Shapiro, “An analysis of genetic algorithms using statistical mechanics,” Physical Review Letters, vol. 72, pp. 1305–1309, 1994.Google Scholar
  27. 27.
    N. J. Radcliffe, “Schema processing,” Handbook of Evolutionary Computation, T. Baeck, D. B. Fogel, and Z. Michalewicz (eds.), Oxford University Press, 1997, pp. B2.5-1-B2.5-10.Google Scholar
  28. 28.
    C. Reeves and J. Rowe, “Genetic algorithms: principles and perspectives,” Kluwer Academic Press, 2003.Google Scholar
  29. 29.
    J. P. Rosca, “Analysis of complexity drift in genetic programming,” in Genetic Programming 1997: Proceedings of the Second Annual Conference, Stanford University, J. R. Koza, K. Deb, M. Dorigo, D. B. Fogel, M. Garzon, H. Iba, and R. L. Riolo (eds.), Morgan Kaufmann: CA, USA, 13-16 July 1997, pp. 286–294.Google Scholar
  30. 30.
    J. E. Rowe, “Population fixed-points for functions of unitation,” in Foundations of Genetic Algorithms 5, W. Banzhaf and C. Reeves (eds.), Morgan Kaufmann: San Francisco, CA, 1999, pp. 69–84.Google Scholar
  31. 31.
    J. E. Rowe, M. D. Vose, and A. H. Wright, “Group properties of crossover and mutation,” Evolutionary Comp., vol. 10, no. 2, pp. 151–184, 2002.Google Scholar
  32. 32.
    G. Rudolph, “Convergence analysis of canonical genetic algorithm,” IEEE Transactions on Neural Networks, vol. 5, no. 1, pp. 96–101, 1994.Google Scholar
  33. 33.
    G. Rudolph, “Genetic algorithms,” in Handbook of Evolutionary Computation, T. Baeck, D. B. Fogel, and Z. Michalewicz (eds.), Oxford University Press, 1997, pp. B2.4-20-B2.4-27.Google Scholar
  34. 34.
    G. Rudolph, “Models of stochastic convergence,” in Handbook of Evolutionary Computation, T. Baeck, D. B. Fogel, and Z. Michalewicz (eds.), Oxford University Press, 1997, pp. B2.3-1-B2.3-3.Google Scholar
  35. 35.
    G. Rudolph, “Stochastic processes,” in Handbook of Evolutionary Computation, T. Baeck, D. B. Fogel, and Z. Michalewicz (eds.), Oxford University Press, 1997, pp. B2.2-1-B2.2-8.Google Scholar
  36. 36.
    W. M. Spears, “Aggregating models of evolutionary algorithms,” in Proceedings of the Congress on Evolutionary Computation, P. J. Angeline, Z. Michalewicz, M. Schoenauer, X. Yao, and A. Zalzala (eds.), Mayflower Hotel, Washington D.C., USA, 6-9 July 1999, IEEE Press, vol. 1, 1999, pp. 631–638.Google Scholar
  37. 37.
    W. M. Spears, “The equilibrium and transiente behaviour of mutation and recombination,” in Foundations of Genetic Algorithms Workshop 6, W. M. Spears and W. Martin (eds.), Morgan Kaufmann: San Francisco, CA, 2001, pp. 241–260.Google Scholar
  38. 38.
    M. R. Spiegel, Probability and Statistics, McGraw-Hill: New York, 1975.Google Scholar
  39. 39.
    C. R. Stephens, “Some exact results from a coarse grained formulation of genetic dynamics,” in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), L. Spector, E. D. Goodman, A. Wu, W. B. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. H. Garzon, and E. Burke (eds.), Morgan Kaufmann: San Francisco, California, USA, 7-11 July 2001, pp. 631–638.Google Scholar
  40. 40.
    C. R. Stephens and H. Waelbroeck, “Effective degrees of freedom in genetic algorithms and the block hypothesis,” in Proceedings of the Seventh International Conference on Genetic Algorithms (ICGA97), T. Bäck (ed.), Morgan Kaufmann: East Lansing, 1997, pp. 34–40.Google Scholar
  41. 41.
    C. R. Stephens and H. Waelbroeck, “Schemata evolution and building blocks,” Evolutionary Comp., vol. 7, no. 2, pp. 109–124, 1999.Google Scholar
  42. 42.
    M. D. Vose, The Simple Genetic Algorithm: Foundations and Theory, MIT Press: Cambridge, MA, 1999.Google Scholar
  43. 43.
    M. D. Vose and G. E. Liepins, “Punctuated equilibria in genetic search,” Complex Systems, vol. 5, no. 1, pp. 31–44, 1991.Google Scholar
  44. 44.
    P. A. Whigham, “A schema theorem for context-free grammars,” in 1995 IEEE Conference on Evolutionary Computation, Perth, Australia, 29 Nov-1 Dec. 1995, IEEE Press, vol. 1, 1995, pp. 178–181.Google Scholar
  45. 45.
    P. A. Whigham, “Grammatical Bias for Evolutionary Learning, 14 Oct. 1996,” PhD thesis, School of Computer Science, University College, University of New South Wales, Australian Defence Force Academy, Canberra, Australia.Google Scholar
  46. 46.
    D. Whitley, “A genetic algorithm tutorial,” Technical Report CS-93-103, Department of Computer Science, Colorado State University, Aug. 1993.Google Scholar

Copyright information

© Kluwer Academic Publishers 2004

Authors and Affiliations

  • Riccardo Poli
    • 1
  • Nicholas Freitag McPhee
    • 2
  • Jonathan E. Rowe
    • 3
  1. 1.Department Computer ScienceUniversity of EssexColchesterUK
  2. 2.Division of Science and MathematicsUniversity of Minnesota, MorrisMorrisUSA
  3. 3.School of Computer ScienceThe University of BirminghamBirminghamUK

Personalised recommendations