# Exact Schema Theory and Markov Chain Models for Genetic Programming and Variable-length Genetic Algorithms with Homologous Crossover

- 136 Downloads
- 27 Citations

## Abstract

Genetic Programming (GP) homologous crossovers are a group of operators, including GP one-point crossover and GP uniform crossover, where the offspring are created preserving the position of the genetic material taken from the parents. In this paper we present an exact schema theory for GP and variable-length Genetic Algorithms (GAs) which is applicable to this class of operators. The theory is based on the concepts of GP crossover masks and GP recombination distributions that are generalisations of the corresponding notions used in GA theory and in population genetics, as well as the notions of hyperschema and node reference systems, which are specifically required when dealing with variable size representations.

In this paper we also present a Markov chain model for GP and variable-length GAs with homologous crossover. We obtain this result by using the core of Vose's model for GAs in conjunction with the GP schema theory just described. The model is then specialised for the case of GP operating on 0/1 trees: a tree-like generalisation of the concept of binary string. For these, symmetries exist that can be exploited to obtain further simplifications.

In the absence of mutation, the Markov chain model presented here generalises Vose's GA model to GP and variable-length GAs. Likewise, our schema theory generalises and refines a variety of previous results in GP and GA theory.

## Preview

Unable to display preview. Download preview PDF.

## References

- 1.L. Altenberg, "Emergent phenomena in genetic programming,” in Evolutionary Programming-Proceedings of the Third Annual Conference, A. V. Sebald and L. J. Fogel (eds.), World Scientific Publishing: San Diego, CA, USA, 24-26 Feb. 1994, pp. 233–241.Google Scholar
- 2.L. Altenberg, “The schema theorem and price's Theorem,” in Foundations of Genetic Algorithms 3, L. D. Whitley and M. D. Vose (eds.), Morgan Kaufmann: San Francisco, CA, USA, 1995, pp. 23–49.Google Scholar
- 3.L. B. Booker, “Recombination distributions for genetic algorithms,” in Foundations of Genetic Algorithms 2, L. D. Whitley (ed.), Morgan Kaufmann: San Francisco, CA, 1993, pp. 29–44.Google Scholar
- 4.T. E. Davis and J. C. Principe, “A Markov chain framework for the simple genetic algorithm,” Evolutionary Comp., vol. 1, no. 3, pp. 269–288, 1993.Google Scholar
- 5.H. Geiringer, “On the probability theory of linkage in Mendelian heredity,” Annals of Mathematical Statistics, vol. 15, no. 1, pp. 25–57, 1944.Google Scholar
- 6.D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Addison-Wesley: Reading, Massachusetts, 1989.Google Scholar
- 7.J. Holland, Adaptation in Natural and Artificial Systems, University of Michigan Press: Ann Arbor, USA, 1975.Google Scholar
- 8.J. R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection, MIT Press: Cambridge, MA, USA, 1992.Google Scholar
- 9.W. B. Langdon, “Size fair and homologous tree genetic programming crossovers,” Genetic Programming and Evolvable Machines, vol. 1, no. 1/2, pp. 95–119, 2000.Google Scholar
- 10.W. B. Langdon and R. Poli, Foundations of Genetic Programming, Springer-Verlag, 2002.Google Scholar
- 11.N. F. McPhee, R. Poli, and J. E. Rowe, "A schema theory analysis of mutation size biases in genetic programming with linear representations,” in Proceedings of the 2001 Congress on Evolutionary Computation CEC2001, COEX, World Trade Center, 159 Samseong-dong, Gangnam-gu, Seoul, Korea, 27-30 May 2001, IEEE Press, 2001, pp. 1078-1085.Google Scholar
- 12.M. Mitchell, An introduction to genetic algorithms, MIT Press: Cambridge, MA, 1996.Google Scholar
- 13.D. J. Montana, “Strongly typed genetic programming,” Evolutionary Compu., vol. 3, no. 2, pp. 199–230, 1995.Google Scholar
- 14.A. E. Nix and M. D. Vose, “Modeling genetic algorithms with Markov chains,” Annals of Mathematics and Artificial Intelligence, vol. 5, pp. 79–88, 1992.Google Scholar
- 15.U.-M. O'Reilly and F. Oppacher, "The troubling aspects of a building block hypothesis for genetic programming,” in Foundations of Genetic Algorithms 3, L. D. Whitley and M. D. Vose (eds.), Morgan Kaufmann: San Francisco, CA, 1995, pp. 73–88.Google Scholar
- 16.R. Poli, “Exact schema theory for genetic programming and variable-length genetic algorithms with one-point crossover,” Genetic Programming and Evolvable Machines, vol. 2, no. 2, pp. 123–163, June 2001.Google Scholar
- 17.R. Poli and W. B. Langdon, “On the search properties of different crossover operators in genetic programming,” in Genetic Programming 1998: Proceedings of the Third Annual Conference, University of Wisconsin, Madison, Wisconsin, USA, 22-25 July 1998, J. R. Koza, W. Banzhaf, K. Chellapilla, K. Deb, M. Dorigo, D. B. Fogel, M. H. Garzon, D. E. Goldberg, H. Iba, and R. Riolo (eds.), Morgan Kaufmann: University of Wisconsin, Madison, Wisconsin, USA, 22-25 July 1998, pp. 293–301.Google Scholar
- 18.R. Poli and W. B. Langdon, “Schema theory for genetic programming with one-point crossover and point mutation,” Evolutionary Computation, vol. 6, no. 3, pp. 231–252, 1998.Google Scholar
- 19.R. Poli and N. F. McPhee, “Exact GP schema theory for headless chicken crossover and subtree mutation,” in Proceedings of the 2001 Congress on Evolutionary Computation CEC2001, IEEE Press: COEX, World Trade Center, 159 Samseong-dong, Gangnam-gu, Seoul, Korea, 27-30 May 2001, pp. 1062–1069.Google Scholar
- 20.R. Poli and N. F. McPhee, “Exact schema theory for GP and variable-length GAs with homologous crossover,” in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), San Francisco, California, USA, 7-11 July 2001, L. Spector, E. D. Goodman, A. Wu, W. B. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. H. Garzon, and E. Burke (eds.), Morgan Kaufmann: San Francisco, California, USA, 7-11 July 2001, pp. 104–111.Google Scholar
- 21.R. Poli and N. F. McPhee, “General schema theory for genetic programming with subtree-swapping crossover: Part I,” Evolutionary Comp., vol. 11, no. 1, pp. 53–66, 2003.Google Scholar
- 22.R. Poli and N. F. McPhee, “General schema theory for genetic programming with subtree-swapping crossover: Part II,” Evolutionary Comp., vol. 11, no. 2, pp. 169–206, 2003.Google Scholar
- 23.R. Poli, J. E. Rowe, and N. F. McPhee, “Markov chain models for GP and variable-length GAs with homologous crossover,” in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), L. Spector, E. D. Goodman, A. Wu, W. B. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. H. Garzon, and E. Burke (eds.), Morgan Kaufmann: San Francisco, California, USA, 7-11 July 2001, pp. 112–119.Google Scholar
- 24.R. Poli, C. R. Stephens, A. H. Wright, and J. E. Rowe, “On the search biases of homologuous crossover in linear genetic programming and variable-length genetic algorithms,” in GECCO 2002: Proceedings of the Genetic and Evolutionary Computation Conference, W. B. Langdon, E. Cantú -Paz, K. Mathias, R. Roy, D. Davis, R. Poli, K. Balakrishnan, V. Honavar, G. Rudolph, J. Wegener, L. Bull, M. A. Potter, A. C. Schultz, J. F. Miller, E. Burke, and N. Jonoska (eds.), Morgan Kaufmann Publishers, New York, 9-13 July 2002, pp. 868–876.Google Scholar
- 25.R. Poli, C. R. Stephens, A. H. Wright, and A. H. Rowe, “A schema-theory-based extension of Geiringer's theorem for linear GP and variable-length GAs under homologous crossover,” in Foundations of Genetic Algorithm 7, K. D. Jong, R. Poli, and J. Rowe (eds.), Morgan Kaufmann: San Francisco, CA, 2003, pp. 45–62.Google Scholar
- 26.A. Prügel-Bennett and J. L. Shapiro, “An analysis of genetic algorithms using statistical mechanics,” Physical Review Letters, vol. 72, pp. 1305–1309, 1994.Google Scholar
- 27.N. J. Radcliffe, “Schema processing,” Handbook of Evolutionary Computation, T. Baeck, D. B. Fogel, and Z. Michalewicz (eds.), Oxford University Press, 1997, pp. B2.5-1-B2.5-10.Google Scholar
- 28.C. Reeves and J. Rowe, “Genetic algorithms: principles and perspectives,” Kluwer Academic Press, 2003.Google Scholar
- 29.J. P. Rosca, “Analysis of complexity drift in genetic programming,” in Genetic Programming 1997: Proceedings of the Second Annual Conference, Stanford University, J. R. Koza, K. Deb, M. Dorigo, D. B. Fogel, M. Garzon, H. Iba, and R. L. Riolo (eds.), Morgan Kaufmann: CA, USA, 13-16 July 1997, pp. 286–294.Google Scholar
- 30.J. E. Rowe, “Population fixed-points for functions of unitation,” in Foundations of Genetic Algorithms 5, W. Banzhaf and C. Reeves (eds.), Morgan Kaufmann: San Francisco, CA, 1999, pp. 69–84.Google Scholar
- 31.J. E. Rowe, M. D. Vose, and A. H. Wright, “Group properties of crossover and mutation,” Evolutionary Comp., vol. 10, no. 2, pp. 151–184, 2002.Google Scholar
- 32.G. Rudolph, “Convergence analysis of canonical genetic algorithm,” IEEE Transactions on Neural Networks, vol. 5, no. 1, pp. 96–101, 1994.Google Scholar
- 33.G. Rudolph, “Genetic algorithms,” in Handbook of Evolutionary Computation, T. Baeck, D. B. Fogel, and Z. Michalewicz (eds.), Oxford University Press, 1997, pp. B2.4-20-B2.4-27.Google Scholar
- 34.G. Rudolph, “Models of stochastic convergence,” in Handbook of Evolutionary Computation, T. Baeck, D. B. Fogel, and Z. Michalewicz (eds.), Oxford University Press, 1997, pp. B2.3-1-B2.3-3.Google Scholar
- 35.G. Rudolph, “Stochastic processes,” in Handbook of Evolutionary Computation, T. Baeck, D. B. Fogel, and Z. Michalewicz (eds.), Oxford University Press, 1997, pp. B2.2-1-B2.2-8.Google Scholar
- 36.W. M. Spears, “Aggregating models of evolutionary algorithms,” in Proceedings of the Congress on Evolutionary Computation, P. J. Angeline, Z. Michalewicz, M. Schoenauer, X. Yao, and A. Zalzala (eds.), Mayflower Hotel, Washington D.C., USA, 6-9 July 1999, IEEE Press, vol. 1, 1999, pp. 631–638.Google Scholar
- 37.W. M. Spears, “The equilibrium and transiente behaviour of mutation and recombination,” in Foundations of Genetic Algorithms Workshop 6, W. M. Spears and W. Martin (eds.), Morgan Kaufmann: San Francisco, CA, 2001, pp. 241–260.Google Scholar
- 38.M. R. Spiegel, Probability and Statistics, McGraw-Hill: New York, 1975.Google Scholar
- 39.C. R. Stephens, “Some exact results from a coarse grained formulation of genetic dynamics,” in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), L. Spector, E. D. Goodman, A. Wu, W. B. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. H. Garzon, and E. Burke (eds.), Morgan Kaufmann: San Francisco, California, USA, 7-11 July 2001, pp. 631–638.Google Scholar
- 40.C. R. Stephens and H. Waelbroeck, “Effective degrees of freedom in genetic algorithms and the block hypothesis,” in Proceedings of the Seventh International Conference on Genetic Algorithms (ICGA97), T. Bäck (ed.), Morgan Kaufmann: East Lansing, 1997, pp. 34–40.Google Scholar
- 41.C. R. Stephens and H. Waelbroeck, “Schemata evolution and building blocks,” Evolutionary Comp., vol. 7, no. 2, pp. 109–124, 1999.Google Scholar
- 42.M. D. Vose, The Simple Genetic Algorithm: Foundations and Theory, MIT Press: Cambridge, MA, 1999.Google Scholar
- 43.M. D. Vose and G. E. Liepins, “Punctuated equilibria in genetic search,” Complex Systems, vol. 5, no. 1, pp. 31–44, 1991.Google Scholar
- 44.P. A. Whigham, “A schema theorem for context-free grammars,” in 1995 IEEE Conference on Evolutionary Computation, Perth, Australia, 29 Nov-1 Dec. 1995, IEEE Press, vol. 1, 1995, pp. 178–181.Google Scholar
- 45.P. A. Whigham, “Grammatical Bias for Evolutionary Learning, 14 Oct. 1996,” PhD thesis, School of Computer Science, University College, University of New South Wales, Australian Defence Force Academy, Canberra, Australia.Google Scholar
- 46.D. Whitley, “A genetic algorithm tutorial,” Technical Report CS-93-103, Department of Computer Science, Colorado State University, Aug. 1993.Google Scholar