Advertisement

Statistics and Computing

, Volume 4, Issue 2, pp 87–112 | Cite as

Genetic programming as a means for programming computers by natural selection

  • John R. Koza
Article

Abstract

Many seemingly different problems in machine learning, artificial intelligence, and symbolic processing can be viewed as requiring the discovery of a computer program that produces some desired output for particular inputs. When viewed in this way, the process of solving these problems becomes equivalent to searching a space of possible computer programs for a highly fit individual computer program. The recently developed genetic programming paradigm described herein provides a way to search the space of possible computer programs for a highly fit individual computer program to solve (or approximately solve) a surprising variety of different problems from different fields. In genetic programming, populations of computer programs are genetically bred using the Darwinian principle of survival of the fittest and using a genetic crossover (sexual recombination) operator appropriate for genetically mating computer programs. Genetic programming is illustrated via an example of machine learning of the Boolean 11-multiplexer function and symbolic regression of the econometric exchange equation from noisy empirical data.

Hierarchical automatic function definition enables genetic programming to define potentially useful functions automatically and dynamically during a run, much as a human programmer writing a complex computer program creates subroutines (procedures, functions) to perform groups of steps which must be performed with different instantiations of the dummy variables (formal parameters) in more than one place in the main program. Hierarchical automatic function definition is illustrated via the machine learning of the Boolean 11-parity function.

Keywords

Genetic programming genetic algorithm crossover hierarchical automatic function definition symbolic regression Boolean 11-multiplexer econometric exchange equation Boolean 11-parity 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Belew, R. and Booker, L. (eds) (1991). Proceedings of the Fourth International Conference on Genetic Algorithms. Morgan Kaufmann, San Mateo, CA.Google Scholar
  2. Citibank (1989). CITIBASE: Citibank Economic Database (Machine Readable Magnetic Data File), 1946-Present. Citibank N.A., New York.Google Scholar
  3. Cramer, N. L. (1985). A representation for the adaptive generation of simple sequential programs. In Proceedings of an International Conference on Genetic Algorithms and Their Applications, ed. J. Grefenstette. Lawrence Erlbaum, Hillsdale, NJ.Google Scholar
  4. Davidor, Y. (1991). Genetic Algorithms and Robotics. World Scientific, Singapore.Google Scholar
  5. Davis, L. (ed) (1987). Genetic Algorithms and Simulated Annealing. Pitman, London.Google Scholar
  6. Davis, L. (1991). Handbook of Genetic Algorithms. Van Nostrand Reinhold, New York.Google Scholar
  7. Forrest, S. (ed). (1990). Emergent Computation: Self-Organizing, Collective, and Cooperative Computing Networks. MIT Press, Cambridge, MA.Google Scholar
  8. Fujiki, C. and Dickinson, J. (1987). Using the genetic algorithm to generate LISP source code to solve the prisoner's dilemma. In Genetic Algorithms and Their Applications: Proceedings of the Second International Conference on Genetic Algorithms, ed. J. Grefenstette. Lawrence Erlbaum, Hillsdale, NJ.Google Scholar
  9. Goldberg, D. (1989). Genetic Algorithms in Search, Optimization, and Machine Learning, Addison-Wesley, Reading, MA.Google Scholar
  10. Goldberg, D. E., Korb, B. and Deb, K. (1989). Messy genetic algorithms: motivation, analysis, and first results. Complex Systems, 3, 493–530.PubMedGoogle Scholar
  11. Gruau, F. (1992). Genetic synthesis of Boolean neural networks with a cell rewriting developmental process. In Proceedings of the Workshop on Combinations of Genetic Algorithms and Neural Networks 1992, ed. J. D. Schaffer and D. Whitley. The IEEE Computer Society Press.Google Scholar
  12. Hallman, J. J., Porter, R. D. and Small, D. H. (1989). M2 per Unit of Potential GNP as an Anchor for the Price Level. Board of Governors of the Federal Reserve System. Staff Study 157, Washington, DC.Google Scholar
  13. Holland, J. H. (1975). Adaptation in Natural and Artificial Systems. University of Michigan Press, Ann Arbor, MI.Google Scholar
  14. Holland, J. H. (1986). Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rulebased systems. In Machine Learning: An Artificial Intelligence Approach, Volume II, ed. R. S. Michalski et al. pp. 593–623. Morgan Kaufmann, Los Altos, CA.Google Scholar
  15. Holland, J. H., Holyoak, K. J., Nisbett, R. E. and Thagard, P. A. (1986). Induction: Processes of Inference, Learning, and Discovery. MIT Press, Cambridge, MA.Google Scholar
  16. Koza, J. R. (1990). Genetic Programming: A Paradigm for Genetically Breeding Populations of Computer Programs to Solve Problems. Stanford University Computer Science Department technical report STAN-CS-90-1314.Google Scholar
  17. Koza, J. R. (1992a). Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge, MA.Google Scholar
  18. Koza, J. R. (1992b). Genetic programming: genetically breeding populations of computer programs to solve problems. In Dynamic, Genetic, and Chaotic Programming, ed. B. Soucek and the IRIS Group. John Wiley, New York.Google Scholar
  19. Koza, J. R. (1992c). Hierarchical automatic function definition in genetic programming. In Proceedings of Workshop on the Foundations of Genetic Algorithms and Classifier Systems, Vail, Colorado 1992, ed. D. Whitley. Morgan Kaufmann, San Mateo, CA.Google Scholar
  20. Koza, J. R. (1992d). Evolution of subsumption using genetic programming. In Proceedings of European Conference on Artificial Life, Paris, December 1991, ed. P. Bourgine and F. Varela. MIT Press, Cambridge, MA.Google Scholar
  21. Koza, J. R. and Keane, M. A. (1990a). Cart centering and broom balancing by genetically breeding populations of control strategy programs. In Proceedings of International Joint Conference on Neural Networks, Washington, January 15–19,1990. Volume I, pp. 198–201. Lawrence Erlbaum, Hillsdale, NJ.Google Scholar
  22. Koza, R. and Keane, M. A. (1990b). Genetic breeding of nonlinear optimal control strategies for broom balancing. In Proceedings of the Ninth International Conference on Analysis and Optimization of Systems. Antibes, France, June, 1990, pp. 47–56. Springer-Verlag, Berlin.Google Scholar
  23. Koza, J. R. and Rice, J. P. (1991). Genetic generation of both the weights and architecture for a neural network. In Proceedings of International Joint Conference on Neural Networks, Seattle, July 1991. Volume II, pp. 397–404. IEEE Press.Google Scholar
  24. Koza, J. R. and Rice, J. P. (1992a). Genetic Programming: The Movie. MIT Press, Cambridge, MA.Google Scholar
  25. Koza, J. R. and Rice, J. P. (1992b). Automatic programming of robots using genetic programming. In Proceedings of Tenth National Conference on Artificial Intelligence, pp. 194–201. AAAI Press/MIT Press, Menlo Park, CA.Google Scholar
  26. Koza, J. R., Keane, M. A. and Rice, J. P. (1993). Performance improvement of machine learning via automatic discovery of facilitating functions as applied to a problem of symbolic system identification. In 1993 IEEE International Conference on Neural Networks, San Francisco, Volume I, pp. 191–198. IEEE Press, Piscataway, NJ.Google Scholar
  27. Langton, C., Taylor, C., Farmer, J. D. and Rasmussen, S. (eds). (1992). Artificial Life II, SFI Studies in the Sciences of Complexity, Volume X. Addison-Wesley, Redwood City, CA.Google Scholar
  28. Meyer, J.-A. and Wilson, S. W. (1991). From Animals to Animats: Proceedings of the First International Conference on Simulation of Adaptive Behavior, Paris. September 24–28, 1990. MIT Press, Cambridge, MA.Google Scholar
  29. Michaelewicz, Z. (1992). Genetic Algorithms + Data Structures = Evolution Programs. Springer-Verlag, New York.Google Scholar
  30. Rawlins, G. (ed) (1991). Proceedings of Workshop on the Foundations of Genetic Algorithms and Classifier Systems, Bloomington, Indiana, July 15–18, 1990. Morgan Kaufmann, San Mateo, CAGoogle Scholar
  31. Samuel, A. L. (1959). Some studies in machine learning using the game of checkers. IBM Journal of Research and Development, 3, 210–229.PubMedGoogle Scholar
  32. Schaffer, J. D. (ed) (1989). Proceedings of the Third International Conference on Genetic Algorithms. Morgan Kaufmann, San Mateo, CA.Google Scholar
  33. Schwefel, H.-P. and Maenner, R. (eds) (1991). Parallel Problem Solving from Nature. Springer-Verlag, Berlin.Google Scholar
  34. Smith, S. F. (1980). A Learning System Based on Genetic Adaptive Algorithms. PhD dissertation, University of Pittsburgh, Pittsburgh, PA.Google Scholar
  35. Whitley, D. (ed) (1992). Proceedings of Workshop on the Foundations of Genetic Algorithms and Classifier Systems, Vail, Colorado 1992. Morgan Kaufmann, San Mateo, CA.Google Scholar
  36. Wilson, S. W. (1987a). Classifier systems and the animat problem. Machine Learning, 3, 199–228.PubMedGoogle Scholar
  37. Wilson, S. W. (1987b). Hierarchical credit allocation in a classifier system. In Proceedings of the Tenth International Joint Conference on Artificial Intelligence, pp. 217–220. Morgan Kaufmann, San Mateo, CA.Google Scholar
  38. Wilson, S. W. (1988). Bid competition and specificity reconsidered. Jounal of Complex Systems, 2, 705–723.Google Scholar

Copyright information

© Chapman & Hall 1994

Authors and Affiliations

  • John R. Koza
    • 1
  1. 1.Computer Science DepartmentStanford UniversityStanfordUSA

Personalised recommendations