Advertisement

Genetic Programming and Evolvable Machines

, Volume 15, Issue 2, pp 115–167 | Cite as

Probabilistic model building in genetic programming: a critical review

  • Kangil Kim
  • Yin Shan
  • Xuan Hoai Nguyen
  • R. I. McKayEmail author
Article

Abstract

Probabilistic model-building algorithms (PMBA), a subset of evolutionary algorithms, have been successful in solving complex problems, in addition providing analytical information about the distribution of fit individuals. Most PMBA work has concentrated on the string representation used in typical genetic algorithms. A smaller body of work has aimed to apply the useful concepts of PMBA to genetic programming (GP), mostly concentrating on tree representation. Unfortunately, the latter research has been sporadically carried out, and reported in several different research streams, limiting substantial communication and discussion. In this paper, we aim to provide a critical review of previous applications of PMBA and related methods in GP research, to facilitate more vital communication. We illustrate the current state of research in applying PMBA to GP, noting important perspectives. We use these to categorise practical PMBA models for GP, and describe the main varieties on this basis.

Keywords

Probabilistic model building Estimation of distribution Ant colony Genetic programming Iterated density estimation Prototype tree Stochastic grammar 

Notes

Acknowledgments

This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology (Project No. 2012-004841). Xuan Hoai Nguyen was partly funded by The Vietnam National Foundation for Science and Technology Development (NAFOSTED), under Grant Number 102.01–2011.08, for doing this work. The ICT at Seoul National University provided research facilities for this study.

References

  1. 1.
    H.A. Abbass, X. Hoai, R.I. McKay. AntTAG: A new method to compose computer programs using colonies of ants. In Proceedings of the 2002 IEEE Congress on Evolutionary Computation, vol. 2, Honolulu, HI, USA. (IEEE Press, New York, 2002), p. 1654–1659Google Scholar
  2. 2.
    P.J. Angeline. Subtree crossover: Building block engine or macromutation? In Genetic Programming 1997: Proceedings of the Second Annual Conference, pages 9–17, Stanford University, CA, USA, 13–16 July 1997. Morgan KaufmannGoogle Scholar
  3. 3.
    P.J. Angeline, J.B. Pollack. Coevolving high-level representations. In Artificial Life III, vol. XVII of Santa Fe Institute Series, pages 55–71, Santa Fe, New Mexico, USA, 15–19 June 1994. Addison-Wesley, USAGoogle Scholar
  4. 4.
    S. Baluja. Population-based incremental learning: a method for integrating genetic searching based function optimization. Technical Report CMU-CS-94-163, Computer Science Dept, Carnegie Mellon University, Pittsburgh, PA, USA, 1994Google Scholar
  5. 5.
    W. Banzhaf. Genetic programming for pedestrians. In Proceedings of the 5th International Conference on Genetic Algorithms, ICGA-93, p. 628, Urbana-Champaign, IL, USA, 17–21 July 1993. Morgan KaufmannGoogle Scholar
  6. 6.
    W. Banzhaf, P. Nordin, R.E. Keller, F.D. Francone. Genetic Programming: An Introduction; On the Automatic Evolution of Computer Programs and Its Applications. Morgan Kaufmann, San Francisco, 1998Google Scholar
  7. 7.
    H.G. Beyer, An alternative explanation for the manner in which genetic algorithms operate. BioSystems 41, 1–15 (1997)CrossRefGoogle Scholar
  8. 8.
    M. Birattari, G. Di Caro, and M. Dorigo. Toward the formal foundation of ant programming. In Proceedings of the Third International Workshop on Ant Algorithms, volume 2463 of Lecture Notes in Computer Science, pages 188–201, Brussels, Belgium September 2002. (Springer, Berlin), p. 12–14Google Scholar
  9. 9.
    C.M. Bishop, Pattern Recognition and Machine Learning (Information Science and Statistics). (Springer, New York, Inc., Secaucus, 2006)Google Scholar
  10. 10.
    M. Boryczka, Eliminating introns in ant colony programming. Fundamenta Informaticae 68(1-2), 1–19 (2005)zbMATHMathSciNetGoogle Scholar
  11. 11.
    M. Boryczka. Ant colony programming with the candidate list. In Proceedings of the Agent and Multi-Agent Systems: Technologies and Applications, Second KES International Symposium, KES-AMSTA 2008, volume 4953 of Lecture Notes in Computer Science, pp. 302–311, Incheon, Korea, 26–28 March 2008. (Springer, Berlin)Google Scholar
  12. 12.
    M. Boryczka, Z.J. Czech. Solving approximation problems by ant colony programming. In GECCO 2002: Proceedings of the Genetic and Evolutionary Computation Conference, New York, USA, 9-13 July 2002. (Morgan Kaufmann Publishers), p. 133Google Scholar
  13. 13.
    P.A.N. Bosman, E.D. de Jong. Grammar transformations in an EDA for genetic programming. In Proceedings of the Optimization by Building and Using Probabilistic Models OBUPM Workshop at the Genetic and Evolutionary Computation Conference—GECCO-2004, Seattle, Washington, USA, June 2004. (Springer, Berlin)Google Scholar
  14. 14.
    P.A.N. Bosman, E.D. de Jong. Learning probabilistic tree grammars for genetic programming. In Parallel Problem Solving from Nature - PPSN VIII, volume 3242 of Lecture Notes in Computer Science Birmingham, UK, Sep 2004. (Springer, Berlin), p. 192–201Google Scholar
  15. 15.
    P.A.N. Bosman, D. Thierens. Continuous iterated density estimation evolutionary algorithms within the IDEA framework. In Proceedings of the Optimization by Building and Using Probabilistic Models OBUPM Workshop at the Genetic and Evolutionary Computation Conference (GECCO-2000), Las Vegas, Nevada, USA, 8–12 July 2000. (Morgan Kaufmann, Burlington), p. 197–200Google Scholar
  16. 16.
    E.K. Burke, M.R. Hyde, G. Kendall, G. Ochoa, E. Ozcan, and J.R. Woodward. Exploring hyper-heuristic methodologies with genetic programming, volume 1 of Intelligent systems Reference Library, chapter 6 Springer, 2009, pp 177–201Google Scholar
  17. 17.
    J. Clegg. Combining cartesian genetic programming with an estimation of distribution algorithm. In GECCO ’08: Proceedings of the 10th annual conference on Genetic and evolutionary computation, Atlanta, GA, USA, 12-16 July 2008. ACM, p. 1333–1334Google Scholar
  18. 18.
    N. L. Cramer. A representation for the adaptive generation of simple sequential programs. In Proceedings of an International Conference on Genetic Algorithms and the Applications, Carnegie Mellon University, Pittsburgh, PA, USA, 24-26 July 1985, p. 183–187Google Scholar
  19. 19.
    A.S. d’Avila Garcez, K. Broda, D.M. Gabbay, Symbolic knowledge extraction from trained neural networks: a sound approach. Artif. Intell. 125(1-2), 155–207 (2001)CrossRefzbMATHMathSciNetGoogle Scholar
  20. 20.
    A.P Dempster, N.M. Laird, D.B. Rubin, Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Statis. Soc. Series B (Methodological) 39(1), 1–38 (1977)zbMATHMathSciNetGoogle Scholar
  21. 21.
    M. Dorigo, T. Stützle, Ant Colony Optimization. (MIT Press, Cambridge, MA, 2004)CrossRefzbMATHGoogle Scholar
  22. 22.
    L. Getoor, B. Taskar, Introduction to Statistical Relational Learning. (The MIT Press, Cambridge, 2007)zbMATHGoogle Scholar
  23. 23.
    A. Geyer-Schulz, Fuzzy Rule-Based Expert Systems and Genetic Machine Learning, volume 3 of Studies in Fuzziness. (Physica-Verlag, Heidelberg, 1995)Google Scholar
  24. 24.
    D.E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning. (Addison-Wesley, USA, 1989)zbMATHGoogle Scholar
  25. 25.
    J. Green, J.L. Whalley, C.G. Johnson. Automatic programming with ant colony optimization. In Proceedings of the 2004 UK Workshop on Computational Intelligence, UK, 6–8 September 2004. (Loughborough University, Loughborough), pp. 70–77Google Scholar
  26. 26.
    F. Gruau. Genetic synthesis of modular neural networks. In Proceedings of the 5th International Conference on Genetic Algorithms, ICGA-93, Urbana-Champaign, IL, USA, 17-21 July 1993. (Morgan Kaufmann, Burlington), pp. 318–325Google Scholar
  27. 27.
    P. Haddawy. Generating bayesian networks from probability logic knowledge bases. In Proceedings of the Tenth Conference on Uncertainty in Artificial Intelligence, Seatle, WA, USA, 29-31 July 1994. (Morgan Kaufmann Publishers Inc, Burlington) pp. 262–269Google Scholar
  28. 28.
    S. Handley. On the use of a directed acyclic graph to represent a population of computer programs. In Proceedings of the First IEEE Conference on Evolutionary Computation, IEEE World Congress on Computational Intelligence, vol. 1, pp. 154–159, 1994Google Scholar
  29. 29.
    N. Hansen and A. Ostermeier. Adapting arbitrary normal mutation distributions in evolution strategies: The covariance matrix adaptation. In Evolutionary Computation, 1996., Proceedings of IEEE International Conference on. IEEE, 1996, pp. 312–317Google Scholar
  30. 30.
    G. Harik, F. Lobo, and K. Sastry. Linkage Learning via Probabilistic Modeling in the Extended Compact Genetic Algorithm (ECGA), volume 33 of Studies in Computational Intelligence. (Springer, Berlin, 2006), pp. 39–61Google Scholar
  31. 31.
    Y. Hasegawa and H. Iba. Estimation of Bayesian network for program generation. In Proceedings of the Third Asian-Pacific workshop on Genetic Programming, pp. 35–46, Military Technical Academy, Hanoi, VietNam, 2006Google Scholar
  32. 32.
    Y. Hasegawa and H. Iba. Estimation of distribution algorithm based on probabilistic grammar with latent annotations. In Proceedings of the 2007 IEEE Congress on Evolutionary Computation, Singapore, 25–28 September 2007. IEEE Computational Intelligence Society, IEEE Press, pp. 1043–1050Google Scholar
  33. 33.
    Y. Hasegawa, H. Iba, A Bayesian network approach to program generation. IEEE Trans. Evol. Comput. 12(6), 750–764 (2008)CrossRefGoogle Scholar
  34. 34.
    Y. Hasegawa, H. Iba, Latent variable model for estimation of distribution algorithm based on a probabilistic context-free grammar. IEEE Trans. Evol. Comput. 13(4), 858–878 (2009)CrossRefGoogle Scholar
  35. 35.
    E. Hemberg, K. Veeramachaneni, J. McDermott, C. Berzan, and U.M. O’Reilly. An investigation of local patterns for estimation of distribution genetic programming. In GECCO ’12: Proceedings of the fourteenth international conference on Genetic and evolutionary computation conference. ACM, 7-11July 2012, pages 767–774Google Scholar
  36. 36.
    H. Iba, Y. Hasegawa, T.K. Paul, Genetic Programming and Machine Learning. (CRC Complex and Enterprise Systems Engineering. CRC Press, Boca Raton, 2009)zbMATHGoogle Scholar
  37. 37.
    C.Z. Janikow. Adapting representation in genetic programming. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2004), volume 3103 of Lecture Notes in Computer Science. (Springer, Berlin , 2004), pp. 507–518Google Scholar
  38. 38.
    A.K. Joshi, L.S. Levy, M. Takahashi, Tree adjunct grammars. J. Comput. Syst. Sci. 10(1), 136–163 (1975)CrossRefzbMATHMathSciNetGoogle Scholar
  39. 39.
    W. Kantschik and W. Banzhaf. Linear-tree GP and its comparison with other GP structures. In Genetic Programming, Proceedings of EuroGP’ 2001, volume 2038 of Lecture Notes in Computer Science, Lake Como, Italy, 18–20 April 2001. Springer, Berlin, pp. 302–312Google Scholar
  40. 40.
    H. Katagiri, K. Hirasama, and J. Hu. Genetic network programming-application to intelligent agents. In 2000 IEEE International Conference on Systems, Man, and Cybernetics, vol 5. IEEE, 8-11 October 2000, pp. 3829–3834Google Scholar
  41. 41.
    H. Katagiri, K. Hirasawa, J. Hu, and J. Murata. Network structure oriented evolutionary model—genetic network programming–and its comparison with. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), San Francisco, California, USA, 7–11 July 2001. (Morgan Kaufmann Publishers Inc., Burlington) p. 179Google Scholar
  42. 42.
    C. Keber, M. G. Schuster. Option valuation with generalized ant programming. In GECCO 2002: Proceedings of the Genetic and Evolutionary Computation Conference, Newyork, USA, 9–13 July 2002. (Morgan Kaufmann Publishers, Burlington), p. 74–81Google Scholar
  43. 43.
    K. Kim, B.(R.I.) McKay, D. Punithan. Sampling bias in estimation of distribution algorithms for genetic programming using prototype trees. In PRICAI 2010: 11th Pacific Rim International Conference on AI, volume 6230 of Lecture Notes in Artificial Intelligence. (Springer, Berlin 2010), pp. 100–111Google Scholar
  44. 44.
    K. Kim, R.I.(Bob) McKay, Stochastic diversity loss and scalability in estimation of distribution genetic programming. IEEE Trans. Evol. Comput. 17(3), 301–320 (2013)CrossRefGoogle Scholar
  45. 45.
    D. Koller, N. Friedman, Probabilistic Graphical Models: Principles and Techniques. (The MIT Press, Cambridge, 2009)Google Scholar
  46. 46.
    J.R. Koza. Genetic programming: A paradigm for genetically breeding populations of computer programs to solve problems. Technical Report STAN-CS-90-1314, Dept. of Computer Science, Stanford University, June 1990Google Scholar
  47. 47.
    J.R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection. (MIT press, Cambridge, 1992)zbMATHGoogle Scholar
  48. 48.
    W.B. Langdon, R. Poli, Foundations of Genetic Programming. (Springer, Berlin, 2002)CrossRefzbMATHGoogle Scholar
  49. 49.
    P. Larranaga, Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation (Genetic Algorithms and Evolutionary Computation). (Springer, Berlin, 2001)Google Scholar
  50. 50.
    P. Larranaga, H. Karshenas, C. Bielza, R. Santana, A review on probabilistic graphical models in evolutionary computation. J. Heuristics 18, 795–819 (2012)CrossRefGoogle Scholar
  51. 51.
    P.P. Le, A. Bah, L. H. Ungar. Using prior knowledge to improve genetic network reconstruction from microarray data. In Silico Biology, 4(27), 2004Google Scholar
  52. 52.
    X. Li, S. Mabu, H. Zhou, K. Shimada, and K. Hirasawa. Genetic network programming with estimation of distribution algorithms and its application to association rule mining for traffic prediction. In Proceedings of ICROS-SICE International Conference, 2009, Fukuoka, Japan, 18–21August 2009. IEEE, pages 3457–3462Google Scholar
  53. 53.
    X. Li, S. Mabu, H. Zhou, K. Shimada, and K. Hirasawa. Genetic network programming with estimation of distribution algorithms for class association rule mining in traffic prediction. In Proceedings of 2010 IEEE Congress on Evolutionary Computation (CEC 2010), Barcelona, Spain, 18–23 July 2010. (IEEE Press, New York) pp. 37–44Google Scholar
  54. 54.
    X. Liu, Y. Wu, and J. Ye. An improved estimation of distribution algorithm in dynamic environments. In 2008 Fourth International Conference on Natural Computation(ICNC’08), volume 6, p. 269–272, 2008Google Scholar
  55. 55.
    M. Looks. Scalable estimation-of-distribution program evolution. In Proceedings of the 9th annual conference on Genetic and evolutionary computation, volume 1 of GECCO ’07, London, UK, 7–11 July 2007. (ACM Press, New York) pp. 539–546Google Scholar
  56. 56.
    M. Looks, B. Goertzel, and C. Pennachin. Learning computer programs with the Bayesian optimization algorithm. In Proceedings of the 2005 conference on Genetic and evolutionary computation, volume 1 of GECCO ’05, pp. 747–748, Washington DC, USA, 25–29 June 2005. ACM Press, New YorkGoogle Scholar
  57. 57.
    C.D. Manning, H. Schutze, Foundations of Statistical Natural Language Processing. (MIT Press, Cambridge, 1999)zbMATHGoogle Scholar
  58. 58.
    T. Matsuzaki, Y. Miyao, and J. Tsujii. Probabilistic CFG with latent annotations. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), Ann Arbor, Michigan, USA, 25-30 June 2005. Association for Computational Linguistics., pp 75–82Google Scholar
  59. 59.
    N.F. McPhee, B. Ohs, T. Hutchison. Semantic building blocks in genetic programming. In Proceedings of the 11th European conference on Genetic programming, EuroGP 2008, volume 4971 of Lecture Notes in Computer Science, Naples, Italy, 26-28 March 2008. Springer, Berlin p. 134–145Google Scholar
  60. 60.
    R.S. Michalski, Learnable evolution model: Evolutionary processes guided by machine learning. Mach Learn 38, 9–40 (2000)CrossRefzbMATHGoogle Scholar
  61. 61.
    R.S. Michalski and J. Wojtusiak. The distribution approximation approach to learning from aggregated data. Technical Report MLI 08-2, Reports of the Machine Learning and Inference Laboratory, George Mason University, 2008Google Scholar
  62. 62.
    J.F. Miller. An empirical study of the efficiency of learning boolean functions using a cartesian genetic programming approach. In Proceedings of the Genetic and Evolutionary Computation Conference, volume 2, Orlando, Florida, USA, 13–17 July 1999. (Morgan Kaufmann Publishers Inc., San Francisco) pp. 1135–1142Google Scholar
  63. 63.
    J.F. Miller and P. Thomson. Cartesian genetic programming. In Genetic Programming, Proceedings of EuroGP’2000, volume 1802 of Lecture Notes in Computer Science, Edinburgh, UK, 15-16 April 2000. Springer, Berlin, pp121–132Google Scholar
  64. 64.
    D.J. Montana. Strongly typed genetic programming. BBN Technical Report #7866, Bolt Beranek and Newman, Inc., 10 Moulton Street, Cambridge, MA 02138, USA, 7 May 1993Google Scholar
  65. 65.
    D.J. Montana, Strongly typed genetic programming. Evol. Comput. 3(2), 199–230 (1995)CrossRefGoogle Scholar
  66. 66.
    H. Mühlenbein, J. Bendisch, and H.-M. Voigt. From recombination of genes to the estimation of distributions: II. continuous parameters. In Proceedings of the 4th International Conference on Parallel Problem Solving from Nature, volume 1141 of Lecture Notes in Computer Science, Berlin, Germany, 22-26 September 1996. (Springer, Berlin) pp. 188–197Google Scholar
  67. 67.
    H. Mühlenbein and G. Paass. From recombination of genes to the estimation of distributions I. binary parameters. In Proceedings of the 4th International Conference on Parallel Problem Solving from Nature, volume 1141 of Lecture Notes in Computer Science, Berlin, Germany, 22-26 September 1996. (Springer, Berlin) pp. 178–187Google Scholar
  68. 68.
    X.H. Nguyen, R.I. McKay, D. Essam, Representation and structural difficulty in genetic programming. IEEE Trans. Evol. Comput. 10(2), 157–166 (2006)CrossRefGoogle Scholar
  69. 69.
    P. Nordin. A compiling genetic programming system that directly manipulates the machine code. In Advances in genetic programming, chapter 14, pp. 311–331. (MIT Press, Cambridge, 1994)Google Scholar
  70. 70.
    P. Nordin and W. Banzhaf. Complexity compression and evolution. In Genetic Algorithms: Proceedings of the Sixth International Conference (ICGA95), Pittsburgh, PA, USA, 15-19July 1995. (Morgan Kaufmann Publisher Inc., San Francisco), pp. 310–317Google Scholar
  71. 71.
    P. Nordin, F. Francone, and W. Banzhaf. Explicitly defined introns and destructive crossover in genetic programming. In Peter J. Angeline and K. E. Kinnear, Jr., editors, Advances in Genetic Programming 2, chapter 6. (MIT Press, Cambridge, 1996), pp. 111–134Google Scholar
  72. 72.
    J.L. Olmo, J. R. Romero, and S. Ventura. A grammar based ant programming algorithm for mining classification rules. In Proceedings of 2010 IEEE Congress on Evolutionary Computation (CEC 2010), Barcelona, Spain, 18–23 July 2010. IEEE Press, pages 225–232Google Scholar
  73. 73.
    O’Neill M., Ryan C. (2001) Grammatical evolution. IEEE Trans. Evol. Comput. 5(4):349–358Google Scholar
  74. 74.
    U.M. O’Reilly. An Analysis of Genetic Programming. PhD thesis, Carleton University, Ottawa-Carleton Institute for Computer Science, Ottawa, Ontario, Canada, 22 September 1995Google Scholar
  75. 75.
    U.M. O’Reilly. Investigating the generality of automatically defined functions. In Genetic Programming 1996: Proceedings of the First Annual Conference, Stanford University, CA, USA, 28–31 July 1996. MIT Press, pages 351–356Google Scholar
  76. 76.
    U.M. O’Reilly and F. Oppacher. The troubling aspects of a building block hypothesis for genetic programming. In Foundations of Genetic Algorithms 3, Estes Park, Colorado, USA, 31 July–2 August 1994. Morgan Kaufmann Publishers Inc. Published 1995, pages 73–88Google Scholar
  77. 77.
    A. Ortega, de la M. Cruz, M. Alfonseca, Christiansen grammar evolution: Grammatical evolution with semantics. IEEE Trans. Evol. Comput. 11(1), 77–90 (2007)CrossRefGoogle Scholar
  78. 78.
    S.J. Pan, Q. Yang, A survey on transfer learning. IEEE Trans. Know. Data Eng. 22(10), 1345–1359 (2010)CrossRefGoogle Scholar
  79. 79.
    M. Pelikan. A simple implementation of bayesian optimization algorithm (boa) in C++ (version 1.0). Technical Report 99011, IlliGAL, University of Illinois at Urbana–Champaign, 1999Google Scholar
  80. 80.
    M. Pelikan. A C++ implementation of bayesian optimization algorithm with decision graphs. Technical Report 2000025, IlliGAL, University of Illinois at Urbana–Champaign, 2000Google Scholar
  81. 81.
    M. Pelikan. Implementation of the dependency-tree estimation of distribution algorithm in C++. Technical Report 2006010, IlliGAL, University of Illinois at Urbana–Champaign, 2006Google Scholar
  82. 82.
    M. Pelikan, D.E. Goldberg. Escaping hierarchical traps with competent genetic algorithms. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO2001), San Francisco, California, July 2001. Morgan Kaufmann Publishers IncGoogle Scholar
  83. 83.
    M. Pelikan, D. E. Goldberg, and E. Cantu-Paz. BOA: The Bayesian optimization algorithm. In Proceedings of the Genetic and Evolutionary Computation Conference GECCO-99, volume 1, Orlando, Florida, USA, 13-17 July 1999. Morgan Kaufamann, pp. 525–532Google Scholar
  84. 84.
    M. Pelikan, D. E. Goldberg, and F. G. Lobo. A survey of optimization by building and using probabilistic models. In Proceedings of the 2000 American Control Conference, volume 5, Baltimore, MD, USA, 28–30 June 2000, pages 3289–3293Google Scholar
  85. 85.
    M. Pelikan, D.E. Goldberg, F.G. Lobo, A survey of optimization by building and using probabilistic models. Comput. Optim. Appl. 21(1), 5–20 (2002)CrossRefzbMATHMathSciNetGoogle Scholar
  86. 86.
    M. Pelikan, M. Hauschild, and P. Lanzi. Transfer learning, soft distance-based bias, and the hierarchical boa. In Parallel Problem Solving from Nature - PPSN XII, volume 7491 of Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2012, pages 173–183Google Scholar
  87. 87.
    Poli R., Langdon W.B. (1998) Schema theory for genetic programming with one-point crossover and point mutation. Evol. Comput. 6(3):231–252Google Scholar
  88. 88.
    R. Poli and N. F. McPhee. A linear estimation-of-distribution GP system. In Proceedings of the 11th European conference on Genetic Programming, EuroGP 2008, volume 4971 of Lecture Notes in Computer Science, Naples, Italy, 26-28 March 2008. (Springer, Berlin), p. 206–217Google Scholar
  89. 89.
    A. Ratle and M. Sebag. Avoiding the bloat with probabilistic grammar-guided genetic programming. In Artificial Evolution 5th International Conference, Evolution Artificielle, EA 2001, volume 2310 of Lecture Notes in Computer Science, Creusot, France, 29-31 October 2001. (Springer, Berlin), p. 255–266Google Scholar
  90. 90.
    C.R. Reeves, J.E. Rowe, Genetic algorithms : principles and perspectives ; a guide to GA theory. (Kluwer, Netherland, 2004)Google Scholar
  91. 91.
    E.N. Regolin, A.T.R. Pozo. Bayesian automatic programming. In Proceedings of the 8th European Conference on Genetic Programming, EuroGP 2005, volume 3447 of Lecture Notes in Computer Science, Lausane, Switzerland, 30 Mar–1 Apr 2005. (Springer, Berlin, p. 38–49)Google Scholar
  92. 92.
    J. Rissanen, Modeling by shortest data description. Automatica 14(5), 465–471 (1978)CrossRefzbMATHGoogle Scholar
  93. 93.
    S.A. Rojas, P.J. Bentley. A grid-based ant colony system for automatic program synthesis. In Late Breaking Papers at the 2004 Genetic and Evolutionary Computation Conference, Seattle, Washington, USA, 26 July 2004Google Scholar
  94. 94.
    J.P. Rosca. Analysis of complexity drift in genetic programming. In Genetic Programming 1997: Proceedings of the Second Annual Conference, Stanford University, CA, USA, 13-16 July 1997. Morgan Kaufmann Publishers Inc., pp. 286–294Google Scholar
  95. 95.
    J.P. Rosca, D.H. Ballard. Genetic programming with adaptive representations. Technical Report TR 489, University of Rochester, Computer Science Department, Rochester, NY, USA, February 1994Google Scholar
  96. 96.
    O. Roux, C. Fonlupt. Ant programming: or how to use ants for automatic programming. In Proceedings of the Second International Conference on Ant Algorithms (ANTS2000), Brussels, Belgium, September 2000, p. 121–129Google Scholar
  97. 97.
    A. Salehi-Abari and T. White. Enhanced generalized ant programming (EGAP). In Proceedings of the 10th annual conference on Genetic and evolutionary computation(GECCO ’10), Portland, Oregon, USA, 7-11 July 2008. ACM New York, pages 111–118Google Scholar
  98. 98.
    R.P. Salustowicz, J. Schmidhüber, Probabilistic incremental program evolution. Evol. Comput. 5(2), 123–141 (1997)CrossRefGoogle Scholar
  99. 99.
    R. Santana, Estimation of distribution algorithms with Kikuchi approximations. Evol. Comput. 13(1), 67–97 (2005)CrossRefGoogle Scholar
  100. 100.
    R. Santana, C. Echegoyen, A. Mendiburu, C. Bielza, J. A. Lozano, P. Larraaga, R. Armaanzas, and S. Shakya. Mateda: A suite of EDA programs in Matlab. Technical Report EHU-KZAA-IK-2/09, University of the Basque Country, Feb 2009Google Scholar
  101. 101.
    R. Santana, R.I. McKay, and J.A. Lozano. Symmetry in evolutionary and estimation of distribution algorithms. In IEEE Congress on Evolutionary Computation, page To Appear. IEEE Computational Intelligence Society, (IEEE Press, New York 2013)Google Scholar
  102. 102.
    R. Santana, A. Mendiburu, and J.A. Lozano. Structural transfer using EDAs: An application to multi-marker tagging SNP selection. In 2012 IEEE Congress on Evolutionary Computation (CEC), p. 1–8, 2012Google Scholar
  103. 103.
    K. Sastry, L. de la Ossa, F.G. Lobo. χ–ary extended compact genetic algorithm for matlab in C++. Technical Report 2006013, IlliGAL, University of Illinois at Urbana–Champaign, 2006Google Scholar
  104. 104.
    K. Sastry, D.E. Goldberg. Probabilistic model building and competent genetic programming. Genetic Programming Theory and Practise, p. 205–220, 2003Google Scholar
  105. 105.
    Y. Shan, R.I. McKay, H.A. Abbass, D. Essam. Program evolution with explicit learning: a new framework for program automatic synthesis. In Proceedings of the 2003 IEEE Congress on Evolutionary Computation, CEC2003, pages 1639–1646, Canberra, Australia, 8-12 December 2003. University of New South Wales, IEEE Press, New YorkGoogle Scholar
  106. 106.
    Y. Shan, R.I. McKay, R. Baxter, H. Abbass, D. Essam, H.X. Nguyen. Grammar model-based program evolution. In Proceedings of the 2004 IEEE Congress on Evolutionary Computation, Portland, Oregon, USA, 20-23 June 2004. (IEEE Press, New York), p. 478–485Google Scholar
  107. 107.
    Y. Shan, R. I. McKay, D. Essam, and H. A. Abbass. A survey of probabilistic model building genetic programming. In Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications, volume 33 of Studies in Computational Intelligence, chapter 6. (Springer, Berlin, 2006), p. 121–160Google Scholar
  108. 108.
    Y. Shan, R. I. McKay, D. Essam, and J. Liu. Modularity and position independence in EDA-GP. In Proceedings of The Second Asian-Pacific Workshop on Genetic Programming, Cairns, Australia, 2004Google Scholar
  109. 109.
    Y. Shan, R. I. McKay, C. J. Lokan, and D. L. Essam. Software project effort estimation using genetic programming. In Proceedings of the IEEE 2002 International Conference on Communications, Circuits and Systems and West Sino Expositions, volume 2. IEEE, 2002, p. 1108–1112Google Scholar
  110. 110.
    J.L. Shapiro, Drift and scaling in estimation of distribution algorithms. Evol. Comput. 13(1), 99–123 (2005)CrossRefGoogle Scholar
  111. 111.
    J.L. Shapiro. Diversity loss in general estimation of distribution algorithms. In Parallel Problem Solving from Nature, volume 4193 of Lecture Notes in Computer Science, Reykjavik, Iceland, September 2006. Springer, pages 92–101Google Scholar
  112. 112.
    S. Shirakawa, S. Ogino, T. Nagao, Dynamic ant programming for automatic construction of programs. IEEJ Trans. Elect. Elect. Eng. 3(5), 540–548 (2008)CrossRefGoogle Scholar
  113. 113.
    B. Su, Y. Shen. Maximum margin transfer learning. In Proceedings of the first ACM/SIGEVO Summit on Genetic and Evolutionary Computation, GEC ’09, New York, NY, USA, 2009. ACM, pages 957–960Google Scholar
  114. 114.
    I. Tanev, Genetic programming incorporating biased mutation for evolution and adaptation of snakebot. Genetic Programming and Evolvable Machines 8(1), 39–59 (2007)CrossRefGoogle Scholar
  115. 115.
    A. Teller. Evolving programmers: The co-evolution of intelligent recombination operators. In Advances in Genetic Programming 2, chapter 3. (MIT Press, Cambridge, 1996), p. 45–68Google Scholar
  116. 116.
    A. Teller, M. Veloso. PADO: Learning tree structured algorithms for orchestration into an object recognition system. Technical Report CMU-CS-95-101, Department of Computer Science, (Carnegie Mellon University, Pittsburgh, 1995)Google Scholar
  117. 117.
    G.G. Towell, J.W. Shavlik, Extracting refined rules from knowledge-based neural networks. Mach. Learn. 13(1), 71–101 (1993)Google Scholar
  118. 118.
    L.G. Valiant. Learning disjunctions of conjunctions. In Proceedings of the Ninth International Joint Conference on Artificial Intelligence, volume 1, Los Angeles, CA, USA, August 1985. (Morgan Kaufmann Publisher Inc., Burlington), pp. 560–566Google Scholar
  119. 119.
    C.S. Wallace, D.M. Boulton, An information measure for classification. Comput. J. 11(2), 185–194 (1968)CrossRefzbMATHGoogle Scholar
  120. 120.
    P.A. Whigham. Grammatically-based genetic programming. In Proceedings of the Workshop on Genetic Programming: From Theory to Real-World Applications, p. 33–41, Tahoe City, California, USA, 9 July 1995Google Scholar
  121. 121.
    P.A. Whigham. Inductive bias and genetic programming. In Proceedings of First International Conference on Genetic Algorithms in Engineering Systems: Innovations and Applications, Sheffield, UK, 12-14 September 1995. IEEE, p. 461–466Google Scholar
  122. 122.
    P.A. Whigham. A schema theorem for context-free grammars. In Proceedings of the 1995 IEEE Conference on Evolutionary Computation, volume 1, Perth, Austrailia, 29 November - 1 December 1995. (IEEE Press, New York) p. 178–181Google Scholar
  123. 123.
    P.A. Whigham. Grammatical bias for evolutionary learning. PhD thesis, School of Computer Science, University College, University of New South Wales, Australian Defence Force Academy, Canberra, Australia, 1996Google Scholar
  124. 124.
    P.A. Whigham, R.I. McKay, Genetic approaches to learning recursive relations. Prog. Evol. Comput. 956, 17–27 (1995)CrossRefGoogle Scholar
  125. 125.
    G. Wilson, M. Heywood, Introducing probabilistic adaptive mapping developmental genetic programming with redundant mappings. Genet. Program. Evolvable Mach. 8(2), 187–220 (2007)CrossRefGoogle Scholar
  126. 126.
    M.L. Wong and K. S. Leung. Applying logic grammars to induce sub-functions in genetic programming. In 1995 IEEE Conference on Evolutionary Computation, volume 2, Perth, Australia, 29 November - 1 December 1995. IEEE Press, pages 737–740Google Scholar
  127. 127.
    M.L. Wong, K.S. Leung, Genetic logic programming and applications. IEEE Expert 10(5), 68–76 (1995)CrossRefGoogle Scholar
  128. 128.
    M.L. Wong, K.S. Leung. An induction system that learns programs in different programming languages using genetic programming and logic grammars. In Proceedings of the 7th IEEE International Conference on Tools with Artificial Intelligence, Herndon, Virginia, USA, 5-8 November 1995Google Scholar
  129. 129.
    K. Yanai, H. Iba. Estimation of distribution programming based on Bayesian network. In Proceedings of the 2003 IEEE Congress on Evolutionary Computation, CEC2003, Canberra, Australia, 8-12 December 2003. IEEE Press, pages 1618–1625Google Scholar
  130. 130.
    K. Yanai, H. Iba. Probabilistic model-building genetic programming based on dependency relationship. In Proceedings of The Second Asian-Pacific Workshop on Genetic Programming, Cairns, Australia, 6-7 December 2004Google Scholar
  131. 131.
    K. Yanai, H. Iba. Program evolution by integrating EDP and GP. In Proceedings of Genetic and Evolutionary Computation—GECCO-2004, Part I, volume 3102 of Lecture Notes in Computer Science, Seattle, WA, USA, 26-30 June 2004. Springer, pages 774–785Google Scholar
  132. 132.
    K. Yanai, H. Iba. Probabilistic distribution models for EDA-based GP. In GECCO 2005: Proceedings of the 2005 conference on Genetic and evolutionary computation, volume 2, Washington DC, USA, 25-29 June 2005. ACM Press, pages 1775–1776Google Scholar
  133. 133.
    M. Zlochin, M. Birattari, N. Meuleau, M. Dorigo, Model-based search for combinatorial optimization: A critical survey. Ann. Oper. Res. 131(1), 373–395 (2004)CrossRefzbMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Kangil Kim
    • 1
  • Yin Shan
    • 2
  • Xuan Hoai Nguyen
    • 3
  • R. I. McKay
    • 1
    Email author
  1. 1.Structural Complexity Laboratory, Department of Computer Science and EngineeringSeoul National UniversitySeoulKorea
  2. 2.Australian Government Department of Human ServicesCanberraAustralia
  3. 3.Hanoi UniversityHanoiVietNam

Personalised recommendations