Genetic Programming and Evolvable Machines

, Volume 9, Issue 2, pp 125–155 | Cite as

Detecting the epistatic structure of generalized embedded landscape

Original Paper


Working under the premise that most optimizable functions are of bounded epistasis, this paper addresses the problem of discovering the linkage structure of a black-box function with a domain of arbitrary-cardinality under the assumption of bounded epistasis. To model functions of bounded epistasis, we develop a generalization of the mathematical model of “embedded landscapes” over domains of cardinality M. We then generalize the Walsh transform as a discrete Fourier transform, and develop algorithms for linkage learning of epistatically bounded GELs. We propose Generalized Embedding Theorem that models the relationship between the underlying decomposable structure of GEL and its Fourier coefficients. We give a deterministic algorithm to exactly calculate the Fourier coefficients of GEL with bounded epistasis. Complexity analysis shows that the epistatic structure of epistatically bounded GEL can be obtained after a polynomial number of function evaluations. Finally, an example experiment of the algorithm is presented.


Linkage detection Epistasis Genetic algorithm Fourier transform Generalized embedded landscape Problem structure 


  1. 1.
    Thierens, D.: Scalability problems of simple genetic algorithms. Evol. Comput. 7(4), 331–352 (1999)CrossRefGoogle Scholar
  2. 2.
    Goldberg, D.E.: The Design of Innovation: Lessons from and for Competent Genetic Algorithms. Kluwer Academic Publishers, Boston (2002)MATHGoogle Scholar
  3. 3.
    Kargupta, H., Bandyopadhyay, S.: A perspective on the foundation and evolution of the linkage learning genetic algorithms. J. Comp. Methods Appl. Mech. Eng. 186, 266–294 (2000)Google Scholar
  4. 4.
    Goldberg, D.E., Deb, K., Kargupta, H., Harik, G.: Rapid, accurate optimization of difficult optimization problems using fast messy genetic algorithms. In: Proceedings of the Fifth International Conference on Genetic Algorithms. San Mateo, USA (1993)Google Scholar
  5. 5.
    Kargupta, H.: The gene expression messy genetic algorithm. In: Proceedings of the IEEE International Conference on Evolutionary Computation. Nogoya, Japan (1996)Google Scholar
  6. 6.
    Harik, G.R.: Learning gene linkage to efficiently solve problems of bounded difficulty using genetic algorithms. Ph.D. dissertation, University of Michigan, Ann Arbor (1997)Google Scholar
  7. 7.
    Larranaga, P., Lozano, J.A.: Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation. Kluwer Academic Publishers, Boston (2002)MATHGoogle Scholar
  8. 8.
    Pelikan, M.: Hierarchical Bayesian Optimization Algorithm: Toward a New Generation of Evolutionary Algorithms. Springer Publication, Berlin (2005)MATHGoogle Scholar
  9. 9.
    Heckendorn, R.B.: Embedded landscapes. Evol. Comput. 10(4), 345–369 (2002)CrossRefGoogle Scholar
  10. 10.
    Kargupta, H., Park, B.: Gene expression and fast construction of distributed evolutionary representation. Evol. Comput. 9(1), 43–59 (2001)CrossRefGoogle Scholar
  11. 11.
    Heckendorn, R.B., Wright, A.: Efficient linkage discovery by limited probing. Evol. Comput. 12(4), 517–545 (2004)CrossRefGoogle Scholar
  12. 12.
    Rana, S., Heckendorn, R.B., Whitley, D.: A tractable Walsh analysis of sat and its implication of genetic algorithms. In: Proceedings of the Fifteenth National Conference on Artificial Intelligence, pp. 392–397. AAAI, Menlo Park, CA (1998)Google Scholar
  13. 13.
    Wright, A.H., Pulavarty, S.: On the convergence of an estimation of distribution algorithm based on linkage discovery and factorization. In: GECCO 2005 Proceedings, pp. 695–702. ACM SIGEVO, June (2005)Google Scholar
  14. 14.
    Mühlenbein, H.: Convergence of estimation of distribution algorithms for finite samples. (2007)
  15. 15.
    Chen, Y.-P.: Extending the scalability of linkage learning genetic algorithms: theory and practice. IlliGAL Report No. 2004018 (2004)Google Scholar
  16. 16.
    Mühlenbein, H., Mahnig, T., Ochoa, A.R.: Schemata, distributions and graphical models in evolutionary optimization. J. Heuristics 5(2), 215–247 (1999)MATHCrossRefGoogle Scholar
  17. 17.
    Mühlenbein, H., Mahnig, T.: Evolutionary optimization and the estimation of search distribution with application to graph bipartitioning. J. Approx. Reason. 31, 157–192 (2002)MATHCrossRefGoogle Scholar
  18. 18.
    Zhang, Q., Müehlenbein, H.: On the convergence of a class of estimation of distribution algorithms. IEEE Trans. Evol. Comput. 8(2), 127–136 (2004)CrossRefGoogle Scholar
  19. 19.
    Zhang, Q.: On stability of fixed points of limit models of univariate marginal distribution algorithm and factorized distribution algorithm. IEEE Trans. Evol. Comput. 8(1), 80–93 (2004)CrossRefGoogle Scholar
  20. 20.
    Munetomo, M., Goldberg, D.E.: Linkage identification by non-monotonicity detection for overlapping functions. Evol. Comput. 7(4), 377–398 (1999)CrossRefGoogle Scholar
  21. 21.
    Koehler, G.J., Bhattacharyya, S., Vose, M.D.: General cardinality genetic algorithms. Evol. Comput. 5(4), 439–549 (1998)CrossRefGoogle Scholar
  22. 22.
    Vose, M.D., Wright, A.H.: The simple genetic algorithm and the Walsh transform: Part I, theory. Evol. Comput. 6(3), 253–273 (1998)CrossRefGoogle Scholar
  23. 23.
    Vose, M.D., Wright, A.H.: The simple genetic algorithm and the Walsh transform: Part II, the inverse. Evol. Comput. 6, 275–289 (1998)CrossRefGoogle Scholar
  24. 24.
    Simon, H.A.: The Science of the Artificial. MIT Press (1981)Google Scholar
  25. 25.
    Goldberg, D.E., Bridges C.L.: An analysis of a reordering operator on a ga-hard problem. Biol. Cybernet. 62, 397–405 (1990)MATHCrossRefGoogle Scholar
  26. 26.
    Goldberg, D.E., Korb, B., Deb, K.: Messy genetic algorithms: motivation, analysis and first results. Complex Syst. 3, 493–530 (1989)MATHMathSciNetGoogle Scholar
  27. 27.
    Bethke, A.D.: Genetic algorithms as function optimizers. Dissertation University of Michigan, Ann Arbor (1981)Google Scholar
  28. 28.
    Heckendorn, R.B., Whitley, D.: Predicting epistasis from mathematical models. Evol. Comput. 7(1), 69–101 (1999)CrossRefGoogle Scholar
  29. 29.
    Heckendorn, R.B., Rana, S., Whitley, D.: Test function generators as embedded landscapes. In: Foundations of Genetic Algorithms – 5. Morgan Kaufmann Publishers (1999)Google Scholar
  30. 30.
    Mühlenbein, H., Mahnig, T.: Fda – a scalable evolutionary algorithm for the optimization of additively decomposed functions. Evol. Comput. 7(4), 353–376 (1999)CrossRefGoogle Scholar
  31. 31.
    Kargupta, H., Bandyopadhyay, S.: Further experimentations on the scalability of the gemga. In: Lecture Notes in Computer Science: Parallel Problem Solving from Nature, pp. 315–324. Springer-Verlag, Amsterdam (1998)Google Scholar
  32. 32.
    Bandyopadhyay, S., Kargupta, H., Wang, G.: Revisiting the gemga: scalable evolutionary optimization through linkage learning. In: Proceedings of the IEEE International Conference on Evolutionary Computation, pp. 603–608. IEEE Press, Piscataway (1998)Google Scholar
  33. 33.
    Bhattacharyya, S., Koehler, G.: An analysis of non-binary genetic algorithm with cardinality 2v. Complex Syst. 8, 227–256 (1994)MATHMathSciNetGoogle Scholar
  34. 34.
    Krishna Garg, H.: Digital Signal Processing Algorithms: Number Theory, Convolution, Fast Fourier Transforms, and Applications. CRC Press, Boston (1998)MATHGoogle Scholar
  35. 35.
    Rotman, J.J.: Advanced Modern Algebra. Prentice Hall (2002)Google Scholar
  36. 36.
    Oei, C.K.: Walsh function analysis of genetic algorithms of nonbinary strings. Unpublished Masters Thesis, UIUC, (1992)Google Scholar
  37. 37.
    Iglesias, M.T., Naudts, B., Verschoren, A., Vidal C.: Walsh transforms, balanced sum theorems and partition coefficients over multary alphabets. In: GECCO 2005 Proceedings, pp. 1303–1308. ACM SIGEVO, June (2005)Google Scholar
  38. 38.
    Oppenheim, A.V., Schafer, R.W.: Digital Signal Processing. Prentice Hall, Inc, Englewood Cliffs, NJ (1975)MATHGoogle Scholar
  39. 39.
    Kargupta, H., Goldberg, D.E.: SEARCH, blackbox optimization, and sample complexity. In: Belew, R., Vose, M. (eds.) Foundations of Genetic Algorithms, pp. 291–324. Morgan Kaufmann (1996)Google Scholar
  40. 40.
    Kargupta, H.: SEARCH, computational processes in evolution, and preliminary development of the gene expression messy genetic algorithm. Complex Syst. 11(4), 233–287 (1997)MathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  1. 1.Department of Computer Science and TechnologyTsinghua UniversityBeijingChina
  2. 2.Department of Computer ScienceUniversity of IdahoMoscowUSA

Personalised recommendations