Advertisement

Genetic Programming and Evolvable Machines

, Volume 6, Issue 1, pp 79–110 | Cite as

Visualizing Tree Structures in Genetic Programming

  • Jason M. Daida
  • Adam M. Hilss
  • David J. Ward
  • Stephen L. Long
Article

Abstract

This paper presents methods to visualize the structure of trees that occur in genetic programming. These methods allow for the inspection of structure of entire trees even though several thousands of nodes may be involved. The methods also scale to allow for the inspection of structure for entire populations and for complete trials even though millions of nodes may be involved. Examples are given that demonstrate how this new way of “seeing” can afford a potentially rich way of understanding dynamics that underpin genetic programming. The examples indicate further studies that might be enabled by visualizing structure at these scales.

Keywords

Genetic Program Binary Tree Evolutionary Computation Steiner Tree Depth Level 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    P. J. Angeline, “Parse trees,” in Handbook of Evolutionary Computation, T. Bäck, D. B. Fogel, and Z. Michalewicz (Eds.), Institute of Physics Publishing: Bristol, 1997, pp. C1.6:1–C1.6:3.Google Scholar
  2. 2.
    W. Banzhaf, P. Nordin, R. E. Keller, and F. D. Francone, Genetic Programming: An Introduction: On the Automatic Evolution of Computer Programs and Its Applications, Morgan Kaufmann Publishers: San Francisco, 1998.zbMATHGoogle Scholar
  3. 3.
    T. Blickle, “Tournament selection,” in Handbook of Evolutionary Computation, T. Bäck, D. B. Fogel, and Z. Michalewicz (Eds.), Institute of Physics Publishing: Bristol, 1997, pp. C2.3:1–C2.3:4.Google Scholar
  4. 4.
    T. Blickle and L. Thiele, “A mathematical analysis of tournament selection,” in ICGA95: Proceedings of the Sixth International Conference on Genetic Algorithms, July 15–19, Pittsburgh, L. J. Eshelman (Ed.), Morgan Kaufmann Publishers: San Francisco, 1995, pp. 9–16.Google Scholar
  5. 5.
    O. A. Chaudhri, J. M. Daida, J. C. Khoo, W. S. Richardson, R. B. Harrison, and W. J. Sloat, “Characterizing a tunably difficult problem in genetic programming,” in GECCO 2000: Proceedings of the Genetic and Evolutionary Computation Conference, July 10–12, 2000, Las Vegas, L. D. Whitley, D. E. Goldberg, E. Cantú-Paz, L. Spector, I. Parmee, and H.-G. Beyer (Eds.), Morgan Kaufmann Publishers: San Francisco, 2000, pp. 395–402.Google Scholar
  6. 6.
    R. Courant, What is Mathematics? An Elementary Approach to Ideas and Methods. Oxford University Press: London, 1941.zbMATHGoogle Scholar
  7. 7.
    J. M. Daida, “Limits to expression in genetic programming: Lattice-aggregate modeling,” in The 2002 IEEE World Congress on Computational Intelligence: Proceedings of the 2002 Congress on Evolutionary Computation, May 12–17, Honolulu, Hawaii. 2002, IEEE: Piscataway, 2002, pp. 273–278.Google Scholar
  8. 8.
    J. M. Daida, “What makes a problem GP-Hard? A look at how structure affects content,” in Theory and Applications in Genetic Programming, R. L. Riolo and W. Worzel (Eds.), Kluwer Academic Publishers: Dordrecht, 2003, pp. 99–118.Google Scholar
  9. 9.
    J. M. Daida, R. B. Bertram, J. A. Polito 2, and S. A. Stanhope, “Analysis of single-node (building) blocks in genetic programming,” in Advances in Genetic Programming 3, L. Spector, W. B. Langdon, U.-M. O’Reilly, and P. J. Angeline (Eds.), The MIT Press: Cambridge, 1999, pp. 217–241.Google Scholar
  10. 10.
    J. M. Daida and A. Hilss, “Identifying structural mechanisms in standard genetic programming,” in Genetic and Evolutionary Computation—GECCO 2003: Genetic and Evolutionary Computation Conference, Chicago, IL, USA, July 2003, E. Cantú-Paz, J. A. Foster, K. Deb, L. D. Davis, R. Roy, U.-M. O’Reilly, H.-G. Beyer, R. Standish, G. Kendall, S. Wilson, M. Harman, J. Wegener, D. Dasgupta, M. A. Potter, A. C. Schultz, K. A. Dowsland, and N. J. J. Miller (Eds.), Springer-Verlag: Berlin, 2003, pp. 1639–1651.Google Scholar
  11. 11.
    J. M. Daida, A. Hilss, D. J. Ward, and S. Long, “Visualizing tree structures in genetic programming,” in Genetic and Evolutionary Computation—GECCO 2003: Genetic and Evolutionary Computation Conference, Chicago, IL, USA, July 2003, E. Cantú-Paz, J. A. Foster, K. Deb, L. D. Davis, R. Roy, U.-M. O’Reilly, H.-G. Beyer, R. Standish, G. Kendall, S. Wilson, M. Harman, J. Wegener, D. Dasgupta, M. A. Potter, A. C. Schultz, K. A. Dowsland, and N. J. J. Miller (Eds.), Springer-Verlag: Berlin, 2003, pp. 1652–1664.Google Scholar
  12. 12.
    J. M. Daida, A. M. Hilss, D. J. Ward, and S. L. Long, Visualization of Tree Structures: Examples from Genetic Programming and A Model of Tree Growth. The University of Michigan: Ann Arbor, 2004.Google Scholar
  13. 13.
    J. M. Daida, H. Li, R. Tang, and A. Hilss, “What makes a problem GP-hard? validating a hypothesis of structural causes,” in Genetic and Evolutionary Computation—GECCO 2003: Genetic and Evolutionary Computation Conference, Chicago, IL, USA, July 2003, E. Cantú-Paz, J. A. Foster, K. Deb, L. D. Davis, R. Roy, U.-M. O’Reilly, H.-G. Beyer, R. Standish, G. Kendall, S. Wilson, M. Harman, J. Wegener, D. Dasgupta, M. A. Potter, A. C. Schultz, K. A. Dowsland, and N. J. J. Miller (Eds.), Springer-Verlag: Berlin, 2003, pp. 1665–1677.Google Scholar
  14. 14.
    J. M. Daida, J. A. Polito 2, S. A. Stanhope, R. R. Bertram, J. C. Khoo, S. A. Chaudhary, and O. Chaudhri, “What makes a problem GP-hard? Analysis of a tunably difficult problem in genetic programming,” Genetic Programming and Evolvable Machines, vol. 2, no. 2, pp. 165–191, 2001.zbMATHCrossRefGoogle Scholar
  15. 15.
    K. Deb, “Introduction [to Issues in Representations],” in Handbook of Evolutionary Computation, T. Bäck, D. B. Fogel, and Z. Michalewicz (Eds.), Institute of Physics Publishing: Bristol, 1997, pp. C1.1:1–C1.1:4.Google Scholar
  16. 16.
    D.-Z. Du, P. M. Pardalos, and W. Wu, Mathematical Theory of Optimization. 273 pp. ed. Nonconvex Optimization and Its Applications, Kluwer Academic Publishers: Dordrecht, 2001.Google Scholar
  17. 17.
    D. B. Fogel and P. J. Angeline, “Guidelines for a suitable encoding,” in Handbook of Evolutionary Computation, T. Bäck, D. B. Fogel, and Z. Michalewicz (Eds.), Institute of Physics Publishing: Bristol, 1997, pp. C1.7:1–C1.7:2.Google Scholar
  18. 18.
    T. M. J. Fruchterman and E. M. Reingold, “Graph drawing by force-directed placement,” Software, Practice and Experience, vol. 21, pp. 1129–1164, 1991.Google Scholar
  19. 19.
    C. Gathercole and P. Ross, “An adverse interaction between crossover and restricted tree depth in genetic programming,” in Genetic Programming 1996: Proceedings of the First Annual Conference: July 28–31, 1996, Stanford University, J. R. Koza, D. E. Goldberg, D. B. Fogel, and R. L. Riolo (Eds.), The MIT Press: Cambridge, 1996, pp. 291–296.Google Scholar
  20. 20.
    D. E. Goldberg and U.-M. O’Reilly, “Where does the good stuff go, and why?,” in Proceedings of the First European Conference on Genetic Programming, Paris, France, W. Banzhaf, R. Poli, M. Schoenauer, and T. C. Fogarty (Eds.), Springer-Verlag: Berlin, 1998, pp. 16–36.Google Scholar
  21. 21.
    T. Granlund, GNU MP: The GNU Multiple Precision Library, The Free Software Foundation, Inc.: Boston, 2002.Google Scholar
  22. 22.
    J. Gray, Mastering Mathematica: Programming Methods and Applications. 2nd editors, Academic Press: San Diego, 1998, p. 629.Google Scholar
  23. 23.
    R. L. Harris, Information Graphics: A Comprehensive Illustrated Reference: Visual Tools for Analyzing, Managing, and Communicating, Oxford University Press: New York, 1999.Google Scholar
  24. 24.
    I. Herman, G. Melançon, and M. S. Marshall, “Graph visualization and navigation in information visualization: A survey,” IEEE Transactions on Visualization and Computer Graphics, vol. 6, no. 1, pp. 24–43, 2000.CrossRefGoogle Scholar
  25. 25.
    G. M. Hunter and K. Steiglitz, “Operations on images using quad trees,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-1, no. 2: pp. 145–153, 1979.CrossRefGoogle Scholar
  26. 26.
    F. Hwang, D. Richards, and P. Winter, The Steiner Tree Problem, North-Holland: Amsterdam, Netherlands, 1992.zbMATHGoogle Scholar
  27. 27.
    A. O. Ivanov and A. A. Tuzhilin, Minimal Networks: The Steiner Problem and Its Generalizations, CRC Press: Boca Raton, FL, 1994.zbMATHGoogle Scholar
  28. 28.
    C. Jacob, Illustrating Evolutionary Computation with Mathematica, Morgan Kaufmann Publishers: San Francisco, 2001, p. 578.Google Scholar
  29. 29.
    T. Kamada and S. Kawai, “An algorithm for drawing general undirected graphs,” Information Processing Letters, vol. 31, pp. 7–15, 1989.zbMATHMathSciNetCrossRefGoogle Scholar
  30. 30.
    P. R. Keller and M. M. Keller, Visual Cues: Practical Data Visualization, IEEE Press: Piscataway, NJ, 1993.Google Scholar
  31. 31.
    G. Kirchoff, Annalen der Physik und Chemie, vol. 72, pp. 497–508, 1847.Google Scholar
  32. 32.
    A. Klinger, “Patterns and search statistics,” in Optimizing Methods in Statistics, J. S. Rustagi (Ed.), Academic: New York, 1971, pp. 303–337.Google Scholar
  33. 33.
    A. Klinger and C. R. Dyer, “Experiments on picture representation using regular decomposition,” Computer Graphics and Image Processing, vol. 5, pp. 68–105, 1976.Google Scholar
  34. 34.
    D. E. Knuth, The Art of Computer Programming: Volume 1: Fundamental Algorithms. 3rd edition, Vol. 1. Reading: Addison–Wesley, 1997.zbMATHGoogle Scholar
  35. 35.
    J. R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection. Complex Adaptive Systems, The MIT Press: Cambridge, 1992.Google Scholar
  36. 36.
    J. Lamping, R. Rao, and P. Pirolli, “Focus+context based on hyperbolic geometry for visualizaing large hierarchies,” in Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems Part 1 of 2, May 7–11, 1995, Denver, CO., ACM: New York, 1995, pp. 401–408.Google Scholar
  37. 37.
    W. B. Langdon, “Quadratic bloat in genetic programming,” in GECCO 2000: Proceedings of the Genetic and Evolutionary Computation Conference, July 10–12, 2000, Las Vegas, L. D. Whitley, D. E. Goldberg, E. Cantú-Paz, L. Spector, I. Parmee, and H.-G. Beyer (Eds.), Morgan Kaufmann Publishers: San Francisco, 2000, pp. 451–458.Google Scholar
  38. 38.
    W. B. Langdon, “Size fair and homologous tree crossovers for tree genetic programming,” Genetic Programming and Evolvable Machines, vol. 1, nos. 1/2, pp. 95–119, 2000.zbMATHCrossRefGoogle Scholar
  39. 39.
    W. B. Langdon and R. Poli, “Fitness causes bloat,” in Soft Computing in Engineering Design and Manufacturing, P. K. Chawdhry, R. Roy, and R. K. Pant (Eds.), Springer-Verlag: London, 1997, pp. 23–27.Google Scholar
  40. 40.
    W. B. Langdon and R. Poli, Foundations of Genetic Programming, Springer-Verlag: Berlin, 2002.zbMATHGoogle Scholar
  41. 41.
    W. B. Langdon, T. Soule, R. Poli, and J. A. Foster, “The evolution of size and shape,” in Advances in Genetic Programming 3, L. Spector, W. B. Langdon, U.-M. O’Reilly, and P. J. Angeline (Eds.), The MIT Press: Cambridge, 1999, pp. 163–190.Google Scholar
  42. 42.
    L. Margulis and K. V. Schwartz, Five Kingdoms: An Illustrated Guide to the Phyla of Life on Earth. Third edition, W.H. Freeman and Company, New York, 1999.Google Scholar
  43. 43.
    A. R. Mast, S. Kelso, A. J. Richards, D. J. Lang, D. M. S. Feller, and E. Conti, “Phylogenetic Relationships in Primula L. and Related Genera (Primulaceae) Based on Noncoding Chloroplast DNA,” International Journal of Plant Science, vol. 162, no. 6, pp. 1381–1400, 2001.CrossRefGoogle Scholar
  44. 44.
    N. F. McPhee and N. J. Hopper, “Analysis of genetic diversity through population history,” in GECCO ’99: Proceeding of the Genetic and Evolutionary Computation Conference, 13–17 July 1999, Orlando, W. Banzhaf, J. M. Daida, A. E. Eiben, M. H. Garzon, V. Honavar, M. Jakiela, and R. E. Smith (Eds.), Morgan Kaufmann Publishers: San Francisco, 1999, pp. 1112–1120.Google Scholar
  45. 45.
    M. Mitchell, S. Forrest, and J. H. Holland, “The royal road for genetic algorithms: fitness landscapes and GA performance,” in Proceedings of the First European Conference on Artificial Life, Toward a Practice of Autonomous Systems, F. J. Varela and P. Bourgine (Eds.), The MIT Press: Cambridge, 1992, pp. 245–254.Google Scholar
  46. 46.
    P. Nordin, Evolutionary Program Induction of Binary Machine Code and Its Applications. 1997, der Universitat Dortmund am Fachereich Informatik: Dortmund.Google Scholar
  47. 47.
    U.-M. O’Reilly and D. E. Goldberg, “How fitness structure affects subsolution acquisition in genetic programming,” in Genetic Programming 1998: Proceedings of the Third Annual Conference, July 22–25, 1998, University of Wisconsin, Madison, J. R. Koza, W. Banzhaf, K. Chellapilla, K. Deb, M. Dorigo, D. B. Fogel, M. H. Garzon, D. E. Goldberg, H. Iba, and R. L. Riolo (Eds.), Morgan Kaufmann Publishers: San Francisco, 1998, pp. 269–277.Google Scholar
  48. 48.
    R. Poli, “General schema theory for genetic programming with subtree-swapping crossover,” in Genetic Programming: Proceedings of EuroGP 2001, April 18–20, 2001, Milan, J. F. Miller, M. Tomassini, P. L. Lanzi, C. Ryan, G. B. Tettamanzi, and W. B. Langdon (Eds.), Springer-Verlag: Berlin, 2001, pp. 143–159.Google Scholar
  49. 49.
    R. Poli and W. B. Langdon, “A new schema theory for genetic programming with one-point crossover and point mutation,” in Genetic Programming 1997: Proceedings of the Second Annual Conference, July 13-16, 1997, Stanford University, J. R. Koza, K. Deb, M. Dorigo, D. B. Fogel, M. Garzon, H. Iba, and R. L. Riolo (Eds.), Morgan Kaufmann Publishers: San Francisco, 1997, pp. 279–85.Google Scholar
  50. 50.
    R. Poli and W. B. Langdon, “Schema theory for genetic programming with one-point crossover and point mutation,” Evolutionary Computation, vol. 6, no. 3, pp. 231–252, 1998.Google Scholar
  51. 51.
    W. Punch, D. Zongker and E. Goodman, “The royal tree problem, a benchmark for single and multiple population genetic programming,” in Advances in Genetic Programming, P. J. Angeline and J. K.E. Kinnear (Eds.), The MIT Press: Cambridge, 1996, pp. 299–316.Google Scholar
  52. 52.
    J. P. Rosca, “Analysis of complexity drift in genetic programming,” in Genetic Programming 1997: Proceedings of the Second Annual Conference, July 13-16, 1997, Stanford University, J. R. Koza, K. Deb, M. Dorigo, D. B. Fogel, M. Garzon, H. Iba, and R. L. Riolo (Eds.), Morgan Kaufmann Publishers: San Francisco, 1997, pp. 286–94.Google Scholar
  53. 53.
    J. P. Rosca and D. H. Ballard, “Rooted-tree schemata in genetic programming,” in Advances in Genetic Programming 3, L. Spector, W. B. Langdon, U.-M. O’Reilly, and P. J. Angeline (Eds.), The MIT Press: Cambridge, 1999, pp. 243–271.Google Scholar
  54. 54.
    H. Samet, “Neighbor finding techniques for images represented by quadtrees,” Computer Graphics and Image Processing, vol. 18, pp. 37–57, 1980.CrossRefGoogle Scholar
  55. 55.
    H. Samet and M. Tamminen, “Computing geometric properties of images represented by linear quadtrees,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-7, no. 2, pp. 229–240, 1985.CrossRefGoogle Scholar
  56. 56.
    P. Schreiber, “On the history of the so-called Steiner Weber problem.” Wiss. Z. Erust-Monitz-Arndt-Univ. Greifswald, Math.-nat.wiss. Reihe, vol. 35, no. 3, 1986.Google Scholar
  57. 57.
    T. Soule and J. A. Foster, “Code size and depth flows in genetic programming,” in Genetic Programming 1997: Proceedings of the Second Annual Conference, July 13-16, 1997, Stanford University, J. R. Koza, K. Deb, M. Dorigo, D. B. Fogel, M. Garzon, H. Iba, and R. L. Riolo (Eds.), Morgan Kaufmann Publishers: San Francisco, 1997, pp. 313–320.Google Scholar
  58. 58.
    T. Soule and J. A. Foster, “Removal bias: A new cause of code growth in tree based evolutionary programming,” in The 1998 IEEE International Conference on Evolutionary Computation Proceedings: IEEE World Congress on Computational Intelligence. IEEE Press: Piscataway, 1998, pp. 781–786.Google Scholar
  59. 59.
    T. Soule, J. A. Foster, and J. Dickinson, “Code growth in genetic programming,” in Genetic Programming 1996: Proceedings of the First Annual Conference: July 28–31, 1996, Stanford University, J. R. Koza, D. E. Goldberg, D. B. Fogel, and R. L. Riolo (Eds.), The MIT Press: Cambridge, 1996, pp. 215–223.Google Scholar
  60. 60.
    R. P. Stanley, Enumerative Combinatorics I, Cambridge Studies in Advanced Mathematics, ed. W. Fulton, D. J. H. Garling, K. Ribet and P. Walters. Vol. 1. Cambridge University Press: Cambridge: 1997.Google Scholar
  61. 61.
    R. P. Stanley, Enumerative Combinatorics II. Cambridge Studies in Advanced Mathematics, W. Fulton, D. J. H. Garling, K. Ribet, and P. Walters (Eds.), vol. 2, Cambridge University Press: Cambridge, 1999.Google Scholar
  62. 62.
    S. H. Strogatz, “Exploring complex networks,” Nature, vol. 410, pp. 268–276, 2001.CrossRefGoogle Scholar
  63. 63.
    E. R. Tufte, Envisioning Information, Graphics Press: Cheshire, CT, 1990.Google Scholar
  64. 64.
    E. R. Tufte, The Visual Display of Quantitative Information. Graphics Press: Cheshire, CT, 1983.Google Scholar
  65. 65.
    E. R. Tufte, Visual Explanations: Images and Quantities, Evidence and Narrative. Graphics Press: Cheshire, CT, 1997.zbMATHGoogle Scholar
  66. 66.
    T. Wickham-Jones, Mathematica Graphics: Techniques and Applications, TELOS: New York, 1994.zbMATHGoogle Scholar
  67. 67.
    S. Wolfram, A New Kind of Science, Wolfram Media, Inc: Champaign, IL, 2002.zbMATHGoogle Scholar
  68. 68.
    D. H. Wolpert and W. G. Macready, “No free lunch theorems for optimization,” IEEE Transactions on Evolutionary Computation, vol. 1, no. 1, pp. 67–82, 1997.CrossRefGoogle Scholar

Copyright information

© Springer Science + Business Media, Inc. 2005

Authors and Affiliations

  • Jason M. Daida
    • 1
  • Adam M. Hilss
    • 1
  • David J. Ward
    • 1
  • Stephen L. Long
    • 1
  1. 1.Center for the Study of Complex Systems and Department of Atmospheric, Oceanic and Space SciencesThe University of MichiganAnn ArborUSA

Personalised recommendations