# Visualizing Tree Structures in Genetic Programming

Article

- 135 Downloads
- 15 Citations

## Abstract

This paper presents methods to visualize the structure of trees that occur in genetic programming. These methods allow for the inspection of structure of entire trees even though several thousands of nodes may be involved. The methods also scale to allow for the inspection of structure for entire populations and for complete trials even though millions of nodes may be involved. Examples are given that demonstrate how this new way of “seeing” can afford a potentially rich way of understanding dynamics that underpin genetic programming. The examples indicate further studies that might be enabled by visualizing structure at these scales.

## Keywords

Genetic Program Binary Tree Evolutionary Computation Steiner Tree Depth Level
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

## Preview

Unable to display preview. Download preview PDF.

## References

- 1.P. J. Angeline, “Parse trees,” in Handbook of Evolutionary Computation, T. Bäck, D. B. Fogel, and Z. Michalewicz (Eds.), Institute of Physics Publishing: Bristol, 1997, pp. C1.6:1–C1.6:3.Google Scholar
- 2.W. Banzhaf, P. Nordin, R. E. Keller, and F. D. Francone, Genetic Programming: An Introduction: On the Automatic Evolution of Computer Programs and Its Applications, Morgan Kaufmann Publishers: San Francisco, 1998.zbMATHGoogle Scholar
- 3.T. Blickle, “Tournament selection,” in Handbook of Evolutionary Computation, T. Bäck, D. B. Fogel, and Z. Michalewicz (Eds.), Institute of Physics Publishing: Bristol, 1997, pp. C2.3:1–C2.3:4.Google Scholar
- 4.T. Blickle and L. Thiele, “A mathematical analysis of tournament selection,” in ICGA95: Proceedings of the Sixth International Conference on Genetic Algorithms, July 15–19, Pittsburgh, L. J. Eshelman (Ed.), Morgan Kaufmann Publishers: San Francisco, 1995, pp. 9–16.Google Scholar
- 5.O. A. Chaudhri, J. M. Daida, J. C. Khoo, W. S. Richardson, R. B. Harrison, and W. J. Sloat, “Characterizing a tunably difficult problem in genetic programming,” in GECCO 2000: Proceedings of the Genetic and Evolutionary Computation Conference, July 10–12, 2000, Las Vegas, L. D. Whitley, D. E. Goldberg, E. Cantú-Paz, L. Spector, I. Parmee, and H.-G. Beyer (Eds.), Morgan Kaufmann Publishers: San Francisco, 2000, pp. 395–402.Google Scholar
- 6.R. Courant, What is Mathematics? An Elementary Approach to Ideas and Methods. Oxford University Press: London, 1941.zbMATHGoogle Scholar
- 7.J. M. Daida, “Limits to expression in genetic programming: Lattice-aggregate modeling,” in The 2002 IEEE World Congress on Computational Intelligence: Proceedings of the 2002 Congress on Evolutionary Computation, May 12–17, Honolulu, Hawaii. 2002, IEEE: Piscataway, 2002, pp. 273–278.Google Scholar
- 8.J. M. Daida, “What makes a problem GP-Hard? A look at how structure affects content,” in Theory and Applications in Genetic Programming, R. L. Riolo and W. Worzel (Eds.), Kluwer Academic Publishers: Dordrecht, 2003, pp. 99–118.Google Scholar
- 9.J. M. Daida, R. B. Bertram, J. A. Polito 2, and S. A. Stanhope, “Analysis of single-node (building) blocks in genetic programming,” in Advances in Genetic Programming 3, L. Spector, W. B. Langdon, U.-M. O’Reilly, and P. J. Angeline (Eds.), The MIT Press: Cambridge, 1999, pp. 217–241.Google Scholar
- 10.J. M. Daida and A. Hilss, “Identifying structural mechanisms in standard genetic programming,” in Genetic and Evolutionary Computation—GECCO 2003: Genetic and Evolutionary Computation Conference, Chicago, IL, USA, July 2003, E. Cantú-Paz, J. A. Foster, K. Deb, L. D. Davis, R. Roy, U.-M. O’Reilly, H.-G. Beyer, R. Standish, G. Kendall, S. Wilson, M. Harman, J. Wegener, D. Dasgupta, M. A. Potter, A. C. Schultz, K. A. Dowsland, and N. J. J. Miller (Eds.), Springer-Verlag: Berlin, 2003, pp. 1639–1651.Google Scholar
- 11.J. M. Daida, A. Hilss, D. J. Ward, and S. Long, “Visualizing tree structures in genetic programming,” in Genetic and Evolutionary Computation—GECCO 2003: Genetic and Evolutionary Computation Conference, Chicago, IL, USA, July 2003, E. Cantú-Paz, J. A. Foster, K. Deb, L. D. Davis, R. Roy, U.-M. O’Reilly, H.-G. Beyer, R. Standish, G. Kendall, S. Wilson, M. Harman, J. Wegener, D. Dasgupta, M. A. Potter, A. C. Schultz, K. A. Dowsland, and N. J. J. Miller (Eds.), Springer-Verlag: Berlin, 2003, pp. 1652–1664.Google Scholar
- 12.J. M. Daida, A. M. Hilss, D. J. Ward, and S. L. Long, Visualization of Tree Structures: Examples from Genetic Programming and A Model of Tree Growth. The University of Michigan: Ann Arbor, 2004.Google Scholar
- 13.J. M. Daida, H. Li, R. Tang, and A. Hilss, “What makes a problem GP-hard? validating a hypothesis of structural causes,” in Genetic and Evolutionary Computation—GECCO 2003: Genetic and Evolutionary Computation Conference, Chicago, IL, USA, July 2003, E. Cantú-Paz, J. A. Foster, K. Deb, L. D. Davis, R. Roy, U.-M. O’Reilly, H.-G. Beyer, R. Standish, G. Kendall, S. Wilson, M. Harman, J. Wegener, D. Dasgupta, M. A. Potter, A. C. Schultz, K. A. Dowsland, and N. J. J. Miller (Eds.), Springer-Verlag: Berlin, 2003, pp. 1665–1677.Google Scholar
- 14.J. M. Daida, J. A. Polito 2, S. A. Stanhope, R. R. Bertram, J. C. Khoo, S. A. Chaudhary, and O. Chaudhri, “What makes a problem GP-hard? Analysis of a tunably difficult problem in genetic programming,” Genetic Programming and Evolvable Machines, vol. 2, no. 2, pp. 165–191, 2001.zbMATHCrossRefGoogle Scholar
- 15.K. Deb, “Introduction [to Issues in Representations],” in Handbook of Evolutionary Computation, T. Bäck, D. B. Fogel, and Z. Michalewicz (Eds.), Institute of Physics Publishing: Bristol, 1997, pp. C1.1:1–C1.1:4.Google Scholar
- 16.D.-Z. Du, P. M. Pardalos, and W. Wu, Mathematical Theory of Optimization. 273 pp. ed. Nonconvex Optimization and Its Applications, Kluwer Academic Publishers: Dordrecht, 2001.Google Scholar
- 17.D. B. Fogel and P. J. Angeline, “Guidelines for a suitable encoding,” in Handbook of Evolutionary Computation, T. Bäck, D. B. Fogel, and Z. Michalewicz (Eds.), Institute of Physics Publishing: Bristol, 1997, pp. C1.7:1–C1.7:2.Google Scholar
- 18.T. M. J. Fruchterman and E. M. Reingold, “Graph drawing by force-directed placement,” Software, Practice and Experience, vol. 21, pp. 1129–1164, 1991.Google Scholar
- 19.C. Gathercole and P. Ross, “An adverse interaction between crossover and restricted tree depth in genetic programming,” in Genetic Programming 1996: Proceedings of the First Annual Conference: July 28–31, 1996, Stanford University, J. R. Koza, D. E. Goldberg, D. B. Fogel, and R. L. Riolo (Eds.), The MIT Press: Cambridge, 1996, pp. 291–296.Google Scholar
- 20.D. E. Goldberg and U.-M. O’Reilly, “Where does the good stuff go, and why?,” in Proceedings of the First European Conference on Genetic Programming, Paris, France, W. Banzhaf, R. Poli, M. Schoenauer, and T. C. Fogarty (Eds.), Springer-Verlag: Berlin, 1998, pp. 16–36.Google Scholar
- 21.T. Granlund, GNU MP: The GNU Multiple Precision Library, The Free Software Foundation, Inc.: Boston, 2002.Google Scholar
- 22.J. Gray, Mastering Mathematica: Programming Methods and Applications. 2nd editors, Academic Press: San Diego, 1998, p. 629.Google Scholar
- 23.R. L. Harris, Information Graphics: A Comprehensive Illustrated Reference: Visual Tools for Analyzing, Managing, and Communicating, Oxford University Press: New York, 1999.Google Scholar
- 24.I. Herman, G. Melançon, and M. S. Marshall, “Graph visualization and navigation in information visualization: A survey,” IEEE Transactions on Visualization and Computer Graphics, vol. 6, no. 1, pp. 24–43, 2000.CrossRefGoogle Scholar
- 25.G. M. Hunter and K. Steiglitz, “Operations on images using quad trees,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-1, no. 2: pp. 145–153, 1979.CrossRefGoogle Scholar
- 26.F. Hwang, D. Richards, and P. Winter, The Steiner Tree Problem, North-Holland: Amsterdam, Netherlands, 1992.zbMATHGoogle Scholar
- 27.A. O. Ivanov and A. A. Tuzhilin, Minimal Networks: The Steiner Problem and Its Generalizations, CRC Press: Boca Raton, FL, 1994.zbMATHGoogle Scholar
- 28.C. Jacob, Illustrating Evolutionary Computation with Mathematica, Morgan Kaufmann Publishers: San Francisco, 2001, p. 578.Google Scholar
- 29.T. Kamada and S. Kawai, “An algorithm for drawing general undirected graphs,” Information Processing Letters, vol. 31, pp. 7–15, 1989.zbMATHMathSciNetCrossRefGoogle Scholar
- 30.P. R. Keller and M. M. Keller, Visual Cues: Practical Data Visualization, IEEE Press: Piscataway, NJ, 1993.Google Scholar
- 31.G. Kirchoff, Annalen der Physik und Chemie, vol. 72, pp. 497–508, 1847.Google Scholar
- 32.A. Klinger, “Patterns and search statistics,” in Optimizing Methods in Statistics, J. S. Rustagi (Ed.), Academic: New York, 1971, pp. 303–337.Google Scholar
- 33.A. Klinger and C. R. Dyer, “Experiments on picture representation using regular decomposition,” Computer Graphics and Image Processing, vol. 5, pp. 68–105, 1976.Google Scholar
- 34.D. E. Knuth, The Art of Computer Programming: Volume 1: Fundamental Algorithms. 3rd edition, Vol. 1. Reading: Addison–Wesley, 1997.zbMATHGoogle Scholar
- 35.J. R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection. Complex Adaptive Systems, The MIT Press: Cambridge, 1992.Google Scholar
- 36.J. Lamping, R. Rao, and P. Pirolli, “Focus+context based on hyperbolic geometry for visualizaing large hierarchies,” in Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems Part 1 of 2, May 7–11, 1995, Denver, CO., ACM: New York, 1995, pp. 401–408.Google Scholar
- 37.W. B. Langdon, “Quadratic bloat in genetic programming,” in GECCO 2000: Proceedings of the Genetic and Evolutionary Computation Conference, July 10–12, 2000, Las Vegas, L. D. Whitley, D. E. Goldberg, E. Cantú-Paz, L. Spector, I. Parmee, and H.-G. Beyer (Eds.), Morgan Kaufmann Publishers: San Francisco, 2000, pp. 451–458.Google Scholar
- 38.W. B. Langdon, “Size fair and homologous tree crossovers for tree genetic programming,” Genetic Programming and Evolvable Machines, vol. 1, nos. 1/2, pp. 95–119, 2000.zbMATHCrossRefGoogle Scholar
- 39.W. B. Langdon and R. Poli, “Fitness causes bloat,” in Soft Computing in Engineering Design and Manufacturing, P. K. Chawdhry, R. Roy, and R. K. Pant (Eds.), Springer-Verlag: London, 1997, pp. 23–27.Google Scholar
- 40.W. B. Langdon and R. Poli, Foundations of Genetic Programming, Springer-Verlag: Berlin, 2002.zbMATHGoogle Scholar
- 41.W. B. Langdon, T. Soule, R. Poli, and J. A. Foster, “The evolution of size and shape,” in Advances in Genetic Programming 3, L. Spector, W. B. Langdon, U.-M. O’Reilly, and P. J. Angeline (Eds.), The MIT Press: Cambridge, 1999, pp. 163–190.Google Scholar
- 42.L. Margulis and K. V. Schwartz, Five Kingdoms: An Illustrated Guide to the Phyla of Life on Earth. Third edition, W.H. Freeman and Company, New York, 1999.Google Scholar
- 43.A. R. Mast, S. Kelso, A. J. Richards, D. J. Lang, D. M. S. Feller, and E. Conti, “Phylogenetic Relationships in
*Primula L.*and Related Genera (Primulaceae) Based on Noncoding Chloroplast DNA,” International Journal of Plant Science, vol. 162, no. 6, pp. 1381–1400, 2001.CrossRefGoogle Scholar - 44.N. F. McPhee and N. J. Hopper, “Analysis of genetic diversity through population history,” in GECCO ’99: Proceeding of the Genetic and Evolutionary Computation Conference, 13–17 July 1999, Orlando, W. Banzhaf, J. M. Daida, A. E. Eiben, M. H. Garzon, V. Honavar, M. Jakiela, and R. E. Smith (Eds.), Morgan Kaufmann Publishers: San Francisco, 1999, pp. 1112–1120.Google Scholar
- 45.M. Mitchell, S. Forrest, and J. H. Holland, “The royal road for genetic algorithms: fitness landscapes and GA performance,” in Proceedings of the First European Conference on Artificial Life, Toward a Practice of Autonomous Systems, F. J. Varela and P. Bourgine (Eds.), The MIT Press: Cambridge, 1992, pp. 245–254.Google Scholar
- 46.P. Nordin, Evolutionary Program Induction of Binary Machine Code and Its Applications. 1997, der Universitat Dortmund am Fachereich Informatik: Dortmund.Google Scholar
- 47.U.-M. O’Reilly and D. E. Goldberg, “How fitness structure affects subsolution acquisition in genetic programming,” in Genetic Programming 1998: Proceedings of the Third Annual Conference, July 22–25, 1998, University of Wisconsin, Madison, J. R. Koza, W. Banzhaf, K. Chellapilla, K. Deb, M. Dorigo, D. B. Fogel, M. H. Garzon, D. E. Goldberg, H. Iba, and R. L. Riolo (Eds.), Morgan Kaufmann Publishers: San Francisco, 1998, pp. 269–277.Google Scholar
- 48.R. Poli, “General schema theory for genetic programming with subtree-swapping crossover,” in Genetic Programming: Proceedings of EuroGP 2001, April 18–20, 2001, Milan, J. F. Miller, M. Tomassini, P. L. Lanzi, C. Ryan, G. B. Tettamanzi, and W. B. Langdon (Eds.), Springer-Verlag: Berlin, 2001, pp. 143–159.Google Scholar
- 49.R. Poli and W. B. Langdon, “A new schema theory for genetic programming with one-point crossover and point mutation,” in Genetic Programming 1997: Proceedings of the Second Annual Conference, July 13-16, 1997, Stanford University, J. R. Koza, K. Deb, M. Dorigo, D. B. Fogel, M. Garzon, H. Iba, and R. L. Riolo (Eds.), Morgan Kaufmann Publishers: San Francisco, 1997, pp. 279–85.Google Scholar
- 50.R. Poli and W. B. Langdon, “Schema theory for genetic programming with one-point crossover and point mutation,” Evolutionary Computation, vol. 6, no. 3, pp. 231–252, 1998.Google Scholar
- 51.W. Punch, D. Zongker and E. Goodman, “The royal tree problem, a benchmark for single and multiple population genetic programming,” in Advances in Genetic Programming, P. J. Angeline and J. K.E. Kinnear (Eds.), The MIT Press: Cambridge, 1996, pp. 299–316.Google Scholar
- 52.J. P. Rosca, “Analysis of complexity drift in genetic programming,” in Genetic Programming 1997: Proceedings of the Second Annual Conference, July 13-16, 1997, Stanford University, J. R. Koza, K. Deb, M. Dorigo, D. B. Fogel, M. Garzon, H. Iba, and R. L. Riolo (Eds.), Morgan Kaufmann Publishers: San Francisco, 1997, pp. 286–94.Google Scholar
- 53.J. P. Rosca and D. H. Ballard, “Rooted-tree schemata in genetic programming,” in Advances in Genetic Programming 3, L. Spector, W. B. Langdon, U.-M. O’Reilly, and P. J. Angeline (Eds.), The MIT Press: Cambridge, 1999, pp. 243–271.Google Scholar
- 54.H. Samet, “Neighbor finding techniques for images represented by quadtrees,” Computer Graphics and Image Processing, vol. 18, pp. 37–57, 1980.CrossRefGoogle Scholar
- 55.H. Samet and M. Tamminen, “Computing geometric properties of images represented by linear quadtrees,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-7, no. 2, pp. 229–240, 1985.CrossRefGoogle Scholar
- 56.P. Schreiber, “On the history of the so-called Steiner Weber problem.” Wiss. Z. Erust-Monitz-Arndt-Univ. Greifswald, Math.-nat.wiss. Reihe, vol. 35, no. 3, 1986.Google Scholar
- 57.T. Soule and J. A. Foster, “Code size and depth flows in genetic programming,” in Genetic Programming 1997: Proceedings of the Second Annual Conference, July 13-16, 1997, Stanford University, J. R. Koza, K. Deb, M. Dorigo, D. B. Fogel, M. Garzon, H. Iba, and R. L. Riolo (Eds.), Morgan Kaufmann Publishers: San Francisco, 1997, pp. 313–320.Google Scholar
- 58.T. Soule and J. A. Foster, “Removal bias: A new cause of code growth in tree based evolutionary programming,” in The 1998 IEEE International Conference on Evolutionary Computation Proceedings: IEEE World Congress on Computational Intelligence. IEEE Press: Piscataway, 1998, pp. 781–786.Google Scholar
- 59.T. Soule, J. A. Foster, and J. Dickinson, “Code growth in genetic programming,” in Genetic Programming 1996: Proceedings of the First Annual Conference: July 28–31, 1996, Stanford University, J. R. Koza, D. E. Goldberg, D. B. Fogel, and R. L. Riolo (Eds.), The MIT Press: Cambridge, 1996, pp. 215–223.Google Scholar
- 60.R. P. Stanley, Enumerative Combinatorics I, Cambridge Studies in Advanced Mathematics, ed. W. Fulton, D. J. H. Garling, K. Ribet and P. Walters. Vol. 1. Cambridge University Press: Cambridge: 1997.Google Scholar
- 61.R. P. Stanley, Enumerative Combinatorics II. Cambridge Studies in Advanced Mathematics, W. Fulton, D. J. H. Garling, K. Ribet, and P. Walters (Eds.), vol. 2, Cambridge University Press: Cambridge, 1999.Google Scholar
- 62.S. H. Strogatz, “Exploring complex networks,” Nature, vol. 410, pp. 268–276, 2001.CrossRefGoogle Scholar
- 63.E. R. Tufte, Envisioning Information, Graphics Press: Cheshire, CT, 1990.Google Scholar
- 64.E. R. Tufte, The Visual Display of Quantitative Information. Graphics Press: Cheshire, CT, 1983.Google Scholar
- 65.E. R. Tufte, Visual Explanations: Images and Quantities, Evidence and Narrative. Graphics Press: Cheshire, CT, 1997.zbMATHGoogle Scholar
- 66.T. Wickham-Jones, Mathematica Graphics: Techniques and Applications, TELOS: New York, 1994.zbMATHGoogle Scholar
- 67.S. Wolfram, A New Kind of Science, Wolfram Media, Inc: Champaign, IL, 2002.zbMATHGoogle Scholar
- 68.D. H. Wolpert and W. G. Macready, “No free lunch theorems for optimization,” IEEE Transactions on Evolutionary Computation, vol. 1, no. 1, pp. 67–82, 1997.CrossRefGoogle Scholar

## Copyright information

© Springer Science + Business Media, Inc. 2005