## Abstract

This paper presents methods to visualize the structure of trees that occur in genetic programming. These methods allow for the inspection of structure of entire trees even though several thousands of nodes may be involved. The methods also scale to allow for the inspection of structure for entire populations and for complete trials even though millions of nodes may be involved. Examples are given that demonstrate how this new way of “seeing” can afford a potentially rich way of understanding dynamics that underpin genetic programming. The examples indicate further studies that might be enabled by visualizing structure at these scales.

### Similar content being viewed by others

### Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.## References

P. J. Angeline, “Parse trees,” in Handbook of Evolutionary Computation, T. Bäck, D. B. Fogel, and Z. Michalewicz (Eds.), Institute of Physics Publishing: Bristol, 1997, pp. C1.6:1–C1.6:3.

W. Banzhaf, P. Nordin, R. E. Keller, and F. D. Francone, Genetic Programming: An Introduction: On the Automatic Evolution of Computer Programs and Its Applications, Morgan Kaufmann Publishers: San Francisco, 1998.

T. Blickle, “Tournament selection,” in Handbook of Evolutionary Computation, T. Bäck, D. B. Fogel, and Z. Michalewicz (Eds.), Institute of Physics Publishing: Bristol, 1997, pp. C2.3:1–C2.3:4.

T. Blickle and L. Thiele, “A mathematical analysis of tournament selection,” in ICGA95: Proceedings of the Sixth International Conference on Genetic Algorithms, July 15–19, Pittsburgh, L. J. Eshelman (Ed.), Morgan Kaufmann Publishers: San Francisco, 1995, pp. 9–16.

O. A. Chaudhri, J. M. Daida, J. C. Khoo, W. S. Richardson, R. B. Harrison, and W. J. Sloat, “Characterizing a tunably difficult problem in genetic programming,” in GECCO 2000: Proceedings of the Genetic and Evolutionary Computation Conference, July 10–12, 2000, Las Vegas, L. D. Whitley, D. E. Goldberg, E. Cantú-Paz, L. Spector, I. Parmee, and H.-G. Beyer (Eds.), Morgan Kaufmann Publishers: San Francisco, 2000, pp. 395–402.

R. Courant, What is Mathematics? An Elementary Approach to Ideas and Methods. Oxford University Press: London, 1941.

J. M. Daida, “Limits to expression in genetic programming: Lattice-aggregate modeling,” in The 2002 IEEE World Congress on Computational Intelligence: Proceedings of the 2002 Congress on Evolutionary Computation, May 12–17, Honolulu, Hawaii. 2002, IEEE: Piscataway, 2002, pp. 273–278.

J. M. Daida, “What makes a problem GP-Hard? A look at how structure affects content,” in Theory and Applications in Genetic Programming, R. L. Riolo and W. Worzel (Eds.), Kluwer Academic Publishers: Dordrecht, 2003, pp. 99–118.

J. M. Daida, R. B. Bertram, J. A. Polito 2, and S. A. Stanhope, “Analysis of single-node (building) blocks in genetic programming,” in Advances in Genetic Programming 3, L. Spector, W. B. Langdon, U.-M. O’Reilly, and P. J. Angeline (Eds.), The MIT Press: Cambridge, 1999, pp. 217–241.

J. M. Daida and A. Hilss, “Identifying structural mechanisms in standard genetic programming,” in Genetic and Evolutionary Computation—GECCO 2003: Genetic and Evolutionary Computation Conference, Chicago, IL, USA, July 2003, E. Cantú-Paz, J. A. Foster, K. Deb, L. D. Davis, R. Roy, U.-M. O’Reilly, H.-G. Beyer, R. Standish, G. Kendall, S. Wilson, M. Harman, J. Wegener, D. Dasgupta, M. A. Potter, A. C. Schultz, K. A. Dowsland, and N. J. J. Miller (Eds.), Springer-Verlag: Berlin, 2003, pp. 1639–1651.

J. M. Daida, A. Hilss, D. J. Ward, and S. Long, “Visualizing tree structures in genetic programming,” in Genetic and Evolutionary Computation—GECCO 2003: Genetic and Evolutionary Computation Conference, Chicago, IL, USA, July 2003, E. Cantú-Paz, J. A. Foster, K. Deb, L. D. Davis, R. Roy, U.-M. O’Reilly, H.-G. Beyer, R. Standish, G. Kendall, S. Wilson, M. Harman, J. Wegener, D. Dasgupta, M. A. Potter, A. C. Schultz, K. A. Dowsland, and N. J. J. Miller (Eds.), Springer-Verlag: Berlin, 2003, pp. 1652–1664.

J. M. Daida, A. M. Hilss, D. J. Ward, and S. L. Long, Visualization of Tree Structures: Examples from Genetic Programming and A Model of Tree Growth. The University of Michigan: Ann Arbor, 2004.

J. M. Daida, H. Li, R. Tang, and A. Hilss, “What makes a problem GP-hard? validating a hypothesis of structural causes,” in Genetic and Evolutionary Computation—GECCO 2003: Genetic and Evolutionary Computation Conference, Chicago, IL, USA, July 2003, E. Cantú-Paz, J. A. Foster, K. Deb, L. D. Davis, R. Roy, U.-M. O’Reilly, H.-G. Beyer, R. Standish, G. Kendall, S. Wilson, M. Harman, J. Wegener, D. Dasgupta, M. A. Potter, A. C. Schultz, K. A. Dowsland, and N. J. J. Miller (Eds.), Springer-Verlag: Berlin, 2003, pp. 1665–1677.

J. M. Daida, J. A. Polito 2, S. A. Stanhope, R. R. Bertram, J. C. Khoo, S. A. Chaudhary, and O. Chaudhri, “What makes a problem GP-hard? Analysis of a tunably difficult problem in genetic programming,” Genetic Programming and Evolvable Machines, vol. 2, no. 2, pp. 165–191, 2001.

K. Deb, “Introduction [to Issues in Representations],” in Handbook of Evolutionary Computation, T. Bäck, D. B. Fogel, and Z. Michalewicz (Eds.), Institute of Physics Publishing: Bristol, 1997, pp. C1.1:1–C1.1:4.

D.-Z. Du, P. M. Pardalos, and W. Wu, Mathematical Theory of Optimization. 273 pp. ed. Nonconvex Optimization and Its Applications, Kluwer Academic Publishers: Dordrecht, 2001.

D. B. Fogel and P. J. Angeline, “Guidelines for a suitable encoding,” in Handbook of Evolutionary Computation, T. Bäck, D. B. Fogel, and Z. Michalewicz (Eds.), Institute of Physics Publishing: Bristol, 1997, pp. C1.7:1–C1.7:2.

T. M. J. Fruchterman and E. M. Reingold, “Graph drawing by force-directed placement,” Software, Practice and Experience, vol. 21, pp. 1129–1164, 1991.

C. Gathercole and P. Ross, “An adverse interaction between crossover and restricted tree depth in genetic programming,” in Genetic Programming 1996: Proceedings of the First Annual Conference: July 28–31, 1996, Stanford University, J. R. Koza, D. E. Goldberg, D. B. Fogel, and R. L. Riolo (Eds.), The MIT Press: Cambridge, 1996, pp. 291–296.

D. E. Goldberg and U.-M. O’Reilly, “Where does the good stuff go, and why?,” in Proceedings of the First European Conference on Genetic Programming, Paris, France, W. Banzhaf, R. Poli, M. Schoenauer, and T. C. Fogarty (Eds.), Springer-Verlag: Berlin, 1998, pp. 16–36.

T. Granlund, GNU MP: The GNU Multiple Precision Library, The Free Software Foundation, Inc.: Boston, 2002.

J. Gray, Mastering Mathematica: Programming Methods and Applications. 2nd editors, Academic Press: San Diego, 1998, p. 629.

R. L. Harris, Information Graphics: A Comprehensive Illustrated Reference: Visual Tools for Analyzing, Managing, and Communicating, Oxford University Press: New York, 1999.

I. Herman, G. Melançon, and M. S. Marshall, “Graph visualization and navigation in information visualization: A survey,” IEEE Transactions on Visualization and Computer Graphics, vol. 6, no. 1, pp. 24–43, 2000.

G. M. Hunter and K. Steiglitz, “Operations on images using quad trees,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-1, no. 2: pp. 145–153, 1979.

F. Hwang, D. Richards, and P. Winter, The Steiner Tree Problem, North-Holland: Amsterdam, Netherlands, 1992.

A. O. Ivanov and A. A. Tuzhilin, Minimal Networks: The Steiner Problem and Its Generalizations, CRC Press: Boca Raton, FL, 1994.

C. Jacob, Illustrating Evolutionary Computation with Mathematica, Morgan Kaufmann Publishers: San Francisco, 2001, p. 578.

T. Kamada and S. Kawai, “An algorithm for drawing general undirected graphs,” Information Processing Letters, vol. 31, pp. 7–15, 1989.

P. R. Keller and M. M. Keller, Visual Cues: Practical Data Visualization, IEEE Press: Piscataway, NJ, 1993.

G. Kirchoff, Annalen der Physik und Chemie, vol. 72, pp. 497–508, 1847.

A. Klinger, “Patterns and search statistics,” in Optimizing Methods in Statistics, J. S. Rustagi (Ed.), Academic: New York, 1971, pp. 303–337.

A. Klinger and C. R. Dyer, “Experiments on picture representation using regular decomposition,” Computer Graphics and Image Processing, vol. 5, pp. 68–105, 1976.

D. E. Knuth, The Art of Computer Programming: Volume 1: Fundamental Algorithms. 3rd edition, Vol. 1. Reading: Addison–Wesley, 1997.

J. R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection. Complex Adaptive Systems, The MIT Press: Cambridge, 1992.

J. Lamping, R. Rao, and P. Pirolli, “Focus+context based on hyperbolic geometry for visualizaing large hierarchies,” in Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems Part 1 of 2, May 7–11, 1995, Denver, CO., ACM: New York, 1995, pp. 401–408.

W. B. Langdon, “Quadratic bloat in genetic programming,” in GECCO 2000: Proceedings of the Genetic and Evolutionary Computation Conference, July 10–12, 2000, Las Vegas, L. D. Whitley, D. E. Goldberg, E. Cantú-Paz, L. Spector, I. Parmee, and H.-G. Beyer (Eds.), Morgan Kaufmann Publishers: San Francisco, 2000, pp. 451–458.

W. B. Langdon, “Size fair and homologous tree crossovers for tree genetic programming,” Genetic Programming and Evolvable Machines, vol. 1, nos. 1/2, pp. 95–119, 2000.

W. B. Langdon and R. Poli, “Fitness causes bloat,” in Soft Computing in Engineering Design and Manufacturing, P. K. Chawdhry, R. Roy, and R. K. Pant (Eds.), Springer-Verlag: London, 1997, pp. 23–27.

W. B. Langdon and R. Poli, Foundations of Genetic Programming, Springer-Verlag: Berlin, 2002.

W. B. Langdon, T. Soule, R. Poli, and J. A. Foster, “The evolution of size and shape,” in Advances in Genetic Programming 3, L. Spector, W. B. Langdon, U.-M. O’Reilly, and P. J. Angeline (Eds.), The MIT Press: Cambridge, 1999, pp. 163–190.

L. Margulis and K. V. Schwartz, Five Kingdoms: An Illustrated Guide to the Phyla of Life on Earth. Third edition, W.H. Freeman and Company, New York, 1999.

A. R. Mast, S. Kelso, A. J. Richards, D. J. Lang, D. M. S. Feller, and E. Conti, “Phylogenetic Relationships in

*Primula L.*and Related Genera (Primulaceae) Based on Noncoding Chloroplast DNA,” International Journal of Plant Science, vol. 162, no. 6, pp. 1381–1400, 2001.N. F. McPhee and N. J. Hopper, “Analysis of genetic diversity through population history,” in GECCO ’99: Proceeding of the Genetic and Evolutionary Computation Conference, 13–17 July 1999, Orlando, W. Banzhaf, J. M. Daida, A. E. Eiben, M. H. Garzon, V. Honavar, M. Jakiela, and R. E. Smith (Eds.), Morgan Kaufmann Publishers: San Francisco, 1999, pp. 1112–1120.

M. Mitchell, S. Forrest, and J. H. Holland, “The royal road for genetic algorithms: fitness landscapes and GA performance,” in Proceedings of the First European Conference on Artificial Life, Toward a Practice of Autonomous Systems, F. J. Varela and P. Bourgine (Eds.), The MIT Press: Cambridge, 1992, pp. 245–254.

P. Nordin, Evolutionary Program Induction of Binary Machine Code and Its Applications. 1997, der Universitat Dortmund am Fachereich Informatik: Dortmund.

U.-M. O’Reilly and D. E. Goldberg, “How fitness structure affects subsolution acquisition in genetic programming,” in Genetic Programming 1998: Proceedings of the Third Annual Conference, July 22–25, 1998, University of Wisconsin, Madison, J. R. Koza, W. Banzhaf, K. Chellapilla, K. Deb, M. Dorigo, D. B. Fogel, M. H. Garzon, D. E. Goldberg, H. Iba, and R. L. Riolo (Eds.), Morgan Kaufmann Publishers: San Francisco, 1998, pp. 269–277.

R. Poli, “General schema theory for genetic programming with subtree-swapping crossover,” in Genetic Programming: Proceedings of EuroGP 2001, April 18–20, 2001, Milan, J. F. Miller, M. Tomassini, P. L. Lanzi, C. Ryan, G. B. Tettamanzi, and W. B. Langdon (Eds.), Springer-Verlag: Berlin, 2001, pp. 143–159.

R. Poli and W. B. Langdon, “A new schema theory for genetic programming with one-point crossover and point mutation,” in Genetic Programming 1997: Proceedings of the Second Annual Conference, July 13-16, 1997, Stanford University, J. R. Koza, K. Deb, M. Dorigo, D. B. Fogel, M. Garzon, H. Iba, and R. L. Riolo (Eds.), Morgan Kaufmann Publishers: San Francisco, 1997, pp. 279–85.

R. Poli and W. B. Langdon, “Schema theory for genetic programming with one-point crossover and point mutation,” Evolutionary Computation, vol. 6, no. 3, pp. 231–252, 1998.

W. Punch, D. Zongker and E. Goodman, “The royal tree problem, a benchmark for single and multiple population genetic programming,” in Advances in Genetic Programming, P. J. Angeline and J. K.E. Kinnear (Eds.), The MIT Press: Cambridge, 1996, pp. 299–316.

J. P. Rosca, “Analysis of complexity drift in genetic programming,” in Genetic Programming 1997: Proceedings of the Second Annual Conference, July 13-16, 1997, Stanford University, J. R. Koza, K. Deb, M. Dorigo, D. B. Fogel, M. Garzon, H. Iba, and R. L. Riolo (Eds.), Morgan Kaufmann Publishers: San Francisco, 1997, pp. 286–94.

J. P. Rosca and D. H. Ballard, “Rooted-tree schemata in genetic programming,” in Advances in Genetic Programming 3, L. Spector, W. B. Langdon, U.-M. O’Reilly, and P. J. Angeline (Eds.), The MIT Press: Cambridge, 1999, pp. 243–271.

H. Samet, “Neighbor finding techniques for images represented by quadtrees,” Computer Graphics and Image Processing, vol. 18, pp. 37–57, 1980.

H. Samet and M. Tamminen, “Computing geometric properties of images represented by linear quadtrees,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-7, no. 2, pp. 229–240, 1985.

P. Schreiber, “On the history of the so-called Steiner Weber problem.” Wiss. Z. Erust-Monitz-Arndt-Univ. Greifswald, Math.-nat.wiss. Reihe, vol. 35, no. 3, 1986.

T. Soule and J. A. Foster, “Code size and depth flows in genetic programming,” in Genetic Programming 1997: Proceedings of the Second Annual Conference, July 13-16, 1997, Stanford University, J. R. Koza, K. Deb, M. Dorigo, D. B. Fogel, M. Garzon, H. Iba, and R. L. Riolo (Eds.), Morgan Kaufmann Publishers: San Francisco, 1997, pp. 313–320.

T. Soule and J. A. Foster, “Removal bias: A new cause of code growth in tree based evolutionary programming,” in The 1998 IEEE International Conference on Evolutionary Computation Proceedings: IEEE World Congress on Computational Intelligence. IEEE Press: Piscataway, 1998, pp. 781–786.

T. Soule, J. A. Foster, and J. Dickinson, “Code growth in genetic programming,” in Genetic Programming 1996: Proceedings of the First Annual Conference: July 28–31, 1996, Stanford University, J. R. Koza, D. E. Goldberg, D. B. Fogel, and R. L. Riolo (Eds.), The MIT Press: Cambridge, 1996, pp. 215–223.

R. P. Stanley, Enumerative Combinatorics I, Cambridge Studies in Advanced Mathematics, ed. W. Fulton, D. J. H. Garling, K. Ribet and P. Walters. Vol. 1. Cambridge University Press: Cambridge: 1997.

R. P. Stanley, Enumerative Combinatorics II. Cambridge Studies in Advanced Mathematics, W. Fulton, D. J. H. Garling, K. Ribet, and P. Walters (Eds.), vol. 2, Cambridge University Press: Cambridge, 1999.

S. H. Strogatz, “Exploring complex networks,” Nature, vol. 410, pp. 268–276, 2001.

E. R. Tufte, Envisioning Information, Graphics Press: Cheshire, CT, 1990.

E. R. Tufte, The Visual Display of Quantitative Information. Graphics Press: Cheshire, CT, 1983.

E. R. Tufte, Visual Explanations: Images and Quantities, Evidence and Narrative. Graphics Press: Cheshire, CT, 1997.

T. Wickham-Jones, Mathematica Graphics: Techniques and Applications, TELOS: New York, 1994.

S. Wolfram, A New Kind of Science, Wolfram Media, Inc: Champaign, IL, 2002.

D. H. Wolpert and W. G. Macready, “No free lunch theorems for optimization,” IEEE Transactions on Evolutionary Computation, vol. 1, no. 1, pp. 67–82, 1997.

## Author information

### Authors and Affiliations

## Rights and permissions

## About this article

### Cite this article

Daida, J.M., Hilss, A.M., Ward, D.J. *et al.* Visualizing Tree Structures in Genetic Programming.
*Genet Program Evolvable Mach* **6**, 79–110 (2005). https://doi.org/10.1007/s10710-005-7621-2

Received:

Revised:

Issue Date:

DOI: https://doi.org/10.1007/s10710-005-7621-2