Improving the Tartarus Problem as a Benchmark in Genetic Programming

  • Thomas D. Griffiths
  • Anikó Ekárt
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10196)


For empirical research on computer algorithms, it is essential to have a set of benchmark problems on which the relative performance of different methods and their applicability can be assessed. In the majority of computational research fields there are established sets of benchmark problems; however, the field of genetic programming lacks a similarly rigorously defined set of benchmarks. There is a strong interest within the genetic programming community to develop a suite of benchmarks. Following recent surveys [7], the desirable characteristics of a benchmark problem are now better defined. In this paper the Tartarus problem is proposed as a tunably difficult benchmark problem for use in Genetic Programming. The justification for this proposal is presented, together with guidance on its usage as a benchmark.


Genetic programming Benchmark Tartarus 


  1. 1.
    Korkmaz, E.E., Üçoluk, G.: Design and usage of a new benchmark problem for genetic programming. In: Yazıcı, A., Şener, C. (eds.) ISCIS 2003. LNCS, vol. 2869, pp. 561–567. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  2. 2.
    Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection (1992)Google Scholar
  3. 3.
    McDermott, J., White, D.R., Luke, S., Manzoni, L., Castelli, M., Vanneschi, L., Jaskowski, W., Krawiec, K., Harper, R., De Jong, K., O’Reilly, U.M.: Genetic programming needs better benchmarks. In: Soule, T., et al. (eds.) Proceedings of the 14th International Conference on Genetic and Evolutionary Computation, GECCO 2012, pp. 791–798 (2012)Google Scholar
  4. 4.
    Sendhoff, B., Roberts, M., Yao, X.: Evolutionary computation benchmarking repository. IEEE Comput. Intell. Mag. 1, 50–60 (2006)Google Scholar
  5. 5.
    Teller, A.: The evolution of mental models. In: Kinnear Jr. K.E. (ed.) Advances in Genetic Programming, pp. 199–217 (1994)Google Scholar
  6. 6.
    Vanneschi, L., Castelli, M., Manzoni, L.: The K landscapes: a tunably difficult benchmark for genetic programming. In: Krasnogor, N., et al. (eds.) Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation, GECCO 2011, pp. 1467–1474 (2011)Google Scholar
  7. 7.
    White, D.R., McDermott, J., Castelli, M., Manzoni, L., Goldman, B.W., Kronberger, G., Jaśkowski, W., O’Reilly, U.M., Luke, S.: Better GP benchmarks: community survey results and proposals. Genet. Program Evolvable Mach. 14(1), 3–29 (2013)CrossRefGoogle Scholar
  8. 8.
    Woodward, J., Martin, S., Swan, J.: Benchmarks that matter for genetic programming. In: Woodward, J., et al. (eds.) 4th Workshop on Evolutionary Computation for the Automated Design of Algorithms, GECCO 2014, pp. 1397–1404 (2014)Google Scholar
  9. 9.
    Jiju, A.: Design of Experiments for Engineers and Scientists. Elsevier, Amsterdam (2003)Google Scholar
  10. 10.
    Koza, J.R.: Scalable learning in genetic programming using automatic function definition. In: Kinnear Jr. K.E. (ed.) Advances in Genetic Programming, pp. 99–117 (1994)Google Scholar
  11. 11.
    Ashlock, D., Joenks, M.: ISAc lists: a different program induction method. In: Koza, J.R., et al. (eds.) Proceedings of the Second Annual Conference on Genetic Programming, pp. 18–26 (1998)Google Scholar
  12. 12.
    Ashlock, D., Freeman, J.: A pure finite state baseline for Tartarus. In: Proceedings of the 2000 Congress on Evolutionary Computation, pp. 1223–1230 (2000)Google Scholar
  13. 13.
    Dick, G.: An effective parse tree representation for Tartarus. In: Blum, C., et al. (eds.) Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation, GECCO 2013, pp. 1397–1404 (2013)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Aston Lab for Intelligent Collectives Engineering (ALICE)Aston UniversityBirminghamUK

Personalised recommendations