Benchmarking Languages for Evolutionary Algorithms

  • J. J. Merelo
  • Pedro Castillo
  • Israel Blancas
  • Gustavo Romero
  • Pablo García-Sanchez
  • Antonio Fernández-Ares
  • Víctor Rivas
  • Mario García-Valdez
Conference paper
First Online:
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9598)

Abstract

Although performance is important, several other issues should be taken into account when choosing a particular language for implementing an evolutionary algorithm, such as the fact that the speed of different languages when carrying out an operation will depend on several factors, including the size of the operands, the version of the language and underlying factors such as the operating system. However, it is usual to rely on compiled languages, namely Java or C/C++, for carrying out any implementation without considering other languages or rejecting them outright on the basis of performance. Since there are a myriad of languages nowadays, it is interesting however to measure their speed when performing operations that are usual in evolutionary algorithms. That is why in this paper we have chosen three evolutionary algorithm operations: bitflip mutation, crossover and the fitness function OneMax evaluation, and measured the speed for several popular, and some not so popular, languages. Our measures confirm that, in fact, Java, C and C++ not only are the fastest, but also have a behaviour that is independent of the size of the chromosome. However, we have found other compiled language such as Go or interpreted languages such as Python to be fast enough for most purposes. Besides, these experiments show which of these measures are, in fact, the best for choosing an implementation language based on its performance.

Keywords

Evolutionary Algorithm Fitness Function Java Virtual Machine Open Source Implementation Popular Language 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access

Notes

Acknowledgements

This paper is part of the open science effort at the university of Granada. It has been written using knitr, and its source as well as the data used to create it can be downloaded from the GitHub repository https://github.com/geneura-papers/2015-ea-languages. It has been supported in part by GeNeura Team.

This work has been supported in part by projects TIN2014-56494-C4-3-P (Spanish Ministry of Economy and Competitiveness), SPIP2014-01437 (Dirección General de Tráfico), PRY142/14 (Fundación Pública Andaluza Centro de Estudios Andaluces en la IX Convocatoria de Proyectos de Investigación), and project V17-2015 of the Microprojects program 2015 from CEI BioTIC Granada.

References

  1. 1.
    Anderson, E., Tucek, J.: Efficiency matters!. ACM SIGOPS Oper. Syst. Rev. 44(1), 40–45 (2010)CrossRefGoogle Scholar
  2. 2.
    Prechelt, L.: An empirical comparison of seven programming languages. Computer 33(10), 23–29 (2000)CrossRefGoogle Scholar
  3. 3.
    Fulgham, B., Gouy, I.: The computer language benchmarks game (2010)Google Scholar
  4. 4.
    Wolpert, D.H., Macready, W.G.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)CrossRefGoogle Scholar
  5. 5.
    Langdon, W.B., Harrison, A.P.: Evolving regular expressions for genechip probe performance prediction. In: Rudolph, G., Jansen, T., Lucas, S., Poloni, C., Beume, N. (eds.) PPSN 2008. LNCS, vol. 5199, pp. 1061–1070. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  6. 6.
    Merelo, J.J., Romero, G., Arenas, M.G., Castillo, P.A., Mora, A.M., Laredo, J.L.J.: Implementation matters: programming best practices for evolutionary algorithms. In: Cabestany, J., Rojas, I., Joya, G. (eds.) IWANN 2011, Part II. LNCS, vol. 6692, pp. 333–340. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  7. 7.
    Grosan, C., Abraham, A.: Evolutionary algorithms. In: Grosan, C., Abraham, A. (eds.) Intelligent Systems. ISRL, vol. 17, pp. 345–386. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  8. 8.
    Merelo-Guervós, J.J., Castillo, P.A., Alba, E.: Algorithm: Evolutionary, a flexible Perl module for evolutionary computation. Soft Comput. 14(10), 1091–1109 (2010)CrossRefGoogle Scholar
  9. 9.
    Lee, W., Kim, H.Y.: Genetic algorithm implementation in Python. In: Fourth Annual ACIS International Conference on Computer and Information Science, 2005, pp. 8–11 (2005)Google Scholar
  10. 10.
    Erb, B., Kargl, F.: A conceptual model for event-sourced graph computing. In: Proceedings of the 9th ACM International Conference on Distributed Event-Based Systems. DEBS 2015, pp. 352–355. ACM, New York (2015)Google Scholar
  11. 11.
    García-Sánchez, P., González, J., Castillo, P.A., Merelo, J.J., Mora, A.M., Laredo, J.L.J., Arenas, M.G.: A distributed service oriented framework for metaheuristics using a public standard. In: González, J.R., Pelta, D.A., Cruz, C., Terrazas, G., Krasnogor, N. (eds.) NICSO 2010. SCI, vol. 284, pp. 211–222. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  12. 12.
    García-Sánchez, P., González, J., Castillo, P.A., García-Arenas, M., Merelo-Guervós, J.J.: Service oriented evolutionary algorithms. Soft. Comput. 17(6), 1059–1075 (2013)CrossRefGoogle Scholar
  13. 13.
    Odersky, M., Altherr, P., Cremet, V., Emir, B., Maneth, S., Micheloud, S., Mihaylov, N., Schinz, M., Stenman, E., Zenger, M.: An overview of the Scala programming language. Technical report, EPFL-Lausanne (2004)Google Scholar
  14. 14.
    Ventura, S., Romero, C., Zafra, A., Delgado, J.A., Hervás, C.: JCLEC: a java framework for evolutionary computation. Soft. Comput. 12(4), 381–392 (2008)CrossRefGoogle Scholar
  15. 15.
    Luke, S., Panait, L., Balan, G., Paus, S., Skolicki, Z., Bassett, J., Hubley, R., Chircop, A.: ECJ: a Java-based evolutionary computation research system. Downloadable versions and documentation can be found at the following (2006). http://cs.gmu.edu/eclab/projects/ecj
  16. 16.
    Liefooghe, A., Jourdan, L., Talbi, E.G.: A software framework based on a conceptual unified model for evolutionary multiobjective optimization: ParadisEO-MOEO. Eur. J. Oper. Res. 209(2), 104–112 (2011)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Alba, E., Almeida, F., Blesa, M.J., Cabeza, J., Cotta, C., Díaz, M., Dorta, I., Gabarró, J., León, C., Luna, J.M., Moreno, L., Pablos, C., Petit, J., Rojas, A., Xhafa, F.: MALLBA: a library of skeletons for combinatorial optimisation. In: Monien, B., Feldmann, R.L. (eds.) Euro-Par 2002. LNCS, vol. 2400, pp. 927–932. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  18. 18.
    Pike, R.: The go programming language. Talk given at Google’s Tech Talks (2009)Google Scholar
  19. 19.
    Ierusalimschy, R., De Figueiredo, L.H., Celes Filho, W.: Lua-an extensible extension language. Softw. Pract. Exper. 26(6), 635–652 (1996)CrossRefGoogle Scholar
  20. 20.
    Merelo, J.J., Castillo, P.A., Mora, A., Fernández-Ares, A., Esparcia-Alcázar, A.I., Cotta, C., Rico, N.: Studying and tackling noisy fitness in evolutionary design of game characters. In: Rosa, A., Merelo, J.J., Filipe, J., eds.: ECTA 2014 - Proceedings of the International Conference on Evolutionary Computation Theory and Applications, pp. 76–85 (2014)Google Scholar
  21. 21.
    Merelo-Guervós, J.J., Mora, A.M., Castillo, P.A., Cotta, C., García-Valdez, M.: A search for scalable evolutionary solutions to the game of MasterMind. In: IEEE Congress on Evolutionary Computation, pp. 2298–2305. IEEE (2013)Google Scholar
  22. 22.
    Merelo-Guervós, J.J., Castillo-Valdivieso, P.A., Mora-García, A., Esparcia-Alcázar, A., Rivas-Santos, V.M.: NodEO, a multi-paradigm distributed evolutionary algorithm platform in JavaScript. In: Genetic and Evolutionary Computation Conference, GECCO 2014, Vancouver, BC, Canada, July 12–16, 2014, pp. 1155–1162. ACM, Companion Material Proceedings (2014)Google Scholar
  23. 23.
    Merelo-Guervós, J.J., García-Sánchez, P.: Modeling browser-based distributed evolutionary computation systems. CoRR abs/1503.06424 (2015)Google Scholar
  24. 24.
    Rivas, V.M., Guervós, J.J.M., López, G.R., Arenas-García, M., Mora, A.M.: An object-oriented library in javascriptto build modular and flexiblecross-platform evolutionary algorithms. In: Esparcia-Alcázar, A.I., Mora, A.M. (eds.) EvoApplications 2014. LNCS, vol. 8602, pp. 853–862. Springer, Heidelberg (2014)Google Scholar
  25. 25.
    Fortin, F.A., Rainville, D., Gardner, M.A.G., Parizeau, M., Gagné, C., et al.: DEAP: Evolutionary algorithms made easy. J. Mach. Learn. Res. 13(1), 2171–2175 (2012)MathSciNetMATHGoogle Scholar
  26. 26.
    Bezanson, J., Karpinski, S., Shah, V.B., Edelman, A.: Julia: A fast dynamic language for technical computing (2012). arXiv preprint arXiv:1209.5145

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • J. J. Merelo
    • 1
  • Pedro Castillo
    • 1
  • Israel Blancas
    • 1
  • Gustavo Romero
    • 1
  • Pablo García-Sanchez
    • 1
  • Antonio Fernández-Ares
    • 1
  • Víctor Rivas
    • 2
  • Mario García-Valdez
    • 3
  1. 1.Department of Computer Architecture and Technlogy and CITICUniversity of GranadaGranadaSpain
  2. 2.Department of InformaticsUniversity of JaénJaénSpain
  3. 3.Instituto Politécnico de TijuanaTijuanaMexico

Personalised recommendations