Benchmarking Languages for Evolutionary Algorithms
Abstract
Although performance is important, several other issues should be taken into account when choosing a particular language for implementing an evolutionary algorithm, such as the fact that the speed of different languages when carrying out an operation will depend on several factors, including the size of the operands, the version of the language and underlying factors such as the operating system. However, it is usual to rely on compiled languages, namely Java or C/C++, for carrying out any implementation without considering other languages or rejecting them outright on the basis of performance. Since there are a myriad of languages nowadays, it is interesting however to measure their speed when performing operations that are usual in evolutionary algorithms. That is why in this paper we have chosen three evolutionary algorithm operations: bitflip mutation, crossover and the fitness function OneMax evaluation, and measured the speed for several popular, and some not so popular, languages. Our measures confirm that, in fact, Java, C and C++ not only are the fastest, but also have a behaviour that is independent of the size of the chromosome. However, we have found other compiled language such as Go or interpreted languages such as Python to be fast enough for most purposes. Besides, these experiments show which of these measures are, in fact, the best for choosing an implementation language based on its performance.
Keywords
Evolutionary Algorithm Fitness Function Java Virtual Machine Open Source Implementation Popular LanguageNotes
Acknowledgements
This paper is part of the open science effort at the university of Granada. It has been written using knitr, and its source as well as the data used to create it can be downloaded from the GitHub repository https://github.com/geneura-papers/2015-ea-languages. It has been supported in part by GeNeura Team.
This work has been supported in part by projects TIN2014-56494-C4-3-P (Spanish Ministry of Economy and Competitiveness), SPIP2014-01437 (Dirección General de Tráfico), PRY142/14 (Fundación Pública Andaluza Centro de Estudios Andaluces en la IX Convocatoria de Proyectos de Investigación), and project V17-2015 of the Microprojects program 2015 from CEI BioTIC Granada.
References
- 1.Anderson, E., Tucek, J.: Efficiency matters!. ACM SIGOPS Oper. Syst. Rev. 44(1), 40–45 (2010)CrossRefGoogle Scholar
- 2.Prechelt, L.: An empirical comparison of seven programming languages. Computer 33(10), 23–29 (2000)CrossRefGoogle Scholar
- 3.Fulgham, B., Gouy, I.: The computer language benchmarks game (2010)Google Scholar
- 4.Wolpert, D.H., Macready, W.G.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)CrossRefGoogle Scholar
- 5.Langdon, W.B., Harrison, A.P.: Evolving regular expressions for genechip probe performance prediction. In: Rudolph, G., Jansen, T., Lucas, S., Poloni, C., Beume, N. (eds.) PPSN 2008. LNCS, vol. 5199, pp. 1061–1070. Springer, Heidelberg (2008)CrossRefGoogle Scholar
- 6.Merelo, J.J., Romero, G., Arenas, M.G., Castillo, P.A., Mora, A.M., Laredo, J.L.J.: Implementation matters: programming best practices for evolutionary algorithms. In: Cabestany, J., Rojas, I., Joya, G. (eds.) IWANN 2011, Part II. LNCS, vol. 6692, pp. 333–340. Springer, Heidelberg (2011)CrossRefGoogle Scholar
- 7.Grosan, C., Abraham, A.: Evolutionary algorithms. In: Grosan, C., Abraham, A. (eds.) Intelligent Systems. ISRL, vol. 17, pp. 345–386. Springer, Heidelberg (2011)CrossRefGoogle Scholar
- 8.Merelo-Guervós, J.J., Castillo, P.A., Alba, E.: Algorithm: Evolutionary, a flexible Perl module for evolutionary computation. Soft Comput. 14(10), 1091–1109 (2010)CrossRefGoogle Scholar
- 9.Lee, W., Kim, H.Y.: Genetic algorithm implementation in Python. In: Fourth Annual ACIS International Conference on Computer and Information Science, 2005, pp. 8–11 (2005)Google Scholar
- 10.Erb, B., Kargl, F.: A conceptual model for event-sourced graph computing. In: Proceedings of the 9th ACM International Conference on Distributed Event-Based Systems. DEBS 2015, pp. 352–355. ACM, New York (2015)Google Scholar
- 11.García-Sánchez, P., González, J., Castillo, P.A., Merelo, J.J., Mora, A.M., Laredo, J.L.J., Arenas, M.G.: A distributed service oriented framework for metaheuristics using a public standard. In: González, J.R., Pelta, D.A., Cruz, C., Terrazas, G., Krasnogor, N. (eds.) NICSO 2010. SCI, vol. 284, pp. 211–222. Springer, Heidelberg (2010)CrossRefGoogle Scholar
- 12.García-Sánchez, P., González, J., Castillo, P.A., García-Arenas, M., Merelo-Guervós, J.J.: Service oriented evolutionary algorithms. Soft. Comput. 17(6), 1059–1075 (2013)CrossRefGoogle Scholar
- 13.Odersky, M., Altherr, P., Cremet, V., Emir, B., Maneth, S., Micheloud, S., Mihaylov, N., Schinz, M., Stenman, E., Zenger, M.: An overview of the Scala programming language. Technical report, EPFL-Lausanne (2004)Google Scholar
- 14.Ventura, S., Romero, C., Zafra, A., Delgado, J.A., Hervás, C.: JCLEC: a java framework for evolutionary computation. Soft. Comput. 12(4), 381–392 (2008)CrossRefGoogle Scholar
- 15.Luke, S., Panait, L., Balan, G., Paus, S., Skolicki, Z., Bassett, J., Hubley, R., Chircop, A.: ECJ: a Java-based evolutionary computation research system. Downloadable versions and documentation can be found at the following (2006). http://cs.gmu.edu/eclab/projects/ecj
- 16.Liefooghe, A., Jourdan, L., Talbi, E.G.: A software framework based on a conceptual unified model for evolutionary multiobjective optimization: ParadisEO-MOEO. Eur. J. Oper. Res. 209(2), 104–112 (2011)MathSciNetCrossRefGoogle Scholar
- 17.Alba, E., Almeida, F., Blesa, M.J., Cabeza, J., Cotta, C., Díaz, M., Dorta, I., Gabarró, J., León, C., Luna, J.M., Moreno, L., Pablos, C., Petit, J., Rojas, A., Xhafa, F.: MALLBA: a library of skeletons for combinatorial optimisation. In: Monien, B., Feldmann, R.L. (eds.) Euro-Par 2002. LNCS, vol. 2400, pp. 927–932. Springer, Heidelberg (2002)CrossRefGoogle Scholar
- 18.Pike, R.: The go programming language. Talk given at Google’s Tech Talks (2009)Google Scholar
- 19.Ierusalimschy, R., De Figueiredo, L.H., Celes Filho, W.: Lua-an extensible extension language. Softw. Pract. Exper. 26(6), 635–652 (1996)CrossRefGoogle Scholar
- 20.Merelo, J.J., Castillo, P.A., Mora, A., Fernández-Ares, A., Esparcia-Alcázar, A.I., Cotta, C., Rico, N.: Studying and tackling noisy fitness in evolutionary design of game characters. In: Rosa, A., Merelo, J.J., Filipe, J., eds.: ECTA 2014 - Proceedings of the International Conference on Evolutionary Computation Theory and Applications, pp. 76–85 (2014)Google Scholar
- 21.Merelo-Guervós, J.J., Mora, A.M., Castillo, P.A., Cotta, C., García-Valdez, M.: A search for scalable evolutionary solutions to the game of MasterMind. In: IEEE Congress on Evolutionary Computation, pp. 2298–2305. IEEE (2013)Google Scholar
- 22.Merelo-Guervós, J.J., Castillo-Valdivieso, P.A., Mora-García, A., Esparcia-Alcázar, A., Rivas-Santos, V.M.: NodEO, a multi-paradigm distributed evolutionary algorithm platform in JavaScript. In: Genetic and Evolutionary Computation Conference, GECCO 2014, Vancouver, BC, Canada, July 12–16, 2014, pp. 1155–1162. ACM, Companion Material Proceedings (2014)Google Scholar
- 23.Merelo-Guervós, J.J., García-Sánchez, P.: Modeling browser-based distributed evolutionary computation systems. CoRR abs/1503.06424 (2015)Google Scholar
- 24.Rivas, V.M., Guervós, J.J.M., López, G.R., Arenas-García, M., Mora, A.M.: An object-oriented library in javascriptto build modular and flexiblecross-platform evolutionary algorithms. In: Esparcia-Alcázar, A.I., Mora, A.M. (eds.) EvoApplications 2014. LNCS, vol. 8602, pp. 853–862. Springer, Heidelberg (2014)Google Scholar
- 25.Fortin, F.A., Rainville, D., Gardner, M.A.G., Parizeau, M., Gagné, C., et al.: DEAP: Evolutionary algorithms made easy. J. Mach. Learn. Res. 13(1), 2171–2175 (2012)MathSciNetzbMATHGoogle Scholar
- 26.Bezanson, J., Karpinski, S., Shah, V.B., Edelman, A.: Julia: A fast dynamic language for technical computing (2012). arXiv preprint arXiv:1209.5145