Flex-GP: Genetic Programming on the Cloud

  • Dylan Sherry
  • Kalyan Veeramachaneni
  • James McDermott
  • Una-May O’Reilly
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7248)


We describe Flex-GP, which we believe to be the first largescale genetic programming cloud computing system. We took advantage of existing software and selected a socket-based, client-server architecture and an island-based distribution model. We developed core components required for deployment on Amazon’s EC2. Scaling the system to hundreds of nodes presented several unexpected challenges and required the development of software for automatically managing deployment, reporting, and error handling. The system’s performance was evaluated on two metrics, performance and speed, on a difficult symbolic regression problem. Our largest successful Flex-GP runs reached 350 nodes and taught us valuable lessons for the next phase of scaling.


Cloud Computing Genetic Program Spot Price Island Model Symbolic Regression 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Armbrust, M., Fox, A., Griffith, R., Joseph, A., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., Zaharia, M.: Above the clouds: A berkeley view of cloud computing. EECS Department, University of California, Berkeley, Tech. Rep. UCB/EECS-2009-28 (2009)Google Scholar
  2. 2.
    Cantú-Paz, E.: A survey of parallel genetic algorithms. Calculateurs Paralleles 10(2) (1998)Google Scholar
  3. 3.
    Dean, J., Ghemawat, S.: MapReduce: Simplified data processing on large clusters. Communications of ACM 51(1), 107–113 (2008)CrossRefGoogle Scholar
  4. 4.
    Laredo, J.L.J., Castillo, P.A., Paechter, B., Mora, A.M., Alfaro-Cid, E., Esparcia-Alcázar, A.I., Merelo, J.J.: Empirical Validation of a Gossiping Communication Mechanism for Parallel EAs. In: Giacobini, M. (ed.) EvoWorkshops 2007. LNCS, vol. 4448, pp. 129–136. Springer, Heidelberg (2007)Google Scholar
  5. 5.
    Luke, S., Panait, L., Balan, G., Paus, S., Skolicki, Z., Bassett, J., Hubley, R., Chircop, A.: ECJ: A Java-based evolutionary computation research system (2007),
  6. 6.
    Ograph, B., Morgens, Y.: Cloud computing. Communications of the ACM 51(7) (2008)Google Scholar
  7. 7.
    Poli, R., Langdon, W., McPhee, N.: A field guide to genetic programming. Lulu Enterprises UK Ltd. (2008)Google Scholar
  8. 8.
    Tomassini, M.: Spatially structured evolutionary algorithms. Springer, Heidelberg (2005)zbMATHGoogle Scholar
  9. 9.
    Vanneschi, L.: Theory and Practice for Efficient Genetic Programming. Ph.D. thesis, Université de Lausanne (2004)Google Scholar
  10. 10.
    Verma, A., Llora, X., Goldberg, D.E., Campbell, R.H.: Scaling genetic algorithms using MapReduce. In: Proceedings of Intelligent Systems Design and Applications, pp. 13–18 (2009)Google Scholar
  11. 11.
    Vladislavleva, E., Smits, G., Den Hertog, D.: Order of nonlinearity as a complexity measure for models generated by symbolic regression via Pareto genetic programming. IEEE Transactions on Evolutionary Computation 13(2), 333–349 (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Dylan Sherry
    • 1
  • Kalyan Veeramachaneni
    • 1
  • James McDermott
    • 1
  • Una-May O’Reilly
    • 1
  1. 1.Massachusetts Institute of TechnologyUSA

Personalised recommendations