A Library to Run Evolutionary Algorithms in the Cloud Using MapReduce

  • Pedro Fazenda
  • James McDermott
  • Una-May O’Reilly
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7248)

Abstract

We discuss ongoing development of an evolutionary algorithm library to run on the cloud. We relate how we have used the Hadoop open-source MapReduce distributed data processing framework to implement a single “island” with a potentially very large population. The design generalizes beyond the current, one-off kind of MapReduce implementations. It is in preparation for the library becoming a modeling or optimization service in a service oriented architecture or a development tool for designing new evolutionary algorithms.

Keywords

MapReduce cloud computing Hadoop evolutionary algorithms 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Web resource: ApacheHadoop, http://hadoop.apache.org/core
  2. 2.
    Gunarathne, T., Wu, T.L., Qiu, J., Fox, G.: MapReduce in the clouds for science. In: 2010 IEEE Second International Conference on Cloud Computing Technology and Science, CloudCom (2010)Google Scholar
  3. 3.
    Web resource: MAHOUT, http://mahout.apache.org/
  4. 4.
    Jin, C., Vecchiola, C., Buyya, R.: MRPGA: An extension of MapReduce for parallelizing genetic algorithms. In: IEEE Fourth International Conference on eScience 2008, pp. 214–221. IEEE (2008)Google Scholar
  5. 5.
    Verma, A., Llora, X., Campbell, R., Goldberg, D.: Scaling genetic algorithms using MapReduce. Technical report, Illigal TR 2009007Google Scholar
  6. 6.
    Verma, A., Llora, X., Venkataraman, S., Goldberg, D., Campbell, R.: Scaling ECGA model building via data-intensive computing. In: 2010 IEEE Congress on Evolutionary Computation, CEC (2010)Google Scholar
  7. 7.
    Verma, A., Llora, X., Goldberg, D., Campbell, R.: Scaling genetic algorithms using MapReduce. In: Ninth International Conference on Intelligent Systems Design and Applications, ISDA 2009 (2009)Google Scholar
  8. 8.
    Wang, S., Gao, B.J., Wang, K., Lauw, H.W.: Parallel learning to rank for information retrieval. In: Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information, SIGIR 2011, pp. 1083–1084. ACM, New York (2011)CrossRefGoogle Scholar
  9. 9.
    Huang, D.W., Lin, J.: Scaling populations of a genetic algorithm for job shop scheduling problems using MapReduce. In: 2010 IEEE Second International Conference on Cloud Computing Technology and Science, CloudCom (2010)Google Scholar
  10. 10.
    Verma, A., Zea, N., Cho, B., Gupta, I., Campbell, R.: Breaking the MapReduce stage barrier. In: 2010 IEEE International Conference on Cluster Computing (CLUSTER), pp. 235–244. IEEE (2010)Google Scholar
  11. 11.
    Web resource: Amazon EC2, http://aws.amazon.com/ec2/
  12. 12.
    Vladislavleva, E., Smits, G., Den Hertog, D.: Order of nonlinearity as a complexity measure for models generated by symbolic regression via pareto genetic programming. IEEE Transactions on Evolutionary Computation 13(2), 333–349 (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Pedro Fazenda
    • 1
    • 2
  • James McDermott
    • 2
  • Una-May O’Reilly
    • 2
  1. 1.Institute for Systems and RoboticsISTLisbonPortugal
  2. 2.Evolutionary Design and Optimization Group, CSAILMITUSA

Personalised recommendations