Evolutionary Algorithms on Volunteer Computing Platforms: The MilkyWay@Home Project

  • Nate Cole
  • Travis Desell
  • Daniel Lombraña González
  • Francisco Fernández de Vega
  • Malik Magdon-Ismail
  • Heidi Newberg
  • Boleslaw Szymanski
  • Carlos Varela
Part of the Studies in Computational Intelligence book series (SCI, volume 269)


Evolutionary algorithms (EAs) require large scale computing resources when tackling real world problems. Such computational requirement is derived from inherently complex fitness evaluation functions, large numbers of individuals per generation, and the number of iterations required by EAs to converge to a satisfactory solution. Therefore, any source of computing power can significantly benefit researchers using evolutionary algorithms. We present the use of volunteer computing (VC) as a platform for harnessing the computing resources of commodity machines that are nowadays present at homes, companies and institutions. Taking into account that currently desktop machines feature significant computing resources (dual cores, gigabytes of memory, gigabit network connections, etc.), VC has become a cost-effective platform for running time consuming evolutionary algorithms in order to solve complex problems, such as finding substructure in the Milky Way Galaxy, the problem we address in detail in this chapter.


Particle Swarm Optimization Dwarf Galaxy Desktop Grid Tidal Stream Astrophysical Journal 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Allen, M.: Do it yourself climate prediction. Nature (1999)Google Scholar
  2. 2.
    Anderson, D.: Boinc: a system for public-resource computing and storage. In: Proceedings of Fifth IEEE/ACM International Workshop on Grid Computing, 2004, pp. 4–10 (2004)Google Scholar
  3. 3.
    Anderson, D.P., Cobb, J., Korpela, E., Lebofsky, M., Werthimer, D.: Seti@home: an experiment in public-resource computing. Commun. ACM 45(11), 56–61 (2002), CrossRefGoogle Scholar
  4. 4.
    Anderson, D.P., Fedak, G.: The computational and storage potential of volunteer computing. In: Proceedings of CCGRID, pp. 73–80 (2006)Google Scholar
  5. 5.
    Andre, D., Koza, J.R.: Parallel genetic programming: a scalable implementation using the transputer network architecture, pp. 317–337 (1996)Google Scholar
  6. 6.
    Belokurov, V., Zucker, D.B., Evans, N.W., Gilmore, G., Vidrih, S., Bramich, D.M., Newberg, H.J., Wyse, R.F.G., Irwin, M.J., Fellhauer, M., Hewett, P.C., Walton, N.A., Wilkinson, M.I., Cole, N., Yanny, B., Rockosi, C.M., Beers, T.C., Bell, E.F., Brinkmann, J., Ivezić, Ž., Lupton, R.: The Field of Streams: Sagittarius and Its Siblings. Astrophysical Journal Letters 642, L137–L140 (2006)CrossRefGoogle Scholar
  7. 7.
    Bennet, F.H.I., Koza, J.R., Shipman, J., Stiffelman, O.: Building a parallel computer system for $18,000 that performs a half peta-flop per day. In: Proceedings of the Genetic and Evolutionary Computation Conference, Orlando, Florida, USA, pp. 1484–1490 (1999)Google Scholar
  8. 8.
    Cahon, S., Melab, N., Talbi, E.: ParadisEO: A Framework for the Reusable Design of Parallel and Distributed Metaheuristics. Journal of Heuristics 10(3), 357–380 (2004)CrossRefGoogle Scholar
  9. 9.
    Cantu-Paz, E.: A survey of parallel genetic algorithms. Calculateurs Paralleles, Reseaux et Systems Repartis 10(2), 141–171 (1998)Google Scholar
  10. 10.
    Cole, N., Jo Newberg, H., Magdon-Ismail, M., Desell, T., Szymanski, B., Varela, C.: Tracing the Sagittarius Tidal Stream with Maximum Likelihood. In: American Institute of Physics Conference Series, vol. 1082, pp. 216–220 (2008), doi:10.1063/1.3059049Google Scholar
  11. 11.
    Cole, N., Newberg, H.J., Magdon-Ismail, M., Desell, T., Dawsey, K., Hayashi, W., Liu, X.F., Purnell, J., Szymanski, B., Varela, C., Willett, B., Wisniewski, J.: Maximum Likelihood Fitting of Tidal Streams with Application to the Sagittarius Dwarf Tidal Tails. The Astrophysical Journal 683, 750–766 (2008)CrossRefGoogle Scholar
  12. 12.
    Desell, T., Cole, N., Magdon-Ismail, M., Newberg, H., Szymanski, B., Varela, C.: Distributed and generic maximum likelihood evaluation. In: 3rd IEEE International Conference on e-Science and Grid Computing (eScience2007), Bangalore, India, pp. 337–344 (2007)Google Scholar
  13. 13.
    Dingxue, Z., Zhihong, G., Xinzhi, L.: An adaptive particle swarm optimization algorithm and simulation. In: IEEE International Conference on Automation and Logistics, pp. 2399–2042 (2007)Google Scholar
  14. 14.
  15. 15.
    Duffau, S., Zinn, R., Vivas, A.K., Carraro, G., Méndez, R.A., Winnick, R., Gallart, C.: Spectroscopy of QUEST RR Lyrae Variables: The New Virgo Stellar Stream. The Astrophysical Journal Letters 636, L97–L100 (2006)CrossRefGoogle Scholar
  16. 16.
    Eberhart, R.C., Kennedy, J.: A new optimizer using particle swarm theory. In: Sixth International Symposium on Micromachine and Human Science, pp. 33–43 (1995)Google Scholar
  17. 17.
    Eggen, O.J., Lynden-Bell, D., Sandage, A.R.: Evidence from the motions of old stars that the Galaxy collapsed. The Astrophysical Journal 136, 748 (1962)CrossRefGoogle Scholar
  18. 18.
    Fernández, F., Tomassini, M.L.V.: An empirical study of multipopulation genetic programming. Genetic Programming and Evolvable Machines (2003)Google Scholar
  19. 19.
    Fedak, G., Germain, C., Neri, V., Cappello, F.: XtremWeb: A Generic Global Computing System. In: Proceedings of the IEEE International Symposium on Cluster Computing and the Grid (CCGRID 2001) (2001)Google Scholar
  20. 20.
    Fernandez, F., Spezzano, G., Tomassini, M., Vanneschi, L.: Parallel genetic programming. In: Alba, E. (ed.) Parallel Metaheuristics, Parallel and Distributed Computing, ch. 6, pp. 127–153. Wiley-Interscience, Hoboken (2005)Google Scholar
  21. 21.
    Fernández, F., Cantú-Paz, E.: Special issue parallel bioinspired algorithms. Journal of Parallel and Distributed Computing 66(8) (2006)Google Scholar
  22. 22.
    Foster, I., Kesselman, C.: Globus: A metacomputing infrastructure toolkit. The International Journal of Supercomputer Applications and High Performance Computing 11(2), 115–128 (1997), CrossRefGoogle Scholar
  23. 23.
    Foundation, F.S.: Gnu lesser general public license, version 3,
  24. 24.
    Freeman, K., Bland-Hawthorn, J.: The New Galaxy: Signatures of Its Formation. Annual Review of Astronomy & Astrophysics 40, 487–537 (2002)CrossRefGoogle Scholar
  25. 25.
    Freeman, K.C.: The Galactic spheroid and old disk. Annual Review of Astronomy & Astrophysics 25, 603–632 (1987)CrossRefGoogle Scholar
  26. 26.
    Gagné, C., Parizeau, M., Dubreuil, M.: Distributed beagle: An environment for parallel and distributed evolutionary computations. In: Proc. of the 17th Annual International Symposium on High Performance Computing Systems and Applications (HPCS) 2003, pp. 201–208 (2003)Google Scholar
  27. 27.
    González, D.L., de Vega, F.F., Trujillo, L., Olague, G., Araujo, L., Castillo, P., Merelo, J.J., Sharman, K.: Increasing gp computing power for free via desktop grid computing and virtualization. In: Proceedings of the 17th Euromicro Conference on Parallel, Distributed and Network-based Processing, Weimar, Germany, pp. 419–423 (2009)Google Scholar
  28. 28.
    Ibata, R., Irwin, M., Lewis, G.F., Stolte, A.: Galactic Halo Substructure in the Sloan Digital Sky Survey: The Ancient Tidal Stream from the Sagittarius Dwarf Galaxy. The Astrophysical Journal Letters 547, L133–L136 (2001)CrossRefGoogle Scholar
  29. 29.
    Ibata, R., Lewis, G.F., Irwin, M., Totten, E., Quinn, T.: Great Circle Tidal Streams: Evidence for a Nearly Spherical Massive Dark Halo around the Milky Way. The Astrophysical Journal 551, 294–311 (2001)CrossRefGoogle Scholar
  30. 30.
    Ibata, R.A., Gilmore, G., Irwin, M.J.: A Dwarf Satellite Galaxy in Sagittarius. Nature 370, 194 (1994)CrossRefGoogle Scholar
  31. 31.
    Kennedy, J., Eberhart, R.C.: Particle swarm optimization. In: IEEE International Conference on Neural Networks, vol. 4, pp. 1942–1948 (1995)Google Scholar
  32. 32.
    Kesselman, C., Foster, I.: The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann, San Francisco (1999)Google Scholar
  33. 33.
    Laure, E., Fisher, S., Frohner, A., Grandi, C., Kunszt, P., Krenek, A., Mulmo, O., Pacini, F., Prelz, F., White, J., et al.: Programming the Grid with gLite. Computational Methods in Science and Technology 12(1), 33–45 (2006)Google Scholar
  34. 34.
    Law, D.R., Johnston, K.V., Majewski, S.R.: A Two Micron All-Sky Survey View of the Sagittarius Dwarf Galaxy. IV. Modeling the Sagittarius Tidal Tails. The Astrophysical Journal 619, 807–823 (2005)CrossRefGoogle Scholar
  35. 35.
    Lombraña, D., Fernández, F., Trujillo, L., Olague, G., Cárdenas, M., Araujo, L., Castillo, P., Sharman, K., Silva, A.: Interpreted applications within boinc infrastructure. In: Ibergrid 2008, Porto, Portugal, pp. 261–272 (2008)Google Scholar
  36. 36.
    Luke, S., Panait, L., Balan, G., Paus, S., Skolicki, Z., Popovici, E., Harrison, J., Bassett, J., Hubley, R., Chircop, A.: Ecj a java-based evolutionary computation research system (2007),
  37. 37.
    Litzkow, M., Tannenbaum, T., Basney, J., Livny, M.: Checkpoint and migration of unix processes in the condor distributed processing system. Tech. rep., University of Wisconsin (1997)Google Scholar
  38. 38.
    McNett, D.: Rc5-64 has been solved!
  39. 39.
    McNett, D.: Secure encryption challenged by internet-linked computers,
  40. 40.
    Newberg, H.J., Yanny, B., Cole, N., Beers, T.C., Re Fiorentin, P., Schneider, D.P., Wilhelm, R.: The Overdensity in Virgo, Sagittarius Debris, and the Asymmetric Spheroid. The Astrophysical Journal 668, 221–235 (2007)CrossRefGoogle Scholar
  41. 41.
    Newberg, H.J., Yanny, B., Rockosi, C., Grebel, E.K., Rix, H.W., Brinkmann, J., Csabai, I., Hennessy, G., Hindsley, R.B., Ibata, R., Ivezić, Z., Lamb, D., Nash, E.T., Odenkirchen, M., Rave, H.A., Schneider, D.P., Smith, J.A., Stolte, A., York, D.G.: The Ghost of Sagittarius and Lumps in the Halo of the Milky Way. The Astrophysical Journal 569, 245–274 (2002)CrossRefGoogle Scholar
  42. 42.
    de la, O, F.C., Guisado, J.L., Lombraña, D., Fernández, F.: Una herramienta de programación genética paralela que aprovecha recursos públicos de computación. In: V Congreso Español sobre Metaheurísticas, Algoritmos Evolutivos y Bioinspirados, Tenerife, Spain, pp. 167–173 (2007) Google Scholar
  43. 43.
    Purnell, J., Magdon-Ismail, M., Newberg, H.: A probabilistic approach to finding geometric objects in spatial datasets of the Milky Way. In: Hacid, M.-S., Murray, N.V., Raś, Z.W., Tsumoto, S. (eds.) ISMIS 2005. LNCS (LNAI), vol. 3488, pp. 475–484. Springer, Heidelberg (2005)Google Scholar
  44. 44.
    Pande Vijay, S., Ian, B., Jarrod, C., Elmer Sidney, P., Siraj, K., Larson Stefan, M., Young Min, R., Shirts Michael, R., Snow Christopher, D., Sorin Eric, J., Zagrovic, B.: Atomistic protein folding simulations on the submillisecond time scale using worldwide distributed computing. Biopolymers 68, 91–109 (2003)CrossRefGoogle Scholar
  45. 45.
    Samples, M., Daida, J., Byom, M., Pizzimenti, M.: Parameter sweeps for exploring GP parameters. In: Proceedings of the 2005 workshops on Genetic and evolutionary computation, pp. 212–219 (2005)Google Scholar
  46. 46.
    Searle, L., Zinn, R.: Compositions of halo clusters and the formation of the galactic halo. The Astrophysical Journal 225, 357–379 (1978)CrossRefGoogle Scholar
  47. 47.
    Sintes, A.M.: Recent results on the search for continuous sources with ligo and geo600 (2005), arXiv.orgGoogle Scholar
  48. 48.
    Storn, R., Price, K.: Minimizing the real functions of the icec 1996 contest by differential evolution. In: Proceedings of the IEEE International Conference on Evolutionary Computation, Nagoya, Japan, pp. 842–844 (1996)Google Scholar
  49. 49.
    Szymanski, B., Desell, T., Varela, C.: The effect of heterogeneity on asynchronous panmictic genetic search. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Wasniewski, J. (eds.) PPAM 2007. LNCS, vol. 4967, pp. 457–468. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  50. 50.
    Team, D.:,
  51. 51.
    Team, D.: Optimal golomb ruler,
  52. 52.
    Team, G.: Greate internet mersenne prime search,
  53. 53.
    Tomassini, M.: Spatially Structured Evolutionary Algorithms. Springer, Heidelberg (2005)zbMATHGoogle Scholar
  54. 54.
    Trujillo, L., Olague, G.: Automated Design of Image Operators that Detect Interest Points, pp. 483–507. MIT Press, Cambridge (2008)Google Scholar
  55. 55.
    Vivas, A.K., Zinn, R., Andrews, P., Bailyn, C., Baltay, C., Coppi, P., Ellman, N., Girard, T., Rabinowitz, D., Schaefer, B., Shin, J., Snyder, J., Sofia, S., van Altena, W., Abad, C., Bongiovanni, A., Briceño, C., Bruzual, G., Della Prugna, F., Herrera, D., Magris, G., Mateu, J., Pacheco, R., Sánchez, G., Sánchez, G., Schenner, H., Stock, J., Vicente, B., Vieira, K., Ferrín, I., Hernandez, J., Gebhard, M., Honeycutt, R., Mufson, S., Musser, J., Rengstorf, A.: The QUEST RR Lyrae Survey: Confirmation of the Clump at 50 Kiloparsecs and Other Overdensities in the Outer Halo. The Astrophysical Journal Letters 554, L33–L36 (2001)CrossRefGoogle Scholar
  56. 56.
    Willman, B., Dalcanton, J.J., Martinez-Delgado, D., West, A.A., Blanton, M.R., Hogg, D.W., Barentine, J.C., Brewington, H.J., Harvanek, M., Kleinman, S.J., Krzesinski, J., Long, D., Neilsen Jr., E.H., Nitta, A., Snedden, S.A.: A New Milky Way Dwarf Galaxy in Ursa Major. The Astrophysical Journal Letters 626, L85–L88 (2005)CrossRefGoogle Scholar
  57. 57.
    Yanny, B., Newberg, H.J., Grebel, E.K., Kent, S., Odenkirchen, M., Rockosi, C.M., Schlegel, D., Subbarao, M., Brinkmann, J., Fukugita, M., Ivezic, Ž., Lamb, D.Q., Schneider, D.P., York, D.G.: A Low-Latitude Halo Stream around the Milky Way. The Astrophysical Journal 588, 824–841 (2003)CrossRefGoogle Scholar
  58. 58.
    Yanny, B., Newberg, H.J., Kent, S., Laurent-Muehleisen, S.A., Pier, J.R., Richards, G.T., Stoughton, C., Anderson Jr., J.E., Annis, J., Brinkmann, J., Chen, B., Csabai, I., Doi, M., Fukugita, M., Hennessy, G.S., Ivezić, Ž., Knapp, G.R., Lupton, R., Munn, J.A., Nash, T., Rockosi, C.M., Schneider, D.P., Smith, J.A., York, D.G.: Identification of A-colored Stars and Structure in the Halo of the Milky Way from Sloan Digital Sky Survey Commissioning Data. The Astrophysical Journal 540, 825–841 (2000)CrossRefGoogle Scholar
  59. 59.
    Zucker, D.B., Belokurov, V., Evans, N.W., Wilkinson, M.I., Irwin, M.J., Sivarani, T., Hodgkin, S., Bramich, D.M., Irwin, J.M., Gilmore, G., Willman, B., Vidrih, S., Fellhauer, M., Hewett, P.C., Beers, T.C., Bell, E.F., Grebel, E.K., Schneider, D.P., Newberg, H.J., Wyse, R.F.G., Rockosi, C.M., Yanny, B., Lupton, R., Smith, J.A., Barentine, J.C., Brewington, H., Brinkmann, J., Harvanek, M., Kleinman, S.J., Krzesinski, J., Long, D., Nitta, A., Snedden, S.A.: A New Milky Way Dwarf Satellite in Canes Venatici. The Astrophysical Journal Letters 643, L103–L106 (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Nate Cole
    • 1
  • Travis Desell
    • 1
  • Daniel Lombraña González
    • 2
  • Francisco Fernández de Vega
    • 2
  • Malik Magdon-Ismail
    • 1
  • Heidi Newberg
    • 1
  • Boleslaw Szymanski
    • 1
  • Carlos Varela
    • 1
  1. 1.Rensselaer Polytechnic Institute, Email: astro@cs.rpi.eduUSA
  2. 2.Centro Universitario de Mérida, Universidad de ExtremaduraMérida (Badajoz)Spain

Personalised recommendations