Advertisement

ComputErl – Erlang-Based Framework for Many Task Computing

  • Michał Ptaszek
  • Maciej Malawski
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6546)

Abstract

This paper shows how Erlang programming language can be used for creating a framework for distributing and coordinating the execution of many task computing problems. The goals of the proposed solution are (1) to disperse the computation into many tasks, (2) to support multiple well-known computation models (such as master-worker, map-reduce, pipeline), (3) to exploit the advantages of Erlang for developing an efficient and scalable framework and (4) to build a system that can scale from small to large number of tasks with minimum effort. We present the results of work on designing, implementing and testing ComputErl framework. The preliminary experiments with benchmarks as well as real scientific applications show promising scalability on a computing cluster.

Keywords

many task computing Erlang grid distributed computing parallelism 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Cesarini, F., Thompson, S.: Erlang Programming. O’Reilly Media, Sebastopol (2009)zbMATHGoogle Scholar
  2. 2.
    DeCandia, G., Hastorun, D., Jampani, M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., Vogels, W.: Dynamo: Amazon’s highly available key-value store. In: SOSP 2007: Proceedings of twenty-first ACM SIGOPS Symposium on Operating Systems Principles, vol. 41, pp. 205–220. ACM, New York (2007)CrossRefGoogle Scholar
  3. 3.
    Foster, I.: Many Tasks Computing: What’s in a Name? (July 2008)Google Scholar
  4. 4.
    Wilde, M., Foster, I., Iskra, K., Beckman, P., Zhang, Z., Espinosa, A., Hategan, M., Clifford, B., Raicu, I.: Parallel scripting for applications at the petascale and beyond. Computer 42(11), 50–60 (2009)CrossRefGoogle Scholar
  5. 5.
    AB Ericsson: OTP Design Principles User’s Guide (February 2010)Google Scholar
  6. 6.
    Foster, I.: Globus toolkit version 4: Software for service-oriented systems. In: Jin, H., Reed, D., Jiang, W. (eds.) NPC 2005. LNCS, vol. 3779, pp. 2–13. Springer, Heidelberg (2005), http://dx.doi.org/10.1007/11577188_2 CrossRefGoogle Scholar
  7. 7.
    Mościcki, J.T.: Diane - distributed analysis environment for grid-enabled simulation and analysis of physics data. In: Nuclear Science Symposium Conference Record, vol. 3, pp. 1617–1620. IEEE, Los Alamitos (2003)Google Scholar
  8. 8.
    Mościcki, J.T., Brochu, F., Ebke, J., Egede, U., Elmsheuser, J., Harrison, K., Jones, R.W.L., Lee, H.C., Liko, D., Maier, A.: Ganga: a tool for computational-task management and easy access to grid resources. Computer Physics Communications (June 2009)Google Scholar
  9. 9.
    Cole, M.: Algorithmic Skeletons: Structured Management of Parallel Computation. MIT Press, Pitman (1989)zbMATHGoogle Scholar
  10. 10.
    Shao, G., Berman, F., Wolski, R.: Master/slave computing on the grid. In: Heterogeneous Computing Workshop, pp. 3–16 (2000)Google Scholar
  11. 11.
    Dean, J., Ghemawat, S.: Mapreduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)CrossRefGoogle Scholar
  12. 12.
    Bryliński, M., Prymula, K., Jurkowski, W., Kochańczyk, M., Stawowczyk, E., Konieczny, L., Roterman, I.: Prediction of functional sites based on the fuzzy oil drop model. PLoS Comput. Biol. 3(5), e94 (2007)CrossRefGoogle Scholar
  13. 13.
    Massie, M.L., Chun, B.N., Culler, D.E.: The Ganglia Distributed Monitoring System: Design, Implementation, and Experience. Parallel Computing 30(7) (July 2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Michał Ptaszek
    • 1
    • 2
  • Maciej Malawski
    • 1
  1. 1.Institute of Computer Science AGHKrakówPoland
  2. 2.Erlang Solutions Ltd.LondonUnited Kingdom

Personalised recommendations