Advertisement

MapReduce Programming Model for .NET-Based Cloud Computing

  • Chao Jin
  • Rajkumar Buyya
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5704)

Abstract

Recently many large scale computer systems are built in order to meet the high storage and processing demands of compute and data-intensive applications. MapReduce is one of the most popular programming models designed to support the development of such applications. It was initially created by Google for simplifying the development of large scale web search applications in data centers and has been proposed to form the basis of a ‘Data center computer’ This paper presents a realization of MapReduce for .NET-based data centers, including the programming model and the runtime system. The design and implementation of MapReduce.NET are described and its performance evaluation is presented.

Keywords

Cloud Computing Runtime System Reduce Task MapReduce Program Model Sort Phase 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    McNabb, A.W., Monson, C.K., Seppi, K.D.: Parallel PSO Using MapReduce. In: Proc. of the Congress on Evolutionary Computation (2007)Google Scholar
  2. 2.
  3. 3.
    Jin, C., Vecchiola, C., Buyya, R.: MRPGA: An Extension of MapReduce for Parallelizing Genetic Algorithms. In: Proc. of 4th International Conference on e-Science (2008)Google Scholar
  4. 4.
    Ranger, C., Raghuraman, R., Penmetsa, A., Bradski, G., Kozyrakis, C.: Evaluating MapReduce for Multi-core and Multiprocessor Systems. In: Proc. of the 13th Intl. Symposium on High-Performance Computer Architecture (2007)Google Scholar
  5. 5.
    Patterson, D.A.: Technical perspective: the data center is the computer. Communications of the ACM 51(1), 105 (2008)CrossRefGoogle Scholar
  6. 6.
    Gregor, D., Lumsdaine, A.: Design and Implementation of a High-Performance MPI for C\(\sharp\) and the Common Language Infrastructure. In: Proc. of the 13th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (2008)Google Scholar
  7. 7.
    Yang, H.C., Dasdan, A., Hsiao, R.L., Stott Parker, D.: Map-Reduce-Merge: simplified relational data processing on large clusters. In: Proc. of SIGMOD (2007)Google Scholar
  8. 8.
    Dean, J., Ghemawat, S.: MapReduce: Simplified Data Processing on Large Clusters. In: Proc. of the 6th Symposium on Operating System Design and Implementation (2004)Google Scholar
  9. 9.
    Varia, J.: Cloud Architectures. White Paper of Amazon (2008), jineshvaria.s3.amazonaws.com/public/cloudarchitectures-varia.pdf
  10. 10.
    Isard, M., Budiu, M., Yu, Y., Birrell, A., Fetterly, D.: Dryad: Distributed Data-Parallel Programs from Sequential Building Blocks. In: Proc. of European Conference on Computer Systems, EuroSys (2007)Google Scholar
  11. 11.
    Kruijf, M., Sankaralingam, K.: MapReduce for the Cell B.E. Architecture. TR1625, Technical Report, The University of Wisconsin-Madison (2007)Google Scholar
  12. 12.
    Buyya, R., Yeo, C.S., Venugopal, S., Broberg, J., Brandic, I.: Cloud Computing and Emerging IT Platforms: Vision, Hype, and Reality for Delivering Computing as the 5th Utility. Future Generation Computer Systems 25(6), 599–616 (2009)CrossRefGoogle Scholar
  13. 13.
    Bryant, R.E.: Data-Intensive Supercomputing: The Case for DISC. CMU-CS-07-128, Technical Report, Carnegie Mellon University (2007)Google Scholar
  14. 14.
    Chen, S., Schlosser, S.W.: Map-Reduce Meets Wider Varieties of Applications. IRP-TR-08-05, Technical Report, Intel Research Pittsburgh (2008)Google Scholar
  15. 15.
    Hey, T., Trefethen, A.: The data deluge: an e-Science perspective. In: Grid Computing: Making the Global Infrastructure a Reality, pp. 809–824 (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Chao Jin
    • 1
  • Rajkumar Buyya
    • 1
  1. 1.Grid Computing and Distributed Systems (GRIDS) Laboratory Department of Computer Science and Software EngineeringThe University of MelbourneAustralia

Personalised recommendations