Abstract
Grid RPC applications, typically master-slave type of applications, often need to share large size of data among workers. For efficient and flexible data sharing among a master and workers, we have designed and developed a data management layer called OmniStorage. This paper enhances the OmniStorage functionality to accommodate several data transfer methods and to specify a hint for data sharing patterns, and develops a set of synthetic benchmarks based on data sharing patterns required by grid RPC applications to evaluate the performance and characteristics of each data transfer method. The performance evaluation and the hint help to select a suitable data transfer method, which improves the efficiency and also scalability of grid RPC applications that need to share large size of data among a master and workers.
This study was supported in part by MEXT KAKENHI (No. 17200002, 17.7324, 19024009), and by the Japan-France collaboration research program (SAKURA) through the JSPS. The study was performed in University of Tsukuba.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Nakada, H., Sato, M., Sekiguchi, S.: Design and Implementations of Ninf: towards a Global Computing Infrastructure. Future Generation Computing Systems 15(5-6), 649–658 (1999)
Arnold, D., Agrawal, S., Blackford, S., Dongarra, J., Miller, M., Seymour, K., Sagi, K., Shi, Z., Vadhiyar, S.: Users’ Guide to NetSolve V1.4.1. Innovative Computing Dept. Technical Report ICL-UT-02-05, University of Tennessee (June 2002)
Caron, E., Desprez, F.: Diet: A scalable toolbox to build network enabled servers on the grid. International Journal of High Performance Computing Applications 20(3), 335–352 (2006)
Sato, M., Boku, T., Takahashi, D.: OmniRPC: a Grid RPC system for Parallel Programming in Cluster and Grid Environment. In: Proceedings of the 3st International Symposium on Cluster Computing and the Grid, pp. 206–213 (2003)
Aida, Y., Nakajima, Y., Sato, M., Sakurai, T., Takahashi, D., Boku, T.: Performance Improvement by Data Management Layer in a Grid RPC System. In: Proceedings of the First International Conference on Grid and Pervasive Computing, pp. 324–335 (2006)
Beck, M., Arnold, D., Bassi, A., Berman, F., Casanova, H., Dongarra, J., Moore, T., Obertelli, G., Plank, J., Swany, M., Vadhiyar, S., Wolski, R.: Middleware for the use of storage in communication. Parallel Comput. 28(12), 1773–1787 (2002)
Del-Fabbro, B., Laiymani, D., Nicod, J.-M., Philippe, L.: Dtm: a service for managing data persistency and data replication in network-enabled server environments: Research articles. Concurr. Comput.: Pract. Exper. 19(16), 2125–2140 (2007)
Nakajima, Y., Sato, M., et al.: Implementation and performance evaluation of CONFLEX-G: grid-enabled molecular conformational space search program with OmniRPC. In: Proceedings of the 18th Annual International Conference on Supercomputing, pp. 154–163 (2004)
BitTorrent, http://www.bittorrent.com/
Tatebe, O., Morita, Y., Matsuoka, S., Soda, N., Sekiguchi, S.: Grid datafarm architecture for petascale data intensive computing. In: Proceedings of 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid, pp. 102–109 (2002)
Qiu, D., Srikant, R.: Modeling and performance analysis of bittorrent-like peer-to-peer networks. In: Proceedings of The 2004 conference on Applications, technologies, architectures, and protocols for computer communications, pp. 367–378 (2004)
Bharambe, A.R., Herley, C., Padmanabhan, V.N.: Some observations on bittorrent performance. In: Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, pp. 398–399 (2005)
Ogura, S., Matsuoka, S., Nakada, H.: Evaluation of the inter-cluster data transfer on Grid environment. In: Protocols of 3rd International Symposium on Cluster Computing and the Grid, pp. 374–381 (2003)
Del-Fabbro, B., Laiymani, D., Nicod, J.-M., Philippe, L.: Data management in grid applications providers. In: The First International Conference on Distributed Frameworks for Multimedia Applications (DFMA 2005), pp. 315–322 (2005)
Bassi, A., Beck, M., Moore, T., Plank, J.S., Swany, M., Wolski, R., Fagg, G.: The Internet Backplane Protocol: a study in resource sharing. Future Generation Computer Systems 19(4), 551–561 (2003)
Tanimura, Y., Nakada, H., Tanaka, Y., Sekiguchi, S.: Design and Implementation of Distributed Task Sequencing on GridRPC. In: Proceedings of the 6th IEEE International Conference on Computer and Information Technology (2006)
Bent, J., Thain, D., Arpaci-Dusseau, A.C., Arpaci-Dusseau, R.H., Livny, M.: Explicit control a batch-aware distributed file system. In: Proceedings of the 1st Symposium on Networked Systems Design and Implementation, p. 27 (2004)
Kosar, T., Livny, M.: Stork: Making data placement a first class citizen in the grid. In: Proceedings of the 24th IEEE International Conference on Distributed Computing Systems (ICDCS 2004), pp. 342–349 (2004)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nakajima, Y., Aida, Y., Sato, M., Tatebe, O. (2008). Performance Evaluation of Data Management Layer by Data Sharing Patterns for Grid RPC Applications. In: Luque, E., Margalef, T., Benítez, D. (eds) Euro-Par 2008 – Parallel Processing. Euro-Par 2008. Lecture Notes in Computer Science, vol 5168. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-85451-7_59
Download citation
DOI: https://doi.org/10.1007/978-3-540-85451-7_59
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-85450-0
Online ISBN: 978-3-540-85451-7
eBook Packages: Computer ScienceComputer Science (R0)