On Fully Decentralized Resource Discovery in Grid Environments
Computational grids provide mechanisms for sharing and accessing large and heterogeneous collections of remote resources such as computers, online instruments, storage space, data, and applications. Resources are identified based on a set of desired attributes. Resource attributes have various degrees of dynamism, from mostly static attributes, like operating system version, to highly dynamic ones, like network bandwidth or CPU load.
In this paper we propose a peer-to-peer architecture for resource discovery in a large and dynamic collection of resources. We evaluate a set of request-forwarding algorithms in a fully decentralized architecture, designed to accommodate heterogeneity (in both sharing policies and resource types) and dynamism. For this, we build a testbed that models two usage characteristics: (1) resource distribution on peers, that varies in the number and the frequency of shared resources; and (2) various requests patterns for resources. We analyzed our resource discovery mechanisms on up to 5000 peers, where each peer provides information about at least one resource. We learned that a decentralized approach is not only desirable from administrative reasons, but it is also supported by promising performance results. Our results also allow us to characterize the correlation between resource discovery performance and sharing characteristics.
KeywordsUser Request Resource Distribution Resource Discovery Forwarding Algorithm Resource Frequency
Unable to display preview. Download preview PDF.
- 1.Adar, E., and Huberman, B. Free riding on gnutella. First Monday 5, 10 (2000).Google Scholar
- 2.Clarke, I., Sandberg, O., Wiley, B., and Hong, T. Freenet: A distributed anonymous information storage and retrieval system. In Workshop on Design Issues in Anonymity and Unobservability (2000).Google Scholar
- 3.Czajkowski, K., Fitzgerald, S., Foster, I., and Kesselman, C. Grid information services for distributed resource sharing. In 10th IEEE Symposium on High Performance Distributed Computing (2001).Google Scholar
- 4.Doar, M. A better model for generating test networks. In IEEE Global Internet (1996), pp. 86–93.Google Scholar
- 5.Foster, I., and Kesselman, C, Eds. The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann, 1999.Google Scholar
- 6.Foster, I., Kesselman, C., and Tuecke, S. The anatomy of the grid: Enabling scalable virtual organizations. International Journal on Supercomputing Applications (2001).Google Scholar
- 7.Plaxton, C. G., Rajaraman, R., and Richa, A. W. Accessing nearby copies of replicated objects in a distributed environment. In ACM Symposium on Parallel Algorithms and Architectures (1997).Google Scholar
- 8.Raman, R., Livny, M., and Solomon, M. Matchmaking: Distributed resource management for high throughput computing. In 7th IEEE International Symposium on High Performance Distributed Computing (1998).Google Scholar
- 9.Ratnasamy, S., Francis, P., Handley, M., Karp, R., and Shenker, S. A scalable content addressable network. In ACM SIGCOMM (2001).Google Scholar
- 10.Ripeanu, M. Peer-to-peer architecture case study: Gnutella network. In International Conference on Peer-to-peer Computing (2001).Google Scholar
- 11.Song, H. J., Liu, X., Jakobsen, D., Bhagwan, R., Zhang, X., Taura, K., and Chien, A. A. The microgrid: a scientific tool for modeling computational grids. In Supercomputing (2000).Google Scholar
- 12.Stoica, I., Morris, R., Karger, D., Kaashoek, M., and Balakrishnan, H. Chord: A scalable peer-to-peer lookup service for internet applications. In ACM SIGCOMM (2001).Google Scholar
- 13.VAN Steen, M., Homburg, P., and Tanenbaum, A. Globe: Awide-area distributed system. IEEE Concurrency (1999), 70–78.Google Scholar
- 14.Zhao, B., Kubiatowicz, J., and Joseph, A. Tapestry: An infrastructure for fault-resilient wide-area location and routing. Tech. Rep. UCB//CSD-01-1141, U. C. Berkeley, 2001.Google Scholar