Cluster Computing

, Volume 17, Issue 4, pp 1185–1201

Scalable and effective peer-to-peer desktop grid system

Article
  • 211 Downloads

Abstract

We have designed a set of protocols that use peer-to-peer techniques to efficiently implement a distributed and decentralized desktop grid. Incoming jobs with different resource requirements are matched with system nodes through proximity in an N-dimensional Content-Addressable Network, where each resource type is represented as a distinct dimension. In this paper, we describe a comprehensive suite of techniques that cooperate to maximize throughput, and to ensure that load is balanced across all peers. We balance load induced by job executions through randomly generated virtual dimension values, which act to disaggregate clusters of nodes/jobs, and also by a job pushing mechanism based on an approximate global view of the system. We improve upon initial job assignments by using a job-stealing mechanism to overcome load imbalance caused by heterogeneity of nodes/jobs and stale load information. We also describe a set of optimizations that combine to reduce the system load created by the management of the underlying peer-to-peer system and the job-monitoring infrastructure. Unlike other systems, we can effectively support resource constraints of jobs during the course of load balancing since we simplify the problem of matchmaking through building a multi-dimensional resource space and mapping jobs and nodes to this space. We use extensive simulation results to show that the new techniques improve scalability, system throughput, and average response time.

Keywords

Peer-to-peer desktop grid Matchmaking Decentralized load balancing Resource constraints 

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.National Institute of Supercomputing and NetworkingKorea Institute of Science and Technology InformationDaejeonSouth Korea
  2. 2.School of Electrical and Computer EngineeringUlsan National Institute of Science and TechnologyUlsanSouth Korea
  3. 3.UMIACS and Department of Computer ScienceUniversity of MarylandCollege ParkUSA

Personalised recommendations