A Multi-agent System Approach to Load-Balancing and Resource Allocation for Distributed Computing
Conference paper
First Online:
Abstract
In this research we use a decentralized computing approach to allocate and schedule tasks on a massively distributed grid. Using emergent properties of multi-agent systems, the algorithm dynamically creates and dissociates clusters to serve the changing resource demands of a global task queue. The algorithm is compared to a standard first-in first-out (FIFO) scheduling algorithm. Experiments done on a simulator show that the distributed resource allocation protocol (dRAP) algorithm outperforms the FIFO scheduling algorithm on time to empty queue, average waiting time, and CPU utilization. Such a decentralized computing approach holds promise for massively distributed processing scenarios like SETI@home and Google MapReduce.
Keywords
Global Cost Decentralize System Timing Metrics Process Queue Centralize Monitor
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Notes
Acknowledgements
We thank Dr. Dorian Arnold for fruitful discussions.
References
- 1.Anderson D, Cobb J, Korpela E, Lebofsky M, Werthimer D (2002) SETI@ home: an experiment in public-resource computing. Commun ACM 45(11):61CrossRefGoogle Scholar
- 2.Banerjee S (2009) An immune system inspired approach to automated program verification. arXiv preprint arXiv:0905.2649. http://arxiv.org/abs/0905.2649
- 3.Banerjee S (2013) Scaling in the immune system. Ph.D. thesis, University of New, MexicoGoogle Scholar
- 4.Banerjee S, Moses M (2009) A hybrid agent based and differential equation model of body size effects on pathogen replication and immune system response. In: Timmis J (ed) The 8th international conference on artificial immune systems (ICARIS). Lecture notes in computer science. Springer, Berlin, pp 14–18. http://www.springerlink.com/content/b786g874642q2j37/ CrossRefGoogle Scholar
- 5.Banerjee S, Moses M (2010) Modular radar: an immune system inspired search and response strategy for distributed systems. In: Artificial immune systems. Springer, Berlin, pp 116–129CrossRefGoogle Scholar
- 6.Banerjee S, Moses M (2010) Scale invariance of immune system response rates and times: perspectives on immune system architecture and implications for artificial immune systems. Swarm Intell 4(4):301–318. http://www.springerlink.com/content/w67714j72448633l/ CrossRefGoogle Scholar
- 7.Banerjee S, Levin D, Moses M, Koster F, Forrest S (2011) The value of inflammatory signals in adaptive immune responses. In: Artificial immune systems. Springer, Berlin, pp 1–14. http://www.springerlink.com/index/U634HJ83W62W5383.pdf CrossRefGoogle Scholar
- 8.Cook M (2004) Universality in elementary cellular automata. Complex Syst 15(1):1–40MathSciNetMATHGoogle Scholar
- 9.Dean J, Sanjay G (2008) MapReduce: simplified data processing on large clusters. Commun ACM 51(1):107–113CrossRefGoogle Scholar
- 10.Gardner M (1970) Mathematical games: the fantastic combinations of John Conway’s new solitaire game ‘Life’. Sci Am 223(4):120–123CrossRefGoogle Scholar
- 11.Hecker JP, Moses ME (2013) An evolutionary approach for robust adaptation of robot behavior to sensor error. In: Proceedings of the 15th annual conference companion on genetic and evolutionary computation (GECCO ’13 companion). ACM, New York, NY, pp 1437–1444. http://doi.acm.org/10.1145/2464576.2482724 Google Scholar
- 12.Hecker JP, Moses ME (2015) Beyond pheromones: evolving error-tolerant, flexible, and scalable ant-inspired robot swarms. Swarm Intell 9(1):43–70Google Scholar
- 13.Hecker J, Wu A, Herweg J, Sciortino J Jr (2008) Team-based resource allocation using a decentralized social decision-making paradigm. In: Proceedings of SPIE, vol 6964, p 696409Google Scholar
- 14.Hecker JP, Letendre K, Stolleis K, Washington D, Moses ME (2012) Formica ex machina: ant swarm foraging from physical to virtual and back again. In: Swarm intelligence: 8th international conference, ANTS 2012. Springer, Berlin, pp 252–259CrossRefGoogle Scholar
- 15.Hecker JP, Stolleis K, Swenson B, Letendre K, Moses ME (2013) Evolving error tolerance in biologically-inspired iAnt robots. In: Proceedings of the twelfth European conference on the synthesis and simulation of living systems (Advances in artificial life, ECAL 2013). MIT Press, Cambridge, MA, pp 1025–1032Google Scholar
- 16.Jacob C, Litorco J, Lee L (2004) Immunity through swarms: agent-based simulations of the human immune system. In: Artificial immune systems. Lecture notes in computer science. Springer, Berlin, pp 400–412CrossRefGoogle Scholar
- 17.Krawczyk S, Bubendorfer K (2008) Grid resource allocation: allocation mechanisms and utilisation patterns. In: Proceedings of the sixth Australasian workshop on grid computing and e-research-Volume 82. Australian Computer Society Inc, Darlinghurst, pp 73–81Google Scholar
- 18.Luke S, Cioffi-Revilla C, Panait L, Sullivan K (2004) Mason: a new multi-agent simulation toolkit. In: Proceedings of the 2004 SwarmFest workshop, vol 8Google Scholar
- 19.Moses M, Banerjee S (2011) Biologically inspired design principles for scalable, robust, adaptive, decentralized search and automated response (radar). In: 2011 IEEE symposium on artificial life (ALIFE), pp 30–37Google Scholar
- 20.Yoo A, Jette M, Grondona M (2003) SLURM: simple linux utility for resource management. In: Job scheduling strategies for parallel processing. Lecture notes in computer science. Springer, Berlin, pp 44–60CrossRefGoogle Scholar
- 21.Zhou S (1992) LSF: load sharing in large-scale heterogeneous distributed systems. In: Proceedings of workshop on cluster computing, pp 1995–1996Google Scholar
Copyright information
© Springer International Publishing Switzerland 2017