Information Systems Frontiers

, Volume 14, Issue 4, pp 925–951 | Cite as

GA-based cloud resource estimation for agent-based execution of bag-of-tasks applications

Article

Abstract

Executing bag-of-tasks applications in multiple Cloud environments while satisfying both consumers’ budgets and deadlines poses the following challenges: How many resources and how many hours should be allocated? What types of resources are required? How to coordinate the distributed execution of bag-of-tasks applications in resources composed from multiple Cloud providers?. This work proposes a genetic algorithm for estimating suboptimal sets of resources and an agent-based approach for executing bag-of-tasks applications simultaneously constrained by budgets and deadlines. Agents (endowed with distributed algorithms) compose resources and coordinate the execution of bag-of-tasks applications. Empirical results demonstrate that the genetic algorithm can autonomously estimate sets of resources to execute budget-constrained and deadline-constrained bag-of-tasks applications composed of more economical (but slower) resources in the presence of loose deadlines, and more powerful (but more expensive) resources in the presence of large budgets. Furthermore, agents can efficiently and successfully execute randomly generated bag-of-tasks applications in multi-Cloud environments.

Keywords

Cloud resource estimation Bag-of-tasks applications Cloud resource management Multiagent systems Genetic algorithms Cloud computing 

References

  1. Amazon EC2. (2011). http://aws.amazon.com/ec2/. Accessed 21 July 2011.
  2. Amazon EC2 FAQs. (2011). http://aws.amazon.com/ec2/faqs/. Accessed 21 July 2011.
  3. Amazon EC2 instance types. (2011). http://aws.amazon.com/ec2/instance-types/. Accessed 21 July 2011.
  4. Amazon EC2 pricing. (2011). http://aws.amazon.com/ec2/pricing/. Accessed 21 July 2011.
  5. Anglano, C., & Canonico, M. (2008). Scheduling algorithms for multiple Bag-of-Task applications on desktop Grids: a knowledge-free approach. In proceedings of the 22nd IEEE International Symposium on Parallel and Distributed Processing, 1–8.Google Scholar
  6. Bellifemine, F., Poggi, A., & Rimassa, G. (1999). JADE - A FIPA-compliant agent framework. In proceedings of the 4th International Conference and Exhibition on the Practical Application of Intelligent Agents and Multi-Agents, 97–108.Google Scholar
  7. Bonami, P., Kilinc, M., & Linderoth, J. (2009). Algorithms and Software for Convex Mixed Integer Nonlinear Programs. Technical Report #1664, Computer Sciences Department, University of Wisconsin-Madison.Google Scholar
  8. Buyya, R., Murshed, M. M., Abramson, D., & Venugopal, S. (2005). Scheduling parameter sweep applications on global Grids: a deadline and budget constrained cost-time optimization algorithm. Software-Practice and Experience, 35(5), 491–512.CrossRefGoogle Scholar
  9. Buyya, R., Yeo, C. S., Venugopal, S., Broberg, J., & Brandic, I. (2009). Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Generation Computer Systems, 25(6), 599–616.CrossRefGoogle Scholar
  10. Candeia, D., Araujo, R., Lopes, R., & Brasileiro, F. (2010). Investigating business-driven cloudburst schedulers for E-science Bag-of-Tasks applications. In proceedings of the 2010 IEEE Second International Conference on Cloud Computing Technology and Science, 343–350.Google Scholar
  11. Coello Coello, C. A. (2006). Evolutionary multi-objective optimization: A historical view of the field. IEEE Computational Intelligence Magazine, 1(1), 28–36.CrossRefGoogle Scholar
  12. Comin, M., Ferrari, C., & Guerra, C. (2004). Grid deployment of bioinformatics applications: a case study in protein similarity determination. Parallel Processing Letters, 14(2), 163–176.CrossRefGoogle Scholar
  13. Da Silva, F. A. B., Carvalho, S., Senger, H., Hruschka, E. R., & De Farias, C. R. G. (2004). Running data mining applications on the grid: A bag-of-tasks approach. In Lagana et al. (Eds.), Computational Science and Its Applications, LNCS 3044 (pp. 168–177). Heidelberg: Springer.Google Scholar
  14. De Jong, K. A., & Spears, W. M. (1989). Using genetic algorithms to solve NP-complete problems. In proceedings of the 3rd International Conference on Genetic algorithms, 124–132.Google Scholar
  15. Deb, K. (2001). Multi-objective optimization using evolutionary algorithms (1st ed.). Chichester: Wiley.Google Scholar
  16. Garey, M. R., & Johnson, D. S. (1990). Computers and intractability: A guide to the theory of NP-Completeness. New York: Freeman.Google Scholar
  17. Ghafoor, A., & Yang, J. (1993). A distributed heterogeneous supercomputing management system. Computer, 26(6), 78–86.CrossRefGoogle Scholar
  18. Grefenstette, J. J. (1986). Optimization of control parameters for genetic algorithms. IEEE Transactions on Systems, Man, and Cybernetics, 16(1), 122–128.CrossRefGoogle Scholar
  19. Gutierrez-Garcia, J. O., & Sim, K. M. (2010). Agent-based service composition in Cloud computing. In T.-H. Kim et al. (Eds.), GDC/CA 2010, CCIS 121 (pp. 1–10). Heidelberg: Springer.Google Scholar
  20. Jarvis, S. A., Spooner, D. P., Lim Choi Keung, H. N., Cao, J., Saini, S., & Nudd, G. R. (2006). Performance prediction and its use in parallel and distributed computing systems. Future Generation Computer Systems, 22(7), 745–754.CrossRefGoogle Scholar
  21. Jung, J. J. (2010). An evolutionary approach to query-sampling for heterogeneous systems. Expert Systems with Applications, 37(1), 226–232.CrossRefGoogle Scholar
  22. Kafil, M., & Ahmad, I. (1998). Optimal task assignment in heterogeneous distributed computing systems. IEEE Concurrency, 6(3), 42–51.CrossRefGoogle Scholar
  23. Konak, A., Coit, D., & Smith, A. (2006). Multi-objective optimization using genetic algorithms: a tutorial. Reliability Engineering and System Safety, 91(9), 992–1007.CrossRefGoogle Scholar
  24. Lee, Y. C., & Zomaya, A. Y. (2007). Practical scheduling of Bag-of-Tasks applications on Grids with dynamic resilience. IEEE Transactions on Computers, 56(6), 815–825.CrossRefGoogle Scholar
  25. Moulton, B. R. (1996). Bias in the consumer price index: what is the evidence? Journal of Economic Perspectives, American Economic Association, 10(4), 159–177.CrossRefGoogle Scholar
  26. Netto, M. A. S., & Buyya, R. (2009). Offer-based scheduling of deadline-constrained Bag-of-Tasks applications for utility computing systems. In proceedings of the 2009 IEEE International Symposium on Parallel and Distributed Processing, 1–11.Google Scholar
  27. O’Brien, A., Newhouse, S., & Darlington, J. (2004). Mapping of scientific workflow within the e-protein project to distributed resources. In proceedings of the UK e-Science All Hands Meeting, 404–409.Google Scholar
  28. Oliveira, M. C., Cirne, W., & De Azevedo-Marques, P. M. (2007). Towards applying content-based image retrieval in the clinical routine. Future Generation Computer Systems, 23(3), 466–474.CrossRefGoogle Scholar
  29. Oprescu, A. M., & Kielmann T. (2010). Bag-of-Tasks scheduling under budget constraints. In proceedings of the 2010 IEEE Second International Conference on Cloud Computing Technology and Science, 351–359.Google Scholar
  30. Rotstan, N., & Meffert, K. (2011). JGAP: java genetic algorithms. http://jgap.sourceforge.net/. Accessed 21 July 2011.
  31. Salehi, M. A., & Buyya, R. (2010). Adapting market-oriented scheduling policies for Cloud computing. In C.-H. Hsu et al. (Eds.), ICA3PP 2010, Part I, LNCS 6081 (pp. 351–362). Heidelberg: Springer.Google Scholar
  32. Silberstein, M., Sharov, A., Geiger, D., & Schuster, A. (2009). Gridbot: execution of bags of tasks in multiple grids. In Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, article 11, 12 pages.Google Scholar
  33. Silva, J. N., Veiga, L., & Ferreira, P. (2008) Heuristic for resources allocation on utility computing infrastructures. In proceedings of the 6th International Workshop on Middleware for Grid Computing, Article 9, p. 6Google Scholar
  34. Sim, K. M. (2009). Agent-based Cloud commerce. In proceedings of the IEEE International Conference on Industrial Engineering and Engineering Management, 717–721.Google Scholar
  35. Sim, K. M. (2010). Towards complex negotiation for cloud economy. In Chang et al. (Eds.), GPC 2010, LNCS 6104 (pp. 395–406). Heidelberg: Springer.Google Scholar
  36. Smith, R. G. (1980). The contract net protocol: high-level communication and control in a distributed problem solver. IEEE Transactions on Computers, 29(12), 1104–1113.CrossRefGoogle Scholar
  37. Smith, W., Foster, I. T., & Taylor, V. E. (1998). Predicting application run times using historical information. In D. G. Feitelson & L. Rudolph (Eds.), JSSPP’98, LNCS 1459 (pp. 122–142). Heidelberg: Springer.Google Scholar
  38. Sugavanam, P., Siegel, H. J., Maciejewski, A. A., Oltikar, M., Mehta, A., Pichel, R., et al. (2007). Robust static allocation of resources for independent tasks under makespan and dollar cost constraints. Journal of Parallel and Distributed Computing, 67(4), 400–416.CrossRefGoogle Scholar
  39. Sulistio, A., & Buyya, R. (2005). A time optimization algorithm for scheduling Bag-of-Task applications in auction-based proportional share systems. In proceedings of the 17th International Symposium on Computer Architecture on High Performance Computing, 235–242.Google Scholar
  40. Wooldridge, M. (2009). An introduction to multiagent systems (2nd ed.). Chichester: Wiley.Google Scholar
  41. Yu, J., Buyya, R., & Khong Tham, C. (2005). Cost-based scheduling of scientific workflow application on utility Grids. In proceedings of the 1st International Conference on e-Science and Grid Computing, 140–147.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Department of Information and CommunicationsGwangju Institute of Science and TechnologyGwangjuRepublic of Korea

Personalised recommendations