Journal of Grid Computing

, Volume 10, Issue 3, pp 553–577 | Cite as

Cost-Aware and SLO-Fulfilling Software as a Service

  • Oliver NiehörsterEmail author
  • André Brinkmann
  • Axel Keller
  • Christoph Kleineweber
  • Jens Krüger
  • Jens Simon


Virtualization technology makes data centers more dynamic and easier to administrate. Today, cloud providers offer customers access to complex applications running on virtualized hardware. Nevertheless, big virtualized data centers become stochastic environments and the simplification on the user side leads to many challenges for the provider. He has to find cost-efficient configurations and has to deal with dynamic environments to ensure service level objectives (SLOs). We introduce a software solution that reduces the degree of human intervention to manage clouds. It is designed as a multi-agent system (MAS) and placed on top of the Infrastructure as a Service (IaaS) layer. Worker agents allocate resources, configure applications, check the feasibility of requests, and generate cost estimates. They are equipped with application specific knowledge allowing it to estimate the type and number of necessary resources. During runtime, a worker agent monitors the job and adapts its resources to ensure the specified quality of service—even in noisy clouds where the job instances are influenced by other jobs. They interact with a scheduler agent, which takes care of limited resources and does a cost-aware scheduling by assigning jobs to times with low costs. The whole architecture is self-optimizing and able to use public or private clouds. Building a private cloud needs to face the challenge to find a mapping of virtual machines (VMs) to hosts. We present a rule-based mapping algorithm for VMs. It offers an interface where policies can be defined and combined in a generic way. The algorithm performs the initial mapping at request time as well as a remapping during runtime. It deals with policy and infrastructure changes. An energy-aware scheduler and the availability of cheap resources provided by a spot market are analyzed. We evaluated our approach by building up an SaaS stack, which assigns resources in consideration of an energy function and that ensures SLOs of two different applications, a brokerage system and a high-performance computing software. Experiments were done on a real cloud system and by simulations.


Cloud computing Autonomic resource management SLO-fulfilling Cost-aware 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Niehörster, O., Brinkmann, A., Fels, G., Krüger, J., Simon, J.: Enforcing SLAs in scientific clouds. In: Proceedings of the IEEE International Conference on Cluster Computing (CLUSTER), Heraklion, Greece (2010)Google Scholar
  2. 2.
    Niehörster, O., Keller, A., Brinkmann, A. An energy-aware SaaS stack. In: Proceedings of the 19th IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS) (2011)Google Scholar
  3. 3.
    Kleineweber, C., Keller, A., Niehörster, O., Brinkmann, A.: Rule based mapping of virtual machines in clouds. In: Proceedings of the 19th Euromicro International Conference on Parallel, Distributed and Network-Based Computing (PDP). Cyprus (2011)Google Scholar
  4. 4.
    VMware: VMware Distributed Resource Scheduler (DRS). Data sheet [Online]. Available: Accessed 8 July 2010
  5. 5.
    RedHat: RedHat Enterprise Virtualization: System Scheduler. Data sheet [Online]. Available: Accessed 8 July 2010
  6. 6.
    Platform: Platform VM Orchestrator. [Online]. Available: Accessed 8 July 2010
  7. 7.
    Clark, C., Fraser, K., Hand, S., Hansen, J.G., Jul, E., Limpach, C., Pratt, I., Warfield, A.: Live migration of virtual machines. In: Proceedings of the 2nd Symposium on Networked Systems Design & Implementation (NSDIS). USENIX Association, Berkeley (2005)Google Scholar
  8. 8.
    Nurmi, D., Wolski, R., Grzegorczyk, C., Obertelli, G., Soman, S., Youseff, L., Zagorodnov, D.: The Eucalyptus open-source cloud-computing system. In: Proceedings of the 9th IEEE Inter. Symposium on Cluster Computing and the Grid (CCGrid). Shanghai, China (2009)Google Scholar
  9. 9.
    Sotomayor, B., Montero, R.S., Llorente, I.M., Foster, I.: Virtual infrastructure management in private and hybrid clouds. IEEE Internet Computing 13(5), 14–22 (2009)CrossRefGoogle Scholar
  10. 10.
    Keahey, K., Foster, I., Freeman, T., Zhang, X.: Virtual workspaces: achieving quality of service and quality of life in the Grid. Sci. Program. J. 13(4), 265–275 (2005)Google Scholar
  11. 11.
    OpenStack Cloud Software: OpenStack Website. [Online]. Available: Accessed 9 June 2012
  12. 12.
    Reservoir: Resources and Services Virtualization without Barriers. [Online]. Available: Accessed 8 July 2010
  13. 13.
    Sotomayor, B., Keahey, K., Foster, I.: Combining batch execution and leasing using virtual machines. In: Proceedings of the 17th International symposium on High Performance Distributed Computing, pp. 87–96. ACM, New York (2008)CrossRefGoogle Scholar
  14. 14.
    Cardosa, M., Korupolu, M.R., Singh, A.: Shares and utilities based power consolidation in virtualized server environments. In: Proceedings of the 11th IFIP/IEEE International Conference on Symposium on Integrated Network Management (IM). IEEE Press, Piscataway (2009)Google Scholar
  15. 15.
    Calheiros, R.N., Buyya, R., De Rose, C.A.F.: A heuristic for mapping virtual machines and links in emulation testbeds. In: Proceedings of the 9th International Conference on Parallel Processing (ICPP), pp. 518–525. IEEE Computer Society, Washington, DC (2009)Google Scholar
  16. 16.
    Verma, A., Ahuja, P., Neogi, A.: pMapper: power and migration cost aware application placement in virtualized systems. In: Proceedings of the 9th ACM/IFIP/USENIX International Conference on Middleware. Springer, New York (2008)Google Scholar
  17. 17.
    Kusic, D., Kephart, J.O., Hanson, J.E., Kandasamy, N., Jiang, G.: Power and performance management of virtualized computing environments via lookahead control. Cluster Comput. 12(1), 1–15 (2009)CrossRefGoogle Scholar
  18. 18.
    Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach, 2nd edn. Prentice-Hall, Englewood Cliffs (2003)Google Scholar
  19. 19.
    Shoham, Y., Leyton-Brown, K.: Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations. Cambridge University Press, Cambridge (2008)CrossRefGoogle Scholar
  20. 20.
    Cao, J., Jarvis, S.A., Saini, S., Kerbyson, D.J., Nudd, G.R.: ARMS: an agent-based resource management system for Grid computing. Sci. Program. 10(2), 135–148 (2002)Google Scholar
  21. 21.
    Frey, J., Tannenbaum, T., Livny, M., Foster, I., Tuecke, S.: Condor-G: a computation management agent for multi-institutional Grids. Cluster Comput. 5(3), 237–246 (2002)CrossRefGoogle Scholar
  22. 22.
    Ouelhadj, D., Garibaldi, J., MacLaren, J., Sakellariou, R., Krishnakumar, K., Meisels, A.: A multi-agent infrastructure and a service level agreement negotiation protocol for robust scheduling in Grid computing. In: Advances in Grid Computing—EGC, vol. 3470, pp. 651–660. Springer, Berlin (2005)Google Scholar
  23. 23.
    Amoonand, M., Mowafey, M., Altameem, T.: A multiagent-based system for scheduling jobs in computational Grids. Int. J. Artif. Intell. Mach. Learn. (AIML) 9(II), 19–27 (2009)Google Scholar
  24. 24.
    Manvi, S.S., Birje, M.N., Prasad, B.: An agent-based resource allocation model for Grid computing. In: Proceedings of the IEEE International Conference on Services Computing (SCC). IEEE Computer Society, Los Alamitos (2005)Google Scholar
  25. 25.
    Burge, J., Ranganathan, P., Wiener, J.L.: Cost-aware scheduling for heterogeneous enterprise machines (CASH’EM). In: Proceedings of the 1st International Workshop on Green Computing (GreenCom) (2007)Google Scholar
  26. 26.
    Verboven, S., Vanmechelen, K., Broeckhove, J.: Multiplexing low and high QoS workloads in virtual environments. In: Proceedings of the 15th Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP) (2010)Google Scholar
  27. 27.
    Wang, Z., Chen, Y., Gmach, D., Singhal, S., Watson, B.J., Rivera, W., Zhu, X., Hyser, C.D.: AppRAISE: application-level performance management in virtualized server environments. In: IEEE Transactions on Network and Service Management, pp. 240–254 (2009)Google Scholar
  28. 28.
    Rice University: RUBiS: Rice University Bidding System. [Online]. Available: Accessed 20 January 2011
  29. 29.
    Padala, P., Shin, K.G., Zhu, X., Uysal, M., Wang, Z., Singhal, S., Merchant, A., Salem, K.: Adaptive control of virtualized resources in utility computing environments. In: Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems (2007)Google Scholar
  30. 30.
    Transaction Processing Performance Council: TPC Benchmark W (TPC-W). [Online]. Available: Accessed 20 January 2011
  31. 31.
    Xu, J., Zhao, M., Fortes, J., Carpenter, R., Yousif, M.: On the use of fuzzy modeling in virtualized data center management. In: Proceedings of the 4th International Conference on Autonomic Computing (ICAC). IEEE Computer Society, Washington, DC (2007)Google Scholar
  32. 32.
    Zhu, X., Young, D., Watson, B.J., Wang, Z., Rolia, J., Singhal, S., Mckee, B., Hyser, C., Gmach, D., Gardner, R., Christian, T., Cherkasova, L.: 1000 islands: an integrated approach to resource management for virtualized data centers. Cluster Comput. 12, 45–57 (2009)CrossRefGoogle Scholar
  33. 33.
    Lim, H.C., Babu, S., Chase, J.S., Parekh, S.S.: Automated control in cloud computing: challenges and opportunities. In: Proceedings of the 1st Workshop on Automated Control for Datacenters and clouds (ACDC) (2009)Google Scholar
  34. 34.
    Rao, J., Bu, X., Xu, C.-Z., Wang, L., Yin, G.: VCONF: a reinforcement learning approach to virtual machines auto-configuration. In: Proceedings of the 6th International Conference on Autonomic Computing (ICAC) (2009)Google Scholar
  35. 35.
    Transaction Processing Performance Council: TPC Benchmark C (TPC-C). [Online]. Available: Accessed 20 January 2011
  36. 36.
    Park, S.-M., Humphrey, M.: Self-tuning virtual machines for predictable eScience. In: Proceedings of the 9th IEEE Inter. Symposium on Cluster Computing and the Grid (CCGrid) (2009)Google Scholar
  37. 37.
    Koh, Y., Knauerhase, R., Brett, P., Bowman, M., Wen, Z., Pu, C.: An analysis of performance interference effects in virtual environments. In: Proceedings of the IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) (2007)Google Scholar
  38. 38.
    Pu, X., Liu, L., Mei, Y., Sivathanu, S., Koh, Y., Pu, C.: Understanding performance interference of I/O workload in virtualized cloud environments. In: Proceedings of the 3rd IEEE International Conference on Cloud Computing, pp. 51–58. Washington, DC (2010)Google Scholar
  39. 39.
    Patterson, D.A., Hennesy, J.L.: Computer Organization and Design: The Hardware/Software Interface, Ch. 4, pp. 240–253. Morgan Kaufmann, San Mateo (2005)Google Scholar
  40. 40.
    Magklis, G., Scott, M.L., Semeraro, G., Albonesi, D.H., Dropsho, S.: Profile-based dynamic voltage and frequency scaling for a multiple clock domain microprocessor. IEEE Micro 23(6), 62–68 (2003)CrossRefGoogle Scholar
  41. 41.
    Reed, D.A., Mendes, C.L., da Lu, C.: Application tuning and adaption. In: Foster, I., Kesselman, C. (eds.) The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann, San Mateo (2003)Google Scholar
  42. 42.
    Krishnaswamy, S., Loke, S.W., Zaslavsky, A.: Estimating computation times of data-intensive applications. IEEE Distrib. Syst. Online (DS Online) 5(4), 1–12 (2004)CrossRefGoogle Scholar
  43. 43.
    Zhu, Y.: Multivariable System Identification for Process Control. Elsevier Science and Technology (1994)Google Scholar
  44. 44.
    Intel: Intelligent Platform Management Interface (IPMI). [Online]. Available: Accessed 9 March 2011
  45. 45.
    Hess, B., Kutzner, C., van der Spoel, D., Lindahl, E.: GROMACS 4: algorithms for highly efficient, load-balanced, and scalable molecular simulation. J. Chem. Theory Comput. 4, 435–447 (2008)CrossRefGoogle Scholar
  46. 46.
    European Energy Exchange AG (EEX): Spot Market. [Online]. Available: Accessed 8 March 2011
  47. 47.
    Yi, S., Kondo, D., Andrzejak, A.: Reducing costs of spot instances via checkpointing in the Amazon elastic compute cloud. In: Proceedings of the IEEE International Conference on Cloud Computing, pp. 236–243. IEEE Computer Society, Los Alamitos (2010)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2012

Authors and Affiliations

  • Oliver Niehörster
    • 1
    Email author
  • André Brinkmann
    • 2
  • Axel Keller
    • 1
  • Christoph Kleineweber
    • 1
  • Jens Krüger
    • 3
  • Jens Simon
    • 1
  1. 1.Paderborn Center for Parallel ComputingUniversität PaderbornPaderbornGermany
  2. 2.Johannes Gutenberg-Universität MainzMainzGermany
  3. 3.Applied Bioinformatics GroupUniversität TübingenTübingenGermany

Personalised recommendations