Cluster Computing

, Volume 20, Issue 1, pp 909–924 | Cite as

An approach for scaling cloud resource management

  • Dan C. Marinescu
  • Ashkan PayaEmail author
  • John P. Morrison
  • Stephen Olariu


Given its current development trajectory, the complexity of cloud computing ecosystems are evolving to where traditional resource management strategies will struggle to remain fit for purpose. These strategies have to cope with ever-increasing numbers of heterogeneous resources, a proliferation of new services, and a growing user-base with diverse and specialized requirements. This growth not only significantly increases the number of parameters needed to make good decisions, it increases the time needed to take these decisions. Consequently, traditional resource management systems are increasingly prone to poor decisions making. Devolving resources management decisions to the local environment of that resource can dramatically increase the speed of decisions making; moreover, the cost of gathering global information can thus be eliminated; saving communication costs. Experimental data, provided in this paper, illustrate that extant cloud deployments can be used as effective vehicles for devolved decision making. This finding strengthens the case for the proposed paradigm shift, since it does not require a change to the architecture of existing cloud systems. This shift would result in systems in which resources decide for themselves how best they can be used. This paper takes this idea to its logical conclusion and proposes a system for supporting self-managing resources in cloud environments. It introduces the concept of coalitions, consisting of collaborating resources, formed for the purpose of service delivery. It suggests the utility of restricting the interactions between the end-user and the cloud service provider to a well-defined services interface. It shows how clouds can be considered functionally, as engines for delivering an appropriate set of resources in response to service requests. And finally, since modern applications are increasingly constructed from sophisticated workflows of complex components, it shows how combinatorial auctions can be used to effectively deliver packages of resources to support those workflows.


Computer clouds Self-organization Over-provisioning Coalition formation Combinatorial auctions 



The work reported in this paper was partially supported by NSF CCR Grant 1525943 “Is the Simulation of Quantum Many-Body Systems Feasible on the Cloud?” to Dan C. Marinescu and collaborators and by a Grant from the EU H2020 program to J. P. Morrison for the CloudLightning consortium.


  1. 1.
    Barossso, L.A., Clidaras, J., Hözle, U.: The Datacenter as a Computer; an Introduction to the Design of Warehouse-Scale Machines. Morgan & Claypool, San Rafael (2013)Google Scholar
  2. 2.
    Chang, V., Wills, G., De Roure, D.: A review of cloud business models and sustainability. In: Proceedings of the IEEE 3rd International Conference on Cloud Computing, pp. 43–50. (2010)Google Scholar
  3. 3.
    Paya, A., Marinescu, D.C.: Energy-aware load balancing and application scaling for the cloud ecosystem. In: IEEE Transaction on Cloud Computing. (2015)Google Scholar
  4. 4.
    Blackburn, M., Hawkins, A.: Unused server survey results analysis. Accessed 6 Dec 2013
  5. 5.
    Marinescu, D.C., Paya, A., Morrison, J.P., Healy, P.: Distributed hierarchical control versus an economic model for cloud resource management. arXiv:1503.01061 (2015)
  6. 6.
    Marinescu, D.C., Paya, A., Morrison, J.P.: A cloud reservation system for big data applications. In: IEEE Transaction on Parallel and Distributed Computing. (2016)Google Scholar
  7. 7.
    Marinescu, D.C.: Complex Systems and Clouds: A Self-Organization and Self-Management Perspective. Morgan Kaufmann, Burlington (2016)Google Scholar
  8. 8.
    Müller, I., Kowalczyk, R., Braun, P.: Towards agent-based coalition formation for service composition. In: Proceedings IEEE/WIC/ACM International Conference on Intelligent Agent Technology, pp. 73–80. (2006)Google Scholar
  9. 9.
    Niyato, D., Vasilakos, A., Kun, Z.: Resource and revenue sharing with coalition formation of cloud providers: game theoretic approach. In: Proceedings IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, pp. 215–224. (2011)Google Scholar
  10. 10.
    Chaisiri, S., Lee, B., Niyato, D.: Optimization of resource provisioning cost in cloud computing. IEEE Trans. Serv. Comput. 5(2), 164–177 (2012)CrossRefGoogle Scholar
  11. 11.
    Li, C., Sycara, K.: Algorithm for combinatorial coalition formation and payoff division in an electronic marketplace. In: Proceedings AAMAS02—First Joint International Conference on Autonomous Agents and Multiagent Systems, pp. 120–127. (2002)Google Scholar
  12. 12.
    Mashayekhy, L., Nejad, M.M., Grosu, D.: Cloud federations in the sky: formation game and mechanisms. IEEE Trans. Cloud Comput. 3(1), 14–27 (2014)CrossRefGoogle Scholar
  13. 13.
    Sandholm, T.W., Larson, K.S., Andersson, M., Shehory, O., Tohm, F.: Coalition structure generation with worst case guarantees. Artif. Intell. 111(1–2), 209–238 (1999)Google Scholar
  14. 14.
    Rahwan, T., Ramchurn, S.D., Jennings, N.R., Giovannucci, A.: An anytime algorithm for optimal coalition structure generation. J. Artif. Intell. Res. 34, 521–567 (2009)MathSciNetzbMATHGoogle Scholar
  15. 15.
    Greco, G., Malizia, E., Palopoli, L., Scarello, F.: On the complexity of the core over coalition structures. In: Proceedings of the 22 International Joint Conference on Artificial Intelligence, pp. 216–221. (2011)Google Scholar
  16. 16.
    Shehory, O., Kraus, S.: Methods for task allocation via agent coalition formation. Artif. Intell. 101(1–2), 165–200 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Marinescu, D.C., Paya, A., Morrison, J.P.: Coalition formation and combinatorial auctions; applications to self-organization and self-management in utility computing. arXiv:1406.7487 (2015)
  18. 18.
    Ausubel, L., Cramton, P., Milgrom, P.: The clock-proxy auction: a practical combinatorial auction design. In: Cramton, P., Shoham, Y., Steinberg, R. (eds.) Combinatorial Auctions. MIT Press, Cambridge (2006)Google Scholar
  19. 19.
    Bradic, I.: Towards self-manageable cloud services. In: Proceedings of the 33 International Conference on Computer Software and Applications, pp. 128–133. (2009)Google Scholar
  20. 20.
    Marinescu, D.C.: Cloud Computing. Theory and Practice. Morgan Kaufmann, New York (2013)Google Scholar
  21. 21.
    Paton, N., de Arago, M.A.T., Lee, K., Fernandes, A.A.A., Sakellariou, R.R.: Optimizing utility in cloud computing through autonomic workload execution. Bull. Tech. Comm. Data Eng. 32(1), 51–58 (2009)Google Scholar
  22. 22.
    Sommerville, I., Cliff, D., Calinescu, R., Keen, J., Kelly, T., Kwiatowska, M., McDermid, J., Paige, R.: Large-scale IT complex systems. Commun. ACM 55(7), 71–77 (2012)CrossRefGoogle Scholar
  23. 23.
    Van, H.N., Tran, F.D., Menaud, J.M.: Autonomic virtual resource management for service hosting platforms. In: Software Engineering Challenges of Cloud Computing, ICSE Workshop at CLOUD09, pp. 1–8. (2009)Google Scholar
  24. 24.
    Minsky, M.: Computation: Finite and Infinite Machines. Prentice Hall, New York (1967)zbMATHGoogle Scholar
  25. 25.
    Gell-Mann, M.: Simplicity and complexity in the description of nature. Eng. Sci. I(3), 3–9 (1988)Google Scholar
  26. 26.
    Mayer, M.W.: Architecting principles for system of systems. Syst. Eng. 1(4), 267–274 (1998)CrossRefGoogle Scholar
  27. 27.
    Marinescu, D.C., Siegel, H.J., Morrison, J.P.: Options and commodity markets for computing resources. In: Buyya, R., Bubendorf, K. (eds.) Market Oriented Grid and Utility Computing, pp. 89–120. Wiley, New York (2009)CrossRefGoogle Scholar
  28. 28.
    Marinescu, D.C., Paya, A., Morrison, J.P., Healy, P.: An auction-driven, self-organizing cloud delivery model. (2013)

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of Central FloridaOrlandoUSA
  2. 2.Computer Science DepartmentUniversity College CorkCorkIreland
  3. 3.Computer Science DepartmentOld Dominion UniversityNorfolkUSA

Personalised recommendations