Skip to main content

Advertisement

SpringerLink
Log in

P-Aware: a proportional multi-resource scheduling strategy in cloud data center

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Concentrating on a single resource cannot efficiently cope with the overall high utilization of resources in cloud data centers. Nowadays multiple resource scheduling problem is more attractive to researchers. Some studies achieve progresses in multi-resource scenarios. However, these previous heuristics have obvious limitations in complex software defined cloud environment. Focusing on energy conservation and load balancing, we propose a preciousness model for multiple resource scheduling in this paper. We give the formulation of the problem and propose an innovative strategy (P-Aware). In P-Aware, a special algorithm PMDBP (Proportional Multi-dimensional Bin Packing) is applied in the multi-dimensional bin packing approach. In this algorithm, multiple resources are consumed in a proportional way. Structure and details of PMDBP are discussed in this paper. Extensive experiments demonstrate that our strategy outperforms others both in efficiency and load balancing. Now P-Aware has been implemented in the resource management system in our cooperative company to cut energy consumption and reduce resource contention.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Hahne, E.L.: Round-robin scheduling for max-min fairness in data networks. IEEE J. Sel. Areas Commun. 9, 1024–1039 (1991)

    Article  Google Scholar 

  2. Ghodsi, A., Zaharia, M., Hindman, B., Konwinski, A., Shenker, S., Stoica, I.: Dominant resource fairness: fair allocation of multiple resource types. In: NSDI 2011, pp. 24–24

  3. Joe-Wong, C., Sen, S., Lan, T., Chiang, M.: Multiresource allocation: fairness-efficiency tradeoffs in a unifying framework. IEEE/ACM Trans. Netw. 21, 1785–1798 (2013)

    Article  Google Scholar 

  4. Parkes, D.C., Procaccia, A.D., Shah, N.: Beyond dominant resource fairness: Extensions, limitations, and indivisibilities. ACM Trans. Econ. Comput. 3(1), 3 (2015)

    Article  MathSciNet  Google Scholar 

  5. Wang, W., Li, B., Liang, B.: Dominant resource fairness in cloud computing systems with heterogeneous servers. In: 2014 Proceedings of the IEEE 2014 INFOCOM, pp. 583–591. IEEE, Piscataway

  6. “Apache Hadoop”. http://hadoop.apache.org

  7. Hindman, B., Konwinski, A., Zaharia, M., Ghodsi, A., Joseph, A.D., Katz, R.H., Shenker, S., Stoica, I.: Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center. In: NSDI 2011, pp. 22–22

  8. “Apache Mesos”. http://mesos.apache.org

  9. Di, S., Wang, C.-L., Zhang, W., Cheng, L.: Probabilistic best-fit multi-dimensional range query in self-organizing cloud. In: 2011 International Conference on Parallel Processing (ICPP), pp. 763–772. IEEE, Piscataway (2011)

  10. Sarood, O., Gupta, A., Kalé, L.V.: Cloud Friendly Load Balancing for HPC Applications: Preliminary Work. In: 41st International Conference on Parallel Processing Workshops (ICPPW), pp. 200–205. IEEE, Piscataway (2012)

  11. Minarolli, D., Freisleben, B.: Distributed resource allocation to virtual machines via artificial neural networks. In: 22nd Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP), pp. 490–499. IEEE, Piscataway (2014)

  12. Chiesi, M., Vanzolini, L., Mucci, C., Scarselli, E.F., Guerrieri, R.: Power-aware job scheduling on heterogeneous multicore architectures. IEEE Trans. Parallel Distrib. Syst. 26(3), 868–877 (2015)

    Article  Google Scholar 

  13. Wu, C., Li, J., Xu, D., Yew, P.-C., Li, J., Wang, Z.: FPS: A Fair-Progress Process Scheduling Policy on Shared-Memory Multiprocessors. IEEE Trans. Parallel Distrib. Syst. 26(2), 444–454 (2015)

    Article  Google Scholar 

  14. Ryoo, I., Na, W., Kim, S.: Information exchange architecture based on software defined networking for cooperative intelligent transportation systems. Clust. Comput. 18(2), 771–782 (2015)

    Article  Google Scholar 

  15. Zeng, D., Li, P., Guo, S., Miyazaki, T., Hu, J., Xiang, Y.: Energy minimization in multi-task software-defined sensor networks. IEEE Trans. Comput. 64(11), 3128–3139 (2015)

    Article  MathSciNet  Google Scholar 

  16. Juliadotter, N.V., Choo, K.-K.R.: Cloud attack and risk assessment taxonomy. IEEE Cloud Comput. 2(1), 14–20 (2015)

    Article  Google Scholar 

  17. Ab Rahman, N.H., Choo, K.-K.R.: A survey of information security incident handling in the cloud. Comput. Secur. 49, 45–69 (2015)

    Article  Google Scholar 

  18. K. S. Tep, B. Martini, R. Hunt, and K.-K. R. Choo,: A taxonomy of Cloud Attack Consequences and Mitigation Strategies: The Role of Access Control and Privileged Access Management. In: 2015 IEEE Trustcom/BigDataSE/ISPA, vol. 1, pp. 1073–1080 (2015)

  19. Jararweh, Y., Al-Ayyoub, M., Benkhelifa, E., Vouk, M., Rindos, A.: Software defined cloud: Survey, system and evaluation. Future Gener. Comput. Syst. 58, 56–74 (2016). doi:10.1016/j.future.2015.10.015

    Article  Google Scholar 

  20. Sandhu, R., Sood, S.K.: Scheduling of big data applications on distributed cloud based on QoS parameters. Clust. Comput. 18(2), 817–828 (2015)

    Article  Google Scholar 

  21. Calheiros, R.N., Buyya, R.: Cost-effective provisioning and scheduling of deadline-constrained applications in hybrid clouds. Web Information Systems Engineering-WISE 2012, pp. 171–184. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  22. Coffman, J., Edward, G., Garey, M.R., Johnson, D.S.: Dynamic bin packing. SIAM J. Comput. 12(2), 227–258 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  23. Beaumont, O., Eyraud-Dubois, L., Thraves Caro, C., Rejeb, H.: Heterogeneous resource allocation under degree constraints. IEEE Trans. Parallel Distrib. Syst. 24(5), 926–937 (2013)

    Article  Google Scholar 

  24. Garey, M.R., Graham, R.L., Johnson, D.S., Yao, A.C.-C.: Resource constrained scheduling as generalized bin packing. J. Comb. Theory Ser. A 21(3), 257–298 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  25. Leinberger, W., Karypis, G., Kumar, V.: Multi-capacity bin packing algorithms with applications to job scheduling under multiple constraints. In: International Conference on Parallel Processing, pp. 404–412. IEEE, Piscataway (1999)

  26. Panigrahy, R., Talwar, K., Uyeda, L., Wieder, U.: Heuristics for vector bin packing. http://www.research.microsoft.com (2011)

  27. Sun, X., Wu, Q., Tan, Y., Wu, F.: MVEI: An Interference Prediction Model for CPU-intensive Application in Cloud Environment. In: 13th International Symposium on Distributed Computing and Applications to Business, Engineering and Science (DCABES), pp. 83–87. IEEE, Piscataway (2014)

  28. Tsai, C.-H., Shin, K.G., Reumann, J., Singhal, S.: Online web cluster capacity estimation and its application to energy conservation. IEEE Trans. Parallel Distrib. Syst. 18(7), 932–945 (2007)

    Article  Google Scholar 

  29. Pham, C., Li, Q., Estrada, Z., Kalbarczyk, Z., Iyer, R.: A Simulation Framework to Evaluate Virtual CPU Scheduling Algorithms. In: 33rd International Conference on Distributed Computing Systems Workshops (ICDCSW), pp. 138–143. IEEE, Piscataway (2013)

  30. Kinger, S., Goyal, K.: Energy-efficient CPU utilization based virtual machine scheduling in Green clouds. In: Communication and Computing (ARTCom 2013), Fifth International Conference on Advances in Recent Technologies, pp. 28–34. IET (2013)

  31. Zhang, P., Chu, R., Wang, H.: MemHole: An Efficient Black-Box Approach to Consolidate Memory in Virtualization Platform. In: 41st International Conference on Parallel Processing Workshops (ICPPW), pp. 608–609. IEEE, Piscataway (2012)

  32. Takouna, I., Meinel, C.: Energy and Performance Efficient Consolidation of Independent VMs in Virtualized Data Center by Exploiting VMs’ Memory Demand Heterogeneity. In: IEEE/ACM 6th International Conference on Utility and Cloud Computing (UCC), pp. 315–320. IEEE, Piscataway (2013)

  33. Liu, L., Chu, R., Zhu, Y., Zhang, P., Wang, L.: DMSS: a dynamic memory scheduling system in server consolidation environments. In: 15th IEEE International Symposium on Object/Component/Service-Oriented Real-Time Distributed Computing Workshops (ISORCW), pp. 70–75. IEEE, Piscataway ( 2012)

  34. Moraveji, R., Taheri, J., Reza, M., Rizvandi, N.B., Zomaya, A.Y.: Data-intensive workload consolidation for the Hadoop distributed file system. In: Proceedings of the 2012 ACM/IEEE 13th International Conference on Grid Computing, pp. 95–103. IEEE Computer Society (2012)

  35. Dong, B., Li, X., Xiao, L., Ruan, L.: Towards minimizing disk I/O contention: a partitioned file assignment approach. Future Gener. Comput. Syst. 37, 178–190 (2014)

    Article  Google Scholar 

  36. Zhou, T., Li, H., Zhu, B., Zhang, Y., Hou, H., Chen, J.: STORE: Data recovery with approximate minimum network bandwidth and disk I/O in distributed storage systems. In: IEEE International Conference on Big Data (Big Data), pp. 33–38. IEEE, Piscataway (2014)

  37. Mijumbi, R., Gorricho, J.-L., Serrat, J., Shen, M., Xu, K., Yang, K.: A neuro-fuzzy approach to self-management of virtual network resources. Expert Syst. Appl. 42(3), 1376–1390 (2015)

    Article  Google Scholar 

  38. Sun, X., Su, S., Xu, P., Chi, S., Luo, Y.: Multi-dimensional resource integrated scheduling in a shared data center. In: 31st International Conference on Distributed Computing Systems Workshops (ICDCSW), pp. 7–13. IEEE, Piscataway (2011)

  39. Jin, H., Pan, D., Xu, J., Pissinou, N.: Efficient VM placement with multiple deterministic and stochastic resources in data centers. In: IEEE Global Communications Conference (GLOBECOM), pp. 2505–2510. IEEE, Piscataway (2012)

  40. Zhu, Q., Zhu, J., Agrawal, G.: Power-aware consolidation of scientific workflows in virtualized environments. In: Proceedings of the 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1–12. IEEE Computer Society (2010)

  41. Xu, F., Liu, F., Jin, H., Vasilakos, A.V.: Managing performance overhead of virtual machines in cloud computing: a survey, state of the art, and future directions. Proc. IEEE 102(1), 11–31 (2014)

    Article  Google Scholar 

  42. Massimo, F., Christian, E., Henry, C., Choo, K.-K.R.: Live migration in emerging cloud paradigms. IEEE Cloud Comput. 3(2), 12–19 (2016)

    Article  Google Scholar 

  43. Xu, F., Liu, F., Liu, L., Jin, H., Li, B., Li, B.: iaware: Making live migration of virtual machines interference-aware in the cloud. IEEE Trans. Comput. 63(12), 3012–3025 (2014)

    Article  MathSciNet  Google Scholar 

  44. Loumiotis, I., Adamopoulou, E., Demestichas, K., Stamatiadi, T., Theologou, M.: On the predictability of next generation mobile network traffic using artificial neural networks. Int. J. Commun. Syst. 28(8), 1484–1492 (2015)

    Article  Google Scholar 

  45. Chen, Z., Zhu, Y., Di, Y., Feng, S.: Self-Adaptive prediction of cloud resource demands using ensemble model and subtractive-fuzzy clustering based fuzzy neural network. Comput. Intell. Neurosci. 2015, 17 (2015)

    Google Scholar 

  46. Chen, Z., Zhu, Y., Di, Y., Feng, S., Geng, J.: A high-accuracy self-adaptive resource demands predicting method in IaaS cloud environment. Neural Netw. World 25(5), 519 (2015)

    Article  Google Scholar 

  47. Mark, S., David, S., Frdric, V., Henri, C.: Resource allocation algorithms for virtualized service hosting platforms. Parallel Distrib. Comput. 70(9), 962–974 (2010)

    Article  MATH  Google Scholar 

  48. Buyya, R., Calheiros, R.N., Son, J., Dastjerdi, A.V., Yoon, Y.: Software-defined cloud computing: Architectural elements and open challenges. In: 2014 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 1–12. IEEE, Piscataway (2014)

  49. Xu, D., Yang, S., Liu, R.: A mixture of HMM, GA, and Elman network for load prediction in cloud-oriented data centers. J. Zhejiang Univ. SCI C 14(11), 845–858 (2013)

    Article  Google Scholar 

  50. Microsoft Systems Center Virtual Machine Manager. http://www.microsoft.com/systemcenter/virtualmachinemanager

  51. Garey, M.R., Johnson, D.S.: Approximation algorithms for bin packing problems: A survey. Anal. Design Algorithms Comb. Optim. 266, 147–172 (1981)

    Google Scholar 

  52. Esposito, C., Castiglione, A., Choo, K.-K.R.: Encryption-based solution for data sovereignty in federated clouds. IEEE Cloud Comput. 3(1), 12–17 (2016)

    Article  Google Scholar 

  53. Osanaiye, O., Choo, K.-K.R., Dlodlo, M.: Distributed denial of service (ddos) resilience in cloud: review and conceptual cloud ddos mitigation framework. J. Netw. Comput. Appl. 67, 147–165 (2016)

    Article  Google Scholar 

  54. Choo, K.-K.R.: A cloud security risk-management strategy. IEEE Cloud Comput. 1(2), 52–56 (2014)

    Article  Google Scholar 

  55. Martini, B., Choo, K.-K.R.: Distributed filesystem forensics: XtreemFS as a case study. Digital Investiga. 11(4), 295–313 (2014)

    Article  Google Scholar 

  56. Quick, D., Choo, K.-K.R.: Google Drive: Forensic analysis of data remnants. J. Netw. Comput. Appl. 40, 179–193 (2014)

    Article  Google Scholar 

  57. Martini, B., Choo, K.-K. R.: Remote programmatic vcloud forensics: a six-step collection process and a proof of concept. In: 2014 IEEE 13th International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), pp. 935–942 (2014)

  58. Martini, B., Choo, K.-K.R.: Remote programmatic vcloud forensics: a six-step collection process and a proof of concept. In: IEEE 13th International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), pp. 935–942. IEEE, Piscataway (2014)

Download references

Acknowledgments

This work was supported in part by NSFC under Grant 61103143, in part by Innovation Action Plan supported by Science and Technology Commission of Shanghai Municipality (15DZ1100305). The authors thank to China Telecom Shanghai branch which is the cooperative company in our current fund. They also thank to Dr. Fei.Xu for his helpful advices to this paper.

Author information

Authors and Affiliations

  1. School of Computer Engineering and Science, Shanghai University, Shanghai, 200444, China

    Hang Zhou, Qing Li, Weiqin Tong & Samina Kausar

  2. School of Computer Science and Technology, Xi’An JiaoTong University, Xi’An, 710049, Shanxi, China

    Hai Zhu

Authors
  1. Hang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weiqin Tong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Samina Kausar
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hai Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hang Zhou.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, H., Li, Q., Tong, W. et al. P-Aware: a proportional multi-resource scheduling strategy in cloud data center. Cluster Comput 19, 1089–1103 (2016). https://doi.org/10.1007/s10586-016-0593-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-016-0593-6

Keywords

Search

Navigation