Skip to main content

Hybrid Cloud Adaptive Scheduling Strategy for Heterogeneous Workloads

Journal of Grid Computing volume 17, pages 419–446 (2019)Cite this article

Abstract

With the advent of the era of big data, many companies have taken the most important steps in the hybrid cloud to handle large amounts of data. In a hybrid cloud environment, cloud burst technology enables applications to be processed at a lower cost in a private cloud and burst into the public cloud when the resources of the private cloud are exhausted. However, there are many challenges in hybrid cloud environment, such as the heterogeneous jobs, different cloud providers and how to deploy a new application with minimum monetary cost. In this paper, the efficient job scheduling approach for heterogeneous workloads in private cloud is proposed to ensure high resource utilization. Moreover, the task scheduling method based on BP neural network in hybrid cloud is proposed to ensure that the tasks can be completed within the specified deadline of the user. The experimental results show that the efficient job scheduling approach can veffectively reduce the job response time and improve the throughput of cluster. The task scheduling method can reduce the response time of tasks, improve QoS satisfaction rate and minimize the cost of public cloud.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

References

  1. Hwang, C.G., Yoon, C.P., Lee, D.: Exchange of data for big data in hybrid cloud environment. Int. J. Softw. Eng. Appl. 9(4), 67–72 (2015)

    Google Scholar 

  2. Clementecastello, F.J., Nicolae, B., Katrinis, K., et al.: Enabling big data analytics in the hybrid cloud using iterative MapReduce. In: Proceeding of 2015 IEEE Conference on Utility and Cloud Computing. IEEE Computer Society, pp. 290–299 (2015)

  3. Cisco: White paper: Cisco vni forecast and methodology (2016)

  4. Guo, T., Sharma, U., Wood, T., et al.: Seagull: intelligent cloud bursting for enterprise applications. Usenix conference on technical conference. USENIX Assoc. 157(10), 33–33 (2014)

    Google Scholar 

  5. Guo, T., Sharma, U., Shenoy, P., et al.: Cost-aware cloud bursting for enterprise applications. ACM Trans. Internet Technol. 13(3), 1–24 (2014)

    Article  Google Scholar 

  6. Zuo, X., Zhang, G., Tan, W.: Self-adaptive learning PSO-based deadline constrained task scheduling for hybrid IaaS cloud. Autom. Sci. Eng. IEEE Trans. 11(2), 564–573 (2014)

    Article  Google Scholar 

  7. Abrishami, H., Rezaeian, A., Tousi, G.K., et al.: Scheduling in hybrid cloud to maintain data privacy. In: Proceeding of 2015 International Conference on Innovative Computing Technology. IEEE, pp. 83–88 (2015)

  8. Clemente-Castelló, F.J., Mayo, R., Fernández, J.C.: Cost model and analysis of iterative MapReduce applications for hybrid cloud bursting. In: 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID), Madrid, pp. 858–864 (2017)

  9. Li, C., Li, L.Y.: Hybrid cloud scheduling method for cloud bursting. Fund. Inform. 138(4), 435–455 (2015)

    MathSciNet  Article  MATH  Google Scholar 

  10. Xue, N., Haugerud, H., Yazidi, A.: On automated cloud bursting and hybrid cloud setups using Apache Mesos. In: 2017 3rd International Conference of Cloud Computing Technologies and Applications (CloudTech), Rabat, pp. 1–8 (2017)

  11. Cao, Y., Lu, L., Yu, J., et al.: Online Cost-Aware service requests scheduling in hybrid clouds for cloud bursting. Web Inf. Syst. Eng. 10569, 259–274 (2017)

    Google Scholar 

  12. Clemente-Castelló, F.J., Nicolae, B., Mayo, R., Fernández, J. C.: Performance Model of MapReduce Iterative Applications for Hybrid Cloud Bursting. IEEE Trans. Parallel Distrib. Syst. 29(8), 1794–1807 (2018)

    Article  Google Scholar 

  13. Wei, H., Meng, F.: A novel scheduling mechanism for hybrid cloud systems. In: International Conference on Cloud Computing, pp. 734–741. IEEE (2017)

  14. Arantes, L., Friedman, R., Marin, O., et al.: Probabilistic byzantine tolerance scheduling in hybrid cloud environments. In: International Conference on Distributed Computing and Networking, pp. 2–12. ACM (2017)

  15. Liu, Y., Li, C., Yang, Z., et al.: Research on cost-optimal algorithm of multi-QoS constraints for task scheduling in hybrid-cloud. J. Softw. Eng. 9(1), 33–49 (2015)

    Article  Google Scholar 

  16. Balagoni, Y., Rao, R.R.: A cost-effective SLA-aware scheduling for hybrid cloud environment. In: IEEE International Conference on Computational Intelligence and Computing Research, pp. 1–7. IEEE (2017)

  17. Muñoz, VM, Ramo, A.C., Albor, V.F., et al.: Rafhyc: an architecture for constructing resilient services on federated hybrid clouds. J. Grid Comput. 11(4), 753–770 (2013)

    Article  Google Scholar 

  18. Caballer, M., Zala, S., García, Á.L., et al.: Orchestrating complex application architectures in heterogeneous clouds. J. Grid Comput. 16(1), 3–18 (2018)

    Article  Google Scholar 

  19. Moreno-Vozmediano, R., Huedo, E., Llorente, I.M.: Implementation and provisioning of federated networks in hybrid clouds. J. Grid Comput. 15(2), 1–20 (2017)

    Article  Google Scholar 

  20. Marosi, A., Kecskemeti, G., Kertesz, A., Kacsuk, P.: FCM: an architecture for integrating IaaS cloud systems. In: Villari, M., et al. (eds.) The 2nd International Conference on Cloud Computing, GRIDs, and Virtualization, pp. 7–12 (2011)

  21. Calatrava, A., Romero, E., Moltó, G., et al.: Self-managed cost-efficient virtual elastic clusters on hybrid Cloud infrastructures. Futur. Gener. Comput. Syst. 61, 13–25 (2016)

    Article  Google Scholar 

  22. Singh, D., Devgan, M., Bhushan, S.: Tasks scheduling with lessen energy usage over a cloud server using hybrid adaptive multi-queue approach. In: 2016 4th International Conference on Parallel, Distributed and Grid Computing (PDGC), Waknaghat, pp. 427–432 (2016)

  23. Zuo, L., Dong, S., Shu, L., Zhu, C., Han, G.: A Multiqueue Interlacing Peak Scheduling Method Based on Tasks’ Classification in Cloud Computing. IEEE Syst. J. 12(2), 1518–1530 (2018)

    Article  Google Scholar 

  24. Shorgin, S., Pechinkin, A., Samouylov, K., et al.: Queuing systems with multiple queues and b6atch arrivals for cloud computing system performance analysis. Science and Technology Conference. IEEE, pp. 1–4 (2015)

  25. Singh, J., Gupta, D.: Towards energy saving with smarter multi queue job scheduling algorithm in cloud computing. J. Eng. Appl. Sci. 12(10), 8944–8948 (2017)

    Google Scholar 

  26. Montes, J., Sánchez, A., Pérez, M.S.: Riding out the storm: how to deal with the complexity of grid and cloud management. J. Grid Comput. 10(3), 349–366 (2012)

    Article  Google Scholar 

  27. Pop, F., Dobre, C., Cristea, V., et al.: Deadline scheduling for aperiodic tasks in inter-Cloud environments: a new approach to resource management. J. Supercomput. 71(5), 1754–1765 (2015)

    Article  Google Scholar 

  28. Yuan, H., Bi, J., Tan, W., et al.: Temporal task scheduling with constrained service delay for profit maximization in hybrid clouds. IEEE Trans. Autom. Sci. Eng. 14(1), 337–348 (2017)

    Article  Google Scholar 

  29. Zuo, L., Shu, L., Dong, S., et al.: A multi-objective hybrid cloud resource scheduling method based on deadline and cost constraints. IEEE Access, pp. 22067–22080 (2016)

  30. Zuo, X., Zhang, G., Tan, W.: Self-adaptive learning PSO-based deadline constrained task scheduling for hybrid IaaS cloud. IEEE Trans. Autom. Sci. Eng. 11(2), 564–573 (2014)

    Article  Google Scholar 

  31. Wang, Y., Xue, G., Qian, S., Li, M.: An online cost-efficient scheduler for requests with deadline constraint in hybrid clouds. In: 2017 International Conference on Progress in Informatics and Computing (PIC), Nanjing, pp. 318–322 (2017)

  32. Tian, C., Zhou, H., He, Y., et al.: A dynamic MapReduce scheduler for heterogeneous workloads. In: Proceeding of 2009 International Conference on Grid and Cooperative Computing, pp. 218–224. ACM (2009)

  33. Spicuglia, S., Chen, L.Y.: On load balancing: a mix-aware algorithm for heterogeneous systems. In: Proceeding of 2013 International Conference on Performance Engineering, pp. 71–76. ACM (2013)

  34. Rasooli, A., Down, D.G.: COSHH: a Classification and optimization based scheduler for heterogeneous Hadoop systems. Futur. Gener. Comput. Syst. 36, 1–15 (2014)

    Article  Google Scholar 

  35. Wang, W.J., Chang, Y.S., Lo, W.T., et al.: Adaptive scheduling for parallel tasks with QoS satisfaction for hybrid cloud environments. J. Super. 66(2), 783–811 (2013)

    Article  Google Scholar 

Download references

Acknowledgments

The work was supported by the National Natural Science Foundation (NSF) under grants (No.61672397, No. 61873341), Application Foundation Frontier Project of WuHan (No. 2018010401011290), Open Foundation of Key Laboratory of Embedded System and Service Computing (Tongji University), Ministry of Education (ESSCKF 2018-2), Open Research Fund of Shaanxi Key Laboratory of Network Data Analysis and Intelligent Processing. Any opinions, findings, and conclusions are those of the authors and do not necessarily reflect the views of the above agencies.

Author information

Authors and Affiliations

  1. The Key Laboratory of Embedded System and Service Computing, Ministry of Education, Tongji University, Shanghai, 200092, China

    Li Chunlin

  2. Department of Computer Science, Wuhan University of Technology, Wuhan, 430063, People’s Republic of China

    Li Chunlin, Tang Jianhang & Luo Youlong

  3. Shaanxi Key Laboratory of Network Data Analysis and Intelligent Processing, Xi’an University of Posts and Telecommunications, Xi’an Shaanxi, 710121, China

    Li Chunlin

Authors
  1. Li Chunlin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tang Jianhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luo Youlong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li Chunlin.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Chunlin, L., Jianhang, T. & Youlong, L. Hybrid Cloud Adaptive Scheduling Strategy for Heterogeneous Workloads. J Grid Computing 17, 419–446 (2019). https://doi.org/10.1007/s10723-019-09481-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10723-019-09481-3

Keywords

  • Hybrid cloud
  • Heterogeneous workloads
  • BP neural network
  • Job scheduling