Abstract
With the advent of the era of big data, many companies have taken the most important steps in the hybrid cloud to handle large amounts of data. In a hybrid cloud environment, cloud burst technology enables applications to be processed at a lower cost in a private cloud and burst into the public cloud when the resources of the private cloud are exhausted. However, there are many challenges in hybrid cloud environment, such as the heterogeneous jobs, different cloud providers and how to deploy a new application with minimum monetary cost. In this paper, the efficient job scheduling approach for heterogeneous workloads in private cloud is proposed to ensure high resource utilization. Moreover, the task scheduling method based on BP neural network in hybrid cloud is proposed to ensure that the tasks can be completed within the specified deadline of the user. The experimental results show that the efficient job scheduling approach can veffectively reduce the job response time and improve the throughput of cluster. The task scheduling method can reduce the response time of tasks, improve QoS satisfaction rate and minimize the cost of public cloud.
Similar content being viewed by others
References
Hwang, C.G., Yoon, C.P., Lee, D.: Exchange of data for big data in hybrid cloud environment. Int. J. Softw. Eng. Appl. 9(4), 67–72 (2015)
Clementecastello, F.J., Nicolae, B., Katrinis, K., et al.: Enabling big data analytics in the hybrid cloud using iterative MapReduce. In: Proceeding of 2015 IEEE Conference on Utility and Cloud Computing. IEEE Computer Society, pp. 290–299 (2015)
Cisco: White paper: Cisco vni forecast and methodology (2016)
Guo, T., Sharma, U., Wood, T., et al.: Seagull: intelligent cloud bursting for enterprise applications. Usenix conference on technical conference. USENIX Assoc. 157(10), 33–33 (2014)
Guo, T., Sharma, U., Shenoy, P., et al.: Cost-aware cloud bursting for enterprise applications. ACM Trans. Internet Technol. 13(3), 1–24 (2014)
Zuo, X., Zhang, G., Tan, W.: Self-adaptive learning PSO-based deadline constrained task scheduling for hybrid IaaS cloud. Autom. Sci. Eng. IEEE Trans. 11(2), 564–573 (2014)
Abrishami, H., Rezaeian, A., Tousi, G.K., et al.: Scheduling in hybrid cloud to maintain data privacy. In: Proceeding of 2015 International Conference on Innovative Computing Technology. IEEE, pp. 83–88 (2015)
Clemente-Castelló, F.J., Mayo, R., Fernández, J.C.: Cost model and analysis of iterative MapReduce applications for hybrid cloud bursting. In: 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID), Madrid, pp. 858–864 (2017)
Li, C., Li, L.Y.: Hybrid cloud scheduling method for cloud bursting. Fund. Inform. 138(4), 435–455 (2015)
Xue, N., Haugerud, H., Yazidi, A.: On automated cloud bursting and hybrid cloud setups using Apache Mesos. In: 2017 3rd International Conference of Cloud Computing Technologies and Applications (CloudTech), Rabat, pp. 1–8 (2017)
Cao, Y., Lu, L., Yu, J., et al.: Online Cost-Aware service requests scheduling in hybrid clouds for cloud bursting. Web Inf. Syst. Eng. 10569, 259–274 (2017)
Clemente-Castelló, F.J., Nicolae, B., Mayo, R., Fernández, J. C.: Performance Model of MapReduce Iterative Applications for Hybrid Cloud Bursting. IEEE Trans. Parallel Distrib. Syst. 29(8), 1794–1807 (2018)
Wei, H., Meng, F.: A novel scheduling mechanism for hybrid cloud systems. In: International Conference on Cloud Computing, pp. 734–741. IEEE (2017)
Arantes, L., Friedman, R., Marin, O., et al.: Probabilistic byzantine tolerance scheduling in hybrid cloud environments. In: International Conference on Distributed Computing and Networking, pp. 2–12. ACM (2017)
Liu, Y., Li, C., Yang, Z., et al.: Research on cost-optimal algorithm of multi-QoS constraints for task scheduling in hybrid-cloud. J. Softw. Eng. 9(1), 33–49 (2015)
Balagoni, Y., Rao, R.R.: A cost-effective SLA-aware scheduling for hybrid cloud environment. In: IEEE International Conference on Computational Intelligence and Computing Research, pp. 1–7. IEEE (2017)
Muñoz, VM, Ramo, A.C., Albor, V.F., et al.: Rafhyc: an architecture for constructing resilient services on federated hybrid clouds. J. Grid Comput. 11(4), 753–770 (2013)
Caballer, M., Zala, S., García, Á.L., et al.: Orchestrating complex application architectures in heterogeneous clouds. J. Grid Comput. 16(1), 3–18 (2018)
Moreno-Vozmediano, R., Huedo, E., Llorente, I.M.: Implementation and provisioning of federated networks in hybrid clouds. J. Grid Comput. 15(2), 1–20 (2017)
Marosi, A., Kecskemeti, G., Kertesz, A., Kacsuk, P.: FCM: an architecture for integrating IaaS cloud systems. In: Villari, M., et al. (eds.) The 2nd International Conference on Cloud Computing, GRIDs, and Virtualization, pp. 7–12 (2011)
Calatrava, A., Romero, E., Moltó, G., et al.: Self-managed cost-efficient virtual elastic clusters on hybrid Cloud infrastructures. Futur. Gener. Comput. Syst. 61, 13–25 (2016)
Singh, D., Devgan, M., Bhushan, S.: Tasks scheduling with lessen energy usage over a cloud server using hybrid adaptive multi-queue approach. In: 2016 4th International Conference on Parallel, Distributed and Grid Computing (PDGC), Waknaghat, pp. 427–432 (2016)
Zuo, L., Dong, S., Shu, L., Zhu, C., Han, G.: A Multiqueue Interlacing Peak Scheduling Method Based on Tasks’ Classification in Cloud Computing. IEEE Syst. J. 12(2), 1518–1530 (2018)
Shorgin, S., Pechinkin, A., Samouylov, K., et al.: Queuing systems with multiple queues and b6atch arrivals for cloud computing system performance analysis. Science and Technology Conference. IEEE, pp. 1–4 (2015)
Singh, J., Gupta, D.: Towards energy saving with smarter multi queue job scheduling algorithm in cloud computing. J. Eng. Appl. Sci. 12(10), 8944–8948 (2017)
Montes, J., Sánchez, A., Pérez, M.S.: Riding out the storm: how to deal with the complexity of grid and cloud management. J. Grid Comput. 10(3), 349–366 (2012)
Pop, F., Dobre, C., Cristea, V., et al.: Deadline scheduling for aperiodic tasks in inter-Cloud environments: a new approach to resource management. J. Supercomput. 71(5), 1754–1765 (2015)
Yuan, H., Bi, J., Tan, W., et al.: Temporal task scheduling with constrained service delay for profit maximization in hybrid clouds. IEEE Trans. Autom. Sci. Eng. 14(1), 337–348 (2017)
Zuo, L., Shu, L., Dong, S., et al.: A multi-objective hybrid cloud resource scheduling method based on deadline and cost constraints. IEEE Access, pp. 22067–22080 (2016)
Zuo, X., Zhang, G., Tan, W.: Self-adaptive learning PSO-based deadline constrained task scheduling for hybrid IaaS cloud. IEEE Trans. Autom. Sci. Eng. 11(2), 564–573 (2014)
Wang, Y., Xue, G., Qian, S., Li, M.: An online cost-efficient scheduler for requests with deadline constraint in hybrid clouds. In: 2017 International Conference on Progress in Informatics and Computing (PIC), Nanjing, pp. 318–322 (2017)
Tian, C., Zhou, H., He, Y., et al.: A dynamic MapReduce scheduler for heterogeneous workloads. In: Proceeding of 2009 International Conference on Grid and Cooperative Computing, pp. 218–224. ACM (2009)
Spicuglia, S., Chen, L.Y.: On load balancing: a mix-aware algorithm for heterogeneous systems. In: Proceeding of 2013 International Conference on Performance Engineering, pp. 71–76. ACM (2013)
Rasooli, A., Down, D.G.: COSHH: a Classification and optimization based scheduler for heterogeneous Hadoop systems. Futur. Gener. Comput. Syst. 36, 1–15 (2014)
Wang, W.J., Chang, Y.S., Lo, W.T., et al.: Adaptive scheduling for parallel tasks with QoS satisfaction for hybrid cloud environments. J. Super. 66(2), 783–811 (2013)
Acknowledgments
The work was supported by the National Natural Science Foundation (NSF) under grants (No.61672397, No. 61873341), Application Foundation Frontier Project of WuHan (No. 2018010401011290), Open Foundation of Key Laboratory of Embedded System and Service Computing (Tongji University), Ministry of Education (ESSCKF 2018-2), Open Research Fund of Shaanxi Key Laboratory of Network Data Analysis and Intelligent Processing. Any opinions, findings, and conclusions are those of the authors and do not necessarily reflect the views of the above agencies.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Chunlin, L., Jianhang, T. & Youlong, L. Hybrid Cloud Adaptive Scheduling Strategy for Heterogeneous Workloads. J Grid Computing 17, 419–446 (2019). https://doi.org/10.1007/s10723-019-09481-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10723-019-09481-3