Skip to main content

Advertisement

SpringerLink
Log in

Energy efficient job scheduling with workload prediction on cloud data center

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Data centers are the backbone of cloud infrastructure platform to support large-scale data processing and storage. More and more business-to-consumer and enterprise applications are based on cloud data center. However, the amount of data center energy consumption is inevitably lead to high operation costs. The aim of this paper is to comprehensive reduce energy consumption of cloud data center servers, network, and cooling systems. We first build an energy efficient cloud data center system including its architecture, job and power consumption model. Then, we combine the linear regression and wavelet neural network techniques into a prediction method, which we call MLWNN, to forecast the cloud data center short-term workload. Third, we propose a heuristic energy efficient job scheduling with workload prediction solution, which is divided into resource management strategy and online energy efficient job scheduling algorithm. Our extensive simulation performance evaluation results clearly demonstrate that our proposed solution has good performance and is very suitable for low workload cloud data center.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Bezawada, B., Liu, A.X., Jayaraman, B., Wang, A.L., Rui, L.: Privacy preserving string matching for cloud computing. In: IEEE 35th International Conference on Distributed Computing Systems, ICDCS 2015, pp. 609–618

  2. Boru, D., Kliazovich, D., Granelli, F., Bouvry, P., Zomaya, A.Y.: Energy-efficient data replication in cloud computing datacenters. Clust. Comput. 18(1), 385–402 (2015)

    Article  Google Scholar 

  3. Kliazovich, D., Bouvry, P., Khan, S.U.: GreenCloud: a packet-level simulator of energy-aware cloud computing data centers. J. Supercomput. 62(3), 1263–1283 (2012)

    Article  Google Scholar 

  4. http://www.datacenterdynamics.com/news/facebook-dat a-centers-energy-use-up-in-2012/80642.article. Accessed 10 Jan 2017

  5. https://www.nrdc.org/energy/files/data-center-efficiency- assessment-IP.pdf. Accessed 25 Feb 2017

  6. Haque, M.A., Aydin, H., Zhu, D.: On reliability management of energy-aware real-time systems through task replication. IEEE Trans. Parallel Distrib. Syst. 28(3), 813–825 (2017)

    Article  Google Scholar 

  7. Awan, M.A., Nelissen, G., Yomsi, P.M., Petters, S.M.: Online slack consolidation in global-EDF for energy consumption minimisation. J. Syst. Archit. 63, 1–15 (2016)

    Article  Google Scholar 

  8. Li, K.: Improving multicore server performance and reducing energy consumption by workload dependent dynamic power management. IEEE Trans. Cloud Comput. 4(2), 122–137 (2016)

    Article  Google Scholar 

  9. Gao, Y., Guan, H., Qi, Z., Hou, Y., Liu, L.: A multi-objective ant colony system algorithm for virtual machine placement in cloud computing. J. Comput. Syst. Sci. 79(8), 1230–1242 (2013)

    Article  MathSciNet  Google Scholar 

  10. Hieu, N., Francesco, M., Yl\(\ddot{a}\)-J\(\ddot{a}\ddot{a}\)ski, A.: Virtual machine consolidation with usage prediction for energy-efficient cloud data centers. In: 2015 IEEE 8th International Conference on Cloud Computing, pp. 750–757 (2015)

  11. Li, D., Chen, C., Guan, J., Zhang, Y., Zhu, J., Yu, R.: DCloud: deadline-aware resource allocation for cloud computing jobs. IEEE Trans. Parallel Distrib. Syst. 27(8), 2248–2260 (2016)

    Article  Google Scholar 

  12. Uddin, M., Darabidarabkhani, Y., Shah, A., Memond, J.: Evaluating power efficient algorithms for efficiency and carbon emissions in cloud data centers: a review. Renew. Sustain. Energy Rev. 51, 1553–1563 (2015)

    Article  Google Scholar 

  13. Guan, X., Choi, B., Song, S.: Energy efficient virtual network embedding for green data centers using data center topology and future migration. Comput. Commun. 69, 50–59 (2015)

    Article  Google Scholar 

  14. Hao, Y., Liu, G.: Evaluation of nine heuristic algorithms with data-intensive jobs and computing-intensive jobs in a dynamic environment. IET Softw. 9(1), 7–16 (2015)

    Article  MathSciNet  Google Scholar 

  15. Li, K., Tang, X., Veeravalli, B., Li, K.: Scheduling precedence constrained stochastic tasks on heterogeneous cluster systems. IEEE Trans. Comput. 64(1), 191–204 (2015)

    Article  MathSciNet  Google Scholar 

  16. Li, J., Qiu, M., Ming, Z., Quan, G., Qin, X., Gu, Z.: Online optimization for scheduling preemptable tasks on IaaS cloud system. J. Parallel Distrib. Comput. 72(3), 666–677 (2012)

    Article  Google Scholar 

  17. Maguluri, S., Srikant, R.: Scheduling jobs with unknown duration in clouds. In: Proceedings IEEE INFOCOM, pp. 1887–1895 (2013)

  18. Tang, X., Li, K., Liao, G., Fang, K., Wu, F.: A stochastic scheduling algorithm for precedence constrained tasks on grid. Future Gener. Comput. Syst. 27, 1083–1091 (2011)

    Article  Google Scholar 

  19. Gerards, M.E.T., Hurink, J.L., Kuper, J.: On the interplay between Global DVFS and scheduling tasks with precedence constraints. IEEE Trans. Comput. 64(6), 1742–1754 (2015)

    MathSciNet  MATH  Google Scholar 

  20. Rizvandi, N., Taheri, J., Zomaya, A.: Some observations on optimal frequency selection in DVFS-based power consumption minimization. J. Parallel Distrib. Comput. 71(8), 1154–1164 (2011)

    Article  Google Scholar 

  21. Li, K., Tang, X., Li, Keqin: Energy-efficient stochastic task scheduling on heterogeneous computing systems. IEEE Trans. Parallel Distrib. Syst. 25(11), 2867–2876 (2014)

    Article  Google Scholar 

  22. Huai, W., Huang, W., Jin, S., Qian, Z.: Towards energy efficient scheduling for online tasks in cloud data centers based on DVFS. In: The 9th International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing, IMIS 2015, pp. 225–232

  23. Barroso, L., Hölzle, U.: The case for energy proportional computing. Computer 40(12), 33–37 (2007)

    Article  Google Scholar 

  24. Guan, X., Choi, B., Song, S.: Topology and migration-aware energy efficient virtual network embedding for green data centers. In: 23rd International Conference on Computer Communication and Networks (ICCCN), pp. 1–8 (2014)

  25. Lee, S., Chung, B.D., Jeon, H.W., Chang, J.: A dynamic control approach for energy-efficient production scheduling on a single machine under time-varying electricity pricing. J. Clean. Prod. 165, 552–563 (2017)

    Article  Google Scholar 

  26. Attia, K.M., El-Hosseini, M.A., Hesham, A.: Dynamic power management techniques in multi-core architectures: a survey study. Ain Shams Eng. J. 8(3), 445–456 (2017)

    Article  Google Scholar 

  27. Tian, W., Zhao, Y., Xu, M., Zhong, Y., Sun, X.: A toolkit for modeling and simulation of real-time virtual machine allocation in a cloud data center. IEEE Rrans. Autom. Sci. Eng. 12(1), 153–161 (2015)

    Article  Google Scholar 

  28. Zhu, W., Zhuang, Y., Zhang, L.: A three-dimensional virtual resource scheduling method for energy saving in cloud computing. Future Gener. Comput. Syst. 69, 66–74 (2017)

    Article  Google Scholar 

  29. Rao, K., Thilagam, P.: Heuristics based server consolidation with residual resource defragmentation in cloud data centers. Future Gener. Comput. Syst. 50, 87–98 (2015)

    Article  Google Scholar 

  30. Wang, Y., Wang, X.: Performance-controlled server consolidation for virtualized datacenters with multi-tier applications. Sustain. Comput. 4, 52–65 (2014)

    Google Scholar 

  31. Menarini, M., Seracini, F., Zhang, X., Rosing, T., Kr\(\ddot{u}\)er, I.: Green web services: improving energy efficiency in data centers via workload predictions. In: The 2013 2nd International Workshop on Green and Sustainable Software (GREENS), pp. 8–15 (2013)

  32. Rahmanian, A.A., Ghobaei-Arani, M., Tofighy, S.: A learning automata-based ensemble resource usage prediction algorithm for cloud computing environment. Future Gener. Comput. Syst. 79, 54–71 (2018)

    Article  Google Scholar 

  33. Liu, B., Lin, Y., Chen, Y.: Quantitative workload analysis and prediction using Google cluster traces. In: IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) 2016, pp. 935–940 (2016)

  34. Alexandridis, A., Zapranis, A.: Wavelet neural networks: a practical guide. Neural Netw. 42(1), 1–27 (2013)

    Article  Google Scholar 

  35. Moschakis, I., Karatza, H.: A meta-heuristic optimization approach to the scheduling of bag-of-tasks applications on heterogeneous clouds with multi-level arrivals and critical jobs. Simul. Model. Pract. Theory 57, 1–25 (2015)

    Article  Google Scholar 

  36. Zhu, D., Aydin, H.: Reliability-aware energy management for periodic real-time tasks. IEEE Trans. Comput. 58(10), 1382–1397 (2009)

    Article  MathSciNet  Google Scholar 

  37. Shi, X., Dong, J., Djouadi, S.M., Wang, Y., Ma, X., Feng, Y.: Power-efficient resource management for co-located virtualized web servers: a stochastic control approach. In: International Green Computing Conference, pp. 1–9 (2014)

  38. Serackis, A., Plonis, D., Krukonis, A., Katkevicius, A.: The prediction of cut-off frequencies of models of gyroelectric waveguides using artificial neural networks. In: IEEE 4th Workshop on Advances in Information, Electronic and Electrical Engineering (AIEEE), pp. 1–5 (2016)

  39. Lui, A., Najmi, A.: Time-frequency decomposition of signals in a current disruption event. Geophys. Res. Lett. 24(24), 3157–3160 (1997)

    Article  Google Scholar 

  40. http://www.cloudbus.org/cloudsim/. Accessed 15 Feb 2017

Download references

Acknowledgements

This research was partially funded by the National Key Research and Development Program of China (Grant No. 2016YFB0201402), National Science Foundation of China (Grant Nos. 61370098, 61672219), Hunan Provincial Natural Science Foundation of China (Grant No. 2015JJ2078), the Hunan Provincial Innovation Foundation For Postgraduate (Grant No. CX2016B316). A lot of thanks should be given to referees and editors,their valuable comments greatly improved the quality of the manuscript. We also thank yufayun data center that provides the workload system log files.

Author information

Authors and Affiliations

  1. Information Science and Technology College/Southern Regional Collaborative Innovation Center for Grain and Oil Crops in China, Hunan Agricultural University, Changsha, 410128, China

    Xiaoyong Tang, Xiaoyi Liao, Jie Zheng & Xiaopan Yang

  2. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210046, China

    Xiaoyong Tang

Authors
  1. Xiaoyong Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoyi Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jie Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaopan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoyong Tang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tang, X., Liao, X., Zheng, J. et al. Energy efficient job scheduling with workload prediction on cloud data center. Cluster Comput 21, 1581–1593 (2018). https://doi.org/10.1007/s10586-018-2154-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-018-2154-7

Keywords

Search

Navigation