Skip to main content
SpringerLink
Log in

E2LG: a multiscale ensemble of LSTM/GAN deep learning architecture for multistep-ahead cloud workload prediction

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Efficient resource demand prediction and management are two main challenges for cloud service providers in order to control dynamic autoscaling and power consumption in recent years. The behavior of cloud workload time-series at subminute scale is highly chaotic and volatile; therefore, traditional machine learning-based time-series analysis approaches fail to obtain accurate predictions. In recent years, deep learning-based schemes are suggested to predict highly nonlinear cloud workloads, but sometimes they fail to obtain excellent prediction results. Hence, demands for more accurate prediction algorithm exist. In this paper, we address this issue by proposing a hybrid E2LG algorithm, which decomposes the cloud workload time-series into its constituent components in different frequency bands using empirical mode decomposition method which reduces the complexity and nonlinearity of prediction model in each frequency band. Also, a new state-of-the-art ensemble GAN/LSTM deep learning architecture is proposed to predict each sub band workload time-series individually, based on its degree of complexity and volatility. Our novel ensemble GAN/LSTM architecture, which employs stacked LSTM blocks as its generator and 1D ConvNets as discriminator, can exploit the long-term nonlinear dependencies of cloud workload time-series effectively specially in high-frequency, noise-like components. By validating our approach using extensive set of experiments with standard real cloud workload traces, we confirm that E2LG provides significant improvements in cloud workload prediction accuracy with respect to the mean absolute and standard deviation of the prediction error and outperforming traditional and state-of-the-art deep learning approaches. It improves the prediction accuracy at least 5% and 12% in average compared to the main contemporary approaches in recent papers such as hybrid methods which employs CNN, LSTM or SVR.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Kumar J, Goomer R, Singh AK (2018) Long short term memory recurrent neural network (LSTM-RNN) based workload forecasting model for cloud datacenters. Procedia Comput Sci 125:676–682. https://doi.org/10.1016/j.procs.2017.12.087

    Article  Google Scholar 

  2. Zhang Q, Yang LT, Yan Z, Chen Z, Li P (2018) An efficient deep learning model to predict cloud workload for industry informatics. IEEE Trans Ind Inform 14:3170–3178. https://doi.org/10.1109/TII.2018.2808910

    Article  Google Scholar 

  3. Mozo A, Ordozgoiti B, Gómez-Canaval S (2018) Forecasting short-term data center network traffic load with convolutional neural networks. PLoS ONE. https://doi.org/10.1371/journal.pone.0191939

    Article  Google Scholar 

  4. Kumar J, Singh AK, Buyya R (2020) Ensemble learning based predictive framework for virtual machine resource request prediction. Neurocomputing. https://doi.org/10.1016/j.neucom.2020.02.014

    Article  Google Scholar 

  5. Jeddi S, Sharifian S (2020) A hybrid wavelet decomposer and GMDH-ELM ensemble model for network function virtualization workload forecasting in cloud computing. Appl Soft Comput J. https://doi.org/10.1016/j.asoc.2019.105940

    Article  Google Scholar 

  6. Amiri M, Mohammad-Khanli L, Mirandola R (2018) An online learning model based on episode mining for workload prediction in cloud. Futur Gener Comput Syst 87:83–101. https://doi.org/10.1016/j.future.2018.04.044

    Article  Google Scholar 

  7. Sharifian S, Barati M (2019) An ensemble multiscale wavelet-GARCH hybrid SVR algorithm for mobile cloud computing workload prediction. Int J Mach Learn Cybern 10:3285–3300. https://doi.org/10.1007/s13042-019-01017-1

    Article  Google Scholar 

  8. Wang H, Li G, Wang G, Peng J, Jiang H, Liu Y (2017) Deep learning based ensemble approach for probabilistic wind power forecasting. Appl Energy 188:56–70. https://doi.org/10.1016/j.apenergy.2016.11.111

    Article  Google Scholar 

  9. Hochreiter S, Schmidhuber JU (1997) Long shortterm memory. Neural Comput 9:1735–1780

    Article  Google Scholar 

  10. Lai G, Chang WC, Yang Y, Liu H (2018) Modeling long- and short-term temporal patterns with deep neural networks. In: 41st International ACM SIGIR Conference Research Development Information Retrieval, SIGIR 2018, pp 95–104. https://doi.org/10.1145/3209978.3210006

  11. Song B, Yu Y, Zhou Y, Wang Z, Du S (2018) Host load prediction with long short-term memory in cloud computing. J Supercomput 74:6554–6568. https://doi.org/10.1007/s11227-017-2044-4

    Article  Google Scholar 

  12. Guo G, Wang C, Chen J, Ge P, Chen W (2019) Who is answering whom? Finding “Reply-To” relations in group chats with deep bidirectional LSTM networks. Cluster Comput 22:2089–2100. https://doi.org/10.1007/s10586-018-2031-4

    Article  Google Scholar 

  13. Xiang Y, Gou L, He L, Xia S, Wang W (2018) A SVR–ANN combined model based on ensemble EMD for rainfall prediction. Appl Soft Comput J 73:874–883. https://doi.org/10.1016/j.asoc.2018.09.018

    Article  Google Scholar 

  14. Calheiros RN, Masoumi E, Ranjan R, Buyya R (2015) Workload prediction using ARIMA model and its impact on cloud applications’ QoS. IEEE Trans Cloud Comput 3:449–458. https://doi.org/10.1109/TCC.2014.2350475

    Article  Google Scholar 

  15. Tan Z, Zhang J, Wang J, Xu J (2010) Day-ahead electricity price forecasting using wavelet transform combined with ARIMA and GARCH models. Appl Energy 87:3606–3610. https://doi.org/10.1016/j.apenergy.2010.05.012

    Article  Google Scholar 

  16. Shafie-Khah M, Moghaddam MP, Sheikh-El-Eslami MK (2011) Price forecasting of day-ahead electricity markets using a hybrid forecast method. Energy Convers Manag 52:2165–2169. https://doi.org/10.1016/j.enconman.2010.10.047

    Article  Google Scholar 

  17. Piacentini M, Rinaldi F (2011) Path loss prediction in urban environment using learning machines and dimensionality reduction techniques. Comput Manag Sci 8:371–385. https://doi.org/10.1007/s10287-010-0121-8

    Article  MATH  Google Scholar 

  18. Bao Y, Wang H, Wang B (2014) Short-term wind power prediction using differential EMD and relevance vector machine. Neural Comput Appl 25:283–289. https://doi.org/10.1007/s00521-013-1482-z

    Article  Google Scholar 

  19. Yaslan Y, Bican B (2017) Empirical mode decomposition based denoising method with support vector regression for time series prediction: a case study for electricity load forecasting. Meas J Int Meas Confed 103:52–61. https://doi.org/10.1016/j.measurement.2017.02.007

    Article  Google Scholar 

  20. Assem H, Ghariba S, Makrai G, Johnston P, Gill L, Pilla F (2017) Urban water flow and water level prediction based on deep learning, lecture notes computer science (Including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics). 10536 LNAI, pp 317–329. https://doi.org/10.1007/978-3-319-71273-4_26.

  21. Hussain AJ, Al-Jumeily D, Al-Askar H, Radi N (2016) Regularized dynamic self-organized neural network inspired by the immune algorithm for financial time series prediction. Neurocomputing 188:23–30. https://doi.org/10.1016/j.neucom.2015.01.109

    Article  Google Scholar 

  22. Jeddi S, Sharifian S (2019) A water cycle optimized wavelet neural network algorithm for demand prediction in cloud computing. Clust Comput 22:1397–1412. https://doi.org/10.1007/s10586-019-02916-2

    Article  Google Scholar 

  23. Mo L, Xie L, Jiang X, Teng G, Xu L, Xiao J (2018) GMDH-based hybrid model for container throughput forecasting: Selective combination forecasting in nonlinear subseries. Appl Soft Comput J 62:478–490. https://doi.org/10.1016/j.asoc.2017.10.033

    Article  Google Scholar 

  24. Humphrey EJ, Bello JP, Lecun Y (2013) Feature learning and deep architectures: new directions for music informatics. J Intell Inf Syst 41:461–481. https://doi.org/10.1007/s10844-013-0248-5

    Article  Google Scholar 

  25. Karim F, Majumdar S, Darabi H (2019) Insights into lstm fully convolutional networks for time series classification. IEEE Access 7:67718–67725. https://doi.org/10.1109/ACCESS.2019.2916828

    Article  Google Scholar 

  26. Zhao R, Yan R, Wang J, Mao K (2017) Learning to monitor machine health with convolutional Bi-directional LSTM networks. Sensors (Switzerland). https://doi.org/10.3390/s17020273

    Article  Google Scholar 

  27. Azar J, Makhoul A, Couturier R, Demerjian J (2020) Robust IoT time series classification with data compression and deep learning. Neurocomputing. https://doi.org/10.1016/j.neucom.2020.02.097

    Article  Google Scholar 

  28. He W (2017) Load forecasting via deep neural networks. Procedia Comput Sci 122:308–314. https://doi.org/10.1016/j.procs.2017.11.374

    Article  Google Scholar 

  29. Zhongda T, Shujiang L, Yanhong W, Yi S (2017) A prediction method based on wavelet transform and multiple models fusion for chaotic time series. Chaos Solitons Fractals 98:158–172. https://doi.org/10.1016/j.chaos.2017.03.018

    Article  MathSciNet  MATH  Google Scholar 

  30. Qin D, Yu J, Zou G, Yong R, Zhao Q, Zhang B (2019) A novel combined prediction scheme based on CNN and LSTM for Urban PM2.5 concentration. IEEE Access 7:20050–20059. https://doi.org/10.1109/ACCESS.2019.2897028

    Article  Google Scholar 

  31. Yu H, Wu Z, Wang S, Wang Y, Ma X (2017) Spatiotemporal recurrent convolutional networks for traffic prediction in transportation networks. Sensors (Switzerland). https://doi.org/10.3390/s17071501

    Article  Google Scholar 

  32. Le T, Vo MT, Vo B, Hwang E, Rho S, Baik SW (2019) Improving electric energy consumption prediction using CNN and Bi-LSTM. Appl Sci. https://doi.org/10.3390/app9204237

    Article  Google Scholar 

  33. Huang CJ, Kuo PH (2018) A deep cnn-lstm model for particulate matter (Pm2.5) forecasting in smart cities. Sensors (Switzerland). https://doi.org/10.3390/s18072220

    Article  Google Scholar 

  34. Zhou X, Pan Z, Hu G, Tang S, Zhao C (2018) Stock market prediction on high-frequency data using generative adversarial nets. Math Probl Eng. https://doi.org/10.1155/2018/4907423

    Article  Google Scholar 

  35. Zhang K, Zhong G, Dong J, Wang S, Wang Y (2019) Stock market prediction based on generative adversarial network. Procedia Comput Sci 147:400–406. https://doi.org/10.1016/j.procs.2019.01.256

    Article  Google Scholar 

  36. Follow JS (2019) Significance of ACF and PACF plots in time series analysis, pp 6–11. https://towardsdatascience.com/significance-of-acf-and-pacf-plots-in-time-series-analysis-2fa11a5d10a8. Accessed 20 July 2019

  37. Chollet F (2017) Deep learning with python. Manning

  38. Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Adv Neural Inf Process Syst 3:2672–2680. https://doi.org/10.3156/jsoft.29.5_177_2

    Article  Google Scholar 

  39. Du S, Li T, Yang Y et al (2019) Multivariate time series forecasting via attention-based encoder–decoder framework. Neurocomputing. https://doi.org/10.1016/j.neucom.2019.12.118

    Article  Google Scholar 

  40. “Online cloud workload traces,” http://ita.ee.lbl.gov/. Accessed 30 June 2018

  41. Barati M, Sharifian S (2015) A hybrid heuristic-based tuned support vector regression model for cloud load prediction. J Supercomput 71:4235–4259. https://doi.org/10.1007/s11227-015-1520-y

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Amirkabir University of Technology, 15914, Tehran, Iran

    Peyman Yazdanian & Saeed Sharifian

Authors
  1. Peyman Yazdanian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Saeed Sharifian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Saeed Sharifian.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yazdanian, P., Sharifian, S. E2LG: a multiscale ensemble of LSTM/GAN deep learning architecture for multistep-ahead cloud workload prediction. J Supercomput 77, 11052–11082 (2021). https://doi.org/10.1007/s11227-021-03723-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-021-03723-6

Keywords

Search

Navigation