Skip to main content
SpringerLink
Log in

RLPAS: Reinforcement Learning-based Proactive Auto-Scaler for Resource Provisioning in Cloud Environment

  • Published:
Mobile Networks and Applications Aims and scope Submit manuscript

Abstract

Public cloud system offers Infrastructure-as-a-Service (IaaS) to deliver the computational resources on demand. Resource requirements of a cloud environment are always fluctuating because of the dynamic nature of the arriving workload, and traditional reactive scaling techniques are employed to deal with this problem. Automated resource provisioning is an effective methodology for handling workload fluctuations by provisioning the resources on demand. Simple reactive approaches affect the performance of elastic system by over-provisioning the resources that substantially increase the costs whereas under-provisioning leads to starvation. An intelligent resource provisioning mechanism overcomes the stated issues by allocating necessary resources by learning the environment dynamically. In this article, RLPAS (Reinforcement Learning based Proactive Auto-Scaler) algorithm is proposed, and it is based on the existing Reinforcement Learning (RL)-SARSA algorithm that learns the environment in parallel and allocates the resources. The performance of RLPAS algorithm is validated using real workloads, and it outperforms existing auto-scaling approaches in terms of CPU utilization, response time and throughput. Further, it also converges at an optimal time step and proves to be feasible for the extensive range of real cloud applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Barham P, Dragovic B, Fraser K, Hand S, Harris T, Ho A, Neugebauer R, Pratt I, Warfield A (2003) Xen and the art of virtualization. In Proceedings of the nineteenth acm symposium on operating systems principles, SOSP ‘03. ACM: New York, pp. 164–177

  2. VMware virtualisation software. (Available from: http://www.vmware.com/in.html [Accessed date:17 Oct 2016]

  3. Abadi DJ, Ahmad Y, Balazinska M, Cetintemel U, Cherniack M, Hwang JH, Lindner W, Maskey AS, Rasin A, Ryvkina E, Tatbul N, Xing Y, Zdonik S (2005) The design of the borealis stream processing engine, in CIDR’05, pp. 277–289

  4. Wu E, Diao Y, Rizvi S (2006) High-performance complex event processing over streams, in Proceedings of the ACM SIGMOD international conference on Management of data, SIGMOD ‘06, New York, pp. 407–418

  5. Armbrust M, Fox A, Griffith R, Joseph AD, Katz R, Konwinski A, Lee G, Patterson D, Rabkin A, Stoica I, Zaharia M (2010) A view of cloud computing. Commun ACM 53(4):50–58

    Article  Google Scholar 

  6. Almeida V, Arlitt M, Rolia J (2002) Analyzing a Web-Based System’s Performance measures at Multiple Time Scales. SIGMETRICS Performance. Eval Rev 30(2):3–9

    Article  Google Scholar 

  7. Jacyno M, Bullock S, Payne T, Luck M (2007) Understanding decentralised control of resource allocation in a minimal multi-agent system, Autonomous Agents and Multi-Agent Systems (AAMAS 07), Article No. 208

  8. Right Scale Cloud Management. http://www.rightscale.com/ [Accessed date :17 Oct 2016]

  9. Botran TL, Alonso JM, Lozano JA (2014) A review of Auto-scaling Techniques for Elastic Applications in Cloud Environments. Journal of Grid Computing 12(4):559–592

    Article  Google Scholar 

  10. Khatua S, Ghosh A, Mukherjee N (2010) Optimizing the utilization of virtual resources in Cloud environment, IEEE International Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems, pp. 82–87

  11. Eldin AA, Tordsson J, Elmroth E. (2012) An adaptive hybrid elasticity controller for cloud infrastructures, In Network Operations and Management Symposium (NOMS), IEEE, pp. 204–212

  12. Patikirikorala T, Colman A. (2010) Feedback controllers in the cloud. APSEC 2010, Cloud workshop, Sydney

  13. Heinze M, Pappalardo V, Jerzak Z, Fetzer C (2014) Auto-scaling Techniques for Elastic Data Stream Processing, Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems, DEBS '14, Mumbai, India, pp. 318–321

  14. Chevaleyre Y, Dunne PE, Endriss U, Lang J, Lemaitre M, Maudet N, Padget J, Phelps S, Aguilar RJA, Sousa P (2006) Issues in Multiagent Resource Allocation. Informatica 30:3–31

    MATH  Google Scholar 

  15. Sutton RS, Barto AG (2012) Reinforcement Learning: An Introduction, First Edition

  16. Maillard OA, Coulom R, Preux P (2005) Parallelization of the TD(λ) Learning Algorithm, In Proceedings of the 7th European Workshop on Reinforcement Learning, EWRL7

  17. Rao J, Bu X, Xu CZ, Wang L, Yin G (2009) VCONF: a reinforcement learning approach to virtual machines auto-configuration. In Proceedings of the 6th international conference on Autonomic computing, ICAC '09, New York, pp. 137–146

  18. Barett E, Howley E, Duggan J (2012) Applying reinforcement learning towards automating resource allocation and application scalability in the cloud. Concurrency and Computation: Practice and Experience 25(12):1656–1674

    Article  Google Scholar 

  19. Tesauro G, Jong NK, Das R, Bennani MN (2007) On the use of hybrid reinforcement learning for autonomic resource allocation. Clust Comput 10:287–299

    Article  Google Scholar 

  20. Dutreilh X, Kirgizov S, Melekhova O, Malenfant J, Rivierre N, Truck I, (2011) Using Reinforcement Learning for Autonomic Resource Allocation in Clouds: towards a fully automated workfloow, In Seventh International Conference on Autonomic and Autonomous Systems, ICAS 2011, IEEE, pp. 67–74, May 2011

  21. Dutreilh X, Moreau A, Malenfant J, Rivierre N, Truck I (2010) From data center resource allocation to control theory and back, In Cloud Computing (CLOUD), IEEE, pp. 410–417

  22. Jamshidi P, Sharifloo AM, Metzger A, Estrada G (2015 Self-Learning Cloud Controllers: Fuzzy Q-Learning for Knowledge Evolution, Proceedings of the International Conference on Cloud and Autonomic Computing, Washington, pp. 208–211

  23. Veni T, Bhanu SMS (2016) Auto-scale: automatic scaling of virtualised resources using neuro-fuzzy reinforcement learning approach. International Journal of Big Data Intelligence 3(3)

  24. Bu X, Rao J, Xu CZ (2013) Coordinated Self-Configuration of Virtual Machines and Appliances Using a Model-Free Learning Approach. IEEE Trans Parallel Distrib Syst 24(4):681–690

    Article  Google Scholar 

  25. RUBiS: Rice University Bidding System Benchmark, http://rubis.ow2.org/. Accessed 17 Oct 2016

  26. RUBBoS: Bulletin Board Benchmark, http://jmob.ow2.org/rubbos.html. Accessed 17 Oct 2016

  27. Olio/PHP, https://svn.apache.org/repos/asf/incubator/olio/docs/php_setup_kit.html. Accessed 17 Oct 2016

  28. Bibal Benifa JV, Dejey D (2017) Performance Improvement of MapReduce for Heterogeneous Clusters Based on Efficient Locality and Replica Aware Scheduling (ELRAS) Strategy. Wireless Pers Commun. https://doi.org/10.1007/s11277-017-3953-5

  29. Arabnejad H, Jamshidi P, Estrada G, Ioini NE, Pahl C (2016) An Auto-Scaling Cloud Controller using Fuzzy Q-Learning - Implementation in OpenStack, Service-Oriented and Cloud Computing, pp. 152–167, Vienna, September 5–7

  30. Kretchmar RM (2002) Parallel Reinforcement Learning, in The 6th World Conference on Systemics, Cybernetics, and Informatics

  31. Amoui M, Salehie M, Mirarab S, Tahvildari L (2005) Adaptive Action Selection in Autonomic Software Using Reinforcement Learning, Proceedings of Second International Conference on Autonomic Computing, Washington, USA

  32. Tesauro G, Das R, Walsh WE, Kephart JO (2005) Utility-Function-Driven Resource Allocation in Autonomic Systems”, Proceedings of the Second International Conference on Autonomic Computing (ICAC’05), Washington, USA

Download references

Acknowledgements

Anna University Regional Campus, Tirunelveli’s support for the work in terms of computing facilities is greatly acknowledged.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Anna University Regional Campus, Tirunelveli, 627007, India

    J. V. Bibal Benifa & D. Dejey

Authors
  1. J. V. Bibal Benifa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. D. Dejey
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to J. V. Bibal Benifa.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bibal Benifa, J.V., Dejey, D. RLPAS: Reinforcement Learning-based Proactive Auto-Scaler for Resource Provisioning in Cloud Environment. Mobile Netw Appl 24, 1348–1363 (2019). https://doi.org/10.1007/s11036-018-0996-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11036-018-0996-0

Keywords

Search

Navigation