The Journal of Supercomputing

, Volume 71, Issue 7, pp 2748–2766 | Cite as

Dynamically hierarchical resource-allocation algorithm in cloud computing environment

Article

Abstract

The development of big data challenges the computing power and communication capability of cloud architecture, but traditional resource-allocation algorithms perform poorly due to the large-scale communication among cloud nodes. In this paper, a dynamically hierarchical, resource-allocation algorithm is proposed for multiple cloud nodes collaborating in big data environment. Using fuzzy pattern recognition, the algorithm dynamically divides tasks and nodes into different levels based on computing power and storage factors. Thus a dynamically adjusted mapping is generated between tasks and nodes. When a new task arrives, only the nodes corresponding to the task level join in the bid. The algorithm takes advantages of dynamical hierarchy to reduce the communication traffic during resource allocation. Both theoretical and experimental results illustrate that the proposed algorithm outperforms the MinMin algorithm in terms of communication traffic and makespan.

Keywords

Cloud computing environment resource-allocation Task level Fuzzy pattern recognition 

Notes

Acknowledgments

This research was supported by the National Natural Science Foundation of China (Grant No. 61173160), Intel Industrial Liaison and Comprehensive Reform Program of Intel Semiconductor (US) LLC in 2014 of Ministry of Education of the People’s Republic of China. We would like to thank the anonymous reviewers for their attentive reading and for their constructive comments that have helped to further strengthen this paper.

References

  1. 1.
    Michael A et al (2010) A view of cloud computing. Commun ACM 53(4):50–58MathSciNetCrossRefGoogle Scholar
  2. 2.
    Seref S, Duygu S (2013) Big data: a review. In: 2013 international conference on collaboration technologies and systems, San Diego, CA, USA, pp 42–47Google Scholar
  3. 3.
    McAffe A, Brynolfsson E (2012) Strategy & competition big data: the management revolution. Harv Bus Rev 90(10):60–66, 68, 128Google Scholar
  4. 4.
    Buyya R, Yeo CS, Venugopal S, Broberg J, Brandic I (2009) Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Gener Comput Syst 25(6):599–616CrossRefGoogle Scholar
  5. 5.
    Amazon EC2 (2015). http://aws.amazon.com/cn/ec2/. Accessed 7 Feb 2015
  6. 6.
    IBM SmartCloud (2015). http://www.ibm.com/cloud-computing/cn/zh/index.html. Accessed 7 Feb 2015
  7. 7.
    Seokho S, Gihun J, Sung CJ (2013) An SLA-based cloud computing that facilitates resource allocation in the distributed data centers of a cloud provider. J Supercomput 64:606–637CrossRefGoogle Scholar
  8. 8.
    Nathuji R, Kansal A, Ghaffarkhah A (2010) Q-clouds: managing performance interference effects for QoS-aware clouds. In: 5th ACM European conference on computer systems (EuroSys 2010), Paris, April 13–16, 2010Google Scholar
  9. 9.
    Marx V (2013) The big challenges of big data. Nature 498(7453):255–260CrossRefGoogle Scholar
  10. 10.
    Sam M (2012) From databases to big data. IEEE Intenet Comput 12:089–7801Google Scholar
  11. 11.
    JiSu P, Hyongsoon K, Young-Sik J, Eunyoung L (2014) Two-phase grouping-based resource management for big data processing in mobile cloud computing. Int J Commun Syst 27:839–851CrossRefGoogle Scholar
  12. 12.
    Hassan MM, Song B, Hossain MS, Alamri A (2014) QoS-aware resource provisioning for big data processing in cloud computing environment. In: 2014 international conference on computational science and computational intelligence, Las Vegas, NV, USA, March 10–13, 2014Google Scholar
  13. 13.
    Simon SW, Jelena M (2014) Optimal application allocation on multiple public clouds. Comput Netw 68:138–148CrossRefGoogle Scholar
  14. 14.
    Liang Q, Zhang J, Zhang YH, Liang JM (2014) The placement method of resources and applications based on request prediction in cloud data center. Inf Sci 279:735–745MathSciNetCrossRefGoogle Scholar
  15. 15.
    Yin C, Huang BQ, Liu F et al (2011) Common key technology system of cloud manufacturing service platform for small and medium enterprises. Comput Integr Manuf Syst 17:495–503Google Scholar
  16. 16.
    Amit N, Sanjay C, Gaurav S (2012) Policy based resource allocation in IaaS cloud. Future Gener Comput Syst 28:94–103CrossRefGoogle Scholar
  17. 17.
    Christian V, Rodrigo NC, Dileban K, Rajkumar B (2012) Deadline-driven provisioning of resources for scientific applications in hybrid clouds with Aneka. Future Gener Comput Syst 28:58–65CrossRefGoogle Scholar
  18. 18.
    Guiyi W, Athanasios V, Vasilakos YZ, Naixue X (2010) A game-theoretic method of fair resource allocation for cloud computing services. J Supercomput 54:252–269CrossRefGoogle Scholar
  19. 19.
    Anton B, Jemal A, Rajkumar B (2012) Energy-aware resource allocation heuristics for efficient management of data centers for cloud computing. Future Gener Comput Syst 28:755–768CrossRefGoogle Scholar
  20. 20.
    Young CL, Albert YZ (2012) Energy efficient utilization of resources in cloud computing systems. J Supercomput 60:268–280CrossRefGoogle Scholar
  21. 21.
    Wu CM, Chang RS, Chan HY (2014) A green energy-efficient scheduling algorithm using the DVFS technique for cloud datacenters. Future Gener Comput Syst 31:141–147MATHCrossRefGoogle Scholar
  22. 22.
    Xiao Z, Song WJ, Chen Q (2013) Dynamic resource allocation using virtual machines for cloud computing environment. IEEE Trans Parallel Distrib Syst 24(6):1107–1117CrossRefGoogle Scholar
  23. 23.
    Daniel W, Odej K (2011) Exploiting dynamic resource allocation for efficient parallel data processing in the cloud. IEEE Trans Parallel Distrib Syst 22(6):985–997CrossRefGoogle Scholar
  24. 24.
    Ahmed S, Kenli L, Aijia O, Zhiyong L (2014) Proactive workload management in dynamic virtualized environments. J Comput Syst Sci 80:1504–1517MATHCrossRefGoogle Scholar
  25. 25.
    Haluk T, Salim H, Wu MY (2002) Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans Parallel Distrib Syst 13(3):260–274CrossRefGoogle Scholar
  26. 26.
    Wang ZJ, Liu JJ (2011) Research on multi-agent-based distributed task scheduling mechanism with multi-objective. J Dalian Univ Technol 51(5):755–760MATHGoogle Scholar
  27. 27.
    Wang ZJ, Fang T (2014) Task scheduling model based on multi-agent and multi-objective dynamical scheduling algorithm. J Netw 9(6):1588–1595Google Scholar
  28. 28.
    Grekovs R (2002) Methods of fuzzy pattern recognition. In: Scientific proceedings of RIGA Technical UniversityGoogle Scholar
  29. 29.
    Pedrycz W (1990) Fuzzy sets in pattern recognition: methodology and methods. Pattern Recognit 23(1–2):121–146CrossRefGoogle Scholar
  30. 30.
    Bezdek JC (1981) Pattern recognition with fuzzy objective function algorithms. Plenum Press, New YorkMATHCrossRefGoogle Scholar
  31. 31.
    Zhao RH, Lu XL, Wang M (2011) Safety evaluation of urban flood control system based on variable fuzzy pattern recognition. Adv Mater Res 159:264–269CrossRefGoogle Scholar
  32. 32.
    Cui XJ, Cao BY (2010) Using two-level fuzzy pattern recognition in the classification of convex quadrilateral. In: 2nd international conference on quantitative logic and soft computing, vol 82, pp 527–534Google Scholar
  33. 33.
    Fatma AO, Rasha MZ (2010) Dynamic task scheduling using fuzzy logic in distributed memory systems. In: 2010 7th international conference on informatics and systems, Cairo, Egypt, vol 5Google Scholar
  34. 34.
    Sepideh A, Ali M, Amir MR, Hamid B, Hengameh DT (2013) A new fuzzy negotiation protocol for grid resource allocation. J Newt Comput Appl. doi:10.1016/j.jnca.2012.12.030 Google Scholar
  35. 35.
    Guo FY, Yu L, Tian S, Yu J (2014) A workflow task scheduling algorithm based on the resources’ fuzzy clustering in cloud computing environment. Int J Commun Syst. doi:10.1002/dac.2743
  36. 36.
    Akihiro K, Nathan S (2008) Optimized algorithms for multi-agent routing. In: AAMAS ’08 proceedings of the 7th international joint conference on autonomous agents and multiagent systems, New York, pp 1585–1588Google Scholar
  37. 37.
    Rodrigo NC, Rajiv R, Anton B, César AFDR, Rajkumar B (2011) CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithm. Softw Pract Exp 41:23–50CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Department of Computer ScienceDalian University of TechnologyDalianChina

Personalised recommendations