Advertisement

Queuing-Oriented Job Optimizing Scheduling In Cloud Mapreduce

  • Ting-Qin HeEmail author
  • Li-Jun Cai
  • Zi-Yun Deng
  • Tao Meng
  • XuAn Wang
Conference paper
Part of the Lecture Notes on Data Engineering and Communications Technologies book series (LNDECT, volume 1)

Abstract

Cloud MapReduce, as an implementation of MapReduce framework on Cloud for big data analysis, is facing the unknown job makespan and long wait time problem, which have seriously affected the service quality. The Inefficient virtual machine allocation is one critical causing factor. Based on the M/M/1 model, a new queuing equation is built to ensure the virtual machine with the high efficiency. By jointing queuing equation and objectives function, a two variables equation group is designed to compute the desired virtual machine number for different jobs. According to the desired virtual machine number of each job, we developed a queuing-oriented job optimizing scheduling algorithm, called QTJS, to optimal job scheduling and enhance the resource utilization in Cloud MapReduce. Extensive experiments show that our QTJS algorithm consumes less job execution time and performs better efficiency than other three algorithms.

Keywords

map-reduce cloud computing queuing theory job scheduling 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    J. Dean and S. Ghemawat, “MapReduce: Simplified data processing on large clusters,” in Proc. 6th Symp. Oper. Syst. Design Implementation (OSDI), Dec. 2004, pp. 137–150.Google Scholar
  2. 2.
    Thabet M, Boufaida M, Kordon F. An approach for developing an interoperability mechanism between cloud providers [J]. International Journal of Space-Based and Situated Computing 27, 2014, 4(2): 88-99.Google Scholar
  3. 3.
    Afrati F N, Ullman J D. Optimizing joins in a map-reduce environment[C]//Proceedings of the 13th International Conference on Extending Database Technology. ACM, 2010: 99-110.Google Scholar
  4. 4.
    Verma A, Cherkasova L, Campbell R H. Orchestrating an ensemble of MapReduce jobs for minimizing their makespan[J]. Dependable and Secure Computing, IEEE Transactions on, 2013, 10(5): 314-327.Google Scholar
  5. 5.
    Xilong Q, Peng X. An energy-efficient virtual machine scheduler based on CPU sharereclaiming policy [J]. International Journal of Grid and Utility Computing, 2015, 6(2): 113-120.Google Scholar
  6. 6.
    Hacker T J, Romero F, Nielsen J J. Secure live migration of parallel applications using container-based virtual machines[J]. International Journal of Space-Based and Situated Computing 1, 2012, 2(1): 45-57.Google Scholar
  7. 7.
    Hashem I A T, Yaqoob I, Anuar N B, et al. The rise of “big data” on cloud computing: review and open research issues [J]. Information Systems, 2015, 47: 98-115.Google Scholar
  8. 8.
    Manoharan M, Selvarajan S. An efficient methodology to improve service negotiation in cloud environment [J]. International Journal of Grid and Utility Computing, 2015, 6(3-4): 150-158.Google Scholar
  9. 9.
    Dahiphale D, Karve R, Vasilakos A V, et al. An advanced MapReduce: cloud MapReduce, enhancements and applications [J]. Network and Service Management, IEEE Transactions on, 2014, 11(1): 101-115.Google Scholar
  10. 10.
    Liu H, Orban D. Cloud MapReduce : a MapReduce implementation on top of a cloud operating system[C]//Proceedings of the 2011 11th IEEE/ACM international symposium on cluster, cloud and grid computing. IEEE Computer Society, 2011: 464-474.Google Scholar
  11. 11.
    Verma A, Kaushal S. Deadline constraint heuristic-based genetic algorithm for workflow scheduling in cloud [J]. International Journal of Grid and Utility Computing, 2014, 5(2): 96-106.Google Scholar
  12. 12.
    Verma A, Cho B, Zea N, et al. Breaking the MapReduce stage barrier [J]. Cluster computing, 2013, 16(1): 191-206.Google Scholar
  13. 13.
    Saez J C, Pousa A, Castro F, et al. ACFS: a completely fair scheduler for asymmetric single-isa multicore systems[C]//Proceedings of the 30th Annual ACM Symposium on Applied Computing. ACM, 2015: 2027-2032.Google Scholar
  14. 14.
    Shengjun X, Delong W, Suhong S. PCSP: A Preemptive Capacity Scheduler Policy for Scheduling Hadoop Jobs[J]. International Journal of Grid and Distributed Computing, 2015, 8(5): 33-46.Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Ting-Qin He
    • 1
    Email author
  • Li-Jun Cai
    • 1
  • Zi-Yun Deng
    • 2
  • Tao Meng
    • 1
  • XuAn Wang
    • 3
  1. 1.College of Information Science and EngineeringHunan UniversityChangshaChina
  2. 2.College of Electrical and Information EngineeringHunan UniversityChangshaChina
  3. 3.Department of electronic technologyEngineering University of CAPFXi’anChina

Personalised recommendations