Advertisement

Optimal online algorithms for MapReduce scheduling on two uniform machines

  • Yiwei Jiang
  • Ping Zhou
  • T. C. E. Cheng
  • Min Ji
Original paper

Abstract

We study online scheduling on two uniform machines in the MapReduce system. Each job consists of two sets of tasks, namely the map tasks and reduce tasks. A job’s reduce tasks can only be processed after all its map tasks are finished. The map tasks are fractional, i.e., they can be arbitrarily split and processed on different machines in parallel. Our goal is to find a schedule that minimizes the makespan. We consider two variants of the problem, namely the cases involving preemptive reduce tasks and non-preemptive reduce tasks. We provide lower bounds for both variants. For preemptive reduce tasks, we present an optimal online algorithm with a competitive ratio of \(\frac{\sqrt{s^{2}+2s+5}+1-s}{2}\), where \(s\ge 1\) is the ratio between the speeds of the two machines. For non-preemptive reduce tasks, we show that the \({ LS}\)-like algorithm is optimal and its competitive ratio is \(\frac{2s+1}{s+1}\) if \(s<\frac{1+\sqrt{5}}{2}\) and \(\frac{s+1}{s}\) if \(s\ge \frac{1+\sqrt{5}}{2}\).

Keywords

Big data MapReduce scheduling Online algorithm Competitive ratio 

Notes

Acknowledgements

Jiang was supported in part by the National Natural Science Foundation of China (Grant No. 11571013). Cheng was supported in part by The Hong Kong Polytechnic University under the Fung Yiu King—Wing Hang Bank Endowed Professorship in Business Administration. Ji was supported in part by Zhejiang Provincial Natural Science Foundation of China (Grant No. LR15G010001).

References

  1. 1.
    Chang, H., Kodialam, M., Kompella, R.R., Lakshman, T.V.: Scheduling in MapReduce-like systems for fast completion time. In: Proceedings of INFOCOM’14, pp. 3074–3082 (2015)Google Scholar
  2. 2.
    Chen, C., Xu, Y., Zhu, Y., Sun, C.: Online MapReduce scheduling problem of minimizing the makespan. J. Comb. Optim. 33(2), 590–608 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Proc. Oper. Syst. Des. Implement. 51(1), 107–113 (2004)Google Scholar
  4. 4.
    Epstein, L., Noga, J., Seiden, S., Sgall, J., Woeginger, G.: Randomized on-line scheduling on two uniform machines. J. Sched. 4, 71–92 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Huang, J., Zheng, F., Xu, Y., Liu, M.: Online MapReduce processing on two identical parallel machines. J. Combin. Optim. 35(1), 216–223 (2018)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Jiang, Y., Zhu, Y., Wu, W., Li, D.: Makespan minimization for MapReduce systems with different servers. Future Gener. Comput. Syst. 67, 13–21 (2017)CrossRefGoogle Scholar
  7. 7.
    Jiang, Y., Zhou, W., Zhou, P.: An optimal preemptive algorithm for online MapReduce scheduling on two parallel machines. Asia-Pac. J. Oper. Res. 35(3), 1850013 (2018)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Luo, T., Zhu, Y., Wu, W., Xu, Y., Du, D.: Online makespan minimization in MapReduce-like systems with complex reduce tasks. Optim. Lett. 11, 271–277 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Le, Y., Liu, J., Ergun, F., Wang, D.: Online load balancing for MapReduce with skewed data input. In: Proceeding of INFOCOM’14, pp. 2004–2012 (2014)Google Scholar
  10. 10.
    Moseley, B., Dasgupta, A., Kumar, R., Sarlós, T.: On scheduling in Map-Reduce and flowshops. In: Proceedings of the Twenty-Third Annual ACM Symposium on Parallelism in Algorithms and Architectures, pp. 289–298 (2011)Google Scholar
  11. 11.
    Wen, J., Du, D.: Preemptive on-line scheduling for two uniform processors. Oper. Res. Lett. 23, 113–116 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Zheng, Y., Shroff, N.B., Sinha, P.: A new analytical technique for designing provably efficient MapReduce schedulers. In: Proceeding of INFOCOM’13, pp. 1600–1608 (2013)Google Scholar
  13. 13.
    Zhu, Y., Jiang, Y., Wu, W., Ding, L., Teredesai, A., Li, D., Lee, W.: Minimizing makespan and total completion time in MapReduce-like systems. In: Proceeding of INFOCOM’14, pp. 2166–2174 (2014)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Management and E-Business, Contemporary Business and Trade Research CenterZhejiang Gongshang UniversityHangzhouChina
  2. 2.College of HumanitiesZhejiang Business CollegeHangzhouChina
  3. 3.Department of Logistics and Maritime StudiesThe Hong Kong Polytechnic UniversityKowloonHong Kong

Personalised recommendations