Abstract
We study machine covering problem in MapReduce system. Each job consists of two sets of tasks, namely the map tasks and reduce tasks. A job’s reduce tasks can only be processed after all its map tasks are finished. The map tasks are fractional, i.e., they can be arbitrarily split and processed on different machines in parallel. Our goal is to maximize the minimum machine completion time. We consider two variants of the problem, namely the cases involving preemptive reduce tasks and non-preemptive reduce tasks. For preemptive reduce tasks, we present optimal solution algorithms for the problem on two and three machines. For non-preemptive reduce tasks, we provide an approximation algorithm with a tight worse-case ratio of \(\frac{4}{3}\) for the problem on two machines.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Chang H, Kodialam M, Kompella RR et al (2015) Scheduling in MapReduce-like systems for fast completion time. Proce IEEE INFOCOM 2(3):3074–3082
Chen C, Xu Y, Zhu Y, Sun C (2017) Online MapReduce scheduling problem of minimizing the makespan. J Comb Optim 33:590–608
Csirik J, Kellerer H, Woeginger G (1992) The exact LPT-bound for maximizing the minimum completion time. Oper Res Lett 11:281–287
Dean J, Ghemawat S (2004) MapReduce: simplified data processing on large clusters. Proc Oper Syst Des Implement (OSDI) 51(1):107–113
Deuermeyer B, Friesen D, Langston M (1982) Scheduling to maximize the minimum processor finish time in a multiprocessor system. SIAM J Discrete Methods 3:190–196
Jiang Y, Tan Z, He Y (2005) Preemptive machine covering on parallel machines. J Comb Optim 10(4):345–363
Jiang Y, Zhu Y, Wu W, Li D (2017) Makespan minimization for MapReduce systems with different servers. Future Gener Comput Syst 67:13–21
Jiang Y, Zhou W, Zhou P (2018) An optimal preemptive algorithm for online MapReduce scheduling on two parallel machines. Asia Pac J Oper Res 35(3):185003
Le Y, Liu J, Ergun F, Wang D (2014) Online load balancing for MapReduce with skewed data input. In: INFOCOM’14, 2004–2012. IEEE
Luo T, Zhu Y, Wu W, Xu Y, Du D (2017) Online makespan minimization in MapReduce-like systems with complex reduce tasks. Optim Lett 11:271–277
Woeginger G (1997) A polynomial time approximation scheme for maximizing the minimum machine completion time. Oper Res Lett 20:149–154
Zheng Y, Shroff NB, Sinha P (2013) A new analytical technique for designing provably efficient MapReduce schedulers. In: Proceedings of IEEE, INFOCOM, 2013. IEEE, pp 1600–1608
Zhu Y, Jiang Y, Ding L, Teredesai A, Li D, Lee W (2014) Minimizing makespan and total completion time in MapReduce-like systems. In: INFOCOM 14. IEEE, pp 2166–2174
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China 11571013.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Jiang, Y., Zhou, P. & Zhou, W. MapReduce machine covering problem on a small number of machines. J Comb Optim 38, 1066–1076 (2019). https://doi.org/10.1007/s10878-019-00436-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10878-019-00436-8