Abstract
Emerging computational requirement for large scale data analysis has resulted in the importance of big data processing. Meanwhile, with virtualization it is now feasible to deploy Hadoop in private or public cloud environment which offers unique benefits like scalability, high availability etc. Live migration is an important feature provided by virtualization that migrate a running VM from one physical host to another to facilitate load balancing, maintenance, server consolidation and avoid SLA violation of VM. However, live migration adds overhead and degrades the performance of the application running inside the VM. This paper discusses the performance of Hadoop when VMs are migrated from one host to another. Experiment shows that job completion time, average downtime as well as average migration time gets increased with increase in the number of VMs that are migrated.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Dean, J., Ghemawat, S.: Mapreduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)
Borthakur, D.: The hadoop distributed file system: architecture and design. Hadoop Proj. Website 11(2007), 21 (2007)
Ibrahim, S., Jin, H., Lu, L., Qi, L., Wu, S., Shi, X.: Evaluating MapReduce on virtual machines: the Hadoop case. In: Jaatun, M.G., Zhao, G., Rong, C. (eds.) CloudCom 2009. LNCS, vol. 5931, pp. 519–528. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-10665-1_47
Xu, G., Xu, F., Ma, H.: Deploying and researching hadoop in virtual machines. In: International Conference on Automation and Logistics (ICAL), 2012 IEEE, pp. 395–399. IEEE (2012)
Ye, K., Jiang, X., Chen, S., Huang, D., Wang, B.: Analyzing and modeling the performance in xen-based virtual cluster environment. In: 12th IEEE International Conference on High Performance Computing and Communications (HPCC), 2010, pp. 273–280. IEEE (2010)
Clark, C., Fraser, K., Hand, S., Hansen, J.G., Jul, E., Limpach, C., Pratt, I., Warfield, A.: Live migration of virtual machines. In: Proceedings of the 2nd Conference on Symposium on Networked Systems Design & Implementation, vol. 2, pp. 273–286. USENIX Association (2005)
Hwang, J., Zeng, S., Wu, F.Y., Wood, T.: A component-based performance comparison of four hypervisors. In: IFIP/IEEE International Symposium on Integrated Network Management (IM 2013), 2013, pp. 269–276. IEEE (2013)
Johnson, C., Chiu, D.: Hadoop in flight: Migrating live mapreduce jobs for power-shifting data centers. In: IEEE 9th International Conference on Cloud Computing (CLOUD), 2016, pp. 92–99. IEEE (2016)
Kambatla, K., Pathak, A., Pucha, H.: Towards optimizing hadoop provisioning in the cloud. HotCloud 9, 12 (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Singh, G., Singh, A.K. (2018). Effect of Live Migration on Virtual Hadoop Cluster. In: Negi, A., Bhatnagar, R., Parida, L. (eds) Distributed Computing and Internet Technology. ICDCIT 2018. Lecture Notes in Computer Science(), vol 10722. Springer, Cham. https://doi.org/10.1007/978-3-319-72344-0_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-72344-0_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-72343-3
Online ISBN: 978-3-319-72344-0
eBook Packages: Computer ScienceComputer Science (R0)