Advertisement

Effect of Live Migration on Virtual Hadoop Cluster

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10722)

Abstract

Emerging computational requirement for large scale data analysis has resulted in the importance of big data processing. Meanwhile, with virtualization it is now feasible to deploy Hadoop in private or public cloud environment which offers unique benefits like scalability, high availability etc. Live migration is an important feature provided by virtualization that migrate a running VM from one physical host to another to facilitate load balancing, maintenance, server consolidation and avoid SLA violation of VM. However, live migration adds overhead and degrades the performance of the application running inside the VM. This paper discusses the performance of Hadoop when VMs are migrated from one host to another. Experiment shows that job completion time, average downtime as well as average migration time gets increased with increase in the number of VMs that are migrated.

Keywords

Virtualization SAN Live migration Hadoop MapReduce Pre-copy 

1 Introduction

Cloud has appeared as a powerful computing technology in the world of distributed computing. With the advancement in cloud and virtualization technology more and more computation is assumed to be done on the cloud. Virtualization is the core driving technology behind cloud that allows running multiple OS instances on the same physical machine concurrently and facilitate load balancing, server consolidation, live migration etc.

Meanwhile, to process large and distributed dataset Google proposed a scalable programing model known as MapReduce [1]. Hadoop [2], is an open source implementation of MapReduce. It has computational power of MapReduce with HDFS(Hadoop Distributed File System). So, with the advancement in virtualization technology and big data analysis it is now feasible to use virtual machine for big data computing [3, 4].

One of the feature provided by virtualization is live VM migration that allows allocation and reallocation of server resources by moving VM from one physical host to another. However, existing migration techniques focused on migration of a single VM only. When a group of VM [5] or cluster need to be migrated simultaneously these techniques may prove inefficient. This paper analyzes the performance of Hadoop cluster when different number of VMs are migrated within the same subnet. The rest of the paper is organized as follows. Section 2 briefly describes the background work. Section 3 presents the related work. Section 4 presents performance analysis of Hadoop on the cloud. Finally, Sect. 5 conclude our work.

2 Background

MapReduce frees user from the complicated task of parallelization, failure handling and data distribution by requiring user to specify map and reduce function only. Map function processes key/value pairs to generate intermediate data which is another set of key/value pairs. This intermediate data is further processed by reduce function which merges values associated with same key to generate the final result.

Live migration primarily transfers VM’s memory, CPU and I/O state from source to destination host. It can be further classified as pre-copy, post-copy and hybrid migration. With pre-copy [6] migration the dirty pages are iteratively transfered across the network followed by a short suspend and copy phase. This iterative phase ends when some terminating condition is reached. This technique is the default migration technique for many hypervisors including Xen which is used in this paper. On the other side, post-copy VM migration first captures and transfers the minimum state to the destination server where the VM is resumed and the remaining memory pages are fetched from the source depending on the read/write request. Lastly, hybrid migration scheme follows bounded number of pre-copy round to transfer a subset of pages followed by a short suspend & resume phase. Next, the remaining pages are pulled from the source.

3 Related Work

Virtualization is useful but it adds overhead to run VM. Hwang et al. in [7] have performed the analysis of the overheads offered by different hypervisor under hardware assisted virtualization. The author has concluded that the overhead incurred by different hypervisor vary depending upon the application hosted on it.

Jhonson and Chiu in [8] have discussed migration of MapReduce task across hosts using pause, migrate and resume strategy. It checkpoint intermediate state and metadata information to facilitate migration. However, this technique transfers only Hadoop MapReduce task, not the entire VM and incurs significant overhead.

Ibrahim et al. in [3] have compared Hadoop performance on physical and virtual machine. The author has discussed that the performance of Hadoop on VM is degraded due to the overhead added due to virtualization. To reduce the cost and improve the performance Kambatla et al. in [9] have optimized the Hadoop provisioning on cloud. However, these approaches does not address the performance of any application when multiple VMs are migrated. This paper studies the performance of Hadoop cluster with live VM migration.
Table 1.

Job completion time(in sec) with different cluster and file size(in MB)

CS:Cluster Size

CS

File Size

128

256

512

1024

1

60

61

104

150

2

60

64

118

154

4

68

70

120

179

6

63

65

173

259

8

63

64

222

600

Table 2.

Reduce task completion time(in msec) with different cluster and file size(in MB)

CS:Cluster Size

CS

File Size

128

256

512

1024

1

4287

4351

47171

66349

2

4330

4813

58137

67581

4

4533

7787

55502

105367

6

5283

5923

55089

106305

8

4759

4690

115218

155218

Table 3.

Map task completion time(in msec) with different cluster and file size(in MB)

CS:Cluster Size

CS

File Size

128

256

512

1024

1

81664

160186

531106

1500951

2

77861

161892

632904

1486645

4

91543

174938

604315

2277456

6

80069

170504

939676

2476982

8

80686

167409

1069634

2557333

4 Performance Analysis

In this section, experimental setup and performance of Hadoop cluster on cloud is discussed.

4.1 Experimental Configuration

The private cloud setup has 6 blade servers each with 2x4 core Intel Xeon CPU E5-2665 @ 2.40GHz processor, 270 GB local disk space and 2 physical NIC with 1 Gbps in both direction. Each blade has hyper threading enabled with two threads running on each CPU core. Each server has Xen 6.5 hypervisor and Linux with kernel version 3.10.0 running in dom0. Ubuntu 10.04 is configured with Hadoop 2.6.1 on each VM. The disk image of each VM is stored on the shared storage. So, when VM migrate disk image is not transfered across hosts. The performance of Hadoop on cloud is analyzed with different cluster size of 2, 4, 6 and 8 nodes. Four different files of 128, 256, 512 and 1024 MB are used as an input to Wordcount program running on Hadoop. Next to study the impact of migration on Hadoop, experiments are conducted on a cluster with 8 slave and 1 master node.

4.2 Performance of Hadoop on Cloud

This section discusses the performance of Hadoop with different cluster sizes. As shown in Table 1, job completion time(in sec) decreases with increase in cluster size. However with small file size, this decrease is small as Hadoop split the file into chunks(known as splits) depending upon the split size(default 64 MB) and for each split it usually has one map task running on it. With small file size the number of splits and accordingly the number of map tasks are less. So, even if the number of VMs in the cluster are increased the job completion time is not reduced much as number of map tasks are not sufficient to utilize the increased computational power. Further, Tables 2 and 3 shows the time(in msec) taken by map and reduce task to process different input file on different cluster size.
Fig. 1.

Effect of live migration on Hadoop cluster

4.3 Performance of Hadoop with VM Migration

This section evaluates the performance of Hadoop cluster when different number of VMs are migrated within the same subnet. Three performance parameters job completion time, migration time and downtime are used to estimate the impact of migration on Hadoop.

As shown in Fig. 1(a), job completion increases with increase in number of VM migration. This is due to the network contention between Hadoop traffic and migration traffic at source and destination. Also, for same number of VMs that are migrated the performance is more degraded when input file size is large as the amount of dirty memory generated with large file size is more. Figures 1(b) and (c) shows that time taken by map and reduce task increases with increase in the number of VM migration which is again due to the network contention of map-reduce traffic with migration traffic.

Next, the effect of migration on average downtime and average migration time is analyzed. The performance of Hadoop cluster running MapReduce job on 1024 MB input file is compared with an idle cluster. As shown in Table 4 the average downtime increases with increase in number of migrations. It is because of sharing of bandwidth reserved for migration among the VMs. So, with more VMs the bandwidth available per VM is reduced resulting in increased migration time. Further, Table 4 shows that average migration time increases with increase in number of VM migrations which is due to increase in migration traffic and reduction in per VM available bandwidth for migration.
Table 4.

Effect of migration on average downtime and migration time

ADT:Avg. Downtime, AMT:Avg. Migration time

Node

ADT/AMT

ADT(idle)

(msec)

ADT(running)

(msec)

AMT(idle)

(sec)

AMT(running)

(sec)

2

219

247

21

22

4

384

408

34

42

6

411

430

48

68

8

424

613

60

81

Table 5.

Effect of migrating master and slave VM on job completion time(in sec) with different file size

Node

File size

128 MB

256 MB

512 MB

1024 MB

Master

65

64

129

182

Slave

61

61

104

150

Lastly, the effect of migrating master and slave node on job completion time is evaluated. As shown in Table 5 migrating the master increases the job completion time for a give file size in comparison to the slave. This is because the master is in frequent communication with the slaves so migrating the master has more impact on performance and job completion time.

5 Conclusion

In this paper a series of experiments are conducted to analyze the performance of Hadoop with VM migration. Experiment shows that job completion time, average downtime as well as average migration time gets increased with increase in the number of VMs that are migrated. Further, for the same number of VM migration, job completion time is higher with large file size. So, in case multiple Hadoop clusters of same size are running and decision has to be made to select appropriate VM to migrate, Hadoop VM running on small file size should be selected to reduce the impact of migration on performance. Further, among master or slave, slave VM should be selected for migration as the job completion time is higher when master is migrated. In future, experiment will be conducted to analyze the performance of Hadoop cluster with VM migration across WAN where network latency is high and no shared storage is available.

References

  1. 1.
    Dean, J., Ghemawat, S.: Mapreduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)CrossRefGoogle Scholar
  2. 2.
    Borthakur, D.: The hadoop distributed file system: architecture and design. Hadoop Proj. Website 11(2007), 21 (2007)Google Scholar
  3. 3.
    Ibrahim, S., Jin, H., Lu, L., Qi, L., Wu, S., Shi, X.: Evaluating MapReduce on virtual machines: the Hadoop case. In: Jaatun, M.G., Zhao, G., Rong, C. (eds.) CloudCom 2009. LNCS, vol. 5931, pp. 519–528. Springer, Heidelberg (2009).  https://doi.org/10.1007/978-3-642-10665-1_47 CrossRefGoogle Scholar
  4. 4.
    Xu, G., Xu, F., Ma, H.: Deploying and researching hadoop in virtual machines. In: International Conference on Automation and Logistics (ICAL), 2012 IEEE, pp. 395–399. IEEE (2012)Google Scholar
  5. 5.
    Ye, K., Jiang, X., Chen, S., Huang, D., Wang, B.: Analyzing and modeling the performance in xen-based virtual cluster environment. In: 12th IEEE International Conference on High Performance Computing and Communications (HPCC), 2010, pp. 273–280. IEEE (2010)Google Scholar
  6. 6.
    Clark, C., Fraser, K., Hand, S., Hansen, J.G., Jul, E., Limpach, C., Pratt, I., Warfield, A.: Live migration of virtual machines. In: Proceedings of the 2nd Conference on Symposium on Networked Systems Design & Implementation, vol. 2, pp. 273–286. USENIX Association (2005)Google Scholar
  7. 7.
    Hwang, J., Zeng, S., Wu, F.Y., Wood, T.: A component-based performance comparison of four hypervisors. In: IFIP/IEEE International Symposium on Integrated Network Management (IM 2013), 2013, pp. 269–276. IEEE (2013)Google Scholar
  8. 8.
    Johnson, C., Chiu, D.: Hadoop in flight: Migrating live mapreduce jobs for power-shifting data centers. In: IEEE 9th International Conference on Cloud Computing (CLOUD), 2016, pp. 92–99. IEEE (2016)Google Scholar
  9. 9.
    Kambatla, K., Pathak, A., Pucha, H.: Towards optimizing hadoop provisioning in the cloud. HotCloud 9, 12 (2009)Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.Motilal Nehru National Institute of Technology AllahabadAllahabadIndia

Personalised recommendations