Abstract
Workload hotspot detection is a key component of virtual machine (VM) management in virtualized environment. One of its challenges is how to effectively collect the resource usage of VMs. Also, since data centers usually have hundreds or even thousands of nodes, workload hotspot detection must be able to handle a large amount of monitoring data. In this paper, we address these two challenges. We first present a novel approach to VM memory monitoring. This approach collects memory usage data by walking through the page tables of VMs and by checking the present bit of page table entry. Second, we present a MapReduce-based approach to efficiently analyze a large amount of resource usage data of VMs and nodes. Leveraging the power of parallelism and robustness of MapReduce can significantly accelerate the detection of hotspots. Extensive simulations have been performed to evaluate the proposed approaches. The simulation results show that our approach can achieve effective estimation of memory usage with low overhead and can quickly detect workload hotspots.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Barham, P., Dragovic, B., Fraser, K.: Xen and the Art of Virtualization. In: Proceedings the nineteenth ACM symposium on operating systems principles, Vol. 37 No. 5, Dec 2003
Hwang, T., Shin, Y., Son, K., Park, H.: Design of a hypervisor-based rootkit detection method for virtualized systems in cloud computing environments In: Proceedings AASRI Winter International Conference on Engineering and Technology (aasri-weit 2013), Atlantis Press (2013)
Ayad, A., Dippel, U.: Agent-based monitoring of virtual machines. In: Proceedings 2010 International Symposium in Information Technology (ITSim), June 2010
Viratanapanu, A., Hamid, A.K.A., Kawahara, Y., Asami, T.: On Demand Fine Grain Resource Monitoring System for Server Consolidation. In: Proceedings kaleidoscope: Beyond the Internet? - Innovations for Future Networks and Services, Dec 2010
Wood, T., Shenoy, P., Venkataramani, A., Yousif, M.: Black-box and gray-box strategies for virtual machine migration. In: Proceedings The 4th USENIX conference on Networked Systems Design & Implementation, 2007
Challa, N.: Detecting workload hotspots and dynamic provisioning of virtual machines in clouds In: Proceedings 2012 IEEE International Conference on Cloud Computing in Emerging Markets (CCEM), 2012
Tusa, F., Paone, M., Villari, M.: CLEVER: A cloud-enabled virtual environment. In: Proceedings 2010 IEEE Symposium on Computers and Communications (ISCC), 2010
Wang, C., Schwan, K., Talwar, V., Eisenhauer, G.: A flexible architecture integrating monitoring and analytics for managing large-scale data centers. In: Proceedings 8th ACM international conference on Autonomic computing. ACM, 2011
Bohm, S., Engelmann, C., Scott, S.L.: Aggregation of real-time system monitoring data for analyzing large-scale parallel and distributed computing environments. In: Proceedings The 12th International Conference on High Performance Computing and Communications, Sept 2010
Apache Hadoop: http://hadoop.apache.org/
Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The hadoop distributed file system. In: Proceedings The 26th Symposium on Mass Storage Systems and Technologies (MSST), May 2010
Shafer, J., Rixner, S., Cox, A.L.: The hadoop distributed silesystem: Balancing portability and performance. In: Proceedings Performance Analysis of Systems & Software (ISPASS), March 2010
Borthakur, D.: The hadoop distributed file system: Architecture and design, http://hadoop.apache.org/common/docs/r0.18.0/
Dean, J., Ghemawat, S.: MapReduce: Simplified data processing on large clusters. In: Proceedings The sixth Symposium on Operating System Design and Implementation, Dec 2004
Shen, Q., Wan, M., Zhang, Z., Qing, S.: ”A covert channel using event channel state on xen hypervisor” Information and Communications Security, pp. 125–134. Springer International Publishing, (2013)
Waldspurger, C.A.: Memory resource management in VMware ESX Server. In: Proceedings The 5th symposium on Operating systems design and implementation, 2002
Ye, L., Lu, G., Kumar, S., Gniady, C., Hartman, J.: ”Energy-efficient storage in virtual machine environments” ACM Sigplan Notices. Vol. 45. No. 7. ACM, 2010
Zhao, W., Wang, Z.: Dynamic memory balancing for virtual machines. In: Proceedings The 2009 ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, 2009
Niu, Y., Yang, C., Cheng, X.: Dynamic memory demand estimating based on the guest operating system behaviors for virtual machines. In: Proceedings The ninth International Symposium on Parallel and Distributed Processing with Applications, May 2011
Xen Architecture Overview: http://wiki.xensource.com/xenwiki/
Garcia, A., Kalva, H., Furht, B.: A study of transcoding on cloud environments for video content delivery. In: Proceedings The 2010 ACM multimedia workshop on Mobile cloud media computing, 2010
Boulon, J., Konwinski, A., Qi, R., Rabkin, A., Yang, E., Yang, M.: Chukwa: A large-scale monitoring system. In: Proceedings Cloud Computing and its Applications, 2008
Payne, B.D., Carbone, M.D.P.A., Lee, W.: Secure and flexible monitoring of virtual machines. In: Proceedings The Twenty-Third Annual Computer Security Applications Conference, Dec 2007
Gorman, M.: Understanding the linux virtual memory, Manager, Feb 2004
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lei, Z., Hu, B., Guo, J. et al. Scalable and efficient workload hotspot detection in virtualized environment. Cluster Comput 17, 1253–1264 (2014). https://doi.org/10.1007/s10586-014-0383-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10586-014-0383-y