Journal of Grid Computing

, Volume 11, Issue 2, pp 167–186 | Cite as

Adaptive Online Compression in Clouds—Making Informed Decisions in Virtual Machine Environments

  • Matthias Hovestadt
  • Odej Kao
  • Andreas Kliem
  • Daniel Warneke
Article

Abstract

Infrastructure as a Service clouds often use virtual machines to host different customers on the same physical hardware. This form of resource sharing can lead to unpredicatable performance degradations for the individual customer, especially with regard to data-intensive applications, which heavily depend on stable I/O characteristics. One traditional approach to cope with I/O fluctuations is adaptive online compression. In this paper we present a new scheme for adaptive online compression which has been explicitly designed to work in co-located virtual machine environments. In contrast to existing adaptive online compression schemes, the decision model of our approach does not rely on the system metrics CPU utilization and I/O bandwidth, which we demonstrate to be often displayed inaccurately inside XEN, KVM, and Amazon EC2-based virtual machines. Instead, it only considers the application data rate. Without requiring any calibration or training phase our adaptive compression scheme can improve the I/O throughput of virtual machines up to a factor of four as shown through extended experimental evaluations.

Keywords

Cloud computing Adaptive compression I/O performance 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Amazon Web Services LLC: Amazon Elastic Compute Cloud (Amazon EC2). http://aws.amazon.com/ec2/ (2010). Accessed 21 Feb 2010
  2. 2.
    Arnold, R., Bell, T.: A corpus for the evaluation of lossless compression algorithms. In: Proc. of the 1997 Data Compression Conference, DCC ’97, pp. 201–210 (1997). doi:10.1109/DCC.1997.582019
  3. 3.
    Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Neugebauer, R., Pratt, I., Warfield, A.: Xen and the art of virtualization. In: Proc. of the 19th ACM symposium on Operating Systems Principles, SOSP ’03, pp. 164–177. ACM, New York (2003). doi:10.1145/945445.945462 CrossRefGoogle Scholar
  4. 4.
    Cherkasova, L., Gardner, R.: Measuring CPU overhead for I/O processing in the Xen virtual machine monitor. In: Proc. of the USENIX Annual Technical Conference, ATEC ’05, pp. 24–24. USENIX Association, Berkeley (2005)Google Scholar
  5. 5.
    Duda, K.J., Cheriton, D.R.: Borrowed-virtual-time (BVT) scheduling: supporting latency-sensitive threads in a general-purpose scheduler. Oper. Syst. Rev. 33, 261–276 (1999). doi:10.1145/319344.319169 CrossRefGoogle Scholar
  6. 6.
    GoGrid, LLC: Cloud hosting, cloud servers, hybrid hosting, cloud infrastructure from GoGrid. http://www.gogrid.com/ (2011). Accessed 31 Jul 2011
  7. 7.
    Hovestadt, M., Kao, O., Kliem, A., Warneke, D.: Evaluating adaptive compression to mitigate the effects of shared I/O in clouds. In: Proc. of the 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and PhD Forum, IPDPSW ’11, pp. 1042–1051. IEEE Computer Society, Washington (2011). doi:10.1109/IPDPS.2011.256 CrossRefGoogle Scholar
  8. 8.
    Jeannot, E., Knutsson, B., Björkman, M.: Adaptive online data compression. In: Proc. of the 11th IEEE International Symposium on High Performance Distributed Computing, HPDC ’11, pp. 379–388 (2002). doi:10.1109/HPDC.2002.1029938
  9. 9.
    Kesavan, M., Gavrilovska, A., Schwan, K.: Differential virtual time (DVT): rethinking I/O service differentiation for virtual machines. In: Proc. of the 1st ACM Symposium on Cloud Computing, SoCC ’10, pp. 27–38. ACM, New York (2010). doi:10.1145/1807128.1807135 CrossRefGoogle Scholar
  10. 10.
    Kivity, A., Kamay, Y., Laor, D., Lublin, U., Liguori, A.: kvm: the Linux virtual machine monitor. In: Proc. of the 2007 Ottawa Linux Symposium, OLS ’07, pp. 225–230 (2007)Google Scholar
  11. 11.
    Krintz, C., Sucu, S.: Adaptive on-the-fly compression. IEEE Trans. Parallel Distrib. Syst. 17(1), 15–24 (2006). doi:10.1109/TPDS.2006.3 CrossRefGoogle Scholar
  12. 12.
    Menon, A., Santos, J.R., Turner, Y., Janakiraman, G.J., Zwaenepoel, W.: Diagnosing performance overheads in the Xen virtual machine environment. In: Proc. of the 1st International Conference on Virtual Execution Environments, VEE ’05, pp. 13–23. ACM, New York (2005). doi:10.1145/1064979.1064984 CrossRefGoogle Scholar
  13. 13.
    Motgi, N., Mukherjee, A.: Network conscious text compression system (NCTCSys). In: Proc. of the International Conference on Information Technology: Coding and Computing, ITCC ’01, p. 440. IEEE Computer Society, Washington (2001). doi:10.1109/ITCC.2001.918836 CrossRefGoogle Scholar
  14. 14.
    Nurmi, D., Wolski, R., Grzegorczyk, C., Obertelli, G., Soman, S., Youseff, L., Zagorodnov, D.: The eucalyptus open-source cloud-computing system. In: Proc. of the 9th IEEE/ACM International Symposium on Cluster Computing and the Grid, CCGRID ’09, pp. 124–131. IEEE Computer Society, Washington (2009). doi:10.1109/CCGRID.2009.93 CrossRefGoogle Scholar
  15. 15.
    Pavlow, I.: LZMA SDK (Software Development Kit). http://www.7-zip.org/sdk.html (2010). Accessed 22 Oct 2010
  16. 16.
    QuickLZ: Fast compression library for C, C# and Java. http://www.quicklz.com/ (2010). Accessed 22 Oct 2010
  17. 17.
    Rackspace US, Inc.: The Rackspace Cloud. http://www.rackspacecloud.com/ (2010)
  18. 18.
    Santos, J.R., Turner, Y., Janakiraman, G., Pratt, I.: Bridging the gap between software and hardware techniques for I/O virtualization. In: Proc. of the USENIX Annual Technical Conference, ATEC ’08, pp. 29–42. USENIX Association, Berkeley, CA, USA (2008)Google Scholar
  19. 19.
    Schad, J., Dittrich, J., Quiané-Ruiz, J.A.: Runtime measurements in the cloud: observing, analyzing, and reducing variance. PVLDB 3(1), 460–471 (2010)Google Scholar
  20. 20.
    Seelam, S.R., Teller, P.J.: Virtual I/O scheduler: a scheduler of schedulers for performance virtualization. In: Proc. of the 3rd International Conference on Virtual Execution Environments, VEE ’07, pp. 105–115. ACM, New York (2007). doi:10.1145/1254810.1254826 CrossRefGoogle Scholar
  21. 21.
    Tickoo, O., Iyer, R., Illikkal, R., Newell, D.: Modeling virtual machine performance: challenges and approaches. Perform. Eval. Rev. 37, 55–60 (2010). doi:10.1145/1710115.1710126 CrossRefGoogle Scholar
  22. 22.
    Walker, E.: Benchmarking Amazon EC2 for high-performance scientific computing. USENIX; login: Magazine 33(5), 18–23 (2008)Google Scholar
  23. 23.
    Wang, G., Ng, T.E.: The impact of virtualization on network performance of Amazon EC2 data center. In: Proc. of the 29th IEEE International Conference on Computer Communications, INFOCOM ’10, pp. 1–9 (2010). doi:10.1109/INFCOM.2010.5461931
  24. 24.
    Warneke, D., Kao, O.: Nephele: efficient parallel data processing in the cloud. In: Proc. of the 2nd Workshop on Many-Task Computing on Grids and Supercomputers, MTAGS ’09, pp. 8:1–8:10. ACM, New York (2009). doi:10.1145/1646468.1646476 Google Scholar
  25. 25.
    Warneke, D., Kao, O.: Exploiting dynamic resource allocation for efficient parallel data processing in the cloud. IEEE Trans. Parallel Distrib. Syst. 22(6), 985–997 (2011). doi:10.1109/TPDS.2011.65 CrossRefGoogle Scholar
  26. 26.
    Wiseman, Y., Schwan, K., Widener, P.: Efficient end to end data exchange using configurable compression. In: Proc. of the 24th International Conference on Distributed Computing Systems, pp. 228–235 (2004). doi:10.1109/ICDCS.2004.1281587
  27. 27.
    Wolski, R.: Experiences with predicting resource performance on-line in computational grid settings. Perform. Eval. Rev. 30(4), 41–49 (2003). doi:10.1145/773056.773064 CrossRefGoogle Scholar
  28. 28.
    Wood, T., Cherkasova, L., Ozonat, K., Shenoy, P.: Profiling and modeling resource usage of virtualized applications. In: Proc. of the 9th ACM/IFIP/USENIX International Conference on Middleware, Middleware ’08, pp. 366–387. Springer New York, Inc., New York (2008)Google Scholar
  29. 29.
    Yamada, H., Kono, K.: Foxytechnique: Tricking operating system policies with a virtual machine monitor. In: Proc. of the 3rd International Conference on Virtual Execution Environments, VEE ’07, pp. 55–64. ACM, New York (2007). doi:10.1145/1254810.1254818 CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  • Matthias Hovestadt
    • 1
  • Odej Kao
    • 2
  • Andreas Kliem
    • 2
  • Daniel Warneke
    • 3
  1. 1.Hannover University of Applied SciencesHannoverGermany
  2. 2.Technische Universität BerlinBerlinGermany
  3. 3.International Computer Science Institute (ICSI)BerkeleyUSA

Personalised recommendations