Advertisement

Journal of Grid Computing

, Volume 13, Issue 2, pp 255–274 | Cite as

SuMo: Analysis and Optimization of Amazon EC2 Instances

  • P. Kokkinos
  • T. A. Varvarigou
  • A. Kretsis
  • P. Soumplis
  • E. A. Varvarigos
Article

Abstract

The analysis and optimization of public clouds gains momentum as an important research topic, due to their widespread exploitation by individual users, researchers and companies for their daily tasks. We identify primitive algorithmic operations that should be part of a cloud analysis and optimization tool, such as resource profiling, performance spike detection and prediction, resource resizing, and others, and we investigate ways the collected monitoring information can be processed towards these purposes. The analyzed information is valuable in driving important virtual resource management decisions. We also present an open-source tool we developed, called SuMo,which contains the necessary functionalities for collecting monitoring data from Amazon Web Services (AWS), analyzing them and providing resource optimization suggestions. SuMo makes easy for anyone to analyze AWS instances behavior, incorporating a set of basic modules that provide profiling and spikef detection functionality. It can also be used as a basis for the development of new such analytic procedures for AWS. SuMo contains a Cost and Utilization Optimization (CUO) mechanism, formulated as an Integer Linear Programming (ILP) problem, for optimizing the cost and the utilization of a set of running Amazon EC2 instances. This CUO mechanism receives information on the currently used set of instances (their number, type, utilization) and proposes a new set of instances for serving the same load that minimizes cost and maximizes utilization and performance efficiency.

Keywords

Public clouds Analysis Optimization Amazon web services Toolkit 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Buyya, R., et al.: Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility. Futur. Gener. Comput. Syst. 25, 599–616 (2009)CrossRefGoogle Scholar
  2. 2.
    Rimal, B.P., Jukan, A., Katsaros, D., Goeleven, I.: Architecture requirementes for cloud computing systems: An enterprise cloud approach. J. Grid Comput. 9(1), 3–26 (2011)CrossRefGoogle Scholar
  3. 3.
    Amazon Web Services – AWS: aws.amazon.com, last seen January 2014
  4. 4.
    RackSpace: www.rackspace.com, last seen January 2014
  5. 5.
    Openstack: www.openstack.org, last seen January 2014
  6. 6.
    OpenNebula: opennebula.org, last seen January 2014Google Scholar
  7. 7.
    Eucaluptus: www.eucalyptus.com, last seen January 2014
  8. 8.
    Amazon data center size: huanliu.wordpress.com/2012/03/13/amazon-data-center-size, last seen January 2014
  9. 9.
    Designs, Lessons and Advice from Building Large Distributed Systems www.cs.cornell.edu/projects/ladis2009/talks/dean-keynote-ladis2009.pdf
  10. 10.
  11. 11.
    Wang, H., et al.: Distributed systems meet economics: pricing in the cloud USENIX Hot Topics in Cloud Computing (HotCloud) (2010)Google Scholar
  12. 12.
    Chen, J., et al.: Tradeoffs Between Profit and Customer Satisfaction for Service Provisioning in the Cloud International Symposium on High performance Distributed Computing (HPDC) (2011)Google Scholar
  13. 13.
    Zanikolas, S., Sakellariou, R.: A Taxonomy of grid monitoring systems. FGCS 21(1), 163–188 (2005)CrossRefGoogle Scholar
  14. 14.
    Kung, H.T., Lin, C.-K., Vlah, D.: CloudSense: Continuous fine-grain cloud monitoring with compressive sensing. USENIX HotCloud (2011)Google Scholar
  15. 15.
    Petcu, D., et al.: Experiences in building a mOSAIC of clouds. J. Cloud Comput. 2(1) (2013)Google Scholar
  16. 16.
    Ferrer, A.J., et al.: OPTIMIS: A holistic approach to cloud service provisioning. Futur. Gener. Comput. Syst. 28(1), 66–77 (2012)CrossRefMathSciNetGoogle Scholar
  17. 17.
    Mallick, S.: Virtualization based cloud capacity prediction. HPCS, 849–852 (2011)Google Scholar
  18. 18.
    De Chaves, S., et al.: Toward an architecture for monitoring private clouds. IEEE Comm. Mag. 49(12), 130–137 (2011)CrossRefGoogle Scholar
  19. 19.
    Ward, J. S., Barker, A.: Semantic Based Data Collection for Large Scale Cloud Systems. DIDC, pp. 13–22 (2012)Google Scholar
  20. 20.
    Shao, J., Wei, H., Wang, Q., Mei, H.: A Runtime Model Based Monitoring Approach for Cloud. IEEE CLOUD, pp. 313–320 (2010)Google Scholar
  21. 21.
    Meng, S., et al.: Reliable State Monitoring in Cloud Datacenters. IEEE CLOUD, pp. 951–958 (2012)Google Scholar
  22. 22.
    Weng, J., et al.: Event Detection in Twitter. HP Laboratories (2011)Google Scholar
  23. 23.
    Kokkinos, P., Kretsis, A., Varvarigou, T., Varvarigos, E.: Social-like Analysis on Virtual Machine Communication Traces. IEEE Cloudnet (2012)Google Scholar
  24. 24.
    Malkowski, S., Hedwig, M., Jayasinghe, D., Pu, C., Neumann, D.: CloudXplor: A tool for configuration planning in clouds based on empirical data. ACM Symposium on Applied Computing (SAC) (2010)Google Scholar
  25. 25.
    M. Frîncu: Scheduling highly available applications on cloud environments. Futur. Gener. Comput. Syst. 32, 138–153 (2014)CrossRefGoogle Scholar
  26. 26.
    Kokkinos, P., et al.: Cost and Utilization Optimization of Amazon EC2 instances. IEEE Sixth International Conference on Cloud Computing, pp. 518–525 (2013)Google Scholar
  27. 27.
    Amazon Elastic Compute Cloud: aws.amazon.com/ec2, last seen January 2014
  28. 28.
    Amazon CloudWatch: aws.amazon.com/cloudwatch, last seen January 2014
  29. 29.
    Nagios: www.nagios.org, last seen January 2014
  30. 30.
    Newvem: www.newvem.com, last seen January 2014
  31. 31.
    Cloudability: cloudability.com, last seen January 2014
  32. 32.
    Cloudvertical. www.cloudvertical.com, last seen January 2014
  33. 33.
    boto - A Python interface to Amazon Web Services: docs.pythonboto.org, last seen January 2014
  34. 34.
    SciPy: www.scipy.org, last seen January 2014
  35. 35.
    NumPy: numpy.scipy.org, last seen January 2014
  36. 36.
    IBM ILOG CPLEX Optimizer: www-01.ibm.com/software/integration/optimization/cplex-optimizer, last seen January 2014
  37. 37.
    On demand instances pricing: aws.amazon.com/ec2/pricing/pricing-on-demand-instances.json, last seen January 2014
  38. 38.
  39. 39.
    mOSAIC project: www.mosaic-cloud.eu, last seen January 2014
  40. 40.
    Optimis project: www.optimis-project.eu, last seen January 2014
  41. 41.
    Aeolus project: www.aeolusproject.org, last seen January 2014
  42. 42.
    Data Set for IMC 2010 Data Center Measurement: pages.cs.wisc.edu/%7Etbenson/IMC10_Data.html, last seen January 2014
  43. 43.
    Google. Google Cluster Data V1. Available: http://code.google.com/p/googleclusterdata/wiki/TraceVersion1, last seen January 2014
  44. 44.
    Kavulya, S., et al.: An analysis of traces from a production mapreduce cluster. Cluster, Cloud and Grid Comput. (CCGrid), pp. 94–103 (2010)Google Scholar
  45. 45.
    Kertesz, A., Kecskemeti, G., Oriol, M., Kotcauer, P., Acs, S., Rodrıguez, M., Merce, O., Marosi, A.C., Marco, J., Franch, X.: Enhancing Federated Cloud Management with an Integrated Service Monitoring Approach. J. Grid Comput. 11(4), 699–720 (2013)CrossRefGoogle Scholar
  46. 46.
    Mendez, V., Casajus, A., Fernandez, V., Graciani, R., Merino, G.: Rafhyc: An Architecture for Constructing Resilient Services on Federated Hybrid Clouds. J. Grid Comput. 11(4), 753–770 (2013)Google Scholar
  47. 47.
    SuMo-tool: https://github.com/SuMo-tool, last seen January 2014

Copyright information

© Springer Science+Business Media Dordrecht 2014

Authors and Affiliations

  • P. Kokkinos
    • 1
  • T. A. Varvarigou
    • 1
  • A. Kretsis
    • 2
  • P. Soumplis
    • 2
  • E. A. Varvarigos
    • 2
  1. 1.Department of Electrical and Computer EngineeringNational Technical University of AthensAthensGreece
  2. 2.Department of Computer Engineering and InformaticsUniversity of PatrasPatraGreece

Personalised recommendations