Skip to main content

Advertisement

SpringerLink
Log in

SuMo: Analysis and Optimization of Amazon EC2 Instances

  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

The analysis and optimization of public clouds gains momentum as an important research topic, due to their widespread exploitation by individual users, researchers and companies for their daily tasks. We identify primitive algorithmic operations that should be part of a cloud analysis and optimization tool, such as resource profiling, performance spike detection and prediction, resource resizing, and others, and we investigate ways the collected monitoring information can be processed towards these purposes. The analyzed information is valuable in driving important virtual resource management decisions. We also present an open-source tool we developed, called SuMo,which contains the necessary functionalities for collecting monitoring data from Amazon Web Services (AWS), analyzing them and providing resource optimization suggestions. SuMo makes easy for anyone to analyze AWS instances behavior, incorporating a set of basic modules that provide profiling and spikef detection functionality. It can also be used as a basis for the development of new such analytic procedures for AWS. SuMo contains a Cost and Utilization Optimization (CUO) mechanism, formulated as an Integer Linear Programming (ILP) problem, for optimizing the cost and the utilization of a set of running Amazon EC2 instances. This CUO mechanism receives information on the currently used set of instances (their number, type, utilization) and proposes a new set of instances for serving the same load that minimizes cost and maximizes utilization and performance efficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Buyya, R., et al.: Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility. Futur. Gener. Comput. Syst. 25, 599–616 (2009)

    Article  Google Scholar 

  2. Rimal, B.P., Jukan, A., Katsaros, D., Goeleven, I.: Architecture requirementes for cloud computing systems: An enterprise cloud approach. J. Grid Comput. 9(1), 3–26 (2011)

    Article  Google Scholar 

  3. Amazon Web Services – AWS: aws.amazon.com, last seen January 2014

  4. RackSpace: www.rackspace.com, last seen January 2014

  5. Openstack: www.openstack.org, last seen January 2014

  6. OpenNebula: opennebula.org, last seen January 2014

  7. Eucaluptus: www.eucalyptus.com, last seen January 2014

  8. Amazon data center size: huanliu.wordpress.com/2012/03/13/amazon-data-center-size, last seen January 2014

  9. Designs, Lessons and Advice from Building Large Distributed Systems www.cs.cornell.edu/projects/ladis2009/talks/dean-keynote-ladis2009.pdf

  10. Amazon Case Studies: aws.amazon.com/solutions/case-studies/

  11. Wang, H., et al.: Distributed systems meet economics: pricing in the cloud USENIX Hot Topics in Cloud Computing (HotCloud) (2010)

  12. Chen, J., et al.: Tradeoffs Between Profit and Customer Satisfaction for Service Provisioning in the Cloud International Symposium on High performance Distributed Computing (HPDC) (2011)

  13. Zanikolas, S., Sakellariou, R.: A Taxonomy of grid monitoring systems. FGCS 21(1), 163–188 (2005)

    Article  Google Scholar 

  14. Kung, H.T., Lin, C.-K., Vlah, D.: CloudSense: Continuous fine-grain cloud monitoring with compressive sensing. USENIX HotCloud (2011)

  15. Petcu, D., et al.: Experiences in building a mOSAIC of clouds. J. Cloud Comput. 2(1) (2013)

  16. Ferrer, A.J., et al.: OPTIMIS: A holistic approach to cloud service provisioning. Futur. Gener. Comput. Syst. 28(1), 66–77 (2012)

    Article  MathSciNet  Google Scholar 

  17. Mallick, S.: Virtualization based cloud capacity prediction. HPCS, 849–852 (2011)

  18. De Chaves, S., et al.: Toward an architecture for monitoring private clouds. IEEE Comm. Mag. 49(12), 130–137 (2011)

    Article  Google Scholar 

  19. Ward, J. S., Barker, A.: Semantic Based Data Collection for Large Scale Cloud Systems. DIDC, pp. 13–22 (2012)

  20. Shao, J., Wei, H., Wang, Q., Mei, H.: A Runtime Model Based Monitoring Approach for Cloud. IEEE CLOUD, pp. 313–320 (2010)

  21. Meng, S., et al.: Reliable State Monitoring in Cloud Datacenters. IEEE CLOUD, pp. 951–958 (2012)

  22. Weng, J., et al.: Event Detection in Twitter. HP Laboratories (2011)

  23. Kokkinos, P., Kretsis, A., Varvarigou, T., Varvarigos, E.: Social-like Analysis on Virtual Machine Communication Traces. IEEE Cloudnet (2012)

  24. Malkowski, S., Hedwig, M., Jayasinghe, D., Pu, C., Neumann, D.: CloudXplor: A tool for configuration planning in clouds based on empirical data. ACM Symposium on Applied Computing (SAC) (2010)

  25. M. Frîncu: Scheduling highly available applications on cloud environments. Futur. Gener. Comput. Syst. 32, 138–153 (2014)

    Article  Google Scholar 

  26. Kokkinos, P., et al.: Cost and Utilization Optimization of Amazon EC2 instances. IEEE Sixth International Conference on Cloud Computing, pp. 518–525 (2013)

  27. Amazon Elastic Compute Cloud: aws.amazon.com/ec2, last seen January 2014

  28. Amazon CloudWatch: aws.amazon.com/cloudwatch, last seen January 2014

  29. Nagios: www.nagios.org, last seen January 2014

  30. Newvem: www.newvem.com, last seen January 2014

  31. Cloudability: cloudability.com, last seen January 2014

  32. Cloudvertical. www.cloudvertical.com, last seen January 2014

  33. boto - A Python interface to Amazon Web Services: docs.pythonboto.org, last seen January 2014

  34. SciPy: www.scipy.org, last seen January 2014

  35. NumPy: numpy.scipy.org, last seen January 2014

  36. IBM ILOG CPLEX Optimizer: www-01.ibm.com/software/integration/optimization/cplex-optimizer, last seen January 2014

  37. On demand instances pricing: aws.amazon.com/ec2/pricing/pricing-on-demand-instances.json, last seen January 2014

  38. Pinterest use case: www.theregister.co.uk/2012/04/30/inside_pinterest_virtual_data_center, last seen January 2014

  39. mOSAIC project: www.mosaic-cloud.eu, last seen January 2014

  40. Optimis project: www.optimis-project.eu, last seen January 2014

  41. Aeolus project: www.aeolusproject.org, last seen January 2014

  42. Data Set for IMC 2010 Data Center Measurement: pages.cs.wisc.edu/%7Etbenson/IMC10_Data.html, last seen January 2014

  43. Google. Google Cluster Data V1. Available: http://code.google.com/p/googleclusterdata/wiki/TraceVersion1, last seen January 2014

  44. Kavulya, S., et al.: An analysis of traces from a production mapreduce cluster. Cluster, Cloud and Grid Comput. (CCGrid), pp. 94–103 (2010)

  45. Kertesz, A., Kecskemeti, G., Oriol, M., Kotcauer, P., Acs, S., Rodrıguez, M., Merce, O., Marosi, A.C., Marco, J., Franch, X.: Enhancing Federated Cloud Management with an Integrated Service Monitoring Approach. J. Grid Comput. 11(4), 699–720 (2013)

    Article  Google Scholar 

  46. Mendez, V., Casajus, A., Fernandez, V., Graciani, R., Merino, G.: Rafhyc: An Architecture for Constructing Resilient Services on Federated Hybrid Clouds. J. Grid Comput. 11(4), 753–770 (2013)

  47. SuMo-tool: https://github.com/SuMo-tool, last seen January 2014

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, National Technical University of Athens, Athens, Greece

    P. Kokkinos & T. A. Varvarigou

  2. Department of Computer Engineering and Informatics, University of Patras, Patra, Greece

    A. Kretsis, P. Soumplis & E. A. Varvarigos

Authors
  1. P. Kokkinos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. T. A. Varvarigou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. Kretsis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. P. Soumplis
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. E. A. Varvarigos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to P. Kokkinos.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kokkinos, P., Varvarigou, T.A., Kretsis, A. et al. SuMo: Analysis and Optimization of Amazon EC2 Instances. J Grid Computing 13, 255–274 (2015). https://doi.org/10.1007/s10723-014-9311-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10723-014-9311-x

Keywords

Search

Navigation