Abstract
Architecting applications for the Cloud is challenging due to significant differences between traditional hosting and Cloud infrastructure setup, unknown and unproven Cloud performance and scalability characteristics, as well as variable quota limitations. Building workable cloud applications therefore requires in-depth insight into the architectural and performance characteristics of each cloud offering, and the ability to reason about tradeoffs and alternatives of application designs and deployments. NICTA has developed a Service Oriented Performance Modeling technology for modeling the performance and scalability of Service Oriented applications architected for a variety of platforms. Using a suite of cloud testing applications we conducted in-depth empirical evaluations of a variety of real cloud infrastructures, including Google App Engine, Amazon EC2, and Microsoft Azure. The insights from these experimental evaluations, and other public/published data, were combined with the modeling technology to predict the resource requirements in terms of cost, application performance, and limitations of a realistic application for different deployment scenarios.
Chapter PDF
Similar content being viewed by others
Keywords
References
Brebner, P., O’Brien, L., Gray, J.: Performance Modeling for e-Government Service Oriented Architectures. In: Experience Report Proceedings, ASWEC 2008, pp. 130–138 (2008)
Brebner, P.: Performance modeling for service oriented architectures. In: ICSE Companion 2008, pp. 953–954 (2008), doi: http://doi.acm.org/10.1145/1370175.1370204
Brebner, P., O’Brien, L., Gray, J.: Performance Modeling Evolving Enterprise Service Oriented Architectures. In: WISA/ECSA 2009, pp. 71–80 (2009), doi: 10.1109/WICSA.2009.5290793
Brebner, P.: Service-Oriented Performance Modeling the MULE Enterprise Service Bus Loan Broker Application. In: SEAA 2009, pp. 404–411 (2009), doi:10.1109/SEAA.2009.57
Brebner, P., O’Brien, L., Gray, J.: Performance Modeling Power Consumption and Carbon Emissions for Server Virtualization of Service Oriented Architectures. In: Proceedings of the IEEE EDOC 2009 Workshops, Middleware for Web Services Workshop 2009, pp. 92–99 (2009), doi:10.1109/EDOCW.2009.5332010
Wood, T., Cherkasova, L., Ozonat, K., Shenoy, P.: Predicting Application Resource Requirements in Virtual Environments. HP Laboratories, Technical Report HPL-2008-122 (2008), http://www.hpl.hp.com/techreports/2008/HPL-2008-122.html
Amazon Elastic Load Balancing, http://docs.amazonwebservices.com/ElasticLoadBalancing/latest/DeveloperGuide/
Amazon Auto Scaling, http://aws.amazon.com/autoscaling/
Amazon EC2, http://aws.amazon.com/ec2/
EC2 Instance Types, http://aws.amazon.com/ec2/instance-types/
Amazon EC2 benchmark, http://www.mnxsolutions.com/blog/linux/amazon-ec2-benchmark-pystone.html
Testing Cloud Computing Performance with PRTG: Performance Comparison of Amazon EC2 Instance Types, http://www.paessler.com/blog/2009/04/03/prtg-7/testing-cloud-computing-performance-with-prtg-performance-comparison-of-amazon-ec2-instance-types/
Lamp performance on the elastic compute cloud: benchmarking drupal on amazon ec2, http://www.johnandcailin.com/blog/john/lamp-performance-elastic-compute-cloud:-benchmarking-drupal-amazon-ec2
MySQL on EC2 Part 1: are all instances equal?, http://www.infibase.com/blog/2009/07/mysql-on-amazon-ec2-part-1/
Walker, E.: Benchmarking Amazon EC2 for high-performance scientific computing, http://www.usenix.org/publications/login/2008-10/openpdfs/walker.pdf
EC2 Pricing, http://aws.amazon.com/ec2/pricing/
EC2 spot price history, http://cloudexchange.org/
EC2 Cloudwatch, http://aws.amazon.com/cloudwatch/
EC2 Spot instances, http://aws.amazon.com/ec2/spot-instances/
Google AppEngine, http://code.google.com/appengine/
Google AppEngine billing, http://code.google.com/appengine/docs/billing.html
Google AppEngine quotas and limits, http://code.google.com/appengine/docs/quotas.html
Google AppEngine - a second look, http://highscalability.com/google-appengine-second-look
Twitmark ported OFF Google AppEngine, http://mrblog.org/2009/10/16/twitmart-ported-off-of-google-app-engine/
CPU (Fibonacci) Latency, http://code.google.com/status/appengine/detail/serving-java/2010/01/30#ae-trust-detail-cpu-fibonacci-java-latency
Barker, S., Shenoy, P.: Empirical Evaluation of Latency-sensitive Applications Performance in the Cloud, http://none.cs.umass.edu/papers/pdf/mmsys10-latency.pdf
Alexandru-Dorin, G.: Network performance in virtual infrastructures – A closer look at Amazon EC2, http://staff.science.uva.nl/~delaat/sne-2009-2010/p29/presentation.pdf
Measuring EC2 system performance, http://tech.mangot.com/roller/dave/entry/ec2_variability_the_numbers_revealed
EC2 Benchmarks, http://www.wafflegrid.com/wiki/index.php?title=Ec2_Benchmarks
http://alan.blog-city.com/has_amazon_ec2_become_over_subscribed.htm
Are Clouds ready for large distributed applications?, http://www.cs.cornell.edu/projects/ladis2009/papers/sripanidkulchai-ladis2009.pdf
Bolden, B.: The Skinny Straw: Cloud Computing’s Bottleneck and How to Address it, http://www.cio.com/article/499137/The_Skinny_Straw_Cloud_Computing_s_Bottleneck_and_How_to_Address_It
Aspera on-demand, http://www.asperasoft.com/en/products/on_demand_17/aspera_on_demand_for_AWS_17
Azure instance limits, http://blog.toddysm.com/2010/01/windows-azure-role-instance-limits-explained.html
Google AppEngine support for Java, http://groups.google.com/group/google-appengine-java/web/will-it-play-in-app-engine
Google AppEngine datastore, http://code.google.com/appengine/docs/java/datastore/overview.html
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Brebner, P., Liu, A. (2011). Performance and Cost Assessment of Cloud Services. In: Maximilien, E.M., Rossi, G., Yuan, ST., Ludwig, H., Fantinato, M. (eds) Service-Oriented Computing. ICSOC 2010. Lecture Notes in Computer Science, vol 6568. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-19394-1_5
Download citation
DOI: https://doi.org/10.1007/978-3-642-19394-1_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-19393-4
Online ISBN: 978-3-642-19394-1
eBook Packages: Computer ScienceComputer Science (R0)