Abstract
Dynamic capacity provisioning is a well studied approach to handling gradual changes in data center load. However, abrupt spikes in load are still problematic in that the work in the system rises very quickly during the setup time needed to turn on additional capacity. Performance can be severely affected even if it takes only 5 seconds to bring additional capacity online.
In this paper, we propose SOFTScale, an approach to handling load spikes in multi-tier data centers without having to over-provision resources. SOFTScale works by opportunistically stealing resources from other tiers to alleviate the bottleneck tier, even when the tiers are carefully provisioned at capacity. SOFTScale is especially useful during the transient overload periods when additional capacity is being brought online.
Via implementation on a 28-server multi-tier testbed, we investigate a range of possible load spikes, including an artificial doubling or tripling of load, as well as large spikes in real traces. We find that SOFTScale can meet our stringent 95th percentile response time Service Level Agreement goal of 500ms without using any additional resources even under some extreme load spikes that would normally cause the system (without SOFTScale) to exhibit response times as high as 96 seconds.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Eckerson, W.W.: Three tier client/server architecture: Achieving scalability, performance, and efficiency in client server applications. Open Information Systems 10 (January 1995)
Schussel, G.: Client/server: Past, present and future (September 2006), http://www.dciexpo.com/geos/dbsejava.htm
Urgaonkar, B., Chandra, A.: Dynamic provisioning of multi-tier internet applications. In: ICAC 2005, Washington, DC (2005)
Trushkowsky, B., Bodík, P., Fox, A., Franklin, M.J., Jordan, M.I., Patterson, D.A.: The SCADS director: scaling a distributed storage system under stringent performance requirements. In: FAST 2011, San Jose, CA, USA (2011)
Bryant, R., Tumanov, A., Irzak, O., Scannell, A., Joshi, K., Hiltunen, M., Lagar-Cavilla, A., de Lara, E.: Kaleidoscope: cloud micro-elasticity via VM state coloring. In: EuroSys 2011, Salzburg, Austria (2011)
Atikoglu, B., Xu, Y., Frachtenberg, E., Jiang, S., Paleczny, M.: Workload analysis of a large-scale key-value store. In: Sigmetrics 2012, London, UK (2012)
Leite, J.C., Kusic, D.M., Mossé, D.: Stochastic approximation control of power and tardiness in a three-tier web-hosting cluster. In: ICAC 2010, Washington, DC, USA (2010)
Nathuji, R., Kansal, A., Ghaffarkhah, A.: Q-clouds: Managing performance interference effects for QoS-aware clouds. In: EuroSys 2010, Paris, France (2010)
Padala, P., Hou, K.Y., Shin, K.G., Zhu, X., Uysal, M., Wang, Z., Singhal, S., Merchant, A.: Automated control of multiple virtualized resources. In: EuroSys 2009, Nuremberg, Germany (2009)
Krioukov, A., Mohan, P., Alspaugh, S., Keys, L., Culler, D., Katz, R.: NapSAC: Design and implementation of a power-proportional web cluster. In: Green Networking 2010, New Delhi, India (2010)
Horvath, T., Skadron, K.: Multi-mode energy management for multi-tier server clusters. In: PACT 2008, Toronto, ON, Canada (2008)
Gmach, D., Krompass, S., Scholz, A., Wimmer, M., Kemper, A.: Adaptive quality of service management for enterprise services. ACM Trans. Web 2(1), 1–46 (2008)
Gandhi, A., Chen, Y., Gmach, D., Arlitt, M., Marwah, M.: Minimizing data center SLA violations and power consumption via hybrid resource provisioning. In: IGCC 2011, Orlando, FL, USA (2011)
LeFebvre, W.: CNN.com: Facing A World Crisis. Invited Talk, USENIX ATC (2002)
Hu, J., Sandoval, G.: Web acts as hub for info on attacks. CNET news (Septemper 2001)
Wald, L.A., Schwarz, S.: The 1999 southern california seismic network bulletin. Seismological Research Letters 71, 401–422 (2000)
Adler, S.: The Slashdot Effect: An Analysis of Three Internet Publications, http://ssadler.phy.bnl.gov/adler/SDE/SlashDotEffect.html
Constine, J.: Walmart’s black friday disaster: Website crippled, violence in stores (November 2011), http://techcrunch.com/2011/11/25/walmart-black-friday
Ohlson, K.: Victoria’s secret knows ads, not the web. Computer World (February 1999)
Arlitt, M., Jin, T.: Workload characterization of the 1998 world cup web site. IEEE Network (1999)
Pachal, P.: Amazon apologizes for cloud outage, issues credit to customers. PCMag. (April 2011)
Schroeder, B., Pinheiro, E., Weber, W.D.: DRAM errors in the wild: a large-scale field study. In: SIGMETRICS 2009, Seattle, WA, USA (2009)
Kivity, A., Kamay, Y., Laor, D., Lublin, U., Liguori, A.: kvm: the Linux virtual machine monitor. In: Linux Symposium 2007, Ottawa, ON, Canada (2007)
Amazon Inc.: Amazon Elastic Compute Cloud, http://aws.amazon.com/ec2/
DeCandia, G., Hastorun, D., Jampani, M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., Vogels, W.: Dynamo: Amazon’s highly available key-value store. In: SOSP 2007, Stevenson, WA, USA (2007)
Mosberger, D., Jin, T.: httperf—A Tool for Measuring Web Server Performance. ACM Sigmetrics: Performance Evaluation Review 26, 31–37 (1998)
Newman, M.E.J.: Power laws, Pareto distributions and Zipf’s law. Contemporary Physics 46, 323–351 (2005)
LaPedus, M.: Facebook Wants New and Cheaper Memories (November 2011), http://semimd.com/blog/2011/11/08/facebook-wants-new-and-cheaper-memories
Personal communication with Facebook
Gandhi, A., Zhu, T., Harchol-Balter, M., Kozuch, M.: SOFTScale: Stealing Opportunistically For Transient Scaling. Technical Report CMU-CS-12-111. Carnegie Mellon University (2012)
Andersen, D.G.: Trace of web site activity on Pi day (3/14/2011) from domains hosted by angio.net. Personal Communication (December 2011)
National Laboratory for Applied Network Research. Anonymized access logs, ftp://ftp.ircache.net/Traces/
Kim, Y., Seshadri, V., Lee, D., Liu, J., Mutlu, O.: A case for exploiting subarray-level parallelism (SALP) in DRAM. In: ISCA 2012, Portland, OR, USA (2012)
Chandra, A., Shenoy, P.: Effectiveness of dynamic resource allocation for handling internet flash crowds. Technical Report TR03-37, Department of Computer Science, University of Massachusetts at Amherst (November 2003)
Lassettre, E., Coleman, D.W., Diao, Y., Froehlich, S., Hellerstein, J.L., Hsiung, L.S., Mummert, T.W., Raghavachari, M., Parker, G., Russell, L., Surendra, M., Tseng, V., Wadia, N., Ye, P.: Dynamic Surge Protection: An Approach to Handling Unexpected Workload Surges with Resource Actions that Have Lead Times. In: Brunner, M., Keller, A. (eds.) DSOM 2003. LNCS, vol. 2867, pp. 82–92. Springer, Heidelberg (2003)
Elson, J., Howell, J.: Handling flash crowds from your garage. In: USENIX ATC 2008, Boston, MA, USA (2008)
Urgaonkar, B., Shenoy, P.: Cataclysm: Scalable overload policing for internet applications. Journal of Network and Computer Applications 31(4), 891–920 (2008)
Adya, A., Bolosky, W.J., Chaiken, R., Douceur, J.R., Howell, J., Lorch, J.: Load management in a large-scale decentralized file system. MSR-TR 2004-60 (2004)
Voigt, T., Tewari, R., Freimuth, D., Mehra, A.: Kernel mechanisms for service differentiation in overloaded web servers. In: USENIX ATC 2001, Boston, MA, USA (2001)
Cherkasova, L., Phaal, P.: Session-based admission control: A mechanism for peak load management of commercial web sites. IEEE Trans. Comput. 51 (June 2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 IFIP International Federation for Information Processing
About this paper
Cite this paper
Gandhi, A., Zhu, T., Harchol-Balter, M., Kozuch, M.A. (2012). SOFTScale: Stealing Opportunistically for Transient Scaling. In: Narasimhan, P., Triantafillou, P. (eds) Middleware 2012. Middleware 2012. Lecture Notes in Computer Science, vol 7662. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-35170-9_8
Download citation
DOI: https://doi.org/10.1007/978-3-642-35170-9_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-35169-3
Online ISBN: 978-3-642-35170-9
eBook Packages: Computer ScienceComputer Science (R0)