Advertisement

Enabling Resource Sharing between Transactional and Batch Workloads Using Dynamic Application Placement

  • David Carrera
  • Malgorzata Steinder
  • Ian Whalley
  • Jordi Torres
  • Eduard Ayguadé
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5346)

Abstract

We present a technique that enables existing middleware to fairly manage mixed workloads: batch jobs and transactional applications. The technique leverages a generic application placement controller, which dynamically allocates compute resources to application instances. The controller works towards a fairness goal while also trying to maximize individual workload performance. We use relative performance functions to drive the application placement controller. Such functions are derived from workload-specific performance models—in the case of transactional workloads, we use queuing theory to build the performance model. For batch workloads, we evaluate a candidate placement by calculating long-term estimates of the completion times that are achievable with that placement according to a scheduling policy. In this paper, we propose a lowest relative performance first scheduling policy as a way to also achieve fair resource allocation among batch jobs. Our technique permits collocation of the workload types on the same physical hardware, and leverages control mechanisms such as suspension and migration to perform online system reconfiguration. In our experiments we demonstrate that our technique maximizes mixed workload performance while providing service differentiation based on high-level performance goals.

Keywords

Completion Time Virtual Machine Relative Performance Physical Machine Earliest Deadline First 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Sun Microsystems: Behavior of mixed workloads consolidated using Solaris Resource Manager software. Technical report (May 2005)Google Scholar
  2. 2.
    Lin, B., Dinda, P.: VSched: Mixing batch and interactive virtual machines using periodic real-time scheduling. In: Proc. ACM/IEEE Supercomputing, Seattle, WA (November 2005)Google Scholar
  3. 3.
    Jensen, E.D., Locke, C.D., Tokuda, H.: A time-driven scheduling model for real-time operating systems. In: IEEE Real-Time Systems Symposium, pp. 112–122 (1985)Google Scholar
  4. 4.
    Ravindran, B., Jensen, E.D., Li, P.: On recent advances in time/utility function real-time scheduling and resource management. In: ISORC 2005: Proceedings of the Eighth IEEE International Symposium on Object-Oriented Real-Time Distributed Computing (ISORC 2005), Washington, DC, USA, pp. 55–60. IEEE Computer Society, Los Alamitos (2005)CrossRefGoogle Scholar
  5. 5.
    Balli, U., Anderson, J.S.: Utility accrual real-time scheduling under variable cost functions. IEEE Trans. Comput. 56(3), 385–401 (2007); Member-Haisang Wu and Senior Member-Binoy Ravindran and Member-E. Douglas JensenMathSciNetCrossRefGoogle Scholar
  6. 6.
    Vengerov, D., Mastroleon, L., Murphy, D., Bambos, N.: Adaptive data-aware utility-based scheduling in resource-constrained systems. Sun Technical Report TR-2007-164, Sun Microsystems (April 2007)Google Scholar
  7. 7.
    Lee, C.B., Snavely, A.E.: Precise and realistic utility functions for user-centric performance analysis of schedulers. In: HPDC 2007: Proceedings of the 16th international symposium on High performance distributed computing, pp. 107–116. ACM, New York (2007)Google Scholar
  8. 8.
    Chun, B.N., Culler, D.E.: User-centric performance analysis of market-based cluster batch schedulers. In: CCGRID 2002: Proceedings of the 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid, Washington, DC, USA, p. 30. IEEE Comp. Society, Los Alamitos (2002)Google Scholar
  9. 9.
    Foster, I., Freeman, T., Keahey, K., Scheftner, D., Sotomayor, B., Zhang, X.: Virtual clusters for grid communities, Singapore (May 2006)Google Scholar
  10. 10.
    Czajkowski, G., Wegiel, M., Daynes, L., Palacz, K., Jordan, M., Skinner, G., Bryce, C.: Resource management for clusters of virtual machines, Cardiff, UK, pp. 382–389 (May 2005)Google Scholar
  11. 11.
    Zhu, X., Wang, Z., Singhal, S.: Utility-driven workload management using nested control design. In: American Control Conference, June 14-16, 2006, p. 6 (2006)Google Scholar
  12. 12.
    Padala, P., Shin, K.G., Zhu, X., Uysal, M., Wang, Z., Singhal, S., Merchant, A., Salem, K.: Adaptive control of virtualized resources in utility computing environments. In: EuroSys 2007: Proceedings of the ACM SIGOPS/EuroSys European Conference on Computer Systems 2007, pp. 289–302. ACM, New York (2007)CrossRefGoogle Scholar
  13. 13.
    Wang, Z., Zhu, X., Padala, P., Singhal, S.: Capacity and performance overhead in dynamic resource allocation to virtual containers. In: 10th IFIP/IEEE International Symposium on Integrated Network Management, 2007. IM 2007, May 21–25, 2007, pp. 149–158 (2007)Google Scholar
  14. 14.
    Kellerer, H., Pferschy, U., Pisinger, D.: Knapsack Problems (2004)Google Scholar
  15. 15.
    Karve, A., Kimbrel, T., Pacifici, G., Spreitzer, M., Steinder, M., Sviridenko, M., Tantawi, A.: Dynamic placement for clustered web applications. In: WWW Conference, Edinburgh, Scotland (May 2006)Google Scholar
  16. 16.
    Kimbrel, T., Steinder, M., Sviridenko, M., Tantawi, A.: Dynamic application placement under service and memory constraints. In: International Workshop on Efficient and Experimental Algorithms, Santorini Island, Greece (May 2005)Google Scholar
  17. 17.
    Wang, X., Lan, D., Wang, G., Fang, X., Ye, M., Chen, Y., Wang, Q.: Appliance-based autonomic provisioning framework for virtualized outsourcing data center. In: ICAC 2007: Proceedings of the Fourth International Conference on Autonomic Computing, Washington, DC, USA, p. 29. IEEE Computer Society, Los Alamitos (2007)Google Scholar
  18. 18.
    Carrera, D., Steinder, M., Whalley, I., Torres, J., Ayguadé, E.: Utility-based placement of dynamic web applications with fairness goals. In: 11th IEEE/IFIP Network Operations and Management Symposium (NOMS 2008), Salvador Bahia, Brazil (2008)Google Scholar
  19. 19.
    Xu, J., Zhao, M., Fortes, J., Carpenter, R., Yousif, M.: On the use of fuzzy modeling in virtualized data center management. In: Xu, J., Zhao, M., Fortes, J., Carpenter, R., Yousif, M. (eds.) ICAC 2007. Fourth International Conference on Autonomic Computing, June 11-15, 2007, p. 25 (2007)Google Scholar
  20. 20.
    Urgaonkar, B., Shenoy, P., Roscoe, T.: Resource overbooking and application profiling in shared hosting platforms. In: Proc. Fifth Symposium on Operating Systems Design and Implementation, Boston, MA (December 2002)Google Scholar
  21. 21.
    Pacifici, G., Spreitzer, M., Tantawi, A., Youssef, A.: Performance management for cluster-based web services. IEEE Journal on Selected Areas in Communications 23(12) (December 2005)Google Scholar
  22. 22.
    Pacifici, G., Segmuller, W., Spreitzer, M., Steinder, M., Tantawi, A., Youssef, A.: Managing the response time for multi-tiered web applications. Technical Report Tech. Rep. RC 23651, IBM (2005)Google Scholar
  23. 23.
    Pacifici, G., Segmuller, W., Spreitzer, M., Tantawi, A.: Dynamic estimation of cpu demand of web traffic. In: VALUETOOLS, Pisa, Italy (October 2006)Google Scholar
  24. 24.
    Carrera, D., Steinder, M., Whalley, I., Torres, J., Ayguadé, E.: Managing SLAs of heterogeneous workloads using dynamic application placement. Technical Report RC 24469, IBM Research (January 2008)Google Scholar
  25. 25.
    Steinder, M., Whalley, I., Carrera, D., Gaweda, I., Chess, D.: Server virtualization in autonomic management of heterogeneous workloads. In: 10th IEEE/IFIP Symposium on Integrated Management (IM 2007), Munich, Germany (2007)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2008

Authors and Affiliations

  • David Carrera
    • 1
  • Malgorzata Steinder
    • 2
  • Ian Whalley
    • 2
  • Jordi Torres
    • 1
  • Eduard Ayguadé
    • 1
  1. 1.Technical University of Catalonia (UPC) - Barcelona Supercomputing Center (BSC)BarcelonaSpain
  2. 2.IBM T.J. Watson Research CenterHawthorne

Personalised recommendations