Enforcing Performance Isolation Across Virtual Machines in Xen

  • Diwaker Gupta
  • Ludmila Cherkasova
  • Rob Gardner
  • Amin Vahdat
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4290)


Virtual machines (VMs) have recently emerged as the basis for allocating resources in enterprise settings and hosting centers. One benefit of VMs in these environments is the ability to multiplex several operating systems on hardware based on dynamically changing system characteristics. However, such multiplexing must often be done while observing per-VM performance guarantees or service level agreements. Thus, one important requirement in this environment is effective performance isolation among VMs. In this paper, we address performance isolation across virtual machines in Xen [1]. For instance, while Xen can allocate fixed shares of CPU among competing VMs, it does not currently account for work done on behalf of individual VMs in device drivers. Thus, the behavior of one VM can negatively impact resources available to other VMs even if appropriate per-VM resource limits are in place.

In this paper, we present the design and evaluation of a set of primitives implemented in Xen to address this issue. First, XenMon accurately measures per-VM resource consumption, including work done on behalf of a particular VM in Xen’s driver domains. Next, our SEDF-DC scheduler accounts for aggregate VM resource consumption in allocating CPU. Finally, ShareGuard limits the total amount of resources consumed in privileged and driver domains based on administrator-specified limits. Our performance evaluation indicates that our mechanisms effectively enforce performance isolation for a variety of workloads and configurations.


Virtual Machine Device Driver Virtual Machine Monitor Performance Isolation Resource Container 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Neugebauer, R., Pratt, I., Warfield, A.: Xen and the art of virtualization. In: Proc. of the 19th ACM SOSP, New York (2003)Google Scholar
  2. 2.
    Kallahalla, M., Uysal, M., Swaminathan, R., Lowell, D.E., Wray, M., Christian, T., Edwards, N., Dalton, C.I., Gittler, F.: SoftUDC: A software based data center for utility computing. IEEE Computer (2004)Google Scholar
  3. 3.
    The Oceano Project, (last accessed, 1/17/2006)
  4. 4.
    Fraser, K., Hand, S., Neugebauer, R., Pratt, I., Warfield, A., Williamson, M.: Reconstructing I/O. Technical Report UCAM-CL-TR-596, University of Cambridge (2005)Google Scholar
  5. 5.
    Cherkasova, L., Gardner, R.: Measuring CPU Overhead for I/O Processing in the Xen Virtual Machine Monitor. In: Proc. of USENIX Annual Technical Conference (2005)Google Scholar
  6. 6.
    Chase, J.S., Levy, H.M., Feeley, M.J., Lazowska, E.D.: Sharing and protection in a single-address-space operating system. ACM Trans. Comput. Syst. 12(4), 271–307 (1994)CrossRefGoogle Scholar
  7. 7.
    Jones, M.B., Leach, P.J., Draves, R.P., Iii Barrera, J.S.: Modular real-time resource management in the Rialto operating system. In: Proc. of the 5th HotOS, Washington, DC, USA, p. 12. IEEE Computer Society, Los Alamitos (1995)Google Scholar
  8. 8.
    Leslie, I.M., McAuley, D., Black, R., Roscoe, T., Barham, P.T., Evers, D., Fairbairns, R., Hyden, E.: The design and implementation of an operating system to support distributed multimedia applications. IEEE Journal of Selected Areas in Communications 14(7) (1996)Google Scholar
  9. 9.
    Druschel, P., Banga, G.: Lazy receiver processing (LRP): a network subsystem architecture for server systems. In: Proc. of the second USENIX OSDI, pp. 261–275 (1996)Google Scholar
  10. 10.
    Bruno, J., Gabber, E., Ozden, B., Silberschatz, A.: The Eclipse Operating System: Providing Quality of Service via Reservation Domains. In: USENIX Annual Technical Conference (1998)Google Scholar
  11. 11.
    Verghese, B., Gupta, A., Rosenblum, M.: Performance isolation: sharing and isolation in shared-memory multiprocessors. In: Proc. of the 8th International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 181–192. ACM Press, New York (1998)Google Scholar
  12. 12.
    Banga, G., Druschel, P., Mogul, J.C.: Resource Containers: a New Facility for Resource Management in Server Systems. In: Proc. of the third USENIX OSDI, New Orleans, Louisiana (1999)Google Scholar
  13. 13.
    Gupta, D., Gardner, R., Cherkasova, L.: XenMon: QoS Monitoring and Performance Profiling Tool. Technical report, HPL-2005-187 (2005)Google Scholar
  14. 14.
  15. 15. (last accessed, 04/02/2006)
  16. 16. (last accessed, 04/02/2006)
  17. 17.
    Mogul, J.C., Ramakrishnan, K.K.: Eliminating receive livelock in an interrupt-driven kernel. ACM Trans. Comput. Syst. 15(3) (1997)Google Scholar
  18. 18.
    Yuan, W., Nahrstedt, K.: Energy-efficient soft real-time cpu scheduling for mobile multimedia systems. In: Proc. of the 19th SOSP, pp. 149–163. ACM Press, New York (2003)Google Scholar
  19. 19.
    Barham, P.: A Fresh Approach to File System Quality of Service. In: Proc. of NOSSDAV (1998)Google Scholar
  20. 20.
    Black, R., Barham, P., Donnelly, A., Stratford, N.: Protocol Implementation in a Vertically Structured Operating System. In: Proc. of IEEE Conference on Computer Networks (1997)Google Scholar
  21. 21.
    Hand, S.M.: Self-paging in the Nemesis operating system. In: Proc. of the third USENIX OSDI, Berkeley, pp. 73–86. USENIX Association (1999)Google Scholar
  22. 22.
    Engler, D.R., Kaashoek, M.F., O’Toole, J.J.: Exokernel: an operating system architecture for application-level resource management. In: Proc. of the 15th ACM SOSP, pp. 251–266. ACM Press, New York (1995)Google Scholar
  23. 23.
    Whitaker, A., Shaw, M., Gribble, S.D.: Scale and performance in the Denali isolation kernel. In: Proc. of the 5th USENIX OSDI, Boston, MA (2002)Google Scholar
  24. 24.
    Karger, P.A.: A retrospective of the VAX VMM security kernel. IEEE Trans. on Software Engineering (1991)Google Scholar
  25. 25.
    Meushaw, R., Simard, D.: NetTop: Commercial Technology in high assurance applications (2005)Google Scholar
  26. 26.
    Bugnion, E., Devine, S., Rosenblum, M.: Disco: running commodity operating systems on scalable multiprocessors. In: Proc. of the 16th ACM SOSP, pp. 143–156. ACM Press, New York (1997)Google Scholar
  27. 27.
    Creasy, R.J.: The origin of the VM/370 time-sharing system. IBM Journal of Research and Development (1982)Google Scholar
  28. 28.
    Waldspurger, C.A.: Memory resource management in VMware ESX server. In: Proc. of the 5th USENIX OSDI (2002)Google Scholar
  29. 29.
    Chun, B., Culler, D., Roscoe, T., Bavier, A., Peterson, L., Wawrzoniak, M., Bowman, M.: PlanetLab: an overlay testbed for broad-coverage services. SIGCOMM Comput. Commun. Rev. 33(3), 3–12 (2003)CrossRefGoogle Scholar

Copyright information

© IFIP International Federation for Information Processing 2006

Authors and Affiliations

  • Diwaker Gupta
    • 1
  • Ludmila Cherkasova
    • 2
  • Rob Gardner
    • 2
  • Amin Vahdat
    • 1
  1. 1.University of CaliforniaSan DiegoUSA
  2. 2.Hewlett-Packard LaboratoriesUSA

Personalised recommendations