Theory of Computing Systems

, Volume 45, Issue 2, pp 280–301 | Cite as

On the Price of Heterogeneity in Parallel Systems



Suppose we have a parallel or distributed system whose nodes have limited capacities, such as processing speed, bandwidth, memory, or disk space. How does the performance of the system depend on the amount of heterogeneity of its capacity distribution? We propose a general framework to quantify the worst-case effect of increasing heterogeneity in models of parallel systems. Given a cost function g(C,W) representing the system’s performance as a function of its nodes’ capacities C and workload W (such as the makespan of an optimum schedule of jobs W on machines C), we say that g has price of heterogeneityα when for any workload, cost cannot increase by more than a factor α if node capacities become arbitrarily more heterogeneous. The price of heterogeneity also upper bounds the “value of parallelism”: the maximum benefit obtained by increasing parallelism at the expense of decreasing processor speed. We give constant or logarithmic bounds on the price of heterogeneity of several well-known job scheduling and graph degree/diameter problems, indicating that in many cases, increasing heterogeneity can never be much of a disadvantage.


Heterogeneity Parallel systems Scheduling Makespan Precedence constrained scheduling 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
  2. 2.
    Almeida, V.A.F., Vasconcelos, I.M.M., Árabe, J.N.C., Menascé, D.A.: Using random task graphs to investigave the potential benefits of heterogeneity in parallel systems. In: Proc. ACM/IEEE Conference on Supercomputing (1992) Google Scholar
  3. 3.
    Almeida, V., Menascé, D.: Cost-performance analysis of heterogeneity in supercomputer architectures. In: Proc. ACM/IEEE Conference on Supercomputing (1990) Google Scholar
  4. 4.
    Chawathe, Y., Ratnasamy, S., Breslau, L., Lanham, N., Shenker, S.: Making Gnutella-like P2P systems scalable. In: Proceedings of ACM SIGCOMM (2003) Google Scholar
  5. 5.
    Chudak, F.A., Shmoys, D.B.: Approximation algorithms for precedence-constrained scheduling problems on parallel machines that run at different speeds. In: Proc. 8th Ann. ACM-SIAM Symp. on Discrete Algorithms (SODA), pp. 581–590 (1997) Google Scholar
  6. 6.
    Comellas, F., Delorme, C.: The (degree, diameter) problem for graphs:
  7. 7.
    Foster, I., Iamnitchi, A.: On death, taxes, and the convergence of peer-to-peer and grid computing. In: Proc. IPTPS (2002) Google Scholar
  8. 8.
    Garey, M.R., Johnson, D.S.: Computers and Intractability: A Guide to the Theory of NP-Completeness. Freeman, New York (1979) MATHGoogle Scholar
  9. 9.
    Godfrey, B., Lakshminarayanan, K., Surana, S., Karp, R., Stoica, I.: Load balancing in dynamic structured P2P systems. In: Proc. IEEE INFOCOM, Hong Kong (2004) Google Scholar
  10. 10.
    Brighten, P.G., Stoica, I.: Heterogeneity and load balance in distributed hash tables. In: Proc. IEEE INFOCOM (2005) Google Scholar
  11. 11.
    Graham, R.L.: Bounds on multiprocessing timing anomalies. Bell Syst. Tech. J. 45, 1563–1581 (1966) Google Scholar
  12. 12.
    Koutsoupias, E.: Coordination mechanisms for congestion games. In: Sigact News, December 2004 Google Scholar
  13. 13.
    Ledlie, J., Seltzer, M.: Distributed, secure load balancing with skew, heterogeneity, and churn. In: Proc. INFOCOM (2005) Google Scholar
  14. 14.
    Lowney, G.: Invited talk: Why intel is designing multi-core processors. In: Proc. SPAA (2006) Google Scholar
  15. 15.
    Marshall, A.W., Olkin, I.: Inequalities: Theory of Majorization and its Applications. Academic Press, New York (1979) MATHGoogle Scholar
  16. 16.
    Moreno, J.H.: Invited talk: Chip-level integration: The new frontier for microprocessor architecture. In: Proc. SPAA (2006) Google Scholar
  17. 17.
    Park, K., Pai, V.: Comon: A monitoring infrastructure for PlanetLab.
  18. 18.
    Ratnasamy, S., Shenker, S., Stoica, I.: Routing algorithms for DHTs: Some open questions. In: Proc. IPTPS (2002) Google Scholar
  19. 19.
    Saroiu, S., Krishna Gummadi, P., Gribble, S.D.: A measurement study of peer-to-peer file sharing systems. In: Proc. MMCN, San Jose, CA, USA, January 2002 Google Scholar
  20. 20.
    Stidham, S.: On the optimality of single-server queueing systems. Oper. Res. 18, 708–732 (1970) MATHCrossRefMathSciNetGoogle Scholar
  21. 21.
    Suri, S., Tóth, C.D., Zhou, Y.: Selfish load balancing and atomic congestion games. In: Proc. SPAA (2004) Google Scholar
  22. 22.
    Wierman, A., Osogami, T., Harchol-Balter, M., Scheller-Wolf, A.: How many servers are best in a dual-priority FCFS system? Perform. Eval. 63(12), 1253–1272 (2006) CrossRefGoogle Scholar
  23. 23.
    Yang, X., de Veciana, G.: Service capacity of peer to peer networks. In: Proc. INFOCOM (2004) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  1. 1.Computer Science DivisionUC BerkeleyBerkeleyUSA

Personalised recommendations