Advertisement

Distributed Computing

, Volume 29, Issue 2, pp 127–142 | Cite as

Tight bounds for parallel randomized load balancing

Article

Abstract

Given a distributed system of \(n\) balls and \(n\) bins, how evenly can we distribute the balls to the bins, minimizing communication? The fastest non-adaptive and symmetric algorithm achieving a constant maximum bin load requires \(\varTheta (\log \log n)\) rounds, and any such algorithm running for \(r\in {\mathcal {O}}(1)\) rounds incurs a bin load of \(\varOmega ((\log n/\log \log n)^{1/r})\). In this work, we explore the fundamental limits of the general problem. We present a simple adaptive symmetric algorithm that achieves a bin load of 2 in \(\log ^* n+{\mathcal {O}}(1)\) communication rounds using \({\mathcal {O}}(n)\) messages in total. Our main result, however, is a matching lower bound of \((1-o(1))\log ^* n\) on the time complexity of symmetric algorithms that guarantee small bin loads. The essential preconditions of the proof are (i) a limit of \({\mathcal {O}}(n)\) on the total number of messages sent by the algorithm and (ii) anonymity of bins, i.e., the port numberings of balls need not be globally consistent. In order to show that our technique yields indeed tight bounds, we provide for each assumption an algorithm violating it, in turn achieving a constant maximum bin load in constant time.

Notes

Acknowledgments

We would like to thank Mohsen Ghaffari, Fabian Kuhn, Thomas Locher, and Reto Spöhel. This work was partly supported by the Swiss National Fund (SNF), the Society of Swiss Friends of the Weizmann Institute of Science, and the German Research Foundation (DFG, reference number Le 3107/1-1).

References

  1. 1.
    Adler, M., Chakrabarti, S., Mitzenmacher, M., Rasmussen, L.: Parallel randomized load balancing. In: Proc. 27th Symposium on Theory of Computing (STOC), pp. 238–247 (1995)Google Scholar
  2. 2.
    auf der Heide, F.M., Scheideler, C., Stemann, V.: Exploiting storage redundancy to speed up randomized shared memory simulations. Theor. Comput. Sci. 162(2), 245–281 (1996)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Awerbuch, B., Azar, Y., Grove, E.F., Kao, M.Y., Krishnan, P., Vitter, J.S.: Load balancing in the \(L_p\) norm. In: Proc. 36th Symposium on Foundations of Computer Science (FOCS), pp. 383–391 (1995)Google Scholar
  4. 4.
    Azar, Y., Broder, A.Z., Karlin, A.R., Upfal, E.: Balanced allocations. SIAM J. Comput. 29(1), 180–200 (1999)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Bast, H., Hagerup, T.: Fast and reliable parallel hashing. In: Proc. 3rd Symposium on Parallel Algorithms and Architectures (SPAA), pp. 50–61 (1991)Google Scholar
  6. 6.
    Berenbrink, P., Friedetzky, T., Goldberg, L.A., Goldberg, P.W., Hu, Z., Martin, R.: Distributed selfish load balancing. SIAM J. Comput. 37(4), 1163–1181 (2007)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Berenbrink, P., Friedetzky, T., Hu, Z., Martin, R.: On weighted balls-into-bins games. Theor. Comput. Sci. 409(3), 511–520 (2008)Google Scholar
  8. 8.
    Berenbrink, P., auf der Heide, F.M., Schröder, K.: Allocating weighted jobs in parallel. In: Proc. 9th Symposium on Parallel Algorithms and Architectures (SPAA), pp. 302–310 (1997)Google Scholar
  9. 9.
    Czumaj, A., Stemann, V.: Randomized allocation processes. Random Struct. Algorithms 18(4), 297–331 (2001)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Sarma, A.D., Holzer, S., Kor, L., Korman, A., Nanongkai, D., Pandurangan, G., Peleg, D., Wattenhofer, R.: Distributed verification and hardness of distributed approximation. In: Proc. 43rd Symposium on Theory of Computing (STOC) (2011)Google Scholar
  11. 11.
    Dubhashi, D., Ranjan, D.: Balls and bins: a study in negative dependence. Random Struct. Algorithms 13, 99–124 (1996)MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Even, G., Medina, M.: Revisiting randomized parallel load balancing algorithms. In: Proc. 16th Colloquium on Structural Information and Communication Complexity (SIROCCO), pp. 209–221 (2009)Google Scholar
  13. 13.
    Even, G., Medina, M.: Parallel randomized load balancing: a lower bound for a more general model. In: Proc. 36th Conference on Theory and Practice of Computer Science (SOFSEM), pp. 358–369 (2010)Google Scholar
  14. 14.
    Fich, F., Ruppert, E.: Hundreds of impossibility results for distributed computing. Distrib. Comput. 16(2–3), 121–163 (2003)CrossRefGoogle Scholar
  15. 15.
    Gil, J., auf der Heide, F.M., Wigderson, A.: The tree model for hashing: lower and upper bounds. SIAM J. Comput. 25(5), 936–955 (1996)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Gonnet, G.H.: Expected length of the longest probe sequence in hash code searching. J. ACM 28(2), 289–304 (1981)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Hagerup, T.: The log-star revolution. In: Proc. 9th Symposium on Theoretical Aspects of Computer Science (STACS), pp. 259–278 (1992)Google Scholar
  18. 18.
    Karp, R.M., Luby, M., auf der Heide, F.M.: Efficient PRAM simulation on a distributed memory machine. Algorithmica 16, 517–542 (1996)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Kenthapadi, K., Panigrahy, R.: Balanced allocation on graphs. In: Proc. 7th Symposium on Discrete Algorithms (SODA), pp. 434–443 (2006)Google Scholar
  20. 20.
    Kleinberg, R., Piliouras, G., Tardos, E.: Load balancing without regret in the bulletin board model. In: Proc. 28th Symposium on Principles of Distributed Computing (PODC), pp. 56–62 (2009)Google Scholar
  21. 21.
    Koutsoupias, E., Mavronicolas, M., Spirakis, P.G.: Approximate equilibria and ball fusion. Theory Comput. Syst. 36(6), 683–693 (2003)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Kuhn, F., Moscibroda, T., Wattenhofer, R.: What cannot Be computed locally! In: Proc. 23rd Symposium on the Principles of Distributed Computing (PODC) (2004)Google Scholar
  23. 23.
    Lenzen, C.: Optimal deterministic routing and sorting on the congested clique. In: Proc. 32nd Symposium on Principles of Distributed Computing (PODC), pp. 42–50 (2013)Google Scholar
  24. 24.
    Lenzen, C., Wattenhofer, R.: Tight bounds for parallel randomized load balancing. Comput. Res. Repos. arXiv:1102.5425 (2011)
  25. 25.
    Lenzen, C., Wattenhofer, R.: Tight bounds for parallel randomized load balancing: extended abstract. In: Proc. 43rd Symposium on Theory of Computing (STOC), pp. 11–20 (2011)Google Scholar
  26. 26.
    Linial, N.: Locality in distributed graph algorithms. SIAM J. Comput. 21(1), 193–201 (1992)MathSciNetCrossRefMATHGoogle Scholar
  27. 27.
    Lotker, Z., Patt-Shamir, B., Peleg, D.: Distributed MST for constant diameter graphs. Distrib. Comput. 18(6), 453–460 (2006)CrossRefMATHGoogle Scholar
  28. 28.
    Lynch, N.: A hundred impossibility proofs for distributed computing. In: Proc. 8th Symposium on Principles of distributed computing (PODC), pp. 1–28 (1989)Google Scholar
  29. 29.
    Matias, Y., Vishkin, U.: Converting high probability into nearly-constant time—with applications to parallel hashing. In: Proc. 23rd Symposium on Theory of Computing (STOC), pp. 307–316 (1991)Google Scholar
  30. 30.
    Mitzenmacher, M.: The Power of Two Choices in Randomized Load Balancing. Ph.D. thesis, University of California, Berkeley (1996)Google Scholar
  31. 31.
    Mitzenmacher, M.: How Useful is Old Information? Tech. rep., Systems Research Center, Digital Equipment Corporation (1998)Google Scholar
  32. 32.
    Mitzenmacher, M.: On the analysis of randomized load balancing schemes. In: Proc. 10th Symposium on Parallel Algorithms and Architectures (SPAA), pp. 292–301 (1998)Google Scholar
  33. 33.
    Mitzenmacher, M., Prabhakar, B., Shah, D.: Load balancing with memory. In: Proc. 43th Symposium on Foundations of Computer Science (FOCS), pp. 799–808 (2002)Google Scholar
  34. 34.
    Mitzenmacher, M., Richa, A., Sitaraman, R.: Handbook of Randomized Computing, vol. 1, chap. The Power of Two Random Choices: A Survey of the Techniques and Results, pp. 255–312. Kluwer Academic Publishers, Dordrecht (2001) Google Scholar
  35. 35.
    Patt-Shamir, B., Teplitsky, M.: The round complexity of distributed sorting: extended abstract. In: Proc. 30th Symposium on Principles of Distributed Computing (PODC), pp. 249–256 (2011)Google Scholar
  36. 36.
    Peleg, D., Rubinovich, V.: A near-tight lower bound on the time complexity of distributed mst construction. In: Proc. 40th Symposium on Foundations of Computer Science (FOCS), pp. 253–261 (1999)Google Scholar
  37. 37.
    Peres, Y., Talwar, K., Wieder, U.: The \((1+\beta )\)-choice process and weighted balls-into-bins. In: Proc. 21th Symposium on Discrete Algorithms (SODA), pp. 1613–1619 (2010)Google Scholar
  38. 38.
    Raman, R.: The power of collision: randomized parallel algorithms for chaining and integer sorting. In: Proc. 10th Conference on Foundations of Software Technology and Theoretical Computer Science (FSTTCS), pp. 161–175 (1990)Google Scholar
  39. 39.
    Schneider, J., Wattenhofer, R.: A new technique for distributed symmetry breaking. In: 29th Symposium on Principles of Distributed Computing (PODC) (2010)Google Scholar
  40. 40.
    Stemann, V.: Parallel balanced allocations. In: Proc. 8th Symposium on Parallel Algorithms and Architectures (SPAA), pp. 261–269 (1996)Google Scholar
  41. 41.
    Talwar, K., Wieder, U.: Balanced allocations: the weighted case. In: Proc. 39th Symposium on Theory of Computing (STOC), pp. 256–265 (2007)Google Scholar
  42. 42.
    Vöcking, B.: How asymmetry helps load balancing. J. ACM 50(4), 568–589 (2003)MathSciNetCrossRefMATHGoogle Scholar
  43. 43.
    Wieder, U.: Balanced allocations with heterogenous bins. In: Proc. 19th Symposium on Parallel Algorithms and Architectures (SPAA), pp. 188–193 (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Computer Science and Artificial Intelligence LaboratoryMassachusetts Institute of TechnologyCambridgeUSA
  2. 2.Computer Engineering and Networks LaboratoryETH ZurichZurichSwitzerland

Personalised recommendations