Performance Evaluation with Heavy Tailed Distributions

(Extended Abstract)
  • Mark E. Crovella
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2221)


Over the last decade an important new direction has developed in the performance evaluation of computer systems: the study of heavy-tailed distributions. Loosely speaking, these are distributions whose tails follow a power-law with low exponent, in contrast to traditional distributions (e.g., Gaussian, Exponential, Poisson) whose tails decline exponentially (or faster). In the late ’80s and early ’90s experimental evidence began to accumulate that some properties of computer systems and networks showed distributions with very long tails [7],[28],[29], and attention turned to heavy-tailed distributions in particular in the mid ’90s [3],[9],[23],[36],[44].


Heavy Tail Task Size Subexponential Distribution Large Task Small Observation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Réka Albert, Hawoong Jeong, and Albert-László Barabási. Diameter of the world wide web. Nature, 401:130–131, 1999.CrossRefGoogle Scholar
  2. 2.
    Virgílio Almeida, Azer Bestavros, Mark Crovella, and Adriana de Oliveira. Characterizing reference locality in the WWW. In Proceedings of 1996 International Conference on Parallel and Distributed Information Systems (PDIS’ 96), pages 92–103, December 1996.Google Scholar
  3. 3.
    Martin F. Arlitt and Carey L. Williamson. Internet web servers: Workload characterization and performance implications. IEEE/ACM Transactions on Networking, 5(5):631–645, 1997.CrossRefGoogle Scholar
  4. 4.
    Paul Barford and Mark E. Crovella. Generating representative Web workloads for network and server performance evaluation. In Proceedings of Performance’ 98/SIGMETRICS’ 98, pages 151–160, July 1998.Google Scholar
  5. 5.
    Lee Breslau, Pei Cao, Li Fan, Graham Phillips, and Scott Shenker. Web caching and zipf-like distributions: Evidence and implications. In Proceedings of INFOCOM’ 99, pages 126–134, 1999.Google Scholar
  6. 6.
    Andrei Broder, Ravi Kumar, Farzin Maghoul, Prabhakar Raghavan, Sridhar Rajagopalan, Raymie Stata, Andrew Tomkins, and Janet Wiener. Graph structure in the web: experiments and models. In Proceedings the Ninth World Wide Web Conference (WWW9), 2000.Google Scholar
  7. 7.
    R. Cáceres, P. B. Danzig, S. Jamin, and D. J. Mitzel. Characteristics of wide-area TCP/IP conversations. Computer Communication Review, 21, 1991.Google Scholar
  8. 8.
    M. E. Crovella, M. Harchol-Balter, and C. D. Murta. Task assignment in a distributed system: Improving performance by unbalancing load. Technical Report TR-97-018, Boston University Department of Computer Science, October 31 1997.Google Scholar
  9. 9.
    Mark E. Crovella and Azer Bestavros. Self-similarity in World Wide Web trafic: Evidence and possible causes. IEEE/ACM Transactions on Networking, 5(6):835–846, December 1997.CrossRefGoogle Scholar
  10. 10.
    Mark E. Crovella, Robert Frangioso, and Mor Harchol-Balter. Connection scheduling in Web servers. In 1999 USENIX Symposium on Internet Technologies and Systems (USITS’ 99), 1999.Google Scholar
  11. 11.
    Mark E. Crovella, Mor Harchol-Balter, and Cristina Duarte Murta. Task assignment in a distributed system: Improving performance by unbalancing load. In Proceedings of SIGMETRICS’ 98 (poster paper), July 1998.Google Scholar
  12. 12.
    Mark E. Crovella and Lester Lipsky. Simulations with heavy-tailed workloads. In Kihong Park and Walter Willinger, editors, Self-Similar Network Trafic and Performance Evaluation. Wiley / Wiley Interscience, New York, 1999.Google Scholar
  13. 13.
    Carlos A. Cunha, Azer Bestavros, and Mark E. Crovella. Characteristics of WWW client-based traces. Technical Report TR-95-010, Boston University Department of Computer Science, April1995.Google Scholar
  14. 14.
    Michalis Faloutsos, Petros Faloutsos, and Christos Faloutsos. On power-law relationships of the internet topology. In Proceedings of SIGCOMM’ 99, 1999.Google Scholar
  15. 15.
    Anja Feldmann, Jennifer Rexford, and Ramon Caceres. Eficient policies for carrying web trafic over flow-switched networks. IEEE/ACM Transactions on Networking, December 1998.Google Scholar
  16. 16.
    Anja Feldmann and Ward Whitt. Fitting mixtures of exponentials to long-tail distributions to analyze network performance models. In Proceedings of IEEE INFOCOM’97, pages 1098–1116, April 1997.Google Scholar
  17. 17.
    Sharad Garg, Lester Lipsky, and Maryann Robbert. The effect of power-taildistributions on the behavior of time sharing computer systems. In 1992 ACM Symposium on Applied Computing, Kansas City, MO, March 1992.Google Scholar
  18. 18.
    Steven Glassman. A caching relay for the World Wide Web. In Proceedings of the First International World Wide Web Conference, pages 69–76, 1994.Google Scholar
  19. 19.
    Charles M. Goldie and Claudia Kluppelberg. Subexponential distributions. In Robert J. Adler, Raisa E. Feldman, and Murad S. Taqqu, editors, A Practical Guide To Heavy Tails, pages 435–460. Chapman & Hall, New York, 1998.Google Scholar
  20. 20.
    Michael Greiner, Manfred Jobmann, and Lester Lipsky. The importance of powertail distributions for telecommunication tra.c models. Operations Research, 41, 1999.Google Scholar
  21. 21.
    S. D. Gribble, G. S. Manku, D. Roselli, E. A. Brewer, T. J. Gibson, and E. L. Miller. Self-similarity in file systems. In Proceedings of SIGMETRICS’ 98, pages 141–150, 1998.Google Scholar
  22. 22.
    M. Harchol-Balter, M. E. Crovella, and S. Park. The case for SRPT scheduling in Web servers. Technical Report MIT-LCS-TR-767, MIT Lab for Computer Science, October 1998.Google Scholar
  23. 23.
    M. Harchol-Balter and A. Downey. Exploiting process lifetime distributions for dynamic load balancing. ACM Transactions on Computer Systems, 15(3):253–285, 1997.CrossRefGoogle Scholar
  24. 24.
    Mor Harchol-Balter, Mark E. Crovella, and Cristina D. Murta. On choosing a task assignment policy for a distributed server system. Journal of Parallel and Distributed Computing, SpecialIssue on Software Support for Distributed Computing, September 1999.Google Scholar
  25. 25.
    Gordon Irlam. Unix file size survey-1993. Available at, September 1994.
  26. 26.
    Cheng Jin, Qian Chen, and Sugih Jamin. Inet: internet topology generator. Technical Report CSE-TR-433-00, U. Michigan Computer Science, 2000.Google Scholar
  27. 27.
    Butler W. Lampson. Hints for computer system design. Proceedings of the Ninth SOSP, in Operating Systems Review, 17(5):33–48, October 1983.CrossRefGoogle Scholar
  28. 28.
    W. E. Leland and T. J. Ott. Load-balancing heuristics and process behavior. In Proceedings of Performance and ACM Sigmetrics, pages 54–69, 1986.Google Scholar
  29. 29.
    W. E. Leland and D. V. Wilson. High time-resolution measurement and analysis of LAN trafic: Implications for LAN interconnection. In Proceeedings of IEEE Infocomm’ 91, pages 1360–1366, Bal Harbour, FL, 1991.Google Scholar
  30. 30.
    W.E. Leland, M.S. Taqqu, W. Willinger, and D.V. Wilson. On the self-similar nature of Ethernet tra.c (extended version). IEEE/ACM Transactions on Networking, 2:1–15, 1994.CrossRefGoogle Scholar
  31. 31.
    Benoit B. Mandelbrot. The Fractal Geometry of Nature. W. H. Freedman and Co., New York, 1983.Google Scholar
  32. 32.
    Alberto Medina, Ibrahim Matta, and John Byers. BRITE: a flexible generator of internet topologies. Technical Report BU-CS-TR-2000-05, Boston University Computer Science, January 2000.Google Scholar
  33. 33.
    Norifumi Nishikawa, Takafumi Hosokawa, Yasuhide Mori, Kenichi Yoshida, and Hiroshi Tsuji. Memory-based architecture for distributed WWW caching proxy. Computer Networks and ISDN Systems, 30:205–214, 1998.CrossRefGoogle Scholar
  34. 34.
    I. Norros. A storage model with self-similar input. Queueing Systems, 16:387–396, 1994.zbMATHCrossRefMathSciNetGoogle Scholar
  35. 35.
    Kihong Park, Gi Tae Kim, and Mark E. Crovella. On the relationship between file sizes, transport protocols, and self-similar network trafic. In Proceedings of the Fourth International Conference on Network Protocols (ICNP’96), pages 171–180, October 1996.Google Scholar
  36. 36.
    Vern Paxson. Empirically-derived analytic models of wide-area tcp connections. IEEE/ACM Transactions on Networking, 2(4):316–336, August 1994.CrossRefGoogle Scholar
  37. 37.
    Vern Paxson and Sally Floyd. Wide-area trafic: The failure of poisson modeling. IEEE/ACM Transactions on Networking, pages 226–244, June 1995.Google Scholar
  38. 38.
    D. Peterson and R. Grossman. Power laws in large shop DASD I/O activity. In CMG Proceedings, pages 822–833, December 1995.Google Scholar
  39. 39.
    David L. Peterson. Data center I/O patterns and power laws. In CMG Proceedings, December 1996.Google Scholar
  40. 40.
    David L. Peterson and David B. Adams. Fractalpatterns in DASD I/O trafic. In CMG Proceedings, December 1996.Google Scholar
  41. 41.
    Matthew Roughan, Darryl Veitch, and Michael Rumsewicz. Computing queuelength distributions for power-law queues. In Proceedings of INFOCOM’ 98, pages 356–363, 1998.Google Scholar
  42. 42.
    Anees Shaikh, Jennifer Rexford, and Kang Shin. Load-sensitive routing of longlived IP flows. In Proceedings of ACM SIGCOMM’ 99, pages 215–226, September 1999.Google Scholar
  43. 43.
    R. W. Weber. On the optimal assignment of customers to parallel servers. Journal of Applied Probability, 15:406–413, 1978.zbMATHCrossRefMathSciNetGoogle Scholar
  44. 44.
    Walter Willinger, Murad S. Taqqu, Robert Sherman, and Daniel V. Wilson. Selfsimilarity through high-variability: Statistical analysis of Ethernet LAN trafic at the source level. IEEE/ACM Transactions on Networking, 5(1):71–86, February 1997.CrossRefGoogle Scholar
  45. 45.
    G. K. Zipf. Human Behavior and the Principle of Least-Effort. Addison-Wesley, Cambridge, MA, 1949.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2001

Authors and Affiliations

  • Mark E. Crovella
    • 1
  1. 1.Department of Computer ScienceBoston UniversityBoston MAUSA

Personalised recommendations