Advertisement

Journal of Grid Computing

, Volume 4, Issue 3, pp 287–309 | Cite as

The Design, Performance, and Use of DiPerF: An automated DIstributed PERformance evaluation Framework

  • Ioan RaicuEmail author
  • Catalin Dumitrescu
  • Matei Ripeanu
  • Ian Foster
Article

Abstract

We present DiPerF, a DIstributed PERformance evaluation Framework, aimed at simplifying and automating performance evaluation of networked services. DiPerF coordinates a pool of machines that access a target service and collect performance measurements, aggregates these measurements, and generates performance statistics. The aggregate data collected provide information on service throughput, service response time, service ‘fairness’ when serving multiple clients concurrently, and on the impact of network connectivity on service performance. We have used DiPerF in various environments (PlanetLab, Grid3, TeraGrid, and a cluster) and with a large number of services. This paper provides data that demonstrates that DiPerF is accurate: The aggregate client view matches the tested service view within a few percents, and scalable: DiPerF can handle more than 10,000 clients and 100,000 transactions per second. Moreover, rapid adoption and extensive use demonstrate that the ability to automate performance characteristics extraction makes DiPerF a valuable tool.

Key words

performance evaluation Grid computing Globus Toolkit 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Peterson, L., Anderson, T., Culler, D., Roscoe, T.: A blueprint for introducing disruptive technology into the internet. In: Proceedings of the First ACM Workshop on Hot Topics in Networking (HotNets), October 2002Google Scholar
  2. 2.
    Foster, I., et al.: The Grid2003 production grid: Principles and practice. In: 13th IEEE Intl. Symposium on High Performance Distributed Computing, 2004Google Scholar
  3. 3.
    Foster, I., Kesselman, C.: The Grid 2: Blueprint for a New Computing Infrastructure, Chapter 1: Perspectives. Elsevier (2003)Google Scholar
  4. 4.
    Dumitrescu, C., Foster, I.: GRUBER: A Grid Resource SLA-based Broker. EuroPar (2005)Google Scholar
  5. 5.
    Dumitrescu, C., Raicu, I., Foster, I.: DI-GRUBER: A Distributed Approach for Grid Resource Brokering. SC (2005)Google Scholar
  6. 6.
    Raicu, I.: A performance study of the Globus Toolkit® and Grid Services via DiPerF, an automated DIstributed PERformance testing framework. University of Chicago, Computer Science Department, MS thesis, May 2005Google Scholar
  7. 7.
    Raicu, I.: Decreasing end-to-end job execution times by increasing resource utilization using predictive scheduling in the Grid. Technical report, Grid Computing Seminar, Department of Computer Science, University of Chicago, March 2005Google Scholar
  8. 8.
    Dumitrescu, C., Raicu, I., Foster, I.: Experiences in running workloads over Grid3”. The 4th International Conference on Grid and Cooperative Computing (GCC’05), Beijing China, December, 2005Google Scholar
  9. 9.
    Allcock, B., Bresnahan, J., Kettimuthu, R., Link, M., Dumitrescu, C., Raicu, I., Foster, I.: Zebra: The Globus Striped GridFTP Framework and Server. sc, p. 54, ACM/IEEE SC 2005 Conference (SC’05), 2005Google Scholar
  10. 10.
    Dumitrescu, C., Raicu, I., Ripeanu, M., Foster, I.: DiPerF: An automated DIstributed PERformance testing Framework. 5th International IEEE/ACM Workshop in Grid Computing, Pittsburg, PA, 2004Google Scholar
  11. 11.
    Internet Protocol Performance Metrics. http://www.advanced.org/ippm/index.html, 1998
  12. 12.
    Paxson, V., Almes, G., Mahdavi, J., Mathis, M.: Framework for IP performance metrics. IETF RFC 2330, 1998Google Scholar
  13. 13.
    The Surveyor Project. http://www.advanced.org/csgippm/, 1998
  14. 14.
    Keynote Systems Inc. http://www.keynote.com, 1998
  15. 15.
    Steigner, Ch., Wilke, J.: Isolating performance bottlenecks in network applications. In: Proceedings of the International IPSI-2003 Conference, Sveti Stefan, Montenegro, October 4–11, 2003Google Scholar
  16. 16.
    Lowekamp, B.B., Miller, N., Karrer, R., Gross, T., Steenkiste, P.: Design, implementation, and evaluation of the Remos Network Monitoring System. Journal of Grid Computing 1(1), 75–93 (2003)CrossRefGoogle Scholar
  17. 17.
    Barford, P., Crovella, M.E.: Measuring Web performance in the wide area. Performance Evaluation Review, Special Issue on Network Traffic Measurement and Workload Characterization, August 1999Google Scholar
  18. 18.
    Banga, G., Druschel, P.: Measuring the capacity of a Web server under realistic loads. World Wide Web Journal (Special Issue on World Wide Web Characterization and Performance Evaluation), 1999Google Scholar
  19. 19.
    Gunter, D., Tierney, B., Tull, C.E., Virmani, V.: On-Demand Grid Application Tuning and Debugging with the NetLogger Activation Service, 4th International Workshop on Grid Computing, Grid2003, Phoenix, Arizona, November 17th, 2003Google Scholar
  20. 20.
    Tsouloupas, G., Dikaiakos, M.: GridBench: A tool for benchmarking grids. 4th International Workshop on Grid Computing, Grid2003, Phoenix, Arizona, November 17th, 2003Google Scholar
  21. 21.
    The Globus Alliance. Globus Toolkit 3.0 Test Results Page. http://www-unix.globus.org/ogsa/tests/gt3_tests_result.html
  22. 22.
    The Globus Alliance: Overview and Status of Current GT Performance Studies. http://www-unix.globus.org/toolkit/docs/development/3.9.5/perf_overview.html
  23. 23.
    Lee, C., Wolski, R., Foster, I., Kesselman, C., Stepanek, J.: A Network Performance Tool for Grid Environments, SC ’99Google Scholar
  24. 24.
    Wolski, R., Spring, N., Hayes, J.: The network weather service: A distributed resource performance forecasting service for metacomputing. Future Generation Computing Systems, 1999Google Scholar
  25. 25.
  26. 26.
    Chandra, A., Mosberger, D.: Scalability of Linux event-dispatch mechanisms. In: Proceedings of the USENIX Annual Technical Conference (USENIX 2001), Boston, Massachusetts, June 2001Google Scholar
  27. 27.
    Minar, N.: A survey of the NTP protocol. MIT Media Lab, 1999, http://xenia.media.mit.edu/~nelson/research/ntp-survey99
  28. 28.
    Williams, T., Kelley, C.: gnuplot, an interactive plotting program. http://www.gnuplot.info/docs/gnuplot.pdf
  29. 29.
    Massie, M.L., Chun, B.N., Culler, D.E.: The ganglia distributed monitoring system: Design, implementation, and experience. Parallel Comput. 30(7), (July 2004)Google Scholar
  30. 30.
  31. 31.
    Chun, B., Culler, D., Roscoe, T., Bavier, A., Peterson, L., Wawrzoniak, M., Bowman, M.: PlanetLab: An overlay testbed for broad-coverage services. ACM Computer Communications Review, vol. 33, no. 3, July 2003Google Scholar
  32. 32.
    Czajkowski, K., Fitzgerald, S., Foster, I., Kesselman, C.: Grid information services for distributed resource sharing. In: Proceedings of the Tenth IEEE International Symposium on High-Performance Distributed Computing (HPDC-10). IEEE (August 2001)Google Scholar
  33. 33.

Copyright information

© Springer Science + Business Media B.V. 2006

Authors and Affiliations

  • Ioan Raicu
    • 1
    Email author
  • Catalin Dumitrescu
    • 2
  • Matei Ripeanu
    • 3
  • Ian Foster
    • 1
    • 4
  1. 1.Computer Science DepartmentThe University of ChicagoChicagoUSA
  2. 2.Electrical Engineering, Mathematical and Computer Science DepartmentDelft University of TechnologyDelftThe Netherlands
  3. 3.Electrical and Computer Engineering DepartmentThe University of British ColumbiaVictoriaUSA
  4. 4.Mathematics and Computer Science DivisionArgonne National LaboratoryArgonneUSA

Personalised recommendations