Skip to main content

An Automated Benchmarking Toolset

  • Conference paper
  • First Online:
High Performance Computing and Networking (HPCN-Europe 2000)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 1823))

Included in the following conference series:

  • 387 Accesses

Abstract

The drive for performance in parallel computing and the need to evaluate platform upgrades or replacements are major reasons frequent running of benchmark codes has become commonplace for application and platform evaluation and tuning. NIST is developing a prototype for an automated benchmarking toolset to reduce the manual effort in running and analyzing the results of such benchmarks. Our toolset consists of three main modules. A Data Collection and Storage module handles the collection of performance data and implements a central repository for such data. Another module provides an integrated mechanism to analyze and visualize the data stored in the repository. An Experiment Control Module assists the user in designing and executing experiments. To reduce the development effort this toolset is built around existing tools and is designed to be easily extensible to support other tools.

This NIST contribution is not subject to copyright in the United States. Certain commercial items may be identified but that does not imply recommendation or endorsement by NIST, nor does it imply that those items are necessarily the best available.

Visiting scientist from University of Maryland, UMIACS.

Guest researcher from the Institut National des Télécommunications, France.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Ruth A. Aydt, “The Pablo Self-Defining Data Format”, http://www.pablo.cs.uiuc.edu.

  2. George E.P. Box, William G. Hunter, “Statistics for Experimenters, an introduction to design, data analysis, and modeling building”, 1978.

    Google Scholar 

  3. “DQS-Distributed Queueing System”, http://www.scri.fsu.edu/~pasko/dqs.html

  4. “IDL-The Interactive Data Language”, http://www.rsinc.com/idl/index.cfm

  5. “MPIProf”, http://www.itl.nist.gov/div895/cmr/mpiprof

  6. “The MathWorks-MATLAB Introduction”. http://www.mathworks.com

  7. B.P. Mille et al., “The Paradyn Parallel Performance Measurement Tools”, IEEE Computer, 28, 1995.

    Google Scholar 

  8. Barton P. Miller, Karen L. Karavanic. “Improving Online Performance Diagnosis by the Use of Historical Performance Data”, SC’99, Portland, Oregon (USA) November 1999.

    Google Scholar 

  9. Alan Mink, “The Multikron Project”, http://www.multikron.nist.gov

  10. “Octave Home Page”, http://www.che.wisc.edu/octave

  11. “PostgreSQL Home Page”, http://www.pyrenet.fr/postgresql

  12. “Scotty-Tcl Extensions for Network Management”, http://wwwhome.cs.utwente.nl/~schoenw/scotty

  13. University of California at Davis, “The UCD-SNMP project home page”, http://www.ece.ucdavis.edu/ucd-snmp

  14. Pallas Gmbh, “VAMPIR”, http://www.pallas.de/pages/vampir.htm, 1999.

  15. J. C. Yan and S. R. Sarukkai, “Analyzing Parallel Program Performance Using Normalized Performance Indices and Trace Transformation Techniques”, Parallel Computing Vol. 22, No. 9, November 1996. pages 1215–1237, 1996.

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2000 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Courson, M., Mink, A., Marçais, G., Traverse, B. (2000). An Automated Benchmarking Toolset. In: Bubak, M., Afsarmanesh, H., Hertzberger, B., Williams, R. (eds) High Performance Computing and Networking. HPCN-Europe 2000. Lecture Notes in Computer Science, vol 1823. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45492-6_50

Download citation

  • DOI: https://doi.org/10.1007/3-540-45492-6_50

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-67553-2

  • Online ISBN: 978-3-540-45492-2

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics