Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

An effective speedup metric for measuring productivity in large-scale parallel computer systems

  • 165 Accesses

  • 9 Citations


With the parallel computer systems scaling-up, the measure index for performance of the systems demands a shift from traditional “high performance” to “high productivity.” This brings a new challenge to defining a synthetic, yet meaningful, measure index of multiple productivity variables; namely computing performance, reliability, energy consumption, parallel software development, etc. Traditional measures for large-scale parallel computer systems merely focus on computing performance, and are incapable of measuring the multiple productivity variables simultaneously in an effective manner. A recently proposed market-related money model, which pursues high utility/cost ratio, relies on money as a measure to consider the multiple productivity variables. Differing from the previous models, this paper proposes a novel system productivity speedup metric for large-scale parallel computer systems. The metric uses speedup instead of money to comprehensively unify the measures of multiple productivity variables. Finally, we propose a trade-off productivity measurement to weigh different productivity variables, to address different design targets. The measurement can facilitate the system evaluation, expose future technique tendencies, and guide future system design.

This is a preview of subscription content, log in to check access.


  1. 1.

    Funk A, Basili V, Hochstein L, Kepner J (2006) Analysis of parallel software development using the relative time productivity metric. CT Watch Q 2(4A), Nov 2006. Available: http://www.ctwatch.org/quarterly/

  2. 2.

    Kumar V, Gupta A (1994) Analysis of scalability of parallel algorithms and architectures. J Parallel Distrib Comput 22(3):379–391

  3. 3.

    Amdahl GM (1967) Validity of the single-processor approach to achieving large scale computing capabilities. In: Proceedings of AFIPS spring joint computer conference, Atlantic City, NJ, 18–20. AFIPS Press, Reston, pp 483–485

  4. 4.

    Gustafson J (1988) Reevaluating Amdahl’s law. Commun ACM 31(5):532–533

  5. 5.

    Sun X, Ni LM (1993) Scalable problems and memory-bounded speedup. J Parallel Distrib Comput 19(1):27–37

  6. 6.

    Faulk S, Gustafson J, Johnson P et al (2004) Measuring HPC productivity. Int J High Perform Comput Appl 18(4):459–473

  7. 7.

    Kepner J (2004) HPC productivity: An overarching view. Int J High Perform Comput Appl 18(4):393–397

  8. 8.

    Yang X, Du Y, Wang P et al (2007) The fault tolerant parallel algorithm: the parallel recomputing based failure recovery. In: Proceedings of the sixteenth international conference on parallel architectures and compilation techniques (PACT 2007), Brasov, Romania

  9. 9.

    Chen J, Li X (2000) A practical scalability metric. In: Proceedings of the HPC Asia 2000, Singapore

  10. 10.

    Adiga NR, Almasi GS et al (2002) An overview of the BlueGene/L supercomputer. In: Proceedings of the 2002 ACM/IEEE conference on supercomputing. ACM, New York, pp 1–22

  11. 11.

    DARPA (2004) High productivity computing systems (HPCS). Available at: http://www.darpa.mil/ipto/programs/hpcs/index.htm

  12. 12.

    Bronevetsky G, Daniel M, Keshav P, Paul S (2003) Automated application-level checkpointing of mpi programs. In: Proceedings of ACM SIGPLAN symposium on principles and practice of parallel programming (PPoPP 2003), San Diego, CA, pp 84–94

  13. 13.

    Engelmann C, Geist A (2003) A diskless checkpointing algorithm for super-scale architectures applied to the fast Fourier transform. In: Proceedings of the 1st international workshop on challenges of large applications in distributed environments (CLADE 2003), Seattle, WA, USA, pp 47–52

  14. 14.

    Chakravorty S, Kale LV (2007) A fault tolerance protocol with fast fault recovery. In: Proceedings of the 21th IEEE international parallel & distributed processing symposium (IPDPS 2007), Long Beach, California, USA

  15. 15.

    Gao Q, Yu W, Huang W et al (2006) Application-transparent checkpoint/restart for MPI programs over InfiniBand. In: Proceedings of the 2006 international conference on parallel processing (ICPP 2006). IEEE Computer Society, Los Alamitos, pp 471–478

  16. 16.

    Bailey DH (2002) 21st century high-end computing. In: Invited talk application, algorithms and architectures workshop for BlueGene/L

  17. 17.

    Feng W (2005) The importance of being low power in high performance computing. Available at: http://www.ctwatch.org/quarterly/articles/2005/08/the-importance-of-being-low-power-in-high-performance-computing/

  18. 18.

    Feng X, Ge R, Cameron KW (2005) Power and energy profiling of scientific applications on distributed systems. In: Proceedings of 19th international parallel and distributed processing symposium (IPDPS 2005). IEEE Computer Society, Los Alamitos

  19. 19.

    Hsu C, Feng W (2005) A power-aware run-time system for high-performance computing. In: Proceedings of the ACM/IEEE SC2005 conference on high performance networking and computing

  20. 20.

    Kadayif I, Kandemir M, Vijaykrishnan N et al. (2004) Exploiting processor workload heterogeneity for reducing energy consumption in chip multiprocessors. In: Proceedings of design, automation and test in Europe (DATE 2004). Paris, France, pp 1158–1163

  21. 21.

    Li J, Martinez JF (2006) Dynamic power-performance adaptation of parallel computation on chip multiprocessors. In: Proceedings of the international symposium on high performance computer architecture (HPCA 2006)

  22. 22.

    Carver J, Hochstein L, Kendall R (2006) Observations about software development for high end computing, CT Watch Q, 2(4A), Nov 2006. Available at: http://www.ctwatch.org/quarterly/

  23. 23.

    Asgari S et al (2005) Generating testable hypotheses from tacit knowledge for high productivity computing. In: Proceedings of SE-HPCS’05, Missouri, USA

  24. 24.

    Kepner J (2004) HPC Productivity Model Synthesis. Int J High Perform Comput Appl 18(4):505–516

  25. 25.

    Snir M, Bade D (2004) A framework for measuring supercomputer productivity. Int J High Perform Comput Appl 18(4), Nov 2004

  26. 26.

    Sterling T (2004) Productivity metrics and models for high performance computing. Int J High Perform Comput Appl 18(4):433–440

  27. 27.

    Funk A, Basili V, Hochstein L, Kepner J (2005) Application of a development time productivity metric to parallel software development. In: Proceedings of SE-HPCS’05, Missouri, USA

  28. 28.

    Brooks FP (1995) The mythical man-month: essays on software engineering. Addison-Wesley, Reading. Anniversary Ed, ISBN 0201835959

  29. 29.

    Perry D, Staudenmayer N, Votta L (1996) Understanding and improving time usage in software development. In: Wolf A, Fuggetta A (eds) Trends in software: software process. Wiley, New York

  30. 30.

    Porter A, Votta L, Basili V (1995) Comparing detection methods for software requirement inspections: a replicated experiment. IEEE Trans Softw Eng 21(6):563–575

  31. 31.

    Boehm B (1981) Software engineering economics. Prentice Hall, Englewood Cliffs

  32. 32.

    Boehm B, Clark B, Horowitz E, Westland C, Madachy R, Selby R (1995) Cost models for future software life cycle processes: COCOMO 2.0. Ann Softw Eng 1:57–94

  33. 33.

    MacCormack A, Kemerer CF, Cusumano M, Crandall B (2003) Trade-offs between productivity and quality in selecting software development practices. IEEE Softw 20:78–85

  34. 34.

    Zelkowitz M, Basili V, Asgari S, Hochstein L, Hollingsworth J, Nakamura T (2005) Measuring productivity on high performance computers. In: IEEE Symp on software metrics, Como, Italy, Sep 2005

  35. 35.

    Grama A, Gupta A, Kumar U (1993) Isoefficiency function: a scalability metric for parallel algorithms and architectures. IEEE Parallel Distrib Technol 1(3):12–21

  36. 36.

    Hwang K, Xu Z (1998) Scalable parallel computing: technique, architecture, programming. McGraw-Hill, Boston

  37. 37.

    Sun X, Rover D (1994) Scalability of parallel algorithm machine combinations. IEEE Trans Parallel Distrib Syst 5(6):99–613

Download references

Author information

Correspondence to Xuejun Yang.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Yang, X., Du, J. & Wang, Z. An effective speedup metric for measuring productivity in large-scale parallel computer systems. J Supercomput 56, 164–181 (2011). https://doi.org/10.1007/s11227-009-0355-9

Download citation

  • Large-scale parallel computer system
  • Productivity speedup
  • Computing performance
  • Reliability
  • Energy consumption
  • Parallel software development