Value-Based Management of Software Testing

  • Rudolf Ramler
  • Stefan Biffl
  • Paul Grünbacher


Testing is one of the most resource-intensive activities in software development and consumes between 30 and 50% of total development costs according to many studies. Testing is however often not organized to maximize business value and not aligned with a project’s mission. Path, branch, instruction, mutation, scenario, or requirement testing usually treat all aspects of software as equally important, while in practice 80% of the value often comes from 20% of the software. In order to maximize the return of investment gained from software testing, the management of testing needs to maximize its value contribution. In this chapter we motivate the need for value-based testing, describe practices supporting the management of value-based testing, outline a framework for value-based test management, and illustrate the framework with an example.


Value-based software engineering value-based testing cost of testing benefits of testing test management 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. (Amland, 1999).
    Amland, S.: Risk Based Testing and Metrics. In: EuroSTAR’99: 5 th European International Conference on Software Testing Analysis and Review, Barcelona, Spain, November 1999Google Scholar
  2. (Bach, 1999).
    Bach, J.: Risk and Requirements-Based Testing. IEEE Computer, 32(6), pp 113–114 (June 1999)Google Scholar
  3. (Beizer, 1990).
    Beizer, B.: Software Testing Techniques, 2nd ed (Van Nostrand Reinhold, New York 1990)Google Scholar
  4. (Biffl et al., 2001)
    Biffl, S., Freimut, B., and Laitenberger, O.: Investigating the Cost-Effectiveness of Reinspections in Software Development. In: ACM/IEEE International Conference on Software Engineering, Toronto, Canada, IEEE Comp. Soc. Press, May 2001Google Scholar
  5. (Black, 2002).
    Black, R.: Managing the Testing Process: Practical Tools and Techniques for Managing Hardware and Software Testing (Wiley, New York 2002)Google Scholar
  6. (Boehm, 1988).
    Boehm, B. W.: A Spiral Model of Software Development and Enhancement. IEEE Computer, 21(5), pp 61–72 (May 1988)CrossRefGoogle Scholar
  7. (Boehm, 2000a).
    Boehm, B. W.: Spiral Development: Experience, Principles, and Refinements, CMU/SEI-2000-SR-008. Spiral Development Workshop, July 2000Google Scholar
  8. (Boehm, 2000b).
    Boehm, B. W.: The Art of Expectations Management. IEEE Computer, 33(1), pp 122–124 (January 2000)CrossRefzbMATHGoogle Scholar
  9. (Boehm, 2003).
    Boehm, B. W.: Value-Based Software Engineering. Software Engineering Notes, 28(2), (March 2003)Google Scholar
  10. (Boehm and Basili, 2001).
    Boehm, B. W., and Basili, V.R.: Software Defect Reduction Top 10 List. IEEE Computer, 34(1), pp 135–137 (January 2001)CrossRefGoogle Scholar
  11. (Boehm and Huang, 2003).
    Boehm, B. W., and Huang, L.G.: Value-Based Software Engineering: A Case Study. IEEE Computer, 36(3), pp 33–41 (March 2003)CrossRefGoogle Scholar
  12. (Boehm et al., 2001)
    Boehm, B. W., Grünbacher, P., and Briggs, R.O.: Developing Groupware for Requirements Negotiation: Lessons Learned. IEEE Software, 18(3), pp 46–55 (May/June 2001)CrossRefzbMATHGoogle Scholar
  13. (Bullock, 2000).
    Bullock, J.: Calculating the Value of Testing. Software Testing and Quality Engineering, 2(3), pp 56–61 (May/June 2000)MathSciNetGoogle Scholar
  14. (Burnstein, 2003).
    Burnstein, I: Practical Software Testing: A Process-oriented Approach (Springer, Berlin Heidelberg New York 2002)zbMATHGoogle Scholar
  15. (Cohen et al., 2004)
    Cohen, C.F., Birkin, S.J., Garfield, M.J., and Webb, H.W.: Managing Conflict in Software Testing. CACM, 47(1), pp 76–81 (January 2004)CrossRefGoogle Scholar
  16. (Crosby, 1979).
    Crosby, P.B.: Quality Is Free: The Art of Making Quality Certain (McGraw-Hill, 1979)Google Scholar
  17. (Ferscha and Mattern, 2004).
    Ferscha, A., Mattern, F. (eds.): PERVASIVE 2004: Pervasive Computing, Second International Conference, Vienna, Austria, Lecture Notes in Computer Science, vol 3001 (Springer, Berlin Heidelberg New York 2004)Google Scholar
  18. (Gause and Weinberg, 1989).
    Gause, D.C., Weinberg, G.M.: Exploring Requirements: Quality before Design (Dorset House Publishing, New York 1989)Google Scholar
  19. (Gryna, 1998).
    Gryna, F.M.: Quality and Costs. In: Juran’s Quality Handbook, 5th edition, ed by Juran, J.M., Godfrey, A.B. (McGraw-Hill, New York 1998), pp 8.1–8.26Google Scholar
  20. (Harrold, 2000).
    Harrold, M.J.: Testing: A Roadmap. In: The Future of Software Engineering, ed by Finkelstein, A., 22th International Conference on Software Engineering, Limerick, Ireland, June 2000, pp 63–72Google Scholar
  21. (Hauston, 2002).
    Hauston, D.: Cost of Software Quality: Justifying Software Process Improvement to Managers. In: Daughtrey, T.: Fundamental Concepts for the Software Quality Engineer (ASQ Quality Press, Milwaukee 2001), pp 85–94Google Scholar
  22. (IEEE 610.12, 1990)
    IEEE Standard Glossary of Software Engineering Terminology, IEEE Std 610.12-1990. (IEEE Computer Society, 1990)Google Scholar
  23. (Kaner et al., 1999)
    Kaner, C., Falk, J., and Nguyen, H.Q.: Testing Computer Software, 2nd edition (Wiley, New York 1999)Google Scholar
  24. (Kaner at al., 2002)
    Kaner, C., Bach, J., Pettichord, B.: Lessons Learned in Software Testing: A Context-Driven Approach (Wiley, New York 2002)Google Scholar
  25. (Kruchten, 2003).
    Kruchten, P.: The Rational Unified Process: An Introduction, 3rd edition (Addison-Wesley, Boston 2003)Google Scholar
  26. (McConnell, 1996).
    McConnell, S.: Rapid Development: Taming Wild Software Schedules (Microsoft Press, Redmond 1996)zbMATHGoogle Scholar
  27. (Myers, 1979).
    Myers, G.J.: The Art of Software Testing (Wiley, New York 1979)Google Scholar
  28. (Pinkster et al., 2004)
    Pinkster, I., Burgt, B.v.d., Janssen, D., Veenendaal, E.v.: Successful Test Management: An Integral Approach (Springer, Berlin Heidelberg New York 2004)CrossRefGoogle Scholar
  29. (PSS-05, 1991).
    SA Board for Software Standardisation and Control (BSSC): ESA Software Engineering Standards PSS-05-0 (European Space Agency (ESA), ESTEC, Noordwijk, The Netherlands, February 1991)Google Scholar
  30. (Redmill, 2004).
    Redmill, F.: Exploring risk-based testing and its implications. Software Testing, Verification and Reliability. 14, pp 3–15 (2004)CrossRefGoogle Scholar
  31. (Reifer, 2002).
    Reifer D.: Making the Software Business Case: Improvement by the Numbers. (Addison Wesley, New York 2002)Google Scholar
  32. (Rosenberg, 2003).
    Rosenberg, L.H.: Lessons Learned in Software Quality Assurance. In: Managing Software Engineering Knowledge. ed by Aurum, A., Jeffery, R., Wohlin, C., Handzic, M. (Springer, Berlin Heidelberg New York 2003), pp 251–268CrossRefGoogle Scholar
  33. (Ruhe and Ngo-The, 2004).
    Ruhe, G., Ngo-The, A.: Hybrid Intelligence in Software Release Planning. International Journal of Hybrid Intelligent Systems. 1(2), pp 99–110 (2004)zbMATHGoogle Scholar
  34. (Slaughter et al., 1998)
    Slaughter, S.A., Harter, D.E., Krishnan, M.S.: Evaluating the Cost of Software Quality. CACM, 41(8), pp 67–73 (August 1998)CrossRefGoogle Scholar
  35. (Wallace and Fujii, 1989).
    Wallace, D.R., Fujii, R.U.: Software Verification and Validation: An Overview. IEEE Software, 6(3), pp 10–17 (May 1989)CrossRefGoogle Scholar
  36. (Weinberg, 2003).
    Weinberg, G.M.: Destroying Communication and Control in Software Development. CrossTalk, pp 4–8 (April 2003)Google Scholar
  37. (Wells, 2002).
    Wells, D.: An Introduction to Testing, XP-Style. In: Marchesi, M., Succi, G., Wells, D., Williams, L.: Extreme Programming Perspectives. (Addison Wesley, 2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Rudolf Ramler
    • 1
  • Stefan Biffl
    • 2
  • Paul Grünbacher
    • 3
  1. 1.Software Competence Center Hagenberg GmbHHagenbergAustria
  2. 2.Institute of Software Technology and Interactive SystemsTechnische Universität WienViennaAustria
  3. 3.Systems Engineering and AutomationJohannes Kepler University LinzLinzAustria

Personalised recommendations