Defect detection activities are generally seen as expensive and time consuming. This article reports on actual experiences from large projects. Based on data, dependencies between the costs of early defect detection, late defect detection and defect fixing are analysed and some guidelines are derived for timing and resource allocation to ensure cost-effective defect detection. Experience reports show that the costs of defect fixing in software development grow dramatically when the defects are found late, i.e. when the defects have already influenced the next phase. This common experience is supported by all the projects, two of which are considered in this paper. However, metrics applied to the data show that it is not always an order of magnitude that lies between the costs of early and late fixing of defects. Rather, the average cost of defects found late decreases with introduction of early defect detection. This is due to the fact that early document reviews leave only minor defects, e.g. in the requirements or the design. Major and therefore expensive defects tend to be found in those early reviews. Further analysis of the data led to the use of metrics that proved very useful when planning and executing the required defect detection activities. For example the number of remarks per 100 lines was used to decide how efficient reviews of the design have been and whether more or less resources should be allocated to these reviews in a running project. As a conclusion, it is emphasized that a substantial amount of a project's budget and time will be saved by introducing systematic defect detection early in the project. It is suggested that defect detection activities should start as soon as the requirements documents are declared finished.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price excludes VAT (USA)
Tax calculation will be finalised during checkout.
BeizerB. (1990) Software Testing Techniques, 2nd edn. (Van Nostrand Rheinhold, New York).
BoehmB.W. (1981) Software Engineering Economics (Prentice Hall, Englewood Cliffs, NJ).
Bons, H. (1994) SQS-TEST as the basis of a test system in a large scale migration, in Proceedings of EuroSTAR '94 (Software Quality Engineering, Jacksonville, USA).
Compuware (1995) Automatisiertes Testen, Hiperstation (Compuware, Düsseldorf).
FaganM.E. (1976) Design and code inspections to reduce errors in program development. IBM Systems Journal, 3, 182–211.
IBM (1981) Implementing Software Inspections, course notes (IBM Systems Sciences Institute, IBM Corporation) (summarised in Pressman 1992.)
ISO 9000-3, Quality management and quality assurance standards, Part 3: Guidelines for the application of ISO 9001 to the development, supply and maintenance of software. International Standards Organization.
van Megen, R. (1994) Testing strategies in software development and maintenance — let's learn from the past, in Proceedings of EuroSTAR '94 (Software Quality Engineering, Jacksonville, USA).
Müllerburg, M., Holenderski, L., Maffeis, O., Merceron, A. and Morley, M. (1995) Systematic testing and verification to validate reactive systems, in this issue.
PressmanR.S. (1992) Software Engineering, A Practitioner's Approach (McGraw-Hill, New York).
ProTerm (1995) Proterm System Guide, Release 2.3 (Reed Software GmbH, Oberhausen).
SOS (1993) SQS-TEST, Das Testsystem zum systematischen Aufbau von regressionsfähigen Tests, Vers. 2.0 (SQS Gesellschaft für Software-Qualitätssicherung mbH, Cologne).
TPNS (1990) TPNS general information (IBM).
YourdonE., ConstantineL.L. (1979) Structured Walkthroughs (Prentice Hall, Englewood Cliffs, NJ).
About this article
Cite this article
van Megen, R., Meyerhoff, D.B. Costs and benefits of early defect detection: experiences from developing client server and host applications. Software Qual J 4, 247–256 (1995). https://doi.org/10.1007/BF00402646