A Quantitative Analysis into the Economics of Correcting Software Bugs

Abstract

Using a quantitative study of in-house coding practices, we demonstrate the notion that programming needs to move from “Lines of Code per day” as a productivity measure to a measure that takes debugging and documentation into account. This could be something such as “Lines of clean, simple, correct, well-documented code per day”, but with bugs propagating into the 6th iteration of patches, a new paradigm needs to be developed. Finding flaws in software, whether these have a security related cost or not, is an essential component of software development. When these bugs result in security vulnerabilities, the importance of testing becomes even more critical. Many studies have been conducted using the practices of large software vendors as a basis, but few studies have looked at in-house development practices. This paper uses an empirical study of in-house software coding practices in Australian companies to both demonstrate that there is an economic limit to how far testing should proceed as well as noting the deficiencies in the existing approaches.