Predicting Bugs in Large Industrial Software Systems
This chapter is a survey of close to ten years of software fault prediction research performed by our group. We describe our initial motivation, the variables used to make predictions, provide a description of our standard model based on Negative Binomial Regression, and summarize the results of using this model to make predictions for nine large industrial software systems. The systems range in size from hundreds of thousands to millions of lines of code. All have been in the field for multiple years and many releases, and continue to be maintained and enhanced, usually at 3 month intervals.
Effectiveness of the fault predictions is assessed using two different metrics. We compare the effectiveness of the standard model to augmented models that include variables related to developer counts, to inter-file calling structure, and to information about specific developers who modified the code.
We also evaluate alternate prediction models based on different training algorithms, including Recursive Partitioning, Bayesian Additive Regression Trees, and Random Forests.
Keywordssoftware fault prediction defect prediction negative binomial model fault-percentile average buggy file ratio calling structure prediction tool
Unable to display preview. Download preview PDF.
- 3.Chipman, H.A., George, E.I., McCulloch, R.E.: BART: Bayesian Additive Regression Trees (2008), http://arxiv.org/abs/0806.3286v1
- 5.Ostrand, T.J., Weyuker, E.J.: The Distribution of Faults in a Large Industrial Software System. In: International Symposium on Software Testing and Analysis (ISSTA 2002), pp. 55–64. ACM Press, New York (2002)Google Scholar
- 7.Ostrand, T.J., Weyuker, E.J., Bell, R.M.: Programmer-based Fault Prediction. In: Predictive Models for Software Engineering (PROMISE 2010). ACM Press, New York (2010)Google Scholar
- 8.Shin, Y., Bell, R.M., Ostrand, T.J., Weyuker, E.J.: On the use of calling structure information to improve fault prediction. Empirical Software Eng. (July 2011), http://www.springerlink.com/content/r4q76v4317148451/
- 9.Weyuker, E.J., Ostrand, T.J., Bell, R.M.: We’re Finding Most of the Bugs, but What are we Missing? In: 3rd International Conference on Software Testing. IEEE Press, New York (2010)Google Scholar