The Adoption of Machine Learning Techniques for Software Defect Prediction: An Initial Industrial Validation
Existing methods for predicting reliability of software are static and need manual maintenance to adjust to the evolving data sets in software organizations. Machine learning has a potential to address the problem of manual maintenance but can also require changes in how companies works with defect prediction. In this paper we address the problem of identifying what the benefits of machine learning are compared to existing methods and which barriers exist for adopting them in practice.
Our methods consist of literature studies and a case study at two companies – Ericsson and Volvo Car Group. By studying literature we develop a framework for adopting machine learning and using case studies we evaluate this framework through a series of four interviews with experts working with predictions at both companies - line manager, quality manager and measurement team leader.
The findings of our research show that the most important perceived benefits of adopting machine learning algorithms for defect prediction are accuracy of predictions and ability to generate new insights from data. The two most important perceived barriers in this context are inability to recognize new patterns and low generalizability of the machine learning algorithms.
We conclude that in order to support companies in making an informed decision to adopt machine learning techniques for software defect predictions we need to go beyond accuracy and also evaluate factors such as costs, generalizability and competence.
KeywordsMachine Learning software defect prediction technology acceptance adoption software quality
Unable to display preview. Download preview PDF.
- 2.Lyu, M.R.: Handbook of software reliability engineering. IEEE Computer Society Press, CA (1996)Google Scholar
- 3.Nagappan, N., Ball, T., Zeller, A.: Mining metrics to predict component failures. In: Proceedings of the 28th International Conference on Software Engineering, pp. 452–461 (2006)Google Scholar
- 11.Rana, R., Staron, M., Nilsson, M.: A framework for adoption of machine learning in industry for software defect prediction. Presented at the Submitted to ICSOFT-EA, 2014 , Vienna, Austria (2014)Google Scholar
- 12.Menzies, T., Ammar, K., Nikora, A., DiStefano, J.: How simple is software defect detec-tion. Submitt. Emprical Softw. Eng. J. (2003)Google Scholar
- 13.Chau, P.Y., Tam, K.Y.: Factors Affecting the Adoption of Open Systems: An Explora-tory Study. Mis Q. 21 (1997)Google Scholar
- 15.Ajzen, I., Fishbein, M.: Understanding attitudes and predicting social behaviour (1980)Google Scholar
- 16.Davis Jr., F.D.: A technology acceptance model for empirically testing new end-user information systems: Theory and results (1986), http://www.researchgate.net/publication/35465050_A_technology_acceptance_model_for_empirically_testing_new_end-user_information_systems__theory_and_results_/file/9c960519fbaddf3ba7.pdf
- 18.Gefen, D., Straub, D.W.: Gender Differences in the Perception and Use of E-Mail: An Extension to the Technology Acceptance Model. MIS Q. 21 (1997)Google Scholar
- 20.Tornatzky, L.G., Fleischer, M., Chakrabarti, A.K.: Processes of technological innovation (1990)Google Scholar