Advertisement

The Adoption of Machine Learning Techniques for Software Defect Prediction: An Initial Industrial Validation

  • Rakesh Rana
  • Miroslaw Staron
  • Christian Berger
  • Jörgen Hansson
  • Martin Nilsson
  • Wilhelm Meding
Part of the Communications in Computer and Information Science book series (CCIS, volume 466)

Abstract

Existing methods for predicting reliability of software are static and need manual maintenance to adjust to the evolving data sets in software organizations. Machine learning has a potential to address the problem of manual maintenance but can also require changes in how companies works with defect prediction. In this paper we address the problem of identifying what the benefits of machine learning are compared to existing methods and which barriers exist for adopting them in practice.

Our methods consist of literature studies and a case study at two companies – Ericsson and Volvo Car Group. By studying literature we develop a framework for adopting machine learning and using case studies we evaluate this framework through a series of four interviews with experts working with predictions at both companies - line manager, quality manager and measurement team leader.

The findings of our research show that the most important perceived benefits of adopting machine learning algorithms for defect prediction are accuracy of predictions and ability to generate new insights from data. The two most important perceived barriers in this context are inability to recognize new patterns and low generalizability of the machine learning algorithms.

We conclude that in order to support companies in making an informed decision to adopt machine learning techniques for software defect predictions we need to go beyond accuracy and also evaluate factors such as costs, generalizability and competence.

Keywords

Machine Learning software defect prediction technology acceptance adoption software quality 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Fenton, N.E., Neil, M.: A critique of software defect prediction models. IEEE Trans. Softw. Eng. 25, 675–689 (1999)CrossRefGoogle Scholar
  2. 2.
    Lyu, M.R.: Handbook of software reliability engineering. IEEE Computer Society Press, CA (1996)Google Scholar
  3. 3.
    Nagappan, N., Ball, T., Zeller, A.: Mining metrics to predict component failures. In: Proceedings of the 28th International Conference on Software Engineering, pp. 452–461 (2006)Google Scholar
  4. 4.
    Staron, M., Meding, W., Söderqvist, B.: A method for forecasting defect backlog in large streamline software development projects and its industrial evaluation. Inf. Softw. Technol. 52, 1069–1079 (2010)CrossRefGoogle Scholar
  5. 5.
    Li, M., Smidts, C.S.: A ranking of software engineering measures based on expert opin-ion. IEEE Trans. Softw. Eng. 29, 811–824 (2003)CrossRefGoogle Scholar
  6. 6.
    Fenton, N., Neil, M., Marsh, W., Hearty, P., Radliński, Ł., Krause, P.: On the effective-ness of early life cycle defect prediction with Bayesian Nets. Empir. Softw. Eng. 13, 499–537 (2008)CrossRefGoogle Scholar
  7. 7.
    Zhang, D., Tsai, J.J.: Machine learning and software engineering. Softw. Qual. J. 11, 87–119 (2003)CrossRefGoogle Scholar
  8. 8.
    Fenton, N., Neil, M., Marsh, W., Hearty, P., Marquez, D., Krause, P., Mishra, R.: Predict-ing software defects in varying development lifecycles using Bayesian nets. Inf. Softw. Technol. 49, 32–43 (2007)CrossRefGoogle Scholar
  9. 9.
    Gondra, I.: Applying machine learning to software fault-proneness prediction. J. Syst. Softw. 81, 186–195 (2008)CrossRefGoogle Scholar
  10. 10.
    Menzies, T., Greenwald, J., Frank, A.: Data mining static code attributes to learn defect predictors. IEEE Trans. on Softw. Eng. 33, 2–13 (2007)CrossRefGoogle Scholar
  11. 11.
    Rana, R., Staron, M., Nilsson, M.: A framework for adoption of machine learning in industry for software defect prediction. Presented at the Submitted to ICSOFT-EA, 2014 , Vienna, Austria (2014)Google Scholar
  12. 12.
    Menzies, T., Ammar, K., Nikora, A., DiStefano, J.: How simple is software defect detec-tion. Submitt. Emprical Softw. Eng. J. (2003)Google Scholar
  13. 13.
    Chau, P.Y., Tam, K.Y.: Factors Affecting the Adoption of Open Systems: An Explora-tory Study. Mis Q. 21 (1997)Google Scholar
  14. 14.
    Wallace, L.G., Sheetz, S.D.: The adoption of software measures: A technology accep-tance model (TAM) perspective. Inf. Manage. 51, 249–259 (2014)CrossRefGoogle Scholar
  15. 15.
    Ajzen, I., Fishbein, M.: Understanding attitudes and predicting social behaviour (1980)Google Scholar
  16. 16.
    Davis Jr., F.D.: A technology acceptance model for empirically testing new end-user information systems: Theory and results (1986), http://www.researchgate.net/publication/35465050_A_technology_acceptance_model_for_empirically_testing_new_end-user_information_systems__theory_and_results_/file/9c960519fbaddf3ba7.pdf
  17. 17.
    Hardgrave, B.C., Johnson, R.A.: Toward an information systems development acceptance model: the case of object-oriented systems development. IEEE Trans. on Eng. Manag. 50, 322–336 (2003)CrossRefGoogle Scholar
  18. 18.
    Gefen, D., Straub, D.W.: Gender Differences in the Perception and Use of E-Mail: An Extension to the Technology Acceptance Model. MIS Q. 21 (1997)Google Scholar
  19. 19.
    Yi, M.Y., Hwang, Y.: Predicting the use of web-based information systems: self-efficacy, enjoyment, learning goal orientation, and the technology acceptance model. Int. J. Hum.-Comput. Stud. 59, 431–449 (2003)CrossRefGoogle Scholar
  20. 20.
    Tornatzky, L.G., Fleischer, M., Chakrabarti, A.K.: Processes of technological innovation (1990)Google Scholar
  21. 21.
    Runeson, P., Höst, M.: Guidelines for conducting and reporting case study research in software engineering. Empir. Softw. Eng. 14, 131–164 (2009)CrossRefGoogle Scholar
  22. 22.
    Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Experimentation in Software Engineering. Springer, New York (2012)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Rakesh Rana
    • 1
  • Miroslaw Staron
    • 1
  • Christian Berger
    • 1
  • Jörgen Hansson
    • 1
  • Martin Nilsson
    • 2
  • Wilhelm Meding
    • 3
  1. 1.Computer Science & Engineering, ChalmersUniversity of GothenburgSweden
  2. 2.Volvo Car GroupGothenburgSweden
  3. 3.EricssonGothenburgSweden

Personalised recommendations