Skip to main content

SMPLearner: learning to predict software maintainability

Abstract

Accurate and practical software maintainability prediction enables organizations to effectively manage their maintenance resources and guide maintenance-related decision making. This paper presents SMPLearner, an automated learning-based approach to train maintainability predictors by harvesting the actual average maintenance effort computed from the code change history as well as employing a much richer set of 44 four-level hierarchical code metrics collected by static code analysis tools. We systematically evaluated SMPLearner on 150 observations partitioned from releases of eight large scale open source software systems. Our evaluation showed that SMPLearner not only outperformed the traditional 4-metric MI model but also the recent learning-based maintainability predictors constructed based on single Class-level metrics, demonstrating that single Class-level metrics were not sufficient for maintainability prediction.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Notes

  1. 1.

    Understand is a static analysis tool for maintaining, measuring, and analyzing critical or large code bases. It calculates the four MI metrics objectively based on the code structure. It can be accessible at http://www.scitools.com/index.php.

  2. 2.

    VirtualBox’s version control system is available at https://www.virtualbox.org/browser/vbox/trunk/src/VBox.

  3. 3.

    A description of Logiscope is available at https://www-304.ibm.com/support/docview.wss?uid=swg24021021.

  4. 4.

    See Sect. 4.2 for details on how the conversion is done and why it is needed.

  5. 5.

    Spearman’s rank correlation coefficient rather than Pearson’s method is used to compute all correlation results in this paper due to its robustness and lack of assumptions about normality.

  6. 6.

    http://www.statstutor.ac.uk/resources/uploaded/spearmans

References

  1. Al-Kilidar, H., Cox, K., Kitchenham, B.: The use and usefulness of the ISO/IEC 9126 quality standard, In IEEE International Symposium on Empirical Software Engineering, pp. 7–17 (2005)

  2. Antonellis, P., Antoniou, D., Kanellopoulos, Y., Makris, C., Theodoridis, E., Tjortjis, C., Tsirakis, N.: A data mining methodology for evaluating maintainability according to ISO/IEC-9126 software engineering Cproduct quality standard. In Special Session on System Quality and Maintainability (SQM) (2007)

  3. Baggen, R., Correia, J.P., Schill, K., Visser, J.: Standardized code quality benchmarking for improvingsoftware maintainability. Software Quality Journal 20(2), 287–307 (2012)

    Article  Google Scholar 

  4. Bhattacharya, P., Iliofotou, M., Neamtiu, I., Faloutsos, M.: Graph-based analysis and prediction for software evolution. ICSE, pp. 419–429 (2012)

  5. Bhattacharya, P., Neamtiu, I.: Assessing programming language impact on development and maintenance: A study on C and C++. ICSE 2011, pp. 171–180 (2011)

  6. Burkett, D. Klein, D.: Two Languages are Better than one (for Syntactic Parsing). In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pp. 877–886 (2008)

  7. Chidamber, S.R., Kemerer, C.F.: A metrics suite for object oriented design. IEEE Transactions on Software Engineering 20(6), 476–493 (1994)

    Article  Google Scholar 

  8. Coleman, D., Ash, D., Lowther, B., Oman, P.: Using Metrics to Evaluate Software System Maintainability. IEEE Computer, pp. 44–49 (1994)

  9. Foss, T., Stensrud, E., Kitchenham, B., Myrtveit, I.: A simulation study of the model evaluation criterion MMRE. IEEE Trans. Softw. Eng. 29(11), 985–995 (2003)

    Article  Google Scholar 

  10. Glass, R.L.: Facts and Fallacies of Software Engineering. Addison Wesley Professional, Boston (2002)

    Google Scholar 

  11. Halstead, M.H.: Elements of Software Science (Operating and Programming Systems Series). Elsevier, New York (1977)

    Google Scholar 

  12. Heitlager, I., Kuipers, T., Visser, J.: A practical model for measuring maintainability. In International Conference on Quality of Information and Communications Technology, pp. 30–39 (2007)

  13. IEEE Std. 610.12-1990: Standard Glossary of Software Engineering Terminology. IEEE Computer Society Press, Los Alamitos (1993)

  14. Kaur, A., Kaur, K.: Statistical comparison of modelling methods for software maintainability prediction. Int. J. Softw. Eng. Knowl. Eng. 23(6), 743–774 (2013)

    Article  MathSciNet  Google Scholar 

  15. Kitchenham, B.A., Pickard, L.M., MacDonell, S.G., Shepperd, M.J.: What accuracy statistics really measure. IEE. Prcc Softw 148(3), 81–85 (2001)

    Article  Google Scholar 

  16. Koten, C.V., Gray, A.R.: An application of bayesian network for predicting object-oriented software maintainability. Inform. Softw. Technol. 48(1), 59–67 (2006)

    Article  Google Scholar 

  17. Li, W., Henry, S.: Object-oriented metrics that predict maintainability. J. Syst. Softw. 23(2), 111–122 (1993)

    Article  Google Scholar 

  18. Liso, A.: Software maintainability metrics model: an improvement in the Coleman-Oman model. Crosstalk, 15–17 (2001)

  19. Malhotra, R., Chug, A.: Software maintainability prediction using machine learning algorithms. Softw. Eng. Int. J. 2(2), 19–36 (2012)

    Google Scholar 

  20. McCabe, T.: A complexity measure. IEEE Trans. Softw. Eng. 2(4), 308–320 (1976)

    Article  MATH  MathSciNet  Google Scholar 

  21. Mittal, H., Bhatia, P.: Software maintainability assessment based on fuzzy logic technique. ACM SIGSOFT Softw. Eng. Notes 34(3), 1–5 (2009)

    Article  Google Scholar 

  22. Muthanna, S., Kontogiannis, K., Ponnambalam, K., Stacey, B.: A maintainability model for industrial software systems using design level metrics. In Proceedings of 7th Working Conference on Reverse Engineering, pp. 248–256 (2000)

  23. Oman, P. W., Hagemeister, J.: Metrics for assessing a software system’s maintainability. In Proceedings of the Conference on Software Maintenance, IEEE Computer Society Press, Los Alamitos, CA, pp. 337–344 (1992)

  24. Ramil, J.F., Cortazar, D.I., Mens, T.: What does it take to develop a million lines of open source code? OSS 299, 170–184 (2009)

    Google Scholar 

  25. Riaz, M., Mendes, E., Tempero, E.: A systematic review of software maintainability prediction and metrics. In Proceedings of the Third International Symposium on Empirical Software Engineering and Measurement, pp. 367–377 (2009)

  26. Shepperd, M., Cartwright, M., Kadoda, G.: On building prediction systems for software engineers. Empir. Softw. Eng. 5, 175–182 (2000)

    Article  MATH  Google Scholar 

  27. Singh, Y., Bhatia, P.K., Sangwan, O.: Predicting software maintenance using fuzzy model. ACM SIGSOFT Softw. Eng. Notes 34(4), 1–6 (2009)

    Article  Google Scholar 

  28. Welker, K.D.: The software maintainability index revisited, crosstalk. J. Def. Softw. Eng. 14, 18–21 (2001)

    Google Scholar 

  29. Welker, K.D., Oman, P.W., Atkinson, G.G.: Development and application of an automated source code maintainability index. J. Softw. Maint. Evol. R 9, 127–159 (1997)

    Article  Google Scholar 

  30. Yu, L.: Indirectly predicting the maintenance effort of open-source software. J. Softw. Maint. Evol. Res. Pract. 18(5), 311–332 (2006)

    Article  Google Scholar 

  31. Zhou, Y., Leung, H.: Predicting object-oriented software maintainability using multivariate adaptive regression splines. J. Syst. Softw. 80, 1349–1361 (2007)

    Article  Google Scholar 

  32. Zhou, Y., Xu, B.: Predicting the maintainability of open source software using design metrics. Wuhan Univ. J. Nat. Sci. 13(1), 14–21 (2008)

    Article  Google Scholar 

Download references

Acknowledgments

We thank the anonymous reviewers for their helpful comments on an earlier draft of this manuscript. This research is supported by the U.S. National Science Foundation (NSF CNS Award # 1126747). It is partially supported by the Oversea Fund of State Key Laboratory for Novel Software Technology of Nanjing University, National Natural Science Foundation, China (No. 61100039, 61021062, 61272188, 91318301), and Natural Science Foundation of Jiangsu Province, China (No. BK20131277).

Author information

Affiliations

Authors

Corresponding author

Correspondence to LiGuo Huang.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhang, W., Huang, L., Ng, V. et al. SMPLearner: learning to predict software maintainability. Autom Softw Eng 22, 111–141 (2015). https://doi.org/10.1007/s10515-014-0161-3

Download citation

Keywords

  • Software Maintainability
  • Maintenance effort
  • Software metric
  • Machine learning