Skip to main content
SpringerLink
Log in

Performance of Maintainability Index prediction models: a feature selection based study

  • Original Paper
  • Published:
Evolving Systems Aims and scope Submit manuscript

Abstract

Numerous design metrics have been studied in the literature to assess software performance and future maintainability issues. Maintainability Index (MI) is an indicator of software maintenance efforts, but that can be computed only after the complete code is developed. MI is complex to calculate and hence has been estimated by several researchers using design metrics. Various prediction models are built using different sets of design metrics as independent variables. Further, researchers have used different prediction models for predicting software maintainability. To identify most prominent design metrics and the prediction models that show consistent performance irrespective of the choice of design metrics, the present empirical study is performed on datasets of 26 open source Java projects with more than one thousand and six hundred class hierarchies. Seven feature selection methods are applied to find out the most prominent metrics that are selected by majority of methods. Then four most frequently used prediction models namely, multi linear regression, multilayer perceptron, support vector regression and M5P regression tree are analyzed for their performance with respect to initial selection of design metrics. The study concludes that MIF, MaxDIT, LCC, TCC are the most significant predictors for Maintainability Index of class hierarchies. Among the all the prediction models, support vector regression model exhibits the best performance both with respect to the choice of metrics and irrespective of initial metric selection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Agarwal S, Tomar D (2014) A feature selection based model for software defect prediction. Int J Adv Sci Technol 65:39–58

    Article  Google Scholar 

  • Aggarwal K, Singh Y, Kaur A, Malhotra R (2006) Empirical study of object-oriented metrics. J Object Technol 5:149–173

    Article  Google Scholar 

  • Ahmed MA, Al-Jamimi HA (2013) Machine learning approaches for predicting software maintainability: a fuzzy-based transparent model. IET Softw 7:317–326

    Article  Google Scholar 

  • Al Dallal J (2011) Measuring the discriminative power of object-oriented class cohesion metrics. IEEE Trans Softw Eng 37:788–804

    Article  Google Scholar 

  • Al Dallal J (2013) Object-oriented class maintainability prediction using internal quality attributes. Inf Softw Technol 55:2028–2048

    Article  Google Scholar 

  • Almugrin S, Albattah W, Melton A (2016) Using indirect coupling metrics to predict package maintainability and testability. J Syst Softw 121:298–310

    Article  Google Scholar 

  • Alshayeb M (2013) On the relationship of class stability and maintainability. IET Softw 7:339–347

    Article  Google Scholar 

  • Anda B (2007) Assessing software system maintainability using structural measures and expert assessments. In: 2007 IEEE International Conference on Software Maintenance. IEEE, pp 204–213

  • Banker RD, Datar SM, Kemerer CF, Zweig D (1993) Software complexity and maintenance costs. Commun ACM 36:81–95

    Article  Google Scholar 

  • Basili VR, Briand LC, Melo WL (1996) A validation of object-oriented design metrics as quality indicators. IEEE Trans Softw Eng 22:751–761

    Article  Google Scholar 

  • Briand LC, Wüst J, Daly JW, Porter DV (2000) Exploring the relationships between design measures and software quality in object-oriented systems. J Syst Softw 51:245–273

    Article  Google Scholar 

  • Catal C, Diri B (2009) Investigating the effect of dataset size, metrics sets, and feature selection techniques on software fault prediction problem. Inf Sci 179:1040–1058

    Article  Google Scholar 

  • Chen J-C, Huang S-J (2009) An empirical analysis of the impact of software development problem factors on software maintainability. J Syst Softw 82:981–992

    Article  Google Scholar 

  • Chidamber SR, Kemerer CF (1994) A metrics suite for object oriented design. IEEE Trans Softw Eng 20:476–493

    Article  Google Scholar 

  • Coleman D, Ash D, Lowther B, Oman P (1994) Using metrics to evaluate software system maintainability. Computer 27:44–49

    Article  Google Scholar 

  • Daly J, Brooks A, Miller J, Roper M, Wood M (1996) Evaluating inheritance depth on the maintainability of object-oriented software. Empir Softw Eng 1:109–132

    Article  Google Scholar 

  • e Abreu FB, Carapuça R (1994) Candidate metrics for object-oriented software within a taxonomy framework. J Syst Softw 26:87–96

    Article  Google Scholar 

  • El Emam K, Melo W, Machado JC (2001) The prediction of faulty classes using object-oriented design metrics. J Syst Softw 56(1):63–75

    Article  Google Scholar 

  • Elish MO (2014) A comparative study of fault density prediction in aspect-oriented systems using MLP, RBF, KNN, RT, DENFIS and SVR models. Artif Intell Rev 42:695–703

    Article  Google Scholar 

  • Elish MO, Aljamaan H, Ahmad I (2015) Three empirical studies on predicting software maintainability using ensemble methods. Soft Comput 19:2511–2524

    Article  Google Scholar 

  • Fenton N, Bieman J (2014) Software metrics: a rigorous and practical approach. CRC Press, Boca Raton

  • Freitag D (2014) Greedy attribute selection. In: Machine learning proceedings 1994: proceedings of the eighth international conference. Morgan Kaufmann, Burlington. p 28

  • Glover F (1989) Tabu search—part I. ORSA J Comput 1:190–206

    Article  MATH  Google Scholar 

  • Gunn SR (1998) Support vector machines for classification and regression. ISIS technical report 14

  • Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182

    MATH  Google Scholar 

  • Gyimothy T, Ferenc R, Siket I (2005) Empirical validation of object-oriented metrics on open source software for fault prediction. IEEE Trans Softw Eng 31:897–910

    Article  Google Scholar 

  • Hanley JA, McNeil BJ (1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143:29–36

    Article  Google Scholar 

  • Harrison R, Counsell S, Nithi R (2000) Experimental assessment of the effect of inheritance on the maintainability of object-oriented systems. J Syst Softw 52:173–179

    Article  Google Scholar 

  • Heiat A (2002) Comparison of artificial neural network and regression models for estimating software development effort. Inf Softw Technol 44:911–922

    Article  Google Scholar 

  • Heitlager I, Kuipers T, Visser J (2007) A practical model for measuring maintainability. In: 6th international conference on the quality of information and communications technology, 2007. QUATIC 2007. IEEE, pp 30–39

  • Hernández-Orallo J (2013) ROC curves for regression. Pattern Recognit 46:3395–3411

    Article  MATH  Google Scholar 

  • Hoque N, Bhattacharyya D, Kalita JK (2014) MIFS-ND: a mutual information-based feature selection method. Expert Syst Appl 41:6371–6385

    Article  Google Scholar 

  • ISO IOFS (2011) ISO/IEC 25010:2011. http://www.iso.org/iso/catalogue_detail.htm?csnumber=35733. 2016

  • Jabangwe R, Börstler J, Šmite D, Wohlin C (2015) Empirical evidence on the link between object-oriented measures and external quality attributes: a systematic literature review. Empir Softw Eng 20:640–693

    Article  Google Scholar 

  • Jorgensen M, Shepperd M (2007) A systematic review of software development cost estimation studies. IEEE Trans Softw Eng 33:33–53

    Article  Google Scholar 

  • Jović A, Brkić K, Bogunović N (2015) A review of feature selection methods with applications. In: 38th international convention on information and communication technology, electronics and microelectronics (MIPRO), 2015. IEEE, pp 1200–1205

  • Kiewkanya M, Jindasawat N, Muenchaisri P (2004) A methodology for constructing maintainability model of object-oriented design. In: Fourth international conference on quality software, 2004. QSIC 2004. Proceedings. IEEE, pp 206–213

  • Kim Y, Street WN, Menczer F (2000) Feature selection in unsupervised learning via evolutionary search. In: Proceedings of the sixth ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 365–369

  • Kiran NR, Ravi V (2008) Software reliability prediction by soft computing techniques. J Syst Softw 81(4):576–583

    Article  Google Scholar 

  • Kitchenham BA, Pickard LM, MacDonell SG, Shepperd MJ (2001) What accuracy statistics really measure [software estimation]. IEE Proc Softw 148:81–85

    Article  Google Scholar 

  • Krogh A, Vedelsby J (1995) Neural network ensembles, cross validation, and active learning. Adv Neural Inf Process Syst 7:231–238

    Google Scholar 

  • Kumar L, Krishna A, Rath SK (2017) The impact of feature selection on maintainability prediction of service-oriented applications. Serv Oriented Comput Appl 11:137–161

    Article  Google Scholar 

  • Laradji IH, Alshayeb M, Ghouti L (2015) Software defect prediction using ensemble learning on selected features. Inf Softw Technol 58:388–402

    Article  Google Scholar 

  • Li W, Henry S (1993) Object-oriented metrics that predict maintainability. J Syst Softw 23:111–122

    Article  Google Scholar 

  • Lu Y, Mao X, Li Z (2016) Assessing software maintainability based on class diagram design: a preliminary case study. Lect Notes Softw Eng 4:53

    Article  Google Scholar 

  • Mishra D (2012) New inheritance complexity metrics for object-oriented software systems: an evaluation with Weyuker’s properties. Comput Inf 30:267–293

    MathSciNet  Google Scholar 

  • Muthanna S, Kontogiannis K, Ponnambalam K, Stacey B (2000) A maintainability model for industrial software systems using design level metrics. In: Seventh working conference on reverse engineering, 2000. Proceedings. IEEE, pp 248–256

  • Naboulsi Z (2011) Code metrics—Maintainability Index. Microsoft. https://blogs.msdn.microsoft.com/zainnab/2011/05/26/code-metrics-maintainability-index/

  • Oh I-S, Lee J-S, Moon B-R (2004) Hybrid genetic algorithms for feature selection. IEEE Trans Pattern Anal Mach Intell 26:1424–1437

    Article  Google Scholar 

  • Ping L (2010) A quantitative approach to software maintainability prediction. In: International forum on information technology and applications (IFITA), 2010. IEEE, pp 105–108

  • Quinlan JR (1992) Learning with continuous classes. In: 5th Australian joint conference on artificial intelligence. Singapore, pp 343–348

  • Reddy BR, Khurana S, Ojha A (2015) Software maintainability estimation made easy: a comprehensive tool COIN. In: Proceedings of the sixth international conference on computer and communication technology 2015. ACM, pp 68–72

  • Riaz M, Mendes E, Tempero E (2009) A systematic review of software maintainability prediction and metrics. In: Proceedings of the 2009 3rd international symposium on empirical software engineering and measurement. IEEE Computer Society, pp 367–377

  • Shi Y, Eberhart R (1998) A modified particle swarm optimizer. In: Evolutionary computation proceedings, 1998. The 1998 IEEE international conference on IEEE world congress on computational intelligence. IEEE, pp 69–73

  • Sjøberg DI, Anda B, Mockus A (2012) Questioning software maintenance metrics: a comparative case study. In: Proceedings of the ACM-IEEE international symposium on empirical software engineering and measurement. ACM, pp 107–110

  • Smola A, Vapnik V (1997) Support vector regression machines. Adv Neural Inf Process Syst 9:155–161

    Google Scholar 

  • Thwin MMT, Quah T-S (2005) Application of neural networks for software quality prediction using object-oriented metrics. J Syst Softw 76:147–156

    Article  Google Scholar 

  • Van Koten C, Gray A (2006) An application of Bayesian network for predicting object-oriented software maintainability. Inf Softw Technol 48:59–67

    Article  Google Scholar 

  • Wang H, Khoshgoftaar TM, Napolitano A (2010) A comparative study of ensemble feature selection techniques for software defect prediction. In: 2010 ninth international conference on machine learning and applications (ICMLA). IEEE, pp 135–140

  • Welker KD (2001) The software Maintainability Index revisited. CrossTalk 14:18–21

    Google Scholar 

  • Zhang W, Huang L, Ng V, Ge J (2015) SMPLearner: learning to predict software maintainability. Autom Softw Eng 22:111–141

    Article  Google Scholar 

  • Zhou Y, Leung H (2007) Predicting object-oriented software maintainability using multivariate adaptive regression splines. J Syst Softw 80:1349–1361

    Article  Google Scholar 

  • Zhou Y, Xu B (2008) Predicting the maintainability of open source software using design metrics. Wuhan Univ J Nat Sci 13:14–20

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. PDPM IIITDMJ, Dumna Airport Road, Jabalpur, India

    B. Ramachandra Reddy & Aparajita Ojha

Authors
  1. B. Ramachandra Reddy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aparajita Ojha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to B. Ramachandra Reddy.

Appendix A

Appendix A

See Tables 6, 7, 8, 9 and 10.

Table 6 Feature selection methods and the resultant metric subsets
Full size table
Table 7 Prediction results for MI using MLR and seven feature selection algorithms
Full size table
Table 8 Performance of predicting MI using MLP and seven feature selection algorithms
Full size table
Table 9 Performance of predicting MI using SVR and seven feature selection algorithms
Full size table
Table 10 Performce of predicting MI using M5P and seven feature selection algorithms
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Reddy, B.R., Ojha, A. Performance of Maintainability Index prediction models: a feature selection based study. Evolving Systems 10, 179–204 (2019). https://doi.org/10.1007/s12530-017-9201-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12530-017-9201-0

Keywords

Search

Navigation