Skip to main content
SpringerLink
Log in

The ability of object-oriented metrics to predict change-proneness: a meta-analysis

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Many studies have investigated the relationships between object-oriented (OO) metrics and change-proneness and conclude that OO metrics are able to predict the extent of change of a class across the versions of a system. However, there is a need to re-examine this subject for two reasons. First, most studies only analyze a small number of OO metrics and, therefore, it is not clear whether this conclusion is applicable to most, if not all, OO metrics. Second, most studies only uses relatively few systems to investigate the relationships between OO metrics and change-proneness and, therefore, it is not clear whether this conclusion can be generalized to other systems. In this paper, based on 102 Java systems, we employ statistical meta-analysis techniques to investigate the ability of 62 OO metrics to predict change-proneness. In our context, a class which is changed in the next version of a system is called change-prone and not change-prone otherwise. The investigated OO metrics cover four metric dimensions, including 7 size metrics, 18 cohesion metrics, 20 coupling metrics, and 17 inheritance metrics. We use AUC (the area under a relative operating characteristic, ROC) to evaluate the predictive effectiveness of OO metrics. For each OO metric, we first compute AUCs and the corresponding variances for individual systems. Then, we employ a random-effect model to compute the average AUC over all systems. Finally, we perform a sensitivity analysis to investigate whether the AUC result from the random-effect model is robust to the data selection bias in this study. Our results from random-effect models reveal that: (1) size metrics exhibit moderate or almost moderate ability in discriminating between change-prone and not change-prone classes; (2) coupling and cohesion metrics generally have a lower predictive ability compared to size metrics; and (3) inheritance metrics have a poor ability to discriminate between change-prone and not change-prone classes. Our results from sensitivity analyses show that these conclusions reached are not substantially influenced by the data selection bias.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. The raw data are available at http://thedata.org.

  2. Assuming a reasonably-sized data set, Hopkins calls a correlation value of less than 0.1 trivial, 0.1–0.3 minor, 0.3–0.5 moderate, 0.5–0.7 large, 0.7–0.9 very large, and 0.9–1 almost perfect (Hopkins 2003).

References

  • acjf_delta.pl. http://www.scitools.com/plugins/perl_scripts.php

  • Aman H, Yamasaki K, Yamada H, Noda MT (2002) A proposal of class cohesion metrics using sizes of cohesive parts. Knowledge-based Software Engineering. In: T. Welzer et al. (eds.) IOS Press, 102–107

  • Arisholm E, Briand LC, Føyen A (2004) Dynamic coupling measurement for object-oriented software. IEEE Trans Softw Eng 30(8):491–506

    Article  Google Scholar 

  • Badri L, Badri M (2004) A proposal of a new class cohesion criterion: an empirical study. J Object Tech 3(4):145–159

    Article  Google Scholar 

  • Bansiya J, Etzkorn L, Davis C, Li W (1999) A class cohesion metric for object-oriented designs. J Object-Oriented Program 11(8):47–52

    Google Scholar 

  • Benlarbi S, Melo WL (1999) Polymorphism measures for early risk prediction. In: Proceedings of the 21st International Conference on Software Engineering, Los Angeles, California, United States, 334–344

  • Bieman JM, Kang BK (1995) Cohesion and reuse in an object-oriented system. ACM SIGSOFT Software Eng Notes 20(Special Issue):259–262

    Article  Google Scholar 

  • Bieman JM, Jain D, Yang HJ (2001) OO design patterns, design structure, and program changes: an industrial case study. In: Proceedings of the 17th International Conference on Software Maintenance, 580–589

  • Bieman JM, Straw G, Wang H, Munger PW, Alexander RT (2003a) Design patterns and change proneness: an examination of five evolving systems. In: Proceedings of the 9th Software Metrics Symposium, 40–49

  • Bieman JM, Andrews AA, Yang HJ (2003b) Understanding change-proneness in OO software through visualization. In: Proceedings of the 11th Workshop on Program Comprehension, 44–53

  • Borenstein M, Hedges LV, Higgins JPT, Rothstein HR (2009) Introduction to Meta-analysis. John Wiley & Sons, Ltd

  • Briand LC, Wüst J (2001) Modeling development effort in object-oriented systems using design properties. IEEE Trans Softw Eng 27(11):963–986

    Article  Google Scholar 

  • Briand LC, Morasca S, Basili VR (1996) Property-based software engineering measurement. IEEE Trans Softw Eng 22(1):68–86

    Article  Google Scholar 

  • Briand LC, Devanbu PT, Melo WL (1997) An investigation into coupling measures for C++. In: Proceedings of ICSE, 412–421

  • Briand LC, Daly JW, Wüst J (1998) A unified framework for cohesion measurement in object-oriented systems. Empir Softw Eng 3(1):65–117

    Article  Google Scholar 

  • Briand LC, Daly JW, Wüst J (1999) A unified framework for coupling measurement in object-oriented systems. IEEE Trans Softw Eng 25(1):91–121

    Article  Google Scholar 

  • Briand LC, Wüst J, Daly JW, Porter DV (2000) Exploring the relationships between design measures and software quality in object-oriented systems. J Syst Softw 51(3):245–273

    Article  Google Scholar 

  • Chidamber SR, Kemerer CF (1991) Towards a metrics suite for object-oriented design. In: the 6th Annual Conference of Object-oriented Programming, Systems, Languages, and Applications, Arizona, 197–211

  • Chidamber SR, Kemerer CF (1994) A metrics suite for object-oriented design. IEEE Trans Softw Eng 20(6):476–493

    Article  Google Scholar 

  • Counsell S, Swift S, Crampton J (2006) The interpretation and utility of three cohesion metrics for object-oriented design. ACM Trans Softw Eng Methodol 15(2):123–149

    Article  Google Scholar 

  • Di Penta M, Cerulo L, Gueheneuc YG, Antoniol G (2008) An empirical study of the relationships between design pattern roles and class change-proneness. In Proceedings of the 24th International Conference on Software Maintenance, 217–226

  • Dickinson W, Leon D, Podgurski A (2001) Finding failures by cluster analysis of execution profiles. In: Proceedings of the 23rd International Conference on Software Engineering, 339–348

  • Duval S, Tweedie R (2000) Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 56(2):455–463

    Article  MATH  Google Scholar 

  • El Emam K, Benlarbi S, Goel N, Rai SN (2001) The confounding effect of class size on the validity of object-oriented metrics. IEEE Trans Softw Eng 27(7):630–650

    Article  Google Scholar 

  • Hanley JA, McNeil BJ (1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143(1):29–36

    Google Scholar 

  • Hannay JE, Dybå T, Arisholm E, Sjøberg DIK (2009) The effectiveness of pair-programming: a meta-analysis. Inf Softw Technol 51(7):1110–1122

    Article  Google Scholar 

  • Hayes W (1999) Research synthesis in software engineering: A case for meta-analysis. In: Proceedings of the 6th IEEE International Software Metrics Symposium, 143–151

  • Henderson-Sellers B (1996) Software metrics. Prentice-Hall, Hemel Hempstaed

    Google Scholar 

  • Higgins J, Thompson S, Deeks J, Altman D (2003) Measuring inconsistency in meta-analyses. Br Med J 327:557–560

    Article  Google Scholar 

  • Hitz M, Montazeri B (1995) Measuring coupling and cohesion in object-oriented systems. In: Proceedings of the International Symposium on Applied Corporate Computing, Monterrey, Mexico

  • Hopkins WG (2003) A new view of statistics. SportScience. Dunedin, New Zealand

    Google Scholar 

  • Julious SA (2004) Using confidence intervals around individual means to assess statistical significance between two means. Pharm Stat 3(3):217–222

    Article  Google Scholar 

  • Kim E, Kusumoto S, Kikuno T (1996) Heuristics for computing attribute values of C++ program complexity metrics. In: Proceedings of the 20th Conference on Computer Software and Applications, 104–109

  • Koru AG, Liu H (2007) Identifying and characterizing change-prone classes in two large-scale open-source products. J Syst Softw 80(1):63–73

    Article  Google Scholar 

  • Koru AG, Tian J (2005) Comparing high-change modules and modules with the highest measurement values in two large-scale open-source products. IEEE Trans Softw Eng 31(8):625–642

    Article  Google Scholar 

  • Lake A, Cook C (1994) Use of factor analysis to develop OOP software complexity metrics. In: Proceedings of the 6th Annual Oregon Workshop on Software Metrics. Silver Falls, Oregon

  • Lee Y, Liang B, Wu S, Wang F (1995) Measuring the coupling and cohesion of an object-oriented program based on information flow. In: Proceedings of the International Conference on Software Quality. Maribor, Slovenia

  • Li W, Henry SM (1993) Object-oriented metrics that predict maintainability. J Syst Softw 23(2):111–122

    Article  Google Scholar 

  • Lindvall M (1998) Are large C++ classes change-prone? An empirical investigation. Software Pract Ex 28(15):1551–1558

    Article  Google Scholar 

  • Lindvall M (1999) Measurement of change: stable and change-prone constructs in a commercial C++ system. In: Proceedings of the 6th Software Metrics Symposium, 40–49

  • Lorenz M, Kidd J (1994) Object-oriented software metrics. Prentice Hall Object-Oriented Series, Englewood Cliffs

    Google Scholar 

  • Miller J (2000) Applying meta-analytical procedures to software engineering experiments. J Syst Softw 54(1):29–39

    Article  Google Scholar 

  • Myers EW (1986) An O(ND) difference algorithm and its variations. Algorithmic 1(2):251–266

    Article  MATH  Google Scholar 

  • Payton ME, Greenstone MH, Schenker N (2003) Overlapping confidence intervals or standard error intervals: what do they mean in terms of statistical significance? J Insect Sci 34(3):1–6

    Google Scholar 

  • Pickard L, Kitchenham B, Jones P (1998) Combining empirical results in software engineering. Inf Softw Technol 40(14):811–821

    Article  Google Scholar 

  • Scientific Toolworks, Inc., Understand for Java: user guide and reference manual. 2005, http://www.scitools.com

  • Succi G, Pedrycz W, Djokic S, Zuliani P, Russo B (2005) An empirical exploration of the distributions of the Chidamber and Kemerer Object-oriented metrics suite. Empir Softw Eng 10(1):81–104

    Article  Google Scholar 

  • Swets JA (1988) Measuring the accuracy of diagnostic systems. Science 240(4857):1285–1293

    Article  MathSciNet  MATH  Google Scholar 

  • Tegarden D, Sheetz S, Monarchi D (1992) A software complexity model of object-oriented systems. Decis Support Syst 13(34):241–262

    Google Scholar 

  • Witten IH, Frank E (2000) Data mining: practical machine learning tools and techniques with Java implementation. Second Edition, Morgan Kaufmann Publishers

  • Zhou Y, Xu B, Leung H (2009) Examining the potentially confounding effect of class size on the associations between object-oriented metrics and change-proneness. IEEE Trans Softw Eng 35(5):607–623

    Article  Google Scholar 

  • Zhou Y, Xu B, Leung H (2010) On the ability of complexity metrics to predict fault-prone classes in object-oriented systems. J Syst Softw 83(4):660–674

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the anonymous reviewers for their very insightful comments and helpful suggestions in greatly improving the quality of this paper. This research is partly supported by the National Natural Science Foundation of China (61073029, 60803008, 90818027, 91018005, and 61021062), the Hong Kong General Research Fund (PolyU5225/08E), the Program for New Century Excellent Talents in University (NCET-08-0274), and the Fundamental Research Funds for the Central Universities (1093020202).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Southeast University, Nanjing, Jiangsu, China

    Hongmin Lu

  2. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China

    Hongmin Lu, Yuming Zhou, Baowen Xu & Lin Chen

  3. Department of Computing, Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong

    Hareton Leung

Authors
  1. Hongmin Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuming Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Baowen Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hareton Leung
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Hongmin Lu or Yuming Zhou.

Additional information

Editor: Sandro Morasca

Appendices

Appendix A. Object-Oriented Metrics Used in This Study

Tables 12, 13, 14, and 15 describe the cohesion, coupling, inheritance, and size metrics used in this study. In each table, column “Name” gives the acronym of each metric, column “Definition” provides an informal definition of the metric, and column “Source” indicates the reference where the metric was originally proposed. Note that inheritance metrics are indeed a form of coupling metrics. In practice, however, many researchers distinguish inheritance metrics from coupling metrics. Our study follows a metric classification framework similar to that in (Briand et al. 2000).

Table 12 Cohesion metrics
Full size table
Table 13 Coupling metrics
Full size table
Table 14 Inheritance metrics
Full size table
Table 15 Size metrics
Full size table

Appendix B. The Plot of Pearson Correlation Coefficients and AUC Distributions

Figures 5 and 6 respectively show, for each metric, the distributions of Pearson correlation coefficients and AUC values on all the investigated systems.

Fig. 5
figure 5

Box plots of Pearson correlation coefficients

Full size image
Fig. 6
figure 6

Box plots of AUC values

Full size image

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lu, H., Zhou, Y., Xu, B. et al. The ability of object-oriented metrics to predict change-proneness: a meta-analysis. Empir Software Eng 17, 200–242 (2012). https://doi.org/10.1007/s10664-011-9170-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-011-9170-z

Keywords

Search

Navigation