Skip to main content
Log in

Understanding the value of considering client usage context in package cohesion for fault-proneness prediction

  • Published:
Automated Software Engineering Aims and scope Submit manuscript

Abstract

By far, many package cohesion metrics have been proposed from internal structure view and external usage view. Based on whether client usage context (i.e., the way packages are used by their clients) is exploited, we group these metrics into two categories: non-context-based and context-based. Currently, there is no comprehensive empirical research devoted to understanding the actual value of client usage context for fault-proneness prediction. In this study, we conduct a thorough empirical study to investigate the value of considering client usage context in package cohesion for fault-proneness prediction. First, we use principal component analysis to examine the relationships between context-based and non-context-based cohesion metrics. Second, we employ univariate logistic regression analysis to investigate the correlation between context-based cohesion metrics and fault-proneness. Then, we build multivariate prediction models to analyze the ability of context-based cohesion metrics for fault-proneness prediction when used alone or used together with non-context-based cohesion metrics. To obtain comprehensive evaluations, we evaluate the effectiveness of these multivariate models in the ranking and classification scenarios from both cross-validation and across-version perspectives. The experimental results show that (1) context-based cohesion metrics are complementary to non-context-based cohesion metrics; (2) most of context-based cohesion metrics have a significantly negative association with fault-proneness; (3) when used alone or used together with non-context-based cohesion metrics, context-based cohesion metrics can substantially improve the effectiveness of fault-proneness prediction in most studied systems under both cross-validation and across-version evaluation. Client usage context has an important value in package cohesion for fault-proneness prediction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. www.scitools.com.

  2. http://www.scitools.com.

  3. http://openscience.us/repo/.

References

  • Abdeen, H., Ducasse, S., Sahraoui, H. : Modularization metrics: assessing package organization in legacy large object-oriented software. In: WCRE, pp. 394-398 (2011)

  • Abdeen, H., Sahraoui, H., Shata, O., Anquetil, N., Ducasse, S.: Towards automatically improving package structure while respecting original design decisions. In: IEEE Working Conference on Reverse Engineering (WCRE), pp. 212–221 (2013)

  • Abdi, H.: Factor Rotations in Factor Analyses. Encyclopedia for Research Methods for the Social Sciences. Sage, Thousand Oaks (2003)

    Google Scholar 

  • Allen, E.B., Khoshgoftaar, T.M., Chen, Y.: Measuring coupling and cohesion of software modules: an information theory approach. In: METRICS, pp. 124–134 (2001)

  • Arisholm, E., Briand, L.C., Fuglerud, M.: Data mining techniques for building fault-proneness models in telecom java software. In: Proceedings of the 18th IEEE International Symposium on Software Reliability, pp. 215–224 (2007)

  • Arisholm, E., Briand, L.C., Johannessen, B.: A systematic and comprehensive investigation of methods to build and evaluate fault prediction models. J. Syst. Softw. 83(1), 2–17 (2010)

    Article  Google Scholar 

  • Basili, V., Briand, L.C., Melo, W.: A validation of object-oriented design metrics as quality indicators. IEEE Trans. Softw. Eng. 22(10), 751–761 (1996)

    Article  Google Scholar 

  • Belsley, D., Kuh, E., Welsch, R.: Regression Diagnostics: Identifying Influential Data and Sources of Collinearity. Wiley, New York (1980)

    Book  MATH  Google Scholar 

  • Belsley, D., Kuh, E., Welsch, R.: Regression Diagnostics: Identifying Influential Data and Sources of Collinearity. Wiley, New York (2005)

    MATH  Google Scholar 

  • Briand, L.C., Morasca, S., Basili, V.R.: Property-based software engineering measurement. IEEE Trans. Softw. Eng. 22(1), 68–86 (1996)

    Article  Google Scholar 

  • Briand, L.C., Morasca, S., Basili, V.R.: Defining and validating measures for object-based high-level design. IEEE Trans. Softw. Eng. 25(5), 722–743 (1999)

    Article  Google Scholar 

  • Briand, L., Wüst, J., Daly, J., Porter, D.: Exploring the relationships between design measures and software quality in object-oriented systems. J. Syst. Softw. 51(3), 245–273 (2000)

    Article  Google Scholar 

  • Chen, L., Fang, B., Shang, Z., Tang, Y.: Negative samples reduction in cross-company software defects prediction. Inf. Softw. Technol. 62, 67–77 (2015)

    Article  Google Scholar 

  • Chowdhury, I., Zulkernine, M.: Using complexity, coupling, and cohesion metrics as early indicators of vulnerabilities. J. Syst. Archit. 57(3), 294–313 (2011)

    Article  Google Scholar 

  • Constantine, L., Yourdon, E.: Structured Design. Prentice Hall, Upper Saddle River (1979)

    MATH  Google Scholar 

  • Couto, C., Pires, P., Valente, M.T., Bigonha, R.S.: Predicting software defects with causality tests. J. Syst. Softw. 93, 24–41 (2014)

    Article  Google Scholar 

  • D’Ambros, M., Lanza, M., Robbes, R.: Evaluating defect prediction approaches: a benchmark and an extensive comparison. Empir. Softw. Eng. 17(4–5), 531–577 (2012)

    Article  Google Scholar 

  • Doval, D., Mancoridis, S., Mitchell, B.S. : Automatic clustering of software systems using a genetic algorithm. In: STEP, pp. 73–81 (1999)

  • Elish, M.O., Al-Yafei, A.H., Al-Mulhem, M.: Empirical comparison of three metrics suites for fault prediction in packages of object-oriented systems: A case study of Eclipse. Adv. Eng. Softw. 42(10), 852–859 (2011)

    Article  Google Scholar 

  • Fenton, N., Pfleeger, S.L.: Software Metrics: A Rigorous and Practical Approach, 2nd edn. International Thomson Computer Press, London (1996)

    Google Scholar 

  • Gorman, J.: OO design principles & metrics. http://www.codemanship.co.uk/parlezuml/metrics/OO%20Design%20Principles%20&%20Metrics.pdf (2006)

  • Gupta, V., Chhabra, J.K.: Package level cohesion measurement in object-oriented software. J. Braz. Comput. Soc. 18(3), 251–266 (2012)

    Article  Google Scholar 

  • Hall, T., Beecham, S., Bowes, D., Gray, D., Counsell, S.: A systematic literature review on fault prediction performance in software engineering. IEEE Trans. Softw. Eng. 38(6), 1276–1304 (2012)

    Article  Google Scholar 

  • Hata, H., Mizuno, O., Kikuno, T.: Bug prediction based on fine-grained module histories. In: Proceedings of the 34th International Conference on Software Engineering, pp. 200–210 (2012)

  • He, P., Li, B., Liu, X., Chen, J., Ma, Y.: An Empirical Study on Software Defect Prediction with a Simplified Metric Set, vol. 59, pp. 170–190. Wuhan University, Wuhan (2014)

  • He, Z., Shu, F., Yang, Y., Li, M., Wang, Q.: An investigation on the feasibility of interproject defect prediction. Autom. Softw. Eng. 19(2), 167–199 (2012)

    Article  Google Scholar 

  • Hocking, R.R.: The analysis and selection of variables in linear regression. Biometrics 32(1), 1–49 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  • Howell, D.C.: Statistical Methods for Psychology. Duxbury Press, Belmont (2002)

    Google Scholar 

  • Janes, A., Scotto, M., Pedrycz, W., Russo, B., Stefanovic, M., Succi, G.: Identification of defect-prone classes in telecommunication software systems using design metrics. Inf. Sci. 176(24), 3711–3734 (2006)

    Article  Google Scholar 

  • Kaiser, H.F.: The varimax criterion for analytic rotation in factor analysis. Psychometrika 23(3), 187–200 (1958)

    Article  MATH  Google Scholar 

  • Kamei, Y., Matsumoto, S., Monden, A., Matsumoto, K., Adams, B. , Hassan, A.E.: Revisiting common bug prediction findings using effort-aware models. In: Proceedings of the 26th IEEE International Conference on Software Maintenance, pp. 1–10 (2010)

  • Kamei, Y., Shihab, E., Adams, B., Hassan, A.E., Mockus, A., Sinha, A., Ubayashi, N.: A large-scale empirical study of just-in-time quality assurance. IEEE Trans. Softw. Eng. 39(6), 757–773 (2013)

    Article  Google Scholar 

  • Kouroshfar, E.: Studying the effect of co-change dispersion on software quality. In: International Conference on Software Engineering (ICSE), pp. 1450–1452 (2013)

  • Lee, Y.S., Liang, B.S.: Measuring the coupling and cohesion of an object-oriented program based on information flow. In: ICSQ, pp. 81–90 (1995)

  • Marcus, A., Poshyvanyk, D., Ferenc, R.: Using the conceptual cohesion of classes for fault prediction in object-oriented systems. IEEE Trans. Softw. Eng. 34(2), 287–300 (2008)

    Article  Google Scholar 

  • Martin, R.C.: Design principles and design patterns. Object Mentor 1, 34 (2000)

    Google Scholar 

  • Martin, R.C.: Agile Software Development: Principles, Patterns, and Practices. Prentice-Hall, Upper Saddle River (2002)

    Google Scholar 

  • Meneely, A., Williams, L., Snipes, W., Osborne, J.: Predicting failures with developer networks and social network analysis. In: Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering. ACM, pp. 13–23 (2008)

  • Menzies, T., Greenwald, J., Frank, A.: Data mining static code attributes to learn defect predictors. IEEE Trans. Softw. Eng. 33(1), 2–13 (2007)

    Article  Google Scholar 

  • Misic, V.B.: Cohesion is structural, coherence is functional: different views, different measures. In: METRICS, pp. 135–144 (2001)

  • Morris, K.: Metrics for object-oriented software development environments. Master’s thesis, Sloan School of Management MIT (1989)

  • Nagappan, N., Ball, T., Zeller, A.: Mining metrics to predict component failures. In: ICSE, pp. 452–461 (2006)

  • Patel, S., Chu, W., Baxter, R.: A measure for composite module cohesion. In: ICSE, pp. 38–48 (1992)

  • Peters, F., Menzies, T., Gong, L., Zhang, H.: Balancing privacy and utility in crosscompany defect prediction. IEEE Trans. Softw. Eng. 39(8), 1054–1068 (2013)

    Article  Google Scholar 

  • Ponisio, L.: Exploiting client usage to manage program modularity. Ph.D. thesis, University of Berne (2006)

  • Ponisio, L., Nierstrasz, O.: Using contextual information to assess package cohesion. Technical Report IAM-06-002, Institute of Applied Mathematics and Computer Sciences, University of Berne (2006)

  • Praditwong, K., Harman, M., Yao, X.: Software module clustering as a multi-objective search problem. IEEE Trans. Softw. Eng. 37(2), 264–282 (2011)

    Article  Google Scholar 

  • Rahman, F., Devanbu, P.: How and why, process metrics are better. In: ICSE, pp. 432–441 (2013)

  • Rahman, F., Posnett, D., Devanbu, P.: Recalling the imprecision of cross-project defect prediction. In : Proceeding of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering, p. 61 (2012)

  • Rahman, F., Posnett, D., Devanbu, P.: Recalling the imprecision of cross-project defect prediction. In: FSE, p. 61 (2012)

  • Reinard, J.C.: Communication Research Statistics. Sage Publications Inc, Thousand Oaks (2006)

    Book  Google Scholar 

  • Sarkar, S., Kak, A.C., Rama, G.M.: Metrics for measuring the quality of modularization of large-scale object-oriented software. IEEE Trans. Softw. Eng. 34(5), 700–720 (2008)

    Article  Google Scholar 

  • Schein, A.I., Saul, L.K., Ungar, L.H.: A generalized linear model for principal component analysis of binary data. In: Proceedings of the 9th International Workshop on Artificial Intelligence and Statistics (2003)

  • Schroter, A., Zimmermann, T., Zeller, A.: Predicting component failures at design time. In: Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering. ACM, pp. 18–27 (2006)

  • Shihab, E., Kamei, Y., Adams, B., Hassan, A.E.: Is lines of code a good measure of effort in effort-aware models. Inf. Softw. Technol. 55(11), 1981–1993 (2013)

    Article  Google Scholar 

  • Shin, Y., Meneely, A., Williams, L., Osborne, J.A.: Evaluating complexity, code churn, and developer activity metrics as indictors of software vulnerabilities. IEEE Trans. Softw. Eng. 37(6), 772–787 (2011)

    Article  Google Scholar 

  • Shin, Y., Williams, L.: Can traditional fault prediction models be used for vulnerability prediction? Empir. Softw. Eng. 18(1), 25–59 (2013)

    Article  Google Scholar 

  • Singh, Y., Kaur, A., Malhotra, R.: Empirical validation of object-oriented metrics for predicting fault proneness models. Softw. Qual. J. 18(1), 3–35 (2010)

    Article  Google Scholar 

  • Tan, M., Tan, L., Dara, S., Mayuex, C.: Online defect prediction for imbalanced data. In: ICSE-SEIP (2015)

  • Turhan, B., Tosun, A.: Misirli, A. Bener, Empirical evaluation of the effects of mixed project data on learning defect predictors. Inf. Softw. Technol. 55(6), 1101–1118 (2013)

    Article  Google Scholar 

  • Vernazza, T., Granatella, G., Succi, G., Benedicenti, L., Mintchev, M.: Defining metrics for software components. In: WMSCI, pp. 16–23 (2000)

  • Weyuker, E.J., Ostrand, T.J., Bell, R.M.: Using developer information as a factor for fault prediction. In: Proceedings of the 3rd International Worksho Data Mining Techniques p on Predictor Models in Software Engineering, p. 8 (2007)

  • Xu, B., Chen, Z., Zhao, J.: Measuring cohesion of packages in Ada95. SIGAda, pp. 62–67 (2003)

  • Yang, Y., Zhou, Y., Lu, H., Chen, L., Chen, Z., Xu, B., Leung, H., Zhang, Z.: Are slice-based cohesion metrics actually useful in effort-aware post-release fault-proneness prediction? An empirical study. IEEE Trans. Softw. Eng. 41(4), 331–357 (2015)

    Article  Google Scholar 

  • Zhang, H., Cheung, S.C.: A cost-effectiveness criterion for applying software defect prediction models. In: ESEC/FSE, pp. 643–646 (2013)

  • Zhou, T., Xu, B., Shi, L., Zhou, Y., Chen, L.: Measuring package cohesion based on context. In: Proceedings of the IEEE international workshop on semantic computing and systems, pp. 127–132 (2008)

  • Zhou, T., Xu, B., Shi, L., Zhou, Y.: Measuring package cohesion based on client usages. J. Softw. 20(2), 256–270 (2009)

    Article  Google Scholar 

  • Zhou, Y., Xu, B., Leung, H., Chen, L.: An in-depth study of the potentially confounding effect of class size in fault prediction. ACM Trans. Softw. Eng. Methodol. 23(1), 10 (2014)

    Article  Google Scholar 

  • Zhou, Y., Leung, H.: Empirical analysis of object-oriented design metrics for predicting high and low severity faults. IEEE Trans. Softw. Eng. 32(10), 771–789 (2006)

    Article  Google Scholar 

  • Zimmermann, T., Nagappan, N.: Predicting defects with program dependencies. In: Proceedings of the 3rd International Symposium on Empirical Software Engineering and Measurement. IEEE Computer Society pp. 435–438 (2009)

  • Zimmermann, T., Premraj, R., Zeller, A.: Predicting defects for eclipse. In: Proceedings of the International Workshop on Predictor Models in Software Engineering. IEEE Computer Society, p. 9 (2007)

Download references

Acknowledgments

The work in this paper is supported by the National Key Basic Research and Development Program of China (2014CB340702), the National Natural Science Foundation of China (91318301, 61432001, 61321491, 61300051, 61272082), and the National Natural Science Foundation of Jiangsu Province (BK20130014)

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yuming Zhou.

Appendices

Appendix 1: Principal component analysis results

Tables 15, 16, 17, 18, 19, 20, 21, 22, 23, and 24 show the rotated components from PCA for the six data sets. The second to fourth rows in the tables show the eigenvalue, the percentage of variance explained by each PC, and the cumulative percentage of variances. The remaining rows, starting with the fifth row, show the loading of each metric in each PC, which indicates the degree of correlation with the PC. In each row, the loading with the largest absolute value is shown in bold. This enables us to observe which metrics capture the same property and hence belong to the same PC.

Appendix 2: The multivariate logistic regression models

Table 25 summarizes the “N”, “C”, and “C+N” models for each data set. The first column indicates the data set and the second column is the type of the multivariate logistic regression model. The third column shows the R square. The remaining columns, starting from the fourth column, show the selected independent variables. Below each variable name, we present the corresponding coefficient and p value.

Appendix 3: Comparison between cohesion metrics and traditional metrics

Table 26 describes the most commonly used twenty traditional metrics, including SLOC, structural complexity, halstead, and process churn metrics. All these metrics are collected in package level in our study. We build two types of multivariate logistic regression models: (1) “Cohesion” model (using 16 package cohesion metrics introduced in Sect. 2); (2) “Tradition” model (using 20 traditional metrics in Table 26); Based on ten Java systems introduced in Sect. 3.6, we perform 30 times threefold cross-validation. Fig. 6 shows the fault prediction performance for each model in both ranking and classification scenarios. Each sub-figure corresponds to one distinct data set, and reveals the mean value in terms of CE at \(\pi =0.2\), ER-BPP, ER-BCE, and ER-MFM.

Fig. 6
figure 6

The fault-prediction performance in terms of CE and ER

From Fig. 6, we can see that, cohesion metrics outperform traditional metrics in six out of the ten systems (i.e., Cxf 2.1, Flume 1.4, Lucene 2.4, Wicket 6.6, Jdtcore 3.4, and Eclipse 2.0). While in Hive 0.12.0, cohesion metrics perform a little worse than traditional metrics. In Camel 2.9.0, cohesion metrics performs worse in terms of CE at \(\pi =0.2\) and ER-BPP, but performs better in terms of ER-BCE and ER_MFM. For the other two systems Hbase0.96.0 and Eclipse3.0, cohesion metrics and traditional metrics have similar performance for fault prediction. Overall, the results show that cohesion metrics have a better performance than traditional metrics in most of the studied systems in both ranking and classification scenarios.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, Y., Yang, Y., Lu, H. et al. Understanding the value of considering client usage context in package cohesion for fault-proneness prediction. Autom Softw Eng 24, 393–453 (2017). https://doi.org/10.1007/s10515-016-0198-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10515-016-0198-6

Keywords

Navigation