Skip to main content
SpringerLink
Log in

Resource oriented selection of rule-based classification models: An empirical case study

  • Published:
Software Quality Journal Aims and scope Submit manuscript

Abstract

The amount of resources allocated for software quality improvements is often not enough to achieve the desired software quality. Software quality classification models that yield a risk-based quality estimation of program modules, such as fault-prone (fp) and not fault-prone (nfp), are useful as software quality assurance techniques. Their usefulness is largely dependent on whether enough resources are available for inspecting the fp modules. Since a given development project has its own budget and time limitations, a resource-based software quality improvement seems more appropriate for achieving its quality goals. A classification model should provide quality improvement guidance so as to maximize resource-utilization.

We present a procedure for building software quality classification models from the limited resources perspective. The essence of the procedure is the use of our recently proposed Modified Expected Cost of Misclassification (MECM) measure for developing resource-oriented software quality classification models. The measure penalizes a model, in terms of costs of misclassifications, if the model predicts more number of fp modules than the number that can be inspected with the allotted resources. Our analysis is presented in the context of our Rule-Based Classification Modeling (RBCM) technique. An empirical case study of a large-scale software system demonstrates the promising results of using the MECM measure to select an appropriate resource-based rule-based classification model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Basili, V.R., Briand, L.C., Melo, W.L. 1996 A validation of object-oriented design metrics as quality indicators. IEEE Transactions on Software Engineering 22(10):751–761.

    Article  Google Scholar 

  • Briand, L.C., Emam, K.E., Morasca, S. 1996 On the application of measurement theory in software engineering. Empricial Software Engineering Journal 1(1):61–88.

    Article  Google Scholar 

  • Conover, W.J. 1971 Practical Nonparametric Statistics. John Wiley and Sons, New York, NY, USA.

    Google Scholar 

  • Ebert, C. 1996 Classification techniques for metric-based software development. Software Quality Journal 5(4):255–272.

    Article  Google Scholar 

  • Fayyad, U.M. 1996 Data mining and knowledge discovery: Making sense out of data. IEEE Expert 11(4):20–25.

    Article  Google Scholar 

  • Fenton, N.E., Pfleeger, S.L. 1997 Software Metrics: A Rigorous and Practical Approach. PWS Publishing Company: ITP, Boston, MA, 2nd edition.

  • Hudepohl, J.P., Aud, S.J., Khoshgoftaar, T.M., Allen, E.B., Mayrand, J. 1996 Emerald: Software metrics and models on the desktop. IEEE Software 13(5):56–60.

    Article  Google Scholar 

  • Imam, K.E., Benlarbi, S., Goel, N., Rai, S.N. 2001 Comparing case-based reasoning classifiers for predicting high-risk software componenets. Journal of Systems and Software, Elsevier Science Publishing 55(3):301–320.

    Article  Google Scholar 

  • Johnson, R.A., Wichern, D.W. 1992 Applied Multivariate Statistical Analysis. Prentice Hall, Englewood Cliffs, NJ, USA, 2nd edition.

  • Khoshgoftaar, T.M., Allen, E.B., Deng, J. 2001 Controlling overfitting in software quality models: Experiments with regression trees and classification. In: Proceedings of 7th International Software Metrics Symposium, London, UK. IEEE Computer Society, pages, 190–198.

  • Khoshgoftaar, T.M. Allen, E.B., Deng, J. 2002 Using regression trees to classify fault-prone software modules. IEEE Transactions on Reliability 51(4):455–462.

    Article  Google Scholar 

  • Khoshgoftaar, T.M., Gao, K. Xiao, Y. 2002 A study of resource-based software quality classification models. Technical Report TR-CSE-02-49, Florida Atlantic University, Boca Raton, FL, USA.

  • Khoshgoftaar, T.M. Seliya, N. 2002 Improving usefulness of software quality classification models based on boolean discriminant functions. In: Proceedings: 13th International Symposium on Software Reliability Engineering, Annapolis, Maryland, USA. IEEE Computer Society, pages, 221–230.

  • Khoshgoftaar, T.M. Seliya, N. 2002 A rule-based classification model for software quality prediction. Technical Report TR-CSE-02-45, Florida Atlantic University, Boca Raton, FL, USA.

  • Khoshgoftaar, T.M. Seliya, N. 2003 Analogy-based practical classification rules for software quality estimation. Empirical Software Engineering Journal 8(4):325–350.

    Article  Google Scholar 

  • Khoshgoftaar, T.M. Seliya, N. 2004 Comparative assessment of software quality classification techniques: An empirical case study. Empirical Software Engineering Journal 9(3):229–257.

    Article  Google Scholar 

  • Khoshgoftaar, T.M., Seliya, N., Herzberg, A. 2005 Resource-oriented software quality classification models. Journal of Systems and Software 76(2):111–126.

    Article  Google Scholar 

  • Nguyen, L. 2002 A comparative study of attribute selection techniques for cbr-based software quality classification models. Master’s thesis, Florida Atlantic University, Boca Raton, FL, USA. Advised by Taghi M. Khoshgoftaar.

  • Ohlsson, N., Helander, M., Wohlin, C. 1996 Quality improvement by identification of fault-prone modules using software design metrics. In: Proceedings: International Conference on Software Quality, Ottawa, Ontario, Canada, pages, 1–13.

  • Ping, Y., Systa, T., Muller, H. 2002 Predicting fault-proneness using OO metrics: An industrial case study. In T. Gyimothy and F. B. Abreu, editors, Proceedings of 6th European Conference on Software Maintenance and Reengineering, Budapest, Hungary, pages, 99–107.

  • Pizzi, N.J., Summers, R., Pedrycz, W. 2002 Software quality prediction using median-adjusted class labels. In Proceedings: International Joint Conference on Neural Networks, volume 3, Honolulu, HI. IEEE Computer Society, pages, 2405–2409.

  • Reformat, M., Pedrycz, W., Pizzi, N.J. 2002 Software quality analysis with the use of computational intelligence. In: Proceedings: IEEE International Conference on Fuzzy Systems, volume 2, Honolulu, HI, USA, pages, 1156–1161.

  • Runeson, P., Ohlsson, M.C., Wohlin, C. 2001 A classification scheme for studies on fault-prone components. Lecture Notes in Computer Science, 2188:341–355. Springer Link.

  • Schneidewind, N.F. 1997 Software metrics model for integrating quality control and prediction. In: Proceedings of 8th International Symposium on Software Reliability Engineering, Albuquerque, NM. IEEE Computer Society, pages, 402–415.

  • Schneidewind, N.F. 2001 Investigation of logistic regression as a discriminant of software quality. In: Proceedings of 7th International Software Metrics Symposium, London, UK. IEEE Computer Society, pages, 328–337.

  • Schneidewind, N.F. 2002 Body of knowledge for software quality measurement. IEEE Computer 35(2):77–83.

    Google Scholar 

  • Suarez, A., Lutsko, J.F. 1999 Globally optimal fuzzy decision trees for classification and regression. Pattern Analysis and Machine Intelligence, IEEE Computer Society 21(12):1297–1311.

    Article  Google Scholar 

  • Votta, L.G., Porter, A.A. 1995 Experimental software engineering: A report on the state of the art. In: Proceedings of the 17th. International Conference on Software Engineering, Seattle, WA USA. IEEE Computer Society, pages, 277–279.

  • Wohlin, C., Runeson, P., Host, M., Ohlsson, M.C., Regnell, B., Wesslen, A. 2000 Experimentation in Software Engineering: An Introduction. Kluwer International Series in Software Engineering. Kluwer Academic Publishers, Boston, MA.

Download references

Author information

Authors and Affiliations

  1. Computer Science and Engineering, Florida Atlantic University, 777 West Glades Road, Boca Raton, Florida, 33431, USA

    Taghi M. Khoshgoftaar & Angela Herzberg

  2. Computer and Information Science, University of Michigan-Dearborn, 4901 Evergreen Road, Dearborn, Michigan, 48128

    Naeem Seliya

Authors
  1. Taghi M. Khoshgoftaar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Angela Herzberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Naeem Seliya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Taghi M. Khoshgoftaar.

Additional information

Taghi M. Khoshgoftaar is a professor of the Department of Computer Science and Engineering, Florida Atlantic University and the Director of the graduate programs and research. His research interests are in software engineering, software metrics, software reliability and quality engineering, computational intelligence applications, computer security, computer performance evaluation, data mining, machine learning, statistical modeling, and intelligent data analysis. He has published more than 300 refereed papers in these areas. He is a member of the IEEE, IEEE Computer Society, and IEEE Reliability Society. He was the general chair of the IEEE International Conference on Tools with Artificial Intelligence 2005.

Naeem Seliya is an Assistant Professor of Computer and Information Science at the University of Michigan - Dearborn. He recieved his Ph.D. in Computer Engineering from Florida Atlantic University, Boca Raton, FL, USA in 2005. His research interests include software engineering, data mining and machine learnring, application and data security, bioinformatics and computational intelligence. He is a member of IEEE and ACM.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Khoshgoftaar, T.M., Herzberg, A. & Seliya, N. Resource oriented selection of rule-based classification models: An empirical case study. Software Qual J 14, 309–338 (2006). https://doi.org/10.1007/s11219-006-0038-1

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11219-006-0038-1

Keyword

Search

Navigation