Skip to main content
SpringerLink
Log in

Empirical validation for effectiveness of fault prediction technique based on cost analysis framework

  • Original Article
  • Published:
International Journal of System Assurance Engineering and Management Aims and scope Submit manuscript

Abstract

Prediction of the faulty or non-faulty modules in a software is an important task in software development life cycle and often carried out by validating them through certain empirical techniques. A good number of techniques have been proposed by different authors to develop fault prediction models and determine best suitable techniques for fault prediction based on certain performance criteria. However it appears that there is a scope to extend the research by giving emphasis on feasibility analysis of fault prediction techniques. In this paper, a cost evaluation framework has been proposed to evaluate the effectiveness of developed fault prediction models. Statistical methods such as linear regression, logistic regression, polynomial regression, Naive Bayes and support vector machine have been applied to develop a classifier in order to predict as to whether any module is a faulty or non-faulty one for an embedded software system. The proposed approach has been applied on five different public domain software system chosen from NASA database consisting of different number of faulty models. The performance of the predicted models are evaluated using different performance evaluation parameters. From cost-based analysis, experimental results reveal that our fault prediction model is best suitable for projects with faulty modules less than a certain threshold value depending on cost and fault identification efficiency of different testing phases. The results also show that the model developed using logistic regression technique obtained promising results in terms of performance and cost when compared with other applied techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. http://promise.site.uottawa.ca/SERepository.

  2. http://promise.site.uottawa.ca/SERepository/datasets/cm1.arff.

  3. http://promise.site.uottawa.ca/SERepository/datasets/kc1.arff.

  4. http://promise.site.uottawa.ca/SERepository/datasets/kc2.arff.

  5. http://promise.site.uottawa.ca/SERepository/datasets/pc1.arff.

  6. http://promise.site.uottawa.ca/SERepository/datasets/jm1.arff.

  7. http://www.locmetrics.com/alternatives.html.

References

  • Abreu FBE, Carapuca R (1994) Object-oriented software engineering: measuring and controlling the development process. In: Proceedings of the 4th international conference on software quality, vol 186, pp 1–8

  • Aggarwal K, Singh Y, Kaur A, Malhotra R (2009) Empirical analysis for investigating the effect of object-oriented metrics on fault proneness: a replicated case study. Softw Process: Improv Pract 14(1):39–62

    Article  Google Scholar 

  • Aggarwal K, Singh Y, Kaur A, Malhotra R (2010) Empirical validation of object-oriented metrics for predicting fault proneness models. Softw Qual J 18(1):3–35

    Article  Google Scholar 

  • Arisholm E, Briand L, Johannessen EB (2010) A systematic and comprehensive investigation of methods to build and evaluate fault prediction models. Empir Softw Eng 83(1):2–17

    Google Scholar 

  • Basili VR, Briand LC, Melo WL (1996) A validation of object-oriented design metrics as quality indicators. IEEE Trans Softw Eng 22(10):751–761

    Article  Google Scholar 

  • Briand LC, Wüst J, Ikonomovski SV, Lounis H (1999) Investigating quality factors in object-oriented designs: an industrial case study. In: Proceedings of the 21st international conference on Software engineering, ACM, pp 345–354

  • Briand LC, Wüst J, Daly JW, Porter DV (2000) Exploring the relationships between design measures and software quality in object-oriented systems. J Syst Softw 51(3):245–273

    Article  Google Scholar 

  • Burrows R, Ferrari FC, Lemos OA, Garcia A, Taiani F (2010) The impact of coupling on the fault-proneness of aspect-oriented programs: an empirical study. In: 2010 IEEE 21st international symposium on software reliability engineering (ISSRE), pp 329–338

  • Cartwright M, Shepperd M (2000) An empirical investigation of an object-oriented software system. IEEE Trans Softw Eng 26(8):786–796

    Article  Google Scholar 

  • Chidamber SR, Darcy DP, Kemerer CF (1998) Managerial use of metrics for object-oriented software: an exploratory analysis. IEEE Trans Softw Eng 24(8):629–639

    Article  Google Scholar 

  • Cruz AEC, Ochimizu K (2009) Towards logistic regression models for predicting fault-prone code across software projects. In: Proceedings of the 3rd international symposium on empirical software engineering and measurement 2009. ESEM 2009, pp 460–463

  • Emam EL, Melo W, Machado JC (2001) The prediction of faulty classes using object-oriented design metrics. J Syst Softw 56(1):63–75

    Article  Google Scholar 

  • Fokaefs M, Mikhaiel R, Tsantalis N, Stroulia E, Lau A (2011) An empirical study on web service evolution. In: IEEE international conference on Web services (ICWS), 2011, pp 49–56

  • Gyimothy T, Ferenc R, Siket I (2005) Empirical validation of object-oriented metrics on open source software for fault prediction. IEEE Trans Softw Eng 31(10):897–910

    Article  Google Scholar 

  • Halstead M (1977) Elements of software science. Elsevier, New York

    MATH  Google Scholar 

  • Huitt R, Wilde N (1992) Maintenance support for object-oriented programs. IEEE Trans Softw Eng 18(12):1038–1044

    Article  Google Scholar 

  • Jabangwe R, Börstler J, Šmite D, Wohlin C (2015) Empirical evidence on the link between object-oriented measures and external quality attributes: a systematic literature review. Empir Softw Eng 20(3):640–693

    Article  Google Scholar 

  • Jain YK, Bhandare SK (2011) Min max normalization based data perturbation method for privacy protection. Int J Comput Commun Technol 2(8):45–50

    Google Scholar 

  • Jiang Y, Cukic B, Ma Y (2008) Techniques for evaluating fault prediction models. Empir Softw Eng 13(5):561–595

    Article  Google Scholar 

  • Jones C (2011) Software quality in 2011: a survey of the state of the art. Capers Jones & Associates LLC, Boston

    Google Scholar 

  • Kanmani S, Uthariaraj VR, Sankaranarayanan V, Thambidurai P (2007) Object-oriented software fault prediction using neural networks. Inf Softw Technol 49(5):483–492

    Article  Google Scholar 

  • Kapila H, Singh S (2013) Analysis of ck metrics to predict software fault-proneness using bayesian inference. Int J Comput Appl 74(2):1–4

    Google Scholar 

  • Kohavi R (1995) A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Proceedings of the 14th international joint conference on artificial intelligence, San Mateo, pp 1137–1143

  • Li W, Henry S (1993) Maintenance metrics for the object-oriented paradigm. In: Proceedings of first international software metrics symposium, pp 52–60

  • Lorenz M, Kidd J (1994) Object-oriented software metrics. Prentice-Hall, Englewood

    Google Scholar 

  • Malhotra R, Jain A (2012) Fault prediction using statistical and machine learning methods for improving software quality. J Inf Process Syst 8(2):241–262

    Article  Google Scholar 

  • Malhotra R, Singh Y (2011) On the applicability of machine learning techniques for object oriented software fault prediction. Softw Eng Int J 1(1):24–34

    Google Scholar 

  • McCabe TJ (1976) A complexity measure. IEEE Trans Softw Eng 2(4):308–320

    Article  MATH  MathSciNet  Google Scholar 

  • Mishra B, Shukla K et al (2012) Defect prediction for object oriented software using support vector based fuzzy classification model. Int J Comput Appl 60(15):8–16

    Google Scholar 

  • Nagappan N, Williams L, Vouk M, Osborne J (2005) Early estimation of software quality using in-process testing metrics: a controlled case study. ACM SIGSOFT Softw Eng Notes 30:1–7

    Article  Google Scholar 

  • Olague HM, Etzkorn LH, Gholston S, Quattlebaum S (2007) Empirical validation of three software metrics suites to predict fault-proneness of object-oriented classes developed using highly iterative or agile software development processes. IEEE Trans Softw Eng 33(6):402–419

    Article  Google Scholar 

  • Pai GJ, Dugan JB (2007) Empirical analysis of software fault content and fault proneness using bayesian methods. IEEE Trans Softw Eng 33(10):675–686

    Article  Google Scholar 

  • Wagner S (2006) A literature survey of the quality economics of defect-detection techniques. In: Proceedings of the ACM/IEEE international symposium on empirical software engineering (ISESE), pp 194–203

  • Shatnawi R, Li W (2008) The effectiveness of software metrics in identifying error-prone classes in post-release software evolution process. J Syst Softw 81(11):1868–1882

    Article  Google Scholar 

  • Singh Y, Kaur A, Malhotra R (2009) Software fault proneness prediction using support vector machines. Proc World Congr Eng 1:1–3

    Google Scholar 

  • Mende T, Koschke R (2009) Revisiting the evaluation of defect prediction models. In: Proceedings of the 5th international conference on predictor models in software engineering (PROMISE), pp 1–10

  • Mende T, Koschke R (2010) Effort-aware defect prediction models. In: Proceedings of 14th IEEE European conference on software maintenance and re-engineering (CSMR), pp 107–116

  • Tomaszewski P, Håkansson J, Grahn H, Lundberg L (2007) Statistical models versus expert estimation for fault prediction in modified code-an industrial case study. J Syst Softw 80(8):1227–1238

    Article  Google Scholar 

  • Zhou Y, Leung H (2006) Empirical analysis of object-oriented design metrics for predicting high and low severity faults. IEEE Trans Softw Eng 32(10):771–789

    Article  Google Scholar 

  • Zhou Y, Leung H (2007) Predicting object-oriented software maintainability using multivariate adaptive regression splines. J Mater Process Technol 80(8):1349–1361

    Google Scholar 

  • Zhou Y, Xu B, Leung H (2010) On the ability of complexity metrics to predict fault-prone classes in object-oriented systems. J Syst Softw 83(4):660–674

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dept. CS&E, National Institute of Technology, Rourkela, India

    Lov Kumar & Santanu Ku. Rath

Authors
  1. Lov Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Santanu Ku. Rath
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lov Kumar.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kumar, L., Rath, S.K. Empirical validation for effectiveness of fault prediction technique based on cost analysis framework. Int J Syst Assur Eng Manag 8 (Suppl 2), 1055–1068 (2017). https://doi.org/10.1007/s13198-016-0566-4

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13198-016-0566-4

Keywords

Search

Navigation