Advertisement

A Diagnostic Reasoning Approach to Defect Prediction

  • Rui Abreu
  • Alberto Gonzalez-Sanchez
  • Arjan J. C. van Gemund
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6704)

Abstract

During software testing, defect prediction approaches measure current reliability status, forecasting future program failures, and provide information on how many defects need to be removed before shipping. Existing approaches often require faults to be detected and identified as a new one, before a model-based trend can be fitted. While during regression testing failures may frequently occur, it is not evident which are related to new faults. Consequently, reliability growth trending can only be performed in sync with fault identification and repair, which is often performed in between regression test cycles. In this paper we present a dynamic, reasoning approach to estimate the number of defects in the system early in the process of regression testing. Our approach, coined Dracon, is based on Bayesian fault diagnosis over abstractions of program traces (also known as program spectra). Experimental results show that Dracon systematically estimates the exact number of (injected) defects, provided sufficient tests cases are available. Furthermore, we also propose a simple, analytic performance model to assess the influence of failed test cases in the estimation. We observe that our empirical findings are in agreement with the model.

Keywords

Defect prediction Bayes diagnosis program spectra 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Abreu, R., van Gemund, A.J.C.: A low-cost approximate minimal hitting set algorithm and its application to model-based diagnosis. In: Proc. of SARA 2009, pp. 2–8 (2009)Google Scholar
  2. 2.
    Abreu, R., Zoeteweij, P., Golsteijn, R., van Gemund, A.: A practical evaluation of spectrum-based fault localization. Journal of Systems and Software 82(11), 1780–1792 (2009)CrossRefGoogle Scholar
  3. 3.
    Abreu, R., Zoeteweij, P., van Gemund, A.J.C.: Localizing software faults simultaneously. In: Proc. of QSIC 2009, pp. 367–376 (2009)Google Scholar
  4. 4.
    Abreu, R., Zoeteweij, P., van Gemund, A.J.C.: A new bayesian approach to multiple intermittent fault diagnosis. In: Proc. of IJCAI 2009, pp. 653–658 (2009)Google Scholar
  5. 5.
    Bertolino, A., Strigini, L.: On the use of testability measures for dependability assessment. IEEE TSE 22(2), 97–108 (1996)Google Scholar
  6. 6.
    de Kleer, J.: Diagnosing multiple persistent and intermittent faults. In: Proc. IJCAI 2009, pp. 733–738 (2009)Google Scholar
  7. 7.
    de Kleer, J., Williams, B.C.: Diagnosing multiple faults. Artif. Intell. 32(1), 97–130 (1987)CrossRefzbMATHGoogle Scholar
  8. 8.
    Elbaum, S., Malishevsky, A., Rothermel, G.: Incorporating varying test costs and fault severities into test case prioritization. In: Proc. of ICSE 2001, pp. 329–338 (2001)Google Scholar
  9. 9.
    Feldman, A., van Gemund, A.J.C.: A two-step hierarchical algorithm for model-based diagnosis. In: Proc. of AAAI 2006, pp. 827–833 (2006)Google Scholar
  10. 10.
    Fenton, N.E., Neil, M.: A critique of software defect prediction models. IEEE TSE 25(5), 675–689 (1999)Google Scholar
  11. 11.
    Goel, A.L., Okumoto, K.: Software Reliability. McGraw-Hill, New York (1987)zbMATHGoogle Scholar
  12. 12.
    Harrold, M.J., Rothermel, G., Wu, R., Yi, L.: An empirical investigation of program spectra. In: Proc. of PASTE 1998, pp. 83–90 (1998)Google Scholar
  13. 13.
    Jiang, B., Zhang, Z., Tse, T.H., Chen, T.Y.: How well do test case prioritization techniques support statistical fault localization. In: Proc. COMPSAC 2009, pp. 99–106 (2009)Google Scholar
  14. 14.
    Jones, J.A., Harrold, M.J., Bowring, J.F.: Debugging in parallel. In: Proc. of ISSTA 2007, pp. 16–26 (2007)Google Scholar
  15. 15.
    Jones, J.A., Harrold, M.J., Stasko, J.: Visualization of test information to assist fault localization. In: Proc. of ICSE 2002, pp. 467–477 (2002)Google Scholar
  16. 16.
    Khoshgoftaar, T.M., Munson, J.C.: Predicting software development errors using software complexity metrics. IEEE Journal on Selected Areas in Communications 8(2), 253–261 (1990)CrossRefGoogle Scholar
  17. 17.
    Li, Z., Harman, M., Hierons, R.M.: Search algorithms for regression test case prioritization. IEEE TSE 33(4), 225–237 (2007)Google Scholar
  18. 18.
    Liu, C., Yan, X., Fei, L., Han, J., Midkiff, S.P.: Sober: Statistical model-based bug localization. In: Proc. of ESEC/FSE-13, pp. 286–295Google Scholar
  19. 19.
    Matsumoto, S., Kamei, Y., Monden, A., ichi Matsumoto, K., Nakamura, M.: An analysis of developer metrics for fault prediction. In: Proc. of PROMISE 2010, pp. 18:1–18:9 (2010)Google Scholar
  20. 20.
    Mayer, W., Stumptner, M.: Evaluating models for model-based debugging. In: Proc. of ASE 2008, pp. 128–137 (2008)Google Scholar
  21. 21.
    Musa, J., Iannino, A., Okumoto, K.: A time dependent error detection model for software reliability and other performance measures. IEEE Trans. Reliability 28(3), 206–211 (1979)Google Scholar
  22. 22.
    Shakeri, M., Raghavan, V., Pattipati, K.R., Patterson-Hine, A.: Sequential testing algorithms for multiple fault diagnosis. IEEE TSMC 30(1), 1–14 (2000)Google Scholar
  23. 23.
    Smith, A.M., Kapfhammer, G.M.: An empirical study of incorporating cost into test suite reduction and prioritization. In: Proc. SAC 2009, pp. 461–467 (2009)Google Scholar
  24. 24.
    Veevers, A., Marshall, A.C.: A relationship between software coverage metrics and reliability. STVR 4(1), 3–8 (1994)Google Scholar
  25. 25.
    Voas, J.M., Miller, K.W.: Software testability: The new verification. IEEE Software 12(3), 17–28 (1995)CrossRefGoogle Scholar
  26. 26.
    Wood, A.: Predicting software reliability. Computer 29(11), 69–77 (1996)CrossRefGoogle Scholar
  27. 27.
    Wotawa, F., Stumptner, M., Mayer, W.: Model-based debugging or how to diagnose programs automatically. In: Hendtlass, T., Ali, M. (eds.) IEA/AIE 2002. LNCS (LNAI), vol. 2358, pp. 746–757. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  28. 28.
    Zhang, H., Zhang, X., Gu, M.: Predicting defective software components from code complexity measures. In: Proc. of PRDC 2007, pp. 93–96 (2007)Google Scholar
  29. 29.
    Zhang, L., Hou, S.-S., Guo, C., Xie, T., Mei, H.: Time-aware test-case prioritization using integer linear programming. In: Proc. of ISSTA 2009, pp. 213–224 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Rui Abreu
    • 1
    • 2
  • Alberto Gonzalez-Sanchez
    • 3
  • Arjan J. C. van Gemund
    • 3
  1. 1.Dept. of Informatics EngineeringUniversity of PortoPortugal
  2. 2.School of Computer ScienceCarnegie Mellon UniversityUSA
  3. 3.Software Technology Dept.Delft University of Tech.The Netherlands

Personalised recommendations