Empirical Software Engineering

, Volume 18, Issue 3, pp 435–477

Predicting the Flow of Defect Correction Effort using a Bayesian Network Model

  • Thomas Schulz
  • Łukasz Radliński
  • Thomas Gorges
  • Wolfgang Rosenstiel
Article

Abstract

The number of defects alone does not provide software companies with enough information on the effort required to fix them. Defects have different impacts on the overall defect correction effort – defects introduced in one phase may be found and corrected in the same or later phase. The later they are found, the more effort is required to correct them. The main aim of this paper is to build and validate a model (Bayesian Network) for predicting the defect correction effort at various phases of the software development process. The procedure of building the model covers the following steps: problem analysis, data analysis, model definition and enhancement, simulation runs, and model validation. Developed Defect Cost Flow Model (DCFM), which is an implementation of the V-model of a software project lifecycle, correctly incorporates known qualitative and quantitative relationships. Application of DCFM in a real industrial process revealed its high potential in finding the appropriate amount of review effort for specific development phases to minimize the overall costs. The model may be used in the industry for decision support. It can be extended and calibrated to meet the needs of specific development environment.

Keywords

Bayesian Network Correction effort Decision support Defect flow Process modeling Software process 

Abbreviations

BN

Bayesian Network

CPT

Conditional Probability Table

DCE

Defect Correction Effort

DCF

Defect Cost Factor

DCFM

Defect Cost Flow Model

DE

Design Stage

IM

Implementation Stage

I&T

Integration and Testing Stage

KPI

Key Performance Indicator

QA

Quality Assurance

RE

Requirements Engineering Stage

References

  1. Abouelela M, Benedicenti L (2010) Bayesian Network Based XP Process Modelling. Int J Softw Eng Appl 1:1–15CrossRefGoogle Scholar
  2. AgenaRisk BN Tool (2009) Agena, http://www.agenarisk.com/
  3. Antoniol G, Di Penta M, Harman M (2011) The use of Search-Based Optimization Techniques to Schedule and Staff Software Projects: an Approach and an Empirical Study. Softw Pract Exp (in press)Google Scholar
  4. Basili VR, Rombach HD (1988) The TAME project: Towards improvement-oriented software environments. IEEE Trans Softw Eng 14:758–773CrossRefGoogle Scholar
  5. Bibi S, Stamelos I (2004) Software Process Modeling with Bayesian Belief Networks. Proc. Int, Software Metrics Symposium, ChicagoGoogle Scholar
  6. Boetticher G, Menzies T, Ostrand T (2007) PROMISE Repository of empirical software engineering data. West Virginia University, Department of Computer Science, http://promisedata.org/?p=224
  7. Chillarege R, Bhandari IS, Chaar JK, Halliday MJ, Moebus DS, Ray BK, Wong MY (1992) Orthogonal Defect Classification-A Concept for In-Process Measurements. IEEE Trans Softw Eng 18:943–956CrossRefGoogle Scholar
  8. CMMI Product Team (2010) CMMI for Development, Version 1.3. Technical Report CMU/SEI-2010-TR-033. Software Engineering Institute (SEI)Google Scholar
  9. Dabney JB, Barber G, Ohi D (2006) Predicting Software Defect Function Point Ratios Using a Bayesian Belief Network. Proc. 2nd Int. Workshop on Predictor Models in Software Engineering, PhiladelphiaGoogle Scholar
  10. Das B (2004) Generating Conditional Probabilities for Bayesian Networks: Easing the Knowledge Acquisition Problem, http://www.citebase.org/cgi-bin/citations?id=oai:arXiv.org:cs/0411034
  11. De Melo ACV, Sanchez AJ (2008) Software maintenance project delays prediction using Bayesian Networks. Exp Syst Appl 34:908–919CrossRefGoogle Scholar
  12. Del Salgado MJ, del Aguina Cano IM (2008) A Bayesian Network for Predicting the Need for a Requirements Review. In: Meziane F, Vadera S (eds) Artificial Intelligence Applications for Improved Software Engineering Development: New Prospects. Information Science Reference, New York, pp 106–128Google Scholar
  13. Durillo JJ, Zhang YY, Alba E, Nebro AJ (2009) A Study of the Multi-objective Next Release Problem. Proc. 2009 1st International Symposium on Search Based Software Engineering (SSBSE '09). IEEE CS, Washington, DC, pp 49–58CrossRefGoogle Scholar
  14. Fenton N, Hearty P, Neil M, Radliński Ł (2008a) Software Project and Quality Modelling Using Bayesian Networks”. In: Meziane F, Vadera S (eds) Artificial Intelligence Applications for Improved Software Engineering Development: New Prospects. Information Science Reference, New York, pp 1–25Google Scholar
  15. Fenton N, Marsh W, Neil M, Cates P, Forey S, Tailor M (2004) Making Resource Decisions for Software Projects. Proc. 26th Int. Conference on Software Engineering, Washington DC, pp 397–406Google Scholar
  16. Fenton N, Neil M, Marsh W, Hearty P, Radliński Ł, Krause P (2008b) On the effectiveness of early life cycle defect prediction with Bayesian Nets. Empir Softw Eng 13:499–537CrossRefGoogle Scholar
  17. Fenton NE, Neil M, Caballero JG (2007a) Using Ranked Nodes to Model Qualitative Judgments in Bayesian Networks. IEEE Trans Knowl Data Eng 19:1420–1432CrossRefGoogle Scholar
  18. Fenton NE, Neil M, Marsh W, Krause P, Mishra R (2007b) Predicting Software Defects in Varying Development Lifecycles using Bayesian Nets. Inf Softw Technol 43:32–43CrossRefGoogle Scholar
  19. Finkelstein A, Harman M, Mansouri S, Ren J, Zhang YY (2009) A search based approach to fairness analysis in requirement assignments to aid negotiation, mediation and decision making. Requir Eng 14:231–245CrossRefGoogle Scholar
  20. Gueorguiev S, Harman M, Antoniol G (2009) Software project planning for robustness and completion time in the presence of uncertainty using multi objective search based software engineering. Proc. 11th Annual conference on Genetic and evolutionary computation (GECCO '09). ACM, New York, pp 1673–1680Google Scholar
  21. Hearty P, Fenton N, Marquez D, Neil M (2009) Predicting Project Velocity in XP using a Learning Dynamic Bayesian Network Model. IEEE Trans Softw Eng 37:124–137CrossRefGoogle Scholar
  22. Helsper EM, van der Gaag LC, Feelders AJ, Loeffen WLA, Geenen PL, Albers ARW (2005) Bringing Order into Bayesian-Network Construction. Proc. 3rd Int. Conf. on Knowledge Capture. Banff, Alberta, pp 121–128Google Scholar
  23. Herbsleb J (2010) MSR: Mining for scientific results?. 2010 7th IEEE Working Conference on Mining Software Repositories. IEEE, Cape Town, pp 1Google Scholar
  24. IABG (1992) The V-Model—General Directive 250. Software Development Standard for the German Federal Armed Forces, Industrieanlagen-Betriebsgesellschaft (IABG), Ottobrunn, GermanyGoogle Scholar
  25. Jensen FV, Nielsen TD (2007) Bayesian Networks and Decision Graphs, 2nd edn. SpringerGoogle Scholar
  26. Jørgensen M, Shepperd M (2007) A Systematic Review of Software Development Cost Estimation Studies. IEEE Trans Softw Eng 33:33–53CrossRefGoogle Scholar
  27. Kjærulff UB, Madsen AL (2008) Bayesian Networks and Influence Diagrams. A Guide to Construction and Analysis. Springer, New YorkMATHGoogle Scholar
  28. Klaes M, Beletski T, Sarishvili A (2007)AP 3.1: Effektivität von QS-Maßnahmen Stand der Wissenschaft. Fraunhofer IESE-Report 096.07/D, KaiserslauternGoogle Scholar
  29. Kraaijeveld P, Druzdzel M, Onisko A, Wasyluk H (2005) GeNIeRate: An Interactive Generator of Diagnostic Bayesian Network Models. Working Notes of the 16th International Workshop on Principles of Diagnosis (DX-05), Monterey, CA, pp 175–180Google Scholar
  30. Kruchten P (1998) The Rational Unified Process: An Introduction. Addison-WesleyGoogle Scholar
  31. Lee I, Leung JYT, Son SH (2007) Handbook of Real-Time and Embedded Systems. Chapman & Hall/CRCGoogle Scholar
  32. Mair C, Kadoda G, Lefley M, Phalp K, Schofield C, Shepperd M, Webster S (2000) An investigation of machine learning based prediction systems. J Syst Softw 53:23–29CrossRefGoogle Scholar
  33. Mendes E, Mosley N (2008) Bayesian Network Models for Web Effort Prediction: A Comparative Study. IEEE Trans Softw Eng 34:723–737CrossRefGoogle Scholar
  34. Nadkarni S, Shenoy PP (2004) A Causal Mapping Approach to Constructing Bayesian Networks. Decis Support Syst 38:259–281CrossRefGoogle Scholar
  35. Neil M, Marquez D, Fenton NE (2010) Improved Reliability Modeling using Bayesian Networks and Dynamic Discretization. Reliab Eng Syst Saf 95:412–425CrossRefGoogle Scholar
  36. Neil M, Tailor M, Marquez D (2007) Inference in Hybrid Bayesian Networks using Dynamic Discretisation. Stat Comput 17:219–233MathSciNetCrossRefGoogle Scholar
  37. O’Hagan A, Buck CE, Daneshkhah A, Eiser JE, Garthwaite PH, Jenkinson DJ, Oakley JE, Rakow T (2006) Uncertain Judgements: Eliciting Expert Probabilities. John Wiley and Sons, ChichesterMATHCrossRefGoogle Scholar
  38. Pearl J (1985) Bayesian Networks: A Model of Self-Activated Memory for Evidential Reasoning. Proc. 7th Conf. of the Cognitive Science Society. University of California, pp 329–334Google Scholar
  39. Pearl J (1988) Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San FranciscoGoogle Scholar
  40. Pendharkar PC, Subramanian GH, Rodger JA (2005) A Probabilistic Model for Predicting Software Development Effort. IEEE Trans Softw Eng 31:615–624CrossRefGoogle Scholar
  41. Pfautz J, Cox Z, Catto G, Koelle D, Campolongo J, Roth E (2007) User-Centered Methods for Rapid Creation and Validation of Bayesian Belief Networks. Proc. 5th Bayesian Modeling Applications Workshop, 23rd Annual Conference on Uncertainty in Artificial Intelligence. Vancouver, British ColumbiaGoogle Scholar
  42. Radlinski L (2008) Improved Software Project Risk Assessment Using Bayesian Nets. Ph.D. Thesis. Queen Mary University, LondonGoogle Scholar
  43. Radlinski L (2010) A Survey of Bayesian Net Models for Software Development Effort Prediction. Int J Softw Eng Comput 2:95–109Google Scholar
  44. Radliński Ł (2010) Software Development Effort and Quality Prediction Using Bayesian Nets and Small Local Qualitative Data. Proc. 22nd International Conference on Software Engineering and Knowledge Engineering. Redwood City, CA, pp. 113–116Google Scholar
  45. Radliński Ł, Fenton N, Neil M (2008) A Learning Bayesian Net for Predicting Number of Software Defects Found in a Sequence of Testing. Pol J Env Stud 17:359–364Google Scholar
  46. Radliński Ł, Fenton N, Neil M, Marquez D (2007) Improved Decision-Making for Software Managers Using Bayesian Networks. Proc. 11th IASTED Int. Conf. Software Engineering and Applications. IASTED, Cambridge, pp 13–19Google Scholar
  47. Schulz T, Radliński LŁ, Gorges T, Rosenstiel W (2010) Defect Cost Flow Model—A Bayesian Network for Predicting Defect Correction Effort. Proc. 6th International Conference on Predictive Models in Software Engineering. ACM, New YorkGoogle Scholar
  48. Schulz T, Radliński Ł, Gorges T, Rosenstiel W (2011) Software Process Model using Dynamic Bayesian Networks. In: Ramachandran M (ed) Knowledge Engineering for Software Development Life Cycles: Support Technologies and Applications. Information Science Reference, Hershey, pp 289–310CrossRefGoogle Scholar
  49. Skaanning C (2000) A Knowledge Acquisition Tool for Bayesian-Network Troubleshooters. Proc. 16th Conference on Uncertainty in Artificial Intelligence. Stanford University, Stanford, CAGoogle Scholar
  50. Song Q, Jia Z, Shepperd M, Ying S, Liu J (2011) A General Software Defect-Proneness Prediction Framework. IEEE Trans Softw Eng. doi:10.1109/TSE.2010.90
  51. Stamelos I, Angelis L, Dimou P, Sakellaris E (2003) On the use of Bayesian belief networks for the prediction of software productivity. Inf Softw Technol 45:51–60CrossRefGoogle Scholar
  52. Stewart B (2002) Predicting project delivery rates using the Naive–Bayes classifier. J Softw Maint Evol Res Pract 14:161–179MATHCrossRefGoogle Scholar
  53. Stolz W, Wagner P (2004) Introduction of a quantitative Quality Management for the ECU Software development at Gasoline Systems. Proceedings of the 2nd Bosch Conference on Software Engineering (BoCSE-2004), Ludwigsburg, Germany, pp 25–27Google Scholar
  54. Torkar R, Adnan NM, Alvi AK, Afzal W (2010) Predicting software test effort in iterative development using a dynamic Bayesian network. Proc. of 21st IEEE International Symposium on Software Reliability Engineering, Industry Practice Track. San JoseGoogle Scholar
  55. Van Koten C, Gray AR (2006) An application of Bayesian network for predicting object-oriented software maintainability. Inf Softw Technol 48:59–67CrossRefGoogle Scholar
  56. Wagner S (2009) A Bayesian network approach to assess and predict software quality using activity-based quality models. Proc. 5th Int. Conf. on Predictor Models in Software Engineering. ACM Press, New YorkGoogle Scholar
  57. Wang H, Peng F, Zhang C, Pietschker A (2006) Software Project Level Estimation Model Framework based on Bayesian Belief Networks. Proc 6th Int. Conf. on Quality Software. IEEE CS, Washington, DC, pp 209–218Google Scholar
  58. Wang X, Wu C, Ma L (2010) Software project schedule variance prediction using Bayesian Network. Proc. 2010 IEEE International Conference on Advanced Management Science. Chengdu, pp 26–30Google Scholar
  59. Weiss SM (1984) A Practical Guide to Designing Expert Systems. Rowman & Littlefield PublishersGoogle Scholar
  60. Wiegmann DA (2005) Developing a Methodology for Eliciting Subjective Probability Estimates During Expert Evaluations of Safety Interventions: Application for Bayesian Belief Networks. Rep. No. AHFD-05-13/NASA-05-4). Aviation Human Factors Division Institute of Aviation, University of IllinoisGoogle Scholar
  61. Winkler RL (2003) An introduction to Bayesian inference and decision, 2nd edn. Probabilistic Publishing, GainesvilleGoogle Scholar
  62. Wooff DA, Goldstein M, Coolen FPA (2002) Bayesian Graphical Models for Software Testing. IEEE Trans Softw Eng 28:510–525CrossRefGoogle Scholar
  63. Zhang D, Tsai JJP (2003) Machine Learning and Software Engineering. Softw Qual J 11:87–119CrossRefGoogle Scholar
  64. Zhou Y, Würsch M, Giger E, Gall HC, Lü J (2008) A Bayesian Network Based Approach for Change Coupling Prediction. Proc. 15th Working Conference on Reverse Engineering. IEEE CS, Washington DC, pp 27–36Google Scholar
  65. Zimmermann K, Hauser HM (2004) The growing importance of embedded software: Managing hybrid hardware-software business. Technical Report. The Boston Consulting GroupGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Thomas Schulz
    • 1
  • Łukasz Radliński
    • 2
  • Thomas Gorges
    • 3
  • Wolfgang Rosenstiel
    • 4
  1. 1.Robert Bosch GmbH, Car Multimedia (CM-IS/EBM1)LeonbergGermany
  2. 2.Department of Information Systems EngineeringUniversity of SzczecinSzczecinPoland
  3. 3.Robert Bosch GmbHLeonbergGermany
  4. 4.Eberhard Karls Universität TübingenTübingenGermany

Personalised recommendations