Skip to main content
SpringerLink
Log in

Neural network models for software development effort estimation: a comparative study

  • Predictive Analytics Using Machine Learning
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Software development effort estimation (SDEE) is one of the main tasks in software project management. It is crucial for a project manager to efficiently predict the effort or cost of a software project in a bidding process, since overestimation will lead to bidding loss and underestimation will cause the company to lose money. Several SDEE models exist; machine learning models, especially neural network models, are among the most prominent in the field. In this study, four different neural network models—multilayer perceptron, general regression neural network, radial basis function neural network, and cascade correlation neural network—are compared with each other based on: (1) predictive accuracy centred on the mean absolute error criterion, (2) whether such a model tends to overestimate or underestimate, and (3) how each model classifies the importance of its inputs. Industrial datasets from the International Software Benchmarking Standards Group (ISBSG) are used to train and validate the four models. The main ISBSG dataset was filtered and then divided into five datasets based on the productivity value of each project. Results show that the four models tend to overestimate in 80 % of the datasets, and the significance of the model inputs varies based on the selected model. Furthermore, the cascade correlation neural network outperforms the other three models in the majority of the datasets constructed on the mean absolute residual criterion.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Notes

  1. The terms software cost and software effort are used interchangeably in this study.

  2. The terms software cost estimation and software cost prediction are used interchangeably in this study.

References

  1. Eck D, Brundick B, Fettig T, Dechoretz J, Ugljesa J (2009) Parametric estimating handbook. In: The International Society of Parametric Analysis (ISPA), 4th edn

  2. Lynch J (2009) Chaos manifesto. The Standish Group, Boston [Online]. http://www.standishgroup.com/newsroom/chaos_2009.php

  3. Jorgensen M, Shepperd M (2007) A systematic review of software development cost estimation studies. IEEE Trans Softw Eng 33(1):33–53

    Article  Google Scholar 

  4. Mendes E, Mosley N, Watson I (2002) A comparison of case-based reasoning approaches. In: Proceedings of the 11th international conference on world wide web, Honolulu, Hawaii, USA, pp 272–280

  5. Jørgensen M (2007) Forecasting of software development work effort: evidence on expert judgment and formal models. Int J Forecast 23(3):449–462

    Article  Google Scholar 

  6. Boehm BW (1981) Software engineering economics. Prentice-Hall, Englewood Cliffs, NJ

    MATH  Google Scholar 

  7. Galorath DD, Evans MW (2006) Software sizing, estimation, and risk management. Auerbach Publications, Boston, MA

    Book  MATH  Google Scholar 

  8. Putnam LH (1978) A general empirical solution to the macro software sizing and estimating problem. IEEE Trans Softw Eng 4(4):345–361

    Article  MATH  Google Scholar 

  9. Lopez-Martin C (2011) A fuzzy logic model for predicting the development effort of short scale programs based upon two independent variables. Appl Soft Comput 11(1):724–732

    Article  MathSciNet  Google Scholar 

  10. Nassif AB, Ho D, Capretz LF (2013) Towards an early software estimation using log-linear regression and a multilayer perceptron model. J Syst Softw 86(1):144–160

    Article  Google Scholar 

  11. Nassif AB, Capretz LF, Ho D (2012) Estimating software effort using an ANN model based on use case points. In: 11th International conference on machine learning and applications (ICMLA 2012), Boca Raton, FL, pp 42–47

  12. Nassif AB, Capretz LF, Ho D (2011) Estimating software effort based on use case point model using sugeno fuzzy inference system. In: 23rd IEEE international conference on tools with artificial intelligence, FL, pp 393–398

  13. Nassif AB, Capretz LF, Ho D (2011) A regression model with mamdani fuzzy inference system for early software effort estimation based on use case diagrams. In: Third international conference on intelligent computing and intelligent systems, Guangzhou, Guangdong, pp 615–620

  14. Idri A, Abran A (2000) COCOMO cost model using fuzzy logic. In: 7th International conference on fuzzy theory and technology, pp 1–4

  15. Huang X, Ho D, Ren J, Capretz LF (2007) Improving the COCOMO model using a neuro-fuzzy approach. Appl Soft Comput 7(1):29–40

    Article  Google Scholar 

  16. Du WL, Capretz LF, Nassif AB, Ho D (2013) A hybrid intelligent model for software cost estimation. J Comput Sci 9(11):1506–1513

    Article  Google Scholar 

  17. Boehm B, Abts C, Chulani S (2000) Software development cost estimation approaches: a survey. Ann Softw Eng 10(1):177–205

    Article  MATH  Google Scholar 

  18. Wen J, Li S, Lin Z, Hu Y, Huang C (2012) Systematic literature review of machine learning based software development effort estimation models. Inf Softw Technol 54(1):41–59

    Article  Google Scholar 

  19. Lopez-Martin C (2011) Applying a general regression neural network for predicting development effort of short-scale programs. Neural Comput Appl 20(3):389–401

    Article  Google Scholar 

  20. Lopez-Martin C (2015) Predictive accuracy comparison between neural networks and statistical regression for development effort of software projects. J Syst Softw 27:434–449

    Google Scholar 

  21. Nassif AB, Capretz LF, Ho D (2012) Software effort estimation in the early stages of the software life cycle using a cascade correlation neural network model. In 13th ACIS international conference on software engineering, artificial intelligence, networking and parallel & distributed computing (SNPD), Kyoto, Japan, pp 589–594

  22. ISBSG. International Software Benchmarking Standards Group. [Online]. http://www.isbsg.org/

  23. Shepperd M, MacDonell S (2012) Evaluating prediction systems in software project estimation. Inf Softw Technol 54(8):820–827

    Article  Google Scholar 

  24. Kassab M (2009) Non-functional requirements: modeling and assessment. VDM Verlag, Germany

  25. Kassab M, Ormandjieva O, Daneva M, Abran A (2008) Non-functional requirements size measurement method (NFSM) with COSMIC-FFP. Softw Process Prod Meas 4895:168–182

    Article  Google Scholar 

  26. Lippman RP (1987) An introduction to computing with neural nets. IEEE ASSP Mag 3(2):4–22

    Article  MathSciNet  Google Scholar 

  27. Møller MF (1993) A scaled conjugate gradient algorithm for fast supervised learning. Neural Netw 6(4):525–533

    Article  Google Scholar 

  28. Park H, Baek S (2008) An empirical validation of a neural network model for software effort estimation. Expert Syst Appl 35(3):929–937

    Article  Google Scholar 

  29. Broomhead DS, Lowe D (1988) Multivariable functional interpolation and adaptive networks. Complex Syst 2:321–355

    MathSciNet  MATH  Google Scholar 

  30. Kitchenham B, Mendes E (2009) Why comparative effort prediction studies may be invalid. In: Proceedings of the 5th international conference on predictor models in software engineering, Vancouver, BC, pp 4:1–4:5

  31. Specht DF (1991) A general regression neural network. IEEE Trans Neural Netw 2(6):568–576

    Article  Google Scholar 

  32. Lopez-Martin C, Isaza C, Chavoya A (2011) Software development effort prediction of industrial projects applying a general regression neural network. Empir Softw Eng 17:1–19

    Google Scholar 

  33. Nassif AB (2012) Software size and effort estimation from use case diagrams using regression and soft computing models. Western University, ON, London

  34. Fahlman SE, Lebiere C (1990) The cascade-correlation learning architecture. Adv Neural Inf Process Syst 2:524–532

    Google Scholar 

  35. Chen S, Hong X, Harris CJ (2005) Orthogonal forward selection for constructing the radial basis function network with tunable nodes. In IEEE international conference on intelligent computing, pp 777–786

  36. Kitchenham BA, Mendes E, Travassos GH (2007) Cross versus within-company cost estimation studies: a systematic review. IEEE Trans Softw Eng 33(5):316–329

    Article  Google Scholar 

Download references

Acknowledgments

Ali Bou Nassif would like to thank the University of Sharjah for supporting this research. Luiz Fernando Capretz, and Danny Ho would like to thank the Natural Sciences and Engineering Research Council of Canada (NSERC) for their support of this work through a Discovery Grant-Team.

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Sharjah, Sharjah, UAE

    Ali Bou Nassif

  2. Department of Software Engineering, Applied Science University, PO BOX 166, Amman, Jordan

    Mohammad Azzeh

  3. Department of Electrical and Computer Engineering, University of Western Ontario, London, ON, Canada

    Luiz Fernando Capretz

  4. NFA Estimation Inc., Richmond Hill, ON, Canada

    Danny Ho

Authors
  1. Ali Bou Nassif
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad Azzeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luiz Fernando Capretz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Danny Ho
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ali Bou Nassif.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Informed consent

This study does not involve any human participants and/or animals.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nassif, A.B., Azzeh, M., Capretz, L.F. et al. Neural network models for software development effort estimation: a comparative study. Neural Comput & Applic 27, 2369–2381 (2016). https://doi.org/10.1007/s00521-015-2127-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-015-2127-1

Keywords

Search

Navigation