Effects of Test-Driven Development: A Comparative Analysis of Empirical Studies

  • Simo Mäkinen
  • Jürgen Münch
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 166)


Test-driven development is a software development practice where small sections of test code are used to direct the development of program units. Writing test code prior to the production code promises several positive effects on the development process itself and on associated products and processes as well. However, there are few comparative studies on the effects of test-driven development. Thus, it is difficult to assess the potential process and product effects when applying test-driven development. In order to get an overview of the observed effects of test-driven development, an in-depth review of existing empirical studies was carried out. The results for ten different internal and external quality attributes indicate that test-driven development can reduce the amount of introduced defects and lead to more maintainable code. Parts of the implemented code may also be somewhat smaller in size and complexity. While maintenance of test-driven code can take less time, initial development may last longer. Besides the comparative analysis, this article sketches related work and gives an outlook on future research.


test-driven development test-first programming software testing software verification software engineering empirical study 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Beck, K.: Test-Driven Development: By Example. Addison-Wesley (2003)Google Scholar
  2. 2.
    Turhan, B., Layman, L., Diep, M., Erdogmus, H., Shull, F.: How Effective is Test-Driven Development. In: Oram, A., Wilson, G. (eds.) Making Software: What Really Works, and Why We Believe It, pp. 207–219. O’Reilly (2010)Google Scholar
  3. 3.
    Jeffries, R., Melnik, G.: Guest Editors’ Introduction: TDD–The Art of Fearless Programming. IEEE Software 24(3), 24–30 (2007)CrossRefGoogle Scholar
  4. 4.
    Rafique, Y., Misic, V.: The Effects of Test-Driven Development on External Quality and Productivity: A Meta Analysis. IEEE Transactions on Software Engineering 39(6), 835–856 (2013)CrossRefGoogle Scholar
  5. 5.
    Desai, C., Janzen, D., Savage, K.: A Survey of Evidence for Test-Driven Development in Academia. SIGCSE Bulletin 40(2), 97–101 (2008)CrossRefGoogle Scholar
  6. 6.
    Runeson, P., Höst, M.: Guidelines for Conducting and Reporting Case Study Research in Software Engineering. Empirical Software Engineering 14, 131–164 (2009)CrossRefGoogle Scholar
  7. 7.
    Merriam, S.B.: Qualitative Research: A Guide to Design and Implementation. John Wiley & Sons (2009)Google Scholar
  8. 8.
    Torraco, R.J.: Writing Integrative Literature Reviews: Guidelines and Examples. Human Resource Development Review 4(3), 356–367 (2005)CrossRefGoogle Scholar
  9. 9.
    Kitchenham, B., Charters, S.: Guidelines for Performing Systematic Literature Reviews in Software Engineering. Technical Report Version 2.3, Keele University and University of Durham (July 2007)Google Scholar
  10. 10.
    Creswell, J.W.: Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Sage Publications (2009)Google Scholar
  11. 11.
    Mäkinen, S.: Driving Software Quality and Structuring Work Through Test-Driven Development. Master’s thesis, Department of Computer Science, University of Helsinki (October 2012)Google Scholar
  12. 12.
    Bhat, T., Nagappan, N.: Evaluating the Efficacy of Test-driven Development: Industrial Case Studies. In: Proceedings of the 2006 ACM/IEEE International Symposium on Empirical Software Engineering, ISESE 2006, pp. 356–363. ACM, New York (2006)Google Scholar
  13. 13.
    Canfora, G., Cimitile, A., Garcia, F., Piattini, M., Visaggio, C.A.: Evaluating Advantages of Test Driven Development: A Controlled Experiment with Professionals. In: Proceedings of the 2006 ACM/IEEE International Symposium on Empirical Software Engineering, ISESE 2006, pp. 364–371. ACM, New York (2006)Google Scholar
  14. 14.
    Dogša, T., Batič, D.: The Effectiveness of Test-Driven Development: An Industrial Case Study. Software Quality Journal 19, 643–661 (2011)CrossRefGoogle Scholar
  15. 15.
    George, B., Williams, L.: An Initial Investigation of Test Driven Development in Industry. In: Proceedings of the 2003 ACM Symposium on Applied Computing, SAC 2003, pp. 1135–1139. ACM, New York (2003)Google Scholar
  16. 16.
    Geras, A., Smith, M., Miller, J.: A Prototype Empirical Evaluation of Test Driven Development. In: Proceedings of the 10th International Symposium on Software Metrics. METRICS 2004, pp. 405–416 (September 2004)Google Scholar
  17. 17.
    Maximilien, E., Williams, L.: Assessing Test-Driven Development at IBM. In: Proceedings of the 25th International Conference on Software Engineering, ICSE 2003, pp. 564–569 (May 2003)Google Scholar
  18. 18.
    Nagappan, N., Maximilien, E., Bhat, T., Williams, L.: Realizing Quality Improvement through Test Driven Development: Results and Experiences of Four Industrial Teams. Empirical Software Engineering 13, 289–302 (2008)CrossRefGoogle Scholar
  19. 19.
    Williams, L., Maximilien, E.M., Vouk, M.: Test-Driven Development as a Defect-Reduction Practice. In: Proceedings of the 14th International Symposium on Software Reliability Engineering, ISSRE 2003, pp. 34–45 (November 2003)Google Scholar
  20. 20.
    Janzen, D.S., Saiedian, H.: Does Test-Driven Development Really Improve Software Design Quality? IEEE Software 25(2), 77–84 (2008)CrossRefGoogle Scholar
  21. 21.
    Madeyski, L., Szała, Ł.: The Impact of Test-Driven Development on Software Development Productivity — An Empirical Study. In: Abrahamsson, P., Baddoo, N., Margaria, T., Messnarz, R. (eds.) EuroSPI 2007. LNCS, vol. 4764, pp. 200–211. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  22. 22.
    Müller, M., Höfer, A.: The Effect of Experience on the Test-Driven Development Process. Empirical Software Engineering 12(6), 593–615 (2007)CrossRefGoogle Scholar
  23. 23.
    Desai, C., Janzen, D.S., Clements, J.: Implications of Integrating Test-Driven Development Into CS1/CS2 Curricula. In: Proceedings of the 40th ACM Technical Symposium on Computer Science Education, SIGCSE 2009, pp. 148–152. ACM, New York (2009)Google Scholar
  24. 24.
    Gupta, A., Jalote, P.: An Experimental Evaluation of the Effectiveness and Efficiency of the Test Driven Development. In: Proceedings of the First International Symposium on Empirical Software Engineering and Measurement, ESEM 2007, pp. 285–294 (September 2007)Google Scholar
  25. 25.
    Huang, L., Holcombe, M.: Empirical Investigation Towards the Effectiveness of Test First Programming. Information and Software Technology 51(1), 182–194 (2009)CrossRefGoogle Scholar
  26. 26.
    Janzen, D., Saiedian, H.: On the Influence of Test-Driven Development on Software Design. In: Proceedings of the 19th Conference on Software Engineering Education and Training, CSEET 2006, pp. 141–148 (April 2006)Google Scholar
  27. 27.
    Madeyski, L.: The Impact of Test-First Programming on Branch Coverage and Mutation Score Indicator of Unit Tests: An Experiment. Information and Software Technology 52(2), 169–184 (2010)CrossRefGoogle Scholar
  28. 28.
    Pančur, M., Ciglarič, M.: Impact of Test-Driven Development on Productivity, Code and Tests: A Controlled Experiment. Information and Software Technology 53(6), 557–573 (2011)CrossRefGoogle Scholar
  29. 29.
    Vu, J., Frojd, N., Shenkel-Therolf, C., Janzen, D.: Evaluating Test-Driven Development in an Industry-Sponsored Capstone Project. In: Proceedings of the Sixth International Conference on Information Technology, ITNG 2009, pp. 229–234. New Generations (April 2009)Google Scholar
  30. 30.
    Wilkerson, J., Nunamaker, J.J., Mercer, R.: Comparing the Defect Reduction Benefits of Code Inspection and Test-Driven Development. IEEE Transactions on Software Engineering 38(3), 547–560 (2012)CrossRefGoogle Scholar
  31. 31.
    Fenton, N.E., Pfleeger, S.L.: Software Metrics: A Rigorous and Practical Approach. PWS Publishing Company, Boston (1997)Google Scholar
  32. 32.
    Pezzé, M., Young, M.: Software Testing and Analysis: Process, Principles and Techniques. Wiley, Chichester (2008)zbMATHGoogle Scholar
  33. 33.
    McCabe, T.: A Complexity Measure. IEEE Transactions on Software Engineering SE 2(4), 308–320 (1976)MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Chidamber, S., Kemerer, C.: A Metrics Suite for Object Oriented Design. IEEE Transactions on Software Engineering 20(6), 476–493 (1994)CrossRefGoogle Scholar
  35. 35.
    Cook, S., He, J., Harrison, R.: Dynamic and Static Views of Software Evolution. In: Proceedings of the IEEE International Conference on Software Maintenance, pp. 592–601 (2001)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Simo Mäkinen
    • 1
  • Jürgen Münch
    • 1
  1. 1.Department of Computer ScienceUniversity of HelsinkiFinland

Personalised recommendations