Test-driven development (TDD) is a software development practice, where test cases are incrementally written before implementing the production code. This paper presents the results of a multi-case study investigating the effectiveness of TDD within an industrial environment. Three comparable medium-sized projects were observed during their development cycle. Two projects were driven without TDD practice, while the third one introduced TDD into the development process. The effectiveness of TDD was expressed in terms of external code quality, productivity, and maintainability. Our results indicate that the TDD developers produced higher quality code that is easier to maintain, although we did observe a reduction in productivity.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
In incremental development, the releases are defined by beginning with one small, functional subsystem and then adding functionality with each new release (Pflegeer and Atlee 2006).
JUnit—A regression testing framework written by Erich Gamma and Kent Beck, http://www.junit.org.
CCCC—C and C ++ Code Counter http://cccc.sourceforge.net/, a free software tool for the measurement of source code related metrics by Tim Littlefair.
Session Initiation Protocol, Internet Engineering Task Force (IETF), RFC 3261, http://datatracker.ietf.org/doc/rfc3261/).
Primavera Progress Reporter is a web-based program that connects project team members across projects, throughout an enterprise.
ISO/IEC 9,126. (2001). Software engineering—product quality—part 1: Quality model. ISO 2001.
Cyclomatic complexity is a measure of the complexity of code related to the number of ways there are to traverse a piece of code (McCabe 1976).
An unacceptable behavior of the software is defined as a failure. A fault is a defect in the program that, when executed with certain input data, causes a failure.
See Albrecht and Gaffney (1983) for details.
Unit test code was excluded.
Sanchez et. al. 2007 reported the average ratio across all ten releases as 0.61.
Albrecht, A. J., & Gaffney, J. E., Jr. (1983). Software function, source lines of code, and development effort prediction: A software science validation. IEEE Transactions on Software Engineering, SE-9(6), 639–648.
Beck, K. (2001). Aim, fire. IEEE Software, 18(5), 87–89.
Beck, K. (2003). Test driven development: by example. USA: Addison Wesley Professional.
Bhat, T., Nagappan, N. (2006). Evaluating the efficacy of test-driven development: industrial case studies. Proceedings of International Symposium, Empirical Software Engineering (ISESE 06), ACM Press, 356–363.
Crispin, Lisa. (2006). Driving software quality: how test-driven development impacts software quality. IEEE Software, 23(6), 70–71. doi:10.1109/MS.2006.157.
Cusumano, M. A., & Selby, R. W. (1997). How microsoft builds software. Communications of the ACM, 40(6), 53–61.
Damm, L.-O., & Lundberg, L. (2006). Results from introducing component-level test automation and test-driven development. Journal of Systems and Software, 79, 1001–1014. Elsevier Science Inc.
Edwards, S. H. (2003). Using test-driven development in the classroom: providing students with concrete feedback on performance. Proceedings of International Conference on Education and Information Systems: Technologies and Applications (EISTA ‘03).
Erdogmus, H., Morisio, M., & Torchiano, M. (2005). On the effectiveness of the test-first approach to programming. IEEE Transactions on Software Engineering, 31(3), 226–237.
Fenton, N. E., & Ohlsson, N. (2000). Quantitative analysis of faults and failures in a complex software system. IEEE Transactions on Software Engineering, 26(8), 797–814.
Fenton, N. E., & Pfleeger, S. L. (1998). Software metrics: A rigorous & practical approach (2nd ed.). Boston, USA: PWS Publishing Company.
Fowler, M., Beck, K., Brant, J., Opdyke, W., & Roberts, D. (1999). Refactoring: Improving the design of existing code. USA: Addison Wesley.
George, B., Williams, L. (2003). An initial investigation of test driven development in industry. ACM Symposium on Applied Computing (SAC), March.
Janzen, D. S., & Saiedian, H. (2008). Does test-driven development really improve software design quality? IEEE Software, 25(2), 77–84.
Jeffries, R. E. (1999). Extreme Testing. Software Testing and Quality Engineering, 1(2), 23–26, March/April.
Jeffries, R., Melnik, G. (2007). TDD: The art of fearless programming. IEEE Software, May/June, 24–30.
Jovanović, D., Dogša, T. (2003). Comparison of software development models and their usage in computer-telephony systems. In ConTEL 2003: Proceedings of the 7th international conference on telecommunications, University of Zagreb, Faculty of Electrical Engineering and Computing, Zagreb, Croatia (vol. 2, pp. 587–592), June 11–13.
Kitchenham, B., Pickard, L., & Pfleeger, S. L. (1995). Case studies for method and tool evaluation. IEEE Software, 12(4), 52–62.
Madeyski, L. (2005). Preliminary analysis of the effects of pair programming and test-driven development on the external code quality. Frontiers in Artificial Intelligence and Applications.Proceeding of the 2005 conference on Software Engineering, vol. 130, pp. 113–123.
Madeyski, L., Szała, Ł. (2007). The impact of test-driven development on software development productivity—An empirical study. Software Process Improvement 14th European Conference, EuroSPI 2007, Potsdam, Germany, September 26–28, 2007 in Proceedings lecture notes in computer science, Springer Berlin/Heidelberg, vol. 4764.
Malik, S., Palencia, J. R. (1999). Synchronize and Stabilize vs. Open-source–An analysis of 2 team-organization models when developing consumer level shrink-wrapped software. Computer Science 95.314A Research Report, Carleton University, Ottawa, Ontario, Canada.
Martin, R. C., Newkirk, J. W., Kess, R. S. (2002). Agile software development, principles, patterns, and practices (1st ed.). Prentice Hall, Englewood Cliffs.
Maurer, F., Martel, S. (2002). On the productivity of agile software practices: An industrial case study. Technical report, University of Calgary, Department of Computer Science, Calgary, Alberta, Canada.
Maximilien, M., Williams, L. (2003). Assessing test-driven development at IBM. Proceedings of 25th International Conference on Software Engineering (pp. 564–569).
McCabe, T. J. (1976). A complexity measure. IEEE Transactions on Software Engineering, 2(4), 308–320.
Mueller, M. M., & Hagner, O. (2002). Experiment about Test-First Programming. IEE Proceedings–Software, 149(5), 131–136.
Newkirk, J., Vorontsov, A. (2004). Test-driven development in microsoft NET. Microsoft Press.
Pfleeger, S. L., & Atlee, J. M. (2006). Software engineering: Theory and practice. New Jersey, USA: Prentice Hall.
Sanchez, J., Williams, L., Maximilien, E. M. (2007). A Longitudinal Study of the Use of a Test-Driven development Practice in Industry. AGILE 2007 Conference (AGILE 2007), 13–17 August, Washington, DC, USA. IEEE Computer Society 2007, ISBN 0-7695-2872-4.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and Quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin Company.
Siniaalto, M., Abrahamsson, P. (2007). A comparative case study on the impact of test-driven development on program design and test coverage. Proceedings of the First International Symposium on Empirical Software Engineering and Measurement, IEEE Computer Society, Washington, pp. 275–284, ISBN ~ ISSN: 1938–6451.
Stephens, M., Rosenberg, D. (2004). The Irony of Extreme Programming. Dr. Dobb’s Journal, May 2004.
About this article
Cite this article
Dogša, T., Batič, D. The effectiveness of test-driven development: an industrial case study. Software Qual J 19, 643–661 (2011). https://doi.org/10.1007/s11219-011-9130-2
- Test-driven development
- Testing and debugging
- Testing strategies
- Software quality/SQA
- Software engineering process