Empirical Software Engineering

, Volume 12, Issue 6, pp 593–615 | Cite as

The effect of experience on the test-driven development process

  • Matthias M. Müller
  • Andreas Höfer


We conducted a quasi-experiment to compare the characteristics of experts’ and novices’ test-driven development processes. Our novices were 11 computers science students who participated in an Extreme Programming lab course, the expert group consisted of seven professionals who had industrial experience in test-driven development. The novices as well as two of the experts worked in a laboratory environment whereas the remaining five experts worked in their office. The experts complied more to the rules of test-driven development and had shorter test-cycles than the novices. The tests written by the experts were of higher quality in terms of statement and block coverage as well. All reported results are statistically significant on the 5% level. We conclude that the results of studies which evaluate performance of test-driven development using subjects inexperienced in TDD are not easily generalisable.


Test-driven development Process Quasi-experiment Experts Novices 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Beck K (2002) Test driven development: by example. Addison-WesleyGoogle Scholar
  2. Bhat T, Nagappan N (2006) Evaluating the efficacy of test-driven development: industrial case studies. In: Proceedings of the 2006 international symposium on empirical software engineering (ISESE’06), Rio de Janeiro, Brazil, September 2006. ACM Press, pp 356–363Google Scholar
  3. Canfora G, Cimitile A, Garcia F, Piattini M, Visaggio CA (2006) Evaluating advantages of test driven development: a controlled experiment with professionals. In: Proceedings of the 2006 international symposium on empirical software engineering (ISESE’06), Rio de Janeiro, Brazil, September 2006. ACM Press, pp 364–371Google Scholar
  4. Cohen J (1988) Statistical power analysis for the behavioral sciences. AcademicGoogle Scholar
  5. Erdogmus H, Morisio M, Torchiano M (2005) On the effectiveness of the test-first approach to programming. IEEE Trans Software Eng 31(3):226–237, MarchCrossRefGoogle Scholar
  6. Erdogmus H, Wang Y (2004) The role of process measurement in test-driven development. In: Proceeding of XP Agile Universe 2004: 4th conference on extreme programming and agile methods, Calgary, Canada, AugustGoogle Scholar
  7. Fowler M (1999) Refactoring: improving the design of existing code. Addison-WesleyGoogle Scholar
  8. George B, Williams L (2003) An initial investigation of test driven development in industry. In: ACM symposium on applied computing, Melbourne, FL, USA, pp 1135–1139Google Scholar
  9. Geras A, Smith M, Miller J (2004) A prototype empirical evaluation of test driven development. In: International symposium on software metrics (Metrics), Chicago, IL, USA, pp 405–416, SeptemberGoogle Scholar
  10. Hollander M, Wolfe D (1999) Nonparametric statistical methods. Wiley, 2nd ednGoogle Scholar
  11. Link J (2003) Unit testing in Java: how tests drive the code. Morgan KaufmannGoogle Scholar
  12. Link J (2005) Softwaretest mit JUnit: techniken der testgetriebenen Entwicklung. dpunkt.verlagGoogle Scholar
  13. Mackinnon T, Freeman S, Craig P (2000) Endo-testing: unit testing with mock objects. In: Extreme programming and flexible processes in software engineering—XP2000, Cagliari, Sardinia, Italy, JuneGoogle Scholar
  14. Malpohl G, Hunt J, Tichy W (2000) Renaming detection. In: Automated software engineering, Grenoble, France, pp 73–80, SeptemberGoogle Scholar
  15. Müller M, Hagner O (2002) Experiment about test-first programming. IEE Proc Software 149(5):131–136, OctoberCrossRefGoogle Scholar
  16. Müller M, Link J, Sand R, Malpohl G (2004) Extreme programming in curriculum: experiences from academia and industry. In: Conference on extreme programming and agile processes in software engineering (XP2004), Garmisch-Partenkirchen, Germany, pp 294–302, JuneGoogle Scholar
  17. Pančur M, Ciglarič M, Trampuš M, Vidmar T (2003) Towards empirical evaluation of test-driven development in a university environment. In: EUROCON 2003. Computer as a Tool. The IEEE Region 8, vol 2, pp 83–86, SeptemberGoogle Scholar
  18. Shadish W, Cook T, Campbell D (2002) Experimental and quasi-experimental designs for generalized causal inference. Houghton MifflinGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  1. 1.Systeme Infrastruktur Support GmbH, EnBW AGKarlsruheGermany
  2. 2.Fakultät für InformatikUniversität KarlsruheKarlsruheGermany

Personalised recommendations