An Empirical Study on the TDD Conformance of Novice and Expert Pair Programmers

  • Andreas Höfer
  • Marc Philipp
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 31)


We conducted a quasi-experiment comparing the confor- mance to the test-driven development (TDD) process of one expert and two novice groups of programmers working in pairs. Besides an insignificant tendency of the expert group toward a higher TDD conformance and instruction coverage, we found that the expert group had refactored their code to a larger extent than the two novice groups. More surprisingly though, the pairs in the expert group were significantly slower than the pairs in one of the novice groups.


test-driven development pair programming experts and novices quasi-experiment 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Müller, M.M., Hagner, O.: Experiment about test-first programming. In: IEE Proceedings – Software, vol. 149, pp. 131–136 (October 2002)Google Scholar
  2. 2.
    Erdogmus, H., Morisio, M., Torchiano, M.: On the Effectiveness of the Test-First Approach to Programming. IEEE Transactions on Software Engineering 31(3), 226–237 (2005)CrossRefGoogle Scholar
  3. 3.
    Pančur, M., Ciglarič, M., Trampuš, M., Vidmar, T.: Towards Empirical Evaluation of Test-Driven Development in a University Environment. In: EUROCON 2003. Computer as a Tool. The IEEE Region 8, vol. 2, pp. 83–86 (September 2003)Google Scholar
  4. 4.
    Edwards, S.H.: Using Software Testing to Move Students from Trial-and-Error to Reflection-in-Action. SIGCSE Bull. 36(1), 26–30 (2004)CrossRefGoogle Scholar
  5. 5.
    Kaufmann, R., Janzen, D.: Implications of Test-Driven development: A Pilot Study. In: OOPSLA 2003: Companion of the 18th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications, pp. 298–299. ACM, New York (2003)Google Scholar
  6. 6.
    Geras, A., Smith, M., Miller, J.: A Prototype Empirical Evaluation of Test Driven Development. In: Proceedings of the 10th International Symposium on Software Metrics, Washington, DC, USA, pp. 405–416. IEEE Computer Society, Los Alamitos (2004)Google Scholar
  7. 7.
    Canfora, G., Cimitile, A., Garcia, F., Piattini, M., Visaggio, C.A.: Evaluating Advantages of Test Driven Development: a Controlled Experiment with Professionals. In: ISESE 2006: Proceedings of the 2006 ACM/IEEE international symposium on empirical software engineering, pp. 364–371. ACM Press, New York (2006)Google Scholar
  8. 8.
    George, B., Williams, L.: An Initial Investigation of Test Driven Development in Industry. In: SAC 2003: Proceedings of the, ACM symposium on Applied computing, pp. 1135–1139. ACM Press, New York (2003)CrossRefGoogle Scholar
  9. 9.
    Maximilien, E.M., Williams, L.: Assessing Test-Driven Development at IBM. In: ICSE 2003: Proceedings of the 25th International Conference on Software Engineering, Washington, DC, USA, pp. 564–569. IEEE Computer Society, Los Alamitos (2003)CrossRefGoogle Scholar
  10. 10.
    Bhat, T., Nagappan, N.: Evaluating the Efficacy of Test-Driven Development: Industrial Case Studies. In: ISESE 2006: Proceedings of the 2006 ACM/IEEE international symposium on empirical software engineering, pp. 356–363. ACM Press, New York (2006)Google Scholar
  11. 11.
    Dybå, T., Arisholm, E., Sjøberg, D.I., Hannay, J.E., Shull, F.: Are Two Heads Better than One? On the Effectiveness of Pair Programming. IEEE Software 24(6), 12–15 (2007)CrossRefGoogle Scholar
  12. 12.
    Domino, M.A., Collins, R.W., Hevner, A.R., Cohen, C.F.: Conflict in collaborative software development. In: SIGMIS CPR 2003: Proceedings of the 2003 SIGMIS conference on Computer personnel research, pp. 44–51. ACM, New York (2003)Google Scholar
  13. 13.
    Chao, J., Atli, G.: Critical personality traits in successful pair programming. In: Proceedings of Agile 2006 Conference, pp. 89–93 (2006)Google Scholar
  14. 14.
    Katira, N., Williams, L., Wiebe, E., Miller, C., Balik, S., Gehringer, E.: On understanding compatibility of student pair programmers. SIGCSE Bull. 36(1), 7–11 (2004)CrossRefGoogle Scholar
  15. 15.
    Sfetsos, P., Stamelos, I., Angelis, L., Deligiannis, I.: Investigating the Impact of Personality Types on Communication and Collaboration-Viability in Pair Programming – An Empirical Study. In: Abrahamsson, P., Marchesi, M., Succi, G. (eds.) XP 2006. LNCS, vol. 4044, pp. 43–52. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  16. 16.
    Mackinnon, T., Freeman, S., Craig, P.: Endo-testing: unit testing with mock objects. In: Extreme programming examined, pp. 287–301. Addison-Wesley Longman Publishing Co., Inc., Boston (2001)Google Scholar
  17. 17.
    Hollander, M., Wolfe, D.A.: Nonparametric Statistical Methods, 2nd edn. Wiley Interscience, Hoboken (1999)zbMATHGoogle Scholar
  18. 18.
    Holland, B.S., DiPonzio Copenhaver, M.: Improved Bonferroni-Type Multiple Testing Procedure. Psychological Bulletin 104(1), 145–149 (1988)CrossRefzbMATHGoogle Scholar
  19. 19.
    Müller, M.M., Höfer, A.: The Effect of Experience on the Test-Driven Development Process. Empirical Software Engineering 12(6), 593–615 (2007)CrossRefGoogle Scholar
  20. 20.
    Arisholm, E., Gallis, H., Dybå, T., Sjøberg, D.I.K.: Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise. IEEE Transactions on Software Engineering 33(2), 65–86 (2007)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Andreas Höfer
    • 1
  • Marc Philipp
    • 2
  1. 1.IPD InstituteUniversität Karlsruhe (TH)KarlsruheGermany
  2. 2.andrena objects agKarlsruheGermany

Personalised recommendations