An industry experiment on the effects of test-driven development on external quality and productivity

  • Ayse Tosun
  • Oscar Dieste
  • Davide Fucci
  • Sira Vegas
  • Burak Turhan
  • Hakan Erdogmus
  • Adrian Santos
  • Markku Oivo
  • Kimmo Toro
  • Janne Jarvinen
  • Natalia Juristo
Article

DOI: 10.1007/s10664-016-9490-0

Cite this article as:
Tosun, A., Dieste, O., Fucci, D. et al. Empir Software Eng (2016). doi:10.1007/s10664-016-9490-0

Abstract

Existing empirical studies on test-driven development (TDD) report different conclusions about its effects on quality and productivity. Very few of those studies are experiments conducted with software professionals in industry. We aim to analyse the effects of TDD on the external quality of the work done and the productivity of developers in an industrial setting. We conducted an experiment with 24 professionals from three different sites of a software organization. We chose a repeated-measures design, and asked subjects to implement TDD and incremental test last development (ITLD) in two simple tasks and a realistic application close to real-life complexity. To analyse our findings, we applied a repeated-measures general linear model procedure and a linear mixed effects procedure. We did not observe a statistical difference between the quality of the work done by subjects in both treatments. We observed that the subjects are more productive when they implement TDD on a simple task compared to ITLD, but the productivity drops significantly when applying TDD to a complex brownfield task. So, the task complexity significantly obscured the effect of TDD. Further evidence is necessary to conclude whether TDD is better or worse than ITLD in terms of external quality and productivity in an industrial setting. We found that experimental factors such as selection of tasks could dominate the findings in TDD studies.

Keywords

Industry experiment Test-driven development External quality Productivity 

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Ayse Tosun
    • 1
  • Oscar Dieste
    • 2
  • Davide Fucci
    • 3
  • Sira Vegas
    • 2
  • Burak Turhan
    • 3
  • Hakan Erdogmus
    • 4
  • Adrian Santos
    • 3
  • Markku Oivo
    • 3
  • Kimmo Toro
    • 5
  • Janne Jarvinen
    • 5
  • Natalia Juristo
    • 2
    • 3
  1. 1.Faculty of Computer Engineering and InformaticsIstanbul Technical UniversityIstanbulTurkey
  2. 2.Escuela Tecnica Superior de Ingenieros InformaticsUPMMadridSpain
  3. 3.Department of Information Processing ScienceUniversity of OuluOuluFinland
  4. 4.Carnegie Mellon UniversityMoffett FieldUSA
  5. 5.FSecure CorporationHelsinkiFinland

Personalised recommendations