A Quantitative Comparison of Test-First and Test-Last Code in an Industrial Project

  • Burak Turhan
  • Ayse Bener
  • Pasi Kuvaja
  • Markku Oivo
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 48)


This paper presents a comparison of test-first and test-last development approaches on a customer account management software of a telecommunication company. While the management had plans for initiating a process that enforces test-first development over test-last, they also had concerns about the tradeoffs. Therefore, an exploratory case study with quantitative analysis is carried out on a pilot project where the code metrics and estimated manual inspection efforts of both approaches are compared. Our results indicate no statistical difference between the two approaches in terms of design (CK) metrics. On the other hand, we observe that test-last development yields significantly simpler code in terms of cyclomatic complexity and less (though not significant) manual inspection effort. Hence, our initial results indicate no superiority of test-first over test-last development in the described industrial context.


Test first Test last Code metrics Case study 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Erdogmus, H., Morisio, M., Torchiano, M.: On the Effectiveness of the Test-First Approach to Programming. IEEE Trans. Softw. Eng. 31(3), 226–237 (2005)CrossRefGoogle Scholar
  2. 2.
    Janzen, D.S., Saiedian, H.: On the Influence of Test-Driven Development on Software Design. In: 19th Conference on Software Engineering Education and Training (CSEET 2006), pp. 141–148. IEEE Press, New York (2006)CrossRefGoogle Scholar
  3. 3.
    Kaufmann, R., Janzen, D.: Implications of Test-Driven Development A Pilot Study. In: 18th Annual ACM Conference on Object-Oriented Programming, Systems, Languages and Applications (OOPSLA 2003), pp. 298–299. ACM, New York (2003)Google Scholar
  4. 4.
    Madeyski, L.: Preliminary Analysis of the Effects of Pair Programming and Test-Driven Development on the External Code Quality. In: Zieliski, K., Szmuc, T. (eds.) Frontiers in Artificial Intelligence and Applications, vol. 130, pp. 113–123. IOS Press, Amsterdam (2005)Google Scholar
  5. 5.
    Müller, M.M.: The Effect of Test-Driven Development on Program Code. In: Abrahamsson, P., Marchesi, M., Succi, G. (eds.) XP 2006. LNCS, vol. 4044, pp. 94–103. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  6. 6.
    Shepperd, M.J., Ince, D.C.: A Critique of Three Metrics. Journal of Systems and Software 26(3), 197–210 (1994)CrossRefGoogle Scholar
  7. 7.
    Siniaalto, M., Abrahamsson, P.: Does Test-Driven Development Improve the Program Code? Alarming Results from a Comparative Case Study. In: Meyer, B., Nawrocki, J.R., Walter, B. (eds.) CEE-SET 2007. LNCS, vol. 5082, pp. 143–156. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  8. 8.
    Vu, J.H., Frojd, N., Shenkel-Therolf, C., Janzen, D.S.: Evaluating Test-Driven Development in an Industry-Sponsored Capstone Project. In: Proceedings of the Sixth international Conference on information Technology: New Generations (ITNG 2009), vol. 00, pp. 229–234. IEEE Computer Society, Washington (2009)CrossRefGoogle Scholar
  9. 9.
    Williams, L., Maximilien, E.M., Vouk, M.: Test-Driven Development as a Defect-Reduction Practice. In: Proceedings of the 14th international Symposium on Software Reliability Engineering (ISSRE 2003), pp. 34–45. IEEE Computer Society, Washington (2003)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Burak Turhan
    • 1
  • Ayse Bener
    • 2
  • Pasi Kuvaja
    • 1
  • Markku Oivo
    • 1
  1. 1.M-Group, Department of Information Processing ScienceUniversity of OuluOuluFinland
  2. 2.Department of Computer EngineeringBogazici UniversityIstanbulTurkey

Personalised recommendations