A Tool to Measure TDD Compliance: A Case Study with Professionals

Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 981)


Context: There are several studies related to Test Driven Development (TDD), but many with divergences of results due to the short time to perform the experiments. Moreover, the environment where they are carried out is generally academic. On the other hand, the environment requires tools not used by practitioners or imposes many technical and training requirements for their application. Goal: The goal of this paper is to provide a tool that supports the evaluation of the TDD process in the software industry and academia settings. The tool focuses on analyzing the effects of verification, validation and test (VV&T). In addition, the compliance of TDD usage in software development was evaluated. Method: This study made use of the Goal Question Metric (GQM) paradigm to characterize a set of objectives using metrics towards TDD effects on software quality. A case study was conducted with IT professionals to evaluate the tool developed. Results: Considering the existing tools that perform TDD compliance assessment, the Butterfly tool was developed to enable the evaluation of the TDD lifecycle as the developer performs the coding of the software. With this tool it is possible to analyze the compliance of TDD usage during software development. Conclusions: The tool allows to measure the effects of TDD when developing software, which will support in the characterization of TDD contributions and interventions applied to software quality in future works.


Test driven development Agile software development TDD conformance TDD lifecycle Software measurement 


  1. 1.
    Basili, V.R.: Software modeling and measurement: the goal/question/metric paradigm. Technical report CS-TR-2956, UMIACS-TR-92-96, p. 24. University of Maryland, College Park, MD, USA, September 1992.
  2. 2.
    Beck, K.: eXtreme Programming Explained: Embrace Change. Addison-Wesley, Boston (1999)Google Scholar
  3. 3.
    Beck, K.: Test-Driven Development: By Example. Addison-Wesley Professional, Boston (2002)Google Scholar
  4. 4.
    Becker, K., de Souza Costa Pedroso, B., Pimenta, M.S., Jacobi, R.P.: Besouro: a framework for exploring compliance rules in automatic TDD behavior assessment. Inf. Softw. Technol. 57, 494–508 (2015)Google Scholar
  5. 5.
    Briand, L., Bianculli, D., Nejati, S., Pastore, F., Sabetzadeh, M.: The case for context-driven software engineering research: generalizability is overrated. IEEE Softw. 34(5), 72–75 (2017)Google Scholar
  6. 6.
    Causevic, A., Punnekkat, S., Sundmark, D.: TDD\(^{\text{HQ}}\): achieving higher quality testing in test driven development. In: Euromicro Conference Series on Software Engineering and Advanced Applications, pp. 33–36, Santander, Spain (2013)Google Scholar
  7. 7.
    Causevic, A., Sundmark, D., Punnekkat, S.: Factors limiting industrial adoption of test driven development: a systematic review. In: International Conference on Software Testing, Verification and Validation, pp. 337–346. IEEE, Berlin, Germany (2011)Google Scholar
  8. 8.
    Causevic, A., Shukla, R., Punnekkat, S., Sundmark, D.: Effects of negative testing on TDD: an industrial experiment. In: Baumeister, H., Weber, B. (eds.) XP 2013. LNBIP, vol. 149, pp. 91–105. Springer, Heidelberg (2013). Scholar
  9. 9.
    Fucci, D., Erdogmus, H., Turhan, B., Oivo, M., Juristo, N.: A dissection of the test-driven development process: does it really matter to test-first or to test-last? Trans. Softw. Eng. 43(7), 597–614 (2017)Google Scholar
  10. 10.
    Garousi, V., Felderer, M., Kuhrmann, M., Herkiloglu, K.: What industry wants from academia in software testing?: Hearing practitioners’ opinions. In: International Conference on Evaluation and Assessment in Software Engineering, pp. 65–69. ACM, Karlskrona, Sweden (2017)Google Scholar
  11. 11.
    George, B., Williams, L.: A structured experiment of test-driven development. Inf. Softw. Technol. 46(5), 337–342 (2004)Google Scholar
  12. 12.
    Hammond, S., Umphress, D.: Test driven development: the state of the practice. In: Smith, R.K., Vrbsky, S.V. (eds.) ACM Annual Southeast Regional Conference, pp. 158–163. ACM, Tuscaloosa, Alabama, USA (2012)Google Scholar
  13. 13.
    Janzen, D., Saiedian, H.: Test-driven development concepts, taxonomy, and future direction. Computer 38(9), 43–50 (2005)Google Scholar
  14. 14.
    Juristo, N., Moreno, A.M.: Basics of Software Engineering Experimentation. Kluwer Academic Publishers, Dordrecht (2001)zbMATHGoogle Scholar
  15. 15.
    Kou, H., Johnson, P.M., Erdogmus, H.: Operational definition and automated inference of test-driven development with Zorro. Ann. Softw. Eng. 17(1), 57–85 (2010)Google Scholar
  16. 16.
    Offutt, J.: Why don’t we publish more TDD research papers? Softw. Test. Verif. Reliab. 28(4), e1670 (2018)Google Scholar
  17. 17.
    Pachulski Camara, B.H., Graciotto Silva, M.A.: A strategy to combine test-driven development and test criteria to improve learning of programming skills. In: Technical Symposium on Computing Science Education, pp. 443–448. ACM, Memphis, TN, USA (2016)Google Scholar
  18. 18.
    Raulamo-Jurvanen, P., Mäntylä, M., Garousi, V.: Choosing the right test automation tool: a grey literature review of practitioner sources. In: International Conference on Evaluation and Assessment in Software Engineering, pp. 21–30. ACM, Karlskrona, Sweden (2017)Google Scholar
  19. 19.
    Shelton, W., Li, N., Ammann, P., Offutt, J.: Adding criteria-based tests to test driven development. In: International Conference on Software Testing, Verification and Validation, pp. 878–886. IEEE, Montreal, QC, Canada (2012)Google Scholar
  20. 20.
    Spinellis, D.: State-of-the-art software testing. IEEE Softw. 34(5), 4–6 (2017)Google Scholar
  21. 21.
    Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Experimentation in Software Engineering: An Introduction. Kluwer Academic Publishers, Sweden (2000)zbMATHGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Graduate Program in Informatics (PPGI)Federal University of Technology – Paraná (UTFPR)ParanáBrazil
  2. 2.Department of Computing (DACOM)Federal University of Technology – Paraná (UTFPR)ParanáBrazil

Personalised recommendations