Skip to main content
SpringerLink
Log in

Critical Issues on Test-Driven Development

  • Conference paper
Product-Focused Software Process Improvement (PROFES 2011)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 6759))

Abstract

During the last decade, Test-Driven Development (TDD) has been actively discussed in the software engineering community. It has been regarded as a useful and beneficial software development practice as well in industry as in academia. After a decade of active research, there is still very little critical discussion on TDD in the literature. This paper is based on a literature review and it is focused on identifying and introducing critical viewpoints on TDD. First, the current evidence on TDD’s benefits is still weak and it includes several issues. Second, the paper presents a number of other possible issues and challenges with TDD that are referred in the literature. Finally, based on the findings, a list of concrete research questions for the future research is presented.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abrahamsson, P., Hanhineva, A., Jäälinoja, J.: Improving Business Agility Through Technical Solutions: A Case Study on Test-Driven Development in Mobile Software Development. In: Business Agility and Information Technology Diffusion. IFIP International Federation for Information Processing, vol. 180, pp. 227–243. Springer, Boston (2006)

    Chapter  Google Scholar 

  2. Ambler, S.W.: Test-driven development of relational databases. IEEE Software 24(3), 37 (2007)

    Article  MathSciNet  Google Scholar 

  3. Astels, D.: Test Driven Development: A Practical Guide. Prentice Hall, Upper Saddle River (2003)

    Google Scholar 

  4. Beck, K.: Aim, fire [test-first coding]. IEEE Software 18(5), 87 (2001)

    Article  Google Scholar 

  5. Beck, K.: Extreme Programming Explained: Embrace Change, 1st edn., p. 224. Addison-Wesley Professional, Reading (1999)

    Google Scholar 

  6. Beck, K.: Test-Driven Development: By Example. The Addison-Wesley Signature Series. Addison-Wesley, Reading (2003)

    Google Scholar 

  7. Bhat, T., Nagappan, N.: Evaluating the efficacy of test-driven development: industrial case studies. In: ISESE 2006: Proceedings of the 2006 ACM/IEEE International Symposium on Empirical Software Engineering, pp. 356–363. ACM, New York (2006)

    Google Scholar 

  8. Boehm, B., Turner, R.: Balancing Agility and Discipline - A Guide for the Perplexed. Addison-Wesley, Reading (2004)

    Book  Google Scholar 

  9. Larman, C., Basili, V.R.: Iterative and incremental developments. a brief history. Computer 36(6), 47–56 (2003)

    Article  Google Scholar 

  10. Canfora, G., Cimitile, A., Garcia, F., Piattini, M., Visaggio, C.A.: Evaluating advantages of test driven development: a controlled experiment with professionals. In: ISESE 2006: Proceedings of the 2006 ACM/IEEE International Symposium on Empirical Software Engineering, pp. 364–371. ACM, New York (2006)

    Google Scholar 

  11. Canfora, G., Cimitile, A., García, F., Piattini, M., Visaggio, C.A.: Productivity of test driven development: A controlled experiment with professionals. In: Münch, J., Vierimaa, M. (eds.) PROFES 2006. LNCS, vol. 4034, pp. 383–388. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  12. Crispin, L.: Driving software quality: How test-driven development impacts software quality. IEEE Software 23(6), 70–71 (2006)

    Article  Google Scholar 

  13. Damm, L.-O., Lundberg, L.: Quality impact of introducing component-level test automation and test-driven development. In: Abrahamsson, P., Baddoo, N., Margaria, T., Messnarz, R. (eds.) EuroSPI 2007. LNCS, vol. 4764, pp. 187–199. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  14. Desai, C., Janzen, D.S., Clements, J.: Implications of integrating test-driven development into cs1/cs2 curricula. In: SIGCSE 2009: Proceedings of the 40th ACM Technical Symposium on Computer Science Education, pp. 148–152. ACM, New York (2009)

    Google Scholar 

  15. van Deursen, A.: Program comprehension risks and opportunities in extreme programming. In: Proceedings of the Eighth Working Conference on Reverse Engineering (2001)

    Google Scholar 

  16. Deursen, A.V., Moonen, L., Bergh, A., Kok, G.: Refactoring test code. In: Proceedings of the 2nd International Conference on Extreme Programming and Flexible Processes in Software Engineering (XP 2001), pp. 92–95 (2001)

    Google Scholar 

  17. Dybå, T., Dingsøyr, T.: Empirical studies of agile software development: A systematic review. Inf. Softw. Technol. 50, 833–859 (2008), http://portal.acm.org/citation.cfm?id=1379905.1379989

    Article  Google Scholar 

  18. Dybå, T., Dingsøyr, T.: Strength of evidence in systematic reviews in software engineering. In: Proceedings of the Second ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2008, pp. 178–187. ACM, New York (2008), http://doi.acm.org/10.1145/1414004.1414034

    Chapter  Google Scholar 

  19. Edwards, S.H.: Using test-driven development in the classroom: Providing students with automatic, concrete feedback on performance. In: Proc. Int”l Conf. Education and Information Systems: Technologies and Applications, EISTA 20 03 (2003)

    Google Scholar 

  20. Edwards, S.H.: Using software testing to move students from trial-and-error to reflection-in-action. In: SIGCSE 2004: Proceedings of the 35th SIGCSE Technical Symposium on Computer Science Education, pp. 26–30. ACM, New York (2004)

    Chapter  Google Scholar 

  21. Erdogmus, H., Morisio, M., Torchiano, M.: On the effectiveness of the test-first approach to programming. IEEE Transactions on Software Engineering 31(3), 226–237 (2005), doi:10.1109/TSE.2005.37

    Article  Google Scholar 

  22. Flohr, T., Schneider, T.: Lessons learned from an XP experiment with students: Test-first needs more teachings. In: Münch, J., Vierimaa, M. (eds.) PROFES 2006. LNCS, vol. 4034, pp. 305–318. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  23. George, B., Williams, L.: An initial investigation of test driven development in industry. In: SAC 2003: Proceedings of the 2003 ACM Symposium on Applied Computing, pp. 1135–1139. ACM, New York (2004)

    Google Scholar 

  24. George, B., Williams, L.: A structured experiment of test-driven development. Information and Software Technology 46(5), 337–342 (2004)

    Article  Google Scholar 

  25. Geras, A., Smith, M., Miller, J.: A prototype empirical evaluation of test driven development. In: METRICS 2004: Proceedings of the Software Metrics, 10th International Symposium, pp. 405–416. IEEE Computer Society, Washington, DC, USA (2004)

    Chapter  Google Scholar 

  26. Grenning, J.: Applying test driven development to embedded software. IEEE Instrumentation Measurement Magazine 10(6), 20–25 (2007)

    Article  Google Scholar 

  27. Gupta, A., Jalote, P.: An experimental evaluation of the effectiveness and efficiency of the test driven development. In: First International Symposium on Empirical Software Engineering and Measurement, ESEM 2007, pp. 285–294 (September 2007)

    Google Scholar 

  28. Hamill, P., Alexander, D., Shasharina, S.: Web service validation enabling test-driven development of service-oriented applications. In: Proceedings of the 2009 Congress on Services - I, pp. 467–470. IEEE Computer Society, Washington, DC, USA (2009), http://portal.acm.org/citation.cfm?id=1590963.1591598

    Chapter  Google Scholar 

  29. Hedin, G., Bendix, L., Magnusson, B.: Teaching extreme programming to large groups of students. Journal of Systems and Software 74(2), 133–146 (2005)

    Article  Google Scholar 

  30. Huang, L., Holcombe, M.: Empirical investigation towards the effectiveness of test first programming. Information and Software Technology 51(1), 182–194 (2009)

    Article  Google Scholar 

  31. Janzen, D., Saiedian, H.: Does test-driven development really improve software design quality? IEEE Software 25(2), 77–84 (2008)

    Article  Google Scholar 

  32. Janzen, D.S., Saiedian, H.: On the influence of test-driven development on software design. In: CSEET 2006: Proceedings of the 19th Conference on Software Engineering Education & Training, pp. 141–148. IEEE Computer Society, Washington, DC, USA (2006)

    Google Scholar 

  33. Janzen, D.S., Saiedian, H.: A leveled examination of test-driven development acceptance. In: 29th International Conference on Software Engineering, ICSE 2007, pp. 719–722 (May 2007)

    Google Scholar 

  34. Janzen, D.S., Turner, C.S., Saiedian, H.: Empirical software engineering in industry short courses. In: 20th Conference on Software Engineering Education Training, CSEET 2007, pp. 89–96 (July 2007)

    Google Scholar 

  35. Kaufmann, R., Janzen, D.: Implications of test-driven development: a pilot study. In: OOPSLA 2003: Companion of the 18th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, pp. 298–299. ACM, New York (2003), doi:10.1145/949344.949421

    Google Scholar 

  36. Kitchenham, B., Charters, S.: Guidelines for performing systematic literature reviews in software engineering. Tech. Rep. EBSE 2007-001, Keele University and Durham University Joint Report (2007), http://www.dur.ac.uk/ebse/resources/Systematic-reviews-5-8.pdf

  37. Kollanus, S.: Test-driven development - still a promising approach? In: Proceedings of the 7th International Conference on the Quality of Information and Communications Technology, pp. 403–408 (2010)

    Google Scholar 

  38. Kollanus, S., Isomöttönen, V.: Test-driven development in education: experiences with critical viewpoints. In: Proceedings of the 13th Annual Conference on Innovation and Technology in Computer Science Education, pp. 124–127. ACM, New York (2008)

    Google Scholar 

  39. Kollanus, S., Isomöttönen, V.: Understanding tdd in academic environment: Experiences from two experiments. In: Pears, A., Malmi, L. (eds.) 8th International Conference on Computing Education Research, Koli Calling 2008, pp. 25–31 (2009)

    Google Scholar 

  40. Lui, K.M., Chan, K.C.: Test driven development and software process improvement in china. In: Extreme Programming and Agile Processes in Software Engineering, pp. 219–222 (2004)

    Google Scholar 

  41. Madeyski, L.: Preliminary analysis of the effects of pair programming and test-driven development on the external code quality. In: Proceeding of the 2005 Conference on Software Engineering: Evolution and Emerging Technologies, pp. 113–123. IOS Press, Amsterdam (2005)

    Google Scholar 

  42. Madeyski, L.: The impact of pair programming and test-driven development on package dependencies in object-oriented design — an experiment. In: Münch, J., Vierimaa, M. (eds.) PROFES 2006. LNCS, vol. 4034, pp. 278–289. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  43. Madeyski, L.: The impact of test-first programming on branch coverage and mutation score indicator of unit tests: An experiment. Inf. Softw. Technol. 52(2), 169–184 (2010)

    Article  Google Scholar 

  44. Madeyski, L., Szała, Ł.: The impact of test-driven development on software development productivity — an empirical study. In: Abrahamsson, P., Baddoo, N., Margaria, T., Messnarz, R. (eds.) EuroSPI 2007. LNCS, vol. 4764, pp. 200–211. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  45. Maximilien, E.M., Williams, L.: Assessing test-driven development at ibm. In: Proceedings of the 25th International Conference on Software Engineering, pp. 564–569 (May 2003)

    Google Scholar 

  46. Muller, M.M., Hagner, O.: Experiment about test-first programming. Software, IEE Proceedings 149(5), 131–136 (2002)

    Article  Google Scholar 

  47. Nagappan, N., Maximilien, E.M., Bhat, T., Williams, L.: Realizing quality improvement through test driven development: results and experiences of four industrial teams. Empirical Softw. Engg. 13(3), 289–302 (2008)

    Article  Google Scholar 

  48. Pancur, M., Ciglaric, M., Trampus, M., Vidmar, T.: Towards empirical evaluation of test-driven development in a university environment. In: EUROCON 2003. Computer as a Tool. The IEEE Region 8, vol. 2, pp. 83–86 (September 2003)

    Google Scholar 

  49. Rahman, S.M.: Applying the tbc method in introductory programming courses. In: 37th Annual Frontiers In Education Conference - Global Engineering: Knowledge Without Borders, Opportunities Without Passports, FIE, pp. T1E–20–T1E–21 (October 2007)

    Google Scholar 

  50. Rasmussen, J.: Introducing xp into greenfield projects: lessons learned. IEEE Software 20(3), 21–28 (2003)

    Article  Google Scholar 

  51. Rendell, A.: Effective and pragmatic test driven development. In: Conference on AGILE 2008, pp. 298–303 (August 2008)

    Google Scholar 

  52. Sanchez, J.C., Williams, L., Maximilien, E.M.: On the sustained use of a test-driven development practice at ibm. In: AGILE 2007, pp. 5–14 (August 2007)

    Google Scholar 

  53. Sangwan, R.S., Laplante, P.A.: Test-driven development in large projects. IT Professional 8(5), 25–29 (2006)

    Article  Google Scholar 

  54. Siniaalto, M., Abrahamsson, P.: A comparative case study on the impact of test-driven development on program design and test coverage. In: First International Symposium on Empirical Software Engineering and Measurement, ESEM 2007, pp. 275–284 (September 2007)

    Google Scholar 

  55. Siniaalto, M., Abrahamsson, P.: Does test-driven development improve the program code? Alarming results from a comparative case study. In: Meyer, B., Nawrocki, J.R., Walter, B. (eds.) CEE-SET 2007. LNCS, vol. 5082, pp. 143–156. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  56. Slyngstad, O.P.N., Li, J., Conradi, R., Rønneberg, H., Landre, E., Wesenberg, H.: The impact of test driven development on the evolution of a reusable framework of components - an industrial case study. In: ICSEA 2008: Proceedings of the 2008 The Third International Conference on Software Engineering Advances, pp. 214–223. IEEE Computer Society, Washington, DC, USA (2008)

    Chapter  Google Scholar 

  57. Steinberg, D.H.: The effect of unit tests on entry points, coupling and cohesion in an introductory java programming course. In: XP Universe Conference 2001 (2001)

    Google Scholar 

  58. Tinkham, A., Kaner, C.: Experiences teaching a course in programmer testing. In: ADC 2005: Proceedings of the Agile Development Conference, pp. 298–305. IEEE Computer Society, Washington, DC, USA (2005)

    Google Scholar 

  59. Vodde, B., Koskela, L.: Learning test-driven development by counting lines. IEEE Software 24(3), 74–79 (2007)

    Article  Google Scholar 

  60. Vu, J.H., Frojd, N., Shenkel-Therolf, C., Janzen, D.S.: Evaluating test-driven development in an industry-sponsored capstone project. In: Sixth International Conference on Information Technology: New Generations, ITNG 2009, pp. 229–234 (April 2009)

    Google Scholar 

  61. Williams, L., Maximilien, E.M., Vouk, M.: Test-driven development as a defect-reduction practice. In: 14th International Symposium on Software Reliability Engineering, ISSRE 2003, pp. 34–45 (November 2003)

    Google Scholar 

  62. Xu, S., Li, T.: Evaluation of test-driven development: An academic case study. In: Software Engineering Research, Management and Applications 2009. Studies in Computational Intelligence, vol. 253, pp. 229–238 (2009)

    Google Scholar 

  63. Ynchausti, R.A.: Integrating unit testing into a software development team’s process. In: Intl. Conf. eXtreme Programming and Flexible Processes in Software Engineering, pp. 79–83 (2001)

    Google Scholar 

  64. Zhang, L., Akifuji, S., Kawai, K., Morioka, T.: Comparison between test driven development and waterfall development in a small-scale project. In: Abrahamsson, P., Marchesi, M., Succi, G. (eds.) XP 2006. LNCS, vol. 4044, pp. 211–212. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Information Technology, University of Jyväskylä, Jyväskylä, Finland

    Sami Kollanus

Authors
  1. Sami Kollanus
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Informatics, University of Bari, Via E. Orabona 4, 70126, Bari, Italy

    Danilo Caivano , Maria Teresa Baldassarre  & Giuseppe Visaggio ,  & 

  2. Department of Information Processing Science, University of Oulu, P.O. Box 3000, 90014, Oulu, Finland

    Markku Oivo

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Kollanus, S. (2011). Critical Issues on Test-Driven Development. In: Caivano, D., Oivo, M., Baldassarre, M.T., Visaggio, G. (eds) Product-Focused Software Process Improvement. PROFES 2011. Lecture Notes in Computer Science, vol 6759. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21843-9_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-21843-9_25

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-21842-2

  • Online ISBN: 978-3-642-21843-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation