Skip to main content
Book cover

International Conference on Agile Software Development

XP 2013: Agile Processes in Software Engineering and Extreme Programming pp 91–105Cite as

Effects of Negative Testing on TDD: An Industrial Experiment

  • Conference paper

Part of the Lecture Notes in Business Information Processing book series (LNBIP,volume 149)

Abstract

In our recent academic experiments, an existence of positive test bias, that is lack of negative test cases, was identified when a test driven development approach was used. At the same time, when defect detecting ability of individual test cases was calculated, it was noted that the probability of a negative test case to detect a defect was substantially higher than that of a positive test case.

The goal of this study is to investigate the existence of positive test bias in test driven development within an industrial context, and measure defect detecting ability of both positive and negative test cases. An industrial experiment was conducted at Infosys Ltd. India, whose employees voluntarily signed up to participate in the study and were randomly assigned to groups utilizing test driven development, test driven development with negative testing, and test last development. Source code and test cases created by each participant during the study were collected and analysed.

The collected data indicate a statistically significant difference between the number of positive and negative test cases created by industrial participants, confirming the existence of positive test bias. The difference in defect detecting ability of positive and negative test cases is also statistically significant. As a result, similarly to our previous academic study, 29% of all test cases were negative, contributing by revealing as much as 71% of all the defects found by all test cases. With this industrial experiment, we confirmed the existence of a positive test bias in an industrial context, as well as significantly higher defect detecting ability of negative test cases.

Keywords

  • Test-driven Development
  • Industrial Experiment
  • Quality of Testing

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-642-38314-4_7
  • Chapter length: 15 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   39.99
Price excludes VAT (USA)
  • ISBN: 978-3-642-38314-4
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   54.99
Price excludes VAT (USA)
Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Beck, K.: Extreme programming explained: embrace change. Addison-Wesley Longman Publishing Co., Inc., Boston (2000)

    Google Scholar 

  2. Koskela, L.: Test driven: practical tdd and acceptance tdd for java developers. Manning Publications Co., Greenwich (2007)

    Google Scholar 

  3. Causevic, A., Sundmark, D., Punnekkat, S.: An industrial survey on contemporary aspects of software testing. In: Proceedings of the 3rd International Conference on Software Testing, Verification and Validation (ICST), pp. 393–401 (2010)

    Google Scholar 

  4. George, B., Williams, L.: A structured experiment of test-driven development. Information and Software Technology 46(5), 337–342 (2003)

    CrossRef  Google Scholar 

  5. Erdogmus, H., Morisio, M., Torchiano, M.: On the effectiveness of the test-first approach to programming. IEEE Transactions on Software Engineering 31, 226–237 (2005)

    CrossRef  Google Scholar 

  6. Janzen, D.S., Saiedian, H.: On the influence of test-driven development on software design. In: Conference on Software Engineering Education and Training, pp. 141–148 (2006)

    Google Scholar 

  7. Gupta, A., Jalote, P.: An experimental evaluation of the effectiveness and efficiency of the test driven development. In: Proceedings of the First International Symposium on Empirical Software Engineering and Measurement, ESEM 2007, pp. 285–294. IEEE Computer Society, Washington, DC (2007)

    Google Scholar 

  8. Vu, J.H., Frojd, N., Shenkel-Therolf, C., Janzen, D.S.: Evaluating test-driven development in an industry-sponsored capstone project. In: Proceedings of the 2009 Sixth International Conference on Information Technology: New Generations, pp. 229–234. IEEE Computer Society, Washington, DC (2009)

    CrossRef  Google Scholar 

  9. Causevic, A., Sundmark, D., Punnekkat, S.: Factors limiting industrial adoption of test driven development: A systematic review. In: 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation (ICST), pp. 337–346 (March 2011)

    Google Scholar 

  10. Causevic, A., Punnekkat, S., Sundmark, D.: Quality of testing in test driven development. In: 2012 Eight International Conference on the Quality of Information and Communications Technology (QUATIC) (September 2012)

    Google Scholar 

  11. Teasley, B.E., Leventhal, L.M., Mynatt, C.R., Rohlman, D.S.: Why Software Testing Is Sometimes Ineffective: Two Applied Studies of Positive Test Strategy. Journal of Applied Psychology 79(1), 142–155 (1994)

    CrossRef  Google Scholar 

  12. Leventhal, L.M., Teasley, B., Rohlman, D.S., Instone, K.: Positive Test Bias in Software Testing Among Professionals: A Review. In: Bass, L.J., Unger, C., Gornostaev, J. (eds.) EWHCI 1993. LNCS, vol. 753, pp. 210–218. Springer, Heidelberg (1993)

    CrossRef  Google Scholar 

  13. Madeyski, L.: The impact of test-first programming on branch coverage and mutation score indicator of unit tests: An experiment. Inf. Softw. Technol. 52, 169–184 (2010)

    CrossRef  Google Scholar 

  14. Shelton, W., Li, N., Ammann, P., Offutt, J.: Adding criteria-based tests to test driven development. In: Proceedings of the 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation, ICST 2012, pp. 878–886. IEEE Computer Society, Washington, DC (2012)

    CrossRef  Google Scholar 

  15. Fenton, N.E., Pfleeger, S.L.: Software Metrics: A Rigorous and Practical Approach, 2nd edn. PWS Publishing Co., Boston (1998)

    Google Scholar 

  16. Causevic, A., Sundmark, D., Punnekkat, S.: Test case quality in test driven development: A study design and a pilot experiment. In: 16th International Conference on Evaluation Assessment in Software Engineering (EASE 2012), pp. 223–227 (May 2012)

    Google Scholar 

  17. Eclipse, http://www.eclipse.org

  18. jUnit Framework, http://www.junit.org

  19. Kollanus, S., Isomöttönen, V.: Understanding tdd in academic environment: experiences from two experiments. In: Proceedings of the 8th International Conference on Computing Education Research, Koli 2008, pp. 25–31. ACM, New York (2008)

    Google Scholar 

  20. Flohr, T., Schneider, T.: Lessons learned from an XP experiment with students: Test-first needs more teachings. In: Münch, J., Vierimaa, M. (eds.) PROFES 2006. LNCS, vol. 4034, pp. 305–318. Springer, Heidelberg (2006)

    CrossRef  Google Scholar 

  21. R Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2012) ISBN 3-900051-07-0

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mälardalen University, Sweden

    Adnan Causevic, Sasikumar Punnekkat & Daniel Sundmark

  2. Infosys Ltd., India

    Rakesh Shukla

Authors
  1. Adnan Causevic
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rakesh Shukla
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sasikumar Punnekkat
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daniel Sundmark
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Applied Mathematics and Computer Science, Technical University of Denmark, Lyngby, Denmark

    Hubert Baumeister

  2. Department of Computer Science, University of Innsbruck, Innsbruck, Austria

    Barbara Weber

Rights and permissions

Reprints and Permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Causevic, A., Shukla, R., Punnekkat, S., Sundmark, D. (2013). Effects of Negative Testing on TDD: An Industrial Experiment. In: Baumeister, H., Weber, B. (eds) Agile Processes in Software Engineering and Extreme Programming. XP 2013. Lecture Notes in Business Information Processing, vol 149. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38314-4_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-38314-4_7

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-38313-7

  • Online ISBN: 978-3-642-38314-4

  • eBook Packages: Computer ScienceComputer Science (R0)