Skip to main content
Book cover

International Conference on Product-Focused Software Process Improvement

PROFES 2019: Product-Focused Software Process Improvement pp 3–19Cite as

An Empirical Assessment on Affective Reactions of Novice Developers When Applying Test-Driven Development

Part of the Lecture Notes in Computer Science book series (LNPSE,volume 11915)

Abstract

We study whether and in which phase Test-Driven Development (TDD) influences affective states of novice developers in terms of pleasure, arousal, dominance, and liking. We performed a controlled experiment with 29 novice developers. Developers in the treatment group performed a development task using TDD, whereas those in the control group used a non-TDD development approach. We compared the affective reactions to the development approaches, as well as to the implementation and testing phases, exploiting a lightweight, powerful, and widely used tool, i.e., Self-Assessment Manikin. We observed that there is a difference between the two development approaches in terms of affective reactions. Therefore, it seems that affective reactions play an important role when applying TDD and their investigation could help researchers to better understand such a development approach.

Keywords

  • Test-Driven Development
  • TDD
  • Affective state
  • SAM

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-35333-9_1
  • Chapter length: 17 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   89.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-35333-9
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   119.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.

Notes

  1. 1.

    https://doi.org/10.6084/m9.figshare.9778019.v1.

  2. 2.

    Although refactoring is part of the process underlying TDD, we did not consider this phase because refactoring could not be performed when following a non-TDD development approach (and some participants who used a non-TDD approach did not refactor their code).

  3. 3.

    In TL development, a developer first implements a feature entirely and then tests it.

  4. 4.

    In ITL development, a developer alternates implementing a code increment with testing that increment until the entire feature is implemented.

  5. 5.

    https://concordion.org/.

  6. 6.

    STR does not meet the normality assumption (Shapiro-Wilk normality test p−value \(=\) 0.0114); this is why we run ATS (rather than ANOVA).

References

  1. Aniche, M.F., Ferreira, T.M., Gerosa, M.A.: What concerns beginner test-driven development practitioners: a qualitative analysis of opinions in an agile conference. In: Proceedings of Brazilian Workshop on Agile Methods. Springer (2011)

    Google Scholar 

  2. Beck, K.: Test-Driven Development: By Example. Addison-Wesley (2003)

    Google Scholar 

  3. Bergersen, G.R., Sjøberg, D.I.K., Dybå, T.: Construction and validation of an instrument for measuring programming skill. IEEE Trans. Softw. Eng. 40(12), 1163–1184 (2014)

    CrossRef  Google Scholar 

  4. Bradley, M.M., Lang, P.J.: Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 25(1), 49–59 (1994)

    CrossRef  Google Scholar 

  5. Brunner, E., Dette, H., Munk, A.: Box-type approximations in nonparametric factorial designs. J. Am. Stat. Assoc. 92(440), 1494–1502 (1997)

    MathSciNet  CrossRef  Google Scholar 

  6. Carver, J., Jaccheri, L., Morasca, S., Shull, F.: Issues in using students in empirical studies in software engineering education. In: Proceedings of International Symposium on Software Metrics, pp. 239–249. IEEE (2003)

    Google Scholar 

  7. Cliff, N.: Ordinal Methods for Behavioral Data Analysis. Psychology Press (1996)

    Google Scholar 

  8. Colomo-Palacios, R., Hernández-López, A., García-Crespo, Á., Soto-Acosta, P.: A study of emotions in requirements engineering. In: Lytras, M.D., Ordonez de Pablos, P., Ziderman, A., Roulstone, A., Maurer, H., Imber, J.B. (eds.) WSKS 2010. CCIS, vol. 112, pp. 1–7. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-16324-1_1

    CrossRef  Google Scholar 

  9. Csikszentmihalyi, M.: Finding Flow: The Psychology of Engagement with Everydaylife. Basic Books (1997)

    Google Scholar 

  10. Erdogmus, H., Morisio, M., Torchiano, M.: On the effectiveness of the test-first approach to programming. IEEE Trans. Softw. Eng. 31(3), 226–237 (2005)

    CrossRef  Google Scholar 

  11. Ford, D., Parnin, C.: Exploring causes of frustration for software developers. In: Proceedings of International Workshop on Cooperative and Human Aspects of Software Engineering, pp. 115–116. IEEE (2015)

    Google Scholar 

  12. Fucci, D., Erdogmus, H., Turhan, B., Oivo, M., Juristo, N.: A dissection of the test-driven development process: does it really matter to test-first or to test-last? IEEE Trans. Softw. Eng. 43(7), 597–614 (2017)

    CrossRef  Google Scholar 

  13. Fucci, D., et al.: A longitudinal cohort study on the retainment of test-driven development. In: Proceedings of International Symposium on Empirical Software Engineering and Measurement, pp. 18:1–18:10. ACM (2018)

    Google Scholar 

  14. Fucci, D., et al.: An external replication on the effects of test-driven development using a multi-site blind analysis approach. In: Proceedings of International Symposium on Empirical Software Engineering and Measurement, pp. 3:1–3:10. ACM (2016)

    Google Scholar 

  15. Gachechiladze, D., Lanubile, F., Novielli, N., Serebrenik, A.: Anger and its direction in collaborative software development. In: Proceedings of International Conference on Software Engineering: New Ideas and Emerging Technologies Results Track, pp. 11–14. IEEE (2017)

    Google Scholar 

  16. Girardi, D., Lanubile, F., Novielli, N., Fucci, D.: Sensing developers’ emotions: the design of a replicated experiment. In: Proceedings of International Workshop on Emotion Awareness in Software Engineering, pp. 51–54. IEEE (2018)

    Google Scholar 

  17. Graziotin, D., Fagerholm, F., Wang, X., Abrahamsson, P.: What happens when software developers are (un)happy. J. Syst. Softw. 140, 32–47 (2018)

    CrossRef  Google Scholar 

  18. Graziotin, D., Wang, X., Abrahamsson, P.: Are happy developers more productive? In: Heidrich, J., Oivo, M., Jedlitschka, A., Baldassarre, M.T. (eds.) PROFES 2013. LNCS, vol. 7983, pp. 50–64. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39259-7_7

    CrossRef  Google Scholar 

  19. Herbon, A., Peter, C., Markert, L., Van Der Meer, E., Voskamp, J.: Emotion studies in HCI-a new approach. In: Proceedings of International Conference on Human-Computer Interaction (2005)

    Google Scholar 

  20. Höst, M., Regnell, B., Wohlin, C.: Using students as subjects–a comparative study of students and professionals in lead-time impact assessment. Empirical Softw. Eng. 5(3), 201–214 (2000)

    CrossRef  Google Scholar 

  21. Jedlitschka, A., Ciolkowski, M., Pfahl, D.: Guide to advanced empirical software engineering. In: Shull, F., Singer, J., Sjoberg, D.I.K. (eds.) Guide to Advanced Empirical Software Engineering, pp. 201–228. Springer, London (2008). https://doi.org/10.1007/978-1-84800-044-5_8

    CrossRef  Google Scholar 

  22. Jeffries, R., Melnik, G.: Guest editors’ introduction: TDD-the art of fearless programming. IEEE Softw. 24(3), 24–30 (2007)

    CrossRef  Google Scholar 

  23. Kaptein, M.C., Nass, C., Markopoulos, P.: Powerful and consistent analysis of likert-type ratingscales. In: Proceedings of International Conference on Human Factors in Computing Systems, pp. 2391–2394. ACM (2010)

    Google Scholar 

  24. Karac, I., Turhan, B.: What do we (really) know about test-driven development? IEEE Softw. 35(4), 81–85 (2018)

    CrossRef  Google Scholar 

  25. Khan, I.A., Brinkman, W.P., Hierons, R.M.: Do moods affect programmers’ debug performance? Cogn. Technol. Work 13(4), 245–258 (2011)

    CrossRef  Google Scholar 

  26. Koelstra, S., et al.: Deap: a database for emotion analysis using physiological signals. IEEE Trans. Affect. Comput. 3(1), 18–31 (2012)

    CrossRef  Google Scholar 

  27. Mäntylä, M., Adams, B., Destefanis, G., Graziotin, D., Ortu, M.: Mining valence, arousal, and dominance: possibilities for detecting burnout and productivity? In: Proceedings of International Conference on Mining Software Repositories, pp. 247–258. ACM (2016)

    Google Scholar 

  28. Müller, S.C., Fritz, T.: Stuck and frustrated or in flow and happy: sensing developers’ emotions and progress. In: International Conference on Software Engineering, vol. 1, pp. 688–699. IEEE (2015)

    Google Scholar 

  29. Munir, H., Moayyed, M., Petersen, K.: Considering rigor and relevance when evaluating test driven development: a systematic review. Inf. Softw. Technol. 56(4), 375–394 (2014)

    CrossRef  Google Scholar 

  30. Murgia, A., Tourani, P., Adams, B., Ortu, M.: Do developers feel emotions? an exploratory analysis of emotions in software artifacts. In: Proceedings of Working Conference on Mining Software Repositories, pp. 262–271. ACM (2014)

    Google Scholar 

  31. Ortu, M., et al.: The emotional side of software developers in JIRA. In: Proceedings of International Conference on Mining Software Repositories, pp. 480–483. ACM (2016)

    Google Scholar 

  32. Rafique, Y., Mišić, V.B.: The effects of test-driven development on external quality and productivity: a meta-analysis. IEEE Trans. Softw. Eng. 39(6), 835–856 (2013)

    CrossRef  Google Scholar 

  33. Romano, J., Kromrey, J., Coraggio, J., Skowronek, J.: Appropriate statistics for ordinal level data: should we really be using t-test and Cohen’sd for evaluating group differences on the NSSE and other surveys? In: Annual Meeting of the Florida Association of Institutional Research, pp. 1–3 (2006)

    Google Scholar 

  34. Romano, S., Fucci, D., Scanniello, G., Turhan, B., Juristo, N.: Results from an ethnographically-informed study in the context of test driven development. In: Proceedings of the International Conference on Evaluation and Assessment in Software Engineering, pp. 10:1–10:10. ACM (2016)

    Google Scholar 

  35. Romano, S., Fucci, D., Scanniello, G., Turhan, B., Juristo, N.: Findings from a multi-method study on test-driven development. Inf. Softw. Technol. 89, 64–77 (2017)

    CrossRef  Google Scholar 

  36. Rudmann, D.S., McConkie, G.W., Zheng, X.S.: Eyetracking in cognitive state detection for HCI. In: Proceedings of international conference on Multimodal interfaces, pp. 159–163. ACM (2003)

    Google Scholar 

  37. Russell, J.A.: Core affect and the psychological construction of emotion. Psychol. Rev. 110(1), 145–172 (2003)

    CrossRef  Google Scholar 

  38. Salman, I., Misirli, A.T., Juristo, N.: Are students representatives of professionals in software engineering experiments? In: International Conference on Software Engineering, vol. 1, pp. 666–676. IEEE (2015)

    Google Scholar 

  39. Turhan, B., Layman, L., Diep, M., Erdogmus, H., Shull, F.: How effective is test-driven development. In: Making Software: What Really Works, and Why We Believe It, pp. 207–217. O’Reilly Media (2010)

    Google Scholar 

  40. Wohlin, C., Runeson, P., Hst, M., Ohlsson, M.C., Regnell, B., Wessln, A.: Experimentation in Software Engineering. Springer, New York (2012). https://doi.org/10.1007/978-1-4615-4625-2

    CrossRef  Google Scholar 

  41. Wrobel, M.R.: Emotions in the software development process. In: Proceedings of International Conference on Human System Interactions, pp. 518–523. IEEE (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Bari, Bari, Italy

    Simone Romano, Maria Teresa Baldassarre & Danilo Caivano

  2. University of Hamburg, Hamburg, Germany

    Davide Fucci

  3. University of Basilicata, Potenza, Italy

    Giuseppe Scanniello

Authors
  1. Simone Romano
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Davide Fucci
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maria Teresa Baldassarre
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Danilo Caivano
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Giuseppe Scanniello
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Simone Romano .

Editor information

Editors and Affiliations

  1. Universitat Politècnica de Catalunya, Barcelona, Spain

    Prof. Dr. Xavier Franch

  2. University of Helsinki, Helsinki, Finland

    Tomi Männistö

  3. Fraunhofer Institute for Experimental Software Engineering, Kaiserslautern, Germany

    Silverio Martínez-Fernández

Rights and permissions

Reprints and Permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Romano, S., Fucci, D., Baldassarre, M.T., Caivano, D., Scanniello, G. (2019). An Empirical Assessment on Affective Reactions of Novice Developers When Applying Test-Driven Development. In: Franch, X., Männistö, T., Martínez-Fernández, S. (eds) Product-Focused Software Process Improvement. PROFES 2019. Lecture Notes in Computer Science(), vol 11915. Springer, Cham. https://doi.org/10.1007/978-3-030-35333-9_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-35333-9_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-35332-2

  • Online ISBN: 978-3-030-35333-9

  • eBook Packages: Computer ScienceComputer Science (R0)