Skip to main content
SpringerLink
Log in

Self-admitted technical debt practices: a comparison between industry and open-source

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Self-admitted technical debt (SATD) consists of annotations, left by developers as comments in the source code or elsewhere, as a reminder about pieces of software manifesting technical debt (TD), i.e., “not being ready yet”. While previous studies have investigated SATD management and its relationship with software quality, there is little understanding of the extent and circumstances to which developers admit TD. This paper reports the results of a study in which we asked developers from industry and open-source about their practices in annotating source code and other artifacts for self-admitting TD. The study consists of two phases. First, we conducted 10 interviews to gather a first understanding of the phenomenon and to prepare a survey questionnaire. Then, we surveyed 52 industrial developers as well as 49 contributors to open-source projects. Results of the study show how the TD annotation practices, as well as the typical content of SATD comments, are very similar between open-source and industry. At the same time, our results highlight how, while open-source code is spread of comments admitting the need for improvements, SATD in industry may be dictated by organizational guidelines but, at the same time, implicitly discouraged by the fear of admitting responsibilities. Results also highlight the need for tools helping developers to achieve a better TD awareness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. See, e.g., the “Todo Comments Considered Harmful” http://wiki.c2.com/?TodoCommentsConsideredHarmful vs. “Todo Comments Considered Useful” http://wiki.c2.com/?TodoCommentsConsideredUseful.

  2. No participant, nor any potential participant reported us to have any particular privacy concern about this channel.

  3. The sum exceeds the number of interviewees as multiple IDEs and programming languages may be used.

References

  • Aghajani E, Nagy C, Vega-Márquez OL, Linares-Vásquez M, Moreno L, Bavota G, Lanza M (2019) Software documentation issues unveiled. In: 2019 IEEE/ACM 41st international conference on software engineering (ICSE). IEEE, pp 1199–1210

  • Aghajani E, Nagy C, Linares-Vásquez M, Moreno L, Bavota G, Lanza M, Shepherd DC (2020) Software documentation: The practitioners’ perspective. In: 2020 IEEE/ACM 42nd international conference on software engineering (ICSE). IEEE

  • Alexeeva Z, Perez-Palacin D, Mirandola R (2016) Design decision documentation: A literature overview. In: Tekinerdogan B, Zdun U, Babar M A (eds) Software Architecture - 10th european conference, ECSA 2016, November 28 - December 2, 2016, Proceedings, Lecture Notes in Computer Science, vol 9839, Denmark, pp 84–101

  • de Almeida RR, Kulesza U, Treude C, Feitosa DC, Lima AHG (2018) Aligning technical debt prioritization with business objectives: A multiple-case study. In: 2018 IEEE international conference on software maintenance and evolution, ICSME 2018, September 23-29, 2018, Spain, pp 655–664

  • de Almeida RR, Treude C, Kulesza U (2019) Tracy: A business-driven technical debt prioritization framework. In: 2019 IEEE international conference on software maintenance and evolution, ICSME 2019, September 29 - October 4, 2019, USA, pp 181–185

  • Alves NSR, Ribeiro LF, Caires V, Mendes TS, Spínola RO (2014) Towards an ontology of terms on technical debt. In: Sixth international workshop on managing technical debt, MTD ICSME 2014, September 30, 2014, Canada, pp 1–7

  • Aranda J, Venolia G (2009) The secret life of bugs: Going past the errors and omissions in software repositories. In: 2009 IEEE 31st international conference on software engineering, pp 298–308

  • Arnaoudova V, Eshkevari LM, Di Penta M, Oliveto R, Antoniol G, Gueheneuc YG (2014) Repent: Analyzing the nature of identifier renamings. IEEE Trans Softw Eng 40(5):502–532

    Article  Google Scholar 

  • Arnaoudova V, Di Penta M, Antoniol G (2016) Linguistic antipatterns: What they are and how developers perceive them. Empir Softw Eng 21 (1):104–158

    Article  Google Scholar 

  • Avelino G, Passos L, Hora A, Valente MT (2016) A novel approach for estimating truck factors. In: 2016 IEEE 24th international conference on program comprehension (ICPC). IEEE, pp 1–10

  • Avelino G, Constantinou E, Valente MT, Serebrenik A (2019) On the abandonment and survival of open source projects: An empirical investigation. In: 2019 ACM/IEEE international symposium on empirical software engineering and measurement, ESEM 2019, September 19-20, 2019. IEEE, Brazil, pp 1–12

  • Baltes S, Ralph P (2020) Sampling in software engineering research: A critical review and guidelines. arXiv:200207764

  • Bavota G, Russo B (2016) A large-scale empirical study on self-admitted technical debt. In: Proceedings of the 13th international conference on mining software repositories, MSR 2016, May 14-22, 2016, USA, pp 315–326

  • Bellomo S, Nord RL, Ozkaya I, Popeck M (2016) Got technical debt?: surfacing elusive technical debt in issue trackers. In: Kim M, Robbes R, Bird C (eds) Proceedings of the 13th International Conference on Mining Software Repositories, MSR 2016, May 14-22, 2016. https://doi.org/10.1145/2901739.2901754. ACM, USA, pp 327–338

  • Benjamini Y, Hochberg Y (1995) Controlling the false discovery rate: A practical and powerful approach to multiple testing. J R Statist Soc Ser B Methodol 57(1):289–300

    MathSciNet  MATH  Google Scholar 

  • Besker T, Martini A, Lokuge RE, Blincoe K, Bosch J (2018) Embracing technical debt, from a startup company perspective. In: 2018 IEEE international conference on software maintenance and evolution, ICSME 2018, September 23-29, 2018, Spain, pp 415–425

  • Brown N, Cai Y, Guo Y, Kazman R, Kim M, Kruchten P, Lim E, MacCormack A, Nord R, Ozkaya I et al (2010) Managing technical debt in software-reliant systems. In: Proceedings of the FSE/SDP workshop on future of software engineering research. ACM

  • Celik A, Knaust A, Milicevic A, Gligoric M (2016) Build system with lazy retrieval for java projects. In: Proceedings of the 2016 24th ACM SIGSOFT international symposium on foundations of software engineering, pp 643–654

  • Chen C, Zhang K, Itoh T (2012) Empirical evidence of tags supporting high-level awareness. In: Luo Y (ed) Cooperative design, visualization, and engineering - 9th international conference, CDVE 2012, September 2-5, 2012. Proceedings, Lecture Notes in Computer Science, vol 7467. Springer, Japan, pp 94–101

  • Cunningham W (1992) The WyCash portfolio management system. In: Addendum to the proceedings on object-oriented programming systems, languages, and applications. ACM

  • Di Nucci D, Palomba F, De Rosa G, Bavota G, Oliveto R, De Lucia A (2017) A developer centered bug prediction model. IEEE Trans Softw Eng 44(1):5–24

    Article  Google Scholar 

  • Ebert F, Castor F, Novielli N, Serebrenik A (2019) Confusion in code reviews: Reasons, impacts, and coping strategies. In: Wang X, Lo D, Shihab E (eds) 26th IEEE international conference on software analysis, evolution and reengineering, SANER 2019, February 24-27, 2019. IEEE, China, pp 49–60

  • Ernst NA, Bellomo S, Ozkaya I, Nord RL, Gorton I (2015) Measure it? manage it? ignore it? software practitioners and technical debt. In: Proceedings of the 2015 10th joint meeting on foundations of software engineering, ESEC/FSE 2015, August 30 - September 4, 2015, Italy, pp 50–60

  • Etco J (2017) todo: Automatically generate new issues. https://todo.jasonet.co/. Accessed 06 May 2020

  • Falessi D, Briand LC, Cantone G, Capilla R, Kruchten P (2013) The value of design rationale information. ACM Trans Softw Eng Methodol 22 (3):21:1–21:32

    Article  Google Scholar 

  • Fisher RA (1922) On the interpretation of chi-square from contingency tables, and the calculation of p. J R Stat Soc 85(1):87–94

    Article  Google Scholar 

  • Fucci G, Zampetti F, Serebrenik A, Di Penta M (2020) Who (self) admits technical debt. In: 2020 IEEE international conference on software maintenance and evolution, ICSME 2020. IEEE

  • Fucci G, Cassee N, Zampetti F, Novielli N, Serebrenik A, Di Penta M (2021) Waiting around or job half-done? sentiment in self-admitted technical debt. In: International conference on mining software repositories. IEEE Computer Society, United States

  • Grissom RJ, Kim JJ (2005) Effect sizes for research: A broad practical approach, 2nd edn. Lawrence Earlbaum Associates, Mahwah

    Google Scholar 

  • Guzzi A (2012) Documenting and sharing knowledge about code. In: 2012 34th international conference on software engineering (ICSE). IEEE, pp 1535–1538

  • Huang Q, Shihab E, Xia X, Lo D, Li S (2018) Identifying self-admitted technical debt in open source projects using text mining. Empir Softw Eng 23(1):418–451

    Article  Google Scholar 

  • Iammarino M, Zampetti F, Aversano L, Di Penta M (2019) Self-admitted technical debt removal and refactoring actions: Co-occurrence or more?. In: 2019 IEEE international conference on software maintenance and evolution, ICSME 2019, September 29 - October 4, 2019. IEEE, USA, pp 186–190

  • Jiang S, Armaly A, McMillan C (2017) Automatically generating commit messages from diffs using neural machine translation. In: Proceedings of the 32nd IEEE/ACM international conference on automated software engineering, ASE 2017, October 30 - November 03, 2017, USA, pp 135–146

  • Kruchten P, Nord RL, Ozkaya I, Falessi D (2013) Technical debt: towards a crisper definition report on the 4th international workshop on managing technical debt. ACM SIGSOFT Softw Eng Notes

  • Kruskal WH, Wallis WA (1952) Use of ranks in one-criterion variance analysis. J Am Stat Assoc 47(260):583–621. http://www.jstor.org/stable/2280779

    Article  Google Scholar 

  • Li Z, Avgeriou P, Liang P (2015) A systematic mapping study on technical debt and its management. J Syst Softw 101:193–220

    Article  Google Scholar 

  • Lim E, Taksande N, Seaman C (2012) A balancing act: what software practitioners have to say about technical debt. IEEE Softw

  • Lin B, Zampetti F, Bavota G, Di Penta M, Lanza M (2019) Pattern-based mining of opinions in q&a websites. In: Proceedings of the 41st international conference on software engineering, ICSE 2019, May 25-31, 2019, Canada, pp 548–559

  • Maipradit R, Treude C, Hata H, Matsumoto K (2020) Wait for it: identifying “on-hold” self-admitted technical debt. Empir Softw Eng :1–29

  • Mensah S, Keung J, Svajlenko J, Bennin KE, Mi Q (2018) On the value of a prioritization scheme for resolving self-admitted technical debt. J Syst Softw

  • Moreno L, Bavota G, Di Penta M, Oliveto R, Marcus A, Canfora G (2017) ARENA: an approach for the automated generation of release notes. IEEE Trans Software Eng 43(2):106–127

    Article  Google Scholar 

  • Newcombe RG (1998) Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat Med 17(8):857–872

    Article  Google Scholar 

  • Padioleau Y, Tan L, Zhou Y (2009) Listening to programmers - taxonomies and characteristics of comments in operating system code. In: Proceedings 31st international conference on software engineering, ICSE 2009, May 16-24, 2009. IEEE, Canada, pp 331–341

  • Potdar A, Shihab E (2014) An exploratory study on self-admitted technical debt. In: 30th IEEE international conference on software maintenance and evolution, September 29 - October 3, 2014, Canada, pp 91–100

  • Rantala L, Mäntylä M, Lo D (2020) Prevalence, contents and automatic detection of kl-satd. arXiv:200805159

  • Rastkar S, Murphy GC, Murray G (2014) Automatic summarization of bug reports. IEEE Trans Softw Eng 40(4):366–380

    Article  Google Scholar 

  • Ren X, Xing Z, Xia X, Lo D, Wang X, Grundy J (2019) Neural network-based detection of self-admitted technical debt: From performance to explainability. ACM Trans Softw Eng Methodol 28(3):15

    Article  Google Scholar 

  • da S Maldonado E, Shihab E (2015) Detecting and quantifying different types of self-admitted technical debt. In: 7th IEEE international workshop on managing technical debt, MTD 2015, October 2, 2015, Germany, pp 9–15

  • da S Maldonado E, Abdalkareem R, Shihab E, Serebrenik A (2017a) An empirical study on the removal of self-admitted technical debt. In: 2017 IEEE international conference on software maintenance and evolution, ICSME 2017, September 17-22, 2017, China, pp 238–248

  • da S Maldonado E, Shihab E, Tsantalis N (2017b) Using natural language processing to automatically detect self-admitted technical debt. IEEE Trans Software Eng 43(11):1044–1062

    Article  Google Scholar 

  • Seaman C, Guo Y (2011) Measuring and monitoring technical debt. Adv Comput

  • Spencer D (2009) Card sorting: Designing usable categories. Rosenfeld Media

  • Steele CM, Aronson J (1995) Stereotype threat and the intellectual test performance of african americans. J Pers Soc Psychol 69(5):797–811

    Article  Google Scholar 

  • Storey MA, Cheng LT, Bull I, Rigby P (2006) Shared waypoints and social tagging to support collaboration in software development. In: Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work, pp 195–198

  • Storey MA, Ryall J, Bull RI, Myers D, Singer J (2008) TODO or to Bug: Exploring how task annotations play a role in the work practices of software developers. In: Proceedings of the 30th international conference on software engineering, association for computing machinery, USA, pp 251–260

  • Storey MA, Ryall J, Singer J, Myers D, Cheng L, Muller MJ (2009) How software developers use tagging to support reminding and refinding. IEEE Trans Softw Eng 35(4):470–483

    Article  Google Scholar 

  • Tan SH, Marinov D, Tan L, Leavens GT (2012) @tcomment: Testing javadoc comments to detect comment-code inconsistencies. In: Fifth IEEE international conference on software testing, verification and validation, ICST 2012, April 17-21, 2012, Canada, pp 260–269

  • Torchiano M, Ricca F, Marchetto A (2011) Is my project’s truck factor low? theoretical and empirical considerations about the truck factor threshold. In: Proceedings of the 2nd international workshop on emerging trends in software metrics, pp 12–18

  • Uddin G, Khomh F (2017) Opiner: an opinion search and summarization engine for APIs. In: Proceedings of the 32nd IEEE/ACM international conference on automated software engineering, ASE 2017, October 30 - November 03, 2017, USA, pp 978–983

  • Vassallo C, Zampetti F, Romano D, Beller M, Panichella A, Di Penta M, Zaidman A (2016) Continuous delivery practices in a large financial organization. In: 2016 IEEE international conference on software maintenance and evolution, ICSME 2016 October 2-7, 2016. IEEE Computer Society, USA, pp 519–528

  • Vassallo C, Proksch S, Gall HC, Di Penta M (2019) Automated reporting of anti-patterns and decay in continuous integration. In: 2019 IEEE/ACM 41st international conference on software engineering (ICSE). IEEE, pp 105–115

  • Wehaibi S, Shihab E, Guerrouj L (2016) Examining the impact of self-admitted technical debt on software quality. In: IEEE 23rd international conference on software analysis, evolution, and reengineering, SANER 2016, March 14-18, 2016. IEEE Computer Society, Japan, pp 179–188

  • Wei L, Liu Y, Cheung SC (2017) Oasis: prioritizing static analysis warnings for android apps based on app user reviews. In: Proceedings of the 2017 11th joint meeting on foundations of software engineering, pp 672–682

  • Wen F, Nagy C, Bavota G, Lanza M (2019) A large-scale empirical study on code-comment inconsistencies. In: 2019 IEEE/ACM 27th international conference on program comprehension (ICPC). IEEE, pp 53–64

  • Wilcoxon F (1945) Individual comparisons by ranking methods. Biom Bull 1(6):80–83

    Article  Google Scholar 

  • Xavier L, Ferreira F, Brito R, Valente MT (2020) Beyond the code: Mining self-admitted technical debt in issue tracker systems. In: 17th International Conference on Mining Software Repositories (MSR), pp 137–146

  • Zampetti F, Noiseux C, Antoniol G, Khomh F, Di Penta M (2017) Recommending when design technical debt should be self-admitted. In: 2017 IEEE international conference on software maintenance and evolution, ICSME 2017, September 17-22, 2017, China, pp 216–226

  • Zampetti F, Serebrenik A, Di Penta M (2018) Was self-admitted technical debt removal a real removal?: an in-depth perspective. In: Proceedings of the 15th international conference on mining software repositories, MSR 2018, May 28-29, 2018, Sweden, pp 526–536

  • Zampetti F, Serebrenik A, Di Penta M (2020) Automatically learning patterns for self-admitted technical debt removal. In: 2020 IEEE 27th international conference on software analysis evolution and reengineering (SANER), pp 355–366

  • Zampetti F, Fucci G, Serebrenik A, Di Penta M (2021) Dataset of the paper “self-admitted technical debt practices: A comparison between industry and open- source”. https://doi.org/10.5281/zenodo.5076096

  • Zazworka N, Shaw MA, Shull F, Seaman CB (2011) Investigating the impact of design debt on software quality. In: Proceedings of the 2nd workshop on managing technical debt, MTD 2011, May 23, 2011, USA, pp 17–23

  • Zimmermann T (2016) Card-sorting: From text to themes. In: Menzies T, Williams L, Zimmermann T (eds) Perspectives on data science for software engineering. Morgan Kaufmann, Boston, pp 137–141

Download references

Author information

Authors and Affiliations

  1. University of Sannio, Via Traiano, 3, Benevento, Italy

    Fiorella Zampetti, Gianmarco Fucci & Massimiliano Di Penta

  2. Eindhoven University of Technology, Eindhoven, The Netherlands

    Alexander Serebrenik

Authors
  1. Fiorella Zampetti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gianmarco Fucci
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexander Serebrenik
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Massimiliano Di Penta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fiorella Zampetti.

Additional information

Communicated by: Lin Tan

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zampetti, F., Fucci, G., Serebrenik, A. et al. Self-admitted technical debt practices: a comparison between industry and open-source. Empir Software Eng 26, 131 (2021). https://doi.org/10.1007/s10664-021-10031-3

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10664-021-10031-3

Keywords

Search

Navigation