Advertisement

Commenting source code: is it worth it for small programming tasks?

  • Sebastian NielebockEmail author
  • Dariusz Krolikowski
  • Jacob Krüger
  • Thomas Leich
  • Frank Ortmeier
Article

Abstract

Maintaining a program is a time-consuming and expensive task in software engineering. Consequently, several approaches have been proposed to improve the comprehensibility of source code. One of such approaches are comments in the code that enable developers to explain the program with their own words or predefined tags. Some empirical studies indicate benefits of comments in certain situations, while others find no benefits at all. Thus, the real effect of comments on software development remains uncertain. In this article, we describe an experiment in which 277 participants, mainly professional software developers, performed small programming tasks on differently commented code. Based on quantitative and qualitative feedback, we i) partly replicate previous studies, ii) investigate performances of differently experienced participants when confronted with varying types of comments, and iii) discuss the opinions of developers on comments. Our results indicate that comments seem to be considered more important in previous studies and by our participants than they are for small programming tasks. While other mechanisms, such as proper identifiers, are considered more helpful by our participants, they also emphasize the necessity of comments in certain situations.

Keywords

Comments Program comprehension Empirical study Documentation Maintenance 

Notes

Acknowledgments

This research is supported by DFG grant LE 3382/2-1.

References

  1. Ali N, Sharafi Z, Guéhéneuc YG, Antoniol G (2015) An empirical study on the importance of source code entities for requirements traceability. Empir Softw Eng (EMSE) 20(2):442–478CrossRefGoogle Scholar
  2. Anquetil N, Lethbridge T (1998) Assessing the relevance of identifier names in a legacy software system. In: Proceedings of the 8th conference of the centre for advanced studies on collaborative research (CASCON). IBM, pp 213–222Google Scholar
  3. Antoniol G, Canfora G, Casazza G, De Lucia A, Merlo E (2002) Recovering traceability links between code and documentation. IEEE Trans Softw Eng (TSE) 28(10):970–983CrossRefGoogle Scholar
  4. Basili VR, Shull F, Lanubile F (1999) Building knowledge through families of experiments. IEEE Trans Softw Eng (TSE) 25(4):456–473CrossRefGoogle Scholar
  5. Beck F, Moseler O, Diehl S, Rey GD (2013) In situ understanding of performance bottlenecks through visually augmented code. In: Proceedings of the 21st international conference on program comprehension (ICPC). IEEE, pp 63–72Google Scholar
  6. Bezerra RMM, da Silva FQB, Santana AM, Magalhaes CVC, Santos RES (2015) Replication of empirical studies in software engineering: an update of a systematic mapping study. In: Proceedings of the 9th international symposium on empirical software engineering and measurement (ESEM). IEEE, pp 1–4Google Scholar
  7. Boehm BW (1981) Software engineering economics. Prentice-HallGoogle Scholar
  8. Börstler J, Paech B (2016) The role of method chains and comments in software readability and comprehension—an experiment. IEEE Trans Softw Eng (TSE) 42(9):886–898CrossRefGoogle Scholar
  9. Briand L, Bunse C, Daly J, Differding C (1997) An experimental comparison of the maintainability of object-oriented and structured design documents. In: Proceedings of the 5th international conference on software maintenance (ICSM). IEEE, pp 130–138Google Scholar
  10. Buse RP, Weimer WR (2010) Learning a metric for code readability. IEEE Trans Softw Eng (TSE) 36(4):546–558CrossRefGoogle Scholar
  11. Carver JC (2010) Towards reporting guidelines for experimental replications: a proposal. In: Proceedings of the 1st international workshop on replication in empirical software engineering (RESER)Google Scholar
  12. Chikofsky EJ, Cross JH (1990) Reverse engineering and design recovery: a taxonomy. IEEE Soft 7(1):13–17CrossRefGoogle Scholar
  13. Cook TD, Campbell DT (1979) Quasi-experimentation: design & analysis issues for field settings. Houghton MifflinGoogle Scholar
  14. Corazza A, Maggio V, Scanniello G (2015) On the coherence between comments and implementations in source code. In: Proceedings of the 41st euromicro conference on software engineering and advanced applications (SEAA). IEEE, pp 76–83Google Scholar
  15. Cornelissen B, Holten D, Zaidman A, Moonen L, van Wijk JJ, Van Deursen A (2007) Understanding execution traces using massive sequence and circular bundle views. In: Proceedings of the 15th international conference on program comprehension (ICPC). IEEE, pp 49–58Google Scholar
  16. Dinno A (2015) Nonparametric pairwise multiple comparisons in independent groups using dunn’s test. Stata J 15(1):292–300Google Scholar
  17. Dunsmore HE (1985) The effect of comments, mnemonic names, and modularity: some university experiment results. In: Empirical foundations of information and software science. Springer, pp 189–196Google Scholar
  18. Elshoff JL, Marcotty M (1982) Improving computer program readability to aid modification. Commun ACM 25(8):512–521CrossRefGoogle Scholar
  19. Feigenspan J, Kästner C, Liebig J, Apel S, Hanenberg S (2012) Measuring programming experience. In: Proceedings of the 20th international conference on program comprehension (ICPC). IEEE, pp 73-82Google Scholar
  20. Feigenspan J, Kästner C, Apel S, Liebig J, Schulze M, Dachselt R, Papendieck M, Leich T, Saake G (2013) Do background colors improve program comprehension in the# ifdef hell? Empir Softw Eng (EMSE) 18(4):699–745CrossRefGoogle Scholar
  21. Fisher RA (1936) Statistical methods for research workers, 6th edn. Oliver and Boyd Edinbrug, Tweeddale Court London: 33 Paternoste R Row, E.C.Google Scholar
  22. Fluri B, Wursch M, Gall HC (2007) Do code and comments co-evolve? On the relation between source code and comment changes. In: Proceedings of the 14th working conference on reverse engineering (WCRE). IEEE, pp 70–79Google Scholar
  23. Gamma E, Helm R, Johnson R, Vlissides J (1995) Design patterns: elements of reusable object-oriented software. Addison-Wesley, ReadingzbMATHGoogle Scholar
  24. Gosling SD, Vazire S, Srivastava S, John OP (2004) Should we trust web-based studies? a comparative analysis of six preconceptions about internet questionnaires. Am Psychol 59(2):93CrossRefGoogle Scholar
  25. Hanenberg S, Kleinschmager S, Robbes R, Tanter É, Stefik A (2014) An empirical study on the impact of static typing on software maintainability. Empir Softw Eng (EMSE) 19(5):1335–1382CrossRefGoogle Scholar
  26. Hofmeister J, Siegmund J, Holt DV (2017) Shorter identifier names take longer to comprehend. In: Proceedings of the 24th international conference on software analysis, evolution and reengineering (SANER). IEEE, pp 217–227Google Scholar
  27. Höst M, Regnell B, Wohlin C (2000) Using students as subjects - a comparative study of students and professionals in lead-time impact assessment. Empir Softw Eng (EMSE) 5(3):201–214CrossRefGoogle Scholar
  28. Jalali S, Wohlin C (2012) Systematic literature studies: database searches vs. backward snowballing. In: Proceedings of the 6th international symposium on empirical software engineering and measurement (ESEM). ACM, pp 29–38Google Scholar
  29. Jbara A, Feitelson DG (2015) How programmers read regular code: a controlled experiment using eye tracking. In: Proceedings of the 23rd international conference on program comprehension (ICPC). IEEE, pp 244–254Google Scholar
  30. Ji W, Berger T, Antkiewicz M, Czarnecki K (2015) Maintaining feature traceability with embedded annotations. In: Proceedings of the 19th international software product line conference (SPLC). ACM, pp 61–70Google Scholar
  31. Jiang ZM, Hassan AE (2006) Examining the evolution of code comments in PostgreSQL. In: Proceedings of the 3rd working conference on mining software repositories (MSR). ACM, pp 179–180Google Scholar
  32. Juristo N, Vegas S (2009) Using differences among replications of software engineering experiments to gain knowledge. In: Proceedings of the 24th international symposium on empirical software engineering and measurement (ESEM). IEEE, pp 356–366Google Scholar
  33. Khamis N, Witte R, Rilling J (2010) Automatic quality assessment of source code comments: the JavadocMiner. In: Proceedings of the 9th international conference on natural language processing and information system (NLDB). Springer, pp 68–79Google Scholar
  34. Knuth DE (1984) Literate programming. Comput J 27(2):97–111CrossRefGoogle Scholar
  35. Kobayashi K, Kamimura M, Yano K, Kato K, Matsuo A (2013) SArF Map: visualizing software architecture from feature and layer viewpoints. In: Proceedings of the 21st international conference on program comprehension (ICPC). IEEE, pp 43–52Google Scholar
  36. Koenemann J, Robertson SP (1991) Expert problem solving strategies for program comprehension. In: Proceedings of the 9th conference on human factors in computing systems (CHI). ACM, pp 125–130Google Scholar
  37. Kosar T, Mernik M, Carver JC (2012) Program comprehension of domain-specific and general-purpose languages: comparison using a family of experiments. Empir Softw Eng (EMSE) 17(3):276–304CrossRefGoogle Scholar
  38. Kramer D (1999) API documentation from source code comments: a case study of Javadoc. In: Proceedings of the 17th annual international conference on computer documentation (SIGDOC). ACM, pp 147–153Google Scholar
  39. Krüger J, Gu W, Shen H, Mukelabai M, Hebig R, Berger T (2018) Towards a better understanding of software features and their characteristics: a case study of marlin. In: Proceedings of the 12th workshop on variability modelling of software-intensive systems (VaMoS). ACM, pp 105–112Google Scholar
  40. Krüger J, Wiemann J, Fenske W, Saake G, Leich T (2018) Do you remember this source code?. In: Proceedings of the 40th international conference on software engineering (ICSE). ACM, pp 764–775Google Scholar
  41. Kruskal WH, Wallis WA (1952) Use of ranks in one-criterion variance analysis. J Am Stat Assoc 47(260):583–621CrossRefGoogle Scholar
  42. Lawrie D, Morrell C, Feild H, Binkley D (2007) Effective identifier names for comprehension and memory. Innov Syst Softw Eng 3(4):303–318CrossRefGoogle Scholar
  43. Mäder P, Egyed A (2015) Do developers benefit from requirements traceability when evolving and maintaining a software system? Empir Softw Eng (EMSE) 20 (2):413–441CrossRefGoogle Scholar
  44. Martin RC (2009) Clean code: a handbook of agile software craftsmanship. Pearson EducationGoogle Scholar
  45. Martinez M, Monperrus M (2015) Mining software repair models for reasoning on the search space of automated program fixing. Empir Soft Eng (EMSE) 20(1):176–205CrossRefGoogle Scholar
  46. von Mayrhauser A, Vans AM (1995) Program comprehension during software maintenance and evolution. IEEE Comput 28(8):44–55CrossRefGoogle Scholar
  47. McBurney PW, Mcmillan C (2014) Automatic documentation generation via source code summarization of method context. In: Proceedings of the 22nd international conference on program comprehension (ICPC). ACM, pp 279–290Google Scholar
  48. McBurney PW, McMillan C (2016) An empirical study of the textual similarity between source code and source code summaries. Empir Soft Eng (EMSE) 21(1):17–42CrossRefGoogle Scholar
  49. Norcio AF (1982) Indentation, documentation and programmer comprehension. In: Proceedings of the 1st conference on human factors in computing systems (CHI). ACM, pp 118–120Google Scholar
  50. Nurvitadhi E, Leung WW, Cook C (2003) Do class comments aid Java program understanding?. In: Proceedings of the 33rd annual frontiers in education, vol 1. IEEE, pp T3C–T3CGoogle Scholar
  51. Perry DE, Porter AA, Votta LG (2000) Empirical studies of software engineering: a roadmap. In: Proceedings of the conference on the future of software engineering. ACM, pp 345–355Google Scholar
  52. Rahman MM, Roy CK, Keivanloo I (2015) Recommending insightful comments for source code using crowdsourced knowledge. In: Proceedings of the 15th international working conference on source code analysis and manipulation (SCAM). IEEE, pp 81–90Google Scholar
  53. Ratol IK (2017) Detecting fragile comments. In: Proceedings of the 32nd international conference on automated software engineering (ASE). IEEE Press, pp 112–122Google Scholar
  54. Runeson P (2003) Using students as experiment subjects – an analysis on graduate and freshmen student data. In: Proceedings of the 7th international conference on evaluation and assessment in software engineering (EASE). Lund University, pp 95–102Google Scholar
  55. Salviulo F, Scanniello G (2014) Dealing with identifiers and comments in source code comprehension and maintenance: results from an ethnographically-informed study with students and professionals. In: Proceedings of the 18th international conference on evaluation and assessment in software engineering (EASE). ACM, pp 48Google Scholar
  56. Schröter I, Krüger J, Siegmund J, Leich T (2017) Comprehending studies on program comprehension. In: Proceedings of the 25th international conference on program comprehension (ICPC). IEEE, pp 308–311Google Scholar
  57. Seiler M, Paech B (2017) Using tags to support feature management across issue tracking systems and version control systems. In: Requirements engineering: foundation for software quality. Springer, pp 174–180Google Scholar
  58. Shakeel Y, Krüger J, von Nostitz-Wallwitz I, Lausberger C, Campero Durand G, Saake G, Leich T (2018) Automated literature analysis - threats and experiences. In: Proceedings of the international workshop on software engineering for science (SE4Science). ACM, pp 20–27Google Scholar
  59. Sharon D (1996) Meeting the challenge of software maintenance. IEEE Soft 13 (1):122–125CrossRefGoogle Scholar
  60. Sheppard S, Borst M, Curtis B, Love L (1978) Predicting programmers’ ability to modify software. Tech Rep TR—7a—3B8100 3, General electricGoogle Scholar
  61. Siegel S (1956) Nonparametric statistics for the behavioral sciences. McGraw-Hill Kogakusha LTD, TokyozbMATHGoogle Scholar
  62. Siegmund J (2016) Program comprehension: past, present, and future. In: Proceedings of the 23rd international conference on software analysis, evolution and reengineering (SANER). IEEE, pp 13–20Google Scholar
  63. Siegmund J, Siegmund N, Apel S (2015) Views on internal and external validity in empirical software engineering. In: Proceedings of the 37th international conference on software engineering (ICSE), vol 1. IEEE, pp 9–19Google Scholar
  64. Sommerlad P, Zgraggen G, Corbat T, Felber L (2008) Retaining comments when refactoring code. In: Proceedings of the 23rd conference on object-oriented programming, systems, languages and applications (OOPSLA). ACM, pp 653–662Google Scholar
  65. Sridhara G (2016) Automatically detecting the up-to-date status of todo comments in Java programs. In: Proceedings of the 9th India software engineering conference (ISEC). ACM, pp 16–25Google Scholar
  66. Sridhara G, Hill E, Muppaneni D, Pollock L (2010) Towards automatically generating summary comments for Java methods. In: Proceedings of the 25th international conference on automated software engineering (ASE). ACM, pp 43–52Google Scholar
  67. Standish TA (1984) An essay on software reuse. IEEE Trans Soft Eng (TSE) SE-10(5):494–497. https://ieeexplore.ieee.org/abstract/document/5010272 CrossRefGoogle Scholar
  68. Steidl D, Hummel B, Juergens E (2013) Quality analysis of source code comments. In: Proceedings of the 21st international conference on program comprehension (ICPC). IEEE, pp 83–92Google Scholar
  69. Storey MAD (2005) Theories, methods and tools in program comprehension: past, present and future. In: Proceedings of the 13th international workshop on program comprehension (IWPC). IEEE, pp 181–191Google Scholar
  70. Storey MAD, Wong K, Muller HH (1997) How do program understanding tools affect how programmers understand programs?. In: Proceedings of the 4th working conference on reverse engineering (WCRE). IEEE, pp 12–21Google Scholar
  71. Svahnberg M, Aurum A, Wohlin C (2008) Using students as subjects - an empirical evaluation. In: Proceedings of the 2nd international symposium on empirical software engineering and measurement (ESEM). ACM, pp 288–290Google Scholar
  72. Takang AA, Grubb PA, Macredie RD (1996) The effects of comments and identifier names on program comprehensibility: an experimental investigation. J Program Lang (JPL) 4(3):143–167Google Scholar
  73. Tan SH, Marinov D, Tan L, Leavens GT (2012) @tComment: testing JavaDoc comments to detect comment-code inconsistencies. In: Proceedings of the 5th international conference on software testing, verification and validation (ICST). IEEE, pp 260–269. https://www.computer.org/csdl/proceedings/icst/2012/4670/00/4670a260-abs.html
  74. Tenny T (1985) Procedures and comments vs. the banker’s algorithm. ACM SIGCSE Bulletin 17(3):44–53CrossRefGoogle Scholar
  75. Tenny T (1988) Program readability: procedures versus comments. IEEE Trans Soft Eng (TSE) 14(9):1271–1279CrossRefGoogle Scholar
  76. Tiarks R (2011) What maintenance programmers really do: an observational study. In: Proceedings of the 13th workshop on software reengineering, pp 36–37Google Scholar
  77. Trochim WM, Donnelly JP, Arora K (2016) Research methods the essential knowledge base, 2nd edn. Cengage Learning, BostonGoogle Scholar
  78. Trumper J, Dollner J, Telea A (2013) Multiscale visual comparison of execution traces. In: Proceedings of the 21st international conference on program comprehension (ICPC). IEEE, pp 53–62Google Scholar
  79. Vermeulen A (2000) The elements of Java (tm) style. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  80. Wohlin C (2014) Guidelines for snowballing in systematic literature studies and a replication in software engineering. In: Proceedings of the 18th international conference on evaluation and assessment in software engineering (EASE). ACM, pp 1–10Google Scholar
  81. Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B, Wesslén A (2012) Experimentation in software engineering. Springer, BerlinCrossRefGoogle Scholar
  82. Wong E, Yang J, Tan L (2013) Autocomment: mining question and answer sites for automatic comment generation. In: Proceedings of the 28th international conference on automated software engineering (ASE). IEEE, pp 562–567Google Scholar
  83. Woodfield SN, Dunsmore HE, Shen VY (1981) The effect of modularization and comments on program comprehension. In: Proceedings of the 5th international conference on software engineering (ICSE). IEEE, pp 215–223Google Scholar
  84. Ying AT, Wright JL, Abrams S (2005) Source code that talks: an exploration of eclipse task comments and their implication to repository mining. ACM SIGSOFT Soft Eng Notes (SEN) 30(4):1–5Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Otto-von-Guericke-University MagdeburgMagdeburgGermany
  2. 2.Otto-von-Guericke-University Magdeburg & Harz University of Applied Sciences WernigerodeMagdeburgGermany
  3. 3.Harz University of Applied Sciences Wernigerode & Metop GmbH MagdeburgMagdeburgGermany

Personalised recommendations