Skip to main content
SpringerLink
Log in

What constitutes debugging? An exploratory study of debugging episodes

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

When debugging, developers engage in activities such as navigating, editing, testing, and inspecting code. Despite being the building blocks of debugging, little is known about how they constitute debugging. To address this gap, we introduce the concept of a “debugging episode” which encompasses the time from when a developer first investigates a defect until when the defect is fixed or the developer stops. We observed 11 professional developers working on open source projects, coding 89 debugging episodes and 2135 instances of activities that occurred during them. Six activities were identified: navigate, edit, test, inspect, consult resources, and other miscellaneous activities. We found that developers spent the most time editing (41%) and testing (29%) during debugging. When addressing time-consuming defects, developers engaged in more diverse types of debugging activities, spent more time inspecting program state, navigating code, and consulting external resources, and spent less time testing. We found that the activities developers do while debugging were more similar than different than the activities that make up implementation work. Developers spent a similar fraction of their time editing and navigating during debugging and implementation work. However, debugging involved significantly more time inspecting (16%) than implementation work (2%), while implementation work involved more time consulting resources (24%) than debugging (6%). We conducted semi-structured interviews with ten developers to gain insights into the challenges that cause developers in longer debugging episodes to engage with more activities. Our findings offer insight into the debugging process and the challenges that developers confront, offering implications for the design of debugging tools, improved debugging education, and future research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The dataset supporting the conclusions of this research is included within this paper.

Notes

  1. https://github.com/downshift-js/downshift.

References

  • Replication package (2022). URL https://figshare.com/s/0e9eac98b8ddd54c384c

  • Abad ZSH, Karras O, Schneider K, Barker K, Bauer M (2018) Task interruption in software development projects: What makes some interruptions more disruptive than others? In: International Conference on Evaluation and Assessment in Software Engineering pp. 122–132

  • Afzal A, Goues CL (2018) A study on the use of ide features for debugging. In: Proceedings of the 15th International Conference on Mining Software Repositories p. 114–117

  • Alaboudi A, LaToza TD (2019a) An exploratory study of live-streamed programming. In: IEEE Symposium on Visual Languages and Human-Centric Computing pp. 5–13

  • Alaboudi A, LaToza TD (2019b) Supporting software engineering research and education by annotating public videos of developers programming. In: International Workshop on Cooperative and Human Aspects of Software Engineering pp. 117–118

  • Alaboudi A, LaToza TD (2020) Using hypotheses as a debugging aid. In: 2020 IEEE Symposium on Visual Languages and Human-Centric Computing pp. 1–9

  • Alaboudi A, LaToza TD (2021) Edit-run behavior in programming and debugging. In: IEEE Symposium on Visual Languages and Human-Centric Computing

  • Amann S, Proksch S, Nadi S, Mezini M (2016) A study of visual studio usage in practice. In: IEEE International Conference on Software Analysis, Evolution, and Reengineering (SANER), pp. 124–134

  • Baltes S, Moseler O, Beck F, Diehl S (2015) Navigate, understand, communicate: How developers locate performance bugs. In: ACM/IEEE International Symposium on Empirical Software Engineering and Measurement pp. 1–10

  • Beller M, Spruit N, Spinellis D, Zaidman A (2018) On the dichotomy of debugging behavior among programmers. In: International Conference on Software Engineering pp. 572–583

  • Böhme M, Soremekun EO, Chattopadhyay S, Ugherughe E, Zeller A (2017) Where is the bug and how is it fixed? an experiment with practitioners. In: The Joint Meeting on Foundations of Software Engineering, pp. 117–128

  • Bragdon A, Zeleznik R, Reiss SP, Karumuri S, Cheung W, Kaplan J, Coleman C, Adeputra F, LaViola Jr, JJ (2010) Code bubbles: a working set-based interface for code understanding and maintenance. In: Conference on Human Factors in Computing Systems pp. 2503–2512

  • Britton T, Jeng L, Carver G, Cheak P, Katzenellenbogen T (2013) Reversible debugging software. Judge Bus. School, Univ. Cambridge, Cambridge, UK, Tech. Rep 229

  • Chatterjee P, Damevski K, Pollock L, Augustine V, Kraft NA (2019) Exploratory study of slack q &a chats as a mining source for software engineering tools. In: International Conference on Mining Software Repositories pp. 490–501

  • Chattopadhyay S, Nelson N, Gonzalez YR, Leon AA, Pandita R, Sarma A (2019) Latent patterns in activities: A field study of how developers manage context. In: International Conference on Software Engineering p. 373–383

  • Coker Z, Widder DG, Le Goues C, Bogart C, Sunshine J (2019) A qualitative study on framework debugging. In: International Conference on Software Maintenance and Evolution pp. 568–579

  • Damevski K, Shepherd DC, Schneider J, Pollock L (2017) Mining sequences of developer interactions in visual studio for usage smells. IEEE Transactions on Software Engineering 43(4):359–371

    Article  Google Scholar 

  • Study dataset: Observe.dev (2022). URL https://bit.ly/3kkbL2W

  • Default dataset: Observe.dev (2022). URL https://bit.ly/3qWdMVA

  • DeMillo RA, Pan H, Spafford EH, DeMillo RA, Pan H, Spafford EH (1996) Critical slicing for software fault localization. In: The International Symposium on Software Testing and Analysis pp. 121–134

  • Eisenstadt M (1993) Tales of debugging from the front lines. In: Empirical Studies of Programmers: Fifth Workshop pp. 86–112

  • Faas T, Dombrowski L, Young A, Miller AD (2018) Watch me code: Programming mentorship communities on twitch.tv. Proceedings of the ACM on Human-Computer Interaction 2, 50:1–50:18

  • Gould JD (1975) Some psychological evidence on how people debug computer programs. International Journal of Man-Machine Studies 7(2):151–182

    Article  MATH  Google Scholar 

  • Gould JD, Drongowski P (1974) An exploratory study of computer program debugging. Human Factors 16(3):258–277

    Article  Google Scholar 

  • Gugerty L, Olson G (1986) Debugging by skilled and novice programmers. In: Conference on Human Factors in Computing Systems pp. 171–174

  • Jiang S, McMillan C, Santelices R (2017) Do programmers do change impact analysis in debugging? Empirical Software Engineering 22:631–669

    Article  Google Scholar 

  • Johnson MS (1982) A software debugging glossary. ACM Sigplan Notices 17(2):53–70

    Article  Google Scholar 

  • Jones JA, Harrold MJ, Stasko J (2002) Visualization of test information to assist fault localization. In: Proceedings of the International Conference on Software Engineering pp. 467–477

  • Katz IR, Anderson JR (1987) Debugging: An analysis of bug-location strategies. Human-Computer Interaction 3(4):351–399

    Article  Google Scholar 

  • Ko AJ, Abraham R, Beckwith L, Blackwell A, Burnett M, Erwig M, Scaffidi C, Lawrance J, Lieberman H, Myers B et al (2011) The state of the art in end-user software engineering. ACM Computing Surveys (CSUR) 43(3):1–44

    Article  Google Scholar 

  • Ko AJ, DeLine R, Venolia G (2007) Information needs in collocated software development teams. In: The International Conference on Software Engineering pp. 344–353

  • Ko AJ, Myers BA, Coblenz MJ, Aung HH (2006) An Exploratory Study of How Developers Seek, Relate, and Collect Relevant Information during Software Maintenance Tasks. IEEE Transactions on Software Engineering 32(12):971–987

    Article  Google Scholar 

  • Lakhani KR, Von Hippel E (2004) How open source software works: “free” user-to-user assistance. In: Produktentwicklung mit virtuellen Communities, pp. 303–339. Springer

  • Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174

    Article  MATH  Google Scholar 

  • LaToza TD, Myers BA (2010a) Developers ask reachability questions. In: The International Conference on Software Engineering, pp. 185–194

  • LaToza TD, Myers BA (2010b) Hard-to-answer questions about code. In: PLATEAU Workshop on Evaluation and Usability of Programming Languages and Tools, pp. 1–6

  • LaToza TD, Venolia G, DeLine R (2006) Maintaining mental models: A study of developer work habits. In: The International Conference on Software Engineering pp. 492–501

  • Lawrance J, Bogart C, Burnett M, Bellamy R, Rector K, Fleming SD (2013) How programmers debug, revisited: An information foraging theory perspective. IEEE Transactions on Software Engineering 39(2):197–215

    Article  Google Scholar 

  • Layman L, Diep M, Nagappan M, DeLine RA (2013) Debugging revisited, toward understanding the debugging needs of contemporary software developers. In: Empirical Software Engineering and Measurement

  • Lukey F (1980) Understanding and debugging programs. International Journal of Man-Machine Studies 12(2):189–202

    Article  Google Scholar 

  • MacLeod L, Storey MA, Bergen A (2015) Code, camera, action: How software developers document and share program knowledge using youtube. In: The International Conference on Program Comprehension pp. 104–114

  • Mamykina L, Manoim B, Mittal M, Hripcsak G, Hartmann B (2011) Design lessons from the fastest q &a site in the west. In: CHI Conference on Human Factors in Computing Systems pp. 2857–2866

  • Mangano N, LaToza TD, Petre M, van der Hoek A (2014) How software designers interact with sketches at the whiteboard. IEEE Transactions on Software Engineering 41(2):135–156

    Article  Google Scholar 

  • Weiser W (1984) Program slicing. In: The International Conference on Software Engineering pp. 439–449

  • Meyer AN, Fritz T, Murphy GC, Zimmermann T (2014) Software developers’ perceptions of productivity. In: Proceedings of the International Symposium on Foundations of Software Engineering pp. 19–29

  • Minelli R, Mocci A, Robbes R, Lanza M (2016) Taming the ide with fine-grained interaction data. In: International Conference on Program Comprehension pp. 1–10

  • Murphy GC, Kersten M, Findlater L (2006) How are java software developers using the eclipse ide? IEEE Softw 23:76–83

    Article  Google Scholar 

  • Parnin C, Orso A (2011) Are automated debugging techniques actually helping programmers? In: The International Symposium on Software Testing and Analysis, pp. 199–209

  • Perscheid M, Siegmund B, Taeumel M, Hirschfeld R (2017) Studying the advancement in debugging practice of professional software developers. Software Quality Journal 25(1):83–110

    Article  Google Scholar 

  • Petrillo F, Guéhéneuc YG, Pimenta M, Freitas CDS, Khomh F (2019) Swarm debugging: The collective intelligence on interactive debugging. Journal of Systems and Software 153:152–174

    Article  Google Scholar 

  • Piorkowski D, Fleming SD, Scaffidi C, Burnett M, Kwan I, Henley AZ, Macbeth J, Hill C, Horvath A (2015) To fix or to learn? how production bias affects developers’ information foraging during debugging. In: 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME) pp. 11–20

  • Piorkowski DJ, Fleming SD, Kwan I, Burnett MM, Scaffidi C, Bellamy RK, Jordahl J (2013) The whats and hows of programmers’ foraging diets. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems pp. 3063–3072

  • Romero P, Du Boulay B, Cox R, Lutz R, Bryant S (2007) Debugging strategies and tactics in a multi-representation software environment. International Journal of Human-Computer Studies 65(12):992–1009

    Article  Google Scholar 

  • Sillito J, Murphy GC, Volder KD (2008) Asking and Answering Questions during a Programming Change Task. IEEE Transactions on Software Engineering 34(4):434–451

    Article  Google Scholar 

  • Singer L, Figueira Filho F, Storey MA (2014) Software engineering at the speed of light: how developers stay current using twitter. In: The International Conference on Software Engineering pp. 211–221

  • de Souza HA, Chaim ML, Kon F (2016) Spectrum-based software fault localization: A survey of techniques, advances, and challenges. arXiv preprint arXiv:1607.04347

  • Vans AM, von Mayrhauser A, Somlo G (1999) Program understanding behavior during corrective maintenance of large-scale software. International Journal of Human-Computer Studies 51(1):31–70

    Article  Google Scholar 

  • Vessey I (1985) Expertise in debugging computer programs: A process analysis. International Journal of Man-Machine Studies 23:459–494

    Article  Google Scholar 

  • Wang Q, Parnin C, Orso A (2015) Evaluating the usefulness of IR-based fault localization techniques. In: The International Symposium on Software Testing and Analysis pp. 1–11

  • Wong WE, Gao R, Li Y, Abreu R, Wotawa F (2016) A survey on software fault localization. IEEE Transactions on Software Engineering 42(8):707–740

    Article  Google Scholar 

  • Xia X, Bao L, Lo D, Li S (2016) “automated debugging considered harmful” considered harmful: A user study revisiting the usefulness of spectra-based fault localization techniques with professionals using real bugs from large systems. In: ICSM IEEE International Conference on Software Maintenance, pp. 267–278

  • Zhang X, Gupta R, Zhang Y (2003) Precise dynamic slicing algorithms. In: The International Conference on Software Engineering pp. 319–329

  • Zhang X, Gupta N, Gupta R (2006) Pruning dynamic slices with confidence. In: PLDI Conference on Programming Language Design and Implementation 6, pp. 169–180

Download references

Funding

This research was funded by NSF award 1845508.

Author information

Authors and Affiliations

  1. Department of Computer Science, George Mason University, Fairfax, VA, USA

    Abdulaziz Alaboudi & Thomas D. LaToza

Authors
  1. Abdulaziz Alaboudi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas D. LaToza
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abdulaziz Alaboudi.

Ethics declarations

Conflicts of Interest/Competing Interests

All authors declare that they have no conflicts of interest.

Additional information

Communicated by: Scott Fleming.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alaboudi, A., LaToza, T.D. What constitutes debugging? An exploratory study of debugging episodes. Empir Software Eng 28, 117 (2023). https://doi.org/10.1007/s10664-023-10352-5

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10664-023-10352-5

Keywords

Search

Navigation