Skip to main content
SpringerLink
Log in

Measuring code comprehension effort using code reading pattern

  • Published:
Sādhanā Aims and scope Submit manuscript

Abstract

In the software industry, 85% of development tasks involve maintenance. A developer’s typical job involves more code reading and less code writing. Since code reading is an integral part of comprehension, efficient code reading emphasizes a better understanding of code execution. The more efficiently a code is read, the more it reduces maintenance time. Thus, the developer’s reading pattern can help to estimate how efficiently a developer can read and comprehend code that can improve software productivity. Quantifying the comprehension effort spent while reading a code can help to explore the developer’s efficiency. The primary goal of this study is to quantify the comprehension effort using code reading patterns. An eye-tracking sensor tracks the developer’s eye movements while reading the code. The backward gaze transitions and internal navigation are identified from the reading pattern of the code and used to compute program comprehension effort. An experiment was to collect the data where 41 subjects were asked to read five short C programs. The findings of the study infer a significant correlation between comprehension effort and fixation points. This study explores the involvement of non-experts in higher comprehension efforts than expert ones.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13

Similar content being viewed by others

Notes

  1. https://www.tobiipro.com/product-listing/nano/.

  2. http://www.ogama.net/.

  3. Link for the dataset used in this work is provided here: https://github.com/sayaniiit/MCCERP.

  4. https://en.wikipedia.org/wiki/Education_in_India.

  5. https://www.scholaro.com/pro/countries/India/Education-System.

  6. https://statistics.laerd.com/spss-tutorials/mann-whitney-u-test-using-spss-statistics.php.

References

  1. Gilmore D J and Green T R G 1988 Programming plans and programming expertise. Q. J. Exp. Psychol., 40(3): 423–442

    Article  Google Scholar 

  2. Robson D J, Bennett K H, Cornelius B J and Munro M 1991 Approaches to program comprehension. J. Syst. Softw., 14(2): 79–84

    Article  Google Scholar 

  3. Blascheck T, Kurzhals K, Raschke M, Burch M, Weiskopf D and Ertl T 2017 Visualization of eye tracking data: A taxonomy and survey. In: Computer Graphics Forum, vol. 36. Wiley Online Library, pp. 260–284

  4. Clark B and Sharif B 2017 itracevis: Visualizing eye movement data within eclipse. In: 2017 IEEE Working Conference on Software Visualization (VISSOFT). IEEE, pp. 22–32

  5. Kumar A, Burch M, van den Brand I, Castelijns L, Ritchi F, Rooks F, de Smeth H, Timmermans N and Mueller K 2018 Eye tracking for exploring visual communication differences. In: An IEEE VIS Workshop on Visualization for Communication (VisComm)

  6. Amann S, Proksch S, Nadi S and Mezini M 2016 A study of visual studio usage in practice. In: 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), vol. 1. IEEE, pp. 124–134

  7. Mondal S and Das P P 2015 An IDE-agnostic system to capture reading behaviour of c++ programs using eye-gaze tracker. In: 2015 Fifth National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics (NCVPRIPG). IEEE, pp. 1–4

  8. Kevic K, Walters B M, Shaffer T R, Sharif B, Shepherd D C and Fritz T 2015 Tracing software developers’ eyes and interactions for change tasks. In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. ACM, pp. 202–213

  9. Fritz T, Shepherd D C, Kevic K, Snipes W and Bräunlich C 2014 Developers’ code context models for change tasks. In: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 7–18

  10. Bednarik R, Myller N, Sutinen E and Tukiainen M 2006 Analyzing individual differences in program comprehension. Technol. Instruct. Cognit. Learn., 3(3/4): 205

    Google Scholar 

  11. De Smet B, Lempereur L, Sharafi Z, Guéhéneuc Y G, Antoniol G and Habra N 2011 Taupe: Visualising and analysing eye tracking data. Sci. Comput. Program.

  12. Shaffer T R, Wise J L, Walters B M, Müller S C, Falcone M and Sharif B 2015 itrace: Enabling eye tracking on software artifacts within the ide to support software engineering tasks. In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. ACM, pp. 954–957

  13. Shima K, Takemura Y and Matsumoto K 2002 An approach to experimental evaluation of software understandability. In: Proceedings International Symposium on Empirical Software Engineering. IEEE, pp. 48–55

  14. Misra S and Akman I 2008 Comparative study of cognitive complexity measures. In: 2008 23rd International Symposium on Computer and Information Sciences. IEEE, pp. 1–4

  15. Rahman A 2018 Comprehension effort and programming activities: related? or not related?. In: 2018 IEEE/ACM 15th International Conference on Mining Software Repositories (MSR). IEEE, pp. 66–69

  16. Scalabrino S, Bavota G, Vendome C, Linares-Vásquez M, Poshyvanyk D and Oliveto R 2017 Automatically assessing code understandability: How far are we?. In: 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, pp. 417–427

  17. Scalabrino S, Bavota G, Vendome C, Poshyvanyk D and Oliveto R 2019 Automatically assessing code understandability. IEEE Trans. Softw. Eng.

  18. Muñoz B M, Wyrich M, and Wagner S 2020 An empirical validation of cognitive complexity as a measure of source code understandability. In: Proceedings of the 14th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), pp. 1–12

  19. Koenemann J and Robertson S P 1991 Problem solving strategies for program comprehension. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, pp. 125–130

  20. Feigenspan J, Siegmund N and Fruth J 2011 On the role of program comprehension in embedded systems. In: Proceedings Workshop Software Reengineering (WSR), pp. 34–35

  21. Siegmund J, Brechmann A, Apel S, Kästner C, Liebig J, Leich T and Saake G 2012 Toward measuring program comprehension with functional magnetic resonance imaging. In: Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering. ACM, p. 24

  22. Turner R, Falcone M, Sharif B and Lazar A 2014 An eye-tracking study assessing the comprehension of c++ and python source code. In: Proceedings of the Symposium on Eye Tracking Research and Applications. ACM, pp. 231–234

  23. Sarkar A 2015 The impact of syntax colouring on program comprehension. In: PPIG, p. 8

  24. Nakayama M and Harada H 2020 Eye movement features in response to comprehension performance during the reading of programs. In: ACM Symposium on Eye Tracking Research and Applications, pp. 1–5

  25. Salvucci D D and Goldberg J H 2000 Identifying fixations and saccades in eye-tracking protocols. In: Proceedings of the 2000 Symposium on Eye Tracking Research & Applications. ACM, pp. 71–78

  26. Likas A, Vlassis N and Verbeek J J 2003 The global k-means clustering algorithm. Pattern Recognit., 36(2): 451–461

    Article  Google Scholar 

  27. Abdi H and Williams L J 2010 Principal component analysis. Wiley Interdiscip. Rev. Comput. Stat., 2(4): 433–459

    Article  Google Scholar 

  28. Benesty J, Chen J, Huang Y and Cohen I 2009 Pearson correlation coefficient. In: Noise Reduction in Speech Processing. Springer, pp. 1–4

  29. Halstead M H 1977 Elements of Software Science (Operating and Programming Systems Series). Elsevier Science Inc.

Download references

Author information

Authors and Affiliations

  1. Rajendra Mishra School of Engineering Entrepreneurship, Indian Institute of Technology Kharagpur, Kharagpur, India

    Sayani Mondal & Titas Bhattacharjee Rudra

  2. Department of Computer Science and Engineering, Indian Institute of Technology Kharagpur, Kharagpur, India

    Partha Pratim Das

Authors
  1. Sayani Mondal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Partha Pratim Das
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Titas Bhattacharjee Rudra
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sayani Mondal.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mondal, S., Das, P.P. & Bhattacharjee Rudra, T. Measuring code comprehension effort using code reading pattern. Sādhanā 47, 117 (2022). https://doi.org/10.1007/s12046-022-01876-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s12046-022-01876-5

Keywords

Search

Navigation