Skip to main content
SpringerLink
Log in

How programmers read regular code: a controlled experiment using eye tracking

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Regular code, which includes repetitions of the same basic pattern, has been shown to have an effect on code comprehension: a regular function can be just as easy to comprehend as a non-regular one with the same functionality, despite being significantly longer and including more control constructs. It has been speculated that this effect is due to leveraging the understanding of the first instances to ease the understanding of repeated instances of the pattern. To verify and quantify this effect, we use eye tracking to measure the time and effort spent reading and understanding regular code. The experimental subjects were 18 students and 2 faculty members. The results are that time and effort invested in the initial code segments are indeed much larger than those spent on the later ones, and the decay in effort can be modeled by an exponential model. This shows that syntactic code complexity metrics (such as LOC and MCC) need to be made context-sensitive, e.g. by giving reduced weight to repeated segments according to their place in the sequence. However, it is not the case that repeated code segments are actually read more and more quickly. Rather, initial code segments receive more focus and are looked at more times, while later ones may be only skimmed. Further, a few recurring reading patterns have been identified, which together indicate that in general code reading is far from being purely linear, and exhibits significant variability across experimental subjects.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  • Bednarik R, Tukiainen M (2006) An eye-tracking methodology for characterizing program comprehension processes. In: Proceedings of the 2006 symposium on eye tracking research & applications. ISBN 1-59593-305-0. ACM, New York, pp 125–132, DOI 10.1145/1117309.1117356, (to appear in print)

  • Beller M, Zaidman A, Karpov A (2015) The last line effect. In: Proceedings of the 2015 IEEE 23rd international conference on program comprehension. http://dl.acm.org/citation.cfm?id=2820282.2820317. IEEE Press, Piscataway, pp 240–243

  • Binkley D, Davis M, Lawrie D, Morrell C (2009) To camelCase or under_score. In: IEEE 17th international conference on program comprehension, pp 158–167. doi:10.1109/ICPC.2009.5090039

  • Brooks A, Daly J, Miller J, Roper M, Wood M (1994) Replication’s Role in Experimental Computer Science. Tech. Rep. EFoCS-5-94 [RR/94/172]. University of Strathclyde

  • Busjahn T, Bednarik R, Begel A, Crosby M, Paterson J H, Schulte C, Sharif B, Tamm S (2015) Eye movements in code reading: relaxing the linear order. In: Proceedings of the 2015 IEEE 23rd international conference on program comprehension. http://dl.acm.org/citation.cfm?id=2820282.2820320. IEEE Press, Piscataway, pp 255–265

  • Busjahn T, Schulte C, Busjahn A (2011) Analysis of code reading to gain more insight in program comprehension. In: Proceedings of the 11th Koli calling international conference on computing education research. ISBN 978-1-4503-1052-9, 10.1145/2094131.2094133. ACM, New York, pp 1–9

  • Cornelissen B, Zaidman A, van Deursen A (2011) A controlled experiment for program comprehension through trace visualization. IEEE Trans Softw Eng 37 (3):341–355. doi:10.1109/TSE.2010.47

    Article  Google Scholar 

  • Crosby M, Stelovsky J (1990) How do we read algorithms? A case study. Computer 23(1):25–35. doi:10.1109/2.48797

    Article  Google Scholar 

  • Feitelson D G (2015) Using students as experimental subjects in software engineering research – a review and discussion of the evidence. arXiv:1512.08409[cs.SE]

  • Fenton N E, Pfleeger S L (1998) Software metrics: a rigorous and practical approach. Course technology, 2nd edn.

  • Fritz T, Begel A, Müller S C, Yigit-Elliott S, Züger M (2014) Using psycho-physiological measures to assess task difficulty in software development. In: Proceedings of the 36th international conference on software engineering. ISBN 978-1-4503-2756-5, 10.1145/2568225.2568266. ACM, New York, pp 402–413

  • Gilchrist I D, Harvey M (2000) Refixation frequency and memory mechanisms in visual search. Current Biol 10(19):1209–1212. doi:10.1016/S0960-9822(00)00729-6

    Article  Google Scholar 

  • Goldberg J H, Kotval X P (1999) Computer interface evaluation using eye movements: methods and constructs. Int J Indus Ergonom 24 (6):631–645. doi:10.1016/S0169-8141(98)00068-7

    Article  Google Scholar 

  • Halstead M (1977) Elements of software science. Elsevier Science Inc.

  • Holmqvist K, Nyström M, Andersson R, Dewhurst R, Jarodzka H, Van de Weijer J (2011) Eye tracking: a comprehensive guide to methods and measures. Oxford University Press

  • Imazato A, Sasaki Y, Higo Y, Kusumoto S (2013) Improving process of source code modification focusing on repeated code. In: Heidrich J, Oivo M, Jedlitschka A, Baldassarre M (eds) Product-focused software process improvement. Lecture Notes in Computer Science. ISBN 978-3-642-39258-0, 10.1007/978-3-642-39259-7_24, vol 7983. Springer, Berlin Heidelberg, pp 298–312

  • Jbara A, Feitelson D G (2014) Quantification of code regularity using preprocessing and compression. Manuscript

  • Jbara A, Feitelson D G (2014a) On the effect of code regularity on comprehension. In: Proceedings of the 22nd international conference on program comprehension. ISBN 978-1-4503-2879-1, 10.1145/2597008.2597140. ACM, New York, pp 189–200

  • Jbara A, Feitelson D G (2014b) JCSD: visual support for understanding code control structure. In: Proceedings of the 22Nd international conference on program comprehension. ISBN 978-1-4503-2879-1, 10.1145/2597008.2597801. ACM, New York, pp 300–303

  • Jbara A, Matan A, Feitelson D (2014c) High-MCC functions in the Linux kernel. Empir Softw Eng 19(5):1261–1298. doi:10.1007/s10664-013-9275-7

  • Just M, Carpenter P (1980) A theory of reading: from eye fixations to comprehension. Psychol Rev 87:329–354

    Article  Google Scholar 

  • Krein J L, Pratt L, Swenson A, MacLean A, Knutson C D, Eggett D (2011) Design patterns in software maintenance: an experiment replication at Brigham Young University. In: 2nd Intl. workshop replication in empirical software engineering research. doi:10.1109/RESER.2011.10, pp 25–34

  • McCabe T (1976) A complexity measure. IEEE Trans Softw Eng 2 (4):308–320. doi:10.1109/TSE.1976.233837 10.1109/TSE.1976.233837

    Article  MathSciNet  MATH  Google Scholar 

  • Mennie N, Hayhoe M, Sullivan B (2007) Look-ahead fixations: anticipatory eye movements in natural tasks. Exper Brain Res 179:427–442

    Article  Google Scholar 

  • Mondal M, Roy C K, Schneider K A (2015) A comparative study on the bug-proneness of different types of code clones. In: 2015 IEEE International conference on software maintenance and evolution (ICSME), pp 91–100. doi:10.1109/ICSM.2015.7332455

  • Murray W S, Kennedy A (1988) Spatial coding in the processing of anaphor by good and poor readers: evidence from eye movement analyses. Q J Exper Psychol: Human Exper Psychol 40:693–718+

    Article  Google Scholar 

  • Nagappan N, Ball T, Zeller A (2006) Mining metrics to predict component failures. In: 28th Intl. Conf. Softw. Eng., pp 452–461. doi:10.1145/1134285.1134349

  • Oman P, Hagemeister J (1994) Construction and testing of polynomials predicting software maintainability. J Syst Softw 24(3):251–266. doi:10.1016/0164-1212(94)90067-1

    Article  Google Scholar 

  • Pelz J B, Canosa R, Babcock J, Barber J (2001) Visual perception in familiar, complex tasks. In: Proceedings of the 2001 international conference on image processing, pp 12–15

  • Rayner K, Fischer M H (1996) Mindless reading revisited: eye movements during reading and scanning are different. Percept Psychophys 58(5):734–747. doi:10.3758/BF03213106

    Article  Google Scholar 

  • Renshaw J A, Finlay J E, Tyfa D, Ward R D (2004) Regressions re-visited: a new definition for the visual display paradigm. In: CHI ’04 Extended abstracts on human factors in computing systems. doi:10.1145/985921.986084. ACM, New York, pp 1437–1440

  • Rodeghero P, McMillan C, McBurney P W, Bosch N, D’Mello S (2014) Improving automated source code summarization via an eye-tracking stud y of programmers. In: Proceedings of the 36th international conference on software engineering. ISBN 978-1-4503-2756-5, 10.1145/2568225.2568247. ACM, New York, pp 390–401

  • Rodeghero P, McMillan C (2015) An empirical study on the patterns of eye movement during summarization tasks. In: 2015 ACM/IEEE International symposium on empirical software engineering and measurement (ESEM), pp 1–10. doi:10.1109/ESEM.2015.7321188

  • Sasaki Y, Ishihara T, Hotta K, Hata H, Higo Y, Igaki H, Kusumoto S (2012) Preprocessing of metrics measurement based on simplifying program structures. In: 19th Asia-Pacific software engineering conference (APSEC), vol 2, pp 120–127. doi:10.1109/APSEC.2012.59

  • Schneider T D, Stephens R M (1990) Sequence logos: a new way to display consensus sequences. Nucleic Acids Res 18

  • Shao J, Wang Y (2003) A new measure of software complexity based on cognitive weights. Canad J Electric Comput Eng 28(2):69–74. doi:10.1109/CJECE.2003.1532511

    Article  Google Scholar 

  • Sharif B, Falcone M, Maletic J I (2012) An eye-tracking study on the role of scan time in finding source code defects. In: Proceedings of the symposium on eye tracking research and applications. ISBN 978-1-4503-1221-9, 10.1145/2168556.2168642. ACM, New York, pp 381–384

  • Sharif B, Maletic J (2010a) An eye tracking study on camelCase and under_score identifier styles. In: IEEE 18th International conference on program comprehension (ICPC), pp 196–205. doi:10.1109/ICPC.2010.41

  • Sharif B, Maletic J (2010b) An eye tracking study on the effects of layout in understanding the role of design patterns. In: IEEE International conference on software maintenance (ICSM), pp 1–10. doi:10.1109/ICSM.2010.5609582

  • Shneiderman B (1977) Measuring computer program quality and comprehension. Intl J Man-Mach Stud 9(4)

  • Siegmund J, Kästner C, Apel S, Parnin C, Bethmann A, Leich T, Saake G, Brechmann A (2014) Understanding understanding source code with functional magnetic resonance imaging. In: Proceedings of the 36th international conference on software engineering. ISBN 978-1-4503-2756-5, 10.1145/2568225.2568252. ACM, New York, pp 378–389

  • Soloway E, Ehrlich K (1984) Empirical studies of programming knowledge. IEEE Trans Softw Eng SE-10(5):595–609. doi:10.1109/TSE.1984.5010283

    Article  Google Scholar 

  • Uwano H, Nakamura M, Monden A, Matsumoto K-i (2006) Analyzing individual performance of source code review using reviewers’ eye movement. In: Proceedings of the 2006 symposium on eye tracking research & applications. ISBN 1-59593-305-0, 10.1145/1117309.1117357. ACM, New York, pp 133–140

  • Vinju J J, Godfrey M W (2012) What does control flow really look like? Eyeballing the cyclomatic complexity metric. In: 12th Working conf. source code analysis and manipulation

  • Wang H, Chignell M, Ishizuka M (2006) Empathic tutoring software agents using real-time eye tracking. In: Proceedings of the 2006 symposium on eye tracking research & applications. ISBN 1-59593-305-0, 10.1145/1117309.1117346. ACM, New York, pp 73–78

  • Welker K D, Oman P W, Atkinson G G (1997) Development and application of an automated source code maintainability index. J Softw Maint 9(3):127–159. doi: 10.1002/(SICI)1096-908X%28199705%299:3%3C127::AID-SMR149%3E3.0.CO;2-S

  • Weyuker E J (1988) Evaluating software complexity measures. IEEE Trans Softw Eng 14(9):1357–1365. doi:10.1109/32.6178

    Article  MathSciNet  Google Scholar 

  • Yusuf S, Kagdi H, Maletic J (2007) Assessing the comprehension of UML class diagrams via eye tracking. In: 15th IEEE International conference on program comprehension, pp 113–122. doi:10.1109/ICPC.2007.10

Download references

Acknowledgments

This research was supported by the ISRAEL SCIENCE FOUNDATION (grant no. 407/13). Many thanks to the reviewers of this extended version who helped to improve the paper considerably in terms of analysis and presentation relative to the original conference version.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Hebrew University, 91904, Jerusalem, Israel

    Ahmad Jbara & Dror G. Feitelson

  2. School of Mathematics and Computer Science, Netanya Academic College, 42100, Netanya, Israel

    Ahmad Jbara

Authors
  1. Ahmad Jbara
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dror G. Feitelson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ahmad Jbara.

Additional information

Communicated by: Christian Bird and Rocco Oliveto

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jbara, A., Feitelson, D.G. How programmers read regular code: a controlled experiment using eye tracking. Empir Software Eng 22, 1440–1477 (2017). https://doi.org/10.1007/s10664-016-9477-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-016-9477-x

Keywords

Search

Navigation