Abstract
Debugging is an intellectually rich and a challenging process when learning a programming language. This process is important for increasing the quality of the program and making it functional. Debugging, by its nature, is thought to be a practice with a state of focus and concentration. This study explored whether the debugging performance could be predicted by attention networks of students studying at the IT department of vocational high schools. 108 vocational high school IT department students participated in the research. First, students’ attention levels were determined by utilizing the Attention Network Test. Second, a Debugging Performance Test was administered to determine their debugging performance. The analysis result indicated that the general attention network statistically predicted the debugging performance but the effect size was found to be small. Furthermore, the singular and dual interactions of the alerting attention network, the orienting attention network, and the executive attention network did not predict debugging performances. Yet, the interactions of the three attention networks on predicting the debugging performances were found to be statistically significant. These findings were discussed together with the existing studies in the literature and suggestions to obtain more in-depth information regarding these results were provoked.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The datasets generated and analyzed during the current study are available from the corresponding author on reasonable request.
References
Ahmadzadeh, M., Elliman, D., & Higgins, C. (2005). An analysis of patterns of debugging among novice computer science students. ACM SIGCSE Bulletin, 37(3), 84–88. https://doi.org/10.1145/1151954.1067472.
Akçay, A., & Altun, A. (2021). Hata ayıklama performansı testi geliştirme: Geçerlik ve güvenirlik çalışması [Test development for debugging performance: Validity and reliability study]. Erzincan University Journal of Education Faculty, 23(3), 667–685. https://doi.org/10.17556/erziefd.815922.
Altadmri, A., & Brown, N. C. C. (2015). 37 million compilations: Investigating novice programming mistakes in large-scale student data. In A. Decker, K. Eiselt, J. Tims, & C. Alphonce (Eds.), Proceedings of the 46th ACM Technical Symposium on Computer Science Education (pp. 522–527). The Association for Computing Machinery. https://doi.org/10.1145/2676723.2677258.
Beckwith, L., Kissinger, C., Burnett, M., Wiedenbeck, S., Lawrance, J., Blackwell, A., & Cook, C. (2006). Tinkering and gender in end-user programmers’ debugging. In R. Grinter, T. Rodden, P. Aoki, E. Cutrell, R. Jeffies, & G. Olson (Eds.), Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 231–240). https://doi.org/10.1145/1124772.1124808.
Bednarik, R. (2012). Expertise-dependent visual attention strategies develop over time during debugging with multiple code representations. International Journal of Human-Computer Studies, 70(2), 143–155. https://doi.org/10.1016/j.ijhcs.2011.09.003.
Bednarik, R., & Tukiainen, M. (2007a). Validating the restricted focus viewer: A study using eye-movement tracking. Behavior Research Methods, 39(2), 274–282. https://doi.org/10.3758/BF03193158.
Bednarik, R., & Tukiainen, M. (2007b). Analysing and interpreting quantitative eye-tracking data in studies of programming: Phases of debugging with multiple representations. 19th Annual Workshop of the Psychology of Programming Interest Group. https://ppig.org/files/2007-PPIG-19th-Bednarik.pdf.
Castelhano, J., Duarte, I. C., Ferreira, C., Duraes, J., Madeira, H., & Castelo-Branco, M. (2019). The role of the insula in intuitive expert bug detection in computer code: An fMRI study. Brain Imaging and Behavior, 13, 623–637. https://doi.org/10.1007/s11682-018-9885-1.
Chandrika, K. R., & Amudha, J. (2017). An eye tracking study to understand the visual perception behavior while source code comprehension. International Journal of Control Theory and Applications, 10(19), 169–175. https://serialsjournals.com/abstract/11377_20-chandrika_k.r.pdf.
Chen, M., & Lim, V. (2013). Eye gaze and mouse cursor relationship in a debugging task. In Stephanidis C. (Ed.), HCI International 2013 - Posters’ Extended Abstracts (pp. 468–472). Springer. https://doi.org/10.1007/978-3-642-39473-7_93.
Ciolkowski, M., Laitenberger, O., Rombach, D. H., Shull, F., & Perry, D. (2002). Software inspections, reviews and walkthroughs. In W. Tracz (Ed.), Proceedings of the 24th International Conference on Software Engineering (pp. 641–642). IEEE. https://doi.org/10.1145/581339.581422.
Dooley, J. (2011). Software development and professional practice. Springer. https://doi.org/10.1007/978-1-4302-3802-7.
Downey, A. B., & Mayfield, C. (2016). Think Java: How to think like a computer scientist. Green Tea Press. https://open.umn.edu/opentextbooks/textbooks/think-java-how-to-think-like-a-computer-scientist.
Ducasse, M., & Emde, A. M. (1988). A review of automated debugging systems: Knowledge, strategies and techniques. In T. N., Nam (Ed.), Proceedings of 11th International Conference on Software Engineering (pp. 162–171). IEEE Computer Society Press. https://doi.org/10.1109/ICSE.1988.93698.
Duraes, J., Madeira, H., Castelhano, J., Duarte, C., & Branco, M. C. (2016). WAP: Understanding the brain at software debugging. In Y. Labiche (Ed.), 2016 IEEE 27th International Symposium on Software Reliability Engineering (pp. 87–92). IEEE. https://doi.org/10.1109/ISSRE.2016.53.
Emhardt, S. N., Kok, E. M., Jarodzka, H., Brand-Gruwel, S., Drumm, C., & van Gog, T. (2020). How experts adapt their gaze behavior when modeling a task to novices. Cognitive Science, 44(9), 1–26. https://doi.org/10.1111/cogs.12893.
Fan, J., McCandliss, B. D., Sommer, T., Raz, A., & Posner, M. I. (2002). Testing the efficiency and independence of attentional networks. Journal of Cognitive Neuroscience, 14(3), 340–347. https://doi.org/10.1162/089892902317361886.
Fan, J., McCandliss, B. D., Fossella, J., Flombaum, J. I., & Posner, M. I. (2005). The activation of attentional networks. Neuroimage, 26(2), 471–479. https://doi.org/10.1016/j.neuroimage.2005.02.004.
Fan, J., Gu, X., Guise, K. G., Liu, X., Fossella, J., Wang, H., & Posner, M. I. (2009). Testing the behavioral interaction and integration of attentional networks. Brain and Cognition, 70(2), 209–220. https://doi.org/10.1016/j.bandc.2009.02.002.
Fields, D. A., Kafai, Y. B., Morales-Navarro, L., & Walker, J. T. (2021). Debugging by design: A constructionist approach to high school students’ crafting and coding of electronic textiles as failure artefacts. British Journal of Educational Technology, 52(3), 1078–1092. https://doi.org/10.1111/bjet.13079.
Finucane, A. M., Whiteman, M. C., & Power, M. J. (2010). The effect of happiness and sadness on alerting, orienting, and executive attention. Journal of Attention Disorders, 13(6), 629–639. https://doi.org/10.1177/1087054709334514.
Fitzgerald, S., Lewandowski, G., McCauley, R., Murphy, L., Simon, B., Thomas, L., & Zander, C. (2008). Debugging: Finding, fixing and flailing, a multi-institutional study of novice debuggers. Computer Science Education, 18(2), 93–116. https://doi.org/10.1080/08993400802114508.
Gao, X., & Hew, K. F. (2023). A flipped systematic debugging approach to enhance elementary students’ program debugging performance and optimize cognitive load. Journal of Educational Computing Research, 0(0), 1–32. https://doi.org/10.1177/07356331221133560.
Hristova, M., Misra, A., Rutter, M., & Mercuri, R. (2003). Identifying and correcting Java programming errors for introductory computer science students. In S. Grissom (Ed.), Proceedings of the 34th SIGCSE technical symposium on Computer science education (pp. 153–156). Association for Computing Machinery. https://doi.org/10.1145/611892.611956.
Hung, J. C., & Wang, C. C. (2021). The influence of cognitive styles and gender on visual behavior during program debugging: A virtual reality eye tracker study. Human-Centric Computing and Information Sciences, 11(22), 1–20. https://doi.org/10.22967/HCIS.2021.11.022.
IEEE Computer Society (2010). IEEE standard classification for software anomalies. https://doi.org/10.1109/IEEESTD.2010.5399061.
Isen, A. M. (2008). Some ways in which positive affect influences decision making and problem solving. In M. Lewis, J. M. Haviland-Jones, & L. F. Barrett (Eds.), Handbook of Emotions (pp. 548–573). The Guilford Press. https://psycnet.apa.org/record/2008-07784-034.
James, W. (1950). The principles of psychology: Volume one. Dover Publications.
Johnson, W. L., & Soloway, E. (1985). PROUST: Knowledge-based program understanding. IEEE Transactions on Software Engineering, SE-11(3), 267–275. https://doi.org/10.1109/TSE.1985.232210.
Johnson, W. L., Soloway, E., Cutler, B., & Draper, S. (1983). Bug catalogue: I. https://cpsc.yale.edu/research/technical-reports/1983-technical-reports.
Khan, I. A., Brinkman, W. P., & Hierons, R. M. (2011). Do moods affect programmers’ debug performance? Cognition Technology and Work, 13(4), 245–258. https://doi.org/10.1007/s10111-010-0164-1.
Ko, A. J. (2002). Individual differences in programming, testing, and debugging strategies in a statistical end-user programming environment [Bachelor Thesis]. Oregon State University.
Ko, A. J., & Myers, B. A. (2005). A framework and methodology for studying the causes of software errors in programming systems. Journal of Visual Languages and Computing, 16, 41–84. https://doi.org/10.1016/j.jvlc.2004.08.003.
Ko, A. J., & Uttl, B. (2003). Individual differences in program comprehension strategies in unfamiliar programming systems. In S. Kawada (Ed.), Proceedings of the 11th IEEE International Workshop on Program Comprehension (pp. 175–184). IEEE Computer Society. https://doi.org/10.1109/WPC.2003.1199201.
Kovari, A. (2020). Study of algorithmic problem-solving and executive function. Acta Polytechnica Hungarica, 17(9), 241–256. https://doi.org/10.12700/APH.17.9.2020.9.13.
Kovari, A., Katona, J., & Costescu, C. (2020). Evaluation of eye-movement metrics ina software debbuging task using GP3 eye tracker. Acta Polytechnica Hungarica, 17(2), 57–76. https://doi.org/10.12700/APH.17.2.2020.2.4.
Lin, Y. T., Wu, C. C., Hou, T. Y., Lin, Y. C., Yang, F. Y., & Chang, C. H. (2016). Tracking students’ cognitive processes during program debugging-An eye-movement approach. IEEE Transactions on Education, 59(3), 175–186. https://doi.org/10.1109/TE.2015.2487341.
Liu, Z., Zhi, R., Hicks, A., & Barnes, T. (2017). Understanding problem solving behavior of 6–8 graders in a debugging game. Computer Science Education, 27(1), 1–29. https://doi.org/10.1080/08993408.2017.1308651.
Lowe, T. (2019). Debugging: The key to unlocking the mind of a novice programmer? In P. K. Imbrie (Ed.), 2019 IEEE Frontiers in Education Conference (pp. 1–9). IEEE. https://doi.org/10.1109/FIE43999.2019.9028699.
Magana, A. J., Fennell, H. W., Vieira, C., & Falk, M. L. (2019). Characterizing the interplay of cognitive and metacognitive knowledge in computational modeling and simulation practices. Journal of Engineering Education, 108(2), 276–303. https://doi.org/10.1002/jee.20264.
Mangaroska, K., Sharma, K., Gašević, D., & Giannakos, M. (2022). Exploring students’ cognitive and affective states during problem solving through multimodal data: Lessons learned from a programming activity. Journal of Computer Assisted Learning, 38(1), 40–59. https://doi.org/10.1111/jcal.12590.
Martella, D., Casagrande, M., & Lupiáñez, J. (2011). Alerting, orienting and executive control: The effects of sleep deprivation on attentional networks. Experimental Brain Research, 210, 81–89. https://doi.org/10.1007/s00221-011-2605-3.
McCauley, R., Fitzgerald, S., Lewandowski, G., Murphy, L., Simon, B., Thomas, L., & Zander, C. (2008). Debugging: A review of the literature from an educational perspective. Computer Science Education, 18(2), 67–92. https://doi.org/10.1080/08993400802114581.
Michaeli, T., & Romeike, R. (2019). Current status and perspectives of debugging in the K12 classroom: A qualitative study. In A. K. Ashmawy & S. Schreiter (Eds.), Proceedings of the 10th IEEE Global Engineering Education Conference (pp. 1030–1038). IEEE. https://doi.org/10.1109/EDUCON.2019.8725282.
Milli Eğitim Bakanlığı [The Ministry of National Education] (2020b). Programlama temelleri ders bilgi formu [Programming fundamentals course information sheet]. http://kitap.eba.gov.tr/panel/dosyalar/upload/0/0/P_0_24_07_2020_1_20_31_342.pdf.
Milli Eğitim Bakanlığı [The Ministry of National Education]. (2020a). Bi̇li̇şi̇m Teknoloji̇leri̇ alani çerçeve öğreti̇m programi [A framework curriculum of Information Technology Department]. Programi&tur=mtal&sinif=9. http://meslek.eba.gov.tr/?p=Ogretim.
Peng, F., Li, C., Song, X., Hu, W., & Feng, G. (2016). An eye tracking research on debugging strategies towards different types of bugs. In S. I. Ahamed, C. K. Chang, W. Chu, I. Crnkovic, P.-A. Hsiung, G. Huang, & J. Yang (Eds.), 2016 IEEE 40th Annual Computer Software and Applications Conference (pp. 130–134). IEEE. https://doi.org/10.1109/COMPSAC.2016.57.
Posner, M. I., & Fan, J. (2008). Attention as an organ system. In R. James, & Pomerantz (Eds.), Topics in Integrative Neuroscience: From cells to Cognition (pp. 31–61). Cambridge University Press. https://doi.org/10.1017/CBO9780511541681.005.
Rahman, M. M., Watanobe, Y., & Nakamura, K. (2020). Source code assessment and classification based on estimated error probability using attentive LSTM language model and its application in programming education. Applied Sciences, 10, 1–21. https://doi.org/10.3390/app10082973.
Raz, A. (2004). Anatomy of attentional networks. The Anatomical Record (Part B: New Anat), 281B, 21–36. https://doi.org/10.1002/ar.b.20035.
Robins, A., Haden, P., & Garner, S. (2006). Problem distributions in a CS1 course. In D. Tolhurst & S. Mann (Eds.), Proceedings of the 8th Australasian Conference on Computing Education (Vol. 52, pp. 165–173). Australian Computer Society, Inc. https://doi.org/10.5555/1151869.1151891.
Romero, P., Cox, R., du Boulay, B., & Lutz, R. (2002). Visual attention and representation Switching during Java program debugging: A study using the restricted focus viewer. In M. Hegarty, B. Meyer, & N. H. Narayanan (Eds.), Diagrammatic Representation and Inference (Vol. 2317, pp. 221–235). Springer. https://doi.org/10.1007/3-540-46037-3_23.
Romero, P., Boulay, B., Cox, R., Lutz, R., & Bryant, S. (2004). Dynamic rich-data capture and analysis of debugging processes. 16th Annual Workshop of Psychology of Programming Interest Group. https://ppig.org/files/2004-PPIG-16th-romero.pdf.
Sharif, B., Falcone, M., & Maletic, J. I. (2012). An eye-tracking study on the role of scan time in finding source code defects. In S. N. Spencer (Ed.), Proceedings of the Symposium on Eye Tracking Research and Applications, (pp. 381–384). Association for Computing Machinery. https://doi.org/10.1145/2168556.2168642.
Soloway, E., & Ehrlich, K. (1984). Empirical studies of programming knowledge. IEEE Transactions on Software Engineering, SE-10(5), 595–609. https://doi.org/10.1109/TSE.1984.5010283.
Spohrer, J. C., Soloway, E., & Pope, E. (1985). A goal/plan analysis of buggy pascal programs. Human-Computer Interaction, 1(2), 163–207. https://doi.org/10.1207/s15327051hci0102_4.
Stefik, A., & Gellenbeck, E. (2011). Empirical studies on programming language stimuli. Software Quality Journal, 19, 65–99. https://doi.org/10.1007/s11219-010-9106-7.
Xie, B., Loksa, D., Nelson, G. L., Davidson, M. J., Dong, D., Kwik, H., Tan, A. H., Hwa, L., Li, M., & Ko, A. J. (2019). A theory of instruction for introductory programming skills. Computer Science Education, 29(2–3), 205–253. https://doi.org/10.1080/08993408.2019.1565235.
Xinogalos, S. (2016). Designing and deploying programming courses: Strategies, tools, difficulties and pedagogy. Education and Information Technologies, 21, 559–588. https://doi.org/10.1007/s10639-014-9341-9.
Yen, C. Z., Wu, P. H., & Lin, C. F. (2012). Analysis of experts’ and novices’ thinking process in program debugging. In K. C. Li, F. L. Wang, K. S. Yuen, S. K. S. Cheung, & R. Kwan (Eds.), Engaging Learners Through Emerging Technologies (Vol. 302, pp. 122–134). Springer. https://doi.org/10.1007/978-3-642-31398-1_12.
Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Informed consent
Informed consent was obtained from all individual participants included in the study.
The study procedures were conducted by attending the Declaration of Helsinki.
CRediT author statement
Conceptualization: [Arif AKÇAY]; Data Curation: [Arif AKÇAY]; Formal Analysis: [Arif AKÇAY]; Investigation: [Arif AKÇAY]; Methodology: [Arif AKÇAY]; Project Administration: [Arif ALTUN]; Resources: [Arif ALTUN]; Software: [Arif AKÇAY]; Supervision: [Arif ALTUN]; Validation: [Arif AKÇAY, Arif ALTUN]; Visualization: [Arif AKÇAY]; Writing - Original Draft: [Arif AKÇAY]; Writing - Review & Editing: [Arif ALTUN]
Conflict of interest
Authors declare they have no competing interests.
Research involving human participants - ethical approval
All procedures performed in study involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. The study was approved by Hacettepe University Ethics Commission under report n° 35853172-300. Informed consent was obtained from all individual participants and their families included in the study.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Akçay, A., Altun, A. The investigation of human attention networks on debugging performance. Educ Inf Technol 29, 2679–2695 (2024). https://doi.org/10.1007/s10639-023-11955-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10639-023-11955-7