Skip to main content

Advertisement

SpringerLink
Log in

Children’s coding experiences in a block-based coding environment: a usability study on code.org

  • Published:
Education and Information Technologies Aims and scope Submit manuscript

Abstract

Programming education is an important educational process that enables the development of children's problem solving and algorithmic thinking skills. It is known that children frequently encounter syntax problems in coding activities. Many block-based programming software has been developed to eliminate this difficulty in the learning process. Block-based programming software is widely used all over the world because of its colorful features and providing a coding environment that children can learn easily. However, analyzes on the usefulness of such widely used block-based programming software cannot be found in the literature. In this study, the usability of code.org block-based coding environment was analyzed through the coding practices of children. The study group was consisted of 14 children aged between 9 and 13. Analyzes were made in terms of efficacy, efficiency, and satisfaction. For the efficacy analysis of the programming environment, it was observed that all the children completed the tasks assigned to them. In efficiency analysis; task times, task step counts, need for assistance in the process of using software, overall focus data, heat maps, eye scanning data and focus levels in the guided area of the participants were examined. In satisfaction analysis; satisfaction level of participants was examined. As a result of the research; usability data for Code.org environment has been tried to be presented in detail. In the efficacy dimension, while there were generally no problems regarding the task completion status of the participants; in efficiency dimension, suggestions were made regarding the placement of the blocks, block sizes and application methods. In satisfaction dimension, it was seen that children faced with problems during the block search process.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

(Adapted from Kraleva, Kralev and Kostadinova, 2019)

Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

Data will be available on request.

Notes

  1. https://scratch.mit.edu/about

  2. https://snap.berkeley.edu/

References

  • Ashrov, A., Marron, A., Weiss, G., & Wiener, G. (2015). A use-case for behavioral programming: An architecture in JavaScript and Blockly for interactive applications with cross-cutting scenarios. Science of Computer Programming, 98, 268–292.

    Article  Google Scholar 

  • Barradas, R., Lencastre, J. A., Soares, S., & Valente, A. (2020). The Code. org Platform in the Developing of Computational Thinking with Elementary School Pupils. In International Conference on Computer Supported Education (pp. 118–145). Springer, Cham.

  • Bevan, N., Barnum, C., Cockton, G., Nielsen, J., Spool, J., and Wixon, D. (2003, April). The magic number 5: is it enough for web testing?. In CHI'03 extended abstracts on Human factors in computing systems (pp. 698–699). ACM.

  • Brooke, J. (2013). SUS: A retrospective. Journal of Usability Studies, 8(2), 29–40.

    Google Scholar 

  • Brown, N. C., Altadmri, A., and Kölling, M. (2016). Frame-based editing: Combining the best of blocks and text programming. In 2016 International Conference on Learning and Teaching in Computing and Engineering (LaTICE) (pp. 47–53). IEEE.

  • Byerly, G. (2007). Look in their eyes-eye tracking, usability, and children. School Library Media Activities Monthly, 23(8), 30.

    Google Scholar 

  • Cheng, G., Poon, L. K., Lau, W. W., & Zhou, R. C. (2019, July). Applying Eye Tracking to Identify Pupils' Use of Learning Strategies in Understanding Program Code. In Proceedings of the 2019 3rd International Conference on Education and Multimedia Technology (pp. 140–144).

  • Choi, H. (2013). Pre-service teachers’ conceptions and reflections of computer programming using Scratch: Technological and pedagogical perspectives. International Journal for Educational Media and Technology, 7(1), 15–25.

    Google Scholar 

  • Code.org (2022). Code.org Statistics, Retrieved January 10, 2022 from https://code.org/statistics

  • Conati, C., Carenini, G., and Steichen, B. (2013). User-adaptive information visualization: using eye gaze data to infer visualization tasks and user cognitive abilities. In Proceedings of the 2013 international conference on Intelligent user interfaces (pp. 317–328). ACM.

  • Cooke, L., and Cuddihy, E. (2005). Using eye tracking to address limitations in think-aloud protocol. In IPCC 2005. Proceedings. International Professional Communication Conference, 2005. (pp. 653–658). IEEE.

  • Creswell, J.W., 2014, Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 4th ed., Sage Publications, Inc.

  • Crowther, M. S., Keller, C. C., & Waddoups, G. L. (2004). Improving the quality and effectiveness of computer-mediated instruction through usability evaluations. British Journal of Educational Technology, 35(3), 289–303. https://doi.org/10.1111/j.0007-1013.2004.00390.x

    Article  Google Scholar 

  • Cui, H. L., Yoon, Y., & Yuan, Y. (2015). A study of convention mobile application characteristics that affect user satisfaction and loyalty. The Journal of the Korea Contents Association, 15(4), 363–372.

  • Davey, L. (1991). The application of case study evaluations. Practical Assessment, Research and Evaluation, 2(9), 1.

    MathSciNet  Google Scholar 

  • Demirer, V., & Nurcan, S. A. K. (2016). Programming education and new approaches around the world and in Turkey. Journal of Theory and Practice in Eduaction, 12(3), 521–546.

    Google Scholar 

  • de Mooij, S. M., Kirkham, N. Z., Raijmakers, M. E., van der Maas, H. L., & Dumontheil, I. (2020). Should online math learning environments be tailored to individuals’ cognitive profiles? Journal of Experimental Child Psychology, 191, 104730.

    Article  Google Scholar 

  • Dix, A., Finlay, J. E., Abowd, G. D., Beale, R. (2003). Human-Computer Interaction, 3rd Edition. Pearson.

  • Duchowski, A. T. (2007). Eye Tracking Methodology. Theory and Practice, 328, 614.

    Google Scholar 

  • Fesakis, G., & Serafeim, K. (2009). Influence of the familiarization with" scratch" on future teachers’ opinions and attitudes about programming and ICT in education. ACM SIGCSE Bulletin, 41(3), 258–262.

    Article  Google Scholar 

  • Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate research in education (Vol. 7, p. 429). New York: McGraw-hill.

  • Giannakos, M. N., Papavlasopoulou, S., & Sharma, K. (2020). Monitoring Children’s Learning Through Wearable Eye-Tracking: The Case of a Making-Based Coding Activity. IEEE Pervasive Computing, 19(1), 10–21.

    Article  Google Scholar 

  • Gomes, A., & Mendes, A. J. (2007, September). Learning to program-difficulties and solutions. In International Conference on Engineering Education–ICEE (Vol. 7).

  • Gomez, M. J., Moresi, M., & Benotti, L. (2019, July). Text-based programming in elementary school: a comparative study of programming abilities in children with and without block-based experience. In Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education (pp. 402–408).

  • Head, A. J. (1999). Design wise: A guide for evaluating the interface design of information resources. CyberAge Books.

    Google Scholar 

  • Holwerda, R., & Hermans, F. (2018). A usability analysis of blocks-based programming editors using cognitive dimensions. In 2018 IEEE symposium on visual languages and human-centric computing (VL/HCC) (pp. 217–225). IEEE.

  • Hsu, T. C., & Hu, H. C. (2017). Application of the Four Phases of Computational Thinking and Integration of Blocky Programming in a Sixth-Grade Mathematics Course (p. 73). Siu-Cheung KONG The Education University of Hong Kong.

    Google Scholar 

  • ISO. (2018). ISO 9241–11:2018(en) Ergonomics of human-system interaction — Part 11: Usability: Definitions and concepts. Retrieved December 5, 2021 from https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en

  • Jacob, R. J., and Karn, K. S. (2003). Eye tracking in human-computer interaction and usability research: Ready to deliver the promises. In The mind's eye (pp. 573–605). North-Holland.

  • Jimenez, Y., Kapoor, A. & Gardner-McCune, C. (2018, October). Usability challenges that novice programmers experience when using scratch for the first time. In 2018 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) (pp. 327–328). IEEE.

  • Kale, U., & Yuan, J. (2021). Still a new kid on the block? Computational thinking as problem solving in Code. org. Journal of Educational Computing Research, 59(4), 620–644.

  • Ke, F., Liu, R., Sokolikj, Z., Dahlstrom-Hakki, I., & Israel, M. (2021, July). Using Eye Tracking for Research on Learning and Computational Thinking. In International Conference on Human-Computer Interaction (pp. 216–228). Springer, Cham.

  • Kraleva, R., Kralev, V., & D., veKostadinova. (2019). Çocuklara uygun blok tabanlı programlama dillerinin analizi için bir metodoloji. Bilgisayar Bilimi Ve Mühendisliği Dergisi, 13(1), 1–10.

    Google Scholar 

  • Lambić, D., Đorić, B., & Ivakić, S. (2021). Investigating the effect of the use of code. org on younger elementary school pupils’ attitudes towards programming. Behaviour & Information Technology, 40(16), 1784–1795.

  • Landauer, T. K. (1996). The trouble with computers: Usefulness, usability, and productivity. MIT press.

  • Leidl, K. D., Bers, M. U., and Mihm, C. (2017). Programming with ScratchJr: a review of the first year of user analytics. In Conference Proceedings of International Conference on Computational Thinking Education (pp. 116–121).

  • Liao, C. N., Chang, K. E., Huang, Y. C., & Sung, Y. T. (2020). Electronic storybook design, kindergartners’ visual attention, and print awareness: An eye-tracking investigation. Computers & Education, 144, 103703.

    Article  Google Scholar 

  • Lin, Y., & Weintrop, D. (2021). The landscape of Block-based programming: Characteristics of block-based environments and how they support the transition to text-based programming. Journal of Computer Languages, 67, 101075.

    Article  Google Scholar 

  • Liu, R., Luo, F., & Israel, M. (2021). What Do We Know about Assessing Computational Thinking? A New Methodological Perspective from the Literature. In Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V. 1 (pp. 269–275).

  • Liu, J., Wimmer, H., & Rada, R. (2016). " Hour of Code”: Can It Change Pupils’ Attitudes Toward Programming? Journal of Information Technology Education: Innovations in Practice, 15, 53.

    Google Scholar 

  • Maloney, J., Resnick, M., Rusk, N., Peppler, K. A., & Kafai, Y. B. (2008). Media designs with Scratch: What urban youth can learn about programming in a computer clubhouse. In Proceedings of the 8th international conference on International conference for the learning sciences-Volume 3 (pp. 81–82). International Society of the Learning Sciences. Retrieved May 17, 2019 from https://dl.acm.org/citation.cfm?id=1599974

  • McKinney, V., Yoon, K., & Zahedi, F. M. (2002). The measurement of web-customer satisfaction: An expectation and disconfirmation approach. Information Systems Research, 13(3), 296–315.

  • Newley, A., Deniz, H., Erdogan, K. A. Y. A., & Yesilyurt, E. (2016). Engaging elementary and middle school pupils in robotics through hummingbird kit with Snap! visual programming language. Journal of Learning and Teaching in Digital Age, 1(2), 20–26.

    Google Scholar 

  • Nielsen, J. (1993). Usability engineering. Ap Professional.

    Book  MATH  Google Scholar 

  • Pelánek, R., & Effenberger, T. (2022). Design and analysis of microworlds and puzzles for block-based programming. Computer Science Education, 32(1), 66–104.

    Article  Google Scholar 

  • Partovi, H. (2015). A comprehensive effort to expand access and diversity in computer science. ACM Inroads, 6(3), 67–72.

    Article  Google Scholar 

  • Piech, C., Huang, J., Nguyen, A., Phulsuksombati, M., Sahami, M., & Guibas, L. (2015). Learning program embeddings to propagate feedback on pupil code. arXiv preprint, retrieved November 5, 2021 from arXiv:1505.05969

  • Popat, S., & Starkey, L. (2019). Learning to code or coding to learn? A systematic review. Computers & Education, 128, 365–376.

    Article  Google Scholar 

  • Sam, N. (2013). "Covert Attention," in PsychologyDictionary.org. Retrieved November 12, 2022 from https://psychologydictionary.org/covert-attention/

  • Sennersten, C. (2004). Eye movements in an action game tutorial. Retrieved October 10, 2021 from https://lup.lub.lu.se/luur/download?func=downloadFile&recordOId=1328850&fileOId=1328851

  • Sim, G., & Bond, R. (2021). Eye tracking in child computer interaction: challenges and opportunities. International Journal of Child-Computer Interaction, 30, 100345.

  • Sprenger, P., & Benz, C. (2020). Children’s perception of structures when determining cardinality of sets—results of an eye-tracking study with 5-year-old children. ZDM, 2(4), 753–765.

    Article  Google Scholar 

  • Wade, N., & Tatler, B. W. (2005). The moving tablet of the eye: The origins of modern eye movement research. Oxford University Press.

    Book  Google Scholar 

  • Wang, J., Antonenko, P., Celepkolu, M., Jimenez, Y., Fieldman, E., & Fieldman, A. (2019). Exploring Relationships Between Eye Tracking and Traditional Usability Testing Data. International Journal of Human-Computer Interaction, 35(6), 483–494.

    Article  Google Scholar 

  • Weichbroth, P. (2020). Usability of mobile applications: A systematic literature study. IEEE Access, 8, 55563–55577.

    Article  Google Scholar 

  • Vlachogianni, P., & Tselios, N. (2022). Perceived usability evaluation of educational technology using the System Usability Scale (SUS): A systematic review. Journal of Research on Technology in Education, 54(3), 392–409.

    Article  Google Scholar 

  • Yauney, J., Bartholomew, S. R., & Rich, P. (2022). A systematic review of “Hour of Code” research. Computer Science Education, 1–33.

Download references

Author information

Authors and Affiliations

  1. Faculty of Education, Computer Education & Instructional Technologies, Yildiz Technical University, Davutpasa Campus, 34210, Davutpasa, Istanbul, Turkey

    Kaan Dilmen, Serhat Bahadır Kert & Tuba Uğraş

Authors
  1. Kaan Dilmen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Serhat Bahadır Kert
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tuba Uğraş
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Serhat Bahadır Kert.

Ethics declarations

Confict of interests

Authors declare that they have no confict of interests.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dilmen, K., Kert, S.B. & Uğraş, T. Children’s coding experiences in a block-based coding environment: a usability study on code.org. Educ Inf Technol 28, 10839–10864 (2023). https://doi.org/10.1007/s10639-023-11625-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10639-023-11625-8

Keywords

Search

Navigation