A valid and reliable tool for examining computational thinking skills

Abstract

The aim of this study was to develop a scale which can be used to measure the computational thinking skills (CTS) of high school students. Validity and reliability testing of the scale was performed with the participation of 785 students. Exploratory and confirmatory factor analysis showed that the five-point Likert scale had a construct consisting of four factors Problem-solving, Cooperative Learning & Critical Thinking, Creative Thinking, and Algorithmic Thinking expressed by 42 items. The factor loadings of the scale varied from .475 to .853. The confirmatory factor analysis performed to reveal the factorial validity of the scale showed that the Chi-square value (χ2 = 2679.07; sd = 815, p = 0.00) was significant. The fitness index values were found to be RMSA = .0075; SRMR = .081; NNFI = .91; CFI = .92; GFI = .90; and AGFI = .88. The Cronbach’s Alpha internal consistency coefficient was .969 for the overall scale. In addition, the stability of the scale was examined to obtain information about its reliability and the test-re-test method was used. It was concluded as a result of the analysis that the scale was a valid and reliable measurement tool which can be used to measure the CTS of high school students.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2

References

  1. Amabile, T. M. (1985). Motivation and creativity: Effects of motivational orientation on creative writers. Journal of Personality and Social Psychology, 48(2), 393–399.

    Article  Google Scholar 

  2. Anderson, C. W., & Smith, E. L. (1984). Children's preconceptions and content-area textbooks in G.G. Duffy. Comprehension Instruction Perspectives and Suggestions. New York: Longman.

    Google Scholar 

  3. Ater-Kranov, A., Bryant, R., Orr, G., Wallace, S., & Zhang, M. (2010). Developing a community definition and teaching modules for computational thinking: Accomplishments and challenges. Proceedings of the 2010 ACM Conference on Information Technology Education, 143–148.

  4. Atmatzidou, S., & Demetriadis, S. (2016). Advancing students’ computational thinking skills through educational robotics : A study on age and gender relevant differences. Robotics and Autonomous Systems, 75, 661–670. https://doi.org/10.1016/j.robot.2015.10.008.

    Article  Google Scholar 

  5. Balcı, A. (2009). Research in Social Sciences: Methods, Techniques and Principles. Ankara: Pegem A.

    Google Scholar 

  6. Barr, D., Harrison, J., & Conery, L. (2011). Computational thinking: A digital age skill for everyone. Retrieved from: http://files.eric.ed.gov/fulltext/EJ918910.pdf.

  7. Bati, H. (2017). Computational thinking and integrative education (STEAM) in science education. Pamukkale University Journal of Education, 41, 91–103. https://doi.org/10.9779/PUJE800.

    Article  Google Scholar 

  8. Berkeley, (2017). The beauty and joy of computing. Retrieved from: http://bjc.berkeley.edu

  9. Boden, M. A. (1998) What is creativity, creativity in human evolution and prehistory (1 ed.). Steven J. Mithen, Routledge.

  10. Brennan, K., & Resnick, M. (2012). New frameworks for studying and assessing the development of computational thinking. 2012 annual meeting of the american educational research association (Vancouver: Canada). Retrieved from: http://scratched.gse.harvard.edu/ct/files/AERA2012.pdf.

  11. Bundy, A. (2007). Computational thinking is pervasive. Retrieved from: http://www.inf.ed.ac.uk/publications/online/1245.pdf.

  12. Büyüköztürk, Ş. (2002). Data analysis handbook. Ankara: Pegem Publication.

    Google Scholar 

  13. Büyüköztürk, Ş. (2008). Manual of data analysis for social sciences. Ankara: Pegem A Publications.

    Google Scholar 

  14. Büyüköztürk, Ş., Çakmak, E. K., Akgün, Ö. E., Karadeniz, Ş., & Demirel, F. (2015). Scientific research methods. Ankara: Pegem A Publications.

    Google Scholar 

  15. Carmines, E. G., & Zeller, R. A. (1982). Reliability and validity assessment (5th ed.). Beverly Hills: Sage Publications Inc..

    Google Scholar 

  16. Chaffee, J. (1994). Thinking critically. Boston: Houghton Mifflin.

    Google Scholar 

  17. Chen, G., Shen, J., Barth-Cohen, L., Jiang, S., Huang, X., & Eltoukhy, M. (2017). Assessing elementary students’ computational thinking in everyday reasoning and robotics programming. Computers & Education, 109, 162–175. https://doi.org/10.1016/j.compedu.2017.03.001.

    Article  Google Scholar 

  18. Code.org. (2017). Hour of code. Retrieved from: https://hourofcode.com/tr/gb.

  19. Cooper, S., Dann, W., & Pausch, R. (2000). Developing algorithmic thinking with Alice. In Proceedings of Isecon, 17, 506–539.

  20. Craft, A. (2003). Creative thinking in the early years of education. Early Years, 23(2), 143–154.

    Article  Google Scholar 

  21. Cropley, A. (2001). Creativity in education and Learning Great Britain: Clays.

  22. Curzon, P. (2015). Computational thinking: Searching to speak. Retrieved from: http://teachinglondoncomputing.org/free-workshops/computational-thinking-searching-to-speak/.

  23. Czerkawski, B. C., & Lyman III, E. W. (2015). Exploring issues about computational thinking in higher education. Tech Trends, 59(2), 57–65. https://doi.org/10.1007/s11528-015-0840-3.

    Article  Google Scholar 

  24. Demir, Ö., & Seferoğlu, S. S. (2017). New concepts, different uses: An evaluation related to computational thinking. Readings of Educational Technology, 41(August), 468–483.

    Google Scholar 

  25. Department for Education England (2013). National curriculum in England:Computing programmes of study key stages 1 and 2. Ref: DFE-00171e2013. Retrieved from: https://www.gov.uk/government/publications/nationalcurriculum-in-england-computing-programmes-of-study.

  26. Ennis, R. H. (1962). A concept of critical thinking. Havard Educational Review, 22, 88–111.

    Google Scholar 

  27. Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272–299.

    Article  Google Scholar 

  28. Facione, P. A., & Facione, N. C. (1993). Test Manual: the California Critical Thinking Skills Test, Form A and Form B. Millbrae: The California Academic Press.

    Google Scholar 

  29. Facione, N. C., Facione, P. A., & Sanchez, C. A. (1994). Critical thinking disposition as a measure of competent clinical judgement: The development of the California thinking disposition inventory. Journal of Nursing Education, 33, 345–350.

    Google Scholar 

  30. Fisher, R. (2004). Creativity across the curriculum. In R. Fisher & M. Williams (Eds.), In unlocking reativity (pp. 160–172). Britain: David Fulton.

    Google Scholar 

  31. Gelbal, S. (1991). Problem solving. Hacettepe University Journal of Education, 6, 167–173.

    Google Scholar 

  32. Gorsuch, R. L. (1983). Factor analysis (2. ed.). Hillsdale: Lawrence Erlbaum Associates.

  33. Guilford, J. P. (1967). Creativity: Yesterday, today and tomorrow. The Journal of Creative Behavior, 1(1), 3–14.

    Article  Google Scholar 

  34. Halpern, D. F. (2013). Thought and knowledge: An introduction to critical thinking. New York: Psychology Press.

    Google Scholar 

  35. Harvard (2017). CS50: Introduction to computer science. Retrieved from: https://cs50.harvard.edu

  36. Heppner, P. P., & Krauskopf, C. J. (1987). The integration of personal problem solving processes within counseling counsel. Psychologist, 15, 371–447.

    Google Scholar 

  37. Heppner, P. P., & Reeder, L. (1983). The relationship between problem solving self-appraisal and psychological adjustment. Cognitive Therapy and Research, 9(4), 415–427.

    Article  Google Scholar 

  38. Heppner, P. P., Baumgardner, A. H., & Jakson, J. (1985). Depression and attributional style: Are they related? Cognitive Therapy and Research, 9, 105–113.

    Article  Google Scholar 

  39. Hu, C. (2011). Computational thinking: What it might meanand what we might do about it. In M. Goldweber (Ed.), 16th annual joint conference on Inno-vation and Technology in Computer Science Education (pp. 223–227). New York: ACM.

    Google Scholar 

  40. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.

    Article  Google Scholar 

  41. ISTE (2015). Computational thinking leadership toolkit (First Edition). Retrieved from: https://www.iste.org/explore/articleDetail?articleid=152&category=Solutions&article=Computational-thinking-for-all.

  42. ISTE (2016). ISTE standards for students. Retrieved from: https://www.iste.org/standards/standards/for-students-2016.

  43. ISTE-CSTA (2011). Computational thinking: Teacher resources (2. edition) (Grant No: CNS-1030054). Retrieved from: https://www.iste.org/explore/articleDetail?articleid=152&category=Solutions&article=Computational-thinking-for-all.

  44. Jane, L. (2001). The relationship between social problem solving and bullying behaviour among male and female adult prisoners. Aggresive Behavior, 27, 297–312.

    Article  Google Scholar 

  45. Johnson, D.W., & Johnson, R.T. (1994). Leading the cooperative school. (Second edition), Edina, Minnesota: Interaction Book Company.

  46. Johnson, D. W., & Johnson, R. T. (1995). Creative controversy: Intellectual challenge in the classroom. Edina: Interaction Book Company.

    Google Scholar 

  47. Johnson, D. W., Johnson, R. T., & Holubec, E. J. (1994). The nuts & bolts of cooperative learning. In Edina. Minnesota: Interaction Book Company.

    Google Scholar 

  48. Jöreskog, K., & Sörbom, D. (1993). Lisrel 8: Structural equation modeling with the Simplis command language. USA: Scientific Software International.

    Google Scholar 

  49. Kalaycı, Ş. (2009). SPSS applied multivariate statistical techniques. Ankara: Asil Publication.

    Google Scholar 

  50. Katai, Z. (2015). The challenge of promoting algorithmic thinking of both sciences- and humanities-oriented learners. Journal of Computer Assisted Learning, 31(4), 287–299. https://doi.org/10.1111/jcal.12070.

    Article  Google Scholar 

  51. Kazimoglu, C., Kiernan, M., Bacon, L., & MacKinnon, L. (2012). Learning programming at the computational thinking level via digital game-play. Procedia Computer Science, 9, 522–531. https://doi.org/10.1016/j.procs.2012.04.056.

    Article  Google Scholar 

  52. Kline, R. B. (2005). Structural equation modeling (2nd ed.). New York: The Guilford Press.

    Google Scholar 

  53. Knuth, D.E. (1980). Algorithms in modern mathematics and computer Science Stanford Department of Computer Science Report No. STAN-CS-80-786, 1980.

  54. Korkmaz, Ö., Çakir, R., & Özden, M. Y. (2017). A validity and reliability study of the computational thinking scales (CTS). Computers in Education, 72, 558–569. https://doi.org/10.1016/j.chb.2017.01.005.

    Google Scholar 

  55. Lau, J. Y. (2011). An introduction to critical thinking and creativity: Think more, think better. Wiley.

  56. Lee, T. Y., Mauriello, M. L., Ahn, J., & Bederson, B. B. (2014). CTArcade: Computational thinking with games in school age children. International Journal of Child-Computer Interaction, 2(1), 26–33. https://doi.org/10.1016/j.ijcci.2014.06.003.

    Article  Google Scholar 

  57. Lu, J. J., & Fletcher, G. H. L. (2009). Thinking about computational thinking. ACM Special Interest Group on Computer Science Education Conference (SIGCSE 2009). Chattanooga: ACM Press.

    Google Scholar 

  58. Mayer, R. E. (1992). Thinking, problem solving, Cognition. New York: W H Freeman and Company.

    Google Scholar 

  59. Mayer, R. E. (1998). Cognitive , metacognitive , and motivational aspects of problem solving. Instructional Science, 26(1–2), 49–63. https://doi.org/10.1023/A:1003088013286.

    Article  Google Scholar 

  60. Mayer, R. E. (1999). Fifity years of creativiy research. Ed by Robert J. Handbook of creativity, Sternberg, Cambridge University Press.

  61. National Research Council. (1999). Being fluent with infor-mation technology. Washington, DC: National AcademyPress.

    Google Scholar 

  62. Newell, A., & Simon, H. A. (1972). Human problem solving (Vol. 104, No. 9). Englewood Cliffs, NJ: Prentice-Hall.

  63. Papert, S. (1996). An exploration in the space of mathematics educations. International Journal of Computers for Mathematical Learning, 1(1), 95–123.

    Google Scholar 

  64. Paul, R. W. (1984). Critical thinking fundamental to education for a free society. Educational Leadership, 1, 5–14.

    Google Scholar 

  65. Paul, R. (1990). Critical thinking. In Rohnert Park. California: Sonoma State University.

    Google Scholar 

  66. Piirto, J. (2004). Understanding creativity. Great potential press, Inc. In USA.

    Google Scholar 

  67. Rouquette, M. L. (2007). Creativity . Ankara: Dost.

  68. Şahiner, A., & Kert, S. B. (2016). Examining studies related with the concept of computational thinking between the years of 2006-2015. European Journal of Science and Technology, 5(9), 38–43.

    Google Scholar 

  69. Seçer, İ. (2013). Practical data analysis and reporting with SPSS and LISREL. Ankara: Anı Publication.

    Google Scholar 

  70. Seferoğlu, S., & Akbıyık, C. (2006). Critical thinking and teaching. Hacettepe Universitesi Journal of Education, 30, 193–200. https://doi.org/10.1080/03634526109377142.

    Google Scholar 

  71. Simpson, E., & Courtney, M. (2002). Critical thinking in nursing education: Literature review. International Journal of Nursing Practice, 8, 89–98.

    Article  Google Scholar 

  72. Şimşek, Ö. F. (2007). Introduction to Structural Equation Modeling; Basic Principles and LISREL Applications. Ankara: Ekinoks Press.

    Google Scholar 

  73. Slavin, R. E. (1980). Cooperative learning: Theory research and Practice. Rentice Hall: Englewood Cliffs.

    Google Scholar 

  74. Slavin, R. E. (1987). Cooperative learning in student teams: What research says to the teacher. Washington, D.C.: National Education Association.

    Google Scholar 

  75. Sönmez, V., & Alacapınar, G. F. (2014). Sampled scientific research methods. Ankara: Anı Publication.

    Google Scholar 

  76. Sternberg, R. J., & Lubart, T. I. (1999). The concept of creativity: Prospects and paradigms. In R. J. Sternberg (Ed.), Handbook of creativity (pp. 3–15). Cambridge: Cambridge University Press.

    Google Scholar 

  77. Tavşancıl, E. (2002). Attitude Measurement and Data Analysis with SPSS. Ankara: Nobel Publications.

    Google Scholar 

  78. Thornton, S. (1998). Kids solve the problem. Istanbul: Gendaş Publishing.

    Google Scholar 

  79. Turgut, M. F., & Baykul, Y. (2011). Measurement and evaluation in education. Ankara: PegemA Publication.

    Google Scholar 

  80. Vee, A. (2013). Understanding computer programming as a literacy. Literacy in Composition Studies, 1(2), 42e64.

    Article  Google Scholar 

  81. Veenman, S., Benthum, N., Bootsma, D., Dieren, J., & Kemp, N. (2002). Cooperative learning and teacher education. Teaching and Teacher Education, 18, 87–103.

    Article  Google Scholar 

  82. Veneziano, L., & Hooper, J. (1997). A method for quantifying content validity of health-related questionnaires. American Journal of Health Behavior, 21(1), 67–70.

    Google Scholar 

  83. Watson, G., & Glaser, E. M. (1980). Watson-Glaser Critical Thinking Appraisal. San Antonio: The Psychological Corporation.

    Google Scholar 

  84. Wing, J. M. (2006). Computational thinking. Communications of the ACM, 49(3), 33–35.

    Article  Google Scholar 

  85. Wing, J. M. (2008). Computational thinking and thinking about computing. Philosophical Transactions of the Royal Society, 366, 3717–3725.

    MathSciNet  Article  MATH  Google Scholar 

  86. Wing, J. M. (2011). Research notebook: Computational thinking What and why? The Link Magazine, Spring. Carnegie Mellon University, Pittsburgh. Retrieved from: https://www.cs.cmu.edu/link/research-notebook-computational-thinking-what-and-why.

  87. Wing, J. M. (2016). Computational thinking. 10 years later. Microsoft Research blog. Retrieved from: https://www.microsoft.com/en-us/research/blog/computational-thinking-10-years-later/.

  88. Wolfram, S. (2016). How to teach computational thinking. Blog Stephen Wolfram. Retrieved from: http://blog.stephenwolfram.com/2016/09/how-to-teachcomputational-thinking/

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Mustafa Yağcı.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Yağcı, M. A valid and reliable tool for examining computational thinking skills. Educ Inf Technol 24, 929–951 (2019). https://doi.org/10.1007/s10639-018-9801-8

Download citation

Keywords

  • Computational thinking
  • Problem solving
  • Algorithmic thinking
  • Teaching & learning strategies