Keywords

1 Introduction

In today’s knowledge and information society, competent handling of information and communication technologies is indispensable in order to meet the diverse requirements of the various areas of life. In this context, it is important that students develop the necessary skills to use these technologies in their daily lives to allow them an active and full participation in today’s digital age [4,5,6,7]. Based on research studies and on recent developments, changes in the digital technologies themselves and changes in the notions of the importance of digital skills, an understanding of the required skills and competences has been expanded to include computational thinking [8, 9]. Since publication of Wing’s [10] influential article, in which she states her grand vision that everyone should have skills in this field and be able to use them, computational thinking has been the subject of research and scientific discourse. However, it also assigns an important role to the school systems and thus to the school as a mediating authority for corresponding competences [2, 11]. As a result, many initiatives have been launched in K-12 schools [12].

In this context, there are differences between educational systems in the integration of computational thinking into compulsory education. This leads to the need to study the results of the implementation of computational thinking and related teaching and evaluation methods [13], also with a view to enabling comparability of computational thinking outcomes within and between educational systems.

However, the broad spectrum of perspectives on computational thinking also means that various elements of definition have emerged, resulting in a lack of clarity as to what computational thinking should be [14].

When it comes to the question of how computational thinking can be conceptualised, it becomes apparent that the construct of computational thinking is known to be poorly defined [15], that there is no universally accepted definition [16] and that there is thus the challenge to assess computational thinking [17]. These circumstances complicate the widespread integration of computational thinking in the learning and teaching context, as educational systems place different emphases on how to learn and teach computational thinking. This is also reflected in teacher education, as there is a great need to prepare well-trained teachers to integrate computational thinking into their daily teaching activities [18]. Yet, there is still only marginal understanding of how non-computer science teachers can be engaged in computational thinking [19, 20].

What is taught, however, are problem-solving skills. Students are expected to work in new environments, overcome problems they are unfamiliar with, and apply multidisciplinary reasoning skills that are not tied to specific content [21]. The majority of existing computational thinking definitions and a few existing studies suggest that there is a high correlation between student competences in computational thinking and problem-solving skills [9, 22,23,24].

If it can be shown that this high correlation between computational thinking and problem-solving does in fact exist, then this finding can be used to structure computational thinking lessons accordingly, which necessitates investigation of the relationship between computational thinking and problem-solving [25].

Measuring this relationship has several benefits. In addition to the referral to well-proven problem-solving theories to explain computational thinking, it might provide an explanatory approach for variation in students’ achievement in computational thinking.

Summarising all these aspects related to the increasing relevance of computational thinking, the inconsistent conceptualisation, the different emphases in learning and teaching computational thinking in different educational systems, and the resulting advantage of measuring the correlation between computational thinking and general problem-solving, leads to three major challenges: conceptualising computational thinking and problem-solving; finding an appropriate research design; and highlighting the benefits and nature of the results. This raises the three research questions, which are addressed in this article:

  1. 1.

    How are computational thinking and general problem-solving to be conceptualised, where do they overlap and how can this be presented on a theoretical level?

  2. 2.

    Which data and which research approach are appropriate for an empirical examination of the theoretical understanding?

  3. 3.

    Which kind of results with respect to the overlap between computational thinking and problem-solving will be obtained in such a research process?

Section 2 is concerned with answering the first question, whereby, firstly, the conceptualisation of computational thinking (Sect. 2.1) and problem-solving (Sect. 2.2) and then their overlapping areas (Sect. 2.3) and, thus, the relationship at the theoretical level, are covered. Section 3 responds to question 2 by introducing the International Computer and Information Literacy Study 2018 and thereby presenting the international option computational thinking (Sect. 3.1) and the German national extension to problem-solving (Sect. 3.2). Finally, Sect. 4 addresses question 3 and outlines the expected results related to the overlap between computational thinking and problem-solving.

2 Computational Thinking and Problem-Solving at a Theoretical Level

2.1 Conceptualisation of Computational Thinking

The identification of the spectrum of computational thinking skills is a balancing act between algorithmic procedural thinking related to computer programming and a wider range of transferable problem-solving skills and dispositions [1, 22, 26].

There are many definitions and conceptualisations of computational thinking, all with different emphases. Wing, for instance, argues that computational thinking should not be limited to programming and that it should be added to the analytical skills of all people [10]. Denning [27] asserts that definitions of computational thinking are “vague and confusing” (p. 33) when they do not originate in the field of computer science. Thus, it makes a difference whether the definition comes from the field of computer science or from the field of education.

In the International Computer and Information Literacy Study 2018 (ICILS 2018), computational thinking is defined as “Computational thinking refers to an individual’s ability to recognize aspects of real-world problems which are appropriate for computational formulation and to evaluate and develop algorithmic solutions to those problems so that the solutions could be operationalized with a Computer” [28] (p. 27).

The computational thinking construct in ICILS 2018 consists of two general conceptual categories (strands) and three or respectively two specific content categories within a strand (aspects). Strand 1 focuses on the conceptualisation of problems, assuming that before developing solutions, problems must first be understood and designed in such a way that algorithmic thinking or system thinking can support the process of solution development. This includes, for instance, the aspect of knowing about and understanding computer systems, whereby students should have the ability to recognise and describe the characteristics of systems. On a declarative level, a person should be able to describe rules and boundary conditions. A second aspect describes the formulation and analysis of problems. For this purpose, a problem is broken down into smaller, manageable parts (decomposition) and the properties of the task are systematised in such a way that a computational solution can be developed. The third aspect of the first strand refers to the meaningful collection and representation of relevant data in order to effectively assess the problem solution within a system.

Strand 2 comprises the operationalisation of solutions in the form of planning, implementation, testing and evaluation of algorithmic solutions to real-world problems. On the one hand, the strand focuses on the aspect of planning and evaluating solutions. Typically, there is a wide range of possible computer-based solutions for a particular problem. It is therefore important to be able to plan and evaluate solutions from different perspectives and to understand the advantages, disadvantages and effects of different solutions. On the other hand, the second strand includes the developing of algorithms, programs and interfaces. This does not assume that the students are familiar with the syntax and functions of a particular programming language, but rather with the logical reasoning underlying the development of algorithms for problem-solving [2, 28].

Computational thinking processes emerge from the framework on three levels. On the level of problem conceptualisation (strand 1), problem identification and definition are important, as is decomposition, in which a problem is broken down into sub-steps to make it easier to deal with. On the second level - the operationalisation of solutions (strand 2) - various processes play a role, such as pattern recognition, pattern matching and algorithmic thinking, which contain abstraction. On the third level - the evaluation of solutions - the focus is on the debugging and evaluation of the solution. This level is closely linked to the second level of operationalisation of solutions, which is reflected in the fact that this is included in aspect 2.1 of the framework under the second strand [29].

2.2 Conceptualisation of Problem-Solving

When considering computational thinking as a competence and/or a competence area, it becomes apparent that it requires the development of both domain-specific and general problem-solving skills [9]. Nevertheless, the investigation of general problem-solving is not the approach considered in the above-mentioned computational thinking construct, but rather it focuses on domain-specific problem-solving (although the framework would also apply to general problem-solving).

Problem-solving can be described as a transformation from an undesirable initial state to a desirable final state [30] by overcoming a barrier. Achieving this requires higher order thinking skills [31].

A problem-solving process is frequently described as a seven-stage cycle [32]. On the level of problem conceptualisation, it comprises the recognition or identification of a problem and the definition and mental representation of the problem. On the level of the operationalisation of solutions, the development of a strategy to solve the problem, the organisation of knowledge concerning the problem, as well as the allocation of mental and physical resources needed to solve the problem, all play a role. Monitoring of progress towards the goal, and evaluation of the solution in terms of accuracy are on the third level, which focuses on evaluating solutions.

2.3 Overlapping Areas of Computational Thinking and Problem-Solving

On a theoretical level, there are apparent indications of a strong correlation between computational thinking and problem-solving processes [29]. When comparing the two constructs, there is considerable overlap between them, as illustrated in Fig. 1.

Fig. 1.
figure 1

Overlapping areas of computational thinking and problem-solving

Full size image

Therefore, introducing students to problem-solving and algorithmic thinking at an early age is useful in enabling them to learn computational thinking step-by-step. This ensures that all students have sufficient skills at the end of their schooling to keep up with technological progress. To further develop these initiatives, it is necessary to know which competences the students already possess and where similarities and differences exist with regard to their general problem-solving skills.

For this purpose, it is necessary to introduce a study in which empirical research can be conducted into the introduced theoretical constructs, also to support students’ acquisition of computational thinking in schools and taking school context and learning as well as student characteristics into account.

3 Computational Thinking and Problem-Solving in ICILS 2018

3.1 Computational Thinking as an International Option of ICILS 2018

Based on the described computational thinking construct, the International Computer and Information Literacy Study 2018 (ICILS 2018) meets the challenge of measuring student competences in the field of computational thinking on a representative basis, with an international comparison, by having integrated computational thinking as an international option (as a response to the increasing relevance of this field in research, scientific discourse and school life).

With ICILS 2018, the International Association for the Evaluation of Educational Achievement (IEA) provides for the second time after ICILS 2013, with the help of an internationally developed and elaborated set of tools, the empirically validated assessment of students’ computer and information literacy (CIL; comparable to information and communication technologies (ICT) literacy) in the participating countries with an international comparison. For the first time, the IEA also supplied the international option computational thinking [8]. In addition, the relationship of these competences to the school and extracurricular context of learning was examined [2].

Educational systems participating in ICILS 2018 could decide whether they also wanted to participate in this international option. Denmark, Finland, France, Germany, Luxembourg, Portugal, the Republic of Korea, the United States of America (USA), as well as North-Rhine-Westphalia (Germany) as benchmark participant, took the opportunity to do so.

The international option for ICILS 2018 is essentially aimed at clarifying the research questions as to: (1) which computational thinking competences students have and how the conditions for acquiring these competences are related to the competence level of the students; and (2) how students’ achievements in computational thinking relate to their computer and information literacy skills. While the first question aims to capture computational thinking as a new area of competence for the first time and to explain the conditions for acquiring competences at student, school and educational system level, the second research question focuses on empirically clarifying the relationship with the area of computer and information literacy.

The sample in ICILS 2018 is 20 students per school in Grade 8 with a minimum age of 13.5 years, 15 teachers per school who teach in Grade 8, and the school principal and ICT coordinator at school level. In all participating countries, a representative random sample of at least 150 schools was drawn and a sample of students and teachers was taken in these schools [28].

The concept of the additional module in computational thinking is that students’ achievement in computational thinking is measured in the same student cohort by extending the computer-based student tests by two test modules.

Computer-based testing with authentic tasks in a software-based test environment is essential for measuring the construct to be captured. A major challenge in test design has been that the computer-based student tests must be applicable to eighth graders, whether they have learned a programming language or not, must be applicable in a wide range of countries and curricula, and must have the least possible overlap with other disciplines (e.g. mathematics). In ICILS 2018, this area is made accessible for the first time using an adequately developed test instrument in the form of computer-based measurement with an international comparison, which can be used without knowledge of programming languages, but includes typical features of computer-based problem-solving processes such as the use of algorithms and loops for systematic and repeated problem-solving steps. A visual coding approach is used to consider algorithmic logic. The international computer-based student tests consist of questions and tasks that are embedded in real-life contexts [28].

All students complete two computer and information literacy modules. In the countries participating in the international option computational thinking, students complete two additional 25-min test modules. According to the computational thinking construct in ICILS 2018 with two strands (problem conceptualisation and solution operationalisation), each of the two computational thinking test modules concentrates on one of these strands. The computational thinking test modules contain information-based response tasks and nonlinear competence tasks.

Since competences are always embedded in a context, the research questions featured in ICILS 2018 focus in addition on the measurement of Grade 8 students’ computer and information literacy as well as the relationship between Computational thinking literacy and Computer and Information literacy in terms of individual characteristics and the school context. By means of supplemented items in the background questionnaires of ICILS 2018, factors related to specific individual, school and teaching contexts and conditions in the field of computational thinking are also considered. The student and teacher questionnaires collect process-related context factors, including students’ reports on the extent of learning about approaches to computational thinking at school, and teachers’ emphasis on teaching approaches to computational thinking in class. School ICT coordinators were asked about their perceptions of school emphasis on teaching computational thinking activities to students. The data obtained from the so-called National Context Survey are intended to support the interpretation of the results of the student, teacher and school questionnaires [28].

The results of ICILS 2018 will be published in the international and national report on November 5, 2019.

3.2 Problem-Solving as a German National Extension to ICILS 2018

In addition, the participating educational systems had the possibility to add national extensions, of which several took advantage [3, 33]. In Germany, reading tests and tests on cognitive abilities [2], items aiming to examine the correlation between computational thinking and general problem-solving skills [3], information about students’ self-reported proficiency in completing computational thinking tasks, and items focusing on computer science and its practice in schools, were therefore also gathered within the context of computational thinking. This means that the relationship between computational thinking and problem-solving could only be addressed for Germany.

The aim is to design an empirically verifiable theoretical analysis model that investigates this relationship with statistical controls for individual student characteristics such as students’ self-reported proficiency in computational thinking and their background characteristics, as well as for the school context such as students’ reports on the extent of learning about approaches to computational thinking at school and teachers’ emphasis on teaching approaches to computational thinking in class.

4 Expected Outcomes and Discussion on the Overlap of Computational Thinking and Problem-Solving

Studying the relationship between students’ computational thinking achievement and their general problem-solving skills, taking into consideration individual student characteristics and the school context, aims to provide a starting point for achieving a deeper understanding from a theoretical and empirical perspective as well as a holistic picture of computational thinking. The benefit of this is that all relationships are examined in the same sample and are calculated in one model on two levels [34].

Focusing on the relationship between computational thinking and general problem-solving and on their links would be of interest for the development of educational systems. As a matter of fact, this would allow in the coming years to work specifically on computational thinking and to use well-proven problem-solving theories, thus making it possible to implement development measures in each considered country (e.g. Germany).

Obviously, the question arises as to whether everyone should be able to learn and apply computational thinking [34] as Wing [3, 10] proposed, but in the spirit of equal opportunities, it would not necessarily be reasonable to deny some students the possibility of acquiring computational thinking skills that are becoming increasingly important for their professional future. When it comes to bringing computational thinking into schools as a cross-curricular competence, there is no alternative to conceptualise computational thinking in such a way that it can also be taught by non-computer science teachers and can also be learned by students who do not attend computer science classes. In doing so, it must also be kept in mind that neither students nor teachers with little knowledge of computer science can be expected to be familiar with a programming language. The logical conditions behind computational thinking can, however, be explained to them and it can then be left up to them to decide whether they want to gain a deeper understanding of the subject matter and acquire knowledge of or skills in a programming language or not.

However, society has the responsibility to offer young people the opportunity to acquire at least basic competences in the field of computational thinking, which they can then upgrade in line with their own particular preferences and needs. Students need a different kind of knowledge that enables them to succeed in a rapidly changing environment. If schools only prepare their students to meet current expectations, the knowledge and skills they may have to use in their private and professional lives can soon become outdated [21].

To ensure students are properly prepared for life in the digital age, it is of great relevance to consider and investigate computational thinking in the context of general problem-solving. This is addressed on a representative basis in the German national extension to ICILS 2018. Germany is currently thus the only educational system, which, with the help of a national extension to ICILS 2018, is able to investigate the relationship between computational thinking and general problem-solving, in an analysis controlling for other individual student characteristics and the school context. Results will be published in November 2019 and in-depth results in 2020.