Keywords

1 Introduction

Problem-solving skills are fundamental in computer science and must therefore be an essential element of computer science education in both schools and universities. While problem solving is recognized in the curricula of German schools, the way in which it should be taught remains largely unspecified. The German educational standards for computer science address certain aspects of problem solving in various process and content areas, such as “Modeling and Implementing” and “Representing and Interpreting” and particularly in the content area of “Algorithms” [3]. In the state of Brandenburg’s school curriculum, problem-solving primarily pertains to the domain of algorithmics, with the various factors influencing this process being outlined [12]. However, it remains unclear how these factors should be implemented in teaching, which ones are particularly important, and which measures can be taken in the classroom to ensure that students acquire the necessary competencies.

The reasons for the lack of specific teaching methods are manifold. Models such as Polya’s [16] that attempt to illustrate problem-solving processes only partially coincide with reality because human problem solvers often do not proceed as systematically as the model suggests. They rather skip, overlap, and repeat individual phases. Moreover, such models can only be an attempt to reflect the complex thought processes involved but cannot provide an answer for all the variables involved. Schoenfeld has identified four essential prerequisites in this regard, namely knowledge, the use of heuristics, self-observation and regulation, as well as self-efficacy [17]. These manifest themselves to varying degrees depending on the problem and the individual, indicating that there is probably no universally applicable recipe for the acquisition of problem-solving competencies. However, at least the aspects that deal with knowledge of computer-science content as well as computer-science specific and more general problem-solving strategies and heuristics can be examined since these can indeed be part of the school education. This paper investigates a segment of these aspects, namely the question of the role of note-taking in the problem-solving process.

2 The Purpose of Note-Taking

Note-taking serves as a general strategy primarily aimed at capturing perceived information in written form. This information can be auditory, visual, or textual in nature. Note-taking is an integral component of classical learning environments and is indispensable in today’s academic settings. The content, scope, and manner of note-taking have been extensively researched, particularly in the context of university lectures. At the latest since DiVesta’s work in 1972 [4], two main functions of note-taking have been identified in this regard.

The first of these functions is that of an external storage. It serves to capture as much information as possible in order to have it available as a basis for learning later on and to be able to reconstruct any gaps in the notes during reading. In this type of note-taking, the focus is less on processing or organizing information, but rather on retaining as many facts as possible or preventing their forgetting. Several studies have shown that note-taking can contribute to better performance in examination situations (see Kiewra [8], for a review). In fact, the mere product of note-taking is already a conducive factor. Additionally, reviewing one’s own notes can lead to even better performance in exam situations.

The second function of note-taking pertains to the active processing and organization of the written contents [13]. The written, yet not fully understood, information is linked to existing knowledge and imbued with meaning over time. Connections between different pieces of information must be identified, not-understood concepts must be paraphrased, or even reacquired from other sources and added to the notes in order to establish understanding. This process, referred to DiVesta [4] as encoding, ideally results in a representation of the writer’s thoughts and mental models in relation to the information. The result is a better understanding and the ability to further process the information in one’s own words. This process can be seen as an active learning process, where the material is linked with the writer’s existing cognitive structure [4]. This second function of note-taking does not contradict the function of external storage, but rather builds on it. Hartley and Davies view this process and its resulting product as a form of analysis [7], which is located in the realm of higher-order thinking according to Bloom’s taxonomy (cf. [2]). This taxonomy of learning objectives, in its revised edition by Krathwohl and Anderson, aims to categorize learning objectives based on the complexity of thinking involved. The three less complex categories are “remember”, “understand”, and “apply”. The three more complex categories, “analyze”, “evaluate”, and “create”, are often grouped together as higher-order thinking skills, indicating a greater level of cognitive demand expected from the learner. Therefore, it can be assumed that intensive processing of previously written material requires a certain level of cognitive maturity, but ideally provides assistance in learning. Note-taking is most helpful when it is done as completely as possible. Aiken showed that the part of the information that is not written down is unlikely to be recalled later [1]. However, according to Kiewra note-takers tend to omit large portions of the presented information, which impairs the encoding function of note-taking [8]. Two additional aspects that could influence the manner of note-taking in the future are the development of technological aids and the increasing ease of accessing information through the internet. Morehead et al. showed that at the time of their study, while much information was still being recorded on paper and with pen, notebook computers and similar devices had also become prevalent. In addition, the amount of note-taking is reduced, especially in online events and when presentation slides are available [13]. A survey of students conducted by Van der Meer et al. revealed that note-taking is often perceived as less necessary when information is readily available on the internet [11]. This indicates, on the one hand, that students are not always aware or familiar with the encoding function of note-taking. On the other hand, it suggests that note-taking may be utilized less frequently in the future due to the presence of easily accessible information.

3 Note-Taking During Problem Solving

A problem contains a multitude of variables that can play a role in finding a solution. The sheer number of variables involved exceeds the capacity of our working memory, as noted by Muesseler et al. [14]. Moreover, the intermediate sub-results generated during the problem-solving process require side calculations. Therefore, it is sensible to make notes and jot down these sub-results. When compared to notes taken during lectures, notes taken during problem solving can initially be classified under the function of external memory storage. In this case, information is written down that is not yet understood and the relationships between the notes are unclear. Following the basic process of problem solving according to Polya71 [16], ideally, one begins with developing an understanding of the problem. All available variables should be considered in relation to the goal. Notes can be helpful in collecting all the details and generating an initial overview. Unlike lecture notes, there is usually no apparent structure at the beginning. The structure emerges only as the problem-solving process unfolds.

After the initial phase, the next step is to look for a solution approach. In this organizational phase [17], a plan needs to be developed from the available variables that leads to the solution of the problem. The previously taken notes can play a significant role in this process. They need to be interpreted, compared, and classified. If they are not recorded, details can be overlooked. The problem is broken down into smaller units, which must be related and assigned to the variables. Ideally, a mental structure of the problem emerges. This structure can be expressed in notes by mapping or clustering. The external memory that the originally recorded variables represented is now further processed in the encoding process. The usefulness of further processing lecture notes was demonstrated by King [9] and others. During problem solving, this phase of mental representation can also manifest itself in drawing diagrams or other graphic representations.

Problem-solving processes typically deviate from the idealized depiction proposed by Polya, wherein a sequential progression of problem understanding, plan formulation and execution, and solution verification occurs [16]. It is more likely that individuals may jump back and forth between phases, forget details, or misinterpret them, resulting in an incorrect mental representation of the problem. Nonetheless, notes are a good starting point for retracing initial thought processes or conducting error checks and reorganizing. Without notes, the risk of forgetting thoughts and intermediate results is significantly higher. Trafton and Trickett showed in a study that the use of notes during problem-solving and self-explanation led to increased learning and more accurate problem solving [18].

4 The Escape Game Room-X

4.1 Escape Games

Escape adventure games are immersive and challenging experiences in which teams of typically two to six individuals work together to collect clues and solve puzzles in order to unlock the door of a physical room in which they are confined. Often set in a story-driven scenario resembling a crime scene, players must locate and piece together various clues and objects to uncover the solution [15].

This engaging and motivating activity has already found its way into educational contexts, with escape games being used in various school subjects, including computer science (see [19]). Previous research has shown that problem-solving competencies can be addressed through this approach, and that learners’ motivation and engagement in such a learning setting is generally high [5].

While educational escape games share many similarities with their recreational counterparts [15], there are notable differences in their aims and scope. In a comparison of different educational formats, approximately one quarter of escape games were found to focus on problem-solving skills as a learning objective [19]. To achieve this, it is necessary to embed the game within a broader problem that needs to be solved alongside other tasks.

4.2 Problem Solving and Note-Taking in the Escape Game Room-X

The escape game Room-X involves a problem of the “simple” category according to [14] with computer science-related content. This means that there is a clear starting and goal state, and some elements of the solution strategy involve tasks from the field of computer science. The game is situated within an educational context and primarily aims to investigate general problem-solving abilities. To ensure inclusivity and avoid excluding students who may lack specific computational problem-solving knowledge, the decision was made to refrain from incorporating problems that necessitate such expertise. Instead, the game includes subtasks that require a foundational understanding of computer science principles, thus preserving its inherent computational character. The objective of the game is to steal the next computer science exam from a fictional teacher and escape from an alarm-secured room within 60 min. Various computer science tasks must be solved to determine the password for a tablet containing the exam and a four-digit combination for a safe that holds the remote control to disable the alarm [6]. A secret message encrypted using the Caesar cipher must be decrypted into plain text, a Morse code needs to be deciphered using a binary tree, and errors need to be identified using the two-dimensional parity check method. Additionally, an automaton’s acceptance of words must be determined based on its state diagram, and a bomb must be defused, which requires knowledge of binary-decimal conversion, understanding of source code, and propositional logic. Furthermore, the game incorporates notable figures from the field of computer science and highlights the complexity involved in solving a monkey puzzle as a form of combinatorial puzzle, thus extending the scope of computer science concepts explored within the game. None of these subtasks is accompanied by a specific written instruction. What needs to be done is derived from the examination of each respective object. Each team is composed of five or six participants who have been receiving computer science education for at least one year and are aged between 13 and 19 years. To facilitate note-taking by students, the room is equipped with two whiteboards. Following the prevailing practice in the surrounding schools, one of the whiteboards is of the conventional type, while the other is a digital board. This arrangement serves not only to enhance the immersive nature of the classroom setting but also allows students to utilize their preferred note-taking system.

In order to solve the problem, participants must find various objects, solve computer science-related tasks, and construct a path to the solution based on the information obtained. Since the purpose of the objects and partial solutions may not be immediately obvious, note-taking is a useful tool, particularly since the game is played in teams, and this allows all participants to access the information gathered. Noteworthy is that the activity of note-taking should be intertwined with the problem-solving process as a continuous activity and not limited to specific moments or tasks.

After this, hypotheses should be formulated regarding how the information can be distinguished from one another, how it relates to each other, and which of the sub-goals they may be relevant to. The resulting content on the board ideally corresponds to the path through the problem space. This second phase in the problem-solving process according to [17] can thus be supported by encoding the notes.

5 Research Methodology

Previous research has shown that note-taking plays a significant role not only in learning but also in problem-solving processes. Depending on how they are used, notes can contribute to a deeper understanding and ultimately to success in solving a task.

Therefore, in the field of computer science, the question arises as to the role that note-taking plays in students’ problem-solving processes. This will be investigated using the aforementioned computer science escape game, where the following two sub-questions are intended to contribute to clarification:

  1. 1.

    What types of notes do participants take during the escape game?

  2. 2.

    How do participants integrate their notes into the problem-solving process during the escape game?

To address these research questions, the notes taken by participants during the escape game Room-X, who agreed to audio and video recording of their activity, were analyzed. The data analyzed here consist of notes made by participants on the whiteboard and the electronic board while they were solving the tasks. For analysis purposes, photos and screenshots of the notes were used, which varied in number per run depending on the changes made by the participants. In some groups, only one photo of the whiteboard was used for analysis, while in others, multiple photos or screenshots were used. The decision on the number of photos or screenshots depended on the extent to which individual notes were erased, replaced, or modified. A total of 54 runs were considered. Four of the 54 data sets used were removed because they either contained no notes or no analyzable notes. The data were collected between May 2018 and January 2023, with no data available between March 2020 and October 2022 due to pandemic-related restrictions. During this time period, the game and the tasks remained unchanged.

The analysis of the data was carried out by two researchers using a structuring qualitative content analysis based on Mayring’s approach [10]. Previous research has shown that notes can either serve as external memory, simply acting as an extension of one’s memory, or as material for encoding, reflecting, and transforming information, serving as tools for active engagement with the content. To approach the question of the role of notes in problem solving in the context of the escape game, a deductive category application approach was used to examine whether these structures were present in the analyzed data. The structuring dimensions derived from the literature and found in the data are henceforth referred to as storage and working tool, respectively. They are intended to contribute to the answer to the first research question. To gain a more detailed understanding of how participants use their notes, we searched for specific manifestations of both dimensions and established a category system based on these findings, which serves to answer the second research question. Hartley and Davies categorize note-taking as an example of analytical thinking when they identify activities such as (1) identifying elements, (2) identifying relationships between elements, and (3) identifying an organizational structure of the material [7]. Translating these categories to note-taking in the escape game, we define them as follows: identifying elements refers not only to the act of noting down found information, but also to evaluating and working with the information to develop clues (Category 1). In the escape game, players must initially locate and identify information as useful or not useful, which requires active engagement with the notes to develop clues. This includes results from (computer science) tasks as well as recognizable solution paths, such as deciphering the Caesar ciphered secret text. Elements that do not contribute to progress in solving tasks must also be identified as such, which can be observed in the notes through erasing or striking out irrelevant information. The active identification of elements suggests that the board is used as a tool, while the mere recording of found information without any editing or deletion indicates note-taking as a storage mechanism. Relationships between elements (category 2) are relevant for solving tasks and were also found in the data. This includes relationships between elements or to the goal of the activity. Notes falling under the category of “element relationships” include cases where there is a clear connection between different notes, such as 14 dashes serving as a placeholder for a password with a reference to additional clues stating “11 letters, 2 special characters, 1 number.” When a relationship to other notes or references to them is visible and the notes help to draw further conclusions or serve as a basis for further work, this suggests that the board was used as a working tool. Notes without a relationship to others or with only a connection to their source (e.g., “Floor \(\rightarrow \) Key”, meaning the key was found on the floor) do not indicate analytical thinking and rather suggest the use of notes as storage.

The organizational structure of the material (Category 3) can also contribute to answering the question of the integration of notes in the problem-solving process. As the escape game Room-X consists of two main tasks, one would expect a visible division of the notes into these two areas. If notes are placed according to whether they contribute to the solution of the password for the tablet or for opening the safe, a structure becomes apparent, indicating a systematic approach. Also, placing elements close to each other that belong to the same (partial) tasks reveals a structure. If, on the other hand, notes are placed freely and without any recognizable connection, the structure is lacking, which suggests that the boards are used more as storage for information rather than as a working tool for problem solving, as in the case of the structured arrangement.

6 Findings

The analysis of the notes taken during the escape game revealed the well-established dichotomy between their use as a working-tool for problem solving or as storage. Out of the 50 data sets examined, 30 were classified as storage, 19 as a working tool, and one data set was disputed between the researchers. Figure 1 clearly illustrates how the boards were used as working tools for problem solving. It can be observed that elements were identified and worked on in a structured manner with visible connections. The development of specific content is visible on the electronic board: the encrypted text and its corresponding plain text “ABSTEIGEnd” (“descending”) were noted, as well as the Morse code and its corresponding decoding. The structure is clearly evident, with everything related to the password noted on the electronic board (top row) and all notes related to the door code on the whiteboard (bottom row). Hints that were initially placed in the wrong location were later erased and transferred to the correct place. For example, “absteigend” (“descending”) was recognized as belonging to the door code and was transferred to the board where the information regarding the door code was collected and erased from the other board. Conversely, “Bill 28” was transferred from the whiteboard to the electronic board and then removed from the original location. Additionally, the participants established connections by drawing lines between hints, such as linking “Buchstaben 6 + 5 Fehlen” (“letters 6 + 5 missing”) to “14 Zeichen” (“14 characters”). The progress of work was also documented, as evidenced by the strikeout and subsequent removal of “Schlüssel auf Boden” (“key on floor”). Likewise, a question mark behind “1 Ziffer” (“1 digit”) was removed after it was found.

Fig. 1.
figure 1

Exemplary presentation of notes as working tool at different points in time during the game. Top left to bottom right: Minute 35, 59, 28 and 33 out of 60.

Full size image

Figure 2 represents an example in which the electronic board was used solely as a storage device for information. The analysis of these notes reveals that the progress of work was not documented, and that the notes were unstructured and lacked any reference to further actions. The word “ABSTEIGEND” (“descending”) needs to be decrypted using the Caesar cipher in the escape game, which apparently happened, but was not recorded in the notes. Furthermore, it remains unclear what purpose this hint serves, as no relation to it was noted. Other notes, such as “Endzustand Doppelkreis” (“finate state double circle”) or “14 = Zeichen, 2 = Sonderzeichen, 1 = Ziffer” (“14 = characters, 2 = special characters, 1 = digit”), were merely copied from found clues or work instructions. Again, there is no apparent connection to what these notes are for. The only reference that was noted is to the origin of a part of the task “Boden \(\rightarrow \) Schlüssel” (“Floor \(\rightarrow \) Key”), which, however, has no relevance for the further solution process. The notes in this example lack any structure that would indicate which clues belong together or lead to the solution of which sub-puzzles. Additionally, the example contains partial results that are of little use in their current form. The underlined item “5 ... GATES” contains a part of the password for the tablet, but it is incorrect and the entire password remains incomplete.

Fig. 2.
figure 2

Example of note-taking as a storage; screenshot taken at minute 59 out of 60 of the game.

Full size image

The use of a category system enables a clear classification of notes as either working tool or storage. The calculation of interrater reliability using Cohen’s Kappa yielded a value of 0.96. Out of the 50 data sets considered, 30 were categorized as storage and 19 as working tool. There was a disagreement between the two researchers regarding the categorization of one data set. It is worth noting that the datasets categorized as working tool also encompassed the function of storage. However, not all notes are as comprehensive as the examples presented. Even in less organized examples, helpful approaches for solving puzzles can become apparent. For example, in some cases, the information about the structure of the password was visualized by drawing 14 strokes on the board, which corresponded to the combined length of the two password parts. However, the two password parts were not linked to each other afterwards. The same applies to the word “ABSTEIGEND” (“descending”), which is frequently found on the boards, even though there might not be any other notes. These examples demonstrate that the types of notes are not always sharply distinguishable from each other. The available data does not provide a clear demonstration of a direct connection between the type of notes taken and the success in the escape game.

7 Discussion

In summary, it can be concluded that in a problem-solving situation, as well as in note-taking situations in lectures, the distinction between external storage and encoded material becomes apparent. This suggests that note-taking serves not only a similar purpose but also a similar benefit, namely that of deeper understanding. However, it should be noted that lecture notes already have a predetermined structure, while in the escape game, the structure needs to be created on a completely blank sheet. Hence, in this case the activities to process and to organize the information must be accomplished by the learners themselves. Although the categorization system from lectures was found here as well, the types of notes are not always clearly distinguishable which indicates a potential transition from storage to working tool. This suggests that some participants who use notes for external storage can also use them in a structured manner, albeit to a limited extent. Nevertheless, our study indicates that the encoding phase often remains in its early stages, raising the question of why this process stalls. The game environment imposes certain constraints that influence the process of note-taking. In a problem-solving scenario where time pressure is a factor, it may seem more intuitive to start immediately rather than dedicating time to note-taking. Additionally, the situation demands heightened cognitive efforts, leading participants to seek ways to mitigate the cognitive load. However, effective note-taking can actually aid in reducing the cognitive burden by alleviating the need to retain all variables mentally, ultimately resulting in a net time gain, as it enables faster resolution of confusion. The question arises how computer science education can contribute here as this concerns problem solving competencies. A deeper look into computer science education contents offers some aspects that could be addressed. Firstly, a lack of competencies are evident in the process area of “Representing and Interpreting” [3], as indicated by the relatively underdeveloped representations. Deficits are also visible in the process area of “Structuring and Networking” [3], particularly in the achievement level II “Reorganization and Transfer” of this process area, which manifests in the lack of systematic approach as well as the less developed structures in the representation of the content. This particular aspect is a fundamental skill. When confronted with a problem, it is crucial to analyze the structure of the initial situation in a deliberate manner, aiming to gain a comprehensive understanding and develop a structured approach towards problem solving. However, the nature of the notes from our study suggests that considerations of the overall structure typically emerge late in the problem-solving process and tend to remain focused on searching or individual tasks. Engaging in visualization and actively working with the notes can help shift the focus towards the broader structural aspects. In the “reflection and problem-solving” achievement level of the process area, the analysis of the notes also revealed that only a few of the students are able to visibly structure their knowledge and their approach to problem solving.

For computer science education, our results can contribute as a starting point for developing these competencies, through which students can be given a universal tool for problem analysis. By implementing note-taking as a working tool, problem-solving processes can be approached more easily, thereby reducing the initial hurdle. Creating connections between different items of information and linking them to the writer’s cognitive structures might be a part of the learning process that should be the focus of future research. One approach for computer science education could be to expose students more frequently to unfamiliar problems and refrain from providing specific solution steps. Instead, the focus could be on practicing systematic problem analysis and decomposition, utilizing visualization techniques and structured note-taking. This approach aims to facilitate the initial transition from a daunting unknown to the development of a concrete computational approach.

It should be noted, however, that the use of notes alone was not decisive for the success of the teams in the escape game. Various other factors, such as team composition, communication within the team, prior knowledge of computer science concepts, motivation, and self-regulation, also played a significant role. Nevertheless, a correlation between the use of notes and progress in problem solving appears likely, and the results can be interpreted in line with those of Trafton and Trickett [18], whereas the use of a free-form note-taking system was helpful depended on whether users could integrate it into the problem-solving process.

If the goal is to promote the use of note-taking in computer science education with regard to problem solving, it is first necessary to understand the reasons for non-use or misuse and then investigate how note-taking can be actively promoted in the problem-solving process. It is reasonable to assume that acquiring this competence cannot be the sole responsibility of computer science education, but rather cross-disciplinary solutions are desirable.