Introduction

This paper presents the development of a method capturing various layers of collaborative problem solving (CPS) for a deeper understanding of CPS as a small-group enactment. CPS has received substantial interest as one of the key competencies of 21st-century learners in educational reforms of national curricula worldwide and international, large-scale assessments (i.e., assessment and teaching of 21st Century skills ATC21S, http://www.atc21s.org; the Organisation for Economic Co-operation and Development OECD, Programme for International Student Assessment PISA 2015, http://www.oecd.org/pisa/). Because of the origins of CPS constructs, which stem from assessing individual problem solving, until recently, research on CPS has often focused on problem solving and the skills individual learners bring to the joint problem space. Against this background, an interest in investigating how the theorized CPS constructs function as broader social units, such as small groups, is relatively recent (Dowell et al., 2020; Funke et al., 2018; Graesser et al., 2018; Scoular et al., 2017).

CPS is a complex and dynamic construct, where the “complexity label” is not based on cognitive demands but refers to the definition of the CPS comprising multiple interacting elements (Scoular et al., 2017). Grounded in the socio-cognitive approach to learning, CPS lies in a two-dimensional space of social and cognitive domains that are tightly coupled and intermingle in the problem-solving process (e.g., Avry et al., 2020; Care et al., 2016; Dowell et al., 2020; Hesse et al., 2015; Swiecki et al., 2020). Given the social nature of CPS, the socio-cognitive properties reside and evolve in interaction between the problem solvers (Dowell et al., 2020). Yet, because of the inherent complexity of the construct, determining how to identify the manifested behaviors of CPS and capture the processes at the group level is challenging (Dowell et al., 2020), especially in open-ended problem spaces that lack clarity of the problem solution (Scoular et al., 2017). Despite their usefulness in certain situations, methods that rely primarily on participants’ perceptions of collaboration and CPS via questionnaires, self- or peer evaluations (for an overview, see Kyllonen et al., 2018), or analyzing the contents of communication—irrespective of their semantic and temporal relations—can afford limited understanding around the actual processes of collaboration (Dowell et al., 2020; Swiecki et al., 2020).

The CPS construct and its social dynamics have been simulated in structured, computer-based environments that automatically generate large-scale data samples (e.g., Dowell et al., 2020; Graesser et al., 2018; Swiecki et al., 2020). These data allow for analysis based on computational linguistics frameworks (i.e., group communication analysis, GCA; Dowell et al., 2018, 2020) that quantify, for example, social dynamics in groups involved in CPS. However, together with simulated CPS activities and automated analysis at a large scale, searching for ecologically valid measures based on detailed capturing of the micro-interaction processes at the small-group level (e.g., Davis et al., 2015)—in authentic learning contexts that large-scale studies can also account for (Reimann, 2021)—is similarly desired (Gauvain, 2018). Consequently, if CPS processes are seen to consist of intertwined and interdependent interactions among the group that evolve over time, adequate methods are needed that allow for detailed capturing and analysis of these interaction-related events (Reimann, 2021) and the complex mechanisms of “coupling minds” (Cress et al., 2018) involved in CPS in this context.

In this study, to better understand CPS as a group-level enactment, a method is developed for capturing various layers of CPS by relying on triangulating analysis techniques (Humble, 2009; Meadows & Morse, 2001). In triangulation, at least two analysis approaches are integrated to form a deeper picture of the object of investigation (Flick, 2004; Kelle & Erzberger, 2004). In this study, triangulation combines the following: (a) directed content analysis on the actualized CPS in groups (e.g., Hsieh & Shannon, 2005; Humble, 2009), (b) process analysis and visualizations (e.g., Abbott, 1990; Kapur, 2011; Reimann, 2009, 2021), and (c) qualitative cases (e.g., Davis et al., 2015; Stahl, 2017). All the analyses are based on the communication stream as video recordings from CPS sessions in small groups (Barron et al., 2015).

In the study, small groups of teacher education students solve open-ended CPS tasks during an experiential mathematics education course in geometry. The tasks include a technology-enhanced task using dynamic software, and a task using physical objects. In a co-located CPS, participants work together in the same physical location and coordinate their understanding and actions by exchanging ideas, knowledge, or resources to collaboratively converge on the correct understanding of the shared problem (Alterman & Harsch, 2017). However, groups often fail to make use of their full potential (Barron, 2000, 2003). Therefore, to better ensure that productive CPS processes evolve, the pedagogical design in this study incorporates the task characteristics (as sufficient complexity of the task) to enhance the process gains of collaboration (Sears & Reagin, 2013) and includes external support as a macro-script, an activity model to engage learners in group interaction processes (e.g., Dillenbourg & Hong, 2008). However, because this paper emphasizes the methodological aspects of capturing various layers of CPS as a small-group enactment, the scope does not cover how the pedagogical design affects the quality of CPS. Next, the two focal points of this study, the CPS construct and the method, are described in more detail.

Collaborative problem solving: a construct definition

In CPS, collaboration and problem solving intermingle, requiring both the social and cognitive contributions of the participants (Care et al., 2016; Funke et al., 2018; Hesse et al., 2015; Scoular & Care, 2020; Scoular et al., 2017). Basically, the social components of CPS are related to how participants coordinate and communicate with one another (e.g., Richardson et al., 2007), as well as how they regulate and resolve differences among the collaborating partners (e.g., Järvelä et al., 2016). The cognitive elements, in turn, are related to how effectively and efficiently participants solve the problem (e.g., Mayer, 1998). To be successful, CPS requires engagement in shared processes and exchanges of ideas, knowledge, or resources, as well as externalizing understandings and efforts in the group.

In observed group interaction, CPS is actualized using verbal and nonverbal indications and constant updates that are transparent to most group members (Graesser et al., 2018). This means that for the group to be aware of most of the substantial problem parts and to work successfully and reach a mutual understanding, the participants are obliged to communicate, exchange, and share knowledge over problem-solving processes at the social and cognitive levels (Graesser et al., 2018; Hesse et al., 2015). This requires a willingness to collaborate and share expertise, as well as the ability to manage interpersonal conflicts (Hesse et al., 2015). To be productive, CPS not only makes use of the full potential of the group’s expertise but also necessitates the full set of social skills coming into force.

In this study, for detailed articulation of the CPS construct in small groups of learners, a comprehensive CPS framework by Hesse et al., (2015; see also Care et al., 2016; Scoular & Care, 2020; Scoular et al., 2017), originally developed for the assessment of CPS skills in the ATC21S project, is applied (Care et al., 2016; Griffin & Care, 2015). The framework is chosen because it profoundly covers both social and cognitive elements of the CPS construct (cognitive, social, and regulatory aspects), and it amalgamates existing theoretical knowledge from social psychology and problem solving (Hesse et al., 2015). Expanding from the original usage, the aim is to explore how the CPS elements intermingle and interact in the group-level processes and over time. The framework gives a detailed description of how the social and cognitive elements of the CPS construct can come into force in a collaborative context (see Appendix 1), and therefore, it is suitable for the purposes of directed content analysis (Hsieh & Shannon, 2005) as employed in this study.

In the framework, CPS is defined as a set of elements consisting of “social” and “cognitive” as the first-order components (see Table 1). As the second-order components, the framework covers three social process elements (participation, perspective taking, social regulation) and two cognitive process elements (task regulation, knowledge building). The five elements are further divided into 19 specific sub-elements as third-order components.

Table 1 Collaborative problem solving (CPS) construct definition
Full size table

From the social set, “participation” (sub-elements: action by individual and interaction with the partner or partners) refers to the willingness and eagerness to share information, externalize one’s thoughts, and be involved in the different stages of problem solving. “Perspective taking” (Hayashi, 2018; sub-elements: adaptive responsiveness, audience awareness) means the ability to understand and consider others’ perspectives and contributions and to tailor one’s contributions to others. “Social regulation” points to the knowledge and awareness of the strengths and weaknesses of the other group members (sub-elements: transactive memory, Lewis & Herndon, 2011; negotiation, Thompson et al., 2010; self-evaluation [metamemory], Wegner, 1986; and responsibility initiative, which signifies taking responsibility for the progress of [the parts of] the task by the group).

From the cognitive set, “task regulation” refers to planning and monitoring skills for developing strategies for problem solving and shared problem representation (“joint problem space”; Roschelle & Teasley, 1995). The sub-elements of task regulation incorporate problem analysis, setting goals, resource management, tolerance for ambiguity, collection of information, and systematicity as four aspects of planning, one of the fundamental activities in CPS (Eichmann et al., 2019). Moreover, from the cognitive set, “knowledge building” refers to the ability to learn and build knowledge through group interaction (Scardamalia & Bereiter, 2006). In CPS, steps in knowledge building include the sub-elements of identification of relationships, cause and effect, and testing of hypotheses (Griffin, 2014). Taken together, these five CPS elements and their 19 sub-elements are considered essential for successful CPS activity. In the framework, CPS may not proceed linearly; rather, the CPS elements can overlap and run parallel between the different stages of the process.

Applying analysis method triangulation for capturing various layers of CPS as a group-level enactment

The concept of triangulation (e.g., Flick, 2002, 2004; Kelle & Erzberger, 2004) has multiple meanings in the literature, but it is generally viewed as a cumulative way of validating the results or as “an enlargement of perspectives that permit a fuller treatment, description and explanation of the subject area” (Kelle & Erzberger, 2004, p. 174). Here, the emphasis is on the latter purpose—to increase the scope, depth, and consistency of research (Flick, 2002). The same problem can be approached using various methods; alternatively, triangulation can be employed to treat different aspects of the same phenomenon (Kelle & Erzberger, 2004). In this study, three different methods of analysis (directed content analysis, process analysis and visualizations, and case examples) are used to capture various layers of CPS as a group-level enactment. All the methods are valued equally and can add to a unified depiction in understanding CPS in this context. Next, the set of methods chosen is discussed and presented with examples of related analysis approaches.

Using a directed approach (e.g., Hsieh & Shannon, 2005; Humble, 2009), content analysis is a deductive category application. It is guided by a structured process where an existing theory or conceptual frame is applied to a novel context, and the categories and the determined operational definitions of each category are based on the theory or framework (Hsieh & Shannon, 2005). In the current work, directed content analysis applies the ATC21S CPS framework and the codebook by Hesse et al. (2015) in analyzing interactional data. The evidence from the coding can be presented, for example, by offering descriptive evidence, or in this study, first calculating the frequency distributions of the categories that enter into the broader framework (Vogel & Weinberger, 2018).

A related approach, quantitative content analysis (e.g., Neuendorf, 2011), also referred to as the “coding and counting approach” in analyzing interaction, is a summarizing, quantitative analysis (Jeong et al., 2014); in contrast to directed content analysis, it applies inductive coding (Kennedy, 2018). In quantitative content analysis, a coding scheme is developed and applied in the data corpus (De Wever et al., 2006). When studying collaborative learning and CPS, the analysis has typically resulted in frequency counts of different categories as a means of comparing different experimental conditions (i.e., concerning the processes or outcomes; Csanadi et al., 2018; Swiecki et al., 2020).

“Coding and counting” analyses are seen as useful in explaining the distribution of process categories, that is, regarding the variations of the outcomes (Kapur, 2011; Reimann, 2009). However, when frequency-based methods are applied in studies of collaborative learning and CPS, it has been found, for example, that groups with similar frequency distribution of categories may well have different temporal dynamics of these categories (Kapur, 2011). As Hesse et al. (2015) stated, CPS processes unfold over time and can vary over the course of problem solving. Therefore, to avoid treating CPS as a set of isolated events, an issue that cumulative accounts or frequencies are often criticized for (e.g., Csanadi et al., 2018; Kapur, 2011; Reimann, 2009; Swiecki et al., 2020), the temporal dimension of the data is also acknowledged. As Kapur (2011) pointed out, “learning in general, and problem solving in particular, is a continuous, dynamic process that evolves over time” (p. 39). In the current work, studying temporality in collaborative learning and CPS processes is inspired by the previous work of several authors (e.g., Avry et al., 2020; Csanadi et al., 2018; Kapur, 2011; Reimann, 2009, 2021; Reimann et al., 2011; Stahl, 2017; Swiecki et al., 2020).

Various methods are applied for analyzing processes and temporality in collaborative learning and CPS studies (for an overview, see Lämsä et al., 2021a; Reimann, 2009, 2021). For example, in epistemic network analysis (ENA), which is a discourse analysis technique that models the temporal co-occurrences of codes in discourse, temporality is studied by focusing on the recent temporal context and immediate events of CPS in the groups (e.g., Andrist et al., 2018; Csanadi et al., 2018; Swiecki et al., 2020; Williamson Shaeffer, 2017).

Lag-sequential analysis (LsA; e.g., Bakeman & Gottman, 1997; Bakeman & Quera, 2011) has been employed for modeling CPS processes using transitional probabilities between sequencies of collaborative action; in this way, researchers have demonstrated, for example, the significant relationship between variation in temporal patterns and variation in group performance (Kapur, 2011). A related sequential analysis approach, sequential pattern mining (e.g., Febrer-Hernández & Hernández-Palancar, 2012), has been applied, for example, for modeling students’ ill-defined problem solving (Norm Lien et al., 2020). Both methods model sequences, but if compared to LsA, sequential pattern mining has been used to mine sequences of events that are frequent (i.e., occur more often than a minimal level of frequency) in a dataset (e.g., Chen et al., 2017).

Temporality not only comes into play in quantitative terms (i.e., duration) but also has significance in terms of the order of events (Reimann, 2009, 2021). In the current study, the indicators of temporal events were the order and intensity of the social and cognitive states and state changes during task completion. The social and cognitive states refer to the coded, timestamped (social and cognitive) sub-elements of CPS (see Appendix 2), which are amalgamated back into broader social and cognitive strands (i.e., the first-order components of the CPS construct). The intensity of the states is related to how many of the different sub-elements of the social or cognitive strands appear at the same point in the CPS session. The state changes are based on calculations of inter-event distances as temporal distances between social states and cognitive states over the CPS session, whereas the calculated averages of states and state changes cover the entire CPS session (e.g., Abbott, 1990; see also Reimann, 2009); all are identified from the timestamped coded interaction of the groups. Accordingly, in this study, the indicators—the order and intensity of the social and cognitive states and the (averages of) states and state changes—are expected to pinpoint the interchange of joint efforts and related communication with the partners, which form the basis of collaboration and CPS (e.g., Dillenbourg et al., 2016; Roschelle & Teasley, 1995).

Finally, via qualitative cases, attention is turned back to the micro-interaction processes in the groups (Davis et al., 2015; Stahl, 2017). All groups are unique in terms of how the actions and interactions come together to form the shared practice (Stahl, 2017). In addition, case examples can provide intensification and offer explanations to help in the interpretation of the previous phases of analysis (Flick et al., 2004). Here, qualitative cases can show whether the differences in the temporal occurrence of CPS from the previous phase of analysis are visible at the micro-interaction level. In general, when carefully justified, cases can provide a thorough view of a complex social phenomenon (Baškarada, 2014). Cases do not aim to generalize to populations, but they can arrive, for example, at conceptualizing the regularities of small-group processes in CPS (Stahl, 2017).

Aims

This study applies analysis method triangulation to capture various layers of CPS enactment in small, co-located groups during a mathematics education course in teacher education. The research questions are as follows:

RQ1: How are the theorized CPS construct (comprising social and cognitive elements and their sub-elements) actualized in the groups’ interactions in two different task designs (i.e., technology-enhanced vs. using physical objects), interpreted as cumulative accounts?

RQ2: How do the elements of the CPS construct vary over time during problem solving in the different groups and different tasks? What differences are there in temporal occurrence in this regard?

RQ3: In what ways are the differences in the temporal occurrence of CPS elements visible in the qualities of interaction in the groups (exemplified as case examples)?

Methods

Participants and context of study

The participants were teacher education students (N = 15, 11 women, 4 men, mean age 27 years) from a Finnish university. At the outset, the participants were divided into four comparable groups of three to four students. Participation in the study was voluntary. As one student declined to participate in the research, only three groups were included in the analysis.

Course design and tasks

The course under study was an applied mathematics education course in experiential geometry. The course included seven meetings (90 min each) and working in groups outside the meeting hours. The meetings included the following sessions:

  • Session 1: Introductory session;

  • Sessions 2 and 3: CPS task demos;

  • Session 4: reflection session to discuss and reflect on the experiences of the previous CPS task demos;

  • Session 5: developing and sharing ideas for forthcoming workshops in schools;

  • Session 6: working in small groups to design, implement, and report on a CPS teaching experiment within a school context; and

  • Session 7: presenting and discussing the workshops with co-students.

The work in small groups resulted in jointly prepared reports that were assessed by the teachers using a scale of 1–5. In this paper, the focus is on the two CPS task demos (Sessions 2 and 3).

During the demos, the groups experienced CPS in terms of two different CPS task designs—a technology-enhanced task relying on virtual manipulatives using the dynamic GeoGebra software and a task using physical objects (giant tiles made from cardboard). The GeoGebra software was employed to support students’ learning by providing more opportunities to explore, analyze, and test their knowledge (Santos‐Trigo & Espinosa-Perez, 2002). Physical objects, or manipulatives, are physical materials (e.g., blocks and tiles) that are used to support learning and have tactile and manipulative properties (Manches et al., 2010).

The tasks (see Table 2) were open ended (Chan & Clarke, 2017), embracing experiential mathematics learning with math–art connections. In open-ended problem solving, the starting situation is not clearly defined, there are multiple correct answers to the problem, or there are many approaches to solving the problem. The difficulty level of the tasks was defined by the teachers in accordance with the preceding math course level.

Table 2 Descriptions of the Collaborative Problem Solving (CPS) Tasks (Sessions 1 and 2)
Full size table

The tasks were designed under the topic of tessellations. A tessellation (or tiling) means an arrangement of figures (i.e., tiles) such that they cover the plane without any gaps or overlaps of the tiles. The tile used in the course was Paul Gailiunas’s tile design, “Princess” (Fig. 1). This was selected because a wide range of different symmetrical patterns can be created based on this tile (Gailiunas, 2007).

Fig. 1
figure 1

Paul Gailiunas’s tile-design “Princess”

Full size image

For the CPS task demos, two settings were applied (Table 2). During the first CPS session, students were advised to use the software program (GeoGebra), and physical objects (giant tiles) were provided as a complementary tool. During the second CPS session, to create giant tessellations, students were advised to use the cardboard tiles as the primary tool and GeoGebra as the complementary tool.

The macro-script

As part of the pedagogical design, a macro-script was applied (Dillenbourg & Hong, 2008; Dillenbourg & Tchounikine, 2007). The script, as a coarse-grained pedagogical model, was aligned with the principles of the socio-cognitive learning approach defined by Zimmerman (2010); this approach emphasizes learning as a complex metacognitive and social process. The script (Näykki et al., 2015, 2017) was orchestrated by the teachers and implemented as physical cards that included question prompts to be answered as a group. In short, the design rationale included three different phases: (a) the forethought phase, where groups set their goals for their learning (“Orientation” card); (b) the monitoring phase (“Checkup”); and (c) reflection (“Reflection” card; see Fig. 2 for the phases and question prompts used). At the onset of the session, teachers gave instructions on how to use the cards and noted the approximate timepoints at which students could focus on them. During the sessions, the groups were gently prompted by the teachers if it seemed they did not follow the given guidelines.

Fig. 2
figure 2

Overview of the three phases of collaboration macro-script for supporting collaborative problem solving (CPS) in small groups

Full size image

Data collection

The data collection included observations, taking notes, and video recordings of group interactions as the primary source of data. Small GoPro® cameras, equipped with suction cups that enabled them to be flexibly positioned in the classroom, were used for video recordings. In addition, when utilizing GeoGebra, CamStudio™ software (http://camstudio.org) was used for the screen activity recordings during the sessions. The full dataset comprised video recordings of the sessions (26 h), screen capture videos (3 h), and students’ written reports on their teaching experiments. In this paper, the emphasis is on video data.

Data analysis

The main analysis process combined directed content analysis (Phase 1), followed by process analysis and visualizations (Phase 2); this set the stage for the qualitative cases (Phase 3; for an overview of the analysis process, see Fig. 3). As a preliminary phase, based on observations during the data collection, a time-indexed narrative was produced that allowed us to go through the data corpus rapidly (e.g., Barron et al., 2015).

Fig. 3
figure 3

Overview of the analysis process, including the preliminary phase and three main phases

Full size image

Phase 1: directed content analysis of the interaction data

In the first phase, to identify the interacting CPS elements at the group level, directed content analysis was applied (e.g., Hsieh & Shannon, 2005; Humble, 2009). The categorization matrix and the operational definitions of each category were modified from the CPS codebook by Hesse et al., (2015; see also Care et al., 2016; Scoular et al., 2017). Since the framework was designed for the targets of automated analysis of CPS skills, it included a large number of code variables. For manual coding, the number was too large. The condensed frame and the selection of codes were based on previous research and the current understanding of the key elements related to the socio-cognitive approach to CPS (Appendix 1).

In this phase, the interest was in how the groups enacted the CPS tasks in the two task designs. That is, the aim of this phase was to uncover how different or similar the groups (Groups B, C, D) were regarding the appearance of the selected CPS elements in the different tasks (technology-enhanced vs. using physical objects) by relying on simple descriptive counts in the coded data.

In brief, the categories applied for analyzing the social elements set (Appendix 1) included “Participation” (originally the “interaction” sub-element) as the willingness to share information, externalize one’s thoughts, and participate in the different stages of problem solving. “Perspective Taking” means, on the one hand, the ability to understand and consider others’ perspectives and contributions (originally named “responding skills,” here labeled “Perspective Taking A”), and on the other, the ability to tailor one’s contributions to others (here called “Perspective Taking B”).

From the cognitive elements set (Appendix 1), “task regulation” (based on the “resource management” sub-element, here called “Coordination”) is included. Sub-element “collects elements of information” is here called “Task Exploration,” and “rules: ‘if…then’” is called “Problem Analysis.” The sub-element “reflects and monitors” is here renamed “Problem Reanalysis.”

The video recordings of the two problem-solving sessions (26 h of video data for all three groups) were analyzed with the assistance of Atlas.ti® data analysis software (Friese, 2014). The first author trained a research assistant for the coding with several data examples until an agreement was reached, and during the process of analysis, they jointly discussed the interpretations and unclear cases. For qualitative rigor (Thomas & Magilvy, 2011), the deductive coding procedure was carried out three times. The initial coding was conducted on the video recordings. Next, to reduce coding errors and to look for similarities within and across groups, the video recordings were transcribed, and the data were re-coded in parallel with transcribing the videos; the codes and the timestamped segments in the video recordings were elaborated and rechecked during this process. Finally, a third round of elaboration of the coding was performed on the transcribed, written data. In the analysis, when reading the transcriptions line-by-line, an episode or passage with natural borders was captured, delimited by a thematic shift. Thus, a minimum unit, where a certain criterion of the predetermined category was observed, was adjusted for analysis (Chenail, 2012). The analysis did not follow a time-based segmentation of events (Sinha et al., 2015), but the length of the coded segments was based on their contents. Therefore, the segments all had unique lengths. Those parts of the video where any criteria segments could not be found were left uncoded. (For representative examples of the coded data and data captures, see Appendix 2).

The coded data were exported from the Atlas.ti® data analysis software to Microsoft Excel for organizing, analyzing, and visualizing the data. Based on the categorized data, descriptive statistics were used to summarize the appearance of CPS elements in different groups and task designs. First, the proportional distribution of the coded CPS elements was calculated in terms of the following: (a) different types of tasks (technology-enhanced vs. utilizing physical objects) and (b) different groups (Groups B, C, and D). In addition, the standard deviation (SD), average, and median were estimated for the CPS elements in the different groups and task designs. These cumulative accounts did not point to individuals but were conceived as group-level constructs (Kapur et al., 2008).

Phase 2: focusing on the temporality in the data

In the second phase, the focus was on the temporal occurrence of CPS elements as inter-event temporal distances between social and between cognitive states over the two task sessions (e.g., Abbott, 1990; see also Reimann, 2009). Accordingly, the order and intensity of the appearance of the social and cognitive states over the course of the two problem-solving tasks was first calculated. Second, to compare the groups, and especially to see whether the general characteristics of the temporal nature of CPS activity in the groups were different in the two tasks, definite ratios were calculated as follows: (a) the averages of the social and cognitive states, (b) the averages of state changes, and (c) the proportional differences between the averages of the state changes at the group level in the two task designs.

The social and cognitive states were created by amalgamating the coded CPS social sub-elements and cognitive sub-elements into broader social and cognitive strands. When calculating the intensity of the social state, the state level takes a value of 1 if all three sub-elements of the social strand (Participation, Perspective Taking A, Perspective Taking B) appear at the same point. (For the values for calculating social states, see Table 3). For the cognitive state, the state level takes a value of 1 if all four of the sub-elements of the cognitive strand (Coordination, Task Exploration, Problem Analysis, Problem Reanalysis) appear at the same point. (For the values for calculating cognitive state, see Table 4). In the visualizations, the intensity values for the social or cognitive state give a quick overview of the possible (co)-occurrence of the CPS sub-elements over the CPS process.

Table 3 Values for calculating social states
Full size table
Table 4 Values for calculating cognitive states
Full size table

For calculating the averages of the social and cognitive states, the sum of values calculated for social (or cognitive) states (see Tables 3 and 4) were divided by the number of these values. The average state change simply describes the average amount of change (i.e., “travel”) from one social (or cognitive) state to another. (For the values for calculating the distances between social states and the distances between cognitive states, see Tables 5 and 6).

Table 5 Distances between social states
Full size table
Table 6 Distances between cognitive states
Full size table

An example of the coded interaction and calculations of the social and cognitive states, as well as the social and cognitive state changes (labeled “travel”), are given in Fig. 4. The example is from the onset of the phase when the script card is discussed (“Orientation”).

Fig. 4
figure 4

Section from the coded excel data file (Task 1, the onset of the task). The coded data file includes the start and end timestamp of the coded interaction, the actor and the (Atlas.ti) id of the coded fragment, the coded seven collaborative problem solving (CPS) sub-elements, the timestamp running from the beginning of the video, the values for cognitive and social states (COGNITIVE, SOCIAL) as well as cognitive state changes (CTRAVEL) and social state changes (STRAVEL)

Full size image

This phase also incorporates visualizing the temporal interchange of social and cognitive states in the groups and the tasks. This visualized interchange, showing the alignment of CPS actions over time, is purported to provide an overview of the actualized CPS activity during the tasks. The visualizations, developed for this study and called “CPS circles,” display (clockwise) the state changes and their intensity during the sessions. It also shows the period when the question prompts of the script cards were discussed (see Fig. 5 for an example of a CPS circle).

Fig. 5
figure 5

Example of a CPS circle. The dark gray color points to the social states of CPS skills, while the light gray color points to the cognitive states. The three areas marked with the transparent, lightest gray color represent the timepoint at which the script cards were discussed. The intensity of the appearance, reflecting the number of social or cognitive sub-elements activated at the specific point, is visualized with the height of the “ray.”

Full size image

The dark gray color points to the social states of CPS skills, whereas the light gray color points to the cognitive states. The three areas marked with the transparent, lightest gray color represent the timepoints at which the script cards were discussed. The intensity of the appearance, reflecting the number of social or cognitive sub-elements activated at a specific point, is visualized by the height of the “ray.”

Phase 3: qualitative cases based on selected interactions

Based on the results of the previous phase (i.e., the proportional differences between the averages of state changes in the two tasks), in Phase 3, the focus returned to the interaction data to better understand the group-level differences in this regard. In the analysis, the aim was to concentrate on identifying the characteristics of interaction when the groups were completing the second task and searching for chains of events that would exemplify the differences.

Results

RQ1: How did the theorized collaborative problem-solving elements actualize in the groups’ interactions?

In general, the cumulative accounts did not indicate a large variation between the different groups in either task design. When focusing on the sub-elements of the CPS, Problem Reanalysis from the cognitive set formed the most frequent sub-element in both the technology-enhanced task (see Table 7) and the task utilizing physical objects (see Table 8). In Task 1, the Participation sub-element from the social set was the second largest sub-element, whereas Task Exploration from the cognitive set was the third largest sub-element, except in Group D, where Task Exploration formed the second largest element from all the sub-elements, followed by Participation as the third largest sub-element. In Task 2, the largest element, Problem Reanalysis, was followed by Participation as the second largest and Coordination from the cognitive set as the third largest sub-element. In both task designs, it seems that cognitive elements were more identifiable from the interaction data but with slightly different task-related emphases: Task 1 involved more task exploration than coordination, while Task 2 involved more coordination than task exploration.

Table 7 Proportions of different collaborative problem solving (CPS) elements in different groups (Task 1, Technology-Enhanced Task Design)
Full size table
Table 8 Proportions of different collaborative problem solving (CPS) elements in different groups (Task 2, Task Design Utilizing Physical Objects)
Full size table

From the descriptive statistics, when focusing on the dispersion of the CPS sub-elements, the low SD implies a more or less equal appearance of CPS elements in the different groups and task designs, as indicated in Tables 9 and 10. The measures of central tendency (average, median) indicate the same result.

Table 9 Standard deviation (SD), average, and median (M) of the collaborative problem solving (CPS) elements in task 1 (Technology-Enhanced Task Design)
Full size table
Table 10 Standard deviation (SD), average, and median (M) of the collaborative problem solving (CPS) elements in task 2 (Task Design Utilizing Physical Objects)
Full size table

RQ2: How did the collaborative problem-solving elements vary over time?

Because the results from the first phase of analysis did not show a large variation in different groups or tasks, the temporal information of these coded data was elaborated further. Here, it was asked how the elements of CPS varied over time by focusing on the order and intensity of social and cognitive states (Figs. 6, 7, 8, 9, 10, 11) and comparing the (average) alignment of the social and cognitive states and state changes in the groups regarding the two different tasks (Table 11).

Fig. 6
figure 6

Collaborative problem solving (CPS) circles from Group B in Task 1 (technology-enhanced task). The figure on the left (dark grey color) presents social states and state changes, the figure on the right (light grey color) cognitive states and state changes, the script discussions are marked with the lightest grey color

Full size image
Fig. 7
figure 7

Collaborative problem solving (CPS) circles from Group C in Task 1 (technology-enhanced task). The figure on the left (dark grey color) presents social states and state changes, the figure on the right (light grey color) cognitive states and state changes, the script discussions are marked with the lightest grey color

Full size image
Fig. 8
figure 8

Collaborative problem solving (CPS) circles from Group D in Task 1 (technology-enhanced task). The figure on the left (dark grey color) presents social states and state changes, the figure on the right (light grey color) cognitive states and state changes, the script discussions are marked with the lightest grey color

Full size image
Fig. 9
figure 9

Collaborative problem solving (CPS) circles from Group B in Task 2 (task design utilizing physical objects). The figure on the left (dark grey color) presents social states and state changes, the figure on the right (light grey color) cognitive states and state changes, the script discussions are marked with the lightest grey color

Full size image
Fig. 10
figure 10

Collaborative problem solving (CPS) circles from Group C in Task 2 (task design utilizing physical objects). The figure on the left (dark grey color) presents social states and state changes, the figure on the right (light grey color) cognitive states and state changes, the script discussions are marked with the lightest grey color

Full size image
Fig. 11
figure 11

Collaborative problem solving (CPS) circles from Group D in Task 2 (task design utilizing physical objects). The figure on the left (dark grey color) presents social states and state changes, the figure on the right (light grey color) cognitive states and state changes, the script discussions are marked with the lightest grey color

Full size image
Table 11 Averages of the social and cognitive states in tasks 1 and 2; averages of state changes; and proportional differences of state changes between task 1 and 2
Full size table

The results suggest that in Group B, the average occurrence of social states is slightly higher in Task 2 than it is in Task 1 (Task 1: 0.115, Task 2: 0.122), but the value for state changes regarding social elements is remarkably lower in Task 2 compared with Task 1 (see Table 11). (The calculated proportional difference between the tasks is 35.2% less). It seems that at the group performance level, more social states appeared in the task with the physical objects than with GeoGebra, but the continued “social moments” in the group interaction, on average, were longer. Figures 6 and 7 visualize the actualized interchange of the social and cognitive states in Group B.

For Group C (see Table 11), there was an increase in the average occurrence of social states in Task 2 compared with Task 1 (Task 1: 0.135, Task 2: 0.164). In addition, the value for the average of social state changes was higher in Task 2 (using physical objects) compared with technology-enhanced Task 1. (For Task 2, the calculated proportional difference is 10.7% more in this regard). This indicates that although more social states occurred on average, the social moments were slightly shorter in Task 2 compared with Task 1. Figures 8 and 9 visualize the actualized interchange of the social and cognitive states in Group C.

In Group D (Table 11), there was a decrease in the averages of the cognitive states (Task 1: 0.246, Task 2: 0.208), as well as a decrease in the average of the cognitive state changes in Task 2. The calculated proportional difference between Tasks 1 and 2 was 24.6% less in this regard. This means that, in Task 2, fewer “cognitive moments” occurred, but on average, these moments lasted longer than they did in Task 1. Figures 10 and 11 visualize the actualized interchange of the social and cognitive states in Group D.

RQ3: In what ways were the differences in the temporal occurrence of CPS elements visible in the qualities of interaction in the groups?

In the third phase, to search for explanations for the results from the second phase of analysis, the focus returned to exploring the interaction processes in the groups. With the case examples, as “a sensible follow-up” to the previous levels of analysis (Flick et al., 2004), the interest was in seeking clarifications for the proportional, group-related differences between the averages of the state changes regarding the two task designs.

Accordingly, different qualities of interaction were found, varying from the lowest quality as chains of individual members’ autonomous actions (Group B, Vignette 1) to strings of synchronized and coordinated actions (Group D, Vignette 2), and finally, contemplative, reflective interactions as a group (Group C, Vignette 3), which could be labeled as the highest quality in terms of interaction in the groups. These different characteristics of the interaction are depicted in Vignettes 1–3.

Vignette 1. Group B: chains of individual members’ autonomous actions

As Table 11 shows, in Group B, the second task resulted in a decrease of the average of state changes of social aspects (− 35.2%). The microlevel perspective on interaction data provides a view of a situation comprising chains of localized moments of independence (Davis et al., 2015). Students seemed to have shared intentions (a conceptual goal to solve the problem), but at many moments, they went off individually or acted as varied subgroups to solve the problem (Stahl et al., 2014). While the subgroups worked in parallel, the members communicated their individual efforts. Whereas two of the group members (Lotta and Samuel) focused on task completion individually, Mia did not develop an individual solution, but instead, moved between the other two members, her contributions remaining at a rather superficial level. She mainly asked questions and commented on the work of Lotta and Samuel, without participating in developing their ideas further. Finally, as Samuel failed with his tessellation, Lotta’s work was taken as the final product of their group. In general, the group unsystematically manipulated the tiles to build giant tessellations. The conceptual goals were not communicated, and the mechanical coordination of the group did not seem intentional (Davis et al., 2015).

Vignette 1

Example from a chain of individual members’ autonomous activities in group interaction

Actor

Quote

1. Student 1 (Mia) (asking Lotta):

“Anything taking shape?”

2. Student 2 (Lotta):

“Nothing significant.”

3. Student 1 (Mia) (asking Samuel):

“Do you have something, then somehow everything is, like, positioned differently?”

4. Student 3 (Samuel):

“I guess so. I don’t know if you can just, like, keep going on with this sort of thing somehow. (Now there’s something starting to –) from there I set out.”

5. Student 1 (Mia) (to Samuel):

“So, as I watched that you made –”

6. Student 3 (Samuel):

“Now it started to come up here, by this kind of a formula.”

[Julia is shaping her pattern; Mia moves to watch Samuel’s working]

7. Student 1 (Mia):

“That’s, like, the same (part of the same pattern somehow), isn’t it?”

The teacher interrupted the work and reminded the group that they should continue working on the final version of the tessellation. At this stage, Lotta and Samuel continued with their individual tessellations in parallel, communicating over their progression.

Actor

Quote

8. Student 2 (Lotta):

“Well then.”

9. Student 3 (Samuel):

“This will be sort of random.”

10. Student 2 (Lotta):

“This will be a flame.”

11. Student 3 (Samuel):

“Yes, this can be extended.”

12. Student 2 (Lotta):

“Help, you’re (Samuel) now coming over to my pattern.”

13. Student 2 (Lotta):

“I got inspired to make a flame. I should just, there’s now a partly red and blue flame, then.”

14. Student 3 (Samuel):

“[whistling] This is working.”

15. Student 2 (Lotta):

“Is it?”

16. Student 3 (Samuel):

“Perhaps. Yes, there’s something for all the holes.”

Lotta then asks whether the group could merge their individual efforts (mainly her and Samuel’s drafts).

Actor

Quote

17. Student 2 (Lotta):

“Hey, shall we still combine these?” [a short laugh]

18. Student 1 (Mia):

“I was thinking that if they should be combined, so that.”

19. Student 3 (Samuel):

“A bit difficult, perhaps.”

20. Student 2 (Lotta):

“Yes, perhaps.”

21. Student 3 (Samuel):

“Because they are at different distances from each other. We would need to move the whole pattern. It wouldn’t really be feasible.”

22. Student 1 (Mia) (to Lotta):

“Take some yellow from there now. (Would it be) still like the same pattern, with different colors? As there’s a blue center –”

23. Student 3 (Samuel):

“Hey, now there’s a problem. Now it’s not working anymore.”

Vignette 2. Group D: strings of synchronized and coordinated actions

Table 11 shows how, in Group D, the second task resulted in a decrease in the average of cognitive states and the average of state changes (− 24.6%). When zooming back in on the group interaction, it seems that the group stayed together over the process and jointly discussed and decided what they were doing or should do; however, the goals were less targeted and immediate. The group members constantly externalized their thoughts, which was not the case in Group B. At the beginning, the group discussed their previous session and decided to set more goals and work more logically than previously. Here, they discovered a “diamond problem,” as they called it; that is, they found themselves in a situation in which they would have needed extra, diamond-shaped tiles to fill all the gaps in their designed patterns (see also Hähkiöniemi et al., 2016).

Vignette 2

Example from synchronized and coordinated actions of the group (Group D)

Actor

Quote

1. Student 1 (Leo):

“((laughs)) Then we would only need a sort of small diamond shape so that we could in a way make this work.”

2. Student 4 (Anna):

“((laughing)) Yes.”

3. Student 2 (Lisa):

“So, this isn’t really like working in any way now, is it?”

4. Student 1 (Leo):

“Nope.”

5. Student 4 (Anna):

“No, it isn’t.”

6. Student 1 (Leo):

“No, as it calls for a sort of diamond. Then it would become like that.”

7. Student 3 (Nea):

“Diamond-shaped sweets.”

8. Student 1 (Leo):

“Or, yes, diamond shapes for here, so then it would become like that.”

9. Student 4 (Anna):

“Yes.”

10. Student 3 (Nea):

“Well what if, could that one be set somehow in another direction, or like, could we get it here somehow?”

11. Student 2 (Lisa):

“Yes.”

12. Student 4 (Anna):

“But isn’t that-”

13. Student 1 (Leo):

“That’s quite an impossible angle; you cannot really do anything about it. ((Notices Nea’s solution)) No, I mean, you actually can do it that way.”

14. Student 4 (Anna):

“Yes, you actually can.”

15. Student 2 (Lisa):

“But I don’t know, I think that the problem will always come up at some point, the problem that -”

16. Student 4 (Anna):

“That it requires some supplementary job there.”

17. Student 3 (Nea):

“The diamond.”

Vignette 3. Group C: contemplative, reflective interactions as a group

As Table 11 shows, in Group C, the second task resulted in an increase in the average of social states and a slight increase in the social state changes (10.7%). Accordingly, the microlevel perspective on the interaction data of Group C provides us with a view of contemplative interactions in which the group members vocalized both immediate conceptual goals and goals going beyond the moment-to-moment interactions (Davis et al., 2015). The overarching goals for their discoveries were verbalized explicitly by relying on mathematical concepts, and the group was willing to verify the final pattern with GeoGebra. As a “warm-up” exercise, the group started by forming a pattern similar to the one made in the previous session with GeoGebra. Next, the group created a new, star-shaped pattern. The students used mathematical terms (i.e., reflection, rotation, translation), but they also wanted to make the pattern esthetic via the use of colors. To ensure the final pattern was mathematically correct, the group decided to test it with GeoGebra. The next excerpt is from the phase after the warm-up exercise, when the group started working on the real pattern. The group was oriented toward joint problem solving and communication, with individual members building on the words and ideas of their co-members (Stahl et al., 2014). The central, primary idea seems to have originated from Student 1 (Linus).

Vignette 3

Example from Contemplative, Reflective Interactions in the Group (Group C)

Actor

Quote

1. Student 1 (Linus):

“I was thinking that, if you make a straight line between these points, you know. No, but it won’t work because (–).”

2. Student 3 (Tina):

“So, is it like, so that—”

3. Student 1 (Linus):

“The way it goes, yes. So, then it, the line, would go with a 90-degree angle through the midsection between these points here, wouldn’t it? So, then this would be mirrored perfectly opposite to each other, wouldn’t it? If it’s like this angle here (even -) and then the line goes exactly there, like in between, like just in between that line, so then this would be mirrored on top of that, yes. And then it could be turned like there, I guess.”

4. Student 3 (Tina):

“Does it go like this?”

5. Student 1 (Linus):

“Yes, because this (–) there. No, no it won’t! No, because now, now it is mirrored like this because now this is the (-).”

 

“Okay, and then this deflects this from there in parallel with this one in this way.”

6. Student 4 (Sara):

“(It is deflected in this direction.) It is deflected to this direction as here it was deflected from here to there. So, wouldn’t it be the same way there, then?”

7. Student 2 (Hanna):

For this [it is] again the same, the line to the middle.”

8. Student 1 (Linus):

“Yes.”

9. Student 3 (Tina):

“Yes, so will it be then -”

10. Student 4 (Sara):

“Is this now deflected here, to this direction?”

11. Student 1 (Linus):

“Yes, so then this is deflected in relation to this. In relation to this line. And then, wait a sec, whereabouts should we find it (symmetry) again?”

12. Student 4 (Sara):

“But then from there again.”

13. Student 2 (Hanna):

“Look, we should get this one here -”

14. Student 3 (Tina):

“[Yes, and if we would have there;] Just put that one there.”

15. Student 2 (Hanna):

“Here in relation to this.”

16. Student 3 (Tina):

“Hmm-m. But will this shape be like, you know.”

17. Student 1 (Linus):

“Hmm-m.”

18. Student 4 (Sara):

“Like what?”

19. Student 3 (Tina):

“Just put those there. And then, was it like, which way did it go now?”

20. Student 1 (Linus):

“Now it goes to its direction.”

21. Student 2 (Hanna):

“Yes, so far, so good!”

22. Student 2 (Hanna):

“(There’s a certain logic.)”

23. Student 4 (Sara):

“Yes, it has.”

24. Student 2 (Hanna):

“How the next layer would start, that’s what I don’t (still really) ((laughs)).”

25. Student 3 (Tina):

“Yes, that’s what I was just thinking that -”

26. Student 2 (Hanna):

“Should we take a picture of this phase? Until now it seems to work.”

27. Student 1 (Linus):

“Yes.”

 

(The group takes pictures of their product.)

Discussion

This paper presented a method, together with empirical investigations, to capture various layers of CPS as a small-group enactment during open-ended, co-located CPS. The data were collected during an experiential mathematics course in geometry in the context of initial teacher education and included two variations of the CPS tasks. The method was based on analysis method triangulation and combined three phases. In the first phase, the descriptive counts on the coded data did not show a large variation in how the CPS construct was actualized in the different groups or tasks. Yet, in the second phase, when further elaborating on the coded data regarding the embedded temporal information, the results revealed both group- and task-related differences. In the third phase, via qualitative cases, the analysis zoomed back in on the microlevel, and exemplified diversity in the interaction quality of the groups.

When focusing on the empirical outcomes, the analysis method triangulation provided access to the diversity of the CPS enactment processes in the different groups and tasks. In terms of the CPS construct, the study made the processes visible as exchanges of actions and understandings that merge into unique CPS situations (Stahl, 2017). To a large extent, as visible in the case-based examples, the processes did not follow linear or rational models of problem solving as “logical deductions in individual minds” (Stahl, 2017, p. 117); instead, they included “breakdown” situations yielding actions, negotiations, and finally, explicit rounds of agreement as a group (Stahl, 2017). Accordingly, collaborative situations are often exploratory, where no clear plans or procedures exist for problem solving or organizing joint efforts (Baker, 2015). However, in these situations, students explore the problem space, and by doing so, they can gain a deeper conceptual understanding of the problem via co-elaboration of knowledge and understanding (Baker, 2015).

In general, the empirical notions from the different layers pinpoint the complexity of the CPS and the unpredictability of its enactment. This resonates well with the many challenges pronounced in earlier research, such as issues related to coordination of joint interactions and contributions (e.g., Andrist et al., 2018; Baker, 2015) and the students’ equal participation in the groups (e.g., Barron, 2000, 2003). Moreover, the different methods implemented in this study were not opposed to each other, and as argued by Kelle and Erzberg (2004), “the results of qualitative and quantitative approaches can converge, complement or contradict one another and each of these possibilities can be beneficial to the research process” (p. 176). Here, the process of quantifying the results of the qualitative analysis, focusing on the temporal characteristics in these data, and returning to qualitative explorations permitted a fuller treatment of the empirical material and its layers, with all the analysis phases still relating to the same concept or phenomenon.

Despite the many critiques of coding and quantifications (i.e., pointing to the fragmentation of the data and the phenomenon; see Ludvigsen et al., 2018), the first approach, combined with simple descriptive statistics (Jeong et al., 2014), was relevant because it allowed an overall understanding of the appearance of the CPS elements in the groups’ interaction to be developed and basic information about the group- and task-related differences to be obtained (Swiecki et al., 2020). In addition, these data formed the input material for the next phases of the analyses. However, manual coding is time-consuming and resource intensive (Graesser et al., 2018). Until now, a variety of automated text-analysis methods based on natural language processing have been developed to analyze the contents of communication, including latent semantic analysis (e.g., Dowell et al., 2020; Landauer et al., 1998, 2007) and automated content analysis for online (e.g., Mu et al., 2012) and manually transcribed face-to-face communication (Lämsä et al., 2021a, 2021b). Although these methods aid and expand the manual approach, ensuring the accuracy of the outcomes is still requisite (Graesser et al., 2018). That is, to guarantee that the linguistic “fine points” of human communication involved in collaboration would not be lost in the analysis; at the same time, to pay attention to the situational context of collaboration (Wise & Schwartz, 2017).

The second approach, relying on the temporal information embedded in the coded interactions, showed differences in the group-related temporal qualities regarding the tasks. When carefully designed, methods that take temporality into account can test and add to the process theory of collaborative learning and CPS (Kapur, 2011; Reimann, 2009; Wise & Schaeffer, 2015). However, Kapur (2011) recalls the limitations of the quantitative, event-based analysis as reductions of the richness and complexity of group processes, and thus, advocates using a comprehensive analytical scheme and triangulating the findings with microlevel, qualitative analysis. In the current study, the second approach directed the analysis toward the selection of units for further investigation via qualitative cases.

Whereas the second phase provided evidence of the differences between the groups and the tasks, the third approach contributed to more detailed clarifications of these analysis outcomes. Accordingly, a detailed interaction analysis of carefully selected cases has the full potential to add to the theoretical understanding of CPS (Gauvain, 2018; Reimann, 2021; Stahl, 2017). In general, providing access to different layers of CPS, the method can intensify and extend our understanding of CPS as a dynamic, group-level construct. However, to reach the theoretical underpinnings of CPS as a group-level enactment requires repeated and refined application of the measures that formed the analysis triangulation. That is, the analysis can be performed on a larger amount of data (both in terms of the sample size and timescale) with advanced, computational methods that make the analysis more automated and thereby more scalable.

Conclusion

Taken together, the paper provided a detailed description of the various layers of CPS enactments as a group-level construct, extending from individually oriented research approaches in CPS. To accomplish this, the study triangulated three analysis approaches. Each approach was beneficial in the process of deepening our understanding of CPS and contributing methodologically to the growing body of research on temporality in data. While the sample size cannot be generalized to wider populations, the case examples revealed diverse group adoption patterns that reproduced the previous findings regarding interpersonal interaction in the groups as being largely explorative, intuitive, and less aligned with linear problem solving. All these aspects relate to the high complexity of the actualized CPS construct and point to the need to consider its manifestations in long-term study designs of wider populations of students, researched with advanced methods that can grasp the causal mechanisms of the multiple interacting elements that enter into successful CPS processes in small groups, aiming for increased and relevant theoretical understanding of CPS.