Introduction

Gaining greater proficiency in problem-solving is vital for future learning, effective participation in society, and success in personal and professional endeavors (OECD, 2013). Problem-solving is defined as a cognitive process that transforms a given problem into a goal when no obvious method of solving it is available (Mayer, 1990; Mayer & Wittrock, 2006; Reeff et al., 2006). Problem-solving competency refers to the ability to engage in cognitive processes in order to understand and solve problems where a solution is not obvious, ultimately enabling people to achieve their full potential as constructive and reflective citizens (OECD, 2013). Problem-solving does not rely solely on prior knowledge and reproduction of accumulated knowledge, but also requires acquiring and using new knowledge or adapting old knowledge to solve new problems. Klieme (2004) argues that problem-solving skills are essential for achieving more than just a basic level of competency, and there is evidence to support the need for further skills that go beyond traditional learning. Surprisingly, PISA 2012 reported that about 20% of 15-year-olds from OECD countries were only able to solve straightforward problems when a familiar situation was presented. Furthermore, only 11.4% of top performers were able to solve complex problems systematically (OECD, 2014). PISA 2012 further reported that the difference between the highest- and lowest-performing OECD countries on problem solving was 113 points, and the difference between the highest- and lowest-performing countries more broadly was 163 points. Within countries and economies, however, even larger gaps separate the highest- and lowest-performing students (OECD, 2014). In light of the aforementioned studies, it is imperative to incorporate problem-solving into school education, and to reduce the gap between high- and low-achieving students.

Importantly, many studies have suggested the notion that problem-solving competency can be developed through education that helps to foster a deeper understanding of knowledge and prepare students to apply their knowledge in novel situations (Adey et al., 2007; Heller & Hollabaugh, 1992; Klauer & Phye, 2008; OECD, 2014). Problem-solving prepares learners to reason effectively and to bridge knowledge gaps through observation, exploration, and interaction with unknown problems. Physics is a crucial subject for students because it forms the essential foundation of their future scientific and technical capabilities (Angell et al., 2004). Physics is widely regarded as one of the more difficult subjects for students because of its abstract nature and subsuming more underlying concepts (Brown, 1993; Osborne & Freyberg, 1985; She, 2002, 2003, 2004a, b, She & Liao, 2010; Yildirim et al., 2021). Consequently, learning physics through problem-solving is plausible to facilitate students' understanding of these underlying abstract concepts and more efficiently apply them to problem-solving.

Collaborative learning offers students the opportunity for co-construction of a shared understanding of knowledge and meaning-making of the content (Fischer et al., 2013; Stahl, 2006). Collaboration is a means of engaging students with learning material and helping them to gain a deeper understanding of the content (Hmelo-Silver, 2004), thereby improving their collaborative skills and learning performance (Chen et al., 2018; Jeong et al., 2019). Some studies have suggested that collaborative learning improves students’ learning outcomes more than does individual learning (Diziol et al., 2007; Mullins et al., 2011), while others have stated that collaborative learning may not always be more effective than individual learning and that peer interaction may cause cognitive interference, resulting in process losses (Kraut, 2003; Wang et al., 2011). In other words, for students able to learn successfully on their own, collaboration was more of a hindrance than a benefit to performance. Although many studies have examined the difference between cooperative learning and individual learning, the evidence remains mixed.

Problem solving has been used as an instruction for decades and has been confirmed as an effective method of facilitating individual students' problem-solving abilities and knowledge development (Cheng et al., 2018; Hambrick & Engle, 2003; Lucangeli et al., 1998; She et al., 2012; Yu et al., 2010). As opposed to assessing individual problem-solving in 2012, PISA 2015 assessed collaborative problem-solving, which caught educators' attention. PISA 2015 and several studies call for collaborative problem-solving due to an ever-increasing demand for both collaboration and problem-solving skills to resolve non-routine problems in today’s workplaces (Deming, 2015; Griffin & Care, 2014; OECD, 2016). Collaborative problem solving can facilitate the development of students’ problem-solving competencies (Malik et al., 2019) and gaining of content knowledge (Harskamp & Ding, 2007; Heller et al., 1992). Studies have shown that collaborative problem-solving (CPS) and individual problem-solving (IPS) can improve students’ problem-solving abilities when applied separately. However, few studies have compared computer-based CPS with computer-based IPS or examined whether the former is more effective than the latter. Therefore, the present study aims to investigate whether computer-based CPS enhances students’ learning of physics concepts and problem-solving skills more than does computer-based IPS.

Problem-solving process

Problem solving involves representing and manipulating various types of knowledge in the problem-solver’s cognitive system (Mayer & Wittrock, 2006). The process of problem-solving begins with identifying the problem to be solved and then planning and implementing a solution with monitoring and evaluation of progress throughout the activity (OECD, 2013). In detail, it involves understanding a problem situation, distinguishing between facts and opinion, formulating a solution, identifying relationships between variables, selecting a strategy, determining cause and effect, communicating the results, and organizing information in a logical manner (OECD, 2013). Each of the stages in the problem-solving process may use one or more reasoning skills, such as deductive, inductive, correlational, analogical, combinatorial, and multidimensional reasoning. In practice, problem-solvers often switch between these reasoning skills when gathering evidence and assessing potential solution paths before settling into a preference for one technique over another to find the answer to a given problem (Adey et al., 2007; Klauer & Phye, 2008). In light of these studies, problem-solving competency including reasoning skills can be taught and modeled in schools. Studying learners' problem-solving approaches provides insight into how they use thinking skills and general cognitive approaches to overcome challenges (Lesh & Zawojewski, 2007). In the present study, computer-based problem solving is used to improve students' conceptual understanding and problem-solving skills in physics, as well as to identify the types of reasoning skills they use.

Computer simulation in physics problem solving

Most physics concepts are abstract and unobservable, subsume underlying concepts, and involve mathematical formulations; these characteristics lead to students' difficulties in understanding physics concepts (Brown, 1993; Osborne & Freyberg, 1985; She, 2002, 2003, 2004a, b, She & Liao, 2010; Yildirim et al., 2021). According to Thong and Gunstone (2007), the electromagnetism concept is highly abstract, and students tend to pay attention to its mathematical form rather than its qualitative representations. From the perspective of constructivism, everyone constructs their knowledge based on existing schema; therefore, actively participating in the learning process is vital. Simulations can reveal invisible, abstract, and microscopic phenomena that are difficult to observe in the real world (Akpınar, 2014; Chou et al., 2022; Lu & Lin, 2017; Sinensis et al., 2019), and thus help students to construct knowledge by observing concrete simulated phenomena (Saab et al., 2012).

Students can observe the animation by modifying the parameters or experimental components of the simulation system. The interactivity of computer simulation provides students the opportunity to actively participate in the process of constructing knowledge and establish cause–effect relations through manipulation of the simulated component and observation of the corresponding results (Franco, 2008; Schwier & Misanchuk, 1993). Many previous studies have reported that simulation-based problem-solving can help students to generate more expert-type scientific procedures, obtain correct solutions more often, and show better problem-solving performance and a positive attitude to learning problem-solving (Andrews-Todd & Forsyth, 2020; Ceberio et al., 2016; Mercier & Higgins, 2014; Rutten et al., 2012). Khan (2011) proposed that simulation-based material supported students’ conceptual understanding and encouraged greater engagement during learning. The OECD (2013) suggested that the available external resources (such as access to tools) and the environment in which the problem-solving takes place (e.g., an examination setting) will affect the way a solver approaches and engages with the problem. In the present study, the computer simulation was specifically designed for physics problem-solving, which allowed students to generate solutions, implement their solutions via simulation, and observe the simulation results.

Learning individually and collaboratively

Learning takes place at an individual level, where knowledge is actively constructed through the assimilation and accommodation of information to move from disequilibrium to equilibrium (Piaget, 1937). Learning collaboratively, where knowledge is constructed through teamwork toward a shared goal, is considered an essential skill for 21st-century learners (Yilmaz & Yilmaz, 2019). Social development theory suggests that students interpret and reflect on their thinking via interactions with a competent partner who improves their cognitive knowledge construction (Vygotsky, 1978) and social communication skills (Kirschner et al., 2015; Kreijns et al., 2007). In the past two decades, the computer-supported collaborative learning (CSCL) approach has evolved to offer students opportunities for co-construction of a shared understanding of knowledge and meaning-making of the content with the use of educational and technological tools (Stahl et al., 2014; Al-Emran et al., 2018; Stahl, 2006). Many studies on CSCL have reported that collaborative learning improves students’ learning (Hmelo-Silver, 2004; Jeong et al., 2019; Leeuwen et al., 2019; Vogel et al., 2017). However, learning in groups is not always associated with better learning compared to individual learning (Clinton & Kohlmeyer, 2005; Morgan & Tindale, 2002; Shibley & Zimmaro, 2002). Mullins et al. (2011) suggested that students’ performance of conceptual tasks is better when working collaboratively than individually, but the opposite is true with procedural tasks. Olsen et al. (2016) found no significant difference between students learning individually or collaboratively, regardless of the conceptual or procedural task. Others suggested that students learn more effectively when they combine cooperative and individual learning (Olsen et al., 2017; Celepkolu et al., 2017). According to Kirschner et al. (2011), collaborative learning may be more beneficial than individual learning when the complexity of the learning material is high (Kirschner et al., 2011). Interestingly, individuals perform better than groups in some highly complex tasks (Retnowati et al., 2016). Students engaging in collaborative learning can be more successful when they realize that they experience less cognitive effort and greater positive interdependence during the learning process than when studying individually (Janssen & Kirschner, 2020; Kirschner et al., 2018; Roseth et al., 2008). Other studies have noted that an idea-sharing group may not always perform better than a collection of non-interacting individuals, both in terms of the quantity and quality of unique ideas (Diehl & Stroebe, 1987; Hill, 1982). A wide range of explanations for process losses have been proposed and empirically tested, including social pressure, social loafing, and production blocking resulting from turn-taking conventions (Connolly, 1993; Diehl & Stroebe, 1987; Kraut, 2003). The idea failure associated with these process losses may be related to cognitive interference during memory retrieval, activation of memory, and idea generation (Wang & Rose, 2007). When brainstorming in groups, peers’ ideas may become sources of cognitive interference leading to process losses (Wang et al., 2011). Overall, the evidence suggesting that collaborative learning is superior to individual learning is mixed. There is also the need for further investigation of why one works better than the other.

Individual Problem-Solving (IPS) Versus Collaborative Problem-Solving (CPS)

Previous research on IPS examines how individuals reproduce their accumulated knowledge and acquire new knowledge while recognizing problem situations, formulating and assessing potential solutions, implementing solutions, and finally, gathering evidence and then evaluating it. The focus on CPS emphasizes how students work in groups to collaboratively analyze a complex problem, develop potential solutions, make decisions about how to achieve the goal, and evaluate the results afterward. In responding to the ever-increasing demand for collaboration skills in modern workplaces and the need for people to cope with different and complicated problems in everyday life (Deming, 2015), CPS has become popular in science education and is considered an effective teaching and learning strategy that facilitates students’ problem-solving ability. CPS involves cognitive and social dimensions during the problem-solving process. One CPS study reported that students believed peer feedback would improve their thinking and team performance (Herro et al., 2021), and the other one suggested that positive group emotions would affect group performance (Dindar et al., 2020). Some studies have reported that CPS can improve students’ science knowledge, processing skills, problem-solving competency, self-confidence, and collaborative learning (Alfin et al., 2019; Chang et al., 2017; Malik et al., 2019; Nordin & Osman, 2018; Prahani et al., 2018). On the other hand, many studies have shown that IPS can effectively enhance individual students’ problem-solving abilities and knowledge development (Amin et al., 2019; Cheng et al., 2018; Yu et al., 2010; She et al., 2012; Wu et al., 2022).

Despite numerous studies examining the effectiveness of CPS or IPS separately, very few have compared the two approaches. A few studies have compared CPS and IPS in mathematics to determine whether they improve students' mathematical problem-solving equally well. For example, Diziol et al. (2007) reported that collaboration yielded improved learning outcomes in post-concept performance compared to individual learning; however, they did not show such differences in procedural performance. Laughlin and colleagues identified that groups performed better than individuals in demonstrable mathematics problem-solving tasks (Laughlin et al., 2003, 2006, 2008). The only one study to compare individual and group scientific problem-solving found that students who learned individually performed better than those who learned in gr when studying biology worked examples; however, the latter performed better than the former when solving biology problems (Kirschner et al., 2011). Three major reasons explain the lack of studies comparing online scientific CPS to IPS. Unlike solving mathematical problems, solving science problems involves implementing PS solutions by running experiments to determine whether their PS solutions are feasible and whether the results match their expectations. The development of a problem-solving program would be more challenging in an online environment where computer simulations are used to implement PS solutions. All these factors make developing online scientific IPS difficult, not to mention developing an online environment for scientific CPS where team members can communicate with each other and collaborate to solve problems. It is therefore difficult to find comparative studies of online scientific IPSs and CPSs, so the need for conducting such study is evident. In light of this, we sought to investigate whether online scientific CPS improves students' conceptual understanding and problem-solving better than online scientific IPS, and furthermore, the mechanisms underlying any variation.

Moreover, no study has been conducted to find out whether low- versus high-achievers would benefit more from CPS than from IPS. Lou et al. (1996) suggested that low-ability students’ learning was significantly improved in heterogeneous ability groups, whereas high-ability students were not affected by group ability composition. Students with complete prior knowledge tend to perform better when they are learning individually than learning collaboratively (Retnowati et al., 2018). Some studies found that expert problem-solvers are highly dependent on domain-specific knowledge and strategies that are specific to the type of problem, an approach that makes them more efficient (Bulu & Pedersen, 2010; Funke & Frensch, 2007; Mayer, 1992). Other studies reported that problem solving practice helps students’ solving problems in scientific domains, such as physics and chemistry, that require scientific conceptual knowledge (Cheng et al., 2018; Hambrick & Engle, 2003; Lucangeli et al., 1998; She et al., 2012). Sears and Reagin (2013) reported that accelerated mathematics students solving mathematics problems in groups performed significantly worse than their peers solving problems individually; in contrast, traditional mathematics students solving problems in groups performed significantly better than their peers working individually. These studies suggests that low-achievers may have more difficulty solving scientific problems independently than high-achievers. When tackling scientific problems, it remains unclear whether high-achievers or low-achievers benefit more from learning individually or collaboratively. Thus, this study intends to examine whether low-ability students may benefit more from CPS than IPS, whereas high-ability students may benefit equally from both or more from IPS.

To date, CPS and IPS have mainly been examined for their effectiveness over a short period, and little attention has been paid to how they affect learning over an extended period of time, such as over an entire unit. Therefore, we designed a complete unit devoted to solving physics problems to examine whether CPS and IPS have the same potential for facilitating students' acquisition of physics concepts, improving their ability to solve physics problems, and minimizing the achievement gaps between high- and low-achievers. This study had five research questions. The first question was whether CPS and IPS were equally effective in improving students' performance on electromagnetism concept tests (ECT) and electromagnetism problem-solving tests (EPST), regardless of whether the students were high or low-achievers. The second question centers on whether online problem-solving classes could narrow the gap between high- and low-achievers in terms of physics concepts and problem-solving performance, regardless of whether they work collaboratively. The third question was to determine whether the effectiveness of online performance, actions, and time allocated for proposing problems-solving solutions (PS solutions), simulations, data recording, and evidence-based reasoning (EB reasoning) across 11 topics differed between CPS and IPS. In the fourth question, we investigated if the CPS group's discussion dialogues differed between high- and low-achievers, including providing support, requesting support, and reminding partner. Lastly, whether students' online problem-solving behavior and performance are correlated with their performance on ECT and EPST.

Methods

Subjects and procedures

A total of 109 eighth grade students were recruited from four classes to participate in the online electromagnetism problem-solving program for five class periods (each of 45 min’ duration) of a physical science course. A consent form was signed both by the student and by the parent prior to participating in this program. Students in two of the classes were randomly assigned to the CPS group (N = 56) and the remaining two classes were assigned to the IPS group (N = 53). Students were classified into high and low achievers based on their school science achievement scores. Students with school science achievement scores higher than 70 were classified as high-achievers (about 50% of students achieved high scores), while the rest were classified as low-achievers. Several studies have shown that groups involving students of different abilities and genders achieve the best results (Johnson & Johnson, 1994; Scheuerell, 2010). Students perform better in collaboration when their knowledge levels are heterogeneous (Janssen & Kirschner, 2020). Therefore, we grouped students into heterogenous pairs in the CPS group according to their respective school science achievement level. All students took the electromagnetic problem-solving test EPST and electromagnetic concept test (ECT) before and after learning with the online electromagnetism problem-solving program, regardless of whether they were in the CPS or IPS group. Furthermore, the students’ online problem-solving processes and behaviors, and CPS group discussion dialogues were collected for further analysis.

Development of an online problem-solving program for learning electromagnetism

Two versions of the online problem-solving program were developed: IPS and CPS. They were created using Unity 3D technologies to develop the simulation and experiments, the photon network to build multi-person collaboration, and a MySQL database to collect data. The electromagnetism problem-solving unit was developed based on the standards for the 8th grade physical science curriculum in Taiwan and consisted of six magnetism topics and five electromagnetism induction topics. A panel composed of a science education professor, a middle school science teacher, and a middle school science pre-service teacher was involved in the development of the electromagnetism problem-solving unit. The six magnetism topics were as follows: (1) magnets and magnetic force; (2) magnets and magnetic poles; (3) magnet segmentation and magnetic poles; (4) magnet combination, direction, and magnetic force; (5) magnetization; and (6) magnetic line of force and magnetic field. The five electromagnetism induction topics were as follows: (1) long straight wire and magnetic force; (2) long straight wire and magnetic field direction; (3) long straight wire and direction of current flow; (4) U-shaped wire and direction of current and magnetic field; and (5) solenoid coil and magnetic effect of current. The design of online versions of CPS and IPS shared the same simulation system and electromagnetism problem-solving content, including proposing PS solutions, simulation, and data recording, EB reasoning, and feedback with the correct answer (Fig. 1). Each problem-solving task prompted students to generate possible solutions to the problem. Students were allowed to implement their PS solutions by running the simulation through manipulation of the equipment or materials (magnet, compass, coil number, battery number and direction, wire thickness, rod types, etc.). Students were required to make a data recording after implementing their PS solutions by running 3D simulation. For example, “Please find at least two solutions to create the strongest magnetic field around a long straight wire.” Or “Please find at least two solutions to make the strongest magnetic field of the helical coil.” Finally, the EB reasoning questions were provided to check whether students were able to transfer their learning. For example, is it possible to only have one of the N or S poles on a magnet? And why? Why can some materials be magnetized by magnets while others cannot? How can you determine the direction of the magnetic field if the current is flowing forward according to Ampere's right-hand rule?

Fig. 1
figure 1

The procedure and design of collaborative problem-solving and individual problem-solving

Full size image

Online collaborative problem-solving

CPS allowed students to work collaboratively with their partners throughout the periods of proposing a PS solution, simulation, data recording, EB reasoning, and system-generated feedback with correct answers. The screen showing manipulation of equipment and the running of simulations and information exchange in the discussion board were shared with partners during collaboration (Fig. 2). In the same CPS group, students had access to the same discussion board and simulation screens. Prior to collaboration, students in the CPS group needed to come up with their own PS solutions. Following the submission of their proposed PS solution, they were allowed to collaborate with their partners and revise their PS solutions. The hardest part of problem solving is generating PS solutions, which students often find challenging if they work independently. Having the opportunity to experience their progress in generating PS solutions after collaboration is crucial for students. As part of the system, students had to take turns running simulations and manipulating equipment. During experiments, they can run simulations, manipulate equipment, and observe their partner's simulations and manipulations. Throughout their online problem-solving processes, the collaborative student pairs could exchange ideas and solutions on the discussion board. The system synchronized all the actions, including students’ manipulation of equipment, running of the simulation, and dialogues.

Fig. 2
figure 2

Images from an online individual problem-solving system

Full size image

Individualized problem-solving

IPS shared the same electromagnetism problem-solving platform and content as CPS, which included proposing PS solutions, stimulation, data recording, EB reasoning, and system-generated feedback with correct answers (Fig. 3). The major difference was that IPS was designed for each student to work individually, while CPS was designed for students to work in pairs. The note room was designed for IPS students to make notes on their problem-solving (Fig. 3, bottom left panel).

Fig. 3
figure 3

Images from an online collaborative problem-solving system

Full size image

Electromagnetism Concept Test (ECT)

The ECT was a two-tier multiple-choice diagnostic instrument designed to measure students’ comprehension of electromagnetism concepts before and after engaging in the online physics problem-solving program. A panel of three experts, including a professor of science education, a middle school science teacher, and a pre-service middle school teacher, developed the items to ensure that they were accurately constructed and relevant to the online electromagnetism problem-solving program. Eighteen questions were developed, and one point was awarded for each correct answer. Among these items, ten of them cover magnetism topics and eight of them cover electromagnetism topics. Cronbach’ α for ECT was 0.760, indicating that the electromagnetism concept assessment had satisfactory reliability in the statistical testing.

Electromagnetism Problem-Solving Test (EPST)

The EPST was an open-ended instrument that included six scenarios covering 11 topics in electromagnetism. It was developed by the same panel of three experts described above to measure students’ problem-solving performance in electromagnetism (Appendix shows an example question). Each scenario consisted of two to three questions to measure students’ physics problem-solving competency in identifying major influential factors and proposing two potential solutions. A coding system was developed to assign students' responses—one point for each correct response and zero points for each incorrect response—leading to a maximum score of 26 points. Two raters were involved in the coding of students’ EPST results based on the coding system, and the inter-rater reliability was 0.97.

Analyses of the online problem-solving process

Each online problem-solving task included four stages: proposing the PS solution, simulation, data recording, and evidence-based (EB) reasoning. There were 11 problem-solving topics, including six on magnetism and five on electromagnetism. During the online problem-solving process, students need to propose PS solutions, implement their PS solution, run a simulation, record data, and provide evidence-based reasoning. These online problem-solving processes and time allocated for these processes and behaviors were all captured by the system. The CPS group’s interactions and discussions between peers were also captured by the system. Two coding systems were developed for PS solution and EB reasoning to classify students’ responses into three levels: level 2 for a correct response; level 1 for a partially correct response; and level 0 for an incorrect response. The coding system for students’ data recording gave one point for entering data fully correctly and half a point for entering data partially correctly. The inter-rater reliability of these three rubrics for PS solution, data recording, and EB reasoning were 0.987, 0.985, and 0.979, respectively. Students' online discussion in the CPS group were classified into three categories: giving support, requesting support, and reminding in accordance with literature (Borup et al., 2020; She, 2001). Hence, the coding system was developed and used to analyze the online discussion of the CPS group, which included providing support, requesting support, and reminding partners. The inter-rater reliability for this coding system was 0.971. A two-factor MANOVA was performed to examine whether the type of problem-solving and achievement level had significant effects on students’ online performance, including PS solutions, EB reasoning, and data recording across the 11 topics. The stepwise regression was used to investigate the relationships between students' performance in ECT and EPST and their online problem-solving performance and behavior.

Results

Electromagnetism Concept Test (ECT)

To answer our first and second research questions, the t-test and two-factor ANCOVA were used to examine the impact of CPS and IPS on students’ performance in the ECT, regardless of whether they were high or low achievers. In addition, we also examined whether taking online physics problem-solving classes, irrespective of whether the problem-solving was collaborative, could narrow the gap between high and low achievers’ ECT scores. The ECT results indicated that both CPS (t = 3.70, p < 0.01) and IPS (t = 4.78, p < 0.001) groups made significant progress from the pretest to the posttest stage. The high-achieving students in both groups also made significant progress from the pretest to the posttest (t = 3.35, p < 0.01; t = 4.70, p < 0.001), however low-achieving students in the CPS group made significant progress (t = 2.17, p < 0.05) while those in the IPS group did not (t = 1.77, p = 0.09).

A two-factor ANCOVA was performed to examine the effects of the type of problem-solving (CPS vs. IPS) and achievement level on students' scores for the posttest ECT, with the pretest scores used as the covariate (Table 1). The results indicated that the type of problem-solving did not have a statistically significant effect on the students’ posttest performance in the ECT (F = 0.10, p = 0.753), while the level of achievement did lead to a statistically significant effect on the students’ posttest performance in the ECT (F = 19.38, p < 0.001, partial η2 = 0.157, effect size = large). It indicated that the high-achievers significantly outperformed the low-achievers in both IPS and CPS groups (F = 11.75, p < 0.01; F = 11.75, p < 0.01).

Table 1 Results of Two-way ANCOVA analysis of electromagnetism concept test (ECT) and electromagnetism problem-solving test (EPST)
Full size table

Electromagnetism Problem-Solving Test (EPST)

To answer our first and second research questions, the t-test and two-factor ANCOVA were used to examine the impact of CPS and IPS on students’ performance in the EPST, regardless of whether they were high or low achievers. The EPST pretest scores for CPS and IPS are 3.18 and 3.35, respectively. Levene's test of homogeneity did not yield significant results for EPST pretest scores (F = 0.78, p = 0.38). The EPST results indicated that both CPS (t = 13.32, p < 0.001) and IPS (t = 11.20, p < 0.001) groups all made significant progress from the pretest to the posttest. In the CPS and IPS groups, both high-achieving students (Mcps = 9.88 vs. Mips = 9.36; t = 9.48, p < 0.001; t = 8.03, p < 0.001) and low-achieving students (Mcps = 9.04 vs. Mips = 6.80; t = 9.18, p < 0.001; t = 7.68, p < 0.001) made significant progress from the pretest to the posttest.

A two-factor ANCOVA was performed to examine the effects of the type of problem-solving (CPS vs. IPS) and achievement level on students' scores on the posttest EPST, with the pretest scores used as the covariate (Table 1). The results indicate that students' EPST posttest performance was significantly affected by the type of problem-solving (F = 5.55, p < 0.05, partial η2 = 0.51, effect size = small), however, achievement level did not have a significant effect on students’ posttest performance on the EPST (F = 0.77, p = 0.383). We further performed a one-factor ANCOVA test to compare the low-achievers in the CPS and IPS groups (Table 2); the results showed that the low-achievers in the CPS group significantly outperformed the low-achievers in the IPS group (F = 4.13, p < 0.05, partial η2 = 0.51, effect size = medium) on the EPST, while high-achievers in the two groups performed the same (F = 1.48, p = 0.229).

Table 2 One factor ANCOVA analysis of electromagnetism concept test (ECT) and electromagnetism problem solving test (EPST) by comparing CPS and IPS within low achievers and within high achievers separately
Full size table

The posttest EPST scores of low-achievers and high-achievers in the CPS group were about the same (Low vs. High = 9.04 vs. 9.88), however, the difference between low-achievers and high-achievers in the IPS group remained substantial (Low vs. High = 6.8 vs. 9.36). As shown in Fig. 4, the scatter plot of marginal histograms compares high- and low-accomplished students in the CPS and IPS groups, showing that CPS was more effective than IPS in reducing the discrepancy in EPST scores between high- and low-achievers.

Fig. 4
figure 4

Scatter plot with marginal histograms for CPS high vs. low achievers (left panel) and IPS high vs. low achievers (right panel)

Full size image

Online problem-solving process

To answer the third research question, MANOVA and a radar map were used to investigate the effectiveness of online performance in proposing a PS solution, simulation, data recording, and EB reasoning in CPS and IPS across 11 topics and the actions and time allocated to the PS solution, simulation, data recording, and EB reasoning.

Online problem-solving solution, evidence-based reasoning, and data recording

The online electromagnetism problem-solving processes, including the six topics in magnetism and the five topics in electromagnetism, were analyzed. Figure 5 summarizes the amount of PS solutions proposed by the CPS and IPS groups across 11 topics. It shows that the CPS group proposing more PS solutions than IPS group across the 11 topics, and the difference was statistically significant for four topics.

Fig. 5
figure 5

CPS and IPS groups’ students’ online PS solutions performance across 11 topics with two units

Full size image

A two-factor MANOVA was performed to examine whether the type of problem-solving and achievement level had significant effects on students’ online performance, including PS solutions, EB reasoning, and data recording across the 11 topics. It indicated that both the type of problem-solving (Wilk’s Λ = 0.82, F = 7.58, p < 0.001) and achievement level (Wilk’s Λ = 0.74, F = 12.03, p < 0.001) had a significant effect, although the interaction was not significant. Type of problem-solving as the main effect indicated that the CPS group significantly outperformed the IPS group in the performance of their PS solution performance (F = 14.42, p < 0.001), regardless of whether they were high or low-achievers (Table 3). Achievement level as the main effect indicated that high-achievers outperformed low-achievers in their PS solution, EB reasoning, and data recording performance (F = 24.28, p < 0.001; F = 29.28, p < 0.001; and F = 5.91, p < 0.005, respectively), regardless of whether they were in the CPS or IPS group (Table 4).

Table 3 The main effect of IPS and CPS groups’ performance and time spending in online problem-solving process
Full size table
Table 4 The main effect of high- and low-achievers’ performance and time spending in online problem-solving process
Full size table

Time spent on PS solution, EB reasoning, and simulation and data recording

A two-factor MANOVA was applied to investigate whether type of problem-solving and achievement level had an effect on the time spent on PS solutions, simulation, data recording, and EB reasoning. The results indicated that type of problem-solving (Wilk’s Λ = 0.50, F = 53.13, p < 0.001) had a significant effect on the time spent on online problem-solving. Type of problem-solving as the main effect indicated that the CPS group allocated significantly more time to PS solutions, EB reasoning, and simulation and data recording than the IPS group (F = 65.05, p < 0.001; F = 119.66, p < 0.001; F = 32.12, p < 0.001), regardless of whether the students were high or low-achievers (Table 4). Level of achievement as the main effect indicated that high-achievers spent about the same amount of time on their PS solutions, EB reasoning, and simulation and data recording as did low-achievers (F = 0.03, p = 0.871; F = 2.82, p = 0.096; and F = 0.43, p = 0.513, respectively).

As part of the second research question, we further compared the CPS and IPS groups for low-achievers’ online problem-solving performance and time spent on online problem-solving process. Our results show that the low-achievers in the CPS group performed significantly better in their PS solutions (F = 4.32, p < 0.05) and allocated significantly more time to the PS solutions, EB reasoning, and simulation and data recording than those in the IPS group (F = 33.32, p < 0.001; F = 61.46, p < 0.001; and F = 35.57, p < 0.001, respectively). Similar patterns were also observed for high-achievers, with those in the CPS group performing significantly better on PS solutions (F = 9.34, p < 0.005) and allocating significantly more time to PS solutions, EB reasoning, and simulation and data recording than those in the IPS group (F = 32.58, p < 0.001; F = 61.43, p < 0.001; and F = 4.74, p < 0.05, respectively).

In Fig. 6, three radar maps are displayed for all students, high achievers, and low achievers. Each radar map includes pretests and posttests for ECT performance, EPST performance, and the online learning process of PS solutions, EB reasoning, data recording, and time devoted to PS solutions, EB reasoning and simulation and data recording (SIM &DR). Figure 6A shows that CPS group had better performance in the EPST posttest, PS solutions, and EB reasoning than the IPS group; and the CPS group allocated a longer time to PS solutions and EB reasoning compared to the IPS group. Figure 6B displays that the high-achievers in the CPS group had higher performance in the EPST posttest, PS solutions, and EB reasoning than IPS group; and CPS group high-achievers allocated a longer time to PS solutions, EB reasoning, and simulation and data recording compared to the IPS high-achievers. Figure 6C shows that low-achievers in the CPS group had higher ECT and EPST posttest scores than IPS low-achievers; and low-achievers in the CPS group allocated more time to PS solution, EB reasoning, and simulation and data recording than the low-achievers in the IPS group.

Fig. 6
figure 6

Radar map for IPS and CPS across variables. ECT Pretest: Electromagnetism Concept Test Pretest; ECT Posttest: Electromagnetism Concept Test Posttest; EPST Pretest: Electromagnetism Problem Solving Test Pretest; EPST Posttest: Electromagnetism Problem Solving Test Posttest;PS Solution: Problem Solving Solution; EB Reasoning: Evidence Based Reasoning; PS Solution Time: Problem Solving Solution Time; EB Reasoning Time: Evidence Based Reasoning Time

Full size image

Analysis of online discussion dialogues of low- and high-achievers in the CPS group

To answer the fourth research question, we examined the discussion contributions between high- and low-achievers in the CPS group regarding providing support, requesting support, and reminding partners. Based on the mean number of online discussion turns, high achievers provide more turns labeled providing support (1.72 vs 0.96, F = 2.66, p = 0.11) and reminding (1.44 vs 1.09, F = 1.22, p = 0.27) to their partners than low-achievers, however, no significant difference was found. Compared to high-achievers, low-achievers requested more support from their partners (0.84 vs. 0.58, F = 1.96, p = 0.17). Based on these results, high achievers provide more support and reminders than low achievers, while low achievers request more support, though the difference was not significant.

Results of stepwise regression

To answer our final research question, stepwise regression was used to investigate the relationships between students’ performance in the ECT and EPST and their online problem-solving performance and behavior. We specifically investigated whether the online PS solutions, EB reasoning, and data recording were able to predict students’ ECT and EPST performance. Table 5 presents the multiple regression model predicting students’ ECT and EPST scores based on online PS solutions, EB reasoning, and data recording performance. The analysis indicates that the online PS solutions was the only factor that predicted students’ ECT posttest score (β = 0.45, p < 0.001), accounting for 20% of the variance (F = 27.38, p < 0.001) and EPST posttest score (β = 0.27, p < 0.01), accounting for 7% of the variance (F = 8.59, p < 0.001). These results new knowledge related to our fifth research question asking whether students' online PS solution performance can best predict their ECT and EPST posttest performance.

Table 5 Stepwise Regression analysis for variables predicting electromagnetism concept test (ECT) and electromagnetism problem-solving test (EPST)
Full size table

Discussion

Our study is the first to design, implement, and examine whether CPS is more effective than IPS in enhancing 8th graders’ physics problem-solving performance after completing five classes of an entire unit on electromagnetism problem-solving. We demonstrated several significant findings that promote students’ problem-solving. First, CPS is superior to IPS in significantly promoting students’ physics problem-solving performance, while both CPS and IPS are able to elevate students’ physics concept performance to the same extent. Second, both CPS and IPS successfully promote high-achievers' problem-solving performance to the same extent. However, CPS was more effective than IPS in maximizing low-achievers' problem-solving performance and minimizing the discrepancy between high- and low-achievers. Third, the CPS group showed significantly higher PS solution performance and allocated more time to PS solutions, EB reasoning, and simulation and data recording than the IPS group during the online problem-solving process. Fourth, the analysis of types of CPS group’s discussion turns offered evidence that low-achievers requested and received more support from high-achievers; thus, the gaps between high- and low-achievers was minimized. Lastly, our study confirms that the performance of students' online PS solutions is the best predictor of their ECT and EPST posttest performance.

In this study, all students who completed five online physics problem-solving classes improved significantly in their performance in the ECT and EPST, except for the low-achievers in the IPS group. Our findings offer empirical evidence to confirm the notion that problem-solving competency can be developed through well-designed instruction that fosters a deeper understanding of knowledge and prepares students to apply their knowledge to novel situations (Cheng et al., 2018; Heller & Hollabaugh, 1992; OECD, 2003; She et al., 2012;), and resolve the difficulty of learning physics that comes due to its abstract nature and subsuming more underlying concepts (Brown, 1993; Osborne & Freyberg, 1985; Yildirim et al., 2021). In addition, other studies have already suggested that the use of simulation-based problem-solving instruction can fill knowledge gaps through observation, exploration, and interaction with unknown systems (Andrews-Todd & Forsyth, 2020; Ceberio et al., 2016; Sinensis et al., 2019) and help students to obtain correct solutions and develop a positive attitude towards learning about problem-solving. These results lead to the conclusion that the well-designed online electromagnetism problem-solving with simulation experience is able to empower students to successfully construct the abstract and hierarchical concepts of electromagnetism and develop electromagnetism-specific problem-solving competency.

Whether and why CPS or IPS is more effective in solving physics problems is less well researched than it should be. We found that the CPS group significantly outperformed the IPS group in the EPST, a finding in line with the results of a previous study that showed that students who had learned in groups performed better than those that learned individually when solving biology problems (Kirschner et al., 2011; Laughlin et al., 2008; Mullins et al., 2011). Our findings of students’ online problem-solving processes indicated that the CPS group showed higher PS solution performance and spent more time on their PS solutions, EB reasoning, and simulation and data recording. In addition, CPS students’ PS solutions outperformed that of the IPS students in 10 of the 11 topics of electromagnetism, with the PS solutions being significantly better in four topics. Problem-solvers often switch between reasoning skills in gathering evidence and assessing potential solution paths before settling into preferring one technique over another when they seek the answer to a given problem (e.g., Adey et al., 2007; Klauer & Phye, 2008). Peer interaction, feedback, and discussions between team members improve task performance during CPS activities (Dindar et al., 2020; Herro et al., 2021; Wagy & Bongard, 2015). Harskamp and Ding (2007) found that students' understanding of the problem deepens when they spend time on analyzing the problem by discussing it with group members. Altogether, these findings lead us to propose that the students in the CPS group allocated more time to interacting and discussing with their partners to analyze the problem, assessing potential solutions, working in simulations, reasoning in gathering evidence, and finding the answer to a given problem, which resulted in better performance than the IPS group.

Interestingly, our results indicated that the low-achievers in the CPS group significantly outperformed their counterparts in the IPS group in the EPST, while the high-achievers in the CPS group performed the same as those in the IPS group in the EPST. This is in line with the radar map discussed earlier in this paper and the findings from students’ online problem-solving processes that the low-achievers in the CPS group performed better in their online PS solutions and allocated more time to the PS solutions, EB reasoning, and simulation and data recording than the low-achievers in the IPS group. Students who have complete prior knowledge tend to perform better when learning individually than when learning collaboratively (Retnowati et al., 2018). It may explain why CPS did not outperform IPS when it came to high achievers in the current study. An interesting pattern emerged from the EPST scatter plot of high- and low-achievers in the CPS and IPS groups, which revealed that the difference in the distribution of posttest EPST scores between the low- and high-achievers in the CPS group had reduced; however, the difference between the high- and low-achievers in the IPS group remained substantial. During the CPS dialogues between high- and low-achievers, high-achievers provided more support and reminded their partners when needed, while low-achievers requested more support. Research shows that collaboration leads to greater learning success than studying alone when students experience less cognitive effort and positive interdependence during the learning process (Janssen & Kirschner, 2020; Kirschner et al., 2018; Roseth et al., 2008). This past work leads us to suggest that low-achievers in CPS groups benefit most from requesting and receiving support from their partners, thereby reducing cognitive effort, and enabling them to successfully move from their existing zone to their potential zone of development. Consequently, we conclude that CPS enhances low-achievers' problem-solving performance better than IPS because it allows students to spend more time exchanging ideas with partners, thus optimizing the performance of low-achievers in problem-solving and bridging the gap between high- and low-achievers more effectively than IPS.

The regression results indicate that students’ online PS solution is the only factor that predicts their posttest performance in the ECT and EPST. They confirm that the performance of the online PS solution contributed to the improvement in their electromagnetism concept learning and problem-solving competency. These findings verified that learning focusing on problem-solving can better develop students’ competency in problem solving and content knowledge (Cheng et al., 2018; Harskamp & Ding, 2007; Heller & Hollabaugh, 1992; Malik et al., 2019; She et al., 2012). Accordingly, the findings of this study may shed light on the future application of CPS and IPS in terms of how and when each of these are able to facilitate physics learning and problem-solving. These findings imply that both IPS and CPS can benefit high-achievers’ physics conceptual understanding and problem-solving to the same extent, whereas CPS is more effective and efficient than IPS at maximizing low-achievers’ performance and minimizing the gaps in attainment between high- and low-achievers. It is highly recommended that future research on CPS practice incorporate heterogeneous groups as a means of facilitating dialogue between high-achieving and low-achieving individuals. Having less cognitive load and support from high-achievers will enhance the ability to move low achievers from their existing zone to their potential zone of development. Additionally, finding PS solutions is the most challenging part of problem solving, which students often find difficult if they work alone. Therefore, we recommend that students in CPS groups be strongly encouraged to develop their own PS solutions prior to collaboration, and revise them afterwards. Having the opportunity to learn collaboratively would motivate students to continue to learn collaboratively, which would lead to problem-solving success. The present study has a limitation of smaller group sizes because of the division of the student population into two factors. Further studies with larger samples would be beneficial to confirm the current findings.

Appendix: An example question of EPST

In 19th-century Denmark, Hans Christian Ørsted accidentally discovered that a magnetic needle deflects around a wire during an experiment. The following experiment equipment are provided for solving the problem.

  • 1 thin copper wire

  • 1 think iron wire

  • 1 thick copper wire

  • 1 thick iron wire

  • 1 magnetic needle

  • 3 pcs Battery

  • 1 box Paperclips

  • (1) What are the factors that may affect the deflection direction and angle of the magnetic needle?

  • (2) Please provide at least two solutions that prove how the factors mentioned above affect the deflection direction of a magnetic needle?

  • (3) Please provide at least two solutions that prove how the factors mentioned above affect the degree of deflection angle of a magnetic needle?