The Power of Tentative Truth: The Impacts of Enhanced Science Instruction on Student Paranormal Beliefs

Abstract

This research explores the impact of epistemic-focused science instruction on college students’ paranormal beliefs and conceptual physics understanding. Despite lacking a scientific foundation, paranormal beliefs are common. Grounded on previous studies, a theoretical model was conceived to tackle this challenge. The model indicates that these beliefs, much like common science alternative ideas and misconceptions, are likely derived from inherent biases in intuitive thinking. Accordingly, an intervention was designed and put into practices in three consecutive semesters. The intervention incorporated epistemic and ontological training. It challenged students’ intuitive idea formation and confirmation and encouraged model-based hypothesis formation backed by empirical evidence. A three-level, mixed-methods study tested the effectiveness of the intervention. Quantitative data at the whole-class level displayed a reduction in paranormal beliefs with a small effect size. Concurrently, a large effect size was observed in enhancing students’ conceptual physics understanding. Moving to the subgroup level, a k-means clustering analysis revealed distinct student clusters characterized by different shifts in paranormal beliefs and conceptual physics learning, indicating differential responses to the intervention. At the individual layer of analysis, qualitative data underscored instances where students creatively misconstrued scientific concepts to reinforce their paranormal beliefs, highlighting the situated and contextual nature of epistemic practices. This work reinforces the critical role of science as a way of knowing for transforming student epistemic practices. It highlights the transition from forming definitive truth based on intuitive idea formation and confirmation, toward model-based hypothesis formation, backed by empirical evidence, to construct a tentative truth until the better one emerges.

1 Introduction

The Scientific Revolution, highlighted by the transformative contributions of Kepler, Galileo, and Newton, initiated a significant paradigm shift in understanding the natural world. Their novel approaches to observation, experimentation, and mathematical reasoning represented a pivotal departure from traditional views and laid the groundwork for modern science (Shapin, 1998). This shift in thinking not only drove the Scientific Revolution forward but also profoundly influenced the Enlightenment era. Enlightenment intellectuals, inspired by these scientific advancements, further developed and integrated these principles into their own thought (Porter, 1990), recognizing science as a way of knowing. This historical evolution, profoundly shaped by the Scientific Revolution and further refined during the Enlightenment, underscores its critical role in our contemporary, science-intensive society (Pinker, 2018). In today’s world, a science-intensive society refers to a setting that heavily relies on scientific knowledge and technological advancements to shape daily life. Despite science playing a pivotal role, it is contrasted with persistent unscientific beliefs, highlighting the ongoing challenge of integrating scientific thinking across all societal domains. This contrast underscores the importance of science education in fostering a critical and empirical approach to understanding the world, where both scientific and paranormal beliefs (PBs) influence perceptions and educational outcomes.

Despite our science-intensive society where science should be the mainstream way of knowing, PBs, a vestige of old and non-scientific thinking, still persist and even flourish in our modern context. Science education has been promoting scientific thinking; however, the prevalence of these beliefs, conspiracy theories, and pseudoscience indicates they are still embedded in our society. As Pinker (2018) suggests in “Enlightenment Now,” confronting contemporary counter-enlightenment ideals such as the supernatural entities requires a revival of Enlightenment thinking. Paranormal beliefs, although their definition varies (Wilson, 2018), broadly refer to accepting phenomena that are not supported by empirical evidence, often violating the natural laws, and lacking conventional explanation. These vestiges of old modes of knowing remain prevalent in our society. For instance, a 2005 Gallup survey reported that 73% of Americans believed in at least one of ten examined paranormal beliefs (Moore, 2005), and a 2018 US survey found that 56% of respondents believed in haunted places and 26% believed in telekinesis, “some people can move object by their mind” (Chapman University, 2018). PBs notably impact student learning in science by promoting alternative, non-empirical explanations that can obstruct scientific understanding. Addressing PBs within science education is crucial for fostering a scientific mindset. This process not only challenges unscientific thinking but also promotes the adoption of scientific methodologies among students. By confronting these beliefs directly in educational settings, students would have more opportunities to critically evaluate and integrate scientific principles in their learning.

This paper examines how an epistemic-focused curriculum can mitigate these beliefs in the context of a higher education physics course. Building on this historical background, this paper explores a critical contemporary challenge: the persistence and impact of paranormal beliefs, vestiges of pre-enlightenment thought, in our science-intensive society. I argue for a revival of Enlightenment thinking, which emphasizes rationality, skepticism, and empirical inquiry, as a vital approach to addressing and mitigating these beliefs. These hallmarks of the Enlightenment are today as essential as ever in confronting PBs, pseudoscience, and conspiracy theories within educational frameworks. This historical continuum highlights the enduring need to apply Enlightenment thinking to modern educational challenges, particularly in fostering scientific literacy and critical thinking in a world where unscientific beliefs persist. Studies suggest a prevalence of such beliefs (Moore, 2005; Chapman University, 2018), underscoring the urgency of this focus. This exploration sets the stage for discussing how a curriculum focused on science as a way of knowing can effectively combat unscientific beliefs and enhance scientific literacy among students in higher education science courses. The emergence of science epistemic practices (SEPs), which evolved from centuries of scientific thought, serves as effective modes of knowing, enabling people to make sense of and interpret everyday phenomena. This evolution from historical scientific thoughts to modern epistemic practices underscores the importance of addressing contemporary challenges in our society. In this light, a key educational objective is to explore how science instruction can empower students to utilize SEPs, transitioning from intuitive to scientific ways of knowing.

Given the multifaceted and situated nature of epistemic practices, which are heavily dependent on their specific settings (Elby et al., 2016; Lidar et al., 2009; Östman & Wickman, 2014), it is reasonable to expect a wide range of such practices when individuals engage in understanding everyday phenomena. These practices often vary significantly from SEPs, highlighting the diverse approaches to knowledge formation. This diversity underscores the vital role of science education in cultivating the “habit” of utilizing SEPs. As Hammer et al. (2005) insightfully noted, and as further discussed by Kelly and Licona (2018), developing “habits of mind to bring to bear on issues of inquiry” (p.159) is crucial for meaningful engagement in knowledge building and reasoning. Sharing this viewpoint, Lehrer and Schauble (2012) underscored the value of fostering these habits through educational scaffolding to enhance students’ grasp of SEPs.

Reflecting on the multifaceted nature of epistemic practices, let us examine a specific scenario to illustrate the varied student epistemic practices applied in everyday contexts. Consider a scenario where someone mixes water and alcohol and observes that the resulting solution has a smaller volume than the sum of the initial water and alcohol volumes. In this scenario, the individual’s choices for modes of knowing, influenced by the context and prior experiences, may range from conducting an internet search for explanations to intuitively drawing on an analogy. The person may search public sources such as blogs or YouTube videos or seeking expert advice by reading scientific websites or posing questions in forums where experts respond. The person may even intuitively attribute the volume decrease to spongy characteristics of molecules, reasoning that alcohol molecules reduce in size due to pressure from water molecules. At this stage, they might be satisfied with their explanation and consider it the definitive truth without seeking empirical evidence.

In contrast, scientists confronted with the same phenomenon, and presumably adopt SEPs, would systematically construct theory-driven or model-based hypotheses to explain the situation. They would then search for ways to empirically test these hypotheses. After these efforts, they arrive at a tentative truth or “the working hypothesis best fitted to open the way to the next better one” (Lorenz, 2002, p.279). The diversity of student ways of knowing and the gap between those ways and SPEs, in essence, highlights the imperative role of science education in establishing SEPs as habits of mind and common modes of knowing, rather than as exclusive practices utilized by scientists within specific contexts. These are modes of knowing that endorse a tentative truth, one that is constructed by model-based and theory-laden analogical ideas and supported by empirical evidence.

The relationship between PBs and epistemic practices extends far beyond just phenomena such as spiritual hauntings; it is a significant influence on epistemic practices. Rodríguez-Ferreiro and Barberia (2021) investigated the impact of varying degrees of PBs on evidence interpretation. Participants were tasked with determining the source jar, filled predominantly with either blue or red beads, by drawing beads from an unseen jar. The study indicated a negative correlation between the number of beads participants drew before deciding and their levels of pseudoscientific and paranormal beliefs. This trend aligns with the “jump-to-conclusions” bias (Irwin et al., 2014) and indicates that individuals with strong paranormal or pseudoscientific beliefs often adopt lower standards of evidence in decision-making. This trend aligns with the “jump-to-conclusions” bias (Irwin et al., 2014) and suggests that individuals with strong paranormal or pseudoscientific beliefs tend to apply lower evidential standards, potentially leading to misinterpretations such as seeing coincidences as causally related events or selectively gathering evidence. Additionally, Blanco et al. (2015) noted that paranormal believers are more vulnerable to developing causal illusions, interpreting coincidences as causal relationships, in the laboratory than nonbelievers due to their biased exposure to information. Furthermore, Metz et al. (2018) observed that people with paranormal beliefs often rely less on empirical methods and more on anecdotal or experiential evidence, leading to a diminished appreciation for empirical validation. This tendency is evident when comparing the justifications of beliefs between creationists, who emphasize intuition and “knowledge from the heart,” and evolutionists, who rely on empirical evidence.

Building on the observation that paranormal beliefs significantly influence epistemic practices, it is important to consider the reverse effect: how these practices might also impact paranormal beliefs. Epistemic practices that emphasize critical thinking, skepticism, and the scientific methods as they relate to PBs reduce the inclination to accept PBs. Banziger (1983) investigated the shift in paranormal beliefs among older learners participating in a short course with a focus on critical and skeptical inquiry about paranormal phenomena. The study methodology included pre-course, post-course, and six-month delayed testing to evaluate changes in PBs and the finding suggested a significant change in participants’ PBs. Similarly, Wilson (2018) studied changes in PBs when participants attended a science and critical thinking course that directly discussed paranormal phenomena. Overall, beliefs in paranormal subcategories of PBS lowered except for superstition. Moreover, enhanced scientific literacy and understanding can demystify paranormal phenomena, leading to a decreased tendency to adopt such beliefs. In their 2023 survey study, Torres et al. 2023 conducted a comprehensive analysis involving a representative sample of the US population. The study revealed that individuals with higher levels of scientific literacy and understanding are less likely to hold epistemically unwarranted beliefs, including those related to the paranormal. Additionally, epistemic practices that prioritize analytical over intuitive thinking affect how individuals interpret ambiguous situations, with analytically inclined individuals being less prone to paranormal explanations and more inclined toward logical, scientific reasoning. The study by Aarnio and Lindeman (2005) investigated the connections between paranormal beliefs and several factors, including educational level, discipline, length of education, gender, and, notably, analytical and intuitive thinking styles. Conducted with a large sample of Finnish students, the study found that university students generally held fewer paranormal beliefs compared to vocational school students. This difference was partially attributed to the stronger preference for analytical thinking among university students.

The relationship between PBs and epistemic practices is both bidirectional and complex, as illustrated in the discussion above. PBs can lead to skewed epistemic practices, characterized by selective evidence gathering, the creation of causal illusions, and a diminished reliance on empirical methods. Conversely, robust epistemic practices that emphasize critical thinking, skepticism, scientific literacy, and analytical thinking have the potential to counteract and reduce these beliefs. Given the significant influence of PBs on the scientific reasoning abilities of students, developing an epistemic-focused curriculum in our general physics course aims to enhance students’ understanding and application of scientific methodologies. This curriculum is designed to challenge and refine students’ epistemic practices, promoting a more empirical and rational approach to interpreting phenomena, which is essential in reducing the acceptance of PBs. Creating this specific intervention necessitates a theoretical model that ties together science as a way of knowing, student epistemic practices, paranormal beliefs, and science instruction. In the rest of this section, it is discussed how this theoretical model was developed drawing upon research from various academic fields, including cognitive and behavioral psychology, parapsychology, sociology, educational psychology, philosophy of science, learning sciences, and science education. This section will end by defining the educational issue, describing the intervention based on the model, outlining the methods for assessing the intervention, and posing the research question.

1.1 Theory of Intuitive Core Knowledge

According to the theory of intuitive core knowledge, by the time they reach kindergarten, children naturally develop basic intuitive knowledge in three areas: physical, biological, and psychological entities and processes (Spelke, 2000). For instance, children intuitively understand that physical entities have independent existence and volume. They also have a basic understanding that animate objects have minds and are intentional agents that can move, which falls under psychological knowledge. Furthermore, they learn intuitively that diseases can be contagious, which is an example of biological knowledge (Kalish, 1999). This theory emphasizes the importance of building on children’s intuitive knowledge to promote more effective learning which can have many implications in science education.

1.1.1 Theory of Intuitive Core Knowledge: Alternative Science Ideas and Paranormal Beliefs

Intuitive core knowledge is a double-edged sword in science learning. While it assists students in making sense of the world around them, it can also foster alternative ideas (i.e., ideas that differ from scientifically accepted explanations) and misconceptions (i.e., incorrect beliefs or understandings about a concept or topic that contradict scientific principles). According to Coley and Tanner (2015), when students employ their intuitive core knowledge to interpret phenomena, sometimes they result in alternative scientific ideas and/or misconceptions. As children grow and are exposed to formal science education, their intuitive understandings, originally rooted in their early experiences, often evolve into more structured beliefs or “ontological commitments” about scientific phenomena, a term used by Slotta and Chi (2006) to refer to the process of attributing specific, tangible entities to abstract scientific concepts.

This classification stems from recognizing that students form beliefs about the nature or existence of these concepts, such as light, force, or electricity. Essentially, they begin to perceive the very being or essence of these phenomena, often attributing physical, tangible characteristics to them. For instance, Reiner et al. (2000) found that students often conceptualize difficult physics concepts, such as light and electricity, by considering physical characteristics for these conceptual constructs. Such attributions can lead to alternative ideas that conflict with empirical evidence. A typical example is students developing an alternative idea that a slowing object is “using up” its force if they assign physical characteristics to “force” and objectify it. The crucial question then becomes will awareness of these ontological commitments help students revise their epistemic practices and form more scientifically accurate ideas?

Lindeman and Aarnio (2007) suggest that paranormal beliefs often stem from a conflation of categories within intuitive core knowledge, similar to the process labeled as “ontological commitment” by Slotta and Chi (2006). They state that when attributes from different categories overlap, people can formulate empirically unsupported explanations of the world. For example, if characteristics of biological or animate entities are applied to psychological ones, a thought (a psychological entity) can be perceived as having life (a characteristic of biological entities), and make decisions (a characteristic of animate entities). This concept of a “living thought” could explain beliefs in telepathy, where thoughts are considered transferable between minds, similar to biological contamination.

Adults with paranormal beliefs and children display similar intuitive thinking confusions. For instance, Lindeman and Saher (2007) found a shared belief in vitalism—the notion that living organisms possess a non-material life energy or force. This common belief underscores the prevalence of paranormal beliefs in society. Furthermore, Svedholm and Lindeman (2013) linked belief in vital energy to alternative medicine practices, emphasizing the broad impact of paranormal beliefs on people’s lives and health.

1.1.2 Effects of Ontological Training on Alternative Science Ideas and Misconceptions

In addressing the issue of alternative conceptions and misconceptions, Slotta and Chi (2006) devised a form of explicit instruction known as ontological training. This approach involves teaching students about the appropriate ontological commitment needed to understand a phenomenon, prior to epistemic practices to make sense of the phenomenon. They found that the training was effective in terms of decreasing student misconceptions about the focused electricity phenomenon. Lindeman and Saher (2007) created a tool to measure lay conceptions of energy and identified four classes of underlying ontological commitments: material substances (e.g., energy can break), living things (e.g., energy can grow or recover from illness), animate beings (e.g., energy can feel pain or hear), mental properties (e.g., energy is conscious), and scientifically valid conceptions (e.g., thermal energy manifests as movement of particles). Building on this work, Svedholm and Lindeman (2013) examined lay conceptions of energy and found similar ontological confusions among students. They observed a decrease in ontological confusions after a targeted lesson on this topic. These sample studies suggest the promising impact of ontological training on student alternative ideas and misconceptions.

1.2 Student Versus Scientist Epistemic Practices

Student and scientist epistemic practices can be compared in many different ways, here I will focus on two differences that can be the bases of an epistemic training with focus on paranormal phenomena. Then, I will describe the possible effects of epistemic training on student alternative ideas and misconceptions.

1.2.1 Spectrum of Idea Formation to Explain a Phenomenon

Students and scientists exhibit a spectrum of epistemic practices when forming ideas to explain phenomena. This spectrum reflects varying degrees of reliance on models and theories, which are critical in developing reliable hypotheses. While modeling helps individuals connect scientific concepts with real-world phenomena, the extent to which models influence thinking varies between students and scientists. For scientists, models and theories are like lenses that illuminate different dimensions of a problem, allowing for a nuanced understanding of complex issues (Street, 2019). They typically employ a combination of abductive, deductive, and inductive reasoning, integrating creativity with rigorous logic. For example, when deducing the sourness of coffee, a scientist might use abductive reasoning to hypothesize its acidity, drawing on a rich understanding of chemical properties (Lawson, 2004). In contrast, students often rely on more intuitive and less structured approaches. Their explanations may originate from similar intuitive processes as scientists but are generally less informed by systematic scientific theories or models (Coley & Tanner, 2015). Consequently, their initial hypotheses may lack the depth and rigor found in scientific hypothesis formation and may not fully leverage the analytical frameworks that models provide (Pluta et al., 2011). This difference does not represent a strict dichotomy but rather stages of development along a continuum where students can evolve toward more sophisticated scientific reasoning as their education progresses.

1.2.2 Spectrum of Using Empirical Evidence to Support Hypotheses

Both scientists and students utilize empirical evidence in their epistemic practices, but they do so to different extents and with varying levels of sophistication. Scientists hold themselves accountable for testing their hypotheses and building empirical evidence to either reject or support them. In contrast, students may consider their intuitive explanations to be true without the need for empirical evidence (Coley & Tanner, 2015). Brem and Rips (2000) found that as students develop their explanations, they use available evidence; otherwise, they do not see the necessity to collect or use data and develop evidence. Instead, they conveniently consider a “plausible causal mechanism.” These crucial differences in approach between students and scientists suggest that students are not familiar with the essential aspect of scientific investigation, in which a claim must be supported or rejected based on empirical evidence (Kuhn & Pearsall, 2000).

1.2.3 Effects of Epistemic Training on Alternative Science Ideas and Misconceptions

Recognizing the differences in epistemic practices between students and scientists, my research focused on exploring strategies to enhance students’ engagement with science practices. This transition aims not to replace one set of beliefs with another but rather to encourage a more rigorous, evidence-based approach to making sense of phenomena. The goal is to cultivate an appreciation for science practices as a powerful, yet continually evolving and tentative way of knowing. In this context, the “Power of Tentative Truth” not only captures the strength of science in progressively constructing truth about the real world but also emphasizes the inherent humility in scientific inquiry. This balance underlines two points: First, scientific knowledge is not absolute but continually evolving as new evidence and perspectives emerge. Second, scientists need to remain open to systematically revise their ideas and challenge their biases in light of new data.

Several studies suggested that science courses are more effective in terms of improving student scientific practices when they include some direct instruction, which involves explicit and structured teaching about paranormal beliefs within the course context (Bensley et al., 2010; Manza et al., 2010; Solon, 2007). Manza et al. (2010) investigated the effect of science courses on student PBs. Their findings suggested that participation in a statistics course emphasizing scientific methods did not significantly alter students’ PBs. However, when they integrated instruction that encouraged students to critically evaluate paranormal beliefs within the context of the statistics course, they observed a statistically significant change in students’ PBs.

Kuhn and Pease (2008) conducted a longitudinal study about the challenges students encountered as they were developing science inquiry skills. They argued that if students become convinced of the scientific epistemic practices, they are likely to utilize these practices in a wide range of questions. They asserted that helping students to become aware of the value of scientific practices should be a priority in science instruction. Coordination of theory and empirical evidence can be a valuable epistemic tool for students to critique and revise their beliefs. For example, consider a scenario where a student intuitively suggests that it is possible to levitate an object using only the power of the mind—a classic paranormal belief. When challenged to support this idea with empirical evidence, they may come to understand that, despite the creativity and intrigue of such a concept, there is no scientific basis or empirical evidence to validate such a claim. This highlights a critical aspect of scientific inquiry: while theoretical creativity is essential for generating hypotheses and ideas, the validation of these ideas must be grounded in empirical evidence. Without such evidence, any claim, no matter how theoretically intriguing or widely believed, remains scientifically unsupported.

1.3 The Iterative Refinement of Educational Theory and Practice: Problem, Model, Solution, Implementation, and Evaluation

This study tackles a broad, overarching problem: Despite the Enlightenment era of the eighteenth century positioning science as a crucial way of knowing, remnants of older, non-scientific thinking, such as paranormal beliefs, persist and flourish in our modern, science-intensive society. Even though science education is used as a tool to disseminate scientific thinking, the widespread prevalence of paranormal beliefs, conspiracy theories, and pseudoscience suggests that non-scientific modes of knowing remain deeply embedded within mainstream society.

This overarching problem can be narrowed down and contextualized within our physics instruction. A general education physics course that utilizes the Next Generation Physics of Everyday Thinking (NGPET) curriculum (Goldberg, 2018), emphasizing the use of models and evidence in scientific explanations, offers a promising context for this exploration. This context led to an investigation of previous literature from various disciplines and construction of a theoretical model which can be summarized in a following nutshell:

Previous studies have indicated that epistemic practices are influenced by intuitive thinking, which is based on intuitive core knowledge learned in early life stages (Spelke, 2000). This convoluted core knowledge can bias intuitive thinking, leading to paranormal beliefs (Lindeman & Aarnio, 2007) and science alternative ideas (Coley & Tanner, 2015). Ontological training has been shown to be effective in helping students become aware of how intuitive biases can develop paranormal beliefs and diminish those beliefs (Slotta & Chi, 2006). Furthermore, epistemic training has been shown to affect student epistemic practices and reduce paranormal beliefs (Kuhn & Pease, 2008; Manza et al., 2010).

According to this model, a hypothetical solution situated in the context of our physics course was constructed. This hypothetical solution is a targeted instruction incorporated NGEPT curriculum enriched by an online learning community (OLC). The OLC includes explicit ontological and epistemic training with the focus on paranormal phenomena.

This hypothetical solution has been evolved, implemented, enhanced, and tested over three consecutive semesters. To test this targeted instruction, three predictions or expectations have been devised. I expected that the intervention would result in a decrease in student paranormal beliefs (Expectation I). Given the intervention included NGPET curriculum, it was expected that the intervention would positively affect student conceptual physics understanding; however, with the added component of OLC, the impact should exceed the effect of standalone NGPET curriculum (Expectation II). Yet, attributing a portion of this improvement to the epistemic and ontological training presents a challenge due to the lack of control group.

This led to the third expectation: As OLC with ontological and epistemic training targets both student conceptual physics understanding and PBs, a correlation was expected between the level of participation in the OLC and both the understanding of conceptual physics and changes in the students’ paranormal beliefs (Expectation III).

This study, beyond its problem–solution educational investigation, involves additional theoretical and practical considerations. These include how this model fits into the context of our physics course, what could be learned from the implementation of the solution, how the targeted instruction could be improved based on collected data, and how the theoretical model could be enhanced by insights from this research. To address these considerations, the guiding research question is as follows:

How does a targeted science instruction, focusing on SEPs and paranormal phenomena, influence college students’ paranormal beliefs and conceptual physics understanding?

Figure 1 illustrates the model of the study as described above. It encapsulates the study’s comprehensive approach and its intended outcomes. The specific details and nuances of the study will be further elaborated in the following section.

Fig. 1

The model of this educational inquiry as described above

2 Methodology

2.1 Context and Participants

The study took place at a Midwestern regional university, classified as a “Master’s Colleges & Universities: Larger Programs” by the Carnegie Classification. This four-year, medium-sized, highly residential university offers a diverse range of academic programs spanning professional, arts, and sciences disciplines, complemented by a comprehensive postbaccalaureate program. It has high undergraduate enrolment and attracts a significant number of transfer-in students.

The university’s student population is diverse, with approximately 56% female and 44% male students, primarily White. This demographic sets the stage for the general education physics course to appeal to a wide spectrum of students who might not typically engage in specialized science courses.

Our introductory physics course, utilizing the studio version of the NGPET curriculum, typically accommodates 25–48 students and engages them in learning model-based physics. For example, in exploring magnetism, students evolve from vernacular explanations to more scientifically aligned models of ferromagnetism through guided investigations and classroom discussions.

To facilitate this learning, students work collaboratively in small groups for about 5 h per week. This approach fosters active engagement and a deeper understanding of scientific models. The course is open to all students and part of the general education curriculum. The participant demographic, comprising about 41% female and 59% male students, mirrors the typical gender distribution seen in this physics course but does not fully correspond with the university’s overall gender ratio. Students majoring in sciences rarely took this course, as they typically opted for calculus or algebra-based physics courses. The study was conducted over three semesters—Phases I, II, and III—with 46, 78, and 61 students, respectively, signing the IRB consent form to participate.

2.2 The Research Methods and Measuring Toolkits

A three-tiered mixed-methods approach was conducted to make sense of the data in this study. At the whole-class level, data was collected through pre- and post-surveys and assessments, along with documentation of participation in classroom activities and participation in the OLC. The general changes at this level were analyzed by comparing pre- and post-survey and assessment results. Moreover, a correlational study was conducted to explore the relationship between the intervention and the changes in the mentioned variables. Moving to the subgroup level, k-means clustering was utilized to distinguish distinct patterns of changes within different clusters of students. Lastly, at the individual level, qualitative analysis was performed to explore student epistemic practices. The specifics of this three-level analysis are outlined below.

2.2.1 Paranormal Belief Survey

I employed the revised Paranormal Belief Scale (PBS) by Tobacyk (2004) to assess participants’ paranormal beliefs. This scale is recognized for its validity and reliability, as mentioned by Tobacyk (2004), Aarnio and Lindeman (2005), and Lobato et al. (2014). The revised PBS version notably improved its subscale test–retest reliability, increasing from ${r}_{tt}=0.61$ in the original to ${r}_{tt}=0.81 ($ Tobacyk, 2004). In the analysis, changes from pre- to post-surveys were examined. It is worth noting that as the subcategories within the PBS are correlated, each student’s PBS result is represented by a single figure: an average score computed from all the survey items. When identifying an unexpected pattern in a subgroup, the Wilcoxon signed-rank test was employed to assess the changes, as the sample size was not normally distributed.

2.2.2 The Monk and Hypothesis Making Activity

In a post-assessment, participants watched a video of a monk claiming to achieve self-levitation and responded to the claim using their newly learned physics models. This assessment (provided I in the supplementary material) was carried out across all study phases with 152 consenting participants. Students’ responses were coded in iterations using a grounded theory approach (Corbin & Strauss, 2008), grouped into categories based on their stance toward levitation, and further classified into themes.

Unexpectedly, some students scored higher on the post-survey than on the pre-survey. To explore this, an extra question was added to the Phase III post-survey, asking students to hypothesize about these survey results. These 53 responses were analyzed using a similar iterative process to the monk activity.

To ensure coding reliability, 30 random responses from both the monk activity and student hypotheses were independently coded by a former colleague, and discrepancies were resolved. A confusion matrix was used to compare the coding results, and the Cohen’s kappa values for the monk activity and student hypotheses were 0.80 and 0.72, respectively, indicating substantial agreement for both sets of data.

2.2.3 Measuring Physics Learning

In this investigation, the Conceptual Physics Assessment (CPA) (Engelhardt, et al., 2018), designed for the NGPET curriculum, was utilized to evaluate student physics learning. CPA comes with NGEPT curriculum to evaluate student physics learning and has been utilized by NGPET instructors. Engelhardt et al. (2018) examined the results of this test on a dataset from 17 instructors and their analysis included Rasch analyses and Hierarchical Linear Modeling (HLM), which confirmed the test’s effectiveness. The study reported statistically significant differences between pre- and post-test scores across various subtests (HLM; p < 0.05), with effect sizes ranging from moderate to large.

In this study, given the sufficiently large sample size in all three study phases and the approximately normal distribution, a paired t-test was employed to examine changes in participants’ scores from pre- to post-test.

2.2.4 Measuring the Student Participation in the Intervention

As described in the following, two variables were used to measure student participation in this intervention as follows.

Classroom Participation

As mentioned, I implemented studio version of NGPET in which students work together in small groups. In each session, students had the opportunity to earn up to 10 points, divided between individual and group efforts. The first five points were allocated for individual work, requiring students to complete specific sections of their workbook independently, yet within the context of a collaborative group setting. The other half of the points were awarded for teamwork and fulfilling group responsibilities. The assessment for both individual and group work was like a qualitative three-tier system: fully completed work merited a “pass” or 5 points, work that was partially completed and showed potential for greater responsibility received 2 or 3 points, and uncompleted work led to a “fail” or 0 points. Generally, students received the full points for their contributions unless non-participation was evident or if the tasks were incomplete. Overall, this dual-focused approach was designed to encourage both individual responsibility in workbook completion and active engagement in collaborative group activities. The total points that they received represented the amount of student classroom participation over the course of a semester.

Participation in Online Learning Community

For building the online learning community (OLC), the Packback (Packback, 2023) platform was utilized. Packback is a platform designed for writing assignments that leverages artificial intelligence to provide real-time, meaningful feedback as students post their writings. The system employs AI to ensure that submissions meet basic standards of originality and a minimum quality threshold set by the instructor before allowing students to submit their questions, discussions, and responses. This quality control mechanism primarily checks for plagiarism and basic grammatical integrity to ensure that submissions contribute constructively to peer discussions.

Students whose submissions do not initially meet the AI’s standards are not prevented from participating; instead, they are encouraged to revise and resubmit their work. This iterative process aims to progressively enhance their writing skills, thereby supporting all students in contributing meaningfully to discussions.

Each week, students are required to submit one question for discussion, along with two responses to their peers, making the total number of posts submitted over the course of a semester a measure of student participation in the OLC. Importantly, the criteria for “minimum quality” are guided by examples of effective writing provided by the instructor rather than strictly quantified metrics. Furthermore, this approach fosters a community of learners where peer critique plays a central role in enhancing the learning environment. The utilized educational methodology promotes open writing within established community norms, which is a recognized and effective educational practice.

2.3 Exploring Changes in Belief and Conceptual Understanding: Clustering and Regression Analysis

At the subgroup level, to identify patterns within clusters of students, I conducted k-means clustering analysis and multinomial regression analysis on normalized variables. The k-means clustering analysis, performed using RapidMiner Studio (RapidMiner, 2023), incorporated the normalized changes in the PBS survey and CPA test. To enhance the reliability of this analysis, data from all three phases were combined, excluding thirteen outliers with z-scores above 2. This analysis enabled the identification of distinct groups with similar changes in beliefs and physics understanding. The optimal cluster number was identified using the elbow method (Nainggolan et al., 2019).

Following the cluster analysis, a multinomial logistic regression (Petrucci, 2009) was conducted to investigate the relationship between cluster memberships (dependent variable) and the initial PBS survey and CPA test scores (independent variables) (Brownlee, 2017). This analysis revealed whether initial paranormal beliefs and physics understanding could predict the observed changes.

2.4 The Design of the Intervention

The educational intervention aimed to improve student epistemic practices. NGPET curriculum mostly covers the indirect aspect of the training, while the epistemic practices related to paranormal phenomena is directly discussed in the OLC. The intervention includes both epistemic and ontological training.

2.4.1 NGPET Curriculum

The NGPET curriculum, which is supported by the National Science Foundation (NSF), focuses on scientific modeling, empirical observation, data collection (both quantitative and qualitative), and constructing scientific explanations. This curriculum provides an effective learning environment for students to implicitly experience science as a way of knowing, emphasizing the use of models to develop ideas and gather evidence. In the first phase of the study, an online learning community (OLC) was implemented, specifically designed to facilitate explicit discussions on SEPs and promote the transition from common student epistemic practices toward science ones. For the subsequent phases of the study, the content of the OLC was further enhanced by the addition of an ontological training. The revised intervention was implemented in the following two semesters.

2.4.2 Complementary Treatment: An OLC Focusing on Epistemic and Ontological Training

The intervention detailed in this study encompasses an NGPET curriculum and an additional educational element, namely, epistemic and ontological training. NGPET involves students in SPEs, but direct, explicit reflection on these practices is somewhat limited. To bridge this gap, an online learning community (OLC) was established under the premise that learning is optimized when active participation and reflection on epistemic practices occur concurrently (Basir, 2019). Most SEPs took place within the NGPET classroom, while the OLC provided a platform for structured and explicit reflection on these SEPs.

Table 1 shows the learning objectives of the OLC over different weeks. The initial five weeks focused on understanding how scientific methodology applies to both natural and paranormal phenomena and how SEPs assist in reshaping ideas. The subsequent five weeks emphasized applying SEPs to comprehend everyday phenomena and using the insights about SEPs to reflect on peers’ work. Weeks 11 and 12 were dedicated to ontological training. The final two weeks of the OLC revolved around deeper reflection on science as a way of knowing. While Table 1 provides an overview of the weekly tasks, detailed instructions for two exemplar weekly writing assignments can be found in the supplementary materials.

Table 1 The topic of the weekly writing assignments for the OLC, the learning objective of those assignments, and the goal of the instruction in that week

2.4.3 The Ontological Training

The ontological training, adapted from Svedholm and Lindeman (2013), was executed in five steps.

Step I: In the OLC, students took the energy conceptions survey (Lindeman & Saher, 2007), analyzing one item of the survey in their writing assignments, complemented by peer discussions. The aim was acquainting students with various energy conceptions.

Step II: In a face-to-face training session, students watched the Heider-Simmel animation (Heider & Simmel, 1944) in class. This experimental animation uses simple shapes to explore how observers attribute personality traits and intentions to abstract objects. Students reflected on the animation and its relevance to their responses in the energy conceptions survey. They discussed their interpretations in small groups, comparing similarities and differences in their narratives.

Step III: After group discussions, students received a text explaining the theory of core knowledge, how intuitive thinking is utilizing the core knowledge to develop interpretations, and its potential for alternative ideas and misconceptions. They were provided with a worksheet containing the energy conceptions survey results. Students were given questions to reflect how core knowledge confusions contribute to the development of conflicting energy conceptions.

Step IV: In small groups, students chose an everyday phenomenon, developed explanations based on alternative energy conceptions from the survey, and elucidated how these ideas resulted in flawed explanations.

Step V: Within the OLC, students received a list of ten energy therapies along with brief descriptions. They were tasked with investigating one of the energy therapies and exploring how alternative energy conceptions were utilized in its development.

Supplementary materials include the necessary resources for each step of this training.

3 Results

3.1 Tracing the Change in Paranormal Beliefs: A Three-Phase Investigation

The PBS pre- and post-survey was administrated to 34, 63, and 40 participants in Phases I, II, and III, respectively. The mean scores for precognition and psychokinesis categories decreased from initial 2.9¹ to 2.6 in Phase I (t =  − 1.9, p = 0.03, Cohen’s d =  − 0.29), overall scores from 3.5 to 3.3 in Phase II (t =  − 2.20, p = 0.02, Cohen’s d =  − 0.21), and from 3.2 to 2.9 in Phase III (t =  − 2.1, p = 0.02, Cohen’s d =  − 0.32). Results signify a significant decrease in paranormal beliefs across all phases (Table 2).

Table 2 The results of the PBS pre- and post-survey in the three phases of the study

3.2 Assessing Replication Reliability Across Phases: A Comparative Analysis of Pre- and Post-Survey Scores

To assess the replication reliability, a comparison was conducted on the results of PBS pre- and post-survey and the changes between PBS pre- and post-surveys in Phase II and Phase III. The aim was to determine if there were any significant differences between the sampled populations in these two phases. As indicated in Table 3, the results suggest that there are no significant differences between Phases II and III. It can be inferred that students in Phases II and III responded to the pre- and post-surveys similarly and experienced a comparable decrease in their paranormal beliefs.

Table 3 Comparing PBS pre- and post-survey responses in Phases II and III

3.3 Evaluating Physics Understanding: A Three-Phase Analysis of Pre- and Post-Test Improvements

Table 4 presents physics test results from all phases. In Phase I, significant score improvement was observed among 41 participants (pre: M = 23.3/42, SD = 7.0; post: M = 31.0, SD = 6.5), supported by a one-tailed paired t-test (t = 9.0, df = 40, p < 0.001; Cohen’s d = 1.13). In Phase II, 63 participants exhibited significant improvement (pre: M = 23.3/42; post: M = 29.1), affirmed by a one-tailed paired t-test (t = 10.7, df = 62, p < 0.001; Cohen’s d = 0.93). In Phase III, 51 participants showed notable improvement (pre: M = 21.8/42; post: M = 26.7), supported by a one-tailed paired t-test (t = 5.5, df = 50, p < 0.001; Cohen’s d = 0.60). Across all phases, significant improvement was consistently found in pre- and post-test scores.

Table 4 The results of the pre- and post-physics test in the three phases of the study. The maximum points in CPA are 42

3.4 Unexpected Increase in PBS Post-Survey Scores: Analyzing Subgroups with Increased Paranormal Beliefs

Contrary to expectations, some students’ post-survey PBS scores were higher than pre-survey scores. Table 5 shows this subgroup’s scores per phase.

Table 5 The results of PBS survey for the subgroup of students who scored in post-survey more than the pre-survey. The results repeated in all three phases of the study

In Phase I, a subgroup of 14 students had a higher post-survey mean (M = 3.08, SD = 0.94) than pre-survey (M = 2.51, SD = 0.81). The Wilcoxon signed-rank one-tailed test verified this positive change (W = 0, n = 14, p < 0.05). Similar patterns appeared in Phases II and III. This unexpected yet consistent pattern was further explored through qualitative individual-level and quantitative subgroup-level analyses, as discussed in the next sections.

3.5 K-Means Clustering Analysis: Distinguishing Student Subgroups by Changes in Conceptual Physics and Paranormal Beliefs

While Table 2 shows an overall decrease in the Paranormal Belief Scale (PBS) pre- and post-survey scores, an unexpected pattern was observed at a subgroup level: some students scored higher in the post-survey across all three study phases.

To make sense of this phenomenon, a k-means clustering was conducted on the combined data set from all study phases. The clusters were based on the changes in PBS (Def of PBS) and CPA (Def of CPA) over a semester. Four distinct clusters emerged, and a Wilcoxon signed-rank test was performed to assess changes. Except for Cluster 4, all clusters demonstrated statistically significant changes in both conceptual physics and paranormal beliefs. Each cluster is detailed as follows:

• Cluster 1: Students in this subgroup showed a statistically significant increase of 4.8 in the CPA test scores, signifying a medium-sized improvement (Cohen’s d = 0.76) in conceptual physics understanding. Furthermore, they also showed a significant increase in paranormal beliefs by 0.3, representing a small effect size (Cohen’s d =  + 0.3). This cluster, encompassing 35% of the students, is characterized by a medium-sized improvement in conceptual physics comprehension, paired with a small-sized increase in paranormal beliefs.

• Cluster 2: This cluster comprises students who demonstrated a significant improvement in conceptual physics understanding (CPA change = 11.2) with a large effect size (Cohen’s d = 2.08). They also showed a significant decrease in paranormal beliefs (PBS change =  − 1.6) with a large effect size (Cohen’s d =  − 2.39). This cluster, representing 10% of the students, is characterized by large-sized changes in both CPA and PBS.

• Cluster 3: Participants in this cluster showed a moderate decrease in paranormal beliefs (PBS change =  − 0.8, Cohen’s d =  − 0.89) along with a medium increase in CPA (CPA change = 4, Cohen’s d = 0.66). This cluster accounts for 32% of the total data.

• Cluster 4: Students in this cluster exhibited no significant change in their paranormal beliefs (PBS change = 0.09), but demonstrated a large-sized improvement (Cohen’s d = 2.63) in the CPA test (CPA change = 13). It represents 23% of the population.

Figures 2 and 3 graphically illustrate these findings. Figure 2 represents the four clusters based on students’ progression in the CPA and changes in the PBS survey. Figure 3 showcases the average learning for each cluster in a two-dimensional space.

Fig. 2

Four clusters of students based on changes in Conceptual Physics Assessment (Def of CPA) and changes in Paranormal Belief Scales (Def of PBS) Note: The scales on both axes are normalized (i.e., each value is divided by the maximum value in its respective axis). N = 98, the k-means clustering analysis is based on the data collected from all three phases of the study

Fig. 3

The amount of change in PBS and CPA within each cluster in two-dimensional learning space Note: The scales on both axes are normalized (i.e., each value is divided by the maximum value in its respective axis), and the bars in both directions represent standard errors

The standard error bars in Fig. 3 suggest no significant overlap in average learning among the clusters, except for between Clusters 1 and 3. This indicates that the clusters are statistically distinct in both learning dimensions, except for Clusters 1 and 3 which show a significant difference solely in their average PBS change. Notably, while the average physics learning in Clusters 1 and 2 is statistically significant, the values are closely aligned.

In summary, Fig. 3 suggests that Clusters 2 and 3 demonstrate high levels of conceptual physics learning, with the key distinction being stable paranormal beliefs in one cluster versus a significant change in the other. Conversely, Clusters 1 and 4, which show lower levels of conceptual physics learning, are primarily distinguished by an increase in paranormal beliefs in one cluster and a decrease in the other.

3.6 Analyzing Correlations: Intervention, Paranormal Beliefs, and Conceptual Physics Learning

The relationship between the intervention, as reflected by the degree of student participation, changes in paranormal beliefs, and conceptual physics learning over the course of a semester was explored through a Pearson correlation analysis and by comparing these variables across identified clusters.

As shown in Table 6, a negative correlation exists between PBS pre-survey and CPA pre-survey (r =  − 0.36, p < 0.001), indicating that higher performance in CPA pre-test is linked with a lower score in the PBS pre-survey.

Table 6 The results of Pearson correlation analysis for four key variables

Significant negative correlations were also observed between “Def of PBS survey” and “classroom participation” (r =  − 0.26, p = 0.008), as well as between changes in PBs and the level of participation in the online learning community (r =  − 0.20, p = 0.04). Furthermore, conceptual physics learning during the semester, as measured by “Def of CPA test,” is positively correlated with “OLC participation” (r = 0.22, p = 0.03). Given that the level of student engagement in the intervention is quantified by these participation variables, these correlations suggest that engagement in ontological and epistemic training is associated with both physics learning and changes in PBs.

As mentioned earlier, the correlation between participation and both dimensions of learning indicates a relationship between the intervention and those learning variables. Moreover, the results of the k-means clustering analysis, as illustrated in Fig. 4, suggest a partial relationship between the extent of participation in the OLC and the variances observed among the clusters. There are significant differences in the levels of participation between Clusters 1 and 2. Thus, it could be inferred that the distinction in conceptual physics learning and changes in paranormal beliefs between Clusters 1 and 2 is related to the varied degrees of participation in the online learning community. These correlational relationships can be considered the evidence for Expectation III.

Fig. 4

The relationship between the amount of participation in OLC and conceptual physics learning within the clusters. Note: The scales on both axes are normalized (i.e., each value is divided by the maximum value in its respective axis), and the bars in both directions represent standard errors

3.7 Exploring Student Interpretations of the Differential Impact of the Intervention

Table 7 presents the results of a qualitative analysis conducted on students’ responses to the question of why some students, as shown in Table 5, scored higher in the PBS post-survey compared to their scores in the PBS pre-survey. The individual level qualitative analysis aimed to explore the students’ perspectives on the factors contributing to this discrepancy.

Table 7 Student hypotheses/codes explaining why some students scored higher in PBS post-survey

Among the participants, 58% hypothesized that the educational intervention had mixed impacts on students’ beliefs about paranormal phenomena. 15% of the participants hypothesized that students’ initial conditions played a role in their reactions to the intervention. Additionally, some students raised concerns about the validity of the results. Others proposed the influence of external factors that could have impacted the students’ belief scores.

A detailed discussion of these qualitative findings will be presented in Sect. 4, providing further insights into the potential factors contributing to the observed variations in the PBS post-survey scores.

3.8 Classification of Student Responses on the Mechanism of Paranormal Levitation

Table 8 presents the classification of student responses obtained during the post-assessment activity. Students reflected on a video clip depicting a monk seemingly self-levitating. The students were asked to provide their hypotheses on the mechanism by which the monk could generate an upward force to counteract gravity based on magnetic, electric, and forces and energy models learned in NGPET curriculum. The responses were categorized as follows: First, supporting paranormal levitation: Some students offered pseudoscientific explanations involving energy release or spiritual/mystical beliefs. Second, against paranormal levitation: Students fell into four categories: those who denied the paranormal aspect without providing an alternative explanation, those suggesting hidden contact forces, those proposing non-contact forces like magnetism, and those focusing on video manipulation to explain the levitation. Third, neutral stance: Some students remained neutral regarding the paranormal aspect. The discussion section will delve deeper into these responses, offering a more thorough analysis and explanation.

Table 8 Categories/codes of student explanation for the monk levitation and the distribution of student responses for each category

4 Discussion

In the following discussion, the varied outcomes of the intervention aimed to influence students’ PBs in a science-intensive educational context are examined. The subsections methodically explore the impact of the intervention from whole-class to individual levels. The analysis begins with an overview of the overall effectiveness of the model-based approach (4.1.1), discusses the specific role of the online learning community (4.1.2), and addresses the nuances of the intervention’s effect size (4.1.3). Subsequent sections scrutinize varying responses among student subgroups (4.2.1) and discuss differential impacts on student beliefs (4.2.2). Following this, a detailed exploration into the individual-level variations in student responses is presented (4.3), where the focus shifts to understanding the unique ways in which different students internalized and reacted to the intervention. This section emphasizes the diversity of individual learning experiences and the complex interplay between personal factors and educational outcomes. The discussion ends with the reflections on the study’s limitations (4.4) and implication for future research and practice (4.5).

4.1 The Whole-Class Level Analysis: Evaluating the Interventions’ Broad Impact

4.1.1 The Impact of the Intervention on PBS and CPA

Drawing on the theoretical model previously discussed, I conjectured that one effective approach to influencing student SEPs is through active engagement of students with a science curriculum like NGPET that has implicit emphasis on the coordination of models and evidence (Kuhn & Pearsall, 2000). In conjunction with NGPET curriculum, I incorporated an OLC that included the explicit epistemic and ontological training infused with paranormal phenomena, targeting three main objectives:

• First, scientific hypotheses are supposed to be developed based on the scientific models and theories rather than vernacular intuitive ideas (Lawson, 2004).

• Second, scientific hypotheses should be supported by empirical evidence; mere plausible explanation are not sufficient to accept a hypothesis (Kuhn & Pearsall, 2000) as a tentative truth.

• Third, confusions in intuitive core knowledge can lead to science alternative ideas and misconceptions (Svedholm & Lindeman, 2013) as well as paranormal beliefs (Lindeman & Aarnio, 2007). These hidden ontological commitments shape intuitive thinking, resulting in ideas about everyday phenomena (Slotta & Chi, 2006) that vastly differ from scientific hypotheses.

In alignment with these objectives, the instructional intervention was implemented. It was anticipated that the intervention should help students to revise their conceptual physics (expectation II), PBs (expectation I), and epistemic practices associated with paranormal phenomena such as levitation or psychokinesis. The findings, as shown in Tables 2 and 4, revealed a statistically significant reduction in the average PBS and CPA. Consistent with earlier studies focusing on epistemic (Kuhn & Pease, 2008) and the ontological training (Slotta & Chi, 2006; Svedholm and Linderman, 2013), it appears that the learning environments effectively helped students to revise their paranormal beliefs.

4.1.2 The Impact of Online Learning Community on PBS and CPA

Investigating Expectation II was not straightforward, since I did not have a control group. This challenge changed the focus of analysis toward the role of participation in the OLC on conceptual physics learning and changes in PBs. Based on the theoretical model, it was expected that the OLC influence on both conceptual physics learning and the change in student PBs (Expectation III).

The correlation study revealed a significant relationship between student participation in the OLC and their learning outcomes. As indicated in Table 6, “OLC participation” is negatively correlated with “Def of PBS” (r =  − 0.26, p = 0.008) and positively correlated with “Def of CPA” (r = 0.22, p = 0.03). These findings suggest that active engagement in the OLC, particularly in ontological and epistemic training is linked to improvements in both conceptual physics understanding and changes in paranormal beliefs, supporting Expectation III of the study.

4.1.3 The Intervention Had Small Effect Size on Student PBs

The results from Phases I and II of the study revealed that the change in PBS and CPA between pre- and post-survey and test were statistically significant. However, while the effect size for CPA was large, the effect size for both Phases I and II on PBS was small, d =  − 0.29 and − 0.21, respectively. This raised questions about the significance of the observed changes and the need for improvement in the implementation of the intervention.

In Phase III, as presented in Table 2, similar results were observed and the effect size for PBS remained small (d =  − 0.32). As illustrated in Table 3, participants in Phases II and III were not significantly different from each other in terms of their scores in PBS pre- and post-survey as well as the amount of change from PBS pre- to post-survey. Moreover, as shown in Table 4, in all three phases students achieved a medium to large-sized improvement in PCA. These similar results from three different semesters, Phases I, II, and III, enhanced the reliability of the statistical findings. However, the study was still in search of a hypothesis to explain the small effect size on PBS and strategies to enhance it.

4.2 The Subgroup Level Analysis: Dissecting Intervention Effects Among Different Student Groups

4.2.1 The Impact of Online Learning Community on PBS and CPA

Figure 3 further illustrates that the average scores of students in Clusters 1 and 3 differ significantly for both paranormal belief changes and conceptual physics understanding. As shown in Fig. 4, these two clusters also differ statistically in the degree of participation in the OLC. Thus, participation in the OLC appears to be a distinguishing factor for these clusters, implying a relationship between OLC participation, changes in paranormal beliefs, and changes in conceptual physics understanding. Insights into the potential reasons behind the differential impact of the intervention on students can be inferred from a qualitative analysis of student responses.

4.2.2 Differential Effect of Intervention and the Small Effect Size on Student PBs

The analysis of subgroup data from pre- and post-surveys revealed a consistent pattern across all three phases: a subset of students scored higher on the PBS post-survey than on the pre-survey, as shown in Table 5. The k-means clustering model seems to provide description for this pattern, offering potential insight into the small effect size seen in the PBS survey changes. Cluster 1, comprising 35% of participants, indicated a small increase in PBs (Cohen’s d =  + 0.3), while no significant change in PBs was observed in Cluster 4, which represented 23% of the sample. The intervention did not impact student PBs in Cluster 4 and resulted in an increase in student PBs in Cluster 1. This contrasted with Clusters 2 and 3, where the intervention resulted in a significant decrease in student PBs. These variations suggest that the intervention had a differential impact on student PBs, which likely contributed to the overall small effect size.

4.3 Individual Level Analysis: The Potential Explanations for the Differential Impact of the Intervention on Students

As detailed in Sect. 2, students were asked to hypothesize why some of them scored higher on the PBS post-survey. Table 7 presents the results of the qualitative examination of these hypotheses. In the following, I will discuss student hypotheses and any preliminary evidence that was obtained to support or weaken those hypotheses. Noteworthily, the limited evidence provided here is insufficient to validate or dismiss most of these hypotheses, necessitating further study for this “why” question.

Three students hypothesized a potential methodological issue: their peers may not have paid enough attention when they responding to the PBS pre- and post-survey. To assess internal consistency of the responses, I computed the Cronbach’s alpha for pre- and post-survey for two last phases of the study. In Phase II, the values were 0.90 and 0.91, respectively. In Phase III, they were 0.86 and 0.91, respectively. This analysis suggests that despite the potential or partial inattentiveness from some students, overall, responses to the survey were constant and reliable.

Three students hypothesized that this may have occurred due to a unique personal experience unrelated to the course, “hysteria may be increased because of the workload of the semester.” Given the statistically significant replication of these patterns across three semesters, this hypothesis appears unlikely.

Eight students hypothesized that the initial conditions might have influenced their responses to the intervention. While this is a plausible hypothesis, it is too broad to investigate without comprehensive demographic and background information. Some students specifically proposed that the degree of their initial beliefs in PBs might have influenced the way they reacted to the intervention and post-survey. To investigate this hypothesis, four clusters were considered the dependent variables, PBS pre-survey and CPA pre-test were considered independent variables, and a multinomial logistic regression was conducted on these variables. The overall fit of the regression model indicated statistical significance (χ² = 28.7, df = 6, p < 0.001). The pseudo-R-squared value of the model was calculated as well, demonstrating that the model accounted for approximately 11% of the variance in the dependent variable, the clusters. Thus, it seems that the initial condition of students, represented by CPA pre-test and PBS pre-survey, can partially predict (11%) how students would be associated with those four clusters. Thus, it can be inferred that the result of CPA and PBS pre-survey could have partially had an impact on changes in CPA and PBS.

Majority of participants hypothesized that student beliefs about paranormal phenomena affected by what they learned in class. Of these participants, 25% hypothesized that the increase in PBs was altered by the course content. However, they did not specify which aspect of the course may have impacted their PBs, “through what we learned in class it either encouraged or discouraged what our original beliefs were.” Roughly 10% proposed that exposure to different kinds of paranormal phenomena during the course enhanced some students’ beliefs in these phenomena, “After sharing stories about paranormal activity some have witnessed, people started believing in the supernatural more.” They did not specify how though.

Half of those students hypothesized that some learners used the course materials such as science models to rationalize the paranormal phenomena and this improved their PBs. For instance, one participant suggested: “It could be possible that students can explain more beliefs with science rather than believing in the unknown.” Here, the implication is that prior to the course some students accepted the paranormal phenomena as face value, with the mechanism of their occurrence being “unknown.” However, during the course, they realized that they can use scientific models and theories to explain these phenomena. It appears that the intervention helped them to construct “plausible causal mechanism” (Brem & Rips, 2000) for the paranormal phenomena. It seems that students were exposed to paranormal beliefs; however, instead of using SEPs to be critical about their PBs, they used science models to develop a science-like, pseudoscientific, explanation for their beliefs.

If students utilized course materials to develop plausible mechanism to support their paranormal beliefs, one would expect to observe those mechanism invoked in the explanations of paranormal phenomena. As could be seen in Table 8, one of the emerged codes is pseudoscience, wherein students used the course materials to explain their beliefs about the paranormal levitation of the monk. For instance, one of the pseudoscientific mechanisms for paranormal levitation suggested by a student was “Probably some gravitational force like mechanism holding him up and lifting him.” Here, the student intuitively used a special kind of gravitational force that is pushing an object upward. The utilized epistemic practice looks like SPEs; however, this creative idea is not drawing from science models and theories. Another student postulated: “He is able to make this force based on the energy in his body. He releases energy and gravity changes on his body.” This student intuitively imagined a force fed by the person’s body energy. Consistently, Lobato et al. (2014) indicated that there is strong association between PBs and pseudoscience.

Finally, about 15% of these students, who felt that the PBs in their peers increased as a result of course content, speculated that learning about the empirical observation and evidence might have somehow contributed to this increase in beliefs, “I think being exposed to empirical evidence can explain why there has been an increased shift in perspective.” However, they did not specify how this occurred.

Given the background literature discussed in this paper, being familiar with the role of evidence in explaining the phenomena should decrease student beliefs in paranormal phenomena, due to lack empirical evidence. This apparently was the case for some students:

“Some students may realize there isn’t enough evidence that would support paranormal activity. While others students may tend to think more on the creative side especially with paranormal activity being the ‘unknown’ and not being proved to be true or false.”

It seems this participant is also suggesting that some students may have developed an alternative idea or misconception about the role of evidence in the process of knowledge formation, epistemic practices. They seem to be under impression that when a claim cannot be supported or rejected, it should be accepted as definite truth. For instance, in the case of someone claiming to read a stranger’s mind, based on this alternative idea about the role of evidence, these students would accept the claim as truth because no empirical evidence exists to refute it. It seems that they are not aware that the burden of supporting the claim with evidence is on the person who makes a claim, not the one who hearing it. The empirical support for this interpretation can be found in the monk activity when one of the students who believed in paranormal levitation stated: “Nothing to disprove the video.” From this statement, the student seemingly expected evidence to “disprove” the monk’s claim and, in the absence of such evidence, accepted the claim to be true. This is a famous fallacy called the burden of proof. According to this fallacy, if a claim has not proven to be false, then it must be true. Similarly, Mukerji and Ernst (2022) observed that using the burden of proof fallacy, proponents of homeopathy argue that “just because we have not found evidence for homoeopathy, we have not found evidence against it” (p.10), as if they make a claim about homeopathy and, by shifting the burden of proof, expect the opponent to provide evidence against their claim.

It is also possible that some students used empirical observation and evidence from the course to strengthen their PBs, as suggested by this statement: “I think being exposed to empirical evidence can explain why there has been an increased shift in perspective.” The evidence to support this hypothesis can also be derived from the pseudoscientific explanations that students provided in the monk activity. For instance: “The human body can generate enough energy to cause it to move.” The student might have had experiences of seeing someone or personally managing to use the internal energy for lifting themselves up using a pole, then, intuitively used this empirical observation to construct an explanation and reinforce their beliefs about paranormal levitation.

In a nutshell, hypothesizing about why some students scored higher in PBS post-survey, I could emphasize on two inclusive explanations: First, as Mukerji and Ernst (2022) reported the misused of epistemic tools by the proponent of homeopathy to defend their beliefs, I conjecture that our students could have developed alternative ideas about SEPs such as the burden of proof fallacy to confirm their PBs. Second, aligned with what Brem and Rips (2000) mentioned about student tendency to develop plausible causal explanation, I conjecture that the intervention helped students to learn some science models, applied them to develop pseudoscientific explanation to support paranormal phenomena, and used them to boost their confidence about the PBs.

4.4 Limitations of the Study

The reduction in paranormal beliefs observed may be influenced by selection bias, as participants were general education physics students potentially more open to belief change. These results may not be representative of all students on campus. Replicating the study in other disciplines, like biology, could address this bias, but it presents significant challenges, such as needing to replace the current curriculum.

There is also a possibility that students simply became more skeptical over time as they progressed through the NGPET curriculum regardless of the epistemic and ontological training. However, studies have shown that science courses focusing on epistemic practices but not providing direct opportunities for students to reflect on those practices are usually ineffective in improving student epistemic practices (Bensley et al., 2010; Manza et al., 2010). Therefore, it is improbable that the standalone NGPET curriculum impacted student paranormal beliefs reported in this study, though a control study could further investigate this limitation.

The study acknowledges that regression to the mean can introduce bias in pre- and post-assessment studies, depending on the specific circumstances. Shifts in PBS scores might partly be a result of students naturally gravitating toward mean scores over time. However, given the distinct response patterns between CPA and PBS, it is unlikely that regression to the mean is the sole explanation for the observed changes. Without random sampling and a control group, it is challenging to completely eliminate the influence of regression to the mean as a potential confounding factor.

Another potential limitation of this study could be social desirability bias. In the OLC and classroom discussions, students may have presented skepticism about paranormal phenomena due to perceived social acceptability, potentially affecting their reported beliefs. This bias may have surfaced in the OLC; however, the private nature of the survey responses likely limited its influence. Moreover, students may unknowingly have a bias toward socially desirable responses, even when answering a private survey. However, the fact that some students scored higher on the post-survey does not entirely support this alternative explanation of the data. Investigations into these limitations can add depth to the understanding of this issue and offers a useful direction for future research.

Finally, based on the theoretical model discussed in this study, it was expected that epistemic and ontological training could impact both paranormal beliefs and conceptual physics learning. While the findings showed a significant correlation between the intervention and these two variables, it is crucial to note that these correlational results are merely evidence to support the study’s hypothesis, not indicative of a causal relationship. Any cause-and-effect inference between the intervention and student changes necessitates future experimental studies employing randomized sampling.

4.5 Implication for Future Research and Practices

The study illustrated that despite a general decrease in PBs among all students in the whole-class level, different subgroups reacted statistically different to the intervention. Similarly, the shift from the subgroup to the individual level provided insights about why the intervention had differential effect on students. Shifting focus from the entire class to distinct subgroups and then to the individual responses helped to see new meanings in the data.

Furthermore, the three-tiered analysis could be instrumental for enhancing the effect of the intervention. For instance, in the subgroup level, the results from the multinomial logistic regression propose that the initial scores on the PBS and CPA can predict with 11% accuracy what subgroup students will belong to. The intriguing question arises, what additional demographic and background information and other pre-assessment could improve the predictability of subgroup membership? If such an educational differentiation would be followed through, it is important to consider any potential risks of stereotyping or profiling—it would be essential to ensure that collected demographic and background information would be solely used to enhance educational effectiveness and respond to the varied needs of each subgroup without compromising individual student identities or experiences. By shifting focus to the individual level, there is greater potential to enhance the intervention. For instance, some students possibly developed alternative science ideas about SPEs, like the burden of proof fallacy, or using scientific models to build pseudoscientific explanations for paranormal phenomena. This underscores the need for proactive teaching strategies that address these common learning challenges.

Table 6 reveals a notable pattern: a significant negative correlation (p < 0.001) of − 0.38 between CPA pre-test scores and PBS pre-survey scores, indicating that students’ initial conceptual physics understanding and paranormal beliefs were interrelated. This correlation was consistently observed across three phases of the study with correlation coefficients of − 0.44, − 0.29, and − 0.36, respectively (all p < 0.05). This consistent negative correlational relationship between these two dependent variables can be explained by a mediator variable. Analytical thinking is one such factor; as Aarnio and Lindeman (2005) suggest, students with a stronger analytical inclination tend to have fewer paranormal beliefs, potentially due to the mutual reinforcement of analytical skills and physics understanding leading to reduced paranormal belief susceptibility. Conversely, “ontological commitments” (Slotta & Chi, 2006), particularly those rooted in “convoluted core knowledge” (Lindeman & Aarnio, 2007), might lead to increased paranormal beliefs and science misconceptions. From the educational implication perspective, this dual-pathway implication underscores the complexity of the cognitive processes at play and highlights the importance of addressing both analytical and intuitive thinking in educational strategies to reduce misconceptions and paranormal beliefs effectively. The correlation pattern observed also raises intriguing questions for further research. It prompts an exploration into which mediators, such as analytical thinking or ontological commitments, might best explain the linkage between conceptual physics understanding and paranormal beliefs. Additionally, it is worthwhile to investigate if this correlation is unique to Conceptual Physics Assessment (CPA) or if it extends to other measurement tools in physics education, like the Force Concept Inventory (Hestenes et al., 1992). Extending the scope, this correlation pattern’s presence in other scientific disciplines, as evidenced by Bensley et al. (2014) who found a similar negative correlation in psychology, further broadens the research horizon, indicating a potentially universal trend across various fields of scientific education.

Research studies with different background theories have underscored the impact of both epistemic and ontological training as means to alter student epistemic practices. However, a deeper understanding is needed to determine the most effective approach: whether it is purely ontological training, epistemic training, or a combination of the two—the latter being current assumption in this study.

Incorporating scientific thinking into everyday life extends far beyond the confines of traditional academic or scientific disciplines. Practices such as skepticism, empirical testing, and evidence-based reasoning are crucial for navigating a world increasingly saturated with information at different level of reliability. For instance, when individuals apply these scientific practices to evaluating health claims, political statements, or news reports, they are more likely to make informed and rational decisions, thereby protecting themselves and their communities from the detrimental effects of misinformation. Promoting science as a way of knowing is not just about understanding scientific principles; it is about applying a scientific approach to all aspects of life. This perspective advocates for a broader view of science as empirical academic inquiry, even in domains traditionally considered beyond scientific scrutiny. For instance, Daniel Dennett (1991) calls for a scientific approach to understanding human consciousness; Hawking and Mlodinow (2010) discuss the idea of moving beyond traditional philosophical approaches toward a philosophy that is more intertwined with empirical science; Harris (2011) advocates for applying scientific rationality to moral and ethical questions, bridging the gap between science and human values; Dawkins (1998) argues that even religious claims should be subjected to empirical scrutiny if they aim to describe reality. This perspective promotes science as a method of critically thinking about any claim related to reality, whether natural or supernatural (Hoffman, 2019).

5 Conclusion

The present study offers important insights into the effects of an educational intervention on university students’ PBs, as well as their grasp of conceptual physics. This study acknowledges paranormal beliefs as widespread thought patterns associated with the old ways of knowing predate the Enlightenment era of the eighteenth century. To tackle this challenge in the realm of science education and drawing from various disciplines, a theoretical model was conceived to connect science instruction, student paranormal beliefs, and conceptual physics understanding. The model suggests that intuitive epistemic practices, grounded in convoluted core knowledge, may develop both paranormal beliefs and alternative scientific conceptions. Consequently, addressing this issue involves a science intervention that places emphasis on epistemic and ontological training within the context of paranormal phenomena.

Proceeding under the assumption of the hypothetical effectiveness of this model-based educational approach, it was anticipated that the intervention would diminish student beliefs in paranormal phenomena (Expectation I) and further amplify conceptual physics learning beyond the impacts of the standalone NGPET curriculum (Expectation II). Collected data from the three phases of the study consistently provided evidence to support Expectation I, demonstrating a statistically significant change in PBs. Alongside this, a consistent and substantial change in student understanding of conceptual physics was indicated by the collected data. However, attributing a portion of this improvement to the epistemic and ontological training presents a challenge due to the lack of control group.

This challenge necessitated a shift in focus toward the role of participation in the OLC on conceptual physics learning and changes in PBs, leading to the development of the third expectation. Based on the model, I expected that the online learning community with a specific focus on epistemic and ontological training on both conceptual physics learning and the change in student PBs (Expectation III). The correlation study indicated that the amount of learning in conceptual physics and changes in PBs were correlated with the level of participation in OLC. Moreover, turning attention to the subgroup level of analysis and utilizing the k-means clustering, it was shown that two subgroups of students that were statistically distinct in terms of the amount of change in CPA and PBS were significantly different in terms of the amount of participation in OLC. The subgroup that scored positively in the PBS had significantly less amount of participation in OLC. This association also provides a support for Expectation III.

Additionally, the interplay between three level of data analyses provided valuable insights about the research question. The subgroup analysis identified four distinct clusters of students, displaying statistically distinct averages in both conceptual physics learning and changes in PBs. One of these clusters, encompassing 35% of the participants, showed a positive average change in PBs. Another cluster including 23% of participants showed no change in PBs. This differential reaction of subgroups of students provides an explanation for why the effect size of the intervention on PBs was small.

Moving from the quantitative whole-class and subgroup analysis of participants to the higher resolution of analysis of individual’s epistemic practices, it was shown that some students used the course materials creatively, not for rejecting their paranormal beliefs but rather for supporting them. It seems that some students misused epistemic tools to defend their beliefs. These students could have developed alternative ideas about SEPs such as the burden of proof fallacy to confirm their PBs. Moreover, some students learned science models and theories of science. Then, they may have applied them to develop pseudoscientific explanation to support paranormal phenomena and used them to boost their confidence about the PBs.

The findings illustrate that targeted teaching strategies can generally diminish paranormal beliefs at the whole-class level. Yet, a more granular look into subgroups of students with different learning patterns reveals a wide range of responses to the intervention, the increase in paranormal beliefs for a cluster of students. With higher resolution to the investigation, qualitative analysis of individual student responses uncovers how the same instruction might develop new alternative ideas about epistemic practices and foster pseudoscientific explanations for paranormal phenomena, emphasizing the situated and contextual nature of epistemic practices (Elby et al., 2016). This three-tiered approach to data analysis was instrumental in deciphering how the intervention impacted both conceptual physics understanding and paranormal beliefs.

These insights serve a dual purpose. They encourage educators to infuse elements of epistemic and ontological training into the curriculum, thereby promoting an environment that nurtures the exploration of personal beliefs such as science alternative ideas and paranormal beliefs. Concurrently, they emphasize that three-level data collection can effectively generate feedbacks to enhance the effectiveness of the educational intervention. This approach allows the prediction of subgroups and individual student responses to the intervention and guides the development of differentiated instruction, thereby amplifying the effect size of the intervention and enhance student learning.

In addressing the persistence of paranormal beliefs within our education system, it is crucial to recognize the underlying value judgments that influence these beliefs. The commitment to scientific rigor and empirical evidence is not merely an epistemological choice but also a reflection of societal values that prioritize reasoned and evidence-based claims over unverified conclusions. By integrating epistemic and ontological training into science education, not only can students’ understanding of phenomena be transformed, but also the value placed on critical thinking and skepticism can be reinforced—essential tools for navigating a world widespread with misinformation. This approach underscores the crucial role of education in shaping not only knowledge but also the values that guide our interpretation and acceptance of that knowledge.

In conclusion, despite the 18th-century Enlightenment era and the Scientific Revolution, non-scientific modes of thinking continue to be deeply embedded in society. The widespread presence of paranormal beliefs, their linkage with science alternative ideas and misconceptions, and their association with the biases in intuitive core knowledge encourage their inclusion as a topic of consideration within science education. The investigation reveals that science instruction, strengthened by epistemic and ontological training, can be an effective tool to transform these beliefs. This model-based instruction underscores a three-step transformation for epistemic practices aiming to generate explanations for phenomena: Firstly, in these modes of knowing, students often intuitively form ideas, influenced by their core knowledge, and confirm these through plausible mechanisms to form a perceived truth. However, through ontological training, they can learn that confusions in intuitive core knowledge may lead to form ideas with insufficient or no exploratory power such as paranormal levitation. Secondly, through epistemic training, they are encouraged to imagine and form ideas intuitively, but should anchor their imagination within scientific theories and models. Finally, they must critically and analytically test the formed ideas against empirical evidence, shaping a tentative truth until better explanations emerge. By reinforcing these practices, not only can students’ scientific literacy be enhanced, but also the kind of informed, analytical thinking essential for individuals to effectively navigate complex issues in modern society can be nurtured. This approach highlights the value of critical thinking and empirical inquiry as foundational skills in a rapidly evolving world.