Introduction

The goal of this chapter is to shed more light on the problem of epistemic inconsistency, that is often encountered when using questionnaires to investigate people’s epistemic beliefs about history. It presents the results of cognitive interviews performed with student teachers after they completed an adapted version of the BLTHQ (Beliefs about Teaching and Learning History Questionnaire).

Reflections about epistemology, and history as a discipline, are age-old, and have occurred since the dawn of academia. Interest towards epistemic beliefs regarding psychological and educational research, however, has particularly grown since the 1970s (Hofer & Bendixen, 2012; Hofer & Pintrich, 1997; VanSledright & Maggioni, 2016). Epistemic beliefs are widely recognized as bearing significant influence on metacognitive learning and teaching processes, and on levels of critical thinking in different domains of knowledge (Buehl & Alexander, 2001; Hofer & Bendixen, 2012; King & Kitchener, 2002; Kuhn, 1999; Stoel et al., 2022). On a larger scale, having societies of people with advanced epistemic beliefs is central to creating healthy democracies which are able to adapt and face both contemporary and future challenges (Garrett & Weeks, 2017).

This growing interest has been embodied by a large body of research, notably concerned with how to assess and map epistemic beliefs, through domain-general or domain-specific criteria and instruments in different disciplines, such as mathematics, science and history (Hofer & Bendixen, 2012; Hofer & Pintrich, 1997; VanSledright & Maggioni, 2016). Different methodologies have been used, but self-report instruments largely dominate the body of the studies conducted, often for practical and economic reasons (Muis et al., 2014).

Self-report instruments—i.e. questionnaires using scales in response to pre-set questions—are usually very valuable to gauge respondents’ thoughts, intentions, attitudes or beliefs. Nonetheless, they still face several issues, such as having little internal consistency and poor factor structure, and issues related to conceptual and theoretical challenges. Further issues include the complexity of epistemic beliefs “per se” for informants and researchers, the domain-specificity of epistemic beliefs and the influence of external factors related to the educational and national contexts (DeBacker et al., 2008; Hofer & Bendixen, 2012; Muis et al., 2014; Nitsche, 2019; Sakki & Pirttilä-Backman, 2019; Voet & De Wever, 2016; Wansink et al., 2016). A way of better understanding and overcoming these challenges is to conduct cognitive interviews with a limited number of participants (Karabenick et al., 2007), asking them to recall how they understood and answered the different questions. There is, however, a relative lack of such studies (Muis et al., 2014).

In history education, the different self-report instruments used to investigate epistemic beliefs—often adapting versions of Maggioni and colleagues’ BLTHQ and BHQ (Beliefs about History Questionnaire) (Maggioni, 2010; Maggioni et al., 2004; Maggioni et al., 2009a) to different languages and national contexts—have not been spared of these problems. One of the main issues identified is epistemic inconsistency, or “wobbling” (Maggioni, 2010; Stoel et al., 2022; VanSledright & Maggioni, 2016).

Epistemic inconsistency is not a new phenomenon; Perry (1970) already noted that interviewees could hold different positions according to the different sectors implied in his research (academic, religious, extracurricular, vocational or interpersonal); and King and Kitchener (2002) asserted that a certain variability of stage reasoning was actually normal. Within history education, epistemic inconsistency, or “wobbling”, has been brought to light by Maggioni and colleagues (Maggioni, 2010; Maggioni et al., 2010; Maggioni et al., 2009b; VanSledright & Maggioni, 2016). A significant number of students and teachers who filled out the BLTHQ or the BHQ agreed (or disagreed) simultaneously with (seemingly) contradictory statements. They “wobbled” between objectivist and subjectivist stances, sometimes between subjectivist and criterialist stances, or even between the three different stances. The same phenomenon of epistemic inconsistency has been identified in other studies (Elmersjö, 2022; Mierwald et al., 2017; Miguel-Revilla, 2022; Stoel et al., 2022; VanSledright & Reddy, 2014; Wagner & Dettweiler, 2024).

The explanations given for epistemic inconsistency are varied.

A general idea is that epistemic beliefs are complex ideas that require a certain degree of awareness to be properly verbalized. As such, they are generally only held consistently by experts. These ideas are most often too complex to be well-integrated, i.e. being well defined and understood, amongst non-experts, whether that be students or teachers (Elmersjö, 2022; Maggioni, 2010; Maggioni et al., 2010; Maggioni et al., 2009a; Mierwald & Junius, 2022).

Another explanation is due to the epistemic ambiguity of many individual items (Elmersjö, 2022; Maggioni, 2010; Mierwald & Junius, 2022; Stoel et al., 2017). Although self-report tools like the BLTHQ have been validated by experts, they fail to grasp non-experts’ epistemic beliefs, because some of their items are understood in different ways, and associated with other stances than the one they are intended to belong to. Moreover, this suggests that context plays an important role, either because informants related their beliefs to memories and experiences in their daily life (Mierwald & Junius, 2022; Muis et al., 2014); or because they responded according to what they thought their students needed to be taught, and not only according to their own epistemic beliefs (Elmersjö, 2022).

Mierwald and Junius (2022) identified two other explanations: one related to the length and comprehensibility of certain items; the other related to irritating references to the school context, an idea also put forward by Miguel-Revilla et al. (2020).

These explanations need further confirmation and investigation in other contexts. Although the investigation of Mierwald and Junius (2022) was informative, it was limited to just four German high school students responding to an adaptation of the BHQ.

Following their work, this article intends to present the results of cognitive interviews we conducted with history student teachers to validate a Norwegian version of the BLTHQ (Maggioni et al., 2004; Maggioni et al., 2009a) that we used in a wider study (Wagner & Dettweiler, 2024). Overall, we hope to contribute to the body of case studies investigating epistemic beliefs about history in different contexts, and we hope to further clarify the problem of epistemic inconsistency.

Method

We recruited eight respondents (three males, four females, one other) from 176 Norwegian history student teachers that had answered our Norwegian version of Maggioni’s BLTHQ (Wagner & Dettweiler, 2024). They volunteered, just after completing the questionnaire, to participate in an individual interview that would investigate their understanding of the questionnaire.

Our BLTHQ in Norwegian is a translated and slightly adapted version of the original 21-item questionnaire, respectively split between an objectivist, a subjectivist and a criterialist stance (see Table 18.2). The order of the different items was randomly set (see each statement’s number) and the respondents answered following a six-point Likert scale system (1 = Strongly disagree; 2 = Disagree; 3 = Somewhat disagree; 4 = Somewhat agree; 5 = Agree; 6 = Strongly agree). Moreover, we added a final question to identify which stance each informant felt closest to. Through an exploratory factor analysis based on the 176 respondents (Wagner & Dettweiler, 2024), we managed to extract a three-factor solution, where each factor clustered items from the same stance and showed acceptable levels of reliability. Factor 1 gathered criterialist items (α = 0.78), while Factor 2 gathered objectivist items (α = 0.74) and the less explaining Factor 3 cumulated the subjectivist items (α = 0.67). However, the three factors stood only for a total of 32.4% of the variance and the p value of the Chi-squared test was significant (p < 0.001), which indicates that the model does not fit optimally. Further, items 6, 8 and 21 had loadings below 0.400.

The individual interviews were performed within a week and lasted an average of 56 minutes (the shortest at 32 minutes, and the longest at 72 min). All interviews were audio-recorded and later transcribed. To begin with, participants were told that the main aim of the interview was to ensure the quality of the questionnaire for further studies, by investigating in greater depth whether each statement was clearly understandable, and how they understood them. They were asked the same questions about each statement: Was it clear and easy to understand? Was it easy to answer? And how did you interpret it? Did you understand the intention behind this statement, what it is aiming at? Each interview followed the same semi-open structure, with the respondents invited to develop their thoughts, if necessary, to clarify their (epistemic) understanding of each statement.

We performed a content analysis of each interview, focusing on the student teachers’ comments about each statement, but also drew on principles of discourse analysis (Gee, 2014) to interpret how each informant positioned him/herself towards the different epistemic stances. Following Mierwald and Junius (2022), we reduced their seven categories to six categories to code the informants’ problems of understanding for each statement (Table 18.1).

Table 18.1 Coding categories for the interviews
Full size table

Validity, Reliability and Ethical Rules

Our sample is a convenience sample that is both geographically and genderly with four women and four men studying at the same university in Norway. However, it is limited to history student teachers in their first semester of study, who volunteered to be interviewed, which may constitute bias regarding their prior interest in History and their self-confidence. Furthermore, our study is qualitative and thereby not representative and generalizable, but performed on a sample that is large enough to reflect the problems of understanding generally faced by Norwegian history student teachers with the BLTHQ.

Although we are aware of the risk that participants might have been unwilling to openly display “too much” lack of understanding in front of an interviewer that they may consider to be an expert/authority, we are confident that the openness and the face-to-face setup of the interviews have given us a fair picture of their thoughts.

We complied with the formal ethical rules of confidentiality and personal data protection endorsed by the Norwegian Agency for Shared Services in Education and Research, and the respondents mentioned in this paper are anonymized. We randomly named them Camilla, Erik, Esteban, Kim, Malin, Martin, Mary and Siri.

Findings

Most Problematic Statements vs. Less or Unproblematic Statements

As shown in Table 18.2, all participants experienced problems of understanding with the questionnaire. However, five statements were entirely unproblematic (i.e., no student teacher experienced problem with the statement), eight statements elicited difficulties for few respondents (one or two respondents). Eight statements elicited difficulties for many respondents (three or more respondents) and cumulated the highest number of problems. Furthermore, we noted that the subjectivist stance comprised most of these problematic items for more than two respondents (6 items), while the objectivist stance had two (items 6 and 15) and the criterialist statements had none.

Table 18.2 Number and nature of the problems experienced by student teachers with the BLTHQ in Norwegian
Full size table

Most problems were due to misunderstanding the intended meaning of the statements, followed by difficulties to choose an answer (reply format and uncertainty of response), and general misunderstandings. Going forwards, we will investigate the problems associated with the eight most problematic items, following their order of appearance in Table 18.2: the subjectivist items 1, 5, 7, 17, 18, 21 and the objectivist items 6 and 15.

The Subjectivist Items

Item 1—“Students Who Read Many History Books Learn That the Past Is What the Historian Makes It To Be”

For Malin, Esteban and Siri, the main problem was that they interpreted the statement from the students’ point of view and partially oversaw that at its core, the statement was about what historians do when they write history, and not about what students do. They cannot really be blamed for this understanding, as the subject of the statement is “students” and not “the historian”. As such, we can gather that the two-part structure of the sentence has been confusing; a more direct statement would have been easier to answer.

As a result, Malin overly focused on “Students who read many history books”; she stressed that it was ambiguous since it depended on how students were reading these books. Esteban was also confused by the first part of the statement but still understood the intended meaning of the statement. He did, however, express doubts about the reply format. As for Siri, the sentence led her to (mis)understand the statement as exclusively centred on the students’ point of view, as if it meant that reading many books is positive because it offers students a multiplicity of different perspectives on a historical topic.

Item 5—“Students Need to Be Aware That History Is Essentially a Matter of Interpretation”

This statement was hugely illustrative of “wobbling” due to epistemic ambiguity. Five student teachers read it as a criterialist item, and not a subjectivist one. As Martin put it, in dialogue with the interviewer:

I understand this question as… how we look at history today and what we think about the past is something that is interpreted, it is something we have discussed and agreed upon. Because, there is a lot we don’t know. There are many holes that are difficult to know about. So, we must discuss to try to find out how to fill these holes with the sources we have

  • But, in the way the question is worded, would you interpret it as if this was about opinions?

  • No, not necessarily opinions, rather interpretations, that is how people discuss and make sense of what is most probable.

  • So, you see history as something that is…

  • Worked with and discussed.

The same kind of epistemic ambiguity was expressed by Camilla: “Interpretation will be how you personally see it, but also how historians interpret it”.

For both respondents, in this statement, interpretation is interpreted as a well-grounded understanding constructed by an academic community of historians according to different sources and perspectives.

As such, we must hold that the epistemic ambiguity here is due to the wording: The item is not formulated precisely enough to belong exclusively to the subjectivist scale, and can be read as a criterialist item.

Item 7—“In Reading a History Book, It Is More Important to Pay Attention to the Perspective of the Historian Than to His or Her Reasoning on the Evidence Discussed”

Interestingly, the student teachers’ reactions to this statement were unambiguous. They saw both parts of the sentence as interdependent, or closely connected, and difficult to separate, which made it tricky to answer. Thanks to Siri’s explanation, we understand that the historian’s subjectivity is seen as an inevitable problem:

It’s two different things, but they are depending on each other. If a historian has a perspectivePerspective X, he will construct his reasoning (…) around it. […] They are two different things, but they work together.

Or, like Erik, some participants asserted and defended their refusal to see perspective and reasoning as different elements, due to personal ways of thinking. It was also a sign of poorly integrated epistemic beliefs:

I think they are both important. I don’t think… For me, it’s not relevant if one thing is… if it is more important to summarize than compare, or if perspectivePerspective is more important than the reasoning. I don’t think that way. I feel it is unnecessary to think like this, actually. Both are important to get the whole picture, you see?

Mary had a similar attitude. She upheld that she could not/would not choose between the historian’s perspective and how s/he built his/her reasoning, because she considered both parts to be equally important. To her, therefore, to choose between them would be wrong. Further, when confronted with the fact that one part of the sentence could hold history as subjective, while the other part could represent a more reflective/interpretative way of thinking about history, she confirmed her own wavering position by replying: “History is both, isn’t it?”

On one hand, we can categorize her answer as within the criterialist position, considering that historians seek the most likely interpretation, whilst well-aware that they still have their own subjectivity to deal with. On the other hand, like Erik, her unwillingness to choose between the two terms is also a clear sign that her epistemic belief on this was not well established. This demonstrates and underscores the difficulty of establishing straightforward, and unambiguous statements for all respondents.

Item 17—“Good Students Know That History Is Basically a Matter of Opinion”

With statement 17, we observed two different problems.

The first one concerned the (school) reference to “good students”, which was considered somewhat misleading, and better to avoid. Kim wondered whether the statement was about the nature of history, or about the nature of “good” students. And good students according to whom? In which context?

Because, good pupils, is it…? (he hesitates). Does it mean that heedful students see that history is often very subjective? Or does it mean that good pupils are subjective? Because it can be unclear (…) Then, one version is that subjectivity is good, while the other means that subjectivity is bad.

The other problem was concerned with the epistemic ambiguity of the sentence, and it was declined in two different versions.

For example, for Martin and Malin, their own preconceptions affected their understandings of the statement. They read the item through the lens of their own criterialist understanding of history. Malin interpreted it as: “Good students understand that history is the product of different subjective sources and interpretations”, while Martin explained the item as follows:

There are lots of sources; History as a whole will be how we interpret them. It will influence points of view, but it’s not only points of view […] because history is how we’ve agreed to use the sources, and it’s often stuff that has been discussed a lot… that must be approved by other scholars to be accepted.

Esteban, on the other hand, demonstrated a meta-understanding of the questionnaire, and described the statement as epistemically ambiguous, since it would be answered similarly by people with different understandings of history. Demonstrating that the questionnaire was meant to spot three different groups, he clearly identified what he called “interpretive people” and “subjective people” as groups that would both agree with the item:

It is mostly the interpretive people that come forward here, I feel, because different points of view are taken into account in a big stew, and then maybe the truth will rise from the steam. But also the subjective people, they are into points of view, them too; and every individual point of view, that’s what matters in their interpretation of history.

Item 18—“History Should Be Taught Like A Story: Some Things Are True, But Some Others Are Just a Matter of Personal Opinion”

With this we noted both the problem of epistemic ambiguity, and with the length of the statement.

For Siri and Malin, the (seemingly) opposed nature of the two parts of the sentence made it paradoxical and difficult to answer. Malin wondered how one could simultaneously argue that “facts are truth” and that “things depend on perspectives”. Siri held that “what is based on the sources” is truth, whilst personal opinions were “what they have imagined”—something separate from the sources. Siri and Malin’s preconceptions of the nature of history appeared to impede and influence their understanding of this (subjectivist) item.

On the other hand, Camilla, Martin and Esteban did not see the item as ambiguous, but interpreted it as a criterialist one, and then agreed with it. They looked past the adjective in “personal opinion” or understood “other things” as different sources that had to be assessed and discussed between historians.

Item 21—“Teachers Need to Make All Historical Interpretations Available and Let the Students Construct Their Own Understanding of Them”

Again, the same misinterpretation of the item led most student teachers to read the statement as a criterialist one. They read it according to their own preconceptions about knowledge in history—as an interpretation of sources and theories that are discussed within a community of inquiry. They did not understand it as if the students were “left alone” to construct their own (undiscussed) understanding.

On the other hand, Mary felt that the two parts of the sentence were contradictory and made it tricky to answer. As she agreed with the first part and disagreed with the second part, she could select both “partly agree” and “partly disagree” for this statement.

The Objectivist Items

Item 6—“Good General Reading and Comprehension Skills Are Enough to Learn History Well”

The problems encountered regarding this statement, which is meant to represent an objectivist stance, were interesting, as they were very varied.

A common problem has been that respondents usually ignored the “general” nature of the skills referred to in the statement (as opposed to “specific” or “disciplinary”). Some student teachers also focused on “reading” and neglected the “comprehension” part. As such, Siri, Erik and Martin saw reading as an important part of an historian’s job and a “basic” skill that would give access to a certain level of understanding. As Siri put it:

As we were told on the first day [at university], history is a discipline where one needs to read a lot. So, it’s an advantage to be able to read well, in a sense, (…) it helps to be able to read, in history, when there are so many written sources.

In contrast, for Kim, the problem was to do with the notion of “comprehension”, which he deemed too vague: Comprehension of what? Facts or more complex things? And how much/what is “enough” to learn “well”, he wondered. As such, he struggled to answer the question.

And finally, for Malin, she misunderstood it in its entirety, because, in addition to these two problems, she understood “general reading and comprehension skills” as contrary to cognitive reading and learning difficulties, like dyslexia. She understood the statement as suggesting that not having any cognitive difficulties to read or understand was an advantage to learn history well.

Item 15—“Corroborating Evidence and Identifying Sources Are Important Learning Strategies in History, But Only After Mastering the Basic Facts”

Here, again, as with item 7, the general issue was the difficulty to prioritize one of two skills that student teachers felt were equally important. As a result, Mary, Erik, Camilla said they were uncertain about how to answer, and that they would have chosen either “partly agree” or “partly disagree”. Obviously, this is a case of poorly integrated epistemic beliefs; the informants did not have a clear idea of what they would or should prioritize and preferred not to.

On the other hand, Martin demonstrated that the statement could also be “misunderstood” by a criterialist, which is the stance we place him within. He read the hierarchy between the two elements as chronological and not in terms of importance. His answer was:

This is not the easiest question to answer, it is not, because you need some basic knowledge in order to build a good argument and to know what to find in sources. Because, if you don’t know who Napoleon was and what kind of reforms he performed, then it will be difficult to search sources about these reforms or the codes he issued and stuff like that. […] So, I would answer “partly agree”. It is important to master basic facts, but you need also to be able to check sources, and identify and verify them.

Discussion and Conclusion

Our goal was to untangle the problem of respondents’ epistemic inconsistency when using a self-report questionnaire like the BLTHQ. We confirmed the explanations from prior studies and distinguished three main sources of problems: an intrinsic epistemic inconsistency; an extrinsic epistemic inconsistency; and respondents’ own uncertainty about their epistemic beliefs.

However, our results are different to those found by Mierwald and Junius (2022), and are difficult to compare, mainly because of two variables: Mierwald and Junius investigated the understanding of the BHQ, while we tested the BLTHQ; and our respective samples of respondents are qualitatively very different: they selected four German high school students, while we interviewed eight Norwegian university student teachers in their first year of history, i.e. more informants and potentially more problems regarding understanding, and informants with a supposed notable interest in the discipline. Further, we had no way of choosing in advance to interview student teachers that had experienced problems with understanding or demonstrated epistemic inconsistencies. It would have been more appropriate to directly interview selected student teachers, but it would have been difficult because of resources and in terms of data protection in Norway (since it would have made the respondents identifiable). We did not apply for this opportunity when we designed our study.

Mierwald and Junius’s (2022) study had a lower proportion of problematic items, and these were evenly split between the three stances, whilst in our study, criterialist items were unproblematic; the problematic statements were mostly the subjectivist items, along with two objectivist items. Still, considering how the BHQ and the BLTHQ share a number of similarities, this confirms that national context and the profile of the participants play an important role in the assessment of epistemic beliefs.

Moreover, our investigation corroborates the explanations given for epistemic inconsistency in prior studies. However, it helped deepen the understanding of two of them: the epistemic ambiguity of certain statements; and the fact that some of the participants’ epistemic beliefs are not well-integrated (VanSledright & Maggioni, 2016), i.e. well-defined.

The problem of the items’ epistemic ambiguity seems twofold: an intrinsic and an extrinsic/perceived epistemic ambiguity.

On one hand, the intrinsic epistemic ambiguity is due to an ambiguous formulation of the problematic statements. The statement is ambiguous “per se” and can be answered similarly by proponents of different stances. In other words, the item was not formulated precisely enough to be clearly understood and associated with the stance it was meant to represent. As shown above, this is the case for items 5, 17, 21. In these cases, the wobbling is not due to the informant’s own inconsistency. The solution is to further clarify these statements by improving their wording and by sharpening their association with the intended stance.

On the other hand, the extrinsic or perceived epistemic ambiguity is due to the participant’s misunderstanding of the item. In some cases (items 1, 7, 18 or 21), the item is a longer sentence composed of two propositions or of different elements. Some informants scrutinized each proposition/element separately, instead of considering the meaning of the sentence as a whole. Alternatively, they focused mainly on one part or certain elements of the statement. It seems that introducing too many subtleties and nuances in the statement is likely to invite different interpretations and to divert the respondents from the main meaning of the item. As a result, they grow indecisive, potentially inciting “wobbling”. Parts of statement that are not strictly necessary and related to the item’s core meaning should be removed. For example, statement 1 should be shortened from “Students who read many history books learn the past is what the historian makes it to be” to “In history books, the past is what the historian makes it to be”.

In addition, our examples also illustrate the great variety of possible interpretations and misunderstandings due to the inherent polysemy of language (item 6).

Further, some student teachers’ interpretations of certain items are biased because of their own epistemic beliefs, especially if these items are epistemically close. As shown with statements 17, 18 and 5, student teachers who are more criterialist tend to interpret subjectivist items as criterialist. They (mis)understood the statement as fitting their own epistemic beliefs about history and, as a result, answered somewhat inconsistently.

The issue of wobbling due to extrinsic epistemic ambiguity is trickier and has no clear solution. In fact, it confronts the respondent’s own subjectivity and potential misinterpretations of a questionnaire. There are potential solutions, however, such as reformulating the statement, avoiding lengthy sentences and statements beginning with a negation, or containing a double negation, as advocated by Mierwald and Junius (2022). That said, the variation in focus and understanding of the same statement by different individuals, as shown in some examples above, will probably always be a variable to deal with.

Finally, in many cases, our results support that student teachers wobble because their own epistemic beliefs about history are not well-integrated. In several cases (items 7, 15 and 18), they did not have a firm grip of what they believed. This issue is difficult because there seems that there is little to do about it. One could add an “I don’t know” alternative to the range of answers. But that offers an easy way out each time the respondents experience uncertainty or discomfort about a question. These answers will still be difficult to interpret and might also distort the results. Alternatively, it may be worth increasing the number of items dedicated to each stance, which may reinforce the internal consistency of the different factors.

In a wider perspective, our study confirms that assessing epistemic beliefs about history deals with complex ideas that require a high level of expertise, awareness and steadiness about one’s own beliefs, to be answered reliably. Obviously, many young participants cannot be expected to have this level of awareness and understanding. If the questionnaire is made to be reliably answered only by experts that comprehend it fully, then we face a precarious task. Further, one could argue that the developmental hierarchy between the three stances potentially questions the validity of the answers of respondents belonging to the “lower stances”, which here are the objectivist and subjectivist stances. In other words, how can we be sure that making the informants aware of statements that they would otherwise not have expressed by themselves may not make their answers to these statements biased? Certain subjectivist and criterialist items may sound wise and easy to agree with, even for an informant who is an objectivist.

The implications of our findings and reflections are multiple.

Firstly, one may still wonder if trying to synthesize such complex ideas into one general stance (out of three, or more) is the right way to go. Defining/Breaking historical thinking into several tasks/dimensions and examining the different ideas about each of them can form a more precise and reliable tool with which to evaluate people’s epistemic beliefs. However, a longer test, including various tasks and dimensions, will raise new challenges and require more time and resources. Further, evaluating how a student teacher solved these tasks will not be easy either. The whole test will still face reliability issues.

Secondly, if one still defends the intentions of establishing a reliable quantitative tool to assess epistemic beliefs about history, an obvious option is to try out a questionnaire with more and sharper formulated items for each stance, in order to increase the validity of the tool. Another possibility is to ask the respondents to additionally hierarchize the different items according to importance. The different items will be weighted according to their given importance, which will align with the developmental hierarchy between the three stances and potentially mitigate the problem of wobbling.

A final alternative is to accept the fact that such a questionnaire, in all its imperfection, is just an approximative tool with which to gauge complex epistemic beliefs, which must be treated as such: a rough indicator, more suited to assess student teachers’ progress over time than an accurate measurement of people’s epistemic beliefs.