Explicitly Teaching Critical Thinking Skills in a History Course
Critical thinking skills are often assessed via student beliefs in non-scientific ways of thinking, (e.g, pseudoscience). Courses aimed at reducing such beliefs have been studied in the STEM fields with the most successful focusing on skeptical thinking. However, critical thinking is not unique to the sciences; it is crucial in the humanities and to historical thinking and analysis. We investigated the effects of a history course on epistemically unwarranted beliefs in two class sections. Beliefs were measured pre- and post-semester. Beliefs declined for history students compared to a control class and the effect was strongest for the honors section. This study provides evidence that a humanities education engenders critical thinking. Further, there may be individual differences in ability or preparedness in developing such skills, suggesting different foci for critical thinking coursework.
Many people in the USA and elsewhere hold beliefs for which there is no or can be no empirical evidence. Such beliefs range from astrology to psychic powers to not scientifically tested alternative medicines to conspiracy theories. Lobato et al. (2014) referred to these beliefs as epistemically unwarranted (henceforth “unwarranted”), meaning that they do not consider all of the evidence available to separate justified belief from opinion. This is the general definition for pseudoscience, thought it covers all domains where conclusions are not based on evidence. For example, the National Science Foundation reported that at least 42% of Americans believe astrology is “somewhat scientific” or “very scientific” and levels of these beliefs are increasing rather than decreasing (NSF 2014). The harm of these beliefs is real: unvaccinated children die, savings are spent on psychics, and in the first month of 2014 two children died during an “exorcism” (Karimi and Sutton 2014).
Less harmful though equally problematic aspects of these beliefs include the popularity of programs such as Ancient Aliens and misrepresentations of ancient cultures (e.g., the Maya in the movie Apocalypto) or popular scholarship about civilizations which do not take into account the ingenuity of ancient cultures. One example is Jared Diamond’s book Collapse: How Societies Choose to Fail or Succeed (2005), where Diamond emphasized natural resource exploitation and socio-political conflicts among the ancient Maya as societal “failures,” rather than bringing attention to their resilience during periods of social, political, and environmental stresses (e.g., centuries of successful agriculture in less than fertile soil (McAnany and Yoffee 2010). Misrepresentations in popular media and incomplete scholarly pictures of ancient civilizations have serious implications for the treatment of their descendants (Freidel 2007; McAnany and Negrón 2009; Pyburn 2006; Ren 2006). Further, unwarranted beliefs are related: belief in one predicts belief in others, suggesting that such beliefs represent a way of thinking (Lobato et al. 2014), even to the point where belief in one conspiracy theory positively predicts belief in other logically contradictory conspiracies (Wood et al. 2012). Educators need proven methods to teach the critical thinking skills needed to battle these unwarranted beliefs and replace them with reasoned beliefs supported by evidence (Smith and Siegel 2004).
Reduction of unwarranted beliefs has been a challenge, with a common supposition being that increasing education, particularly science education, will result in a public better equipped for critical thinking and analysis of unsupported claims. Critical thinking is a particular focus, defined as “purposeful, self-regulatory judgment which results in interpretation, analysis, evaluation, and inference, as well as explanation of the evidential, conceptual, methodological, criteriological or contextual considerations upon which that judgment is based” (Facione 1990, p. 2). An important further component in critical thinking is the acceptance of the results of critical thinking, rather than reverting back to previous beliefs, or at least an acknowledgement that the evidence supports some beliefs but not others (Smith and Siegel 2004). Unfortunately, knowledge of science is not a panacea for unwarranted beliefs with studies finding small or no differences for higher levels of science knowledge (Aarnio and Lindeman 2005; Bridgstock 2003; DeRobertis and Delaney 1993; Goode 2002; Johnson and Pigliucci 2004; Ryan et al. 2004) and some finding higher beliefs in the more highly educated (Rice 2003). Even teachers did not differ greatly from the general public (Losh and Nzekwe 2011). Studies with 1000s of archeology students across multiple universities found that students across decades and within the last few years showed belief in the existence of “ancient astronauts,” Atlantis, Big Foot, or a curse on King Tut’s tomb (Feder 1984, 1995, 2010; Harrold and Eve 1987). These studies have shown little change in students’ preconceptions about the past over several decades, finding consistently over 10% of students responding that they either have “no opinion” or “do not know” when asked about any pseudo-history or pseudo-archeological claim. This indicates that they feel ill-equipped to assess unwarranted claims. (Feder 2010). It may seem common sense that critical thinking will develop merely from exposure to collegiate courses, but unfortunately such is not the case (Arum and Roksa 2011; Pascarella et al. 2011). The conclusion was that it is not enough to teach science; how science is being taught and how to think (Paul 1995) needs to be revisited by explicitly defining critical thinking and expectations for students (Fitzgerald and Baird 2011).
Science courses created specifically to engender critical thinking have reduced pseudoscientific beliefs (e.g., Dougherty 2004; Franz and Green 2013; Gray 1985; Kane et al. 2010; Wesp and Montgomery 1998). However, as mentioned by Kane et al. (2010), many of these studies did not contain a control group, did not use appropriate statistical tests, or did not provide anonymity to respondents, meaning that the results could come from self-selection into the course or demand characteristics. Other evidence that demand characteristics did not affect the data would be selective belief change, for example, when some beliefs may decrease while others would not. This would indicate that the participants were not universally lowering their reported beliefs due to demand characteristics. However, prior studies did not investigate beliefs at this level (see Kane et al. 2010, for an exception). Studies of engendering critical thinking skills have shown similar results, with the most effective courses providing direct instruction on critical thinking along with class exercises that require critical thinking on materials from the course (Abrami et al. 2008; Niu et al. 2013).
Although most courses meant to teach critical thinking and reduce pseudoscientific and paranormal beliefs have been science courses, critical thinking is not the sole property of the sciences. Ryan et al. (2004) put it well by stating:
The liberal arts – literature, philosophy, history, art—and the natural sciences belong to the same intellectual tradition and as such, they have the same general goal … which is “to develop the faculties of the mind.” … Certainly the humanities invite reflection and personal experience as bases for decision-making; however, like the sciences, the humanities also require a rigorous approach to reasoning and reaching sound conclusions.
Indeed, students with previous courses in the physical and social sciences and humanities were less likely to believe certain problematic claims about the past (Feder 1984, 2010), but pseudoscientific beliefs remain pervasive.
In the current study, we tested the effectiveness of a history course focused on exposing “pseudo-history,” “pseudo-science,” and “pseudo-archeology” in reducing unwarranted beliefs when compared to a research methods course in psychology. Pseudo- science, history, and archeology can all be used pejoratively and acts of these often, though not always, involve sensational and outrageous claims about the past and/or the ways the world works. These practices can utilize existing facts but in ways that are inconsistent with standard disciplinary practices and may also simply exaggerate existing evidence to support false claims. Although it has been suggested that a pseudo-history course is a good way to address pseudoscientific beliefs (Allchin 2004) and that pseudoscience and pseudo-history overlap in their features (Boudry 2013), no empirical studies have been undertaken. We measured beliefs pre- and post-semester, informed students their responses would be anonymous and could not adversely affect their grade in the courses, took general measures of scientific knowledge, and a measure of hindsight bias at post-test. Last, the history course included an honors section, allowing us to investigate individual differences in belief change as well as a comparison of each history course to the control.
Participant demographics and pre-test scores
Year in college
Pew science score
n = 29
n = 33
n = 70
Number of majors
Colleges of science or engineering
College of humanities and social science
Science knowledge was assessed using a 13-item quiz developed by the Pew Research center (Appendix 1; Pew 2013). Items covered science in daily life and “textbook science” such as chemical reactions. Norming data for the Pew quiz reported that those with a college education averaged 9.8 correct out of 13 (Pew 2013). The Pew measure was not correlated to pre-test beliefs, and thus is used as a descriptor of the sample rather than a covariate in analyses.
The main measure of beliefs in pseudoscience and the paranormal (Appendix 2) was composed of statements across a variety of beliefs, adapted from belief scales used in similar experiments (e.g., Bridgstock 2003; Feder 2010; Kane et al. 2010; Tobacyk 2004). Each belief was rated on a seven-point Likert scale with verbal anchors of “(1) I do not believe this at all,” “(2) I doubt this very much,” “(3) I doubt this,” “(4) I am unsure,” “(5) I believe this,” “(6) I believe this very much,” and “(7) I strongly believe this.” Belief scores were calculated as the mean rating of each item. The difference between ratings for an item pre-semester and post-semester was the dependent measure used in analyses. Reliability of the belief test questions was assessed using the pre-test data with the resulting Cronbach’s alpha of 0.892, indicating good reliability.
2.3 Course Topics
The syllabus for the control course (Psychological Research Methods) is available at www.lacelab.org/PSY230Fall2014.pdf. The course covered research design and measurement. The syllabus for the history course (Frauds and Mysteries in History) is available at http://history.ncsu.edu/images/uploads/HI%20298-001_McGill%20F14%281%29.pdf. The course used the textbooks “Frauds, myths, and mysteries: science and pseudoscience in archaeology” (Feder 2010) and “From Stonehenge to Las Vegas: archaeology as popular culture” (Holtorf 2005) and covered such topics as the Piltdown Hoax and the building of the Egyptian pyramids. The Feder text presented an empirical, positivist, and critical approach to analyzing and debunking frauds, mysteries, and misconceptions about the past. The Holtorf text provided a more constructivist, post-processual archeology-based, and post-modern approach to understanding interests in the past in popular culture and the ways people connect with the past in the present.
Class time was devoted to analyzing how these different authors approached knowledge and misinformation about the past and the values of both of these approaches to understanding frauds and myths. The course contained direct instruction on critical thinking, e.g., including Feder’s “Quick Start Guide” (Feder 2010) and Carl Sagan’s “baloney detection kit” (Sagan 1996). These tools were then used throughout the course by the students, such as evaluating the likelihood of aliens being the source of ingenuity in ancient cultures using Sagan’s rule that “If there’s a chain of argument, every link in the chain must work (including the premise) – not just most of them.” Students also learned common logical fallacies and fallacies of rhetoric, tropes in historical myths, and then applied them to course topics, such as the “appeal to ignorance — the claim that whatever has not been proved false must be true, and vice versa.” Fallacies embedded in the frauds and mysteries of history were identified by the students throughout the semester to build their critical thinking skills. Students also practiced analysis, evaluation, critical thinking, rational skepticism, and were to be critical consumers of popular culture. Students honed their critical thinking skills through website analyses, research on historical frauds and misconceptions, and class debates about misuses and misconceptions about the past.
Not all of the examples that students were surveyed on were explicit examples of pseudo-history, pseudo-archeology, or pseudoscience. However, most of these are connected to these concepts in some way. For example, the concept of race as a biological construct is not an explicit example of pseudo-history, pseudo-archeology, or pseudoscience, but it is taught as a kind of social and historical “myth.” The course covered the history of racial science, the ways that scientific and historical information and concepts (e.g., connections between biology, race, and skin color; or connections between genes, skin color, and race) was and in some cases continues to be misused to be make arguments about biological racial differences and racial superiority and inferiority.
Topics in the belief measure were classified as to whether they were covered in the history course (Appendix 2). The honors section used the same texts but the course demands differed in that students were also assigned a group project to explore evidence behind a purported paranormal activity. For example, one group researched claims about the “Brown Mountain lights” that reportedly appear over Brown Mountain, NC, and presented to the class.
2.4 Design and Analytical Methods
Class Type (treatment: honors history, history, control: psychology) was a quasi-independent between-participant grouping variable. Belief Type (studied or unstudied in the history courses) was a within-participant variable. No belief topics were covered in the control course, but the topics were still analyzed by Belief Type. This was to check that there were no differential effects for the topics studied in history compared to those not studied. For example, it might have been that the studied topics in the history courses were beliefs most likely to change over time. If that were true, then we would see those beliefs change more for the control course as well. By dividing the topics into Belief Type for the control course, we would know whether such a confound affected the results: if a confound were present, we would see differences in belief change for the studied and unstudied topics, despite receiving no instruction on either. The dependent measure was their belief change (as measured on the seven-point Likert scale for all questions) from the pre-semester assessment to the post-semester assessment.
Analyses were performed using SPSS 18. Correlations were performed on continuous variables, such as pre-test scores on the various scales and demographic descriptors. The main analysis was a mixed-model ANOVA, with the between-participant factors of Class and the within-participant factor of Belief Type.
Survey responses were collected online, outside of the classroom, and by a researcher who was not the instructor for any of the courses. Participants were invited via emails sent to the course roster that contained a link to the pre-semester survey. Students who followed the link were presented with a consent form guaranteeing their answers would be confidential and they would receive two extra credit points on an exam for completing both surveys. Consenting students continued to the pre-semester survey that contained the demographic questionnaire, science knowledge test, beliefs in pseudoscience, and open-ended questions concerning their motivation for taking the course. Students who completed the pre-semester survey were emailed 1 week before their last class and given the link to the post-semester survey, which contained beliefs in pseudoscience, and open-ended questions about their experience in the course. The duration of each course was approximately 15 weeks.
Participants of different classes differed at pre-test on their Pew Science scores, where the honors history students scored significantly higher than the psychology course students (Table 1). All groups scored higher than the norm for college graduates (Pew 2013). There was no correlation between Pew Science score and pre-test overall belief scores (r2 = 0.02, p = 0.11, n = 117). Classes did not significantly differ on the belief measures at pre-test. An ANOVA revealed initial belief ratings differed according to biological sex, with female students reporting higher beliefs, F(1115) = 8.33, p < 0.001. ω2 = 0.07. Mean belief scores for each class were low and similar to Dougherty’s findings for college students (Dougherty 2004).
Because there were differences in male and female levels of belief at pre-test, a second mixed-model ANOVA was performed as above, adding the between-subjects variable of sex. There were no changes in significance from the previous analysis, no main effect of sex, and no interactions with sex (all ps >/05).
Unwarranted beliefs were low across all three classes at pre-test, similar to findings by Dougherty (2004), and scientific knowledge was high. Even such already low unwarranted beliefs were further significantly reduced after exposure to a critical thinking-based history course, but not to a control science course in research methods. Important findings included the differences in belief change for students in the two history classes, both receiving similar course materials. The honors students scored more highly on a test of science knowledge and trended toward less unwarranted beliefs at pre-test. These differences likely contributed to the honors class lowering their beliefs the most of any class at the post-semester assessment, both on topics covered by the course and topics not covered by the course. Further, the honors class most reduced their beliefs for topics explicitly covered in the class. In essence, the effectiveness of a course on critical thinking via history was most effective for those who were best prepared to absorb it and transfer it to other domains: honors students with some background in science. Those in the non-honors history course also reduced their beliefs, replicating previous findings regarding the effectiveness of skepticism-oriented courses (Gray 1985). These findings support the importance (and effectiveness) of explicit instruction (Abrami et al. 2008).
There is a long literature on the difficulties in transfer of knowledge, where even closely related information between subjects or courses often fails to transfer (Barnett and Ceci 2002). We find it promising that there was transfer of critical thinking to unstudied beliefs for students of both history courses, even if it was less for those in the non-honors course. Our findings via the belief measures in the study were bolstered by student comments on the post-semester survey that indicated application of critical thinking to other parts of their lives. Representative comments included “I think the most applicable things I learned from this class are guides on being a healthy skeptic. Being able to interpret current events and discoveries armed with Feder’s quick tips, Sagan’s baloney-detection kit, and with knowledge of credibility and the scientific method will allow me to make more informed decisions on what I believe to be true, as well as on what I need more evidence in order to make a more definite conclusion” and “… I want to dive into the details and I always look for poorly constructed arguments. In my other classes, I find myself challenging what my professors say when they use false dichotomies, circular arguments, or reference questionable sources....” Last, students referenced their chance to apply their critical thinking skills in class with comments such as “The course ... analyzed a lot of misconceptions of history and did a good job of providing the students with the ability to do this as well. The in-class activities were very helpful and were a hands-on way to learn the class material. I can easily say that I now have the skills and knowledge to properly assess historical information and form my own opinion on the reliability and credibility of the information.”
Although initial science knowledge did not protect against pseudoscientific beliefs, similarly to previous studies (Aarnio and Lindeman 2005; Bridgstock 2003; DeRobertis and Delaney 1993; Goode 2002; Johnson and Pigliucci 2004; Ryan et al. 2004), the honors history class did show the largest reduction in such beliefs due to the course, indicating that their higher science knowledge and cognitive abilities may have prepared them for the critical thinking introduced by the course.
Limitations to consider include the potential effect of instructor on belief change and the possibility that students responded to demand characteristics in the history class by reducing their belief ratings for the post-semester assessment. Because the control and treatment courses were taught by different people, it may also be that the history instructor was particularly engaging compared to the psychology instructor. However, because we found differences in studied beliefs compared to unstudied beliefs in the history courses, belief change was not entirely due to the instructor. We cannot rule out the possibility that the history students’ belief change was due to them thinking that their instructor expected that their beliefs should be lower by the end of the semester (and thus reporting lower than actual beliefs post-semester). However, we believe the anonymity of study participation reduced the chances of such demand characteristics driving the results. Further, if students were responding to demand characteristics, we would expect them to lower their beliefs similarly on all topics, rather than more for the studied beliefs than the unstudied. One last limitation was that the control and treatment groups differed in years of college experience, with the history classes averaging about 1 year older. However, we do not believe that this age difference was driving their beliefs, as groups had no differences in their pre-semester belief scores.
In conclusion, the current study has shown that critical thinking skills can be taught via a non-science course, resulting in a reduction in pseudoscientific and paranormal beliefs. This effect was strongest for beliefs directly addressed by the course, but transferred to other beliefs. Such generalization supports Davies’ (2013) supposition that critical thinking skills can be generalized beyond the specific discipline in which they are instructed and practiced. Individual differences in preparation and ability were related to belief reduction, where the most able students benefitted most from taking the history course.
We recommend that instructors in the sciences and the humanities explicitly identify the ways the class allows practice of critical thinking skills, define critical reflection, and integrate content-based instruction with critical thinking tasks. Such tasks offer the opportunity to reveal to students their own current beliefs while giving them a framework on which to alter those beliefs (Turgut 2011). Additionally, we recommend that other researchers consider the role of the humanities in the reduction of pseudoscientific thought, as suggested by Bunge (2010). Cross-disciplinary and collaborative scholarship on teaching and learning of critical thinking skills could also be particularly valuable.
Compliance with Ethical Standards
This research was approved by the North Carolina State University IRB and informed consent was obtained from all participants.
Conflict of Interest
The authors have no conflict of interest regarding this project.
- Arum, R., & Roksa, J. (2011). Academically adrift: limited learning on college campuses. Chicago: University of Chicago Press.Google Scholar
- Bridgstock, M. (2003). Paranormal beliefs among science students. Australasian Science, 24(4), 33–35.Google Scholar
- Bunge, M. (2010). Knowledge: genuine and bogus. Science & Education, 20(5–6), 411–438.Google Scholar
- DeRobertis, M. M., & Delaney, P. A. (1993). A survey of the attitudes of university students to astrology and astronomy. Journal of the Royal Astronomical Society of Canada, 87(1), 34–50.Google Scholar
- Diamond, J. (2005). Collapse: how societies choose to fail or succeed. New York: Penguin Group.Google Scholar
- Dougherty, M. J. (2004). Educating believers: research demonstrates that courses in skepticism can effectively decrease belief in the paranormal. Skeptic, 10(4), 31–35.Google Scholar
- Facione, P. A. (1990). Critical thinking: a statement of expert consensus for purposes of educational assessment and instruction. Millbrae: The California Academic Press.Google Scholar
- Feder, K. (1995). Ten years after: surveying misconceptions about the human past. Cultural Resource Management, 18(3), 10–14.Google Scholar
- Feder, K. (2010). Frauds, myths, and mysteries: science and pseudoscience in archaeology (7th ed.). New York: McGraw-Hill.Google Scholar
- Franz, T. M., & Green, K. H. (2013). The impact of an interdisciplinary learning community course on pseudoscientific reasoning in first-year science students. Journal of the Scholarship of Teaching and Learning, 13(5), 90–105.Google Scholar
- Freidel, D. (2007). Betraying the Maya. Archaeology Magazine, 60(2), 36–41.Google Scholar
- Goode, E. (2002). Education, scientific knowledge, and belief in the paranormal. Skeptical Inquirer, 26(1), 24–27.Google Scholar
- Harrold, F. B., & Eve, R. A. (Eds.). (1987). Cult archaeology and creationism: understanding pseudoscientific beliefs about the past. Iowa City: University of Iowa Press.Google Scholar
- Holtorf, C. (2005). From Stonehenge to Las Vegas: archaeology as popular culture. Walnut Creek, CA: AltaMira Press.Google Scholar
- Karimi, F., & Sutton, J. (2014). Maryland mom kills two of her children during attempted exorcism. CNN. http://www.cnn.com/2014/01/19/justice/maryland-exorcism-deaths/. Accessed 20 Feb.
- McAnany, P. A., & Negrón, T. G. (2009). Bellicose rulers and climatological peril? Retrofitting 21st century woes on 8th century Maya society. In In questioning collapse: human resilience, ecological vulnerability, and the aftermath of empire. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
- McAnany, P. A., & Yoffee, N. (Eds.) (2010). Questioning collapse: human resilience, ecological vulnerability, and the aftermath of empire. New York: Cambridge University Press.Google Scholar
- NSF. (2014). Chapter 7: Science and technology: public attitudes and understanding. In Science and Engineering Indicators 2014. National Science Foundation, 7–1–7-37.Google Scholar
- Paul, R. (1995). Critical thinking: how to prepare students for a rapidly changing world. Rohnert Park: Foundation for Critical Thinking.Google Scholar
- Pew (2013). Public’s knowledge of science and technology. Pew Research Center, Washington, D.C. (April). http://www.people-press.org/files/legacy-pdf/04-22-13%20Science%20knowledge%20Release.pdf. Accessed 23 Nov 2014.
- Ryan, T. J., Brown, J., Johnson, A., Sanburg, C., & Schildmeier, M. (2004). Science literacy and belief in the paranormal—an empirical test. Skeptic, 10(4), 12–13.Google Scholar
- Sagan, C. (1996). The demon-haunted world: science as a candle in the dark. New York: Ballantine Books.Google Scholar
- Tobacyk, J. (2004). A revised paranormal belief scale. International Journal of Transpersonal Studies, 23, 94–98.Google Scholar