Background

Despite an overwhelming acceptance among scientists and science education organizations of the veracity of evolution, its centrality for understanding all of biology, and its power in unifying the sciences, only about 20% of high school students, 52% of college graduates, and 65% of postgraduates accept evolution as a scientific theory well supported by evidence (Brumfield 2005). Emerging research suggests that even the very youngest students can understand basic evolutionary concepts (see review in Pobiner 2016), yet many studies suggest that students struggle to develop an understanding of evolution even when the content is part of a science course (e.g. Smith 2010a, b). It therefore seems likely that both cognitive and non-cognitive factors play an important role in this phenomenon. Indeed, there is a diverse array of affective, cognitive, cultural, epistemological, pedagogical, political, religious, and social factors that contribute to a reluctance to recognize evolution as an essential organizing principle of the natural world, or even outright rejection of the scientific evidence for evolution (e.g. Allmon 2011; Nehm and Schonfeld 2007; Smith 2010a, b; Thagard and Findlay 2010).

Research strongly suggests that students who hold cultural and religious beliefs that preclude acceptance of biological evolution are unlikely to learn about evolution until these issues are addressed—and that doing so explicitly can be more effective in changing attitudes towards evolution than ignoring them entirely (Smith 2010b; Verhey 2005). Acknowledging the cultural or religious controversy that may exist for some students in a classroom and fostering positive dialogue about it, coupled with emphasizing the nature of science, may provide teachable moments about why a difference exists between acceptance of evolution among scientists and among the public. This approach may be one of the most effective instructional methods for teaching evolution in places where it is a socially controversial issue (Andersson and Wallin 2006; Hermann 2008).

There is perhaps no more direct way to address cultural conflicts than by using examples of human evolution because reluctance to accept evolution is often due to a conflict between cultural, religious, or social beliefs about what it means to be human and an evolutionary perspective on the same question. Although it seems counterintuitive, we argue that a pedagogical focus on examples from human evolution may provide an enjoyable, engaging, and effective approach to helping students overcome their reluctance to study the concepts, more fully consider and understand the evidence for evolution, and accept evolution as a scientifically valid and meaningful tool in the study of biology. Even students who see a conflict between their beliefs and accepting evolution are likely to be at least curious about the evidence for human evolution because it is a highly personal context for learning scientific concepts. Helping students make connections between the subject matter they are learning and personal experiences or “real-world” examples can result in deeper learning of many subject domains (e.g., National Research Council 2009). Incorporating compelling examples of practical applications of evolution that are relevant to students’ lives and familiar social issues may increase their motivation to learn and retain evolutionary concepts (Beardsley 2011; Hillis 2007; Scharmann 1990; Thanukos 2010). Adolescents are keenly interested in themselves and in their own development, so there may be no more relevant examples to use than those from human evolution (Pobiner 2012, 2016). There are also compelling pedagogical reasons to teach evolution in the context of humans. Using human examples to teach evolutionary concepts may be beneficial because people can see variation from one person to another more easily than variation among animals (Nettle 2010), and students who appreciate the extent of individual-level variability are more likely to have a correct mechanistic grasp of natural selection (Shtulman and Schulz 2008).

Data suggest using human examples to teach evolution can be effective in college classes for both biology and non-biology majors (Nettle 2010; Paz-y-Mino and Espinosa 2009; Wilson 2005), but prior to this project, this approach had not been the primary focus of investigations of student learning of evolution in advanced high school biology classes. One reason for this is that cultural barriers to understanding evolution are rarely discussed in teacher preparation or in-service programs, nor are teachers provided with strategies they could use to overcome these obstacles (Alters and Alters 2001; Branch et al. 2010). In order to use human examples, in particular, to teach evolution, teachers need resources to help them address potential conflicts and challenges that arise from cultural barriers to teaching and learning about evolution.

The Teaching Evolution through Human Examples project

The goal of the Teaching Evolution through Human Examples (TEtHE) project was to develop and field test (a) four mini-units (curriculum supplements) that use case studies of human evolution to address specific core evolutionary concepts included in the high school A.P. biology curriculum, and (b) a cultural and religious sensitivity (CRS) teaching strategies resource that provides teachers with instructional strategies to address potential classroom conflicts and challenges related to teaching evolution, including two classroom activities that can be integrated into the mini-units. The project focused on advanced placement (A.P.) biology because of the centrality of evolution in its curriculum after it underwent a major revision based on recommendations from reports from the National Research Council (2002a, b). Evolution is the first of four “big ideas” in the new A.P. biology course which was introduced in the 2012–2013 school year. In addition, A.P. biology classrooms provided a “best case” learning context for the formative evaluation of the field-test version of the mini-units because A.P. students are generally more motivated to learn and are more sufficiently aware of the impact of teaching materials on their own learning which together suggest that these students will provide feedback of value for the design of usable and feasible materials.

The main questions guiding the research and evaluation were:

  1. 1.

    To what extent can the project team develop a set of human evolution-centered curriculum mini-units that align with A.P. biology learning objectives, are scientifically rigorous and accurate, and are relevant to students?

  2. 2.

    To what extent can the project team develop a set of cultural and religious sensitivity (CRS) resources that provide teachers with strategies that create a supportive classroom environment for the teaching of evolution and support an understanding of the nature of science?

  3. 3.

    To what extent does the use of the curriculum mini-units alone, and the curriculum mini-units used in conjunction with the CRS activities, affect student understanding of evolutionary concepts and their acceptance of evolution?

Methods

Research framework

The TEtHE project was conducted within a design-based research framework. Design-based research is situated in a real educational context and involves a collaborative partnership between researchers and practitioners, focuses on the design and testing of a significant intervention, uses mixed methods to explore research questions, involves multiple iterations, and is intended to have a practical impact on practice (Anderson and Shattuck 2012). This framework provides a backdrop for understanding the processes involved in addressing the first two research questions and a perspective for considering how the third research question might suggest whether future research on the materials’ effectiveness using experimental or quasi-experimental designs is warranted.

Mini-unit development process

We used a curriculum development process based on understanding by design (Wiggins and McTighe 2005) that involved an advisory board with five informal science educators, four expert teachers, four scientists, and three science education researchers, and an internal writing team (which was led by Beardsley). An advisory board meeting was convened at the beginning of the development process to identify seven content and science practice criteria to guide the curriculum development process (Table 1), key examples for the case studies for the mini-units, and the content areas for the case studies (Table 2). The group then prioritized the criteria and content areas from among the research-based best practices the board discussed. The criteria agreed on by the advisory board for deciding which concepts and topics to incorporate included: (1) evolutionary topics for which there are teacher misconceptions (summarized in Nehm and Schonfeld 2007) because students’ knowledge structure about evolution tends to reflect that of their teachers (e.g. Moore and Cotner 2009), as well as topics for which student misconceptions are common (summarized in Smith 2010b); (2) topics identified by teachers in the Understanding Evolution website evaluation as those with the highest needs for content, which include evidence for evolution, mutation, extinction, cladistics and phylogenetics (Scotchmoor and Thanukos 2007); (3) genetics, a poor understanding of which was identified by Miller et al. (2006) as one of the three main causes of low acceptance of evolution among Americans; and (4) the then-new A.P. biology curriculum’s enduring understandings (i.e., major concepts) and specific learning objectives (i.e., what students should be able to do at the end of an A.P. biology course) for the evolution “big idea”, which includes the seven science practices that A.P. biology students should be able to apply (Wood 2009). The advisory board agreed that potential content areas to focus on could include: (1) the human fossil record, which would capitalize on many students’ fascination with fossils; (2) morphological and behavioral diversity of primates, including humans as a part of the primate family, to underscore our connectedness with the tree of life; (3) recent evolutionary trends in humans (such as modern human variation, migration, and adaptation) to pique students’ interest in their geographic origins and variation; (4) examples of evolution that focus on current topics and everyday experience (such as health, disease, and medicine; agricultural practices; and using evolutionary analyses in forensic science to detect crimes or catch criminals) to highlight the relevance and applicability of evolutionary theory (McKeachie et al. 2002; Sinatra et al. 2008) and the idea that humans are still evolving (Shields 2004; Andrews et al. 2011); and (5) common misconceptions about evolution and experiences to help students confront their misconceptions. The materials would be built on the paired misconceptions and corrective responses in the educator guide developed for the Hall of Human Origins (Smithsonian Human Origins Program 2010a) and other common issues raised by antievolutionist students, parents, and others (Smith 2010b).

Table 1 Curriculum mini-unit content and pedagogy criteria
Full size table
Table 2 Short descriptions of each curriculum mini-unit
Full size table

The curriculum developers first developed a detailed outline for each lesson describing the content and pedagogical approach. This outline was evaluated by project advisors using the content and pedagogical criteria with each criterion (Table 1) being scored as “Yes”, “Yes, but (insert comment)”, “No”, or “Unsure”. Feedback was used to develop a draft of each mini-unit which was then pilot tested (see below). Feedback from a small initial group of field test teachers, students, and the advisory board led to revisions for a second draft of each mini-unit which was used in the field test. The final version of the mini-units, which are briefly described in Table 2, include (a) a brief overview of lessons (4–5 per mini-unit), (b) a summary of A.P. biology essential knowledge and learning objectives, (c) a suggested timeline of 5–9 days depending on the mini-unit use of the full or condensed version, (d) a master materials list including handouts and overheads, and (e) a summary of each lesson that includes a brief description, objectives, teacher preparation (materials/handouts and preparation instructions), and the procedure. The materials include a teacher guide, teacher PowerPoints, and student workbooks. The curriculum mini-units are freely available on the Smithsonian’s Human Origins Program website (Smithsonian Human Origins Program 2010b).

CRS resource development process

The project advisory board meeting determined the CRS resource structure, content, and focus, and determined the criteria to guide the CRS resource development process (Table 3). The advisory board agreed that the resource should include a variety of strategies for teachers, providing both background information and classroom activities. The goals of the CRS resource are to (1) encourage and equip high school teachers to help students manage any tension they may experience between a scientific study of evolution and their religious and cultural beliefs and (2) create a classroom environment that supports an increased scientific understanding of evolution. The approach is to help teachers proactively acknowledge and manage cultural and religious controversies with directed discussions, or if the teachers choose not to be proactive, to help them manage these controversies should they arise in the classroom. Its use was optional for teachers during the pilot and field tests. The intention is not for the teacher to specifically resolve any conflict the student may see between their personal worldview and the scientific account of human origins, but to help create a non-threatening classroom environment.

Table 3 CRS resource criteria
Full size table

The CRS resource is designed to provide background information on the nature of science as pertinent to managing a conflict between science and cultural or religious beliefs; the range of creationists views; the variety of possible relationships between science and religion, including examples of how individuals accommodate evolution and religion; and the historical context and background on legal cases dealing with the teaching of evolution. For teachers seeking a proactive approach that acknowledges students’ cultural and religious conflicts with evolution and encourages a classroom exploration of the impact of these conflicts on the understanding of evolution, the resource also provides two activities to engage students in two 50–75 min directed classroom discussions (Table 4). The classroom activities use a procedural neutrality approach to teaching controversial topics (Hermann 2008) in which information about the cultural conflict surrounding evolution and different points of view about this conflict are elicited from students and from resource materials. The teacher does not make a value judgement about these views, but helps students come to a correct understanding of the nature of science. Classroom Activity 1 is designed to be used at the start of instruction on evolutionary theory, in classrooms where teachers are aware that many of their students have been exposed to only negative and/or mistaken notions of evolutionary theory. Classroom Activity 2 is designed to be used at the end of instruction on evolution and to reinforce that instruction in classrooms where teachers believe that an anti-evolutionism is nonexistent or a minority viewpoint.

Table 4 Short descriptions of each CRS resource classroom activity
Full size table

The CRS author (Bertka) developed a first draft of the CRS resource; feedback from the advisory board led to revisions for a second draft of the resource which was used in the field test. Feedback from field test teachers and students was used to create a final version, as well as accompanying PowerPoint files, for each of the classroom activities. Results from the field testing of the CRS resource, including from student focus groups, are detailed in Bertka et al. (in review).

Field test and design-based research

Results presented here are from the National Field Test during the 2013–2014 school year of the three curriculum mini-units which focus on natural selection (excluding the What Does It Mean to Be Human? mini-unit which focuses on phylogenetic relationships). Ten teachers from each of 10 schools in eight states (California, Colorado, Connecticut, Maryland, New Jersey, New York, Utah, and Virginia) piloted the mini-units. Teachers were recruited through the researchers’ and advisory board members’ networks of former students and colleagues, an email announcement from the National Association of Biology Teachers (NABT), and members of the Teacher Advisory Council of the National Academy of Sciences. Teachers who responded to the recruitment were typically well-respected, experienced teachers, all of whom already taught Advanced Placement Biology. Many of the teachers were therefore knowledgeable about and experienced in the kinds of research-based practices embedded in the mini-units. The intent was to identify a sample of teachers who could engage with the research team in a partnership to produce and test high quality curriculum materials in a “best case” environment so that we could attempt to explore the impacts of the curriculum materials as the focus of the research. The sample included teachers at a mix of public/private and urban/suburban/rural schools with a range of socioeconomic and ethnic diversity, as well as a mix of teachers at varying career stages (Table 5). Each teacher field tested one of the curriculum mini-units. A total of 304 students field tested the curriculum mini-units and 234 students of seven of the teachers also field tested one of the two CRS Teaching Strategies classroom activities.

Table 5 Summary of school data for each teacher who taught using the Adaptation to Altitude, Evolution of Human Skin Color, or Malaria curriculum mini-units
Full size table

Teachers were asked to implement the lessons within the mini-unit in sequence over a 2-3 week period and to use the mini-unit “as intended,” meaning that they were asked to teach all the lessons without modification in the sequence in which they were provided within recommended time duration. Fidelity of implementation (Table 6) was assessed using a self-reporting document in which the teacher recorded (1) whether each step in the lesson was completed as written, adapted from what was written, or skipped; (2) approximately how long the step took to implement; (3) any notes about that step’s implementation, including whether any adaptations were made and any noteworthy student responses to that step; and (4) whether or not the teacher introduced any extra/non-TEtHE material during that step. Unlike the mini-unit implementation, both CRS activities were optional for teachers and teachers were asked to only use one of the activities—either before or after their instruction on evolution depending on the activity they chose. We asked teachers to record their fidelity for two reasons. First, if there were consistent patterns of deviation from the intent of the mini-units, then we could use teachers’ adaptations as a source of potential revisions. Second, the extent to which teachers were faithful to the intent of the mini-units provided useful data for interpreting the extent to which any changes we might see in student understanding of evolution concepts could be attributable to the mini-units, the CRS, or both.

Table 6 Summary of implementation characteristics outcomes for students of each teacher who taught the Adaptation to Altitude, Evolution of Human Skin Color, and Malaria mini-units
Full size table

To answer research questions 1 and 2 we collected survey data from advisory board members, teachers, and students in which they rated the degree to which the classroom materials aligned to the criteria/core strategies and the degree to which the materials were useable/feasible. Also, one of the authors (Bertka) observed the classroom implementation at five of the schools that field tested the CRS classroom activities and conducted student focus groups at four of those schools.

To answer research question 3 we collected student responses to two questions derived from the ACORNS (Assessing COntextual Reasoning about Natural Selection; Nehm et al. 2012) and GAENE (Generalized Acceptance of EvolutioN Evaluation; Smith et al. 2016; Table 8) instruments as a pretest and a posttest. The pretest–posttest design is considered a pre-experimental design (Shadish et al. 2002). It has limited utility for making causal inferences because of multiple threats to internal validity in that it does not adequately address the potential influences of history, maturation, testing effects, instrumentation, regression to the mean, or any interactions between them. Given the design-based nature of the project and research, a more robust quasi-experimental or experimental design was premature. Our use of the pretest–posttest design limits interpretation of results related to research question 3 to suggestive of a potential impact and gives an indication of whether or not the approach warrants further study. Nevertheless, we argue that the inclusion of data relevant to student understanding and acceptance of evolution to determine whether the materials “first do no harm” (i.e., student understanding and acceptance appear to increase as a result of using them) is warranted.

To assess student understanding of evolution, teachers were asked to distribute a two question ACORNS short-answer diagnostic test. The ACORNS is designed with a sixteen item scoring rubric that standardizes student responses that differ across contextual variables for evolution (e.g., gain vs. loss of traits, plants vs. animals, within vs. between species differences). The instrument included one human-based and one non-human based question, both of which focused on trait gain (Table 7). The non-human based example is directly from Nehm et al. (2012), and the human question was constructed by the research team to have a question similar in stem structure to the non-human example. Each student was asked both questions in the same order both pre- and post-curriculum mini-unit: first the human evolution question, then the non-human evolution question. This sequence was chosen to attempt to mitigate the possibility of learning and transfer, particularly within the pretest, from the potentially more accepted non-human example to the human example, as well as to more accurately reflect a real-world testing situation. The trade-off is that results might be affected by item sequence effects identified for the ACORNS in previous research (e.g., Federer et al. 2015).

Table 7 The two ACORNS questions that were used in this project
Full size table

Teachers were asked to distribute the ACORNS instrument as a pretest the day before implementation of the mini-unit and as a posttest the day after implementation of the mini-unit. We recognize that this is a short duration for a change in understanding to occur. However, it seemed reasonable to attempt to detect such change because of the similarity of the mini-unit to normal classroom practice and the time frame for instruction to time frames typically allocated to instruction in evolution and assessment of student understanding as a result of such instruction.

To assess student acceptance of evolution, teachers were asked to distribute the 16 item Likert-scale GAENE instrument as close as possible to the beginning of the school year for the pre-test and as close as possible to the end of the school year for the post-test. This timing was used to increase the likelihood that an affective change in disposition toward evolution could be detected, as attitude change is unlikely over the recommended implementation window of 2-3 weeks for the mini-units. While this approach weakens the validity of attributing attitude change that occurs over the academic year to one 2–3 week intervention, it nevertheless provides the foundation for further inquiry if attitude change is detected. The version of the GAENE survey used in this project is the version that was available during the time of field testing, and differs somewhat from the final published version (Smith et al. 2016).

Results

Research question 1: To what extent can the project team develop a set of human evolution-centered curriculum mini-units that align with A.P. biology learning objectives, are scientifically rigorous and accurate, and are relevant to students?

The advisory board decided on seven content and science practice criteria to guide the development of the curriculum mini-units (Table 1). In a survey in which respondents were asked the whether the curriculum mini-units aligned to the criteria, 95% of the advisory board responses and 86% of field test teacher responses were “Yes”, 5% of advisory panel responses and 13% of teacher responses were “Yes, but…” (small suggestions for revision were then added) and none of the advisory members and only 1% of field test teacher responses were “No”. Teachers and students were surveyed about the usability of the materials and 78% of the student responses judged the materials as “Just Right”. 80% of teachers responded “Just Right” to questions about the amount of information to conduct the activity, however 55% suggested that more time was needed for the activities. As a response to these formative data, we developed a revised set of “condensed” lessons for each mini-unit. In response to an open-ended question on the student feedback form “How did your experience in learning about evolution through human examples compare with the other ways you’ve learned about evolution?”, 286 student responses could be coded as positive (human examples were preferred), neutral, or negative. Most students answered in the positive (64.7%) versus neutral (28.7%) or negative (6.6%).

Research question 2: To what extent can the project team develop a set of cultural and religious sensitivity resources that provide teachers with strategies that create a supportive classroom environment and support an understanding of the nature of science?

The advisory board decided on six criteria to guide the development of the CRS (Table 3). In a survey in which respondents were asked the whether the CRS aligned to the criteria, 97% of the advisory panel responses and 88% of field test teacher responses were “Yes”, 3% of advisory panel responses and 6% of teacher responses were “Yes, but…” (small suggestions for revision were then added), none of the responses were “No,” and 6% of teacher responses were “Not applicable.” Teachers and students were surveyed about the usability of the materials and 81% of the student and 88% of the teacher responses judged the materials as “Just Right.” A detailed description of the focus groups and their transcript analyses is described elsewhere (Bertka et al. in review).

Research question 3: To what extent does the use of the curriculum mini-units alone, and the curriculum mini-units used in conjunction with the CRS activities, affect student understanding of evolutionary concepts and their acceptance of evolution?

The ACORNS scoring rubrics include rubrics for seven key concepts, including the three core concepts of variation, heritability, and differential reproduction and survival; 3 cognitive biases; and 6 misconceptions (Nehm et al. 2010). Three researchers rated subsets of all student pretests and posttests, with approximately 5% of assessments rated by all three to assess Cohen (1960) kappa interrater reliability (IRR), which was calculated for each pair of coders for each of the 16 ACORNS rubrics. Calculation for 10 of the 16 items for the human item and 8 of the 16 items for the non-human item was compromised because too few student papers showed evidence of the concept, bias, or misconception measured by those rubrics to be meaningful for interpretation. We therefore determined that our measure of evolution should be restricted to core concepts, for which we had sufficient variation in student responses for meaningful analysis and interpretation. Cohen’s kappa IRR for the variation, heritability, and differential survival concepts was 0.77, 0.77, and 0.78, respectively for the human item and for the mouse item was 0.76, 0.94, and 0.77, respectively, indicating substantial agreement among raters for each of these concepts (Landis and Koch 1977).

We reasoned that student understanding of the core concepts of evolution could be ascertained by taking a sum of the ratings (either 1, which indicated the presence of evidence, or 0, which indicated no evidence) for the three core concepts for the mouse item and the human item (for a total of 6 ratings) together. While this limited our ability to interpret the contributions of all evolution concepts to student understanding as well as the potential effects of cognitive bias and misunderstandings, this approach seemed prudent given the limitations of our data. At minimum, the sum of ratings provides a measure of the extent to which students are able to identify core components of evolution in the items to which they responded. These items clustered together conceptually as the three key concepts related to evolution across the two contexts (human and mouse). A principal components analysis on post-test ACORNS data for these three concepts across both contexts (6 ratings, total) revealed a one factor solution, with an Eigenvalue of 1.587 when these six variables were considered, which accounted for 52.9% of the variance. Cronbach’s alpha for the six items was 0.74.

To analyze student understanding of evolution concepts, we used a paired t test with ACORNS assessment scores as the dependent measure. Analyses were conducted separately for each teacher due to the exploratory nature of the analysis and the limitations of the study design and sample size for meaningfully interpreting the hierarchical school, teacher, and supplement variables. Separate analyses for each teacher might also mitigate some of the threats to internal validity described by Shadish et al. (2002) if similar patterns are seen across the classrooms. The results are shown in Table 9 and Fig. 1. We note that the interpretations that follow are tentative given the limitations of the study design.

Fig. 1
figure 1

Results of the analysis of ACORNS pre- and post-curriculum mini-unit implementation scores. Light grey bars indicate pre-unit scores, and dark grey bars indicate post-unit scores. A, Adaptation to Altitude mini-unit; M, Malaria mini-unit; S, Evolution of Human Skin color mini-unit

Full size image

Four of the five teachers who taught the Altitude mini-unit show a significant increase (p < 0.05) in student understanding scores, with the fifth teacher trending in this direction as well (Table 9, Fig. 1). It is worth noting that the one teacher whose class did not show a significant increase in student understanding had a low fidelity of implementation (this teacher taught the lessons throughout the year, rather than in a package) and assessments (this teacher gave the post-ACORNS assessment at the end of the school year, since the lessons were not taught as a package; see Table 8). This statistically significant increase was evident for one of the two teachers who implemented only the mini-unit as well as for each of the three teachers who also implemented one of the CRS classroom activities. Students of both teachers who implemented the Malaria mini-unit—one who also implemented a CRS Classroom Activity and one who did not—showed a statistically significant increase (p < 0.05) in understanding. While there were no significant differences in student understanding from pretest to posttest for students who had engaged in the Evolution of Human Skin Color mini-unit, the variability of these scores (as measured by standard error [SE]) was greater for students exposed to this mini-unit, which may have made it more difficult to detect an effect over and above this error variance. Additionally, the number of students per class was much lower for the two classes that experienced the skin color mini-unit than for the other mini-units, which may explain the higher SE. Cohen’s d effect sizes were also positive for all teachers but one who taught the Skin Color mini-unit.

Table 8 The version of the GAENE survey that was used in this project
Full size table

The final GAENE score used in data analysis was the sum of the 16 items on the GAENE instrument. These items clustered together into one factor using a principal components factor analysis. The factor had an Eigenvalue of 8.45 and accounted for 52.80% of the variance in the data. The alpha reliability coefficient for the 16-item scale was 0.94. A separate paired samples t-test was used each teacher using the same rationale as for the ACORNS analysis. The results are shown in Table 9 and Fig. 2. Again, our interpretations are tentative given the limitations of the study design.

Table 9 Summary of changes in ACORNS and GAENE scores for students of each teacher who taught the Adaptation to Altitude, Evolution of Human Skin Color, and Malaria mini-units
Full size table
Fig. 2
figure 2

Results of the analysis of GAENE beginning and end of school year scores. Light grey bars indicate pre-unit scores, and dark grey bars indicate post-unit scores. A, Adaptation to Altitude mini-unit; M, Malaria mini-unit; S, Evolution of Human Skin Color mini-unit

Full size image

Mean scores for students of 5 of the 9 teachers who submitted pre- and post- GAENE acceptance data increased significantly (p < 0.05), and all but one effect size (Cohen’s d) was positive (range − 0.12 to 0.42) (Figs. 2, 3). Increases, including those that were significant, were generally small, suggesting that non-significant gains could be due to low sample size in some classes. Although acceptance increased for students who completed the Evolution of Human Skin Color mini-unit, these differences were not statistically significant. A possible explanation for this finding was the increased variability for Skin Color attitude data (i.e., larger SE’s) making it more difficult to detect an actual difference. Student mean attitude scores went down for one teacher, although not significantly (n = 11 for this class). Mean pretest scores were also generally very high; therefore, the potential for significant gain was limited. It is difficult to attribute increases or a failure to significantly increase in GAENE to the mini-units because GAENE instruments were usually distributed at the beginning and end of the school year. At minimum, the curriculum mini-units appear to have “done no harm” to students’ scientific perspectives on evolution, which is an important finding given their focus on human evolution.

Fig. 3
figure 3

Comparisons of ACORNS and GAENE effect sizes for the curriculum mini-units. Darker grey bars indicate pre-unit scores, and medium grey bars indicate post-unit scores. A single star indicates the teacher used CRS Classroom Activity 1. A double star indicates the teacher used CRS Classroom Activity 2

Full size image

Overall, the Adaptation to Altitude and Malaria mini-units showed significant increases in student understanding, and all three mini-units showed positive effect sizes when they were taught as intended and assessments were given on schedule (Tables 8, 9). Low sample size might account for the failure to find significant gains in the three cases in which ACORNS gains were not significant. Significant increases were found in schools with varying demographic profiles, among teachers who used both, one, or none of the CRS activities, and for students who experienced the mini-units as intended or modified. The biggest increases in understanding of evolution were usually for students whose teachers had used one of the CRS activities, suggesting they may be effective in helping to create a classroom environment conducive to learning evolution (Fig. 3).

Conclusions

The Teaching Evolution through Human Examples (TEtHE) project successfully developed and field tested three guided, structured inquiry curriculum mini-units for advanced placement biology (Adaptation to Altitude, Malaria, and Evolution of Human Skin Color). The mini-units focus on case studies of natural selection in modern human populations to teach core evolution concepts and align with A.P. biology learning objectives, are scientifically rigorous and accurate, and are relevant to students. The TEtHE project also created and field tested a cultural and religious sensitivity (CRS) teaching strategies resource that provides teachers with strategies that create a supportive classroom environment and encourage a scientific understanding of evolution, including background information and two classroom activities. The teaching materials development process was based on understanding by design (Wiggins and McTighe 2005) and involved an advisory board with informal science educators, expert teachers, scientists, and science education researchers. The mini-units were field tested in 10 classrooms in eight states with 340 students, and the CRS classroom activities were field tested in a subset of seven of those classrooms with 148 of those students to determine to what extent the use of the curriculum mini-units alone, and the curriculum mini-units used in conjunction with the CRS activities, affects student understanding and acceptance of evolution. The project used published, validated assessments to measure student understanding (ACORNS; Nehm et al. 2012) and student acceptance (GAENE; Smith et al. 2016) of evolution.

The Adaptation to Altitude and Malaria curriculum mini-units show the most promise for potential increases in student understanding. The mean total number of core evolution concepts identified on the ACORNS for students of all but one teacher who taught the Adaptation to Altitude and Malaria mini-units increased significantly from the pretest to the posttest- and that teacher whose students’ scores did not significantly increase had low fidelity of implementation, teaching the materials over the entire course of the school year rather than as a mini-unit, and gave the assessments at inopportune times (Table 8). Significant increases were found in schools with varying demographic profiles, among teachers who used both, one, or none of the CRS activities, and for students who experienced the mini-units as intended or modified. The consistency of the results partially mitigates the threats to internal validity inherent in using a pre-experimental, pretest–posttest only design and the limitations of our use of the ACORNS given the limited variation of student responses for several of the items. Nevertheless, we urge caution in considering the potential impact of the materials due to the reality that these threats and limitations were not sufficiently addressed within the framework of design-based research that we used.

While there were no systematic differences in ACORNS or GAENE scores among students of teachers who used either of the CRS activities and those who did not, it is likely that our design was not able to detect the impacts that the CRS activities had. It is impossible to know what outcomes would have been for classes that used CRS activities had they not used them. The evidence from student focus groups and classroom observations, which is presented elsewhere (Bertka et al. in review), are better positioned to detect the subtle impacts of the CRS activities on students.

This is the first study of which we are aware that assessed advanced high school students understanding and acceptance of evolution before and after using curriculum materials that use human examples to teach evolution. Despite a focus on A.P. biology students, who may start with a relatively high level of understanding and acceptance of evolution (versus students who have not had a high school biology class), data indicate an overall increase in both measures. This result, particularly when combined with CRS classroom activities designed to manage tension around the topic of evolution in biology classrooms, indicates that despite the perceived or real controversial nature of the subject of human evolution in particular, this approach can be an engaging and effective way to teach high school students core concepts of evolution. This study provides a concrete example of materials based on human evolution case studies and creating a supportive environment for teaching evolution that appear to have increased A.P. biology students’ acceptance and understanding of evolution, a subject that is notoriously difficult to teach effectively. The results suggest that further testing of the materials with different student populations in the context of a quasi-experimental or experimental design would be worthwhile to more thoroughly explore their potential.