Addressing Undergraduate Student Misconceptions about Natural Selection with an Interactive Simulated Laboratory
- First Online:
Although evolutionary theory is considered to be a unifying foundation for biological education, misconceptions about basic evolutionary processes such as natural selection inhibit student understanding. Even after instruction, students harbor misconceptions about natural selection, suggesting that traditional teaching methods are insufficient for correcting these confusions. This has spurred an effort to develop new teaching methods and tools that effectively confront student misconceptions. In this study, we designed an interactive computer-based simulated laboratory to teach the principles of evolution through natural selection and to correct common student misconceptions about this process. We quantified undergraduate student misconceptions and understanding of natural selection before and after instruction with multiple-choice and open-response test questions and compared student performance across gender and academic levels. While our lab appeared to be effective at dispelling some common misconceptions about natural selection, we did not find evidence that it was as successful at increasing student mastery of the major principles of natural selection. Student performance varied across student academic level and question type, but students performed equally across gender. Beginner students were more likely to use misconceptions before instruction. Advanced students showed greater improvement than beginners on multiple-choice questions, while beginner students reduced their use of misconceptions in the open-response questions to a greater extent. These results suggest that misconceptions can be effectively addressed through computer-based simulated laboratories. Given the level of misconception use by beginner and advanced undergraduates and the gains in performance recorded after instruction at both academic levels, natural selection should continue to be reviewed through upper-level biology courses.
KeywordsActive learning Evolution Education Misconceptions Natural selection
Evolutionary theory, the unifying foundation for most biological concepts, is also one of the most commonly misunderstood areas in biology. It is, therefore, imperative to identify the source of student confusions about evolution and to develop effective teaching tools that can correct them (National Research Council 1996; Novak 2002; Modell et al. 2005; Wescott and Cunningham 2005). Although cultural and political viewpoints affect student understanding (Almquist and Cronin 1988; Sinclair et al. 1997; Hokayem and BouJaoude 2008), even students who claim to accept evolutionary theory often demonstrate little understanding of its basic principles (e.g., Bishop and Anderson 1990; Demastes et al. 1995). These misunderstandings are instead often directly linked to students having their own incorrect conception of the functioning of the world (known as misconceptions), which prevents them from being able to use scientifically accepted concepts in thinking through some scientific problems (Greene 1990; Ferrari and Chi 1998; but see Hamza and Wickman 2008).
Description, source, and frequency of student use of misconceptions about natural selection in pretest open-response questions
# Pretests (N = 338)
Organisms willfully change their traits or the traits of their offspring
e.g. Bishop and Anderson 1990
Variation arises due to environmental or selection pressure
e.g. Bishop and Anderson 1990
Trait change occurs within a generation
e.g. Brumby 1984
Populations change their traits together as a whole
e.g. Bishop and Anderson 1990
Inheritable Phenotypic Change
Acquired changes to phenotype of parent are inherited genetically by offspring
e.g. Lawson and Thompson 1988
Variation through Drift
Genetic drift can create trait variation
e.g. Nehm and Reilly 2007
Fitness is directly related to physical strength
e.g. Bishop and Anderson 1990
Dominant alleles are always selectively advantageous
e.g. Nehm and Reilly 2007
Although many of these misconceptions about natural selection are formed early in life (Evans 2000; Samarapungavan and Wiers 1997; Beardsley 2004), students in high school, college, and even graduate school often retain their misconceptions despite receiving formal training in biology (Brumby 1984; Lawson and Thompson 1988; Bishop and Anderson 1990; Ferrari and Chi 1998; Crawford et al. 2005; Nehm and Reilly 2007; Robbins and Roy 2007). These confusions can inhibit understanding of related biological concepts (Greene 1990; Jensen and Finley 1996) and likely limit students′ ability to explain and comprehend other relevant phenomena such as antibiotic resistance in bacteria, impacts of climate change, or biological engineering.
Traditional teaching methods are not well suited for identifying and correcting misconceptions (Morrison and Lederman 2003; Nelson 2008). However, efforts to develop new pedagogical tools for teaching evolutionary biology have achieved some success. Bishop and Anderson (1990) found moderate improvement in undergraduate student understanding of natural selection after intervention with a unit on that subject designed to dispel misconceptions, while Demastes et al. (1995) found minimal improvement using the same instructional methods. However, high school students showed greater gains, perhaps due to differences in the presentation of instructional materials. Robbins and Roy (2007) developed an inquiry-based teaching unit that yielded drastic improvements in college student explanations and overall acceptance of modern evolutionary theory. Recently, interactive computer-based simulations designed to support student and teacher training in biology have also successfully improved learner understanding of biological concepts and dispelled common learner misconceptions (Crawford et al. 2005; Meir et al. 2005a; Kara and Yesilyurt 2007; Perry et al. 2008). This approach may prove to be particularly useful, as it can increase student access to expert knowledge and offer a greater flexibility in teaching situations. Additionally, as a platform through which students can create experiments and modify parameters, interactive simulations offer an opportunity for students to correct their own misconceptions (Crawford et al. 2005; Perry et al. 2008).
In this study, we sought to develop an interactive computer-based simulation that effectively addresses student misconceptions about natural selection. We first used the literature to identify common misconceptions about the process of natural selection held by undergraduate biology students. We used this list to direct the development of a virtual laboratory (henceforward, Darwinian Snails Lab) within the EvoBeaker software package (Meir et al. 2005b) to teach natural selection principles and to design the test items on the pre- and post tests constructed for this study. We then conducted a summative assessment of student knowledge before and after using the Darwinian Snails Lab to determine the efficacy of that exercise at correcting misconceptions and teaching the principles of natural selection. In addition to studying the lab itself, we also report results on differences in learning between introductory versus advanced biology students and on differences in information gained from the several types of questions included on our tests.
Description of Subject Population and Data Collection
We recruited 637 students from 25 two- and four-year colleges and universities for this study. Students were 18 years of age or older and enrolled in introductory and upper-level biology courses. Henceforward, these students are referred to as “beginner” and “advanced,” respectively (beginner = 170, advanced = 426, unreported = 43). Course instructors volunteered to use the software in their courses; while some instructors had used previous versions of the software, others were newly recruited contacts. In exchange for their help, we provided the software to the instructors and students at no charge. However, instructors received no form of monetary payment. We directed instructors to give the pretest to their students no earlier than one week before using the Darwinian Snails Lab. Students were assigned the lab during the laboratory section of their biology courses and worked alone or with a partner. Within a week of completing the exercise, students were given the post test. A participant profile sheet was attached to each pretest, in which subjects were asked their gender. The majority of students completed the profile sheet (female = 327, male = 248).
The Committee on the Use of Humans as Experimental Subjects, the institutional review board at the Massachusetts Institute of Technology in Cambridge, MA, approved this study before data collection.
The Darwinian Snails Lab in EvoBeaker
Each of the EvoBeaker labs includes a series of interactive simulations with which students design experiments and collect data. Students are provided with a workbook for each lab that directs them through different experiments and asks them to organize and interpret data they collect. We (Herron, Maruca, Meir, Perry, Stal) designed the Darwinian Snails Lab to teach the basic principles of natural selection and to correct the most commonly held misconceptions about natural selection. In this lab, students are presented with a re-creation of a New England rocky shore habitat. The simulated habitat includes populations of the native flat periwinkle snail and their predator, the nonnative European green crab, and is based on the work of Seeley (1986) and, for the final exercise, Trussell (1996). The snails vary in their shell thickness, which affects the efficiency of predation by the European green crab.
Students first read a short section about the simulated system, and then they are introduced to the system by acting as European green crabs feeding on snails. Thicker shells require more effort by the students (i.e., number of mouse clicks needed to feed on the snail). As the students feed on snails, they observe changes in the average shell thickness in the snail population. The population of snails then reproduces, and students are shown how the traits of the remaining snails are inherited by their offspring.
After this initial exercise, students explore three basic requirements for natural selection based on shell thickness: variation in thickness, heritability of thickness, and differential survival of individuals with different shell thicknesses. During this portion of the lab, students sequentially violate each assumption and make predictions about what will then occur when the predator is introduced. Students then quantify changes in the average shell thickness in the snail population.
A third section demonstrates the origin of variation through mutation (but not genetic recombination through sexual reproduction). Students introduce crabs into the system but prevent mutation from occurring in the snail population. They quantify the change in average shell thickness after several generations. Students then allow mutations to occur in the snails, examine individual offspring to see that mutations are random, and compare the average snail shell thickness in the new population after several generations of predation from crabs.
Instrument Design and Validation
The initial design of the test was a series of written open-response questions. We (Herron) asked for feedback on the test from several evolutionary biology instructors and refined the test based on their comments. We (Meir, Perry) then pilot-tested the exams with 20 Boston-area students to further refine problems with test items. We interviewed students after they took the exam, allowed them to further explain their answers in interviews, and used student responses in tests and interviews to develop distracters (incorrect options) for multiple-choice test items.
The final version of the tests designed for this study included nine multiple-choice and seven open-response items about natural selection principles. Four of the multiple-choice items were taken from Settlage and Odum (1995). The remaining multiple-choice and open-response test items presented scenarios from a hypothetical situation and asked students to analyze or predict situations based on the information provided. The majority of the multiple-choice questions included distracters based on student responses and the most common misconceptions about natural selection in the literature. The pre- and post test were structured identically, but we changed the specific information in each item. We analyzed the internal consistency of the pretest multiple-choice questions using the Kuder–Richardson 20 method, which yielded a reliability coefficient of 0.68. Sample short- and long-response test questions are shown in Appendix A. Full copies of our tests are available by writing to SimBiotic Software® (www.simbio.com)—we avoid posting openly to retain their usefulness for instructors.
To quantify the presence or absence of misconceptions on the open-response test items, we (Abraham, Herron, Meir) first developed a rubric based on the list of misconceptions culled from the literature (Table 1). We then independently coded misconceptions and correct concepts in student responses on a subset of tests. Initial agreement among the authors was 85%. Disagreements about misconceptions in the responses were discussed until all of the authors agreed on 100% of the coding. One author (Abraham, who did not participate in designing the lab or tests) then coded misconceptions in open responses based on the revised rubric (N = 338 students). Coding for the presence of misconceptions was done conservatively; we only assigned a misconception to a response when the student explicitly stated it. Instances where a misconception was indirectly suggested but not clearly stated and instances where an incorrect answer was provided that did not link to a misconception were both coded as unclassifiable.
We analyzed student performance on the multiple-choice selection and open-response type items separately. We used the full dataset for the multiple-choice responses (N = 637) but used a subset of those exams for analysis of the open-responses (N = 338). The subset, including tests from twelve institutions, was chosen so that it included a sufficient number of exams from beginner and advanced students and represented the diversity of institution types we included in the study. We first compared the average proportion of correct answers between the pre- and post test multiple-choice items with a one-tailed paired sample Wilcoxon sign-rank nonparametric test. We then used two-tailed Wilcoxon rank sum nonparametric tests to compare the average improvement in student score on the multiple-choice items between two student subgroups: female (n = 187) and male (n = 145) and beginner (n = 128) and advanced (n = 210) students. We calculated Cohen′s d effect sizes for each comparison of pre- and post test scores (Cohen′s Open image in new window, χ = mean, s = pooled standard deviation).
In one open-response test item, we asked students to describe what had occurred in the hypothetical situations in the pre- and post tests to elicit descriptions of the process of natural selection (Question 15, Appendix 1). We designated this question as a long-response question because a correct answer necessarily involved more than one sentence. We compared the number of correct concepts ((1) variation in traits; (2) heritability of traits; (3) differential survival to reproduction; (4) change in average trait value in population over generations) provided by students before and after instruction with a one-tailed paired sample Wilcoxon sign-rank test. We then compared average improvement between gender and academic level with two-tailed Wilcoxon rank-sum tests. We used the statistical software package JMP 7.0.2 (SAS Institute 2008) for each of the preceding analyses.
To analyze student performance on the other open-response questions (short response), we scored each misconception as present or absent in the exam. Thus, a student who used a misconception a single time was scored the same as a student who used it multiple times. Students sometimes failed to provide answers for some of the open-response test items. Before analysis, we compared the frequency of incomplete responses to open-response test items between pre- and post tests. We found no difference between tests, so we included incomplete exams in our analysis. We also compared student use of misconceptions between public and private institutions. We found no difference, so this factor was dropped from the analysis.
We compared the prevalence of the four most common misconceptions (MC1 = willful change, MC2 = directed variation, MC3 = intra generational change, MC4 = population change; defined in Table 1) between the pre- and post tests with a series of McNemar′s paired sample chi-square tests. Other misconceptions did not occur frequently enough to analyze statistically.
We next compared improvement in students who exhibited misconceptions in either test between male or female and beginner or advanced students with a series of chi-square tests. For these comparisons, we defined improvement as a misconception present in the pretest open responses that was absent in the post test. We defined a lack of improvement as either the presence of a misconception in both the pre- and post test or a misconception present in the post test that was absent in the pretest. Thus, students who did not exhibit a given misconception in either test were excluded from this portion of the analysis.
Student Understanding as Evidenced in Multiple-Choice Questions
Student Understanding as Evidenced in a Long-Response Question
Student: “The packrats came eating to cooler fruits in the valley, allowing only hot fruits to reproduce (Variation, Differential Survival/Reproduction). Since spiciness is heritable (Heritability of Trait), the hot fruits would survive and natural selection would eventually wipe out the cool population and that gene (Change in Gene/Trait Frequency).”
Student: “After the shrikes arrived, they began eating horned lizards. The horned lizards with longer horns were better equipped to defend themselves against the shrikes than the lizards with shorter horns (Variation). As a result, the shrikes ended up eating mostly lizards with shorter horns (Differential Survival/Reproduction). Since the lizards that survived were mostly long-horned, the offspring inherited the long-horned trait (Heritability of Trait) and long-horn lizards eventually dominated the population. The average horn length of the entire population was longer than that of the original population (Change in Gene/Trait Frequency).”
Student: “The packrats ate the cooler fruits so they didn′t reproduce. The hotter plants weren′t eaten and thus reproduced (Variation, Differential Survival/Reproduction).”
Student: “When the shrikes arrived they were eating the shorter horn lizards and in order for the lizards to stay alive they had to develop a mutation for long horns so they could pass off to their offspring (Differential Survival/Reproduction, Heritability of Trait, Directed Variation (MC2), Need (MC1))”
Student Understanding as Evidenced in Short-Response Questions
The open responses on the test were designed primarily to elicit these misconceptions: Willful Change (MC1), Directed Variation (MC2), Intra-Generational Change (MC3), and Population Change (MC4). As expected, these were the most commonly exhibited misconceptions in the pretest (Table 1). Of the 338 sets of student open responses on the pretests, slightly more than half exhibited at least one of the four most common misconceptions. Each of these misconceptions is discussed below.
Student: “It would have needed to grow longer horns out of necessity for survival.”
Student: “The lizards were getting along fine with their short/medium horn lengths, but they suddenly became threatened by a predator, so, in order to adapt to their new environment, they had to develop longer horns”
Student: “If there is no change in the environment there would be no need for change in horn sizes there wouldn′t be anything to adapt to out of normal standards”
Student: “Need directional force (predation, etc.)”
Student: “This lizard would have grown longer horns because it had to adapt to the environment it was in”
Student: “The arrival of shrikes will cause the adaptation of longer horns to evolve during the life span of the newly hatched lizards”
Student: “As generations went on the population developed with longer horns, better adapting them to environment”
Student: “As the shrikes attack the lizards counter them by trying to grow horns. Short horns are not as effective as long ones, so they grow longer horns, and each generation grows a bit longer”
Impact of Darwinian Snails Lab on Student Performance
Overall, our interactive simulation software appears to reduce student misconceptions around natural selection. The prevalence of three of the misconceptions dropped 35–49% after use of the Darwinian Snails Lab. Student performance on the multiple-choice items significantly improved after instruction as well, although the magnitude of change (4%) was quite small (Fig. 2). Together, these results suggest that the Darwinian Snails Lab may be an effective tool for addressing some of the most common student misconceptions about the process of natural selection.
The majority of instructors gave the post test immediately following instruction, but some waited up to a week. Although we asked instructors to refrain from explicit instruction on natural selection between testing phases, it may have been unavoidable in some instances. This information was not consistently reported. Use of the lab may also have prompted students to further explore these topics outside of class. Thus, we cannot rule out the possibility that additional instruction affected student post-test performance or that students who were post tested later performed differently than those tested immediately after instruction. However, given the difficulty of correcting misconceptions through traditional teaching methods or self-teaching (Morrison and Lederman 2003; Nelson 2008), we feel that the gains or stasis in student scores are likely due to the strengths and shortcomings of our software. In future studies, we hope to track post-instruction performance over time to get a better appreciation of student retention of concepts.
Of the misconceptions we tested for, Directed Variation (MC2) was the only one that did not significantly decrease in the open responses after instructional intervention (Fig. 6). This misconception was also the most common in our pretest (Fig. 6). This is not surprising, as we and others have found in previous studies that processes involving randomness are difficult for biology students to understand (Meir et al. 2005a; Garvin-Doxas and Klymkowsky 2008). While we attempted to address this misconception with a separate section in the Darwinian Snails lab, that section was added late in the design process and may have suffered from a lack of clarity. In it, we ask students to attempt to increase average snail shell thickness over time through natural selection, with and without mutations in the system. This may have mistakenly reinforced the misconception rather than identifying mutation as a source of random variation. Alternately, the visuals and student exercises may not have been adequate to facilitate the understanding of the random nature of variation. The section of the lab on the role of mutations comprised only 10–15 minutes of a 1.5–2-hour lab, and the visualization of the mutations was less clear than other aspects of the lab such as the selective advantage of thick shells. We suspect that students′ lack of improvement on this misconception partially results from poor lab design rather than it’s being a harder misconception to dispel than the others.
In contrast to the pattern of misconception use by students, we found less convincing evidence that the Darwinian Snails Lab was associated with improvements in overall student use of correct natural selection principles. On the long-response test item, where students wrote a paragraph explaining how natural selection works, the number of principles of natural selection used by students was nearly identical before and after instruction (Fig. 5). While this may be a shortcoming of the software, our study design may have also played a role in this pattern (see below). But to the extent that this result is real, it indicates that our software does not perform as well in this aspect as we had hoped. Some studies have had greater success at increasing student use of principles of natural selection in their explanations (e.g., Jensen and Finley 1996; Robbins and Roy 2007). These studies used instructional methods that explicitly compared the principles of natural selection to misconceptions to better improve student understanding. This may be something we can improve with a future version of the Darwinian Snails Lab.
Differences between Multiple-Choice, Short-Response, and Long-Response Test Items
Although not directly comparable, in general, students performed better on the multiple-choice questions than they did on open-response questions. While students answered nearly three quarters of the multiple-choice questions correctly, the majority of open-response questions were answered incorrectly or incompletely. This fits with other work that suggests multiple-choice questions are not as powerful at capturing students’ underlying conceptual knowledge as other types of questions (e.g., Almquist and Cronin 1988; Nehm and Reilly 2007). However, the use of student misconceptions as distracters in multiple-choice questions allows for a more complete assessment of student conceptual knowledge (Anderson et al. 2002; Tanner and Allen 2005; Settlage and Odom 1995). We have successfully assessed conceptual knowledge with multiple-choice questions in a previous study (Perry et al. 2008). In that case, we saw substantial improvements on multiple-choice questions between pre- and post tests and were able to verify these improvements with student interviews.
In contrast, students in this study showed many more misconceptions, and greater improvement, on the open-response items. The open format of these questions may have provided a better opportunity to assess student understanding. Students frequently offered misconceptions in their responses to questions that were designed to elicit other misconceptions and sometimes mixed correct concepts with misconceptions in their responses.
We used a single long-response test item (Question 15, Appendix 1) near the end of the test to directly assess student understanding of the principles of natural selection. In contrast with the above trend, overall, students showed no change between their pre- and post test responses on this item. Given that 100 students used fewer correct concepts on the post test, but many of those same students improved on those same concepts in the short-answer questions, we believe that the study design may have caused some fatigue. Most students complete the lab in 1.5–2 hours, not including the time spent on the pretest. By the time they reached this question, some students may have been tired of the experience and provided less complete answers than they did on the pretest. A slightly higher percentage of advanced students (31%) performed worse on the pretest than did beginner students (27%), which may reinforce the idea that boredom or fatigue played a role in the pattern rather than a change in understanding.
The results of this and previous studies suggest that when possible, multiple-choice questions should be augmented with open-response questions. Although the utility of the test may decrease, the inclusion of multiple question types allows for a more complete assessment of student conceptual knowledge. This is especially true of ongoing assessment during instruction, as the identification of unforeseen student confusions can aid in the development of future instructional priorities.
Gender and Academic Level Effects on Student Performance
In addition to total improvement among all undergraduate biology students, we also gauged the role played by student gender and academic level on student response to the Darwinian Snail Lab. Several studies have found that student gender impacts the use of misconceptions or replacement of those misconceptions after instruction (Almquist and Cronin 1988; Pearsall et al. 1996). However, we found no impact of gender in any of our analyses here or in previous studies (Meir et al. 2005a, 2007; Perry et al. 2008). Male and female biology students did not differ in their performance on multiple-choice questions, use of misconceptions, or improvement after instruction.
In contrast to the similarity between male and female students, we found significant, if varying, effects of student academic level on performance and improvement. In this study, advanced students were responsible for essentially all of the improvement on the multiple-choice questions (Fig. 3). In contrast, beginner students used significantly fewer principles of natural selection in the pretest than did advanced students but showed greater improvement after instruction (Fig. 5). We found that relatively more beginner students used Willful Change (MC1) in their pretest responses than did advanced student but that there was no effect of academic level on the reductions in misconception use that we recorded (Fig. 7).
While the more frequent use of a misconception and the less frequent use of correct principles by beginner students may be intuitive, we were surprised by the greater improvement of advanced students on multiple-choice items after instruction (Fig. 3). Although the reason for this difference is impossible to determine without interviews, we may have recorded a threshold effect from repeated exposure to the topic. Advanced students have already received instruction on natural selection principles; they are perhaps better able to pick up on the nuances of language differences between the correct answer key versus the distracters presented in the multiple-choice items. Had we seen a similar pattern of greater improvement in misconception use by advanced students after instruction, our results would have provided stronger evidence for this hypothesis. In either case, this study, in agreement with previous research on student misconceptions, provides a strong argument for continued review of natural selection principles through upper-level courses. Student use of misconceptions and inability to explain the process of natural selection is high, even after instruction. However, the significant improvement by advanced and beginner students on either the multiple-choice or open-response questions demonstrates that pedagogical tools designed to challenge misconceptions can be effective at multiple academic levels.
Student use of some of the most common misconceptions about natural selection decreased after exposure to the Darwinian Snails Lab, suggesting that this computer-based interactive simulation is an effective tool. However, the lab was not as successful at improving student mastery of the principles of natural selection. Both beginner and advanced students showed misconceptions about natural selection and were largely unable to fully explain the process of natural selection. These results suggest that basic evolutionary principles should be reviewed in introductory through advanced biology courses. Continued development of teaching practices and tools designed to confront student misconceptions will aid in identifying and confronting student misconceptions and help increase student understanding of the fundamentals of evolutionary theory.
We thank Eric Klopfer and Joel Kingsolver for their support and advice on this project as well as the evolutionary biologists who reviewed our tests. We also thank Mark Hartman for help with data collection and the professors and students who assisted us with this study (names withheld for confidentiality). We are also grateful for the help and advice provided by other employees at SimBiotic Software®. Finally, we thank two anonymous reviewers for their feedback on this manuscript. This research was funded in part by grants #0341202 and #0717495 from the National Science Foundation. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. EvoBeaker is a trademark of SimBiotic Software® for Research and Teaching, Inc.
Conflict of Interest
Abraham, an employee of SimBiotic Software®, had no connection with the company at the planning or data collection phase of this study. Perry was paid by SimBiotic Software® for her work in the planning and data collection phase of the study but no longer had any financial ties to the company at the time of data analysis or paper submission. Meir, Maruca, and Stal are currently employed by SimBiotic Software®. Herron contracted with SimBiotic Software® on this project and has a financial interest in EvoBeaker.