Abstract
In order to test the effect of exam-question level on fostering student conceptual understanding, low-level and high-level quizzes and exams were administered in two sections of an introductory biology course. Each section was taught in a high-level inquiry based style but was assigned either low-level questions (memory oriented) on the quizzes and exams, or high-level questions (application, evaluation, and analysis) on the quizzes and exams for the entirety of the semester. A final exam consisting of 20 low-level and 21 high-level questions was given to both sections. We considered several theoretical perspectives based on testing effect, test expectancy, and transfer-appropriate processing literature as well as the theoretical underpinnings of Bloom’s taxonomy. Reasoning from these theoretical perspectives, we predicted that high-level exams would encourage not only deeper processing of the information by students in preparation for the exam but also better memory for the core information (learned in the service of preparing for high-level questions). Results confirmed this prediction, with students in the high-level exam condition demonstrating higher performance on both the low-level final-exam items and the high-level final exam items. This pattern suggests that students who are tested throughout the semester with high-level questions acquire deep conceptual understanding of the material and better memory for the course information, and lends support to the proposed hierarchical nature of Bloom’s taxonomy.
This is a preview of subscription content,
to check access.

Similar content being viewed by others
Notes
Analysis without the covariate (ANOVA) indicates the same effects, F (1, 167) = 13.04, MSE = .01, p < .0001, hp 2 = .072
Again, analysis without the covariate (ANOVA) indicates the same effects, F (1, 167) = 7.73, MSE = 002, p < .01, hp 2 = .044
Again, analysis without the covariate (ANOVA) indicates the same effects, F (1, 167) = 12.27, MSE = .02, p < .01, hp 2 = .068
We appreciate an anonymous reviewer for raising this possibility.
References
AAAS. (2010). Vision and change: a call to action. Washington: AAAS.
Anderson, G. L., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., et al. (Eds.). (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Boston: Allyn & Bacon.
Becker, H. S., Geer, B., & Hughes, E. C. (1968). Making the grade: The academic side of college life. New Brunswick: Transaction.
Biggs, J. B. (1987). Student approaches to studying and learning. Hawthorn: Australian Council for Educational Research.
Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-regulated learning: Beliefs, techniques, and illusions. Annual Review of Psychology, 64, 417–444.
Bloom, B. S. (1984). Taxonomy of educational objectives. Boston: Allyn and Bacon.
Bybee, R. (1993). An instructional model for science education: Developing biological literacy. Colorado Springs: Biological Sciences Curriculum Studies.
Campbell, N. A., & Reece, J. B. (2005). Biology, 7th ed. San Francisco: Benjamin Cummings.
Carpenter, S. K. (2012). Testing enhances the transfer of learning. Current Directions in Psychological Science, 21(5), 279–283. doi:10.1177/0963721412452728.
Carpenter, S. K., & DeLosh, E. L. (2006). Impoverished cue support enhances subsequent retention: Support for the elaborative retrieval explanation of the testing effect. Memory & Cognition, 34(2), 268–276. doi:10.3758/Bf03193405.
Carpenter, S. K., & Pashler, H. (2007). Testing beyond words: Using tests to enhance visuospatial map learning. Psychonomic Bulletin & Review, 14(3), 474–478. doi:10.3758/Bf03194092.
Carpenter, S. K., Pashier, H., Wixted, J. T., & Vul, E. (2008). The effects of tests on learning and forgetting. Memory & Cognition, 36(2), 438–448. doi:10.3758/Mc.36.2.438.
Carpenter, S. K., Pashler, H., & Cepeda, N. J. (2009). Using tests to enhance 8th grade students’ retention of US history facts. Applied Cognitive Psychology, 23(6), 760–771. doi:10.1002/Acp.1507.
Carrier, M., & Pashler, H. (1992). The influence of retrieval on retention. Memory & Cognition, 20(6), 633–642. doi:10.3758/Bf03202713.
Chan, J. C. K., & McDermott, K. B. (2007). The testing effect in recognition memory: A dual process account. [Reports - Evaluative]. Journal of Experimental Psychology: Learning, Memory, and Cognition, 33(2), 431–437.
Crowe, A., Dirks, C., & Wenderoth, M. P. (2008). Biology in bloom: Implementing Bloom’s Taxonomy to enhance student learning in biology. CBE - Life Sciences Education, 7, 368–381.
Dickie, L. O. (2003). Approach to learning, the cognitive demands of assessment, and achievement in physics. [Reports - Research]. Canadian Journal of Higher Education, 33(1), 87–111.
Finley, J. R., & Benjamin, A. S. (2012). Adaptive and qualitative changes in encoding strategy with experience: Evidence from the test-expectancy paradigm. Journal of Experimental Psychology: Learning Memory and Cognition, 38(3), 632–652. doi:10.1037/A0026215.
Fisher, R. P., & Craik, F. I. M. (1977). Interaction between encoding and retrieval operations in cued recall. Journal of Experimental Psychology: Human Learning and Memory. 3(6), 710–711.
Harlen, W., & Deakin Crick, R. (2002). A systematic review of the impact of summative assessment and tests on students’ motivation for learning (EPPI-Centre Review, version 1.1*). Research Evidence in Education Library (Vol. 1). London: EPPI-Centre, Social Science Research Unit, Institute of Education.
Hill, P. W., & Mcgaw, B. (1981). Testing the simplex assumption underlying Blooms Taxonomy. American Educational Research Journal, 18(1), 93–101. doi:10.3102/00028312018001093.
Johnson, C. I., & Mayer, R. E. (2009). A testing effect with multimedia learning. Journal of Educational Psychology, 101(3), 621–629. doi:10.1037/A0015183.
Joughin, G. (2010). The hidden curriculum revisited: A critical review of research into the influence of summative assessment on learning. Assessment & Evaluation in Higher Education, 35(3), 335–345.
Kang, S. H. K., McDermott, K. B., & Roediger, H. L., III. (2007). Test format and corrective feedback modulate the effect of testing on memory retention. European Journal of Cognitive Psychology, 19, 528–558.
Krathwohl, D. R. (2002). A revision of Bloom’s Taxonomy: An overview. Theory Into Practice, 41(4), 212–218.
Kropp, R. P., et al., & Florida State Univ. Tallahassee. Inst. of Human Learning. (1966). The construction and validation of tests of the cognitive processes as described in the “Taxonomy of Educational Objectives”. (pp. 444).
Lawson, A. E. (1978). The development and validation of a classroom test of formal reasoning. Journal of Research in Science Teaching. 15(1), 11–24.
Lawson, A. E. (2002). Science teaching and development of thinking. Belmont: Wadsworth/Thompson Learning.
Lawson, A. E., Alkhoury, S., Benford, R. B. C., & Falconer, K. A. (2000a). What kinds of scientific concepts exist? Concept construction and intellectual development in college biology. Journal of Research in Science Teaching, 37(9), 996–1018.
Lawson, A. E., Alkhoury, S., Benford, R., Clark, B. R., & Falconer, K. A. (2000b). What kinds of scientific concepts exist? Concept construction and intellectual development in college biology. [Reports - Research]. Journal of Research in Science Teaching, 37(9), 996–1018.
Lawson, A. E., Clark, B., Cramer-Meldrum, E., Falconer, K. A., Sequist, J. M., & Kwon, Y.-J. (2000c). Development of scientific reasoning in college biology: Do two levels of general hypothesis-testing skills exist? [Reports - Research]. Journal of Research in Science Teaching, 37(1), 81–101.
Madaus, G. F. (1973). A causal model analysis of Bloom’s Taxonomy. American Educational Research Journal, 10(4), 253–262.
Mayer, R. E. (2003). Learning and instruction. Upper Saddle River: Prentice Hall. 10(4), 253–262.
McDaniel, M. A., & Donnelly, C. M. (1996). Learning with analogy and elaborative interrogation. Journal of Educational Psychology, 88(3), 508–519. doi:10.1037//0022-0663.88.3.508.
McDaniel, M. A., Blischak, D. M., & Challis, B. (1994). The effects of test expectancy on processing and memory of prose. Contemporary Educational Psychology, 19, 230–248.
McDaniel, M. A., Anderson, J. L., Derbish, M. H., & Morrisette, N. (2007). Testing the testing effect in the classroom. European Journal of Cognitive Psychology, 19(4-5), 494–513. doi:10.1080/09541440701326154.
McDaniel, M. A., Wildman, K. M., & Anderson, J. L. (2012). Using quizzes to enhance summative-assessment performance in a web-based class: An experimental study. Journal of Applied Research in Memory and Cognition, 1, 18–26.
McDaniel, M. A., Friedman, A., & Bourne, L. E., Jr. (1978). Remembering the levels of information in words. Memory & Cognition, 6, 156-164.
McDaniel, M. A., Thomas, R. C., Agarwal, P. K., McDermott, K. B., & Roediger, H. L., III. (2013). Quizzing in middle school science: Successful transfer performance on classroom exams. Applied Cognitive Psychology, 27, 360–372.
McDermott, K. B., Agarwal, P. K., D’Antonio, L., Roediger, H. L., III, & McDaniel, M. A. (in press). Both multiple-choice and short-answer quizzes enhance later exam performance in middle and high school classes. Journal of Experimental Psychology: Applied.
Miller, C. M. L., & Parlett, M. (1974). Up to the mark: A study of the examination game. London: Society for Research into Higher Education.
Momsen, J. L., Long, T. M., Wyse, S. A., & Ebert-May, D. (2010). Just the facts? Introductory undergraduate biology courses focus on low-level cognitive skills. [Reports - Evaluative]. CBE - Life Sciences Education, 9(4), 435–440.
Momsen, J. L., Offerdahl, E., Kryjevskaia, M., Montplaisir, L., Anderson, E., & Grosz, N. (2013). Using assessments to investigate and compare the nature of learning in undergraduate science courses. CBE - Life Sciences Education, 12, 239–249.
National Research Council. (1996). National science education standards. Washington: National Academy.
Roediger, H. L., III, & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17, 249–255.
Roediger, H. L., III, Agarwal, P. K., McDaniel, M. A., & McDermott, K. B. (2011). Test-enhanced learning in the classroom: Long-term improvements from quizzing. Journal of Experimental Psychology: Applied, 17, 382–395.
Rohrer, D., Taylor, K., & Sholar, B. (2010). Tests enhance the transfer of learning. [Reports - Research]. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36(1), 233–239.
Seddon, G. M. (1978). The Properties of Bloom’s Taxonomy of Educational Objectives for the Cognitive Domain. Review of Educational Research, r.
Snyder, G. (1951). The hidden curriculum. New York: Knopf.
Sternberg, R. J., Grigorenko, E. L., & Zhang, L. (2008). Styles of learning and thinking matter in instruction and assessment. Perspectives on Psychological Science, 3, 486–506.
Struyven, K., Dochy, F., & Janssens, S. (2005). Students’ perceptions about evaluation and assessment in higher education: A review. Assessment & Evaluation in Higher Education, 30(4), 325–341.
Thiede, K. W., Wiley, J., & Griffing, T. D. (2011). Test expectancy affects metacomprehension accuracy. British Journal of Educational Psychology, 81, 264–273.
Thomas, A. K., & McDaniel, M. A. (2007). The negative cascade of incongruent generative study-test processing in memory and metacomprehension. Memory & Cognition, 35(4), 668–678. doi:10.3758/Bf03193305.
Van Etten, S., Pressley, M., McInerney, D. M., & Liem, A. D. (2008). College senior’s theory of their academic motivation. Journal of Educational Psychology, 100(4), 812–828.
Wiggins, G., McTighe, J., & Association for Supervision and Curriculum Development Alexandria VA. (1998). Understanding by Design. (pp. 214): Association for Supervision and Curriculum Development, 1703 North Beauregard Street, Alexandria, VA 22311–1714 (stock number 198199, members:.
Zawicki, P., & Witas, H. W. (2007). HIV-1 protecting CCR5-D32 allele in medieval Poland. Infection, Genetics and Evolution, 8(2): 146-151.
Zoller, U. (1993). Are lecture and learning compatible? Maybe for LOCS: unlikely for HOCS (SYM). Journal of Chemical Education 70: 195–197.
Conflict of Interest
The authors declare that they have no conflict of interest.
Author information
Authors and Affiliations
Corresponding author
Appendix
Appendix
Sample Test Items
Low-Level Unit Exam Items
Term/Concept: Haploid/Meiosis
-
i.
In humans, the haploid number, n equals:
-
a)
2n.
-
b)
44.
-
c)
23.
-
d)
46.
-
a)
Term/Concept: Nondisjunction/Meiosis
-
ii.
Trisomy is a consequence of _______________________, the unequal distribution of chromosomes during meiosis.
-
a)
recombination
-
b)
cytokinesis
-
c)
nondisjunction
-
d)
crossing over
-
a)
Term/Concept: Types of Selection/Evolution
-
iii.
Divergent selection occurs when:
-
a)
low-ranking, “sneaker” males disrupt the mating between a dominant female and a male.
-
b)
individuals with one extreme value of a trait have greater fitness than individuals with the other extreme value of the trait.
-
c)
individuals with extreme values of a trait have greater fitness than individuals with intermediate values of the trait.
-
d)
there is selection against the extreme ends of a trait’s distribution.
-
a)
Term/Concept: Hardy–Weinberg Equilibrium/Microevolution
The pygmies are a group of humans who are thought to have lived continuously in the rainforest of central Africa since the ice age, in geographically separated areas called refuges. There are two distinct groups of pygmies, the eastern and the western, living in two different refuges. The eastern population has an HbS allele frequency of 8 %. The western population has an HbS allele frequency of 18 %.
-
iv.
Are the eastern and western pygmies in Hardy–Weinberg equilibrium?
-
a)
Yes
-
b)
No
-
c)
Impossible to tell
-
a)
Term/Concept: Predicted Allelic Frequencies/Microevolution
-
v.
Given the HbS allele frequency in the western population is 0.18 (q), what is the frequency of the normal allele (HbA) in the western population?
-
a)
0.2
-
b)
0.18
-
c)
0.82
-
d)
0.08
-
a)
Term/Concept: Interpreting the Periodic Table/Atomic theory

-
vi.
Looking at the figure to the right, how many protons does chlorine have?
-
a)
17
-
b)
18
-
c)
35.5
-
d)
71
-
a)
Term/Concept: Electron Shells/Atomic Bonding
-
vii.
If the normal electron-shell diagram of Sodium (Na) is pictured to the right, what does Na+ look like?
Low-level Final Exam Items
Term/Concept: Types of Bonds/Atomic bonding
-
viii.
____________ bonds deal with the sharing of electrons, while __________bonds deal with the transferring of electrons completely to another atom.
-
a)
Ionic, Hydrogen
-
b)
Ionic, Covalent
-
c)
Covalent, Hydrogen
-
d)
Covalent, Ionic
-
e)
Hydrogen, Covalent
-
f)
Hydrogen, Ionic
-
a)
Term/Concept: G1/Cell Cycle
-
ix.
Within a Eukaryotic cell cycle, the primary stage during which normal cellular functioning occurs is
-
a)
G1
-
b)
S
-
c)
G2
-
d)
Mitosis
-
e)
Meiosis
-
a)
Term/Concept: Hardy–Weinberg Equilibrium/Microevolution
-
x.
Which of the following violations to Hardy–Weinberg assumptions would be most likely to change the allele frequency of a small population of pigmy monkeys living on an island?
-
a)
Mutation
-
b)
Changing environment
-
c)
Genetic drift
-
d)
Gene flow
-
e)
Selection
-
f)
Non-random mating
-
g)
All of the above have an equal chance of moving the population out of Hardy–Weinberg equilibrium.
-
a)
High-Level Unit Exam Items
Term/Concept: Haploid/Meiosis
You and your spouse are expecting your first child. Because you and your spouse have an extensive history of genetic diseases on both sides of the family, you are concerned about the health of your unborn baby and request that an amniocentesis be performed so that you can prepare yourself for whatever lies ahead. During this procedure, a large needle is inserted through the mother’s belly and into the amniotic sack in order to collect a sample of amniotic fluid containing sloughed cells from the embryo. These cells are then grown in a laboratory for about a week in order to analyze the chromosomes.
-
xi.
The DNA from a typical human sperm cell weighs approximately 3.3 picograms (a picogram is a 10−12 g). If all chromosomes weighed approximately the same, how much does a typical chromosome weigh?
-
a)
0.07 picograms
-
b)
0.14 picograms
-
c)
0.28 picograms
-
d)
75.9 picograms
-
e)
151.8 picograms
-
a)
Term/Concept: Nondisjunction/Meiosis
-
xii.
Doctors find that in the majority of embryo’s cells, the DNA weighs nearly 6.8 picograms. What is a possible explanation for this?
-
a)
The embryonic cells remained haploid, meaning that the sperm’s chromosomes never fused with the egg’s and were most likely kicked out with the polar body.
-
b)
The embryonic cells contain all duplicated chromosomes; thus, there must be a problem with Meiosis II and sister chromatids are never separating.
-
c)
The embryonic cells contain extra DNA indicating a possible trisomy.
-
d)
The embryo is normal but is definitely a girl, since girls carry an extra X chromosome.
-
a)
Term/Concept: Types of Selection/Evolution
Zawicki and Witas (2007) studied the C-C chemokine receptor 5 (CCR5), a receptor on the white blood cell that is used by many pathogens to gain access to the cell, including HIV, Bubonic plague, and others. It has been found that a mutation (the deletion of 32 base pairs; designated CCR5-D32) prevents HIV from entering the cell. Individuals who are homozygous for the CCR5-D32 mutation are nearly completely resistant to HIV infection and individuals who are heterozygous have slower disease progression and better survival rates.
-
xiii.
There are no known disadvantages to being homozygous for the CCR5-D32 mutation. Predict what type of selection will most likely occur, given enough time?
-
a)
Stabilizing
-
b)
Directional
-
c)
Diversifying
-
a)
Term/Concept: Hardy–Weinberg Equilibrium/Microevolution
-
xiv.
Scientists have found a high incidence of this allele in European and Mediterranean people. The frequency of the CCR5-D32 allele on the European continent is 10 %. What percentage of the people are likely carriers of the CCR5-D32 allele?
-
a)
90 %
-
b)
10 %
-
c)
18 %
-
d)
9 %
-
a)
Term/Concept: Predicted Allelic Frequencies/Microevolution
-
xv.
Given that the CCR5-Δ32 allele confers immunity to the bubonic plague, which of the following phenomena would you predict?
-
a)
The bubonic plague of the 14 th century would closely correspond in geographic distribution as well as timing with the suspected origin and increasing frequency of the CCR5- Δ32 mutation.
-
b)
The bubonic plague of the 14th century would directly precede (come before) the emergence of the CCR5- Δ32 mutation in the same geographic distribution indicating that the plague caused the mutation of the CCR5 receptor.
-
c)
The bubonic plague of the 14th century would be only in geographic areas where the CCR5- Δ32 mutation was not prevalent since those are the people most susceptible to plague infection.
-
d)
The bubonic plague of the 14th century would have emerged shortly after and in the same geographic location as the emergence and increasing frequency of the CCR5- Δ32 mutation indicating that the CCR5- Δ32 mutation increases plague infection rate.
-
a)
Term/Concept: Electron Shells/Atomic Bonding
A salt is defined as “a compound resulting from the formation of an ionic bond; also called an ionic compound” (Campbell & Reece, 2005). That means that a salt can be formed from any two ions. Below are the electron-shell diagrams of magnesium (Mg) and chlorine (Cl).

-
xvi.
If table salt (the kind you eat) is sodium chloride (NaCl), what is the molecular formula of magnesium chloride?
-
a)
MgCl
-
b)
Mg2Cl
-
c)
MgCl 2
-
d)
Mg2Cl2
-
a)
High-Level Final Exam Items
Term/Concept: Predicting Selection/Microevolution
-
xvii.
The incidence of spinal muscular atrophy (an autosomal recessive disease) in the United States is about 1 case in every 17,000. Whereas, in North Dakota, the prevalence is 1 in 6720. Which of the following would support the hypothesis that genetic drift was responsible for the increased allele frequency in North Dakota?
-
a)
There is an abnormally high concentration of mutagenic chemicals in the ground water causing an increased mutation rate in North Dakota.
-
b)
One of the best treatment centers for SMA is located in North Dakota causing migration of SMA carriers into the area at an abnormally high frequency.
-
c)
The original settlers of North Dakota were a small group of pioneers who happened to have an abnormally high frequency of the SMA allele in their population.
-
d)
A particular mosquito-born parasite native to North Dakota causes high infant mortality; carriers of the SMA allele are less likely to catch the disease.
-
a)
Term/Concept: Electron Shells/Atomic Bonding
The hydrogen car runs off the power of hydrogen created from water in the process of electrolysis (i.e., the splitting of water). In this process, water is split into hydrogen and oxygen gases by running an electrical current through the water. This is done by placing two metal rods in the water and attaching them to an electrical source. The cathode donates electrons to the solution; whereas the anode accepts electrons from the solution. The reactions at each node are shown below (keep in mind that these reactions are happening at the same time):
At the Cathode, where hydrogen gas bubbles are produced: 2H+ + 2e- → H2 (gas)
At the Anode, where oxygen gas bubbles are produced: 2H2O → O2 (gas) + 4H+ + 4e-
-
xviii.
How many electrons must come from an outside source (like the anode or cathode) to make this reaction run?
-
a)
Two electrons are being donated by the cathode
-
b)
Two electrons are being donated by the anode
-
c)
Four electrons are being donated by the anode
-
d)
Eight electrons are being donated by the anode
-
e)
No electrons are being donated, they are all contained within the original water molecule
-
a)
Term/Concept: Homologous Chromosomes/Meiosis
-
xix.
Cystic fibrosis, an autosomal recessive disease, is located on chromosome #7. If you viewed the 7th chromosome pair of an individual who is a carrier for cystic fibrosis during prophase of mitosis, which of the following is the best representation of what you would see?
Term/Concept: Interpreting the Periodic Table/Atomic Theory
-
xx.
Let’s say I have a hypothetical element, called “Element J.” It tends to make three bonds and the most common isotope has 3 more neutrons than protons. Which of the following is the most accurate representation of Element J?
Rights and permissions
About this article
Cite this article
Jensen, J.L., McDaniel, M.A., Woodard, S.M. et al. Teaching to the Test…or Testing to Teach: Exams Requiring Higher Order Thinking Skills Encourage Greater Conceptual Understanding. Educ Psychol Rev 26, 307–329 (2014). https://doi.org/10.1007/s10648-013-9248-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10648-013-9248-9