Skip to main content

Optimizing multiple-choice tests as tools for learning

Memory & Cognition volume 43, pages 14–26 (2015)Cite this article

Abstract

Answering multiple-choice questions with competitive alternatives can enhance performance on a later test, not only on questions about the information previously tested, but also on questions about related information not previously tested—in particular, on questions about information pertaining to the previously incorrect alternatives. In the present research, we assessed a possible explanation for this pattern: When multiple-choice questions contain competitive incorrect alternatives, test-takers are led to retrieve previously studied information pertaining to all of the alternatives in order to discriminate among them and select an answer, with such processing strengthening later access to information associated with both the correct and incorrect alternatives. Supporting this hypothesis, we found enhanced performance on a later cued-recall test for previously nontested questions when their answers had previously appeared as competitive incorrect alternatives in the initial multiple-choice test, but not when they had previously appeared as noncompetitive alternatives. Importantly, however, competitive alternatives were not more likely than noncompetitive alternatives to be intruded as incorrect responses, indicating that a general increased accessibility for previously presented incorrect alternatives could not be the explanation for these results. The present findings, replicated across two experiments (one in which corrective feedback was provided during the initial multiple-choice testing, and one in which it was not), thus strongly suggest that competitive multiple-choice questions can trigger beneficial retrieval processes for both tested and related information, and the results have implications for the effective use of multiple-choice tests as tools for learning.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3

References

  • Anderson, R. C., & Biddle, W. B. (1975). On asking people questions about what they are reading. In G. H. Bower (Ed.), The psychology of learning and motivation (Vol. 9, pp. 89–132). New York, NY: Academic Press.

    Google Scholar 

  • Anderson, M. C., Bjork, R. A., & Bjork, E. L. (1994). Remembering can cause forgetting: Retrieval dynamics in long-term memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 20, 1063–1087. doi:10.1037/0278-7393.20.5.1063

    PubMed  Google Scholar 

  • Anderson, M. C., & Spellman, B. A. (1995). On the status of inhibitory mechanisms in cognition: Memory retrieval as a model case. Psychological Review, 102, 68–100. doi:10.1037/0033-295X.102.1.68

    Article  PubMed  Google Scholar 

  • Bjork, R. A. (1975). Retrieval as a memory modifier: An interpretation of negative recency and related phenomena. In R. L. Solso (Ed.), Information processing and cognition: The Loyola symposium (pp. 123–144). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Bjork, E. L., Little, J. L., & Storm, B. C. (2014). Multiple-choice testing as a desirable difficulty in the classroom. Journal of Applied Research in Memory and Cognition.

  • Boker, J. R. (1974). Immediate and delayed retention effects of interspersing questions in written instructional passages. Journal of Educational Psychology, 66, 96–98.

    Article  Google Scholar 

  • Camp, G., Pecher, D., & Schmidt, H. G. (2007). No retrieval-induced forgetting using item-specific independent cues: Evidence against a general inhibitory account. Journal of Experimental Psychology: Learning, Memory, and Cognition, 33, 950–958. doi:10.1037/0278-7393.33.5.950

    PubMed  Google Scholar 

  • Carpenter, S. K., & DeLosh, E. L. (2006). Impoverished cue support enhances subsequent retention: Support for the elaborative retrieval explanation of the testing effect. Memory & Cognition, 34, 268–276. doi:10.3758/BF03193405

    Article  Google Scholar 

  • Carrier, M., & Pashler, H. (1992). The influence of retrieval on retention. Memory & Cognition, 20, 633–642. doi:10.3758/BF03202713

    Article  Google Scholar 

  • Carroll, M., Campbell-Ratcliffe, J., Murnane, H., & Perfect, T. (2007). Retrieval-induced forgetting in educational contexts: Monitoring, expertise, text integration, and test format. European Journal of Cognitive Psychology, 19, 580–606. doi:10.1080/09541440701326071

    Article  Google Scholar 

  • Chan, J. C. K. (2009). When does retrieval induce forgetting and when does it induce faciliation? Implications for retrieval inhibition, testing effect, and text processing. Journal of Memory and Language, 61, 153–170. doi:10.1016/j.jml.2009.04.004

    Article  Google Scholar 

  • Chan, J. C. K., McDermott, K. B., & Roediger, H. L., III. (2006). Retrieval-induced facilitation: Initially nontested material can benefit from prior testing of related material. Journal of Experimental Psychology: General, 135, 553–571. doi:10.1037/0096-3445.135.4.553

    Article  Google Scholar 

  • Dodd, M. D., Castel, A. D., & Roberts, K. E. (2006). A strategy disruption component to retrieval-induced forgetting. Memory & Cognition, 34, 102–111. doi:10.3758/BF03193390

    Article  Google Scholar 

  • Duchastel, P. C. (1981). Retention of prose following testing with different types of test. Contemporary Educational Psychology, 6, 217–226.

    Article  Google Scholar 

  • Foos, P. W., & Fisher, R. P. (1988). Using tests as learning opportunities. Journal of Educational Psychology, 80, 179–183.

    Article  Google Scholar 

  • Frase, L. T. (1967). Learning from prose material: Length of passage, knowledge of results and position of questions. Journal of Educational Psychology, 58, 266–272.

    Article  PubMed  Google Scholar 

  • Frase, L. T. (1968). Effect of question location, pacing, and mode of retention of prose material. Journal of Educational Psychology, 59, 244–249.

    Article  PubMed  Google Scholar 

  • Frase, L. T. (1971). Effect of incentive variables and type of adjunct questions upon text learning. Journal of Educational Psychology, 62, 371–375.

    Article  Google Scholar 

  • Glover, J. A. (1989). The “testing” phenomenon: Not gone but nearly forgotten. Journal of Educational Psychology, 81, 392–399.

    Article  Google Scholar 

  • Hamaker, C. (1986). The effects of adjunct question on prose learning. Review of Educational Research, 56, 212–242.

    Article  Google Scholar 

  • Jacoby, L. L., Shimizu, Y., Daniels, K. A., & Rhodes, M. G. (2005). Modes of cognitive control in recognition and source memory: Depth of retrieval. Psychonomic Bulletin & Review, 12, 852–857. doi:10.3758/BF03196776

    Article  Google Scholar 

  • Johansson, M., Aslan, A., Bäuml, K., Gäbel, A., & Mecklinger, A. (2007). When remembering causes forgetting: Electrophysiological correlates of retrieval-induced forgetting. Cerebral Cortex, 17, 1335–1341. doi:10.1093/cercor/bhl044

    Article  PubMed  Google Scholar 

  • Little, J. L., & Bjork, E. L. (2012). The persisting benefits of using multiple-choice tests as learning events. In N. Miyake, D. Peebles, & R. P. Cooper (Eds.), Proceedings of the 34th Annual Conference of the Cognitive Science Society (pp. 683–688). Austin, TX: Cognitive Science Society.

    Google Scholar 

  • Little, J. L., Bjork, E. L., Bjork, R. A., & Angello, G. (2012). Multiple-choice tests exonerated, at least of some charges: Fostering test-induced learning and avoiding test-induced forgetting. Psychological Science, 23, 1337–1344.

    Article  PubMed  Google Scholar 

  • Little, J. L., Storm, B. C., & Bjork, E. L. (2011). The costs and benefits of testing text materials. Memory, 19, 346–359. doi:10.1080/09658211.2011.569725

    Article  PubMed  Google Scholar 

  • MacLeod, C. M., Dodd, M. D., Sheard, E. D., Wilson, D. E., & Bibi, U. (2003). In opposition to inhibition. In B. H. Ross (Ed.), The psychology of learning and motivation (Vol. 43, pp. 163–214). San Diego, CA: Academic Press.

    Google Scholar 

  • Macrae, C. N., & MacLeod, M. D. (1999). On recollections lost: When practice makes imperfect. Journal of Personality and Social Psychology, 77, 463–473. doi:10.1037/0022-3514.77.3.463

    Article  Google Scholar 

  • McGaw, B., & Grotelueschen, A. (1972). Direction of the effect of questions in prose material. Journal of Educational Psychology, 63, 586–588.

    Article  Google Scholar 

  • Perfect, T. J., Stark, L.-J., Tree, J. J., Moulin, C. J. A., Ahmed, L., & Hunter, R. (2004). Transfer appropriate forgetting: The cued-dependent nature of retrieval-induced forgetting. Journal of Memory and Language, 51, 399–417. doi:10.1016/j.jml.2004.06.003

    Article  Google Scholar 

  • Rickards, J. P. (1976). Interaction of position and conceptual level of adjunct questions on immediate and delayed retention of text. Journal of Educational Psychology, 68, 210–217.

    Article  Google Scholar 

  • Roediger, H. L., III, & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1, 181–210. doi:10.1111/j.1745-6916.2006.00012.x

    Article  Google Scholar 

  • Rothkopf, E. Z. (1966). Learning from written instructive materials: An exploration of the control of inspection behavior by test-like events. American Educational Research Journal, 3, 241–249.

    Article  Google Scholar 

  • Rothkopf, E. Z., & Bisbicos, E. E. (1967). Selective facilitative effects of interspersed questions on learning from written materials. Journal of Educational Psychology, 58, 56–61.

    Article  PubMed  Google Scholar 

  • Rothkopf, E. Z., & Bloom, R. D. (1970). Effects of interpersonal interaction on the instructional value of adjunct questions in learning from written material. Journal of Educational Psychology, 61, 417–422.

    Article  Google Scholar 

  • Storm, B. C., Bjork, E. L., & Bjork, R. A. (2012). On the durability of retrieval-induced forgetting. Journal of Cognitive Psychology, 24, 617–629.

    Article  Google Scholar 

  • Watts, G. H., & Anderson, R. C. (1971). Effects of three types of inserted questions on learning from prose. Journal of Educational Psychology, 62, 387–394.

    Article  Google Scholar 

  • Whitten, W. B., & Leonard, J. M. (1980). Learning from tests: Facilitation of delayed recall by initial recognition alternatives. Journal of Experimental Psychology: Human Learning and Memory, 6, 127–134. doi:10.1037/0278-7393.6.2.127

    Google Scholar 

  • Williams, C. C., & Zacks, R. T. (2001). Is retrieval-induced forgetting an inhibitory process? American Journal of Psychology, 114, 329–354.

    Article  PubMed  Google Scholar 

Download references

Author Note

J.L.L. is now at the Department of Psychology, Hillsdale College. A Collaborative Activity Award from the James S. McDonnell Foundation funded this research. We thank Ashley Kees for creating the materials and helping with data collection. We thank Robert Bjork, Barbara Knowlton, and the members of CogFog for helpful insights. Aspects of this research were reported in a poster presented at the 32nd Annual Conference of the Cognitive Science Society in Portland, Oregon, and appear as part of the dissertation of J.L.L.

Author information

Authors and Affiliations

  1. Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA

    Jeri L. Little & Elizabeth Ligon Bjork

  2. Department of Psychology, Hillsdale College, 33 E. College St., Hillsdale, MI, 49242, USA

    Jeri L. Little

Authors
  1. Jeri L. Little
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Elizabeth Ligon Bjork
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jeri L. Little.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Little, J.L., Bjork, E.L. Optimizing multiple-choice tests as tools for learning. Mem Cogn 43, 14–26 (2015). https://doi.org/10.3758/s13421-014-0452-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.3758/s13421-014-0452-8

Keywords

  • Memory
  • Educational psychology
  • Retrieval processes
  • Learning
  • Multiple-choice tests
  • Testing effects