Test-enhanced learning for pairs and triplets: When and why does transfer occur?


In four experiments, we explored conditions under which learning due to retrieval practice (i.e., testing) transfers to the case in which the cue and response words are rearranged (e.g., a training test on gift, rose, ?, wherein the target is wine, and a final test on gift, ?, wine, wherein the answer is rose). In both Experiment 1 and a supplementary experiment, we observed divergent results for pairs and triplets: Relative to a restudy control condition, strong transfer was observed for pairs, but none for triplets. In Experiments 2 and 3, the theoretical basis of the specificity of learning for triplets was explored. The results rule out the possibilities that transfer is wholly absent for triplets and that transfer occurs only for the case of exact cue–response reversal on the final test. Rather, it appears that, for both pairs and triplets, transfer will occur unless both of the following conditions hold: (1) two or more independent cues are presented on the training test, and (2) the correct responses on the training and final tests are different. We show that the majority of the results can be explained by combining the dual-memory theory of the testing effect with an inclusive-OR representation that forms when two or more cues are presented on the training test. Follow-up analyses that were conditionalized on training test accuracy suggest that specificity of learning is greater on a correct than on an incorrect training test trial, although selection confounds and contradictory experimental results preclude a strong conclusion.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8


  1. Caplan, J. B., Glaholt, M. G., & McIntosh, A. R. (2006). Linking associative and serial list memory: Pairs versus triples. Journal of Experimental Psychology: Learning, Memory, and Cognition, 32(6), 1244–1265. https://doi.org/10.1037/0278-7393.32.6.1244

    Article  PubMed  Google Scholar 

  2. Carpenter, S. K., Pashler, H., & Vul, E. (2006). What types of learning are enhanced by a cued recall test? Psychonomic Bulletin & Review, 13(5), 826–830. https://doi.org/10.3758/BF03194004

    Article  Google Scholar 

  3. Delaney, P. F., Verkoeijen, P. P. J. L., & Spirgel, A. (2010). Spacing and testing effects: A deeply critical, lengthy, and at times discursive review of the literature. In B. H. Ross (Ed.), The psychology of learning and motivation: Advances in research and theory Vol. 53 (pp. 63–147). San Diego, CA: Academic Press. https://doi.org/10.1016/S0079-7421(10)53003-2

    Google Scholar 

  4. Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), 4–58. https://doi.org/10.1177/1529100612453266.

  5. Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191.

    Article  Google Scholar 

  6. Hinze, S. R., & Wiley, J. (2011). Testing the limits of testing effects using completion tests. Memory, 19(3), 290–304. https://doi.org/10.1080/09658211.2011.560121

    Article  PubMed  Google Scholar 

  7. Horner, A. J., & Burgess, N. (2013). The associative structure of memory for multi-element events. Journal of Experimental Psychology: General, 142(4), 1370.

    Article  Google Scholar 

  8. Horner, A. J., & Burgess, N. (2014). Pattern completion in multielement event engrams. Current Biology, 24(9), 988–992.

    Article  Google Scholar 

  9. Horner, A. J., Bisby, J. A., Bush, D., Lin, W. J., & Burgess, N. (2015). Evidence for holistic episodic recollection via hippocampal pattern completion. Nature Communications, 6, 7462.

    Article  Google Scholar 

  10. Kahana, M. J. (2002). Associative symmetry and memory theory. Memory & Cognition, 30(6), 823–840. https://doi.org/10.3758/BF03195769

    Article  Google Scholar 

  11. Kahana, M. J., & Caplan, J. B. (2002). Associative asymmetry in probed recall of serial lists. Memory & Cognition, 30(6), 841–849.

  12. Kornell, N., & Vaughn, K. E. (2016). How retrieval attempts affect learning: A review and synthesis. In B. H. Ross (Ed.), The psychology of learning and motivation; the psychology of learning and motivation (pp. 183–215). San Diego, CA: Elsevier Academic Press.

    Google Scholar 

  13. Kornell, N., Hays, M. J., & Bjork, R. A. (2009). Unsuccessful retrieval attempts enhance subsequent learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35(4), 989.

    PubMed  Google Scholar 

  14. Loftus, G. R., & Masson, M. E. J. (1994). Using confidence intervals in within-subject designs. Psychonomic Bulletin & Review, 1(4), 476–490.

    Article  Google Scholar 

  15. McDaniel, M. A., Anderson, J. L., Derbish, M. H., & Morrisette, N. (2007). Testing the testing effect in the classroom. European Journal of Cognitive Psychology, 19(4/5), 494–513. https://doi.org/10.1080/09541440701326154

    Article  Google Scholar 

  16. McDaniel, M. A., Bugg, J. M., Liu, Y., & Brick, J. (2015). When does the test-study-test sequence optimize learning and retention? Journal of Experimental Psychology: Applied, 21(4), 370–382. https://doi.org/10.1037/xap0000063

    Article  PubMed  Google Scholar 

  17. Meiser, T., & Bröder, A. (2002). Memory for multidimensional source information. Journal of Experimental Psychology: Learning, Memory, and Cognition, 28(1), 116–137.

  18. Pan, S. C., & Rickard, T. C. (2017). Does retrieval practice enhance learning and transfer for term-definition facts? Journal of Experimental Psychology: Applied, 23(3). https://doi.org/10.1037/xap0000124

  19. Pan, S. C., & Rickard, T. C. (2018). Transfer of test-enhanced learning: Meta-analytic review and synthesis. Psychological Bulletin, 144(7), 710–756. https://doi.org/10.1037/bul0000151

    Article  PubMed  Google Scholar 

  20. Pan, S. C., Gopal, A., & Rickard, T. C. (2015). Testing with feedback yields potent, but piecewise, learning of history and biology facts. Journal of Educational Psychology. https://doi.org/10.1037/edu0000074

  21. Pan, S. C., Wong, C. M., Potter, Z. E., Mejia, J., & Rickard, T. C. (2016). Does test-enhanced learning transfer for triple associates? Memory & Cognition https://doi.org/10.3758/s13421-015-0547-x

  22. Pan, S. C., Hutter, S., D’Andrea, D., Unwalla, D., & Rickard, T. C. (2018). In search of transfer following cued recall practice: The case of process-based biology concepts. Applied Cognitive Psychology. https://doi.org/10.1002/acp.3506

  23. Pashler, H., Bain, P., Bottge, B., Graesser, A., Koedinger, K., McDaniel, M., et al. (2007). Organizing instruction and study to improve student learning (NCER 2007–2004). Washington, DC: National Center for Education Research, Institute of Education Sciences, U.S. Department of Education. Available from: http://ncer.ed.gov.

  24. Rawson, K. A., & Dunlosky, J. (2011). Optimizing schedules of retrieval practice for durable and efficient learning: How much is enough? Journal of Experimental Psychology: General, 140(3), 283–302. https://doi.org/10.1037/a0023956

    Article  Google Scholar 

  25. Rickard, T. C., & Pan, S. C. (2018). A dual memory theory of the retrieval practice effect. Psychonomic Bulletin & Review https://doi.org/10.3758/s13423-017-1298-4

  26. Roediger, H. L., & Butler, A. C. (2011). The critical role of retrieval practice in long-term retention. Trends in Cognitive Sciences, 15(1), 20–27. https://doi.org/10.1016/j.tics.2010.09.003

    Article  PubMed  Google Scholar 

  27. Roediger, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1(3), 181–210. https://doi.org/10.1111/j.1745-6916.2006.00012

    Article  PubMed  Google Scholar 

  28. Rowland, C. A. (2014). The effect of testing versus restudy on retention: A meta-analytic review of the testing effect. Psychological Bulletin, 140(6), 1432-1463. https://doi.org/10.1037/a0037559

    Article  PubMed  Google Scholar 

  29. Starns, J. J., & Hicks, J. L. (2005). Source dimensions are retrieved independently in multidimensional monitoring tasks. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31(6), 1213.

    PubMed  Google Scholar 

  30. Starns, J. J., & Hicks, J. L. (2008). Context attributes in memory are bound to item information, but not to one another. Psychonomic Bulletin & Review, 15(2), 309–314.

    Article  Google Scholar 

  31. Vaughn, K. E., & Rawson, K. A. (2014). Effects of criterion level on associative memory: Evidence for associative asymmetry. Journal of Memory and Language, 75, 14–26. https://doi.org/10.1016/j.jml.2014.04.004

    Article  Google Scholar 

Download references


The authors thank John Barry, Maxim Deinitchenko, Kayla Hartman, Yangyang Liu, Jonathan Mejia, and Thomas Ting for assistance with data collection. Thanks also to Dina Rodgers for expert assistance with subject pool management.

Author information



Corresponding author

Correspondence to Timothy C. Rickard.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Data and materials for this study are accessible via the Open Science Framework (https://osf.io/95b6r/).



Replication of Experiment 1 under conditions of consistent spatial element arrangement

In a supplementary experiment, we explored whether the results of Experiment 1 hold under conditions of consistent spatial arrangement of the words for each set across all phases of the experiment. This experiment, which was designed to address potential comparisons between the retrieval practice paradigm employed in the present experiments versus those used in other literatures (e.g., work on associative symmetry), replicated the design of Experiment 1 using a 24-hr delay (the 1-week delay was dropped), with the exception that the spatial position of the stimulus elements for each pair and triplet was no longer randomized across the three experimental phases. Specifically, the stimulus words for both pairs and triplets were presented in columnar form throughout all phases of the experiment (whereas in Experiment 1 such a format was used for only the first two phases, as illustrated in Fig. 1), and each word for each set always filled the same columnar position (for both pairs and triplets, the missing word during both training and the final test in the tested-same condition was always the bottom-most word of the column; the missing word in the tested-rearranged condition of the final test was always the highest word of the column; in the restudy condition, half of the missing words were in the lowest position, and half were in the highest position). This provided an implicit cue to subjects that columnar word order was held constant throughout all phases. Thus, in this experiment, memory for relative word location can in principle be a driver of final test performance, including the magnitude of the testing and transfer effects.

Seventy-four undergraduate students participated for course credit. Six subjects were dropped due to not returning for Session 2 or computer errors; analysis was limited to the 68 subjects (33 in the pairs condition, and 35 in the triplets condition) that completed both sessions.

Results and discussion

In the training phase, mean proportion correct on the training test was 0.60 (SE = 0.025) and 0.63 (SE = 0.041), in the pairs and triplets conditions, respectively. Those mean differences were not statistically significant, t(103) = 0.9, p = .37, d = 0.088.

Final test results are depicted in Fig. 9. A factorial ANOVA with the factors stimulus type (between subjects), and final test condition (tested identical vs. tested rearranged vs. restudy; within subjects) indicated no significant main effect of stimulus type, F(1, 66) = 1.79, p = .19, ηp2 = 0.026, replicating Experiment 1. Also as observed in Experiment 1, there was a significant main effect of final test condition, F(2, 132) = 32.4, p < .0001, ηp2 = 0.329. Of most interest is the Stimulus Type × Final Test Condition interaction, F(2, 132) = 3.73, p = 0.026, ηp2 = 0.054; as in Experiment 1, for triplets there was no trend toward positive transfer relative to restudy in the tested-rearranged condition, whereas for pairs there was substantial transfer.

Fig. 9

Mean proportion correct for pairs and triplets in the tested-same, tested-rearranged, and restudy final test conditions of the supplemental experiment. Error bars are standard errors based on the error term of a within-subjects ANOVA conducted separately for pairs and triplets (Loftus & Masson, 1994)

A cross-experiment analysis of the 24-hr delay groups of Experiments 1 and this experiment was performed to more formally investigate the effects of consistent versus varied word location over experimental phases. In an ANOVA with the factors experiment (between subjects), stimulus type (between subjects), and final test condition (within subjects), there was again a significant interaction between final test condition and stimulus type, F(2, 258) = 9.7, p < .0001, ηp2 = 0.07. There were, however, no main or interaction effects involving experiment (all ps > .07). Thus, there is no statistical evidence that consistency of spatial word order affected any aspect of final test performance. It appears that either the spatial position of words was weakly encoded during training or was not retained over the delay between training and the final test. In any case, that factor appears to play a minimal role in testing and transfer effects in this paradigm.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Rickard, T.C., Pan, S.C. Test-enhanced learning for pairs and triplets: When and why does transfer occur?. Mem Cogn 48, 1146–1160 (2020). https://doi.org/10.3758/s13421-020-01048-y

Download citation


  • Memory
  • Testing effect
  • Retrieval practice
  • Transfer
  • Paired associates
  • Triplets