Skip to main content
SpringerLink
Log in

Modeling Theories and Theorizing Models: an Attempted Replication of Miller-Cotto & Byrnes’ (2019) Comparison of Working Memory Models Using ECLS-K Data

  • Replication
  • Published:
Educational Psychology Review Aims and scope Submit manuscript

Abstract

Working memory is an essential mechanism in the cognitive learning process. However, its definitions and mechanisms remain a topic of debate. Miller-Cotto and Byrnes (Journal of Educational Psychology, 112(5), 1074–1084, 2020) reported a comparison of three models of working memory to determine which best accounted for data obtained from a national US dataset of young children over time (Early Childhood Longitudinal Study of Kindergarten). In comparing these models, which they derived from competing theories of working memory, they found that (1) one of the tested models best fit the available data and (2) the model parameters were similar across content domains, concluding that one specific theory of working memory was best supported. The study reported here is an attempted replication of this work. We implemented a multi-phase effort at replication, first undertaking a review of the literature surrounding competing theories of working memory to aid interpretation of results, then attempting a direct replication, and lastly applying an alternative modeling technique to appropriately differentiate between between- and within-person variance. Neither effort succeeded in fully replicating the original findings. Instead, we found that parsing between- and within-person variance is an essential strategy for appropriately interpreting the relationships between working memory and domain learning. Doing so led to support for different models across domains, and interpretation of aspects of these models through the lens of relevant theory suggested alternative strategies for examining these issues in future research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. All data files and analysis scripts are available online at https://www.dropbox.com/sh/68uxrimic948uac/AACE66vNoWVcklfjXGqdjRNqa?dl=0

  2. We were unable to ascertain with certainty whether the sample we used are exactly the same as the original author, as the authors refused to provide the data sample they used for analysis.

References

  • Ackerman, P. (2003). Aptitude complexes and trait complexes. Educational Psychologist, 38(2), 85–93.

  • Adams, E., Nguyen, A., & Cowan, N. (2018). Theories of working memory: Differences in definition, degree of modularity, role of attention, and purpose. Language, Speech, and Hearing Services in Schools, 49(3), 340–355.

  • Alloway, T. P., & Alloway, R. G. (2010). Investigating the predictive roles of working memory and IQ in academic attainment. Journal of Experimental Child Psychology, 106(1), 20–29.

    Google Scholar 

  • Alloway, T. P., Gathercole, S. E., Willis, C., & Adams, A.-M. (2004). A structural analysis of working memory and related cognitive skills in young children. Journal of Experimental Child Psychology, 87(2), 85–106.

    Google Scholar 

  • Baddeley, A. (2000). The episodic buffer: A new component of working memory? Trends in Cognitive Sciences, 4(11), 417–423.

  • Baddeley, A. (2007). Working memory, thought, and action. Oxford: Oxford University Press.

    Google Scholar 

  • Baddeley, A. (2010). Working memory. Current Biology, 20(4), 136–140.

    Google Scholar 

  • Baddeley, A., & Hitch, G. (1974). Working memory. Psychology of Learning and Motivation, 8, 47–89.

  • Barrett, L. F., Tugade, M. M., & Engle, R. W. (2004). Individual differences in working memory capacity and dual-process theories of the mind. Psychological Bulletin, 130(4), 553–573.

    Google Scholar 

  • Bayliss, D., Jarrold, C., Gunn, D., & Baddeley, A. (2003). The complexities of complex span: Explaining individual differences in working memory in children and adults. Journal of Experimental Psychology: General, 132(1), 71–92.

    Google Scholar 

  • Berry, D., & Willoughby, M. (2017). On the practical interpretability of cross-lagged panel models: Rethinking a developmental workhorse. Child Development, 88, 1186–1206.

  • Bollen, K., & Curran, P. (2004). Autoregressive latent trajectory (ALT) models: A synthesis of two traditions. Sociological Methods & Research, 32, 336–383.

    Google Scholar 

  • Bollen, K., & Pearl, J. (2013). Eight myths about causality and structural equation models. In S. Morgan (Ed.), Handbook of Causal Analysis for Social Research (pp. 301-328). New York: Springer.

  • Borsboom, D., Kieveit, R., Cervone, D., & Hood, S. B. (2009). The two disciplines of scientific psychology, or: The disunity of psychology as a working hypothesis. In J. Valsiner, P. Molenaar, M. Lyra, & N. Chaudhary (Eds.), Dynamic process methodology in the social and developmental sciences (pp. 67–97). New York: Springer.

    Google Scholar 

  • Brose, A., Schmiedek, F., Lövdén, M., Molenaar, P., & Linenberger, U. (2010). Adult age differences in covariation of motivation and working memory performance: Contrasting between-person and within-person findings. Research in Human Development, 7, 61–78.

  • Burkart, J., Schubiger, M., & Van Schaik, C. (2017). The evolution of general intelligence. Behavioral and Brain Sciences, 40, E195.

    Google Scholar 

  • Byrnes, J. Personal communication, 2019.

    Google Scholar 

  • Camp, G., Paas, F., Rikers, R., & van Merrienboer, J. (2001). Dynamic problem selection in air traffic control training: A comparison between performance mental effort and mental efficiency. Computers in Human Behavior, 17, 575–595.

  • Cowan, N. (1999). An embedded-processes model of working memory. New York: Cambridge University Press.

    Google Scholar 

  • Cowan, N. (2016). Working memory maturation: Can we get at the essence of cognitive growth? Perspectives on Psychological Science, 11(2), 239–264.

    Google Scholar 

  • Cowan, N. (2017). The many faces of working memory and short-term storage. Psychonomic Bulletin & Review, 24(4), 1158–1170.

    Google Scholar 

  • Cowan, N. (2019). Short-term memory based on activated long-term memory: A review in response to Norris (2017). Psychological Bulletin, 145(8), 822–847.

    Google Scholar 

  • Cowan, N., Nugent, L. D., Elliott, E. M., Ponomarev, I., & Saults, J. S. (1999). The role of attention in the development of short-term memory: Age differences in the verbal span of apprehension. Child Development, 70(5), 1082–1097.

    Google Scholar 

  • Cowan, N., Saults, J. S., & Blume, C. L. (2014). Central and peripheral components of working memory storage. Journal of Experimental Psychology: General, 143(5), 1806–1836.

  • Cowan, N., Li, Y., Glass, B. A., & Scott Saults, J. (2018). Development of the ability to combine visual and acoustic information in working memory. Developmental Science, 21(5), e12635.

    Google Scholar 

  • Cronbach, L. J., & Snow, R. E. (1977). Aptitudes and instructional methods: A handbook for research on interactions. New York: Irvington.

    Google Scholar 

  • Curran, P. J., & Bauer, D. J. (2011). The disaggregation of within-person and between-person effects in longitudinal models of change. Annual Review of Psychology, 62(1), 583–619.

    Google Scholar 

  • Curran, P. J., Howard, A. L., Bainter, S. A., Lane, S. T., & McGinley, J. S. (2014). The separation of between-person and within-person components of individual change over time: A latent curve model with structured residuals. Journal of Consulting and Clinical Psychology, 82(5), 879–894.

    Google Scholar 

  • Dormann, C., & Griffin, M. (2015). Optimal time lags in panel studies. Psychological Methods, 20(4), 489–505.

    Google Scholar 

  • Engle, R. W. (2002). Working memory capacity as executive attention. Current Directions in Psychological Science, 11(1), 19–23.

    Google Scholar 

  • Engle, R. W., & Kane, M. J. (2004). Executive attention, working memory capacity, and a two-factor theory of cognitive control. In B. H. Ross (Ed.), The psychology of learning and motivation: Advances in research and theory (Vol. 44, pp. 145–199). Amsterdam: Elsevier Science.

    Google Scholar 

  • Engle, R. W., Tuholski, S. W., Laughlin, J. E., & Conway, A. R. A. (1999). Working memory, short-term memory, and general fluid intelligence: A latent-variable approach. Journal of Experimental Psychology: General, 128(3), 309–331.

    Google Scholar 

  • Ericsson, K. A., & Kintsch, W. (1995). Long-term working memory. Psychological Review, 102(2), 211–245.

    Google Scholar 

  • Feldon, D. F. (2007). Implications of research on expertise for curriculum and pedagogy. Educational Psychology Review, 19, 91–110.

  • Feldon, D. F. (2012). Validity of learning. In N. Seel (Ed.), Encyclopedia of the sciences of learning (pp. 3381–3383). New York: Springer.

    Google Scholar 

  • Fenesi, B., Sana, F., Kim, J., & Shore, D. (2015). Reconceptualizing working memory in educational research. Educational Psychology Review, 27(2), 333–351.

    Google Scholar 

  • Fraley, R. C., Roisman, G., & Haltigan, J. (2013). The legacy of early experiences in development: Formalizing alternative models of how early experiences are carried forward over time. Developmental Psychology, 49(1), 109–126.

    Google Scholar 

  • Friso-van den Bos, I., van der Ven, S., Kroesbergen, E., & van Luit, J. (2013). Working memory and mathematics in primary school children: A meta-analysis. Educational Research Review, 10, 29–44.

    Google Scholar 

  • Geary, D. C., Nicholas, A., Li, Y., & Sun, J. (2017). Developmental change in the influence of domain-general abilities and domain-specific knowledge on mathematics achievement: An eight-year longitudinal study. Journal of Educational Psychology, 109(5), 680–693.

    Google Scholar 

  • Gobet, F., & Simon, H. A. (1996). Templates in chess memory: A mechanism for recalling several boards. Cognitive Psychology, 31(1), 1–40.

    Google Scholar 

  • Hamaker, E., Nesselroade, J., & Molenaar, P. (2007). The integrated trait-state model. Journal of Research in Personality, 41(2), 295–315.

  • Hamaker, E., Kuiper, R., & Grasman, R. (2015). A critique of the cross-lagged panel model. Psychological Methods, 20(1), 102–116.

  • Hoffman, L. (2015). Longitudinal analysis: Modeling within-person fluctuation and change. New York: Routledge.

  • Horn, J., Donaldson, G., & Engstrom, R. (1981). Apprehension, memory, and fluid intelligence decline in adulthood. Research on Aging, 3(1), 33–84.

    Google Scholar 

  • Jöreskog, K., & Long, J. (1993). Testing structural equation models. Thousand Oaks, CA: SAGE.

    Google Scholar 

  • Kalyuga, S. (2007). Expertise reversal effect and its implications for learner-tailored instruction. Educational Psychology Review, 19, 509–539.

  • Kalyuga, S. (2011). Cognitive load theory: How many types of load does it really need? Educational Psychology Review, 23(1), 1–19.

    Google Scholar 

  • Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). The expertise reversal effect. Educational Psychologist, 38(1), 23–31.

    Google Scholar 

  • Kane, M. J., & Engle, R. W. (2002). The role of prefrontal cortex in working-memory capacity, executive attention, and general fluid intelligence: An individual-differences perspective. Psychonomic Bulletin & Review, 9(4), 637–671.

    Google Scholar 

  • Kane, M. J., Hambrick, D. Z., Tuholski, S. W., Wilhelm, O., Payne, T. W., & Engle, R. W. (2004). The generality of working memory capacity: A latent-variable approach to verbal and visuospatial memory span and reasoning. Journal of Experimental Psychology: General, 133(2), 189–217.

    Google Scholar 

  • Kane, M., Conway, A., Hambrick, D., & Engle, R. (2007). Variation in working memory capacity as variation in executive attention and control. In A. Conway, C. JArrold, M. Kane, A. Miyake, & J. Towse (Eds.), Variation in working memory (pp. 21–48). New York: Oxford University Press.

    Google Scholar 

  • Kline, R. (2016). Principles and practice of structural equation modeling (4th ed.). New York: Guilford Press.

    Google Scholar 

  • Korbach, A., Brunken, R., & Park, B. (2016). Learner characteristics and information processing in multimedia learning: A moderated mediation of the seductive details effect. Learning and Individual Differences, 51, 59–68.

  • Kovacs, K., & Conway, A. R. A. (2016). Process overlap theory: A unified account of the general factor of intelligence. Psychological Inquiry, 27(3), 151–177.

    Google Scholar 

  • Kuhn, T. S. (1970). The structure of scientific revolutions. Chicago: University of Chicago Press.

    Google Scholar 

  • Kuiper, R., & Ryan, O. (2018). Drawing conclusions from cross-lagged relationships: Re-considering the role of the time-interval. Structural Equation Modeling, 25(5), 809–823.

    Google Scholar 

  • Lehmann, J., Goussios, C., & Seufert, T. (2016). Working memory capacity and disfluency effect: An aptitude-treatment-interaction study. Metacognition and Learning, 11(1), 89–105.

  • Logie, R. (2016). Retiring the central executive. Quarterly Journal of Experimental Psychology, 69, 2093–2109.

  • MacCallum, R. C., Roznowski, M., & Necowitz, L. B. (1992). Model modifications in covariance structure analysis: The problem of capitalization on chance. Psychological Bulletin, 111(3), 490–504.

    Google Scholar 

  • Masunaga, H., & Horn, J. (2000). Characterizing mature human intelligence Expertise development. Learning and Individual Differences, 12(1), 5–33.

    Google Scholar 

  • Masunaga, H., & Horn, J. (2001). Expertise and age-related changes in components of intelligence. Psychology and Aging, 16(2), 293–311.

    Google Scholar 

  • Molenaar, P. (2004). A manifesto on psychology as idiographic science: Bringing the person back into scientific psychology, this time forever. Measurement, 2(4), 201–218.

  • Miller-Cotto, D., & Byrnes, J. P. (2020). What’s the best way to characterize the relationship between working memory and achievement?: An initial examination of competing theories. Journal of Educational Psychology, 112(5), 1074–1084.

    Google Scholar 

  • Munafo, M., Nosek, B., Bishop, D., Button, K., Chambers, C., et al. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1, 1–9.

    Google Scholar 

  • Muthen, L. K., & Muthen, B. O. (1998-2019). Mplus user’s guide [Computer software manual]. Los Angeles: Muthen & Muthen.

  • Najarian, M., Tourangeau, K., Nord, C., & Wallner-Allen, K. (2015). Early Childhood Longitudinal Study, Kindergarten Class of 2010–11 (ECLSK:2011), Kindergarten Psychometric Report. National Center for Education Statistics, Institute of Education Sciences, U.S. DC: Department of Education. Washington.

  • Najarian, M., Tourangeau, K., Nord, C., & Wallner-Allen, K. (2018). Early Childhood Longitudinal Study, Kindergarten Class of 2010–11 (ECLS-K:2011), Kindergarten Psychometric Report (NCES 2018-182). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education, Washington DC. Retrieved from http://nces.ed.gov/pubsearch.

  • Nosek, B., Alter, G., Banks, G., Borsboom, D., Bowman, S., et al. (2015). Promoting an open research culture. Science, 348(6242), 1422–1425.

    Google Scholar 

  • Paas, F., & Sweller, J. (2012). An evolutionary upgrade of cognitive load theory: Using the human motor system and collaboration to support the learning of complex cognitive tasks. Educational Psychology Review, 24(1), 27–45.

    Google Scholar 

  • Paas, F., & Sweller, J. (2014). Implications of cognitive load theory for multimedia learning. In R. E. Mayer (Ed.), Cambridge handbooks in psychology. The Cambridge handbook of multimedia learning (pp. 27–42). Cambridge: Cambridge University Press.

    Google Scholar 

  • Robison, M. K., Miller, A. L., & Unsworth, N. (2018). Individual differences in working memory capacity and filtering. Journal of Experimental Psychology: Human Perception and Performance, 44(7), 1038–1053.

    Google Scholar 

  • Salden, R., Aleven, V., Schwonke, R., & Renkl, A. (2010). The expertise reversal effect and worked examples in tutored problem solving. Instructional Science, 38, 289–307.

  • Satorra, A., & Bentler, P. M. (2010). Ensuring positiveness of the scaled difference chi-square test statistic. Psychometrika, 75(2), 243–248.

    Google Scholar 

  • Schermelleh-Engel, K., Moosbrugger, H., & Müller, H. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8(2), 23–74.

    Google Scholar 

  • Schweppe, J., & Rummer, R. (2014). Attention, working memory, and long-term memory in multimedia learning: An integrated perspective based on process models of working memory. Educational Psychology Review, 26(2), 285–306.

    Google Scholar 

  • Stanovich, K. (2009). Matthew effects in reading: Some consequences of individual differences in the acquisition of literacy. Journal of Education, 189(1-2), 23–55.

    Google Scholar 

  • Stelzl, I. (1986). Changing a causal hypothesis without changing the fit: Some rules for generating equivalent path models. Multivariate Behavioral Research, 21(3), 309–331.

    Google Scholar 

  • Swanson, H. L., & Fung, W. (2016). Working memory components and problem-solving accuracy: Are there multiple pathways? Journal of Educational Psychology, 108(8), 1153–1177.

    Google Scholar 

  • Sweller, J. (2011). Cognitive load theory. In J. P. Mestre & B. H. Ross (Eds.), The psychology of learning and motivation: Vol. 55. The psychology of learning and motivation: Cognition in education (pp. 37–76). Amsterdam: Elsevier Academic Press.

    Google Scholar 

  • Sweller, J. (2012). Human cognitive architecture: Why some instructional procedures work and others do not. In K. R. Harris, S. Graham, T. Urdan, C. B. McCormick, G. M. Sinatra, & J. Sweller (Eds.), APA handbooks in psychology®. APA educational psychology handbook, Vol. 1. Theories, constructs, and critical issues (pp. 295–325). Washington DC: American Psychological Association.

    Google Scholar 

  • Sweller, J., van Merrienboer, J. J. G., & Paas, F. G. W. C. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251–296.

    Google Scholar 

  • Sweller, J., van Merriënboer, J. J. G., & Paas, F. (2019). Cognitive architecture and instructional design: 20 years later. Educational Psychology Review, 31(2), 261–292.

    Google Scholar 

  • Tourangeau, K., Nord, C., Lê, T., Sorongon, A. G., Hagedorn, M. C., et al. (2015a). Early Childhood Longitudinal Study, Kindergarten Class of 2010–11 (ECLS-K:2011) User’s Manual for the ECLS-K:2011 Kindergarten Data File and Electronic Codebook, Public Version (NCES 2015-074). Washington, DC: National Center for Education Statistics.

    Google Scholar 

  • Tourangeau, K., Nord, C., Lê, T., Wallner-Allen, K., Hagedorn, M. C., Leggitt, J., Najarian, M., & Mulligan, G. (2015b). Early Childhood Longitudinal Study, Kindergarten Class of 2010–11 (ECLS-K:2011), User’s Manual for the ECLS-K:2011 Kindergarten–First Grade Data File and Electronic Codebook, Public Version (NCES 2015-078). U.S. Department of Education. Washington, DC: National Center for Education Statistics.

    Google Scholar 

  • Tourangeau, K., Nord, C., Lê, T., Wallner-Allen, K., Vaden-Kiernan, N., Blaker, L., Najarian, M., & Mulligan, G. (2017). Early Childhood Longitudinal Study, Kindergarten Class of 2010–11 (ECLS-K:2011) User’s Manual for the ECLS-K:2011 Kindergarten–Second Grade Data File and Electronic Codebook, Public Version (NCES 2017-285). U.S. Department of Education. Washington, DC: National Center for Education Statistics.

    Google Scholar 

  • Turner, M. L., & Engle, R. W. (1989). Is working memory capacity task dependent? Journal of Memory and Language, 28(2), 127–154. https://doi.org/10.1016/0749-596X(89)90040-5.

    Article  Google Scholar 

  • Unsworth, N. (2010). Interference control, working memory capacity, and cognitive abilities: A latent variable analysis. Intelligence, 38(2), 255–267.

    Google Scholar 

  • Unsworth, N., & Engle, R. W. (2007). The nature of individual differences in working memory capacity: Active maintenance in primary memory and controlled search from secondary memory. Psychological Review, 114(1), 104–132.

    Google Scholar 

  • Unsworth, N., McMillan, B. D., Brewer, G. A., & Spillers, G. J. (2012). Everyday attention failures: An individual differences investigation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 38(6), 1765–1772.

    Google Scholar 

  • Usami, S., Murayama, K., & Hamaker, E. (2019). A unified framework of longitudinal models to examine reciprocal relations. Psychological Methods, 24(5), 637–657.

    Google Scholar 

  • van Merriënboer, J. J. G., & Sweller, J. (2005). Cognitive load theory and complex learning: Recent developments and future directions. Educational Psychology Review, 17(2), 147–177.

    Google Scholar 

  • Voelkle, M., Brose, A., Schmiedek, F., & Lindenberger, U. (2014). Toward a unified framework for the study of between-person and within-person structures: Building a bridge between two research paradigms. Multivariate Behavioral Research, 49, 193–213.

  • Wiemers, E., & Redick, T. (2018). Working memory capacity and intra-individual variability of proactive control. Acta Psychologica, 182, 21–31.

    Google Scholar 

  • Woodcock, R. W., McGrew, K. S., & Mather, N. (2001). Woodcock-Johnson III Tests of Cognitive Abilities. Riverside Publishing.

Download references

Acknowledgements

The authors would like to express their gratitude to Dr. John Sweller and Dr. James Peugh for their insightful comments on previous versions of this manuscript.

Author information

Authors and Affiliations

  1. Utah State University, 2830 Old Main Hill, Logan, UT, 84322-2830, USA

    David F. Feldon & Kaylee Litson

Authors
  1. David F. Feldon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kaylee Litson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David F. Feldon.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Feldon, D.F., Litson, K. Modeling Theories and Theorizing Models: an Attempted Replication of Miller-Cotto & Byrnes’ (2019) Comparison of Working Memory Models Using ECLS-K Data. Educ Psychol Rev 33, 1907–1934 (2021). https://doi.org/10.1007/s10648-021-09596-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10648-021-09596-8

Keywords

Search

Navigation