Skip to main content
SpringerLink
Log in

Research Replication

  • Chapter
The Palgrave Handbook of Applied Linguistics Research Methodology

Abstract

This chapter traces the history of replication research in the field of applied linguistics, culminating in a discussion of current views of replication research as a means of evaluating the internal and external validity of a study, illuminating phenomena of interest, and ultimately, driving both theory and pedagogy forward. It provides an overview of different types of replication studies (exact, approximate, conceptual) with recent examples from the field. Challenges concerning the interpretation of replication results, as well as ongoing controversies over the replication of qualitative studies, are discussed. The author concludes with current recommendations for facilitating replication research, including those pertaining to reporting and data sharing.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 299.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 379.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 379.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Abbuhl, R. (2012a). Practical methods for teaching replication to applied linguistics studies. In G. Porte (Ed.), Replication research in applied linguistics (pp. 135–150). Cambridge: Cambridge University Press.

    Google Scholar 

  • Abbuhl, R. (2012b). When, when, and how to replicate research. In A. Mackey & S. Gass (Eds.), Research methods in second language acquisition (pp. 296–312). Oxford: Wiley-Blackwell.

    Chapter  Google Scholar 

  • Basturkmen, H. (2014). Replication research in comparative genre analysis in English for academic purposes. Language Teaching, 47(3), 377–386.

    Article  Google Scholar 

  • Benson, L., & Borrego, M. (2015). The role of replication in engineering education research. Journal of Engineering Education, 104(4), 388–392.

    Article  Google Scholar 

  • Bikowski, D., & Schulze, M. (2015). Replication and evaluation in CALL. CALICO Journal, 32(2), i–v.

    Article  Google Scholar 

  • Bitchener, J., & Knoch, U. (2015). Written corrective feedback studies: Approximate replication of Bitchener & Knoch (2010a) and Van Beuningen, De Jong & Kuiken (2012). Language Teaching, 48(3), 405–414.

    Article  Google Scholar 

  • Booth, P. (2013). Vocabulary knowledge in relation to memory and analysis: An approximate replication of Milton’s (2007) study on lexical profiles and learning style. Language Teaching, 46(3), 335–354.

    Article  Google Scholar 

  • Bronstein, R. (1990). Publication politics, experimenter bias and the replication process in social science research. Journal of Social Behavior and Personality, 5(4), 71–81.

    Google Scholar 

  • Burman, L., Reed, W., & Alm, J. (2010). A call for replication studies. Public Finance Review, 38(6), 787–793.

    Article  Google Scholar 

  • Casanave, C. (2012). Heading in the wrong direction? A response to Porte and Richards. Journal of Second Language Writing, 21(3), 296–297.

    Article  Google Scholar 

  • Chun, D. (2012). Replication studies in CALL research. CALICO Journal, 29(4), 591–600.

    Article  Google Scholar 

  • Crandall, C., & Sherman, J. (2016). On the scientific superiority of conceptual replications for scientific progress. Journal of Experimental Social Psychology, 66, 93–99.

    Article  Google Scholar 

  • Earp, B., & Trafimow, D. (2015). Replication, falsification, and the crisis of confidence in social psychology. Frontiers in Psychology, 6(621), 1–11.

    Google Scholar 

  • Easley, R., Madden, C., & Dunn, M. (2000). Conducting market science: The role of replication in the research process. Journal of Business Research, 48(1), 83–92.

    Article  Google Scholar 

  • Frankenberg-Garcia, A. (2014). The use of corpus examples for language comprehension and production. ReCALL, 26(2), 128–146.

    Article  Google Scholar 

  • Gass, S., & Valmori, L. (2015). Replication in interaction and working memory research: Révész (2012) and Goo (2012). Language Teaching, 48(4), 545–555.

    Article  Google Scholar 

  • Golden, M. (1995). Replication and non-quantitative research. PS: Political Science and Politics, 28(3), 481–483.

    Google Scholar 

  • Hendrik, C. (1990). Replications, strict replications, and conceptual replications: Are they important? Journal of Social Behavior and Personality, 5(4), 41–49.

    Google Scholar 

  • Hubbard, R., & Armstrong, J. (1994). Replications and extensions in marketing: Rarely published but quite contrary. International Journal of Research in Marketing, 11(3), 233–248.

    Article  Google Scholar 

  • Ioannidis, J. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124.

    Article  Google Scholar 

  • John, L., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23(5), 524–532.

    Article  Google Scholar 

  • Johnson, M., Mercado, L., & Acevedo, A. (2012). The effect of planning sub-processes on L2 writing fluency, grammatical complexity, and lexical complexity. Journal of Second Language Writing, 21(3), 264–282.

    Article  Google Scholar 

  • Johnson, M., & Nicodemus, C. (2015). Testing a threshold: An approximate replication of Johnson, Mercado & Acevedo 2012. Language Teaching, 49(2), 251–274.

    Article  Google Scholar 

  • King, K., & Mackey, A. (2016). Research methodology in second language studies: Trends, concerns, and new directions. Modern Language Journal, 100(s1), 209–227.

    Article  Google Scholar 

  • Klein, R. A., Ratliff, R. A., Vianello, M., Adams, R. A., Jr., Bahník, S., Bernstein, M. J., … Nosek, B. A. (2014). Investigating variation in replicability: A “ManyLabs” replication project. Social Psychology, 45(3), 142–152.

    Article  Google Scholar 

  • Language Teaching Review Panel. (2008). Replication studies in language learning and teaching: Questions and answers. Language Teaching, 41(1), 1–14.

    Google Scholar 

  • Larson-Hall, J., & Plonsky, L. (2015). Reporting and interpreting quantitative research findings: What gets reported and recommendations for the field. Language Learning, 65(S1), 127–159.

    Article  Google Scholar 

  • LeBel, E. (2015). A new replication norm for psychology. Collabra, 1(1), 1–13.

    Article  Google Scholar 

  • Leow, R. (2015). The roles of attention and (un)awareness in SLA: Conceptual replication of N.C. Ellis & Sagarra (2010a) and Leung & Williams (2012). Language Teaching, 48(1), 117–129.

    Article  Google Scholar 

  • Lim, H., & Godfroid, A. (2015). Automatization in second language sentence processing: A partial, conceptual replication of Hulstijn, Van Gelderen, and Schoonen’s 2009 study. Applied Psycholinguistics, 36(5), 1247–1282.

    Article  Google Scholar 

  • Liu, Q., & Brown, D. (2015). Methodological synthesis of research on the effectiveness of corrective feedback in L2 writing. Journal of Second Language Writing, 30, 66–81.

    Article  Google Scholar 

  • Lu, X. (2011). A corpus-based evaluation of syntactic complexity measures as indices of college-level ESL writers’ language development. TESOL Quarterly, 45(1), 36–62.

    Article  Google Scholar 

  • Mackey, A., & Marsden, E. (2016). Advancing methodology and practice: The IRIS repository of instruments for research into second languages. New York: Routledge.

    Google Scholar 

  • Makel, M., & Plucker, J. (2015). An introduction to replication research in gifted education: Shiny and new is not the same as useful. Gifted Child Quarterly, 59(3), 157–164.

    Article  Google Scholar 

  • Makel, M., Plucker, J., & Hegarty, B. (2012). Replications in psychology research: How often do they really occur? Perspectives on Psychological Science, 7(6), 532–542.

    Article  Google Scholar 

  • Markee, N. (2017). Are replication studies possible in qualitative second/foreign language classroom research? A call for comparative re-production research. Language Teaching, 50(3), 367–383.

    Article  Google Scholar 

  • Marsden, E., & Mackey, A. (2013, June). IRIS and replication. Paper presented at the 9th International Symposium on Bilingualism, Singapore.

    Google Scholar 

  • Marsden, E., Mackey, A., & Plonsky, L. (2016). The IRIS repository: Advancing research practice and methodology. In A. Mackey & E. Marsden (Eds.), Advancing methodology and practice: The IRIS repository for research into second languages (pp. 1–21). New York: Routledge.

    Google Scholar 

  • Matsuda, P. (2012). On the nature of second language writing: Replication in a postmodern field. Journal of Second Language Writing, 21(3), 300–302.

    Article  Google Scholar 

  • Maxwell, S., Lau, M., & Howard, G. (2015). Is psychology suffering from a replication crisis? What does “failure to replicate” really mean? American Psychologist, 70(6), 487–498.

    Article  Google Scholar 

  • McNeeley, S., & Warner, J. (2015). Replication in criminology: A necessary practice. European Journal of Criminology, 12(5), 581–597.

    Article  Google Scholar 

  • Mezias, S., & Regnier, M. (2007). Walking the walk as well as talking the talk: Replication and the normal science paradigm in strategic management research. Strategic Organization, 5(3), 283–296.

    Article  Google Scholar 

  • Moravcsik, A. (2014). Transparency: The revolution in qualitative research. PS: Political Science & Politics, 47(1), 48–53.

    Google Scholar 

  • Mu, C., & Matsuda, P. (2016). Replication in L2 writing research: Journal of second language writing authors’ perceptions. TESOL Quarterly, 50(1), 201–219.

    Article  Google Scholar 

  • Nassaji, H. (2012). Significance tests and generalizability of research results: A case for replication. In G. Porte (Ed.), Replication research in applied linguistics (pp. 92–115). Cambridge: Cambridge University Press.

    Google Scholar 

  • Neuliep, J., & Crandall, R. (1990). Editorial bias against replication research. Journal of Social Behavior and Personality, 5(4), 85–90.

    Google Scholar 

  • Open Science Collaboration. (2012). An open, large-scale, collaborative effort to estimate the reproducibility of psychological science. Perspectives on Psychological Science, 7(6), 657–660.

    Article  Google Scholar 

  • Pashler, H., & Harris, C. (2012). Is the replicability crisis overblown? Three arguments examined. Perspectives on Psychological Science, 7(6), 531–536.

    Article  Google Scholar 

  • Pashler, H., & Wagenmakers, E.-J. (2012). Editors’ introduction to the special section on replicability in psychological science: A crisis of confidence? Perspectives on Psychological Science, 7(6), 528–530.

    Article  Google Scholar 

  • Plonsky, L. (2012). Replication, meta-analysis, and generalizability. In G. Porte (Ed.), Replication research in applied linguistics (pp. 116–132). Cambridge: Cambridge University Press.

    Google Scholar 

  • Plonsky, L. (2013). Study quality in SLA: An assessment of designs, analyses, and reporting practices in quantitative L2 research. Studies in Second Language Acquisition, 35(4), 655–687.

    Article  Google Scholar 

  • Plonsky, L. (2015). Quantitative considerations for improving replicability in CALL and applied linguistics. CALICO Journal, 32(2), 232–244.

    Google Scholar 

  • Plonsky, L., Egbert, J., & Laflair, G. (2015). Bootstrapping in applied linguistics: Assessing its potential using shared data. Applied Linguistics, 36(5), 591–610.

    Google Scholar 

  • Polio, C. (2012a). No paradigm wars please! Journal of Second Language Writing, 21(3), 294–295.

    Article  Google Scholar 

  • Polio, C. (2012b). Replication in published applied linguistics research: An historical perspective. In G. Porte (Ed.), Replication research in applied linguistics (pp. 47–91). Cambridge: Cambridge University Press.

    Google Scholar 

  • Polio, C., & Gass, S. (1997). Replication and reporting: A commentary. Studies in Second Language Acquisition, 19(4), 499–508.

    Article  Google Scholar 

  • Porte, G. (2012). Replication research in applied linguistics. Cambridge: Cambridge University Press.

    Google Scholar 

  • Porte, G. (2013). Who needs replication research? CALICO Journal, 30(1), 10–15.

    Article  Google Scholar 

  • Porte, G., & Richards, K. (2012). Focus article: Replication in second language writing research. Journal of Second Language Writing, 21(3), 284–293.

    Article  Google Scholar 

  • Reis, H., & Lee, K. (2016). Promise, peril, and perspective: Addressing concerns about reproducibility in social-personality psychology. Journal of Experimental Social Psychology, 66, 148–152.

    Article  Google Scholar 

  • Richards, K. (2009). Trends in qualitative research in language teaching since 2000. Language Teaching, 42(2), 147–180.

    Article  Google Scholar 

  • Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641.

    Article  Google Scholar 

  • Rott, S., & Gavin, B. (2015). Comprehending and learning from Internet sources: A conceptual replication study of Goldman, Braasch, Wiley, Greasser and Brodowinska (2012). CALICO Journal, 32(2), 323–354.

    Article  Google Scholar 

  • Sakaluk, J. (2016). Exploring small, confirming big: An alternative system to the new statistics for advancing cumulative and replicable psychological research. Journal of Experimental Social Psychology, 66, 47–54.

    Article  Google Scholar 

  • Santos, T. (1989). Replication in applied linguistics research. TESOL Quarterly, 23(4), 699–702.

    Article  Google Scholar 

  • Sasaki, M. (2012). An alternative approach to replication studies in second language writing: An ecological perspective. Journal of Second Language Writing, 21(3), 303–305.

    Article  Google Scholar 

  • Schmitt, N., Cobb, T., Horst, M., & Schmitt, D. (2017). How much vocabulary is needed to use English? Replication of van Zeeland & Schmitt (2012), Nation (2006) and Cobb (2007). Language Teaching, 50(2), 212–226.

    Article  Google Scholar 

  • Schofield, J. (2002). Increasing the generalizability of qualitative research. In A. Huberman & M. Miles (Eds.), The qualitative researcher’s companion (pp. 171–203). Thousand Oaks, CA: SAGE.

    Google Scholar 

  • Schweinsberg, M., Madan, N., Vianello, M., Amy Sommer, S., Jordan, J., Tierney, W., … Uhlmann, E. L. (2016). The pipeline project: Pre-publication independent replications of a single laboratory’s research pipeline. Journal of Experimental Social Psychology, 66, 55–67.

    Article  Google Scholar 

  • Simmons, J., Nelson, L., & Simonsohn, U. (2011). False-positive psychology undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366.

    Article  Google Scholar 

  • Smith, B., & Schulze, M. (2013). Thirty years of the CALICO Journal—Replicate, replicate, replicate. CALICO Journal, 30(1), i–iv.

    Article  Google Scholar 

  • Stake, R. (2005). Qualitative case studies. In N. Denzin & Y. Lincoln (Eds.), Handbook of qualitative research (pp. 443–466). Thousand Oaks, CA: SAGE.

    Google Scholar 

  • Valdman, A. (1993). Replication study. Studies in Second Language Acquisition, 15(4), 505.

    Article  Google Scholar 

  • Webb, S. (2015). Learning vocabulary through meaning-focused input: Replication of Elley (1989) and Liu & Nation (1985). Language Teaching, 49(1), 129–140.

    Article  Google Scholar 

  • Willans, F., & Leung, C. (2016). Empirical foundations for medium of instruction policies: Approximate replications of Afolayan (1976) and Siegel (1997b). Language Teaching, 49(4), 549–563.

    Article  Google Scholar 

  • Yoon, H.-J., & Polio, C. (2017). The linguistic development of students of English as a second language in two written genres. TESOL Quarterly, 51(2), 275–301.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Linguistics, California State University at Long Beach, Long Beach, CA, USA

    Rebekha Abbuhl

Authors
  1. Rebekha Abbuhl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rebekha Abbuhl .

Editor information

Editors and Affiliations

  1. Sydney School of Education and Social Work, University of Sydney, Sydney, NSW, Australia

    Aek Phakiti

  2. Department of Linguistics, Germanic, Slavic, Asian and African Languages, Michigan State University, East Lansing, MI, USA

    Peter De Costa

  3. Applied Linguistics, Northern Arizona University, Flagstaff, AZ, USA

    Luke Plonsky

  4. School of Education, UNSW Sydney, Sydney, NSW, Australia

    Sue Starfield

Copyright information

© 2018 The Author(s)

About this chapter

Cite this chapter

Abbuhl, R. (2018). Research Replication. In: Phakiti, A., De Costa, P., Plonsky, L., Starfield, S. (eds) The Palgrave Handbook of Applied Linguistics Research Methodology. Palgrave Macmillan, London. https://doi.org/10.1057/978-1-137-59900-1_7

Download citation

  • DOI: https://doi.org/10.1057/978-1-137-59900-1_7

  • Publisher Name: Palgrave Macmillan, London

  • Print ISBN: 978-1-137-59899-8

  • Online ISBN: 978-1-137-59900-1

  • eBook Packages: Social SciencesSocial Sciences (R0)

Publish with us

Policies and ethics

Search

Navigation