Skip to main content
SpringerLink
Log in

Replication and preregistration

  • Published:
Quality & Quantity Aims and scope Submit manuscript

Abstract

Replicability of an experiment’s results is necessary for relying on its results but is extremely difficult to achieve. The difficulty is that besides having the same valid measure across the intended replicates for each dimension and the same comprehensive case population sample coverage, each replicate must have the same fully sufficient set of causally relevant input and effect relevant outcome dimensions. Otherwise the un-included necessary input and outcome dimensions may be at different levels across the intended replicates. Case sample or dimension measure differences across intended replicates can have similar consequences. To ensure all the necessary comparability and so accurate replication of an experiment preregistration and precise oversight of its intended replicates are necessary for every attempt to replicate an experiment in order to promote complete conformity to all the necessary features of its design.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Allen, M., Preiss, R.: Replication and meta-analysis: a necessary connection. J. Soc. Behav. Pers. 8(6), 9–20 (1993)

    Google Scholar 

  • Anderson, S.E., Maxwell, S.E.: There’s more than one way to conduct a replication study: beyond statistical significance. Psychol. Methods 21(1), 1–12 (2015)

    Article  Google Scholar 

  • Bissell, M.: The risks of the replication drive. Nature 503, 333–334 (2013)

    Article  Google Scholar 

  • Bosson, J.K., Swann Jr., W.B., Pennebaker, J.W.: Stalking the perfect measure of implicit self-esteem: the blind men and the elephant revisited? J. Pers. Soc. Psychol. 79(4), 631–643 (2000)

    Article  Google Scholar 

  • Brandt, M.J., Ijzerman, H., Dijksterhuis, A., Farach, F.J., Geller, J., Giner-Sorolla, R., et al.: The replication recipe: what makes for a convincing replication? J. Exp. Soc. Psychol. 50, 217–224 (2014)

    Article  Google Scholar 

  • Bullock, R.J., Svyantek, D.J.: Analyzing meta-analysis: potential problems, an unsuccessful replication, and evaluation criteria. J. Appl. Psychol. 70(1), 108–115 (1985)

    Article  Google Scholar 

  • Cybulski, L., Mayo-Wilson, E., Grant, S.: Improving transparency and reproducibility through registration: the status of intervention trials published in clinical psychology journals. J. Consult. Clin. Psychol. 84(9), 753–767 (2016)

    Article  Google Scholar 

  • de Angelis, C., Drazen, J.M., Frizelle, F.A., Haug, C., Hoey, J., Horton, R., Van der Weyden, M.B.: Clinical trial registration: a statement from the international Committee of Medical Journal Editors. N. Engl. J. Med. 351, 1250–1251 (2004)

    Article  Google Scholar 

  • Franco, A., Malhotra, N., Simonovits, G.: Publication bias in the social sciences: unlocking the file drawer. Science 345, 1502–1505 (2014)

    Article  Google Scholar 

  • Glaser, B.G., Holton, J.: Remodeling grounded theory. Forum. Qualit. Soc. Res. 5(2), 1–22 (2004)

    Google Scholar 

  • Hedges, L.V., Schauer, J.M.: Statistical analyses for studying replication: meta-analytic perspectives. Advance online publication, Psychological Methods (2018). https://doi.org/10.1037/met0000189

    Book  Google Scholar 

  • Hohn, R.E., Slaney, K.L., Tafreshi, D.: Primary study quality in psychological meta-analysis: an empirical assessment of recent practice. Front. Psychol. (2019). https://doi.org/10.3389/fpsyg.2018.02667

    Article  Google Scholar 

  • Hüffmeier, J., Mazei, J., Schultze, T.: Reconceptualizing replication as a sequence of different studies: a replication typology. J. Exp. Soc. Psychol. 66, 81–92 (2016)

    Article  Google Scholar 

  • Ioannidis, J.P., Greenland, S., Hlatky, M.A., Khoury, M.J., Macleod, M.R., Moher, D., et al.: Increasing value and reducing waste in research design, conduct, and analysis. The Lancet 383(9912), 166–175 (2014)

    Article  Google Scholar 

  • Klein, R.A., et al.: Many Labs 2: investigating variation in replicability across samples and settings. Adv. Methods Pract. Psychol. Sci. 1(4), 443–490 (2018)

    Article  Google Scholar 

  • Krause, M.S.: Trying to discover sufficient condition causes. Methodology 6, 59–70 (2010)

    Article  Google Scholar 

  • Krause, M.S.: Measurement validity is fundamentally a matter of definition, not correlation. Rev. Gen. Psychol. 16, 391–400 (2012)

    Article  Google Scholar 

  • Krause, M.S.: The incompatibility of achieving a fully specified linear model and assuming that residual dependent-variable variance is random. Qual. Quant. 47, 3201–3204 (2013)

    Article  Google Scholar 

  • Krause, M.S.: Mathematical expression and sampling issues of treatment-contrasts: beyond significance testing and meta-analysis to clinically useful research synthesis. Psychother. Res. 28(1), 58–75 (2018a)

    Article  Google Scholar 

  • Krause, M.S.: Associational versus correlational research study design and data analysis. Qual. Quant. 52(6), 2691–2707 (2018b). https://doi.org/10.1007/s11135-018-0687-8

    Article  Google Scholar 

  • Krause, M.S.: Randomness is problematic for social science research purposes. Qual. Quant. (2018c) (in press)

  • Lancaster, G.A., Dodd, S., Williamson, P.R.: Design and analysis of pilot studies: recommendations for good practice. J. Eval. Clin. Pract. 10(2), 307–312 (2004)

    Article  Google Scholar 

  • Lobchuk, M.M., Degner, L.F.: Patients with cancer and next-of-kin response comparability on physical and psychological symptom well-being: trends and measurement issues. Cancer Nurs. 25(5), 358–374 (2002)

    Article  Google Scholar 

  • Makel, M.C., Plucker, J.A., Hegarty, B.: Replications in psychology research: how often do they really occur? Perspect. Psychol. Sci. 7, 537–542 (2012)

    Article  Google Scholar 

  • Maxwell, S.E., Lau, M.Y., Howard, G.S.: Is Psychology suffering from a replication crisis? What does “failure to replicate” really mean? Am. Psychol. 70(6), 487–498 (2015)

    Article  Google Scholar 

  • McElreath, R., Smaldino, P.E.: Replication, communication, and the population dynamics of scientific discovery. PLoS ONE 10(8), e0136088 (2015)

    Article  Google Scholar 

  • Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K.M., Gerber, A., et al.: Promoting transparency in social science research. Science 343(6166), 30–31 (2014)

    Article  Google Scholar 

  • Munafò, M.R., Nosek, B.A., Bishop, D.V., Button, K.S., Chambers, C.D., Du Sert, N.P., et al.: A manifesto for reproducible science. Nat. Hum. Behav. 1(1), 0021 (2017)

    Article  Google Scholar 

  • Nickerson, R.S.: Null hypothesis significance testing: a review of an old and continuing controversy. Psychol. Methods 5, 241–301 (2000)

    Article  Google Scholar 

  • Nosek, B.A., Ebersole, C.R., DeHaven, A.C., Mellor, D.T.: The preregistration revolution. Proc. Natl. Acad. Sci. U.S.A. 115, 2600–2606 (2018). https://doi.org/10.1073/pnas.1708274114

    Article  Google Scholar 

  • Nuijten, M.B., van Assen, M.A.L.M., Veldkamp, C.L.S., Wicherts, J.M.: The replication paradox: combining studies can decrease accuracy of effect size estimates. Rev. Gen. Psychol. 19(2), 172–182 (2015)

    Article  Google Scholar 

  • Pearce, N.: Registration of protocols for observational research is unnecessary and would do more harm than good. Occup. Environ. Med. 68(2), 86–88 (2011)

    Article  Google Scholar 

  • Peat, G., Riley, R.D., Croft, P., Morley, K.I., Kyzas, P.A., Moons, K.G., et al.: Improving the transparency of prognosis research: the role of reporting, data sharing, registration, and protocols. PLoS Med. 11(7), e1001671 (2014)

    Article  Google Scholar 

  • Rubin, M.: An evaluation of four solutions to the forking paths problem: adjusted alpha, preregistration, sensitivity analyses, and abandoning the Neyman–Pearson approach. Rev. Gen. Psychol. 21(4), 321–329 (2017)

    Article  Google Scholar 

  • Ruzzene, A.: Drawing lessons from case studies by enhancing comparability. Philos. Soc. Sci. 42(1), 99–120 (2012)

    Article  Google Scholar 

  • Scheuch, E.K.: The cross-cultural use of sample surveys: problems of comparability. Hist. Soc. Res. 18(2), 104–138 (1993)

    Google Scholar 

  • Schmidt, F.L., Oh, I.-S.: The crisis of confidence in research findings in psychology: is lack of replication the real problem? Or is it something else? Arch. Sci. Psychol. 4, 32–33 (2016)

    Google Scholar 

  • Schmidt, S.: Replication. In: Makel, M.C., Plucker, J.A. (eds.) Towards a more perfect psychology: improving trust, accuracy, and transparency in research, pp. 233–253. APA, Washington, DC (2017)

    Chapter  Google Scholar 

  • Schulze, M.B., Kroke, A., Saracci, R., Boeing, H.: The effect of differences in measurement procedure on the comparability of blood pressure estimates in multi-center studies. Blood Press. Monit. 7(2), 95–104 (2002)

    Article  Google Scholar 

  • Stroebe, W., Strack, F.: The alleged crisis and the illusion of exact replication. Perspect. Psychol. Sci. 9(1), 59–71 (2014)

    Article  Google Scholar 

  • Tryon, W.W.: Replication is about effect size: comment on Maxwell, Lau, and Howard (2015). Am. Psychol. 77(3), 236–237 (2016)

    Article  Google Scholar 

  • van’t Veer, A.E., Giner-Sorolla, R.: Pre-registration in social psychology—a discussion and suggested template. J. Exp. Soc. Psychol. 67, 2–12 (2016)

    Article  Google Scholar 

  • Wagenmakers, E.J., Wetzels, R., Borsboom, D., van der Maas, H.L., Kievit, R.A.: An agenda for purely confirmatory research. Perspect. Psychol. Sci. 7(6), 632–638 (2012)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Evanston, IL, USA

    Merton S. Krause

Authors
  1. Merton S. Krause
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Merton S. Krause.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Krause, M.S. Replication and preregistration. Qual Quant 53, 2647–2652 (2019). https://doi.org/10.1007/s11135-019-00877-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11135-019-00877-7

Keywords

Search

Navigation