Statistical Analysis: Getting to Insight Through Collaboration and Critical Thinking

  • Matthew LineberryEmail author
  • David A. Cook


Statistical analyses are key for deriving important insights from quantitative data in educational research. While the technical aspects of statistics can seem daunting, and expert consultation is well-advised, we do not advise thinking of analysis as a task to be “handed off” to a statistician after data is collected. Instead, analyses are best completed in close collaboration with a broad research team beginning early in the conceptualization of the research. This chapter outlines foundational concepts in statistics with the goal of familiarizing non-statisticians with key terms and concepts. We also share tips and tricks for running analyses and writing about your findings. Our hope is that readers who consider themselves statistically “challenged” will appreciate that their critical thinking about analyses can be essential to conducting sound and insightful research, provided a basic understanding of content and a team-centered approach.


Statistics Analysis Quantitative Psychometrics Simulation 



Null hypothesis statistical testing


  1. 1.
    Cook DA, Bordage G, Schmidt HG. Description, justification and clarification: a framework for classifying the purposes of research in medical education. Med Educ. 2008;42(2):128–33.CrossRefGoogle Scholar
  2. 2.
    Rosenthal R. The file drawer problem and tolerance for null results. Psychol Bull. 1979;86(3):638–41.CrossRefGoogle Scholar
  3. 3.
    Silberzahn R, Uhlmann EL, Martin D, Anselmi P, Aust F, Awtrey EC, et al. Many analysts, one dataset: making transparent how variations in analytical choices affect results. PsyArXiv [Internet]. 2017 24 [cited 2018 Jan 19]. Available from:
  4. 4.
    Gelman A, Loken E. The garden of forking paths: why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time. Unpublished Manuscript [Internet]. 2013. Available from:
  5. 5.
    Norman GR, Streiner DL. PDQ statistics. 3rd ed. Hamilton: People’s Medical Publishing House; 2003. p. 218.Google Scholar
  6. 6.
    Motulsky H. Essential biostatistics [Internet]. Oxford: Oxford University Press; 2016 [cited 2018 Aug 30]. Available from:
  7. 7.
    Morey RD, Hoekstra R, Rouder JN, Lee MD, Wagenmakers E-J. The fallacy of placing confidence in confidence intervals. Psychon Bull Rev. 2016;23(1):103–23.CrossRefGoogle Scholar
  8. 8.
    Lineberry M, Walwanis M, Reni J. Comparative research on training simulators in emergency medicine: a methodological review. Simul Healthc J Soc Simul Healthc. 2013;8(4):253–61.CrossRefGoogle Scholar
  9. 9.
    Murphy K. Using power analysis to evaluate and improve research. In: Rogelberg SG, editor. Handbook of research methods in industrial organizational psychology. Malden: Blackwell Publishers; 2002. p. 119–37.Google Scholar
  10. 10.
    Ioannidis JPA. The proposal to lower p value thresholds to.005. JAMA. 2018;319(14):1429–30.CrossRefGoogle Scholar
  11. 11.
    Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. New York: Taylor & Francis; 1988.Google Scholar
  12. 12.
    Cohen J. The earth is round (p<. 05). Am Psychol. 1994;49(12):997–1003.CrossRefGoogle Scholar
  13. 13.
    Miller J. What is the probability of replicating a statistically significant effect? Psychon Bull Rev. 2009;16(4):617–40.CrossRefGoogle Scholar
  14. 14.
    Meehl PE. High school yearbooks: a reply to Schwarz. J Abnorm Psychol. 1971;77(2):143–8.CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Zamierowski Institute for Experiential LearningUniversity of Kansas Medical Center and Health SystemKansas CityUSA
  2. 2.Mayo Clinic Multidisciplinary Simulation Center, Office of Applied Scholarship and Education Science, and Division of General Internal MedicineMayo Clinic College of Medicine and ScienceRochesterUSA

Personalised recommendations