Statistical Analysis: Getting to Insight Through Collaboration and Critical Thinking
Statistical analyses are key for deriving important insights from quantitative data in educational research. While the technical aspects of statistics can seem daunting, and expert consultation is well-advised, we do not advise thinking of analysis as a task to be “handed off” to a statistician after data is collected. Instead, analyses are best completed in close collaboration with a broad research team beginning early in the conceptualization of the research. This chapter outlines foundational concepts in statistics with the goal of familiarizing non-statisticians with key terms and concepts. We also share tips and tricks for running analyses and writing about your findings. Our hope is that readers who consider themselves statistically “challenged” will appreciate that their critical thinking about analyses can be essential to conducting sound and insightful research, provided a basic understanding of content and a team-centered approach.
KeywordsStatistics Analysis Quantitative Psychometrics Simulation
Null hypothesis statistical testing
- 3.Silberzahn R, Uhlmann EL, Martin D, Anselmi P, Aust F, Awtrey EC, et al. Many analysts, one dataset: making transparent how variations in analytical choices affect results. PsyArXiv [Internet]. 2017 24 [cited 2018 Jan 19]. Available from: https://psyarxiv.com/qkwst/.
- 4.Gelman A, Loken E. The garden of forking paths: why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time. Unpublished Manuscript [Internet]. 2013. Available from: http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf.
- 5.Norman GR, Streiner DL. PDQ statistics. 3rd ed. Hamilton: People’s Medical Publishing House; 2003. p. 218.Google Scholar
- 6.Motulsky H. Essential biostatistics [Internet]. Oxford: Oxford University Press; 2016 [cited 2018 Aug 30]. Available from: https://global.oup.com/ushe/product/essential-biostatistics-9780199365067?cc=us&lang=en&.
- 9.Murphy K. Using power analysis to evaluate and improve research. In: Rogelberg SG, editor. Handbook of research methods in industrial organizational psychology. Malden: Blackwell Publishers; 2002. p. 119–37.Google Scholar
- 11.Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. New York: Taylor & Francis; 1988.Google Scholar