Abstract
Transparency in the reporting of empirical studies is foundational to a credible knowledge base. Higher levels of transparency, in addition to clarity in writing, also make research more accessible to a diverse readership. Previous research reviewed how transparently reported qualitative, interview-based, studies were in contemporary technology education research (Buckley et al. in Int J Technol Des Educ, 2021a. https://doi.org/10.1007/s10798-021-09695-1). The results illustrated that no article was fully transparent and that authors tended to be less transparent in some areas, such as the management of power imbalances and a saturation point, than in others, such as the methodology adopted and research setting. This article presents a similar study, however the focus of the investigation was on contemporary quantitative technology education research. An analysis of 46 articles revealed again that no article was fully transparent and that authors tended to be more transparent in some areas than others. Interestingly, the areas where authors or quantitative research tended to be more or less transparent were similar to the areas in which authors of qualitative research tended to be more or less transparent. These results have use in supporting researchers in the clear and transparent reporting of the empirical work and could be useful in the development of guides or support material for academic writing.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The coding rubric used in this study, raw data, and analysis code are available in the Supplementary Material located at https://osf.io/wh735.
References
Aguinis, H., & Solarino, A. M. (2019). Transparency and replicability in qualitative research: The case of interviews with elite informants. Strategic Management Journal, 40(8), 1291–1315. https://doi.org/10.1002/smj.3015
Anvari, F., & Lakens, D. (2018). The replicability crisis and public trust in psychological science. Comprehensive Results in Social Psychology, 3(3), 266–286. https://doi.org/10.1080/23743603.2019.1684822
Baker, M. (2016). 1,500 scientists lift the lid on reproducibility. Nature, 533(7604), 452–454. https://doi.org/10.1038/533452a
Barba, L. (2018). Terminologies for reproducible research. https://arxiv.org/pdf/1802.03311.
Begley, C. G., & Ellis, L. M. (2012). Raise standards for preclinical cancer research. Nature, 483(7391), 531–533. https://doi.org/10.1038/483531a
Buckley, J., Adams, L., Aribilola, I., Arshad, I., Azeem, M., Bracken, L., Breheny, C., Buckley, C., Chimello, I., Fagan, A., Fitzpatrick, D. P., Garza Herrera, D., Gomes, G. D., Grassick, S., Halligan, E., Hirway, A., Hyland, T., Imtiaz, M. B., Khan, M. B., & Zhang, L. (2021a). An assessment of the transparency of contemporary technology education research employing interview-based methodologies. International Journal of Technology and Design Education. https://doi.org/10.1007/s10798-021-09695-1
Buckley, J., Hyland, T., & Seery, N. (2021b). Examining the replicability of contemporary technology education research. Techne Series: Research in Sloyd Education and Craft Sciences, 28(2), 1–9.
Buckley, J., Hyland, T., & Seery, N. (2022a). Estimating the replicability of technology education research. International Journal of Technology and Design Education. https://doi.org/10.1007/s10798-022-09787-6
Buckley, J., Seery, N., & Kimbell, R. (2022b). A review of the valid methodological use of adaptive comparative judgment in technology education research. Frontiers in Education, 7(787926), 1–6.
Camerer, C. F., Dreber, A., Forsell, E., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Almenberg, J., Altmejd, A., Chan, T., Heikensten, E., Holzmeister, F., Imai, T., Isaksson, S., Nave, G., Pfeiffer, T., Razen, M., & Wu, H. (2016). Evaluating replicability of laboratory experiments in economics. Science. https://doi.org/10.1126/science.aaf0918
Charmaz, K. (2006). Constructing grounded theory (2nd ed.). SAGE.
Closa, C. (2021). Planning, implementing and reporting: Increasing transparency, replicability and credibility in qualitative political science research. European Political Science. https://doi.org/10.1057/s41304-020-00299-2
Cochran, W. (1977). Sampling techniques (3rd ed.). Wiley.
Cova, F., Strickland, B., Abatista, A., Allard, A., Andow, J., Attie, M., Beebe, J., Berniūnas, R., Boudesseul, J., Colombo, M., Cushman, F., Diaz, R., N’Djaye Nikolai van Dongen, N., Dranseika, V., Earp, B. D., Torres, A. G., Hannikainen, I., Hernández-Conde, J. V., Hu, W., & Zhou, X. (2021). Estimating the reproducibility of experimental philosophy. Review of Philosophy and Psychology, 12(1), 9–44. https://doi.org/10.1007/s13164-018-0400-9
Diedenhofen, B., & Musch, J. (2015). cocor: A comprehensive solution for the statistical comparison of correlations. PLoS ONE, 10(4), e0121945. https://doi.org/10.1371/journal.pone.0121945
Errington, T. M., Mathur, M., Soderberg, C. K., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021). Investigating the replicability of preclinical cancer biology. eLife, 10, e71601. https://doi.org/10.7554/eLife.71601
Gibbons, M. (1999). Science’s new social contract with society. Nature, 402(6761), C81–C84. https://doi.org/10.1038/35011576
Goldenberg, L. R., Owens, E. F., & Pickar, J. G. (2007). Recruitment of research volunteers: Methods, interest, and incentives. The Journal of Chiropractic Education, 21(1), 28–31.
Hiles, D., & Čermák, I. (2007). Qualitative research: Transparency and narrative oriented inquiry. In 10th European congress of psychology (pp. 1–8).
Kidwell, M. C., Lazarević, L. B., Baranski, E., Hardwicke, T. E., Piechowski, S., Falkenberg, L.-S., Kennett, C., Slowik, A., Sonnleitner, C., Hess-Holden, C., Errington, T. M., Fiedler, S., & Nosek, B. A. (2016). Badges to acknowledge open practices: A simple, low-cost, effective method for increasing transparency. PLoS Biology, 14(5), e1002456. https://doi.org/10.1371/journal.pbio.1002456
Klein, O., Hardwicke, T. E., Aust, F., Breuer, J., Danielsson, H., Mohr, A. H., IJzerman, H., Nilsonne, G., Vanpaemel, W., & Frank, M. C. (2018). A practical guide for transparency in psychological science. Collabra: Psychology, 4(1), 20. https://doi.org/10.1525/collabra.158
Lakens, D. (2021). Sample size justification. PsyArXiv. https://doi.org/10.31234/osf.io/9d3yf
Lakens, D., Adolfi, F. G., Albers, C. J., Anvari, F., Apps, M. A. J., Argamon, S. E., Baguley, T., Becker, R. B., Benning, S. D., Bradford, D. E., Buchanan, E. M., Caldwell, A. R., Van Calster, B., Carlsson, R., Chen, S.-C., Chung, B., Colling, L. J., Collins, G. S., Crook, Z., & Zwaan, R. A. (2018). Justify your alpha. Nature Human Behaviour, 2(3), 168–171. https://doi.org/10.1038/s41562-018-0311-x
Lawson, H. (2004). The logic of collaboration in education and the human services. Journal of Interprofessional Care, 18(3), 225–237. https://doi.org/10.1080/13561820410001731278
Locke, K. (2001). Grounded theory in management research. SAGE.
Low, J. (2019). A pragmatic definition of the concept of theoretical saturation. Sociological Focus, 52(2), 131–139. https://doi.org/10.1080/00380237.2018.1544514
Lupia, A., & Elman, C. (2014). Openness in political science: Data access and research transparency. PS Political Science & Politics, 47(1), 19–42. https://doi.org/10.1017/S1049096513001716
Mason-Bish, H. (2019). The elite delusion: Reflexivity, identity and positionality in qualitative research. Qualitative Research, 19(3), 263–276. https://doi.org/10.1177/1468794118770078
Merton, R. K. (1973). The sociology of science: Theoretical and empirical investigations. University of Chicago Press.
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., Contestabile, M., Dafoe, A., Eich, E., Freese, J., Glennerster, R., Goroff, D., Green, D. P., Hesse, B., Humphreys, M., & Yarkoni, T. (2015). Promoting an open research culture. Science (new York, N. Y.), 348(6242), 1422–1425. https://doi.org/10.1126/science.aab2374
Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), 943. https://doi.org/10.1126/science.aac4716
Pashler, H., & Wagenmakers, E.-J. (2012). Editors’ introduction to the special section on replicability in psychological science: A crisis of confidence? Perspectives on Psychological Science, 7(6), 528–530.
Popper, K. (2002). Conjectures and refutations: The growth of scientific knowledge (2nd ed.). Psychology Press.
Pratt, M., Kaplan, S., & Whittington, R. (2020). Editorial essay: The tumult over transparency: Decoupling transparency from replication in establishing trustworthy qualitative research. Administrative Science Quarterly, 65(1), 1–19. https://doi.org/10.1177/0001839219887663
R Core Team. (2020). R: A language and environment for statistical computing (Version 4.0.3 ‘Bunny-Wunnies Freak Out’) [Computer software]. R Foundation for Statistical Computing. https://www.R-project.org/
Rowe, W. (2014). Positionality. In The SAGE encyclopedia of action research (Vol. 1–2, pp. 627–628). SAGE Publications Ltd. https://doi.org/10.4135/9781446294406.
Saldaña, J. (2013). The coding manual for qualitative researchers (2nd ed.). SAGE.
Sedgwick, P., & Greenwood, N. (2015). Understanding the Hawthorne effect. BMJ, 351, 4672. https://doi.org/10.1136/bmj.h4672
Strauss, A., & Corbin, J. (1998). Basics of qualitative research. SAGE.
Vazire, S. (2018). Implications of the credibility revolution for productivity, creativity, and progress. Perspectives on Psychological Science, 13(4), 411–417. https://doi.org/10.1177/1745691617751884
Zutlevics, T. (2016). Could providing financial incentives to research participants be ultimately self-defeating? Research Ethics, 12(3), 137–148. https://doi.org/10.1177/1747016115626756
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Buckley, J., Araujo, J.A., Aribilola, I. et al. How transparent are quantitative studies in contemporary technology education research? Instrument development and analysis. Int J Technol Des Educ 34, 461–483 (2024). https://doi.org/10.1007/s10798-023-09827-9
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10798-023-09827-9