Abstract
Open Science practices bear great promise for making research in general more reproducible and transparent, and these goals are very important for preventive intervention research. From my perspective as a program co-developer, I note potential concerns and issues of how open science practices can be used in intervention research. Key issues considered are in the realms of pre-registration (making pre-registration a living document; providing rewards for hypothesis-generating research, in addition to hypothesis-testing research), data archiving (resources for data archiving of large datasets; ethical issues related to need for strong de-identification), and research materials (intervention manuals and materials, and characteristics, training and supervision of intervention staff). The paper focuses on easier-to-address and considerably harder-to-address issues and concerns in these three areas.
This is a preview of subscription content, log in via an institution to check access.
References
Allen, C., & Mehler, D. M. A. (2019). Open science challenges, benefits and tips in early career and beyond. PLoS Biology, 17, e3000246. https://doi.org/10.1371/journal.obio.3000246
Bahlia, C., Bartlett, L. J., Burgio, K. R., Fournier, A. M. V., Keiser, C. R., Poisot, T., & Whitney, K. S. (2019). Open science isn’t always open to all scientists. American Scientist, 107(2), 78. https://doi.org/10.1511/2019.107.2.78
Cook, B. G., Lloyd, J. W., Mellor, D., Nosek, B. A., & Therrien, W. J. (2018). Promoting open science to increase the trustworthiness of evidence in special education. Exceptional Children, 85, 104-118.
David, P. A. (2004). Understanding the emergence of ‘open science’ institutions: Functionalist economics in historical context. Industrial and Corporate Change, 13, 571-589.
David, P. (2008). The historical origins of ‘Open Science’: An essay on patronage, reputation, and common agency contracting in the scientific revolution. Capitalism and Society, 3 (2): Article 5.
Dickerson, K. (2019). Lost knowledge: Open science is one solution for hidden data. Hopkins Bloomberg Public Health, Spring. https://mgazine.jhsph.edu/2019/lost-knowledge-open-science-one-solution-hidden-data
Elliott, K. C., & Resnick, D. B. (2019). Making open science work for science and society. Environmental Health Perspectives, 127 (7), 075002-1-6.
Gottfredson, D. C., Cook, T. D., Gardner, F. E. M., Gorman-Smith, D., Howe, G. H., Sandler, I. N., & Zafft, K. M. (2015). Standards of evidence for efficacy, effectiveness, and scale-up research in prevention science: Next generation. Prevention Science, 16, 893-926.
Grant, S., Wendt, K. E., Leadbeater, B. J., Supplee, L. H., Mayo-Wilson, E., Gardner, F., & Bradshaw, C. P. (under review). Transparent, open, and reproducible prevention science. Prevention Science
Grand, A., Wilkinson, C., Bultitude, K., Winfield, A. F. (2016). Mapping the hinterland: Data issues in open science. Public Understanding of Science, 25, 88-103.
Hillary, F. G. & Medaglia, J. D. (2019). What the replication crisis means for intervention science. International Journal of Psychophysiology. https://doi.org/10.1016/.ijpsycho.2019.05.006 2019.05.006
Howe, G. W., Dagne, G. A., Brown, C. H., Brincks, A. M., Beardslee, W., Perrino, T., & Pantin, H. (2019) Evaluating construct equivalence of youth depression measures across multiple measures and multiple studies. Psychological Assessment, 31, 1154–1167.
Kaplan, R. M., & Irvin, V. L. (2015). Likelihood of null effects of large NHLBI clinical trials has increased over time. PLOS ONE, 10(8), e0132382. https://doi.org/10.1371/journal.pone.013238
Lochman, J. E., Boxmeyer, C., Powell, N., Qu, L., Wells, K., & Windle, M. (2009a). Dissemination of the coping power program: Importance of intensity of counselor training. Journal of Consulting and Clinical Psychology, 77, 397-409.
Lochman, J.E., Dishion, T.J., Boxmeyer, C.L., Powell, N.P., & Qu, L. (2017). Variations in response to evidence-based group preventive intervention for disruptive behavior problems: A view from 938 Coping Power sessions. Journal of Abnormal Child Psychology, 45, 1271-1284. https://doi.org/10.1007/s10802-016-0252-7
Lochman, J. E., Dishion, T. J., Powell, N. P., Boxmeyer, C. L., Qu, L., & Sallee, M. (2015a). Evidence-based preventive intervention for preadolescent aggressive children: One-year outcomes following randomization to group versus individual delivery. Journal of Consulting and Clinical Psychology, 83, 728-735.
Lochman, J. E., Powell, N. P., Boxmeyer, C. L., Qu, L., Sallee, M., Wells, K. C., & Windle, M. (2015b). Counselor-level predictors of sustained use of an indicated preventive intervention for aggressive children. Prevention Science, 16, 1075-1085. https://doi.org/10.1007/s11121-014-0511-1
Lochman, J. E., Powell, N., Boxmeyer, C., Qu, L., Wells, K., & Windle, M. (2009b). Implementation of a school-based prevention program: Effects of counselor and school characteristics. Professional Psychology: Research and Practice, 40, 476-497.
Munafo, M., Nosek, B., Bishop, D., Button, K., Chambers, C., du Percie, S. N. et al. (2017). A manifesto for reproducible science. Natural Human Behavior, 1. https://doi.org/10.1038/s41562-016-0021
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115, 2600-2606.
Rosenthal, R. (1979). The “file-drawer problem” and tolerance for null results. Psychological Bulletin, 86, 638-641.
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359-1366.
Tanner-Smith, E. E., Durlak, J. A., & Marx, R. A. (2018). Empirically based mean effect size distributions for universal prevention programs targeting school-aged youth: A review of meta-analyses. Prevention Science, 19, 1091-1101.
Wagenmakers, E. J., Wetzels, R., Borsboom, D., van der Maas H. L., & Kievit, R. A. (2012). An agenda for purely confirmatory research. Perspectives on Psychological Science, 7, 632-638.
Woelfle, M., Olliaro, P., & Todd, M. H. (2011). Open science is a research accelerator. Nature Chemistry, 3, 745-748.
Funding
The preparation of this paper has been supported by grants from the National Institute of Minority Health and Health Disparities (R01 MD013806) and the National Institute of Mental Health (R01 MH124438)
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The author receives royalties from Oxford University Press, and Guilford Press for the books containing intervention manuals for the Coping Power Program, Anger Coping Program, and the Fast Track Program.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Lochman, J.E. Open Science and Intervention Research: a Program Developer’s and Researcher’s Perspective on Issues and Concerns. Prev Sci 23, 844–851 (2022). https://doi.org/10.1007/s11121-021-01219-6
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11121-021-01219-6