Abstract
When seeking to inform and improve prevention efforts and policy, it is important to be able to robustly synthesize all available evidence. But evidence sources are often large and heterogeneous, so understanding what works, for whom, and in what contexts can only be achieved through a systematic and comprehensive synthesis of evidence. Many barriers impede comprehensive evidence synthesis, which leads to uncertainty about the generalizability of intervention effectiveness, including inaccurate titles/abstracts/keywords terminology (hampering literature search efforts), ambiguous reporting of study methods (resulting in inaccurate assessments of study rigor), and poorly reported participant characteristics, outcomes, and key variables (obstructing the calculation of an overall effect or the examination of effect modifiers). To address these issues and improve the reach of primary studies through their inclusion in evidence syntheses, we provide a set of practical guidelines to help prevention scientists prepare synthesis-ready research. We use a recent mindfulness trial as an empirical example to ground the discussion and demonstrate ways to ensure the following: (1) primary studies are discoverable; (2) the types of data needed for synthesis are present; and (3) these data are readily synthesizable. We highlight several tools and practices that can aid authors in these efforts, such as using a data-driven approach for crafting titles, abstracts, and keywords or by creating a repository for each project to host all study-related data files. We also provide step-by-step guidance and software suggestions for standardizing data design and public archiving to facilitate synthesis-ready research.
This is a preview of subscription content, log in via an institution to check access.
source files, locations, and relationships between the files; d provides permanent author information and licensing information
Similar content being viewed by others
Notes
Although FAIR principles primarily refer to enhancing the ability of machines to find and use data, these recommendations also enhance data reuse by individuals. Where appropriate, we have adapted the FAIR components to more directly apply to human-readability efforts for the purposes of this manuscript.
Comparing the full suite of licenses available is beyond the scope of this article; online resources such as ChooseALicence (https://choosealicense.com/non-software/) and Creative Commons (https://creativecommons.org/licenses/) provide detailed descriptions of the range of licenses available for data sharing.
References
Abbott, S., Christensen, H., Welton, N. J., & Brooks-Pollock, E. (2019). Estimating the effect of the 2005 change in BCG policy in England: A retrospective cohort study, 2000 to 2015. Eurosurveillance, 24(49), 1,900,220. https://doi.org/10/gg9nqk
Acabchuk, R. L., Simon, M. A., Low, S., Brisson, J. M., & Johnson, B. T. (2021). Measuring meditation progress with a consumer-grade EEG device: Caution from a randomized controlled trial. Mindfulness, 12, 68–81. https://doi.org/10.1007/s12671-020-01497-1
Banks, G. C., Field, J. G., Oswald, F. L., O’Boyle, E. H., Landis, R. S., Rupp, D. E., & Rogelberg, S. G. (2019). Answers to 18 questions about open science practices. Journal of Business and Psychology, 34, 257–270. https://doi.org/10.1007/s10869-018-9547-8
Barton, C., Smith, R., & Weaver, R. (2010). Data practices, policy, and rewards in the information era demand a new paradigm. Data Science Journal, 9, IGY95–IGY99. https://doi.org/10.2481/dsj.SS_IGY-003
Beugelsdijk, S., van Witteloostuijn, A., & Meyer, K. E. (2020). A new approach to data access and research transparency (DART). Journal of International Business Studies, 51, 887–905. https://doi.org/10.1057/s41267-020-00323-z
Bishop, L. (2009). Ethical sharing and reuse of qualitative data. Australian Journal of Social Issues, 44(3), 255-272.
Borah, R., Brown, A. W., Capers, P. L., & Kaiser, K. A. (2017). Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open, 7, e012545. https://doi.org/10.1136/bmjopen-2016-012545
Chan, A. W., Tetzlaff, J. M., Altman, D. G., Laupacis, A., Gøtzsche, P. C., Krleža-Jerić, K., .& Doré, C. J. (2013). SPIRIT 2013 statement: Defining standard protocol items for clinical trials. Annals of Internal Medicine; Ann Intern Med. https://doi.org/10.7326/0003-4819-158-3-201302050-00583
Choose Muse. (2018, June 25). A deep dive into brainwaves: Brainwave frequencies explained. Muse. https://choosemuse.com/blog/a-deep-dive-into-brainwaves-brainwave-frequencies-explained-2/
Christensen, G., Dafoe, A., Miguel, E., Moore, D. A., & Rose, A. K. (2019). A study of the impact of data sharing on article citations using journal policies as a natural experiment. PLoS One, 14, e0225883. https://doi.org/10.1371/journal.pone.0225883
Colavizza, G., Hrynaszkiewicz, I., Staden, I., Whitaker, K., & McGillivray, B. (2020). The citation advantage of linking publications to research data. PLoS One, 15, 023–0416. https://doi.org/10.1371/journal.pone.0230416
Crosas, M. (2013, October 30). Joint Declaration of Data Citation Principles—FINAL. FORCE11. https://www.force11.org/datacitationprinciples
Dryad. (2020). https://datadryad.org/stash/our_mission
Evans, S. R. (2016). Gauging the purported costs of public data archiving for long-term population studies. PLoS Biology, 14, e1002432. https://doi.org/10.1371/journal.pbio.1002432
Falk Delgado, A., & Falk Delgado, A. (2017). Outcome switching in randomized controlled oncology trials reporting on surrogate endpoints: A cross-sectional analysis. Scientific Reports, 7. https://doi.org/10.1038/s41598-017-09553-y
Federer, L. M., Belter, C. W., Joubert, D. J., Livinski, A., Lu, Y. L., Snyders, L. N., & Thompson, H. (2018). Data sharing in PLOS ONE: An analysis of Data Availability Statements. PLoS One, 13, e0194768. https://doi.org/10/gdhb7j
Forstmeier, W., Wagenmakers, E. J., & Parker, T. H. (2017). Detecting and avoiding likely false-positive findings – a practical guide. Biological Reviews, 92, 1941–1968. https://doi.org/10.1111/brv.12315
Frankenhuis, W. E., & Nettle, D. (2018). Open science is liberating and can foster creativity. Perspectives on Psychological Science, 13, 439–447. https://doi.org/10.1177/1745691618767878
Gaba, J. F., Siebert, M., Dupuy, A., Moher, D., & Naudet, F. (2020). Funders’ data-sharing policies in therapeutic research: A survey of commercial and non-commercial funders. PloS One, 15, e0237464. https://doi.org/10.1371/journal.pone.0237464
Gerstner, K., Moreno-Mateos, D., Gurevitch, J., Beckmann, M., Kambach, S., Jones, H. P., & Seppelt, R. (2017). Will your paper be used in a meta-analysis? Make the reach of your research broader and longer lasting. Methods in Ecology and Evolution, 8, 777–784. https://doi.org/10.1111/2041-210X.12758
Graf, C., Flanagan, D., Wylie, L., & Silver, D. (2020). The open data challenge: An analysis of 124,000 data availability statements and an ironic lesson about data management plans. Data Intelligence, 1–15. https://doi.org/10.1162/dint_a_00061
Grames, E. M., Stillman, A. N., Tingley, M. W., & Elphick, C. S. (2019). An automated approach to identifying search terms for systematic reviews using keyword co-occurrence networks. Methods in Ecology and Evolution, 10, 1645–1654.
Grant, S., Mayo-Wilson, E., Montgomery, P., Macdonald, G., Michie, S., Hopewell, S., Moher, D., Aber, J. L., Altman, D., Bhui, K., Booth, A., Clark, D., Craig, P., Eisner, M., Fraser, M. W., Gardner, F., Grant, S., Hedges, L., Hollon, S., & CONSORT-SPI 2018. (2018). CONSORT-SPI 2018 explanation and elaboration: Guidance for reporting social and psychological intervention trials. Trials, 19, 406. https://doi.org/10.1186/s13063-018-2735-z
Gupta, Y. K., Meenu, M., & Mohan, P. (2015). The Tamiflu fiasco and lessons learnt. Indian Journal of Pharmacology, 47, 11–16. https://doi.org/10.4103/0253-7613.150308
Haddaway, N. R., & Westgate, M. J. (2019). Predicting the time needed for environmental systematic reviews and systematic maps. Conservation Biology, 33(2), 434–443. https://doi.org/10.1111/cobi.13231
Hardwicke, T. E., Wallach, J. D., Kidwell, M. C., Bendixen, T., Crüwell, S., & Ioannidis, J. P. A. (2020). An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014–2017). Royal Society Open Science, 7, 1908706. https://doi.org/10.1098/rsos.190806
Haynes, R. B. (2006). Forming research questions. Journal of Clinical Epidemiology, 59, 881–886. https://doi.org/10.1016/j.jclinepi.2006.06.006
ICPSR. (2021). Retrieved February 24, 2021, from https://www.icpsr.umich.edu/web/pages/
Inter-university Consortium for Political and Social Research (2010). Guide to social science data preparation and archiving best practice throughout the data life cycle. ICPSR Institute for Social Research University of Michigan.
Johnson, B. T., & Hennessy, E. A. (2019). Systematic reviews and meta-analyses in the health sciences: Best practice methods for research syntheses. Social Science & Medicine, 233, 237–251. https://doi.org/10.1016/j.socscimed.2019.05.035
Johnson, B. T., Cromley, E. K., & Marrouch, N. (2017) Spatiotemporal meta-analysis: Reviewing health psychology phenomena over space and time. Health Psychology Review, 11, 280–291.
Kahan, B. C., & Jairath, V. (2018). Outcome pre-specification requires sufficient detail to guard against outcome switching in clinical trials: A case study. Trials, 19, 265. https://doi.org/10.1186/s13063-018-2654-z
Kerr, N. L. (1998). HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, 2, 196–217.
Marshall, I. J., Johnson, B. T., Wang, Z., Rajasekaran, S., & Wallace, B. C. (2020). Semi-Automated evidence synthesis in health psychology: Current methods and future prospects. Health Psychology Review, 14, 145–158. https://doi.org/10.1080/17437199.2020.1716198
Miyakawa, T. (2020). No raw data, no science: Another possible source of the reproducibility crisis. Molecular Brain, 13, 24. https://doi.org/10.1186/s13041-020-0552-2
Montgomery, P., Grant, S., Mayo-Wilson, E., Macdonald, G., Michie, S., Hopewell, S., Moher, D., Lawrence Aber, J., Altman, D., Bhui, K., Booth, A., Clark, D., Craig, P., Eisner, M., Fraser, M. W., Gardner, F., Grant, S., Hedges, L., Hollon, S., & CONSORT-SPI Group. (2018). Reporting randomised trials of social and psychological interventions: The CONSORT-SPI 2018 Extension. Trials, 19, 407–407. https://doi.org/10.1186/s13063-018-2733-1
Nakagawa, S., Dunn, A. G., Lagisz, M., Bannach-Brown, A., Grames, E. M., Sánchez-Tójar, A., O’Dea, R. E., Noble, D. W. A., Westgate, M. J., Arnold, P. A., Barrow, S., Bethel, A., Cooper, E., Foo, Y. Z., Geange, S. R., Hennessy, E., Mapanga, W., Mengersen, K., Munera, C., & Hackathon, E. S. (2020). A new ecosystem for evidence synthesis. Nature Ecology & Evolution, 4, 498–501. https://doi.org/10.1038/s41559-020-1153-2
NIH data sharing policy and guidance. (2003). National Institutes of Health. https://grants.nih.gov/grants/policy/data_sharing/data_sharing_guidance.htm
Nuijten, M. B., Hartgerink, C. H. J., van Assen, M. A. L. M., Epskamp, S., & Wicherts, J. M. (2016). The prevalence of statistical reporting errors in psychology (1985–2013). Behavior Research Methods, 48, 1205–1226. https://doi.org/10.3758/s13428-015-0664-2
Ohmann, C., Banzi, R., Canham, S., Battaglia, S., Matei, M., Ariyo, C., Becnel, L., Bierer, B., Bowers, S., Clivio, L., Dias, M., Druml, C., Faure, H., Fenner, M., Galvez, J., Ghersi, D., Gluud, C., Groves, T., Houston, P., & Demotes-Mainard, J. (2017). Sharing and reuse of individual participant data from clinical trials: Principles and recommendations. BMJ Open, 7, e018647. https://doi.org/10.1136/bmjopen-2017-018647
Open Science Framework. (2020). https://osf.io/
ORCID iD. (2009). "ORCID website". orcid.org. Retrieved 18 April 2021.
Pasquetto, I. V., Borgman, C. L., & Wofford, M. F. (2019). Uses and reuses of scientific data: The data creators’ advantage. Harvard Data Science Review, 1. https://doi.org/10.1162/99608f92.fc14bf2d
Pescosolido, B., Lee, B., & Kafadar, K. (2020). Replication data for: Socio-demographic similarity at the contextual level alters individual risk for completed suicide [Data set]. Harvard Dataverse. https://doi.org/10.7910/DVN/35IV23
Popkin, G. (2019). Data sharing and how it can benefit your scientific career. Nature, 569, 445–447.
Quintana, D. S. (2020). A synthetic dataset primer for the biobehavioural sciences to promote reproducibility and hypothesis generation. Elife, 9, e53275.
R Core Team. (2019). R: A language and environment for statistical computing. https://www.R-project.org/
Riley, R. D., Lambert, P. C., & Abo-Zaid, G. (2010). Meta-analysis of individual participant data: Rationale, conduct, and reporting. BMJ, 340. https://doi.org/10.1136/bmj.c221
Schulz, K. F., Altman, D. G., Moher, D., & CONSORT Group. (2010). CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials. BMJ (clinical Research Ed.), 340, c332. https://doi.org/10.1136/bmj.c332
Sholler, D., Ram, K., Boettiger, C., & Katz, D. S. (2019). Enforcing public data archiving policies in academic publishing: A study of ecology journals. Big Data & Society, 6, epub. https://doi.org/10.1177/2053951719836258
Slep, A. M. S., Heyman, R. E., Lorber, M. F., Baucom, K. J. W., & Linkh, D. J. (2020). Evaluating the Effectiveness of NORTH STAR: A Community-Based Framework to Reduce Adult Substance Misuse, Intimate Partner Violence, Child Abuse, Suicidality, and Cumulative Risk. Prevention Science. https://doi.org/10/gg898m
Stern, C., Jordan, Z., & McArthur, A. (2014). Developing the review question and inclusion criteria. AJN the American Journal of Nursing, 114, 53–56.
Strasser, C. (2012). Primer on data management: What you always wanted to know.
Ten Percent Happier: Mindfulness Meditation Courses. (2020). Ten percent happier. https://www.tenpercent.com
Tennant, J. P., Crane, H., Crick, T., Davila, J., Enkhbayar, A., Havemann, J., Kramer, B., Martin, R., Masuzzo, P., Nobes, A., Rice, C., Rivera-López, B., Ross-Hellauer, T., Sattler, S., Thacker, P. D., & Vanholsbeeck, M. (2019). Ten hot topics around scholarly publishing. Publications, 7, 34. https://doi.org/10.3390/publications7020034
Tenopir, C., Allard, S., Douglass, K., Aydinoglu, A. U., Wu, L., Read, E., Manoff, M., & Frame, M. (2011). Data sharing by scientists: Practices and perceptions. PLoS One, 6, e21101. https://doi.org/10.1371/journal.pone.0021101
Tenopir, C., Dalton, E. D., Allard, S., Frame, M., Pjesivac, I., Birch, B., Pollock, D., & Dorsett, K. (2015). Changes in data sharing and data reuse practices and perceptions among scientists worldwide. PLoS One, 10, e0134826. https://doi.org/10.1371/journal.pone.0134826
Thorogood, A., & Knoppers, B. M. (2017). Can research ethics committees enable clinical trial data sharing? Ethics, Medicine and Public Health, 3, 56–63.
Tomczyk, S., Schomerus, G., Stolzenburg, S., Muehlan, H., & Schmidt, S. (2020). Ready, willing and able? An investigation of the theory of planned behaviour in help-seeking for a community sample with current untreated depressive symptoms. Prevention Science, 21(6), 749–760. https://doi.org/10/gg898n
Transpose. (2020). Transpose Database: A database of journal policies on peer review, co-reviewing, and preprinting. https://transpose-publishing.github.io/#/
van Panhuis, W. G., Paul, P., Emerson, C., Grefenstette, J., Wilder, R., Herbst, A. J., Heymann, D., & Burke, D. S. (2014). A systematic review of barriers to data sharing in public health. BMC Public Health, 14, 1144. https://doi.org/10.1186/1471-2458-14-1144
Vasilevsky, N. A., Minnier, J., Haendel, M. A., & Champieux, R. E. (2017). Reproducible and reusable research: Are journal data sharing policies meeting the mark? PeerJ, 5.
Vicente-Saez, R., & Martinez-Fuentes, C. (2018). Open Science now: A systematic literature review for an integrated definition. Journal of Business Research, 88, 428–436. https://doi.org/10.1016/j.jbusres.2017.12.043
Walters, W. H. (2020). Data journals: Incentivizing data access and documentation within the scholarly communication system. Insights, 33, 18. https://doi.org/10.1629/uksg.510
Wellcome Trust Public Health Research Data Forum. (2012). Summary of Funders’ data sharing policies. https://wellcome.ac.uk/sites/default/files/summary-of-phrdf-funder-data-sharing-policies.pdf
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J. W., da Silva Santos, L. B., Bourne, P. E., Bouwman, J., Brookes, A. J., Clark, T., Crosas, M., Dillo, I., Dumon, O., Edmunds, S., Evelo, C. T., Finkers, R., & Mons, B. (2016). The FAIR guiding principles for scientific data management and stewardship. Scientific Data, 3, 160018. https://doi.org/10.1038/sdata.2016.18
Wilson, S. J., & Tanner-Smith, E. E. (2014). Meta-analysis in prevention science. In Z. Sloboda & H. Petras (Eds.), Defining prevention science (pp. 431–452). Springer US. https://doi.org/10.1007/978-1-4899-7424-2_19
Yardley, S. J., Watts, K. M., Pearson, J., & Richardson, J. C. (2014). Ethical issues in the reuse of qualitative data: Perspectives from literature, practice, and participants. Qualitative Health Research, 24(1), 102-113.
Yenni, G. M., Christensen, E. M., Bledsoe, E. K., Supp, S. R., Diaz, R. M., White, E. P., & Ernest, S. K. M. (2019). Developing a modern data workflow for regularly updated data. PLoS Biology, 17, e3000125. https://doi.org/10.1371/journal.pbio.3000125
Zenodo—Research Shared. (2020). https://about.zenodo.org/
Acknowledgements
The paper was first developed during the 2019 Evidence Synthesis Hackathon.
Funding
The Fenner School of Environment & Society (ANU), the University of New South Wales, and the Faculty of Humanities at the University of Johannesburg funded the Evidence Synthesis Hackathon 2019. EAH has support from NIAAA (K01 AA028536-01). MJP has support from an Australian Research Council Discovery Early Career Researcher Award (DE200101618). LAM has support by an NIHR Doctoral Research Fellowship (DRF-2018–11-ST2-048).
Author information
Authors and Affiliations
Contributions
Project administration (EAH; LAM), roles/writing — original draft (EAH; LAM), visualization (EAH; LAM); data curation (RLA; LAM); resources (LAM); funding acquisition (NRH); investigation (EAH; LAM; RLA), methodology (EAH; LAM); software (EAH; LAM); conceptualization (All authors), writing — review and editing (all authors).
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare no competing interests.
Disclaimer
The views expressed in this article are those of the authors and do not necessarily represent those of the NIAAA, NHS, the NIHR, or the Department of Health and Social Care.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Hennessy, E.A., Acabchuk, R.L., Arnold, P.A. et al. Ensuring Prevention Science Research is Synthesis-Ready for Immediate and Lasting Scientific Impact. Prev Sci 23, 809–820 (2022). https://doi.org/10.1007/s11121-021-01279-8
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11121-021-01279-8