Abstract
We can learn a great deal about the research questions being addressed in a field by examining the study designs used in that field. This manuscript examines the research questions being addressed in prevention research by characterizing the distribution and trends of study designs included in primary and secondary prevention research supported by the National Institutes of Health through grants and cooperative agreements, together with the types of prevention research, populations, rationales, exposures, and outcomes associated with each type of design. The Office of Disease Prevention developed a taxonomy to classify new extramural NIH-funded research projects and created a database with a representative sample of 14,523 research projects for fiscal years 2012–2019. The data were weighted to represent the entirety of the extramural research portfolio. Leveraging this dataset, the Office of Disease Prevention characterized the study designs proposed in NIH-funded primary and secondary prevention research applications. The most common study designs proposed in new NIH-supported prevention research applications during FY12-19 were observational designs (63.3%, 95% CI 61.5%–65.0%), analysis of existing data (44.5%, 95% CI: 42.7–46.3), methods research (23.9%, 95% CI: 22.3–25.6), and randomized interventions (17.2%, 95% CI: 16.1%–18.4%). Observational study designs dominated primary prevention research, while intervention designs were more common in secondary prevention research. Observational designs were more common for exposures that would be difficult to manipulate (e.g., genetics, chemical toxin, and infectious disease (not pneumonia/influenza or HIV/AIDS)), while intervention designs were more common for exposures that would be easier to manipulate (e.g., education/counseling, medication/device, diet/nutrition, and healthcare delivery). Intervention designs were not common for outcomes that are rare or have a long latency (e.g., cancer, neurological disease, Alzheimer’s disease) and more common for outcomes that are more common or where effects would be expected earlier (e.g., healthcare delivery, health related quality of life, substance use, and medication/device). Observational designs and analyses of existing data dominated, suggesting that much of the prevention research funded by NIH continues to focus on questions of association and on questions of identification of risk and protective factors. Randomized and non-randomized intervention designs were included far less often, suggesting that a much smaller fraction of the NIH prevention research portfolio is focused on questions of whether interventions can be used to modify risk or protective factors or to change some other health-related biomedical or behavioral outcome. The much heavier focus on observational studies is surprising given how much we know already about the leading risk factors for death and disability in the USA, because those risk factors account for 74% of the county-level mortality in the USA, and because they play such a vital role in the development of clinical and public health guidelines, whose developers often weigh results from randomized trials much more heavily than results from observational studies. Improvements in death and disability nationwide are more likely to derive from guidelines based on intervention research to address the leading risk factors than from additional observational studies.
Similar content being viewed by others
Notes
R01, R03, R21, R43, R44, R56, P01, P50, U01, U19, U54, and UM1 (https://grants.nih.gov/grants/funding/ac_search_results.htm).
A higher percentage of projects was selected for quality control at the beginning of the coding process for each activity code, and in weeks when the quality control results indicated a kappa < 0.70.
References
Bernal, J. L., Cummins, S., & Gasparrini, A. (2016). Interrupted time series regression for the evaluation of public health interventions: a tutorial. International Journal of Epidemiology. https://doi.org/10.1093/ije/dyw098
Boon, M. H., Craig, P., Thomson, H., Campbell, M., & Moore, L. (2021). Regression discontinuity designs in health: a systematic review. Epidemiology, 32(1), 87–93. https://doi.org/10.1097/EDE.0000000000001274. (Epidemiology. 2021 Jul 1;32(4):e15. https://doi.org/10.1097/EDE.0000000000001362. PMID: 34042078)
Brown, C. H., & Liao, J. (1999). Principles for designing randomized preventive trials in mental health: an emerging developmental epidemiology paradigm. American Journal of Community Psychology, 27(5), 673–710. https://doi.org/10.1023/a:1022142021441
Colditz, G. A., & Taylor, P. R. (2010). Prevention trials: Their place in how we understand the value of prevention strategies. Annual Review of Public Health, 31, 105–120. https://doi.org/10.1146/annurev.publhealth.121208.131051
Dwyer-Lindgren, L., Bertozzi-Villa, A., Stubbs, R. W., Morozoff, C., Mackenbach, J. P., van Lenthe, F. J., Mokdad, A. H., & Murray, C. J. L. (2017). Inequalities in life expectancy among US counties, 1980 to 2014: Temporal trends and key drivers. JAMA Internal Medicine, 177(7), 1003–1011. https://doi.org/10.1001/jamainternmed.2017.0918
Friedman, L. M., Furberg, C. D., DeMets, D. L., Reboussin, D. M., & Granger, C. B. (2015). Fundamentals of Clinical Trials (5th ed.). Springer.
Gail, M. H., Altman, D. G., Cadarette, S. M., Collins, G., Evans, S. J., Sekula, P., Williamson, E., & Woodward, M. (2019). Design choices for observational studies of the effect of exposure on disease incidence. British Medical Journal Open, 9, e031031. https://doi.org/10.1136/bmjopen-2019-031031
He, K. Y., Ge, D., & He, M. M. (2017). Big data analytics for genomic medicine. International Journal of Molecular Science, 18(2). https://doi.org/10.3390/ijms18020412
Krist, A. H., Wolff, T. A., Jonas, D. E., Harris, R. P., LeFevre, M. L., Kemper, A. R., Mangione, C. M., Tseng, C. W., & Grossman, D. C. (2018). Update on the methods of the U.S. preventive services task force: Methods for understanding certainty and net benefit when making recommendations. American Journal of Preventive Mededicine, 54(1S1), S11–S18. https://doi.org/10.1016/j.amepre.2017.09.011
Lash, T. L., VanderWeele, T. J., Haneuse, S., & Rothman, K. J. (2021). Modern Epidemiology. Wolters Kluwer.
Martin-Sanchez, F. J., Aguiar-Pulido, V., Lopez-Campos, G. H., Peek, N., & Sacchi, L. (2017). Secondary use and analysis of big data collected for patient care. Yearbook of Medical Informatics, 26(1), 28–37. https://doi.org/10.15265/iy-2017-008
Meinert, C. L. (2012). Clinical Trials Handbook: Design, Conduct, and Analysis (2nd ed.). Wiley.
Mokdad, A. H., Ballestros, K., Echko, M., Glenn, S., Olsen, H. E., Mullany, E., Lee, A., Khan, A. R., Ahmadi, A., Ferrari, A. J., Kasaeian, A., Werdecker, A., Carter, A., Zipkin, B., Sartorius, B., Serdar, B., Sykes, B. L., Troeger, C., Fitzmaurice, C., Rehm, C. D., Santomauro, D., Kim, D., Colombara, D., Schwebel, D. C., Tsoi, D., Kolte, D., Nsoesie, E., Nichols, E., Oren, E., Charlson, F. J., Patton, G. C., Roth, G. A., Hosgood, H. D., Whiteford, H. A., Kyu, H., Erskine, H. E., Huang, H., Martopullo, I., Singh, J. A., Nachega, J. B., Sanabria, J. R., Abbas, K., Ong, K., Tabb, K., Krohn, K. J., Cornaby, L., Degenhardt, L., Moses, M., Farvid, M., Griswold, M., Criqui, M., Bell, M., Nguyen, M., Wallin, M., Mirarefin, M., Qorbani, M., Younis, M., Fullman, N., Liu, P., Briant, P., Gona, P., Havmoller, R., Leung, R., Kimokoti, R., Bazargan-Hejazi, S., Hay, S. I., Yadgir, S., Biryukov, S., Vollset, S. E., Alam, T., Frank, T., Farid, T., Miller, T., Vos, T., Bärnighausen, T., Gebrehiwot, T. T., Yano, Y., Al-Aly, Z., Mehari, A., Handal, A., Kandel, A., Anderson, B., Biroscak, B., Mozaffarian, D., Dorsey, E. R., Ding, E. L., Park, E. K., Wagner, G., Hu, G., Chen, H., Sunshine, J. E., Khubchandani, J., Leasher, J., Leung, J., Salomon, J., Unutzer, J., Cahill, L., Cooper, L., Horino, M., Brauer, M., Breitborde, N., Hotez, P., Topor-Madry, R., Soneji, S., Stranges, S., James, S., Amrock, S., Jayaraman, S., Patel, T., Akinyemiju, T., Skirbekk, V., Kinfu, Y., Bhutta, Z., Jonas, J. B., & Murray, C. J. L. (2018). The state of US health, 1990–2016: Burden of diseases, injuries, and risk factors among US states. JAMA, 319(14), 1444–1472. https://doi.org/10.1001/jama.2018.0158
Mues, K. E., Liede, A., Liu, J., Wetmore, J. B., Zaha, R., Bradbury, B. D., Collins, A. J., & Gilbertson, D. T. (2017). Use of the Medicare database in epidemiologic and health services research: a valuable source of real-world evidence on the older and disabled populations in the US. Clinical Epidemiology, 9, 267–277. https://doi.org/10.2147/clep.S105613
Murray, D. M., Cross, W. P., Simons-Morton, D., Engel, J., Portnoy, B., Wu, J., Watson, P. A., & Olkkola, S. (2015). Enhancing the quality of prevention research supported by the National Institutes of Health. American Journal of Public Health, 105, 9–12.
Murray, D. M., Ganoza, L. F., Vargas, A. J., Ellis, E. M., Oyedele, N. K., Schully, S. D., & Liggins, C. A. (2021). New NIH primary and secondary prevention research during 2012–2019. American Journal of Preventive Medicine. https://doi.org/10.1016/j.amepre.2021.01.006
Murray, D. M., Taljaard, M., Turner, E. L., & George, S. M. (2020). Essential ingredients and innovations in the design and analysis of group-randomized trials. Annual Review of Public Health, 41, 1–19. https://doi.org/10.1146/annurev-publhealth-040119-094027
Patterson, J. G., Jabson, J. M., & Bowen, D. J. (2017). Measuring sexual and gender minority populations in health surveillance. LGBT Health, 4(2), 82–105. https://doi.org/10.1089/lgbt.2016.0026
Pennell, M. L., Hade, E. M., Murray, D. M., & Rhoda, D. A. (2011). Cutoff designs for community-based intervention studies. Statistics in Medicine, 30(15), 1865–1882. https://doi.org/10.1002/sim.4237
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin Company.
Vargas, A. J., Schully, S. D., Villani, J., Caballero, L. G., & Murray, D. M. (2019). Assessment of prevention research measuring leading risk factors and causes of mortality and disability supported by the US National Institutes of Health. JAMA Network Open, 2, e1914718–e1914718.
Venkataramani, A. S., Bor, J., & Jena, A. B. (2016). Regression discontinuity designs in healthcare research. BMJ, 352, i1216. https://doi.org/10.1136/bmj.i1216
Villani, J., Schully, S. D., Meyer, P., Myles, R. L., Lee, J. A., Murray, D. M., & Vargas, A. J. (2018). A machine learning approach to identify NIH-funded applied prevention research. American Journal of Preventive Medicine, 55(6), 926–931. https://doi.org/10.1016/j.amepre.2018.07.024
Zelen, M. (1988). Are primary cancer prevention trials feasible? Journal of National Cancer Institure, 80(18), 1442–1444. https://doi.org/10.1093/jnci/80.18.1442
Acknowledgements
All authors were employees of the NIH Office of Disease Prevention when they were actively engaged in work related to this paper. NIH is the sole source of support for the work reported. The authors would like to thank the team of research analysts and developers at IQ Solutions and Westat for their work on coding.
Funding
No funding was received for conducting this study.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethics Approval
Not applicable.
Consent to Participate
Not applicable.
Conflicts of Interest
The authors have no relevant financial or non-financial interests to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Oyedele, N.K., Ganoza, L.F., Schully, S.D. et al. NIH Primary and Secondary Prevention Research in Humans: a Portfolio Analysis of Study Designs Used in 2012–2019. Prev Sci 23, 477–487 (2022). https://doi.org/10.1007/s11121-022-01337-9
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11121-022-01337-9