Purpose of Review
Our purpose was to discuss the methodological limitations of observational nutritional epidemiology research, using observational studies on coffee intake and health as a case example.
A number of recent observational studies on the potential health effects of daily coffee intake have reported protective associations between higher coffee intake and a variety of health outcomes, including death. This is inconsistent with the findings from classic studies showing an increased risk of coronary heart disease events, performed in young adults with a homogeneous education level, and adjusting for tobacco use.
Many nutritional epidemiological studies have important limitations, which limit their validity. These include the use of prevalent user designs, risk of reverse causality, measurement error particularly of the exposure of interest, and residual confounding by socioeconomic status. In this review, we discuss these potential issues and provide constructive recommendations intended to help minimize them.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Papers of particular interest, published recently, have been highlighted as: • Of importance •• Of major importance
•• Lachat C, Hawwash D, Ocké MC, Berg C, Forsum E, Hörnell A, et al. Strengthening the Reporting of Observational Studies in Epidemiology-Nutritional Epidemiology (STROBE-nut): an extension of the STROBE statement. PLoS Med. 2016;13(6):e1002036. https://doi.org/10.1371/journal.pmed.1002036. Extension of the STROBE statement, setting standards and recommendations specific to nutritional epidemiological research.
Mandrola J. Enough with the coffee research and other distractions. Medscape. Available at: http://www.medscape.com/viewarticle/883709.
• Freedman ND, Park Y, Abnet CC, Hollenbeck AR, Sinha R. Association of coffee drinking with total and cause-specific mortality. N Engl J Med. 2012;366(20):1891–904. https://doi.org/10.1056/NEJMoa1112010. The authors observed a protective association between higher coffee intake and all-cause mortality in a US population.
• Mostofsky E, Rice MS, Levitan EB, Mittleman MA. Habitual coffee consumption and risk of heart failure: a dose-response meta-analysis. Circ Heart Fail. 2012;5(4):401–5. https://doi.org/10.1161/CIRCHEARTFAILURE.112.967299. In this meta-analysis, the authors found a U-shaped association between higher coffee intake and incident heart failure.
• Kokubo Y, Iso H, Saito I, Yamagishi K, Yatsuya H, Ishihara J, et al. The impact of green tea and coffee consumption on the reduced risk of stroke incidence in Japanese population: the Japan public health center-based study cohort. Stroke. 2013;44(5):1369–74. https://doi.org/10.1161/STROKEAHA.111.677500. The authors report a protective association between higher coffee intake and the incidence of stroke.
• Gunter MJ, Murphy N, Cross AJ, Dossus L, Dartois L, Fagherazzi G, et al. Coffee drinking and mortality in 10 European countries: a multinational cohort study. Ann Intern Med. 2017; https://doi.org/10.7326/M16-2945. Study reporting a protective association between higher coffee intake and death in 10 European countries.
• Park SY, Freedman ND, Haiman CA, Le Marchand L, Wilkens LR, Setiawan VW. Association of coffee consumption with total and cause-specific mortality among nonwhite populations. Ann Intern Med. 2017; https://doi.org/10.7326/M16-2472. The authors observed a protective association between higher coffee intake and mortality among non-white populations living in the USA.
• Jick H, Miettinen OS, Neff RK, Shapiro S, Heinonen OP, Slone D. Coffee and myocardial infarction. N Engl J Med. 1973;289(2):63–7. https://doi.org/10.1056/NEJM197307122890203. Classic paper reporting an increased risk of acute myocardial infraction with higher coffee intake.
• MacMahon B, Yen S, Trichopoulos D, Warren K, Nardi G. Coffee and cancer of the pancreas. N Engl J Med. 1981;304(11):630–3. https://doi.org/10.1056/NEJM198103123041102. Classic report of a positive association between higher coffee intake and occurrence of pancreatic cancer.
• Marrett LD, Walter SD, Meigs JW. Coffee drinking and bladder cancer in Connecticut. Am J Epidemiol. 1983;117(2):113–27. https://doi.org/10.1093/oxfordjournals.aje.a113522. The authors found an association between higher coffee intake and bladder cancer in a US population.
• LaCroix AZ, Mead LA, Liang KY, Thomas CB, Pearson TA. Coffee consumption and the incidence of coronary heart disease. N Engl J Med. 1986;315(16):977–82. https://doi.org/10.1056/NEJM198610163151601. In the PRECURSORS cohort (mean age at baseline 26 years), the authors observed an association between higher coffee intake and risk of incident coronary heart disease events.
• Klag MJ, Mead LA, LaCroix AZ, Wang NY, Coresh J, Liang KY, et al. Coffee intake and coronary heart disease. Ann Epidemiol. 1994;4(6):425–33. https://doi.org/10.1016/1047-2797(94)90001-9. Update of the prior study, using a longer follow-up period and updated analytic techniques.
Margetts BM, Nelson M. Design concepts in nutritional epidemiology. 2nd ed. Oxford: Oxford University Press; 1997.
Bekkering GE, Harris RJ, Thomas S, Mayer AM, Beynon R, Ness AR, et al. How much of the data published in observational studies of the association between diet and prostate or bladder cancer is usable for meta-analysis? Am J Epidemiol. 2008;167(9):1017–26. https://doi.org/10.1093/aje/kwn005.
Palatini P. Letter by Palatini regarding article, “Habitual coffee consumption and risk of heart failure: a dose-response meta-analysis”. Circ Heart Fail. 2012;5(6):e98; author reply e99, DOI: https://doi.org/10.1161/CIRCHEARTFAILURE.112.970111.
Aubin HJ, Berlin I. Coffee drinking and mortality. N Engl J Med. 2012;367(6):576. author reply 576-7
Saloustros E, Stratakis CA. Coffee drinking and mortality. N Engl J Med. 2012;367(6):575–6. author reply 576-7
Aberegg SK. Coffee drinking and mortality. N Engl J Med. 2012;367(6):575. author reply 576-7
Bellach B, Kohlmeier L. Energy adjustment does not control for differential recall bias in nutritional epidemiology. J Clin Epidemiol. 1998;51(5):393–8. https://doi.org/10.1016/S0895-4356(97)00302-8.
Lee BY. Forbes. No, These two studies don’t prove that coffee leads to longer life. Available at: https://www.forbes.com/sites/brucelee/2017/07/11/no-these-2-studies-dont-prove-that-coffee-leads-to-longer-life/#7beb00ef2d2f.
•• Barnard ND, Willett WC, Ding EL. The misuse of meta-analysis in nutrition research. JAMA. 2017;318(15):1435–6. https://doi.org/10.1001/jama.2017.12083. Insightful discussion on the limitations of meta-analyses in the context of nutritional epidemiological research.
Wu JN, Ho SC, Zhou C, Ling WH, Chen WQ, Wang CL, et al. Coffee consumption and risk of coronary heart diseases: a meta-analysis of 21 prospective cohort studies. Int J Cardiol. 2009;137(3):216–25. https://doi.org/10.1016/j.ijcard.2008.06.051.
National Coffee Association USA. National coffee drinking trends. Available at: http://www.ncausa.org/Industry-Resources/Market-Research/National-Coffee-Drinking-Trends-Report.
Coffee Association of Canada. Coffee Facts. Available at: https://www.coffeeassoc.com/media-coffee-facts/.
Moride Y, Abenhaim L. Evidence of the depletion of susceptibles effect in non-experimental pharmacoepidemiologic research. J Clin Epidemiol. 1994;47(7):731–7.
Shirlow MJ, Mathers CD. A study of caffeine consumption and symptoms; indigestion, palpitations, tremor, headache and insomnia. Int J Epidemiol. 1985;14(2):239–48. https://doi.org/10.1093/ije/14.2.239.
Bruce MS, Lader M. Caffeine abstention in the management of anxiety disorders. Psychol Med. 1989;19(1):211–4. https://doi.org/10.1017/S003329170001117X.
Liu H, Yao K, Zhang W, Zhou J, Wu T, He C. Coffee consumption and risk of fractures: a meta-analysis. Arch Med Sci. 2012;8(5):776–83. https://doi.org/10.5114/aoms.2012.31612.
O’Keefe JH, Bhatti SK, Patil HR, DiNicolantonio JJ, Lucan SC, Lavie CJ. Effects of habitual coffee consumption on cardiometabolic disease, cardiovascular health, and all-cause mortality. J Am Coll Cardiol. 2013;62(12):1043–51. https://doi.org/10.1016/j.jacc.2013.06.035.
Ray WA. Evaluating medication effects outside of clinical trials: new-user designs. Am J Epidemiol. 2003;158(9):915–20. https://doi.org/10.1093/aje/kwg231.
Hernán MA, Alonso A, Logan R, Grodstein F, Michels KB, Willett WC, et al. Observational studies analyzed like randomized experiments: an application to postmenopausal hormone therapy and coronary heart disease. Epidemiology. 2008;19(6):766–79. https://doi.org/10.1097/EDE.0b013e3181875e61.
Danaei G, Tavakkoli M, Hernán MA. Bias in observational studies of prevalent users: lessons for comparative effectiveness research from a meta-analysis of statins. Am J Epidemiol. 2012;175(4):250–62. https://doi.org/10.1093/aje/kwr301.
•• Sattar N, Preiss D. Reverse causality in cardiovascular epidemiological research: more common than imagined? Circulation. 2017;135(24):2369–72. https://doi.org/10.1161/CIRCULATIONAHA.117.028307. Expert comment on the relevance that reverse causality may have in cardiovascular epidemiology and its potential implications.
Maselko J, Hayward RD, Hanlon A, Buka S, Meador K. Religious service attendance and major depression: a case of reverse causality? Am J Epidemiol. 2012;175(6):576–83. https://doi.org/10.1093/aje/kwr349.
Goldstein AM, Hodge SE, Haile RW. Selection bias in case-control studies using relatives as the controls. Int J Epidemiol. 1989;18(4):985–9. https://doi.org/10.1093/ije/18.4.985.
Mendez MA. Invited commentary: dietary misreporting as a potential source of bias in diet-disease associations: future directions in nutritional epidemiology research. Am J Epidemiol. 2015;181(4):234–6. https://doi.org/10.1093/aje/kwu306.
Schatzkin A, Kipnis V, Carroll RJ, Midthune D, Subar AF, Bingham S, et al. A comparison of a food frequency questionnaire with a 24-hour recall for use in an epidemiological cohort study: results from the biomarker-based Observing Protein and Energy Nutrition (OPEN) study. Int J Epidemiol. 2003;32(6):1054–62. https://doi.org/10.1093/ije/dyg264.
Miller TM, Abdel-Maksoud MF, Crane LA, Marcus AC, Byers TE. Effects of social approval bias on self-reported fruit and vegetable consumption: a randomized controlled trial. Nutr J. 2008;7(1):18. https://doi.org/10.1186/1475-2891-7-18.
Galea S, Tracy M, Hoggatt KJ, Dimaggio C, Karpati A. Estimated deaths attributable to social factors in the United States. Am J Public Health. 2011;101(8):1456–65. https://doi.org/10.2105/AJPH.2010.300086.
Borrell LN, Diez Roux AV, Rose K, Catellier D, Clark BL. Neighbourhood characteristics and mortality in the Atherosclerosis Risk in Communities Study. Int J Epidemiol. 2004;33(2):398–407. https://doi.org/10.1093/ije/dyh063.
Diez Roux AV, Merkin SS, Arnett D, Chambless L, Massing M, Nieto FJ, et al. Neighborhood of residence and incidence of coronary heart disease. N Engl J Med. 2001;345(2):99–106. https://doi.org/10.1056/NEJM200107123450205.
Stringhini S, Carmeli C, Jokela M, Avendaño M, Muennig P, Guida F, et al. Socioeconomic status and the 25 × 25 risk factors as determinants of premature mortality: a multicohort study and meta-analysis of 1.7 million men and women. Lancet. 2017;389(10075):1229–37. https://doi.org/10.1016/S0140-6736(16)32380-7.
Pedersen SS, von Känel R, Tully PJ, Denollet J. Psychosocial perspectives in cardiovascular disease. Eur J Prev Cardiol. 2017;24(3_suppl):108–15. https://doi.org/10.1177/2047487317703827.
Giltay EJ, Kamphuis MH, Kalmijn S, Zitman FG, Kromhout D. Dispositional optimism and the risk of cardiovascular death: the Zutphen Elderly Study. Arch Intern Med. 2006;166(4):431–6. https://doi.org/10.1001/archinte.166.4.431.
Nabi H, Koskenvuo M, Singh-Manoux A, Korkeila J, Suominen S, Korkeila K, et al. Low pessimism protects against stroke: the Health and Social Support (HeSSup) prospective cohort study. Stroke. 2010;41(1):187–90. https://doi.org/10.1161/STROKEAHA.109.565440.
Van der Weele TJ, Hernán MA. Results on differential and dependent measurement error of the exposure and the outcome using signed directed acyclic graphs. Am J Epidemiol. 2012;175(12):1303–10.
Hernán MA, Cole SR. Invited commentary: causal diagrams and measurement bias. Am J Epidemiol. 2009;170(8):959–62. https://doi.org/10.1093/aje/kwp293.
Xu S, Shetterly S, Raebel MA, Ho PM, Tsai TT, Magid D. Estimating the effects of time-varying exposures in observational studies using Cox models with stabilized weights adjustment. Pharmacoepidemiol Drug Saf. 2014;23(8):812–8. https://doi.org/10.1002/pds.3601.
Freedman LS, Schatzkin A, Midthune D, Kipnis V. Dealing with dietary measurement error in nutritional cohort studies. J Natl Cancer Inst. 2011;103(14):1086–92. https://doi.org/10.1093/jnci/djr189.
•• The European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP). Guide on methodological standards in pharmacoepidemiology (Revision 6). EMA/95098/2010. Available at http://www.encepp.eu/standards_and_guidances. Methodological guidance for the conduct of pharmacoepidemiological studies, most of which may also apply to observational nutritional research.
Gilbertson DT, Bradbury BD, Wetmore JB, Weinhandl ED, Monda KL, Liu J, et al. Controlling confounding of treatment effects in administrative data in the presence of time-varying baseline confounders. Pharmacoepidemiol Drug Saf. 2016;25(3):269–77. https://doi.org/10.1002/pds.3922.
Lopez-Garcia E. Long-term coffee consumption associated with reduced risk of total and cause-specific mortality. Evid Based Med. 2013;18(3):116–7. https://doi.org/10.1136/eb-2012-100878.
Experian Simmons. Coffee in America. Available at: https://www.experian.com/assets/simmons-research/white-papers/demographic-and-preferences-of-coffee-drinkers-in-america.pdf.
Gallup News. Drinking highest among educated, Upper-Income Americans. Available at: http://news.gallup.com/poll/184358/drinking-highest-among-educated-upper-income-americans.aspx.
United States Department of Agriculture. U.S. food commodity consumption broken down by demographics, 1994–2008. Available at: https://www.ers.usda.gov/webdocs/publications/45526/57057_err-206.pdf?v=42459.
Oakes JM, Andrade KE. The measurement of socioeconomic status. In: Oakes JM, Kaufman JS, editors. Methods in social epidemiology. 2nd ed. New York: Jossey-Bass; 2017.
Van der Weele TJ, Ding P. Sensitivity analysis in observational research: introducing the E-value. Ann Intern Med. 2017;167(4):268–74.
•• Guallar E, Blasco-Colmenares E, Arking DE, Zhao D. Moderate coffee intake can be part of a healthy diet. Ann Intern Med. 2017;167(4):283–4. https://doi.org/10.7326/M17-1503. Interesting discussion on some of the key strengths and limitations of recent studies on coffee intake and mortality.
Cappuccio FP, Cooper D, D’Elia L, Strazzullo P, Miller MA. Sleep duration predicts cardiovascular outcomes: a systematic review and metaanalysis of prospective studies. Eur Heart J. 2011;32(12):1484–92. https://doi.org/10.1093/eurheartj/ehr007.
Kim CW, Chang Y, Zhao D, Cainzos-Achirica M, Ryu S, Jung HS, et al. Sleep duration, sleep quality, and markers of subclinical arterial disease in healthy men and women. Arterioscler Thromb Vasc Biol. 2015;35(10):2238–45. https://doi.org/10.1161/ATVBAHA.115.306110.
Sabanayagam C, Shankar A. Sleep duration and cardiovascular disease: results from the National Health Interview Survey. Sleep. 2010;33(8):1037–42. https://doi.org/10.1093/sleep/33.8.1037.
Bao Y, Han J, Hu FB, Giovannucci EL, Stampfer MJ, Willett WC, et al. Association of nut consumption with total and cause-specific mortality. N Engl J Med. 2013;369(21):2001–11. https://doi.org/10.1056/NEJMoa1307352.
Estruch R, Ros E, Salas-Salvadó J, Covas MI, Corella D, Arós F, et al. Primary prevention of cardiovascular disease with a Mediterranean diet. N Engl J Med. 2013;368(14):1279–90. https://doi.org/10.1056/NEJMoa1200303.
Gore MO, McGuire DK. A test in context: hemoglobin A1c and cardiovascular disease. J Am Coll Cardiol. 2016;68(22):2479–86. https://doi.org/10.1016/j.jacc.2016.08.070.
•• Goodman SN, Schneeweiss S, Baiocchi M. Using design thinking to differentiate useful from misleading evidence in observational research. JAMA. 2017;317(7):705–7. https://doi.org/10.1001/jama.2016.19970. Insightful discussion on key methodological considerations relevant to the design and conduct of observational research studies.
Conflict of Interest
Dr. Bilal, Dr. Kapoor, Dr. Quispe, Dr. McEvoy, Dr. Pladevall-Vila, and Dr. Blumenthal declare that they have no conflict of interest.
Dr. Cainzos-Achirica reports that he collaborates with RTI Health Solutions, an independent nonprofit research organization that does work for government agencies and pharmaceutical companies.
Dr. Blaha reports grants from NIH, grants from AHA, grants and personal fees from FDA and Amgen, grants from Aetna Foundation, personal fees from Novartis, Siemens, Medimmune, Akcea, Sanofi, and Regeneron.
Human and Animal Rights and Informed Consent
This article does not contain any studies with human or animal subjects performed by the authors.
This article is part of the Topical Collection on Novel and Emerging Risk Factors
About this article
Cite this article
Cainzos-Achirica, M., Bilal, U., Kapoor, K. et al. Methodological Issues in Nutritional Epidemiology Research—Sorting Through the Confusion. Curr Cardiovasc Risk Rep 12, 4 (2018). https://doi.org/10.1007/s12170-018-0567-8
- Nutritional epidemiology
- Epidemiologic methods