Methodological Issues in Nutritional Epidemiology Research—Sorting Through the Confusion

Abstract

Purpose of Review

Our purpose was to discuss the methodological limitations of observational nutritional epidemiology research, using observational studies on coffee intake and health as a case example.

Recent Findings

A number of recent observational studies on the potential health effects of daily coffee intake have reported protective associations between higher coffee intake and a variety of health outcomes, including death. This is inconsistent with the findings from classic studies showing an increased risk of coronary heart disease events, performed in young adults with a homogeneous education level, and adjusting for tobacco use.

Summary

Many nutritional epidemiological studies have important limitations, which limit their validity. These include the use of prevalent user designs, risk of reverse causality, measurement error particularly of the exposure of interest, and residual confounding by socioeconomic status. In this review, we discuss these potential issues and provide constructive recommendations intended to help minimize them.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3

Abbreviations

SES:

Socioeconomic status

References

Papers of particular interest, published recently, have been highlighted as: Of importance •• Of major importance

  1. 1.

    •• Lachat C, Hawwash D, Ocké MC, Berg C, Forsum E, Hörnell A, et al. Strengthening the Reporting of Observational Studies in Epidemiology-Nutritional Epidemiology (STROBE-nut): an extension of the STROBE statement. PLoS Med. 2016;13(6):e1002036. https://doi.org/10.1371/journal.pmed.1002036. Extension of the STROBE statement, setting standards and recommendations specific to nutritional epidemiological research.

    Article  PubMed  PubMed Central  Google Scholar 

  2. 2.

    Mandrola J. Enough with the coffee research and other distractions. Medscape. Available at: http://www.medscape.com/viewarticle/883709.

  3. 3.

    • Freedman ND, Park Y, Abnet CC, Hollenbeck AR, Sinha R. Association of coffee drinking with total and cause-specific mortality. N Engl J Med. 2012;366(20):1891–904. https://doi.org/10.1056/NEJMoa1112010. The authors observed a protective association between higher coffee intake and all-cause mortality in a US population.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  4. 4.

    • Mostofsky E, Rice MS, Levitan EB, Mittleman MA. Habitual coffee consumption and risk of heart failure: a dose-response meta-analysis. Circ Heart Fail. 2012;5(4):401–5. https://doi.org/10.1161/CIRCHEARTFAILURE.112.967299. In this meta-analysis, the authors found a U-shaped association between higher coffee intake and incident heart failure.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  5. 5.

    • Kokubo Y, Iso H, Saito I, Yamagishi K, Yatsuya H, Ishihara J, et al. The impact of green tea and coffee consumption on the reduced risk of stroke incidence in Japanese population: the Japan public health center-based study cohort. Stroke. 2013;44(5):1369–74. https://doi.org/10.1161/STROKEAHA.111.677500. The authors report a protective association between higher coffee intake and the incidence of stroke.

    CAS  Article  PubMed  Google Scholar 

  6. 6.

    • Gunter MJ, Murphy N, Cross AJ, Dossus L, Dartois L, Fagherazzi G, et al. Coffee drinking and mortality in 10 European countries: a multinational cohort study. Ann Intern Med. 2017; https://doi.org/10.7326/M16-2945. Study reporting a protective association between higher coffee intake and death in 10 European countries.

  7. 7.

    • Park SY, Freedman ND, Haiman CA, Le Marchand L, Wilkens LR, Setiawan VW. Association of coffee consumption with total and cause-specific mortality among nonwhite populations. Ann Intern Med. 2017; https://doi.org/10.7326/M16-2472. The authors observed a protective association between higher coffee intake and mortality among non-white populations living in the USA.

  8. 8.

    • Jick H, Miettinen OS, Neff RK, Shapiro S, Heinonen OP, Slone D. Coffee and myocardial infarction. N Engl J Med. 1973;289(2):63–7. https://doi.org/10.1056/NEJM197307122890203. Classic paper reporting an increased risk of acute myocardial infraction with higher coffee intake.

    CAS  Article  PubMed  Google Scholar 

  9. 9.

    • MacMahon B, Yen S, Trichopoulos D, Warren K, Nardi G. Coffee and cancer of the pancreas. N Engl J Med. 1981;304(11):630–3. https://doi.org/10.1056/NEJM198103123041102. Classic report of a positive association between higher coffee intake and occurrence of pancreatic cancer.

    CAS  Article  PubMed  Google Scholar 

  10. 10.

    • Marrett LD, Walter SD, Meigs JW. Coffee drinking and bladder cancer in Connecticut. Am J Epidemiol. 1983;117(2):113–27. https://doi.org/10.1093/oxfordjournals.aje.a113522. The authors found an association between higher coffee intake and bladder cancer in a US population.

    CAS  Article  PubMed  Google Scholar 

  11. 11.

    • LaCroix AZ, Mead LA, Liang KY, Thomas CB, Pearson TA. Coffee consumption and the incidence of coronary heart disease. N Engl J Med. 1986;315(16):977–82. https://doi.org/10.1056/NEJM198610163151601. In the PRECURSORS cohort (mean age at baseline 26 years), the authors observed an association between higher coffee intake and risk of incident coronary heart disease events.

    CAS  Article  PubMed  Google Scholar 

  12. 12.

    • Klag MJ, Mead LA, LaCroix AZ, Wang NY, Coresh J, Liang KY, et al. Coffee intake and coronary heart disease. Ann Epidemiol. 1994;4(6):425–33. https://doi.org/10.1016/1047-2797(94)90001-9. Update of the prior study, using a longer follow-up period and updated analytic techniques.

    CAS  Article  PubMed  Google Scholar 

  13. 13.

    Margetts BM, Nelson M. Design concepts in nutritional epidemiology. 2nd ed. Oxford: Oxford University Press; 1997.

    Google Scholar 

  14. 14.

    Bekkering GE, Harris RJ, Thomas S, Mayer AM, Beynon R, Ness AR, et al. How much of the data published in observational studies of the association between diet and prostate or bladder cancer is usable for meta-analysis? Am J Epidemiol. 2008;167(9):1017–26. https://doi.org/10.1093/aje/kwn005.

    Article  PubMed  Google Scholar 

  15. 15.

    Palatini P. Letter by Palatini regarding article, “Habitual coffee consumption and risk of heart failure: a dose-response meta-analysis”. Circ Heart Fail. 2012;5(6):e98; author reply e99, DOI: https://doi.org/10.1161/CIRCHEARTFAILURE.112.970111.

  16. 16.

    Aubin HJ, Berlin I. Coffee drinking and mortality. N Engl J Med. 2012;367(6):576. author reply 576-7

    PubMed  Google Scholar 

  17. 17.

    Saloustros E, Stratakis CA. Coffee drinking and mortality. N Engl J Med. 2012;367(6):575–6. author reply 576-7

    Article  PubMed  Google Scholar 

  18. 18.

    Aberegg SK. Coffee drinking and mortality. N Engl J Med. 2012;367(6):575. author reply 576-7

    Article  PubMed  Google Scholar 

  19. 19.

    Bellach B, Kohlmeier L. Energy adjustment does not control for differential recall bias in nutritional epidemiology. J Clin Epidemiol. 1998;51(5):393–8. https://doi.org/10.1016/S0895-4356(97)00302-8.

    CAS  Article  PubMed  Google Scholar 

  20. 20.

    Lee BY. Forbes. No, These two studies don’t prove that coffee leads to longer life. Available at: https://www.forbes.com/sites/brucelee/2017/07/11/no-these-2-studies-dont-prove-that-coffee-leads-to-longer-life/#7beb00ef2d2f.

  21. 21.

    •• Barnard ND, Willett WC, Ding EL. The misuse of meta-analysis in nutrition research. JAMA. 2017;318(15):1435–6. https://doi.org/10.1001/jama.2017.12083. Insightful discussion on the limitations of meta-analyses in the context of nutritional epidemiological research.

  22. 22.

    Wu JN, Ho SC, Zhou C, Ling WH, Chen WQ, Wang CL, et al. Coffee consumption and risk of coronary heart diseases: a meta-analysis of 21 prospective cohort studies. Int J Cardiol. 2009;137(3):216–25. https://doi.org/10.1016/j.ijcard.2008.06.051.

    Article  PubMed  Google Scholar 

  23. 23.

    National Coffee Association USA. National coffee drinking trends. Available at: http://www.ncausa.org/Industry-Resources/Market-Research/National-Coffee-Drinking-Trends-Report.

  24. 24.

    Coffee Association of Canada. Coffee Facts. Available at: https://www.coffeeassoc.com/media-coffee-facts/.

  25. 25.

    Moride Y, Abenhaim L. Evidence of the depletion of susceptibles effect in non-experimental pharmacoepidemiologic research. J Clin Epidemiol. 1994;47(7):731–7.

    CAS  Article  PubMed  Google Scholar 

  26. 26.

    Shirlow MJ, Mathers CD. A study of caffeine consumption and symptoms; indigestion, palpitations, tremor, headache and insomnia. Int J Epidemiol. 1985;14(2):239–48. https://doi.org/10.1093/ije/14.2.239.

    CAS  Article  PubMed  Google Scholar 

  27. 27.

    Bruce MS, Lader M. Caffeine abstention in the management of anxiety disorders. Psychol Med. 1989;19(1):211–4. https://doi.org/10.1017/S003329170001117X.

    CAS  Article  PubMed  Google Scholar 

  28. 28.

    Liu H, Yao K, Zhang W, Zhou J, Wu T, He C. Coffee consumption and risk of fractures: a meta-analysis. Arch Med Sci. 2012;8(5):776–83. https://doi.org/10.5114/aoms.2012.31612.

    Article  PubMed  PubMed Central  Google Scholar 

  29. 29.

    O’Keefe JH, Bhatti SK, Patil HR, DiNicolantonio JJ, Lucan SC, Lavie CJ. Effects of habitual coffee consumption on cardiometabolic disease, cardiovascular health, and all-cause mortality. J Am Coll Cardiol. 2013;62(12):1043–51. https://doi.org/10.1016/j.jacc.2013.06.035.

    Article  PubMed  Google Scholar 

  30. 30.

    Ray WA. Evaluating medication effects outside of clinical trials: new-user designs. Am J Epidemiol. 2003;158(9):915–20. https://doi.org/10.1093/aje/kwg231.

    Article  PubMed  Google Scholar 

  31. 31.

    Hernán MA, Alonso A, Logan R, Grodstein F, Michels KB, Willett WC, et al. Observational studies analyzed like randomized experiments: an application to postmenopausal hormone therapy and coronary heart disease. Epidemiology. 2008;19(6):766–79. https://doi.org/10.1097/EDE.0b013e3181875e61.

    Article  PubMed  PubMed Central  Google Scholar 

  32. 32.

    Danaei G, Tavakkoli M, Hernán MA. Bias in observational studies of prevalent users: lessons for comparative effectiveness research from a meta-analysis of statins. Am J Epidemiol. 2012;175(4):250–62. https://doi.org/10.1093/aje/kwr301.

    Article  PubMed  PubMed Central  Google Scholar 

  33. 33.

    •• Sattar N, Preiss D. Reverse causality in cardiovascular epidemiological research: more common than imagined? Circulation. 2017;135(24):2369–72. https://doi.org/10.1161/CIRCULATIONAHA.117.028307. Expert comment on the relevance that reverse causality may have in cardiovascular epidemiology and its potential implications.

    Article  PubMed  Google Scholar 

  34. 34.

    Maselko J, Hayward RD, Hanlon A, Buka S, Meador K. Religious service attendance and major depression: a case of reverse causality? Am J Epidemiol. 2012;175(6):576–83. https://doi.org/10.1093/aje/kwr349.

    Article  PubMed  PubMed Central  Google Scholar 

  35. 35.

    Goldstein AM, Hodge SE, Haile RW. Selection bias in case-control studies using relatives as the controls. Int J Epidemiol. 1989;18(4):985–9. https://doi.org/10.1093/ije/18.4.985.

    CAS  Article  PubMed  Google Scholar 

  36. 36.

    Mendez MA. Invited commentary: dietary misreporting as a potential source of bias in diet-disease associations: future directions in nutritional epidemiology research. Am J Epidemiol. 2015;181(4):234–6. https://doi.org/10.1093/aje/kwu306.

    Article  PubMed  Google Scholar 

  37. 37.

    Schatzkin A, Kipnis V, Carroll RJ, Midthune D, Subar AF, Bingham S, et al. A comparison of a food frequency questionnaire with a 24-hour recall for use in an epidemiological cohort study: results from the biomarker-based Observing Protein and Energy Nutrition (OPEN) study. Int J Epidemiol. 2003;32(6):1054–62. https://doi.org/10.1093/ije/dyg264.

    Article  PubMed  Google Scholar 

  38. 38.

    Miller TM, Abdel-Maksoud MF, Crane LA, Marcus AC, Byers TE. Effects of social approval bias on self-reported fruit and vegetable consumption: a randomized controlled trial. Nutr J. 2008;7(1):18. https://doi.org/10.1186/1475-2891-7-18.

    Article  PubMed  PubMed Central  Google Scholar 

  39. 39.

    Galea S, Tracy M, Hoggatt KJ, Dimaggio C, Karpati A. Estimated deaths attributable to social factors in the United States. Am J Public Health. 2011;101(8):1456–65. https://doi.org/10.2105/AJPH.2010.300086.

    Article  PubMed  PubMed Central  Google Scholar 

  40. 40.

    Borrell LN, Diez Roux AV, Rose K, Catellier D, Clark BL. Neighbourhood characteristics and mortality in the Atherosclerosis Risk in Communities Study. Int J Epidemiol. 2004;33(2):398–407. https://doi.org/10.1093/ije/dyh063.

    Article  PubMed  Google Scholar 

  41. 41.

    Diez Roux AV, Merkin SS, Arnett D, Chambless L, Massing M, Nieto FJ, et al. Neighborhood of residence and incidence of coronary heart disease. N Engl J Med. 2001;345(2):99–106. https://doi.org/10.1056/NEJM200107123450205.

    CAS  Article  PubMed  Google Scholar 

  42. 42.

    Stringhini S, Carmeli C, Jokela M, Avendaño M, Muennig P, Guida F, et al. Socioeconomic status and the 25 × 25 risk factors as determinants of premature mortality: a multicohort study and meta-analysis of 1.7 million men and women. Lancet. 2017;389(10075):1229–37. https://doi.org/10.1016/S0140-6736(16)32380-7.

    Article  PubMed  PubMed Central  Google Scholar 

  43. 43.

    Pedersen SS, von Känel R, Tully PJ, Denollet J. Psychosocial perspectives in cardiovascular disease. Eur J Prev Cardiol. 2017;24(3_suppl):108–15. https://doi.org/10.1177/2047487317703827.

    Article  PubMed  Google Scholar 

  44. 44.

    Giltay EJ, Kamphuis MH, Kalmijn S, Zitman FG, Kromhout D. Dispositional optimism and the risk of cardiovascular death: the Zutphen Elderly Study. Arch Intern Med. 2006;166(4):431–6. https://doi.org/10.1001/archinte.166.4.431.

    PubMed  Google Scholar 

  45. 45.

    Nabi H, Koskenvuo M, Singh-Manoux A, Korkeila J, Suominen S, Korkeila K, et al. Low pessimism protects against stroke: the Health and Social Support (HeSSup) prospective cohort study. Stroke. 2010;41(1):187–90. https://doi.org/10.1161/STROKEAHA.109.565440.

    Article  PubMed  Google Scholar 

  46. 46.

    Van der Weele TJ, Hernán MA. Results on differential and dependent measurement error of the exposure and the outcome using signed directed acyclic graphs. Am J Epidemiol. 2012;175(12):1303–10.

    Article  Google Scholar 

  47. 47.

    Hernán MA, Cole SR. Invited commentary: causal diagrams and measurement bias. Am J Epidemiol. 2009;170(8):959–62. https://doi.org/10.1093/aje/kwp293.

    Article  PubMed  PubMed Central  Google Scholar 

  48. 48.

    Xu S, Shetterly S, Raebel MA, Ho PM, Tsai TT, Magid D. Estimating the effects of time-varying exposures in observational studies using Cox models with stabilized weights adjustment. Pharmacoepidemiol Drug Saf. 2014;23(8):812–8. https://doi.org/10.1002/pds.3601.

    PubMed  PubMed Central  Google Scholar 

  49. 49.

    Freedman LS, Schatzkin A, Midthune D, Kipnis V. Dealing with dietary measurement error in nutritional cohort studies. J Natl Cancer Inst. 2011;103(14):1086–92. https://doi.org/10.1093/jnci/djr189.

    Article  PubMed  PubMed Central  Google Scholar 

  50. 50.

    •• The European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP). Guide on methodological standards in pharmacoepidemiology (Revision 6). EMA/95098/2010. Available at http://www.encepp.eu/standards_and_guidances. Methodological guidance for the conduct of pharmacoepidemiological studies, most of which may also apply to observational nutritional research.

  51. 51.

    Gilbertson DT, Bradbury BD, Wetmore JB, Weinhandl ED, Monda KL, Liu J, et al. Controlling confounding of treatment effects in administrative data in the presence of time-varying baseline confounders. Pharmacoepidemiol Drug Saf. 2016;25(3):269–77. https://doi.org/10.1002/pds.3922.

    Article  PubMed  Google Scholar 

  52. 52.

    Lopez-Garcia E. Long-term coffee consumption associated with reduced risk of total and cause-specific mortality. Evid Based Med. 2013;18(3):116–7. https://doi.org/10.1136/eb-2012-100878.

    Article  PubMed  Google Scholar 

  53. 53.

    Experian Simmons. Coffee in America. Available at: https://www.experian.com/assets/simmons-research/white-papers/demographic-and-preferences-of-coffee-drinkers-in-america.pdf.

  54. 54.

    Gallup News. Drinking highest among educated, Upper-Income Americans. Available at: http://news.gallup.com/poll/184358/drinking-highest-among-educated-upper-income-americans.aspx.

  55. 55.

    United States Department of Agriculture. U.S. food commodity consumption broken down by demographics, 1994–2008. Available at: https://www.ers.usda.gov/webdocs/publications/45526/57057_err-206.pdf?v=42459.

  56. 56.

    Oakes JM, Andrade KE. The measurement of socioeconomic status. In: Oakes JM, Kaufman JS, editors. Methods in social epidemiology. 2nd ed. New York: Jossey-Bass; 2017.

    Google Scholar 

  57. 57.

    Van der Weele TJ, Ding P. Sensitivity analysis in observational research: introducing the E-value. Ann Intern Med. 2017;167(4):268–74.

    Article  Google Scholar 

  58. 58.

    •• Guallar E, Blasco-Colmenares E, Arking DE, Zhao D. Moderate coffee intake can be part of a healthy diet. Ann Intern Med. 2017;167(4):283–4. https://doi.org/10.7326/M17-1503. Interesting discussion on some of the key strengths and limitations of recent studies on coffee intake and mortality.

    Article  PubMed  Google Scholar 

  59. 59.

    Cappuccio FP, Cooper D, D’Elia L, Strazzullo P, Miller MA. Sleep duration predicts cardiovascular outcomes: a systematic review and metaanalysis of prospective studies. Eur Heart J. 2011;32(12):1484–92. https://doi.org/10.1093/eurheartj/ehr007.

    Article  PubMed  Google Scholar 

  60. 60.

    Kim CW, Chang Y, Zhao D, Cainzos-Achirica M, Ryu S, Jung HS, et al. Sleep duration, sleep quality, and markers of subclinical arterial disease in healthy men and women. Arterioscler Thromb Vasc Biol. 2015;35(10):2238–45. https://doi.org/10.1161/ATVBAHA.115.306110.

    CAS  Article  PubMed  Google Scholar 

  61. 61.

    Sabanayagam C, Shankar A. Sleep duration and cardiovascular disease: results from the National Health Interview Survey. Sleep. 2010;33(8):1037–42. https://doi.org/10.1093/sleep/33.8.1037.

    Article  PubMed  PubMed Central  Google Scholar 

  62. 62.

    Bao Y, Han J, Hu FB, Giovannucci EL, Stampfer MJ, Willett WC, et al. Association of nut consumption with total and cause-specific mortality. N Engl J Med. 2013;369(21):2001–11. https://doi.org/10.1056/NEJMoa1307352.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  63. 63.

    Estruch R, Ros E, Salas-Salvadó J, Covas MI, Corella D, Arós F, et al. Primary prevention of cardiovascular disease with a Mediterranean diet. N Engl J Med. 2013;368(14):1279–90. https://doi.org/10.1056/NEJMoa1200303.

    CAS  Article  PubMed  Google Scholar 

  64. 64.

    Gore MO, McGuire DK. A test in context: hemoglobin A1c and cardiovascular disease. J Am Coll Cardiol. 2016;68(22):2479–86. https://doi.org/10.1016/j.jacc.2016.08.070.

    Article  PubMed  Google Scholar 

  65. 65.

    •• Goodman SN, Schneeweiss S, Baiocchi M. Using design thinking to differentiate useful from misleading evidence in observational research. JAMA. 2017;317(7):705–7. https://doi.org/10.1001/jama.2016.19970. Insightful discussion on key methodological considerations relevant to the design and conduct of observational research studies.

    Article  PubMed  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Michael J. Blaha.

Ethics declarations

Conflict of Interest

Dr. Bilal, Dr. Kapoor, Dr. Quispe, Dr. McEvoy, Dr. Pladevall-Vila, and Dr. Blumenthal declare that they have no conflict of interest.

Dr. Cainzos-Achirica reports that he collaborates with RTI Health Solutions, an independent nonprofit research organization that does work for government agencies and pharmaceutical companies.

Dr. Blaha reports grants from NIH, grants from AHA, grants and personal fees from FDA and Amgen, grants from Aetna Foundation, personal fees from Novartis, Siemens, Medimmune, Akcea, Sanofi, and Regeneron.

Human and Animal Rights and Informed Consent

This article does not contain any studies with human or animal subjects performed by the authors.

Additional information

This article is part of the Topical Collection on Novel and Emerging Risk Factors

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Cainzos-Achirica, M., Bilal, U., Kapoor, K. et al. Methodological Issues in Nutritional Epidemiology Research—Sorting Through the Confusion. Curr Cardiovasc Risk Rep 12, 4 (2018). https://doi.org/10.1007/s12170-018-0567-8

Download citation

Keywords

  • Nutritional epidemiology
  • Observational
  • Confounding
  • Bias
  • Epidemiologic methods
  • Coffee