Journal of General Internal Medicine

, Volume 29, Issue 4, pp 653–660 | Cite as

Multimorbidity and Evidence Generation

  • Carlos O. Weiss
  • Ravi Varadhan
  • Milo A. Puhan
  • Andrew Vickers
  • Karen Bandeen-Roche
  • Cynthia M. Boyd
  • David M. KentEmail author
Multimorbidity Symposium


Most people with a chronic disease actually have more than one, a condition known as multimorbidity. Despite this, the evidence base to prevent adverse disease outcomes has taken a disease-specific approach. Drawing on a conference, Improving Guidelines for Multimorbid Patients, the goal of this paper is to identify challenges to the generation of evidence to support the care of people with multimorbidity and to make recommendations for improvement. We identified three broad categories of challenges: 1) challenges to defining and measuring multimorbidity; 2) challenges related to the effects of multimorbidity on study design, implementation and analysis; and 3) challenges inherent in studying heterogeneity of treatment effects in patients with differing comorbid conditions. We propose a set of recommendations for consideration by investigators and others (reviewers, editors, funding agencies, policymaking organizations) involved in the creation of evidence for this common type of person that address each of these challenges. The recommendations reflect a general approach that emphasizes broader inclusion (recruitment and retention) of patients with multimorbidity, coupled with more rigorous efforts to measure comorbidity and comorbidity burden and the influence of multimorbidity on outcomes and the effects of therapy. More rigorous examination of heterogeneity of treatment effects requires careful attention to prioritizing the most important comorbid-related questions, and also requires studies that provide greater statistical power than conventional trials have provided. Relatively modest changes in the orientation of current research along these lines can be helpful in pointing to and partially addressing selected knowledge gaps. However, producing a robust evidence base to support patient-centered decision making in complex individuals with multimorbidity, exposed to many different combinations of potentially interacting factors that can modify the risks and benefits of therapies, is likely to require a clinical research enterprise fundamentally restructured to be more fully integrated with routine clinical practice.


evidence-based medicine chronic disease guidelines comorbidity clinical trials 



Developing Guidelines for Multimorbidity Core Group

Cynthia M. Boyd, Johns Hopkins Medical Institutes (JHMI), Syndey M. Dy, Johns Hopkins School of Public Health (JHSPH), David M. Kent, Tufts Medical Center (TMC), Bruce Leff, JHMI, Thomas A. Trikalinos, Brown University, Katrin Uhlig, TMC, Ravi Varadhan, JHMI, Carlos O. Weiss, JHMI.

Evidence Generation Work Group

Lawrence Appel, JHU, Karen Bandeen-Roche, JHU, Erica Breslau, NCI, Joseph Cappelleri, Pfizer, Dean Follmann, NIAID, Sherrie Kaplan, University of California-Irvine, David Kent (Co-PI), TMC, Todd Lee, University of Illinois at Chicago, Matthew McNabney, JHU, Erin Murphy, JHU, Milo Puhan, Johns Hopkins University (JHU), Sergei Romashkan, NIA, A. Sedrakyan, FDA/Cornell, Vincenza Snow, Pfizer, Ravi Varadhan (Co-I), JHU, Carlos Weiss (Co-I), JHU.


This manuscript was partially funded by grants R21 HS18597 and R21 HS017653 from the Agency for Healthcare Research and Quality and UL1RR025752 from the National Institutes of Health.

Conflict of Interest

The authors declare they do not have any conflicts of interest.


  1. 1.
    Weiss CO, Boyd CM, Yu Q, Wolff JL, Leff B. Patterns of prevalent major chronic disease among older adults in the United States. JAMA. 2007;298:1160–2.PubMedCrossRefGoogle Scholar
  2. 2.
    Wang R, Lagakos SW, Ware JH, Hunter DJ, Drazen JM. Statistics in medicine—reporting of subgroup analyses in clinical trials. N Engl J Med. 2007;357:2189–94.PubMedCrossRefGoogle Scholar
  3. 3.
    Boyd CM, Darer J, Boult C, Fried LP, Boult L, Wu AW. Clinical practice guidelines and quality of care for older patients with multiple comorbid diseases: implications for pay for performance. JAMA. 2005;294:716–24.PubMedCrossRefGoogle Scholar
  4. 4.
    Trikalinos TA, Segal JB, Boyd CM. Addressing Multimorbidity in evidence integration and synthesis. J Gen Intern Med. 2013, doi:  10.1007/s11606-013-2661-4.
  5. 5.
    Uhlig K, Leff B, Kent DM, Dy S, Brunnhuber K, Burgers JS, Greenfield S, Guyatt G, High K, Leipzig R, Mulrow C, Schmader K, Schunemann H, Walter LC, Woodcock J, Boyd CM. A framework for crafting clinical practice guidelines that are relevant to the care and management of people with multimorbidity. J Gen Intern Med. 2013, doi: 10.1007/s11606-013-2659-y.
  6. 6.
    Werner RM, Greenfield S, Fung C, Turner BJ. Measuring quality of care in patients with multiple clinical conditions: summary of a conference conducted by the Society of General Internal Medicine. J Gen Intern Med. 2007;22:1206–11.PubMedCentralPubMedCrossRefGoogle Scholar
  7. 7.
    Lash TL, Mor V, Wieland D, Ferrucci L, Satariano W, Silliman RA. Methodology, design, and analytic techniques to address measurement of comorbid disease. J Gerontol A Biol Sci Med Sci. 2007;62:281–5.PubMedCentralPubMedCrossRefGoogle Scholar
  8. 8.
    Boyd CM, Weiss CO, Halter J, Han KC, Ershler WB, Fried LP. Framework for evaluating disease severity measures in older adults with comorbidity. J Gerontol A Biol Sci Med Sci. 2007;62:286–95.PubMedCrossRefGoogle Scholar
  9. 9.
    Yancik R, Ershler W, Satariano W, Hazzard W, Cohen HJ, Ferrucci L. Report of the national institute on aging task force on comorbidity. J Gerontol A Biol Sci Med Sci. 2007;62:275–80.PubMedCentralPubMedCrossRefGoogle Scholar
  10. 10.
    Karlamangla A, Tinetti M, Guralnik J, Studenski S, Wetle T, Reuben D. Comorbidity in older adults: nosology of impairment, diseases, and conditions. J Gerontol A Biol Sci Med Sci. 2007;62:296–300.PubMedCrossRefGoogle Scholar
  11. 11.
    Reuters T. Web of Science 2010. Available at: Accessed September, 2013.
  12. 12.
    Guralnik JM. Assessing the impact of comorbidity in the older population. Ann Epidemiol. 1996;6:376–80.PubMedCrossRefGoogle Scholar
  13. 13.
    Guralnik JM, LaCroix AZ, Everett DF, Kovar MG. Aging in the Eighties: The Prevalence of Comorbidity and its Association With Disability. Hyattsville: National Center for Health. Statistics. 1989;1989:170.Google Scholar
  14. 14.
    Verbrugge LM, Lepkowski JM, Imanaka Y. Comorbidity and its impact on disability. Milbank Q. 1989;67:450–84.PubMedCrossRefGoogle Scholar
  15. 15.
    Crews JE, Campbell VA. Vision impairment and hearing loss among community-dwelling older Americans: implications for health and functioning. Am J Public Health. 2004;94:823–9.PubMedCentralPubMedCrossRefGoogle Scholar
  16. 16.
    Guralnik JM. The impact of vision and hearing impairments on health in old age. J Am Geriatr Soc. 1999;47:1029–31.PubMedGoogle Scholar
  17. 17.
    Safford MM, Allison JJ, Kiefe CI. Patient complexity: more than comorbidity. the vector model of complexity. J Gen Intern Med. 2007;22(Suppl 3):382–90.PubMedCentralPubMedCrossRefGoogle Scholar
  18. 18.
    de Groot V, Beckerman H, Lankhorst GJ, Bouter LM. How to measure comorbidity. a critical review of available methods. J Clin Epidemiol. 2003;56:221–9.PubMedCrossRefGoogle Scholar
  19. 19.
    Diederichs C, Berger K, Bartels DB. The measurement of multiple chronic diseases—a systematic review on existing multimorbidity indices. J Gerontol A Biol Sci Med Sci. 2011;66:301–11.PubMedCrossRefGoogle Scholar
  20. 20.
    Piette JD, Kerr EA. The impact of comorbid chronic conditions on diabetes care. Diabetes Care. 2006;29:725–31.PubMedCrossRefGoogle Scholar
  21. 21.
    Hayward RA, Kent DM, Vijan S, Hofer TP. Multivariable risk prediction can greatly enhance the statistical power of clinical trial subgroup analysis. BMC Med Res Methodol. 2006;6:18.PubMedCentralPubMedCrossRefGoogle Scholar
  22. 22.
    Green SB, Byar DP. Using observational data from registries to compare treatments: the fallacy of omnimetrics. Stat Med. 1984;3:361–73.PubMedCrossRefGoogle Scholar
  23. 23.
    Byar DP. Why data bases should not replace randomized clinical trials. Biometrics. 1980;36:337–42.PubMedCrossRefGoogle Scholar
  24. 24.
    Van Spall HG, Toren A, Kiss A, Fowler RA. Eligibility criteria of randomized controlled trials published in high-impact general medical journals: a systematic sampling review. JAMA. 2007;297:1233–40.PubMedCrossRefGoogle Scholar
  25. 25.
    Fortin M, Dionne J, Pinho G, Gignac J, Almirall J, Lapointe L. Randomized controlled trials: do they have external validity for patients with multiple comorbidities? Ann Fam Med. 2006;4:104–8.PubMedCentralPubMedCrossRefGoogle Scholar
  26. 26.
    Kravitz RL, Duan N, Braslow J. Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages. Milbank Q. 2004;82:661–87.PubMedCentralPubMedCrossRefGoogle Scholar
  27. 27.
    Treweek S, Zwarenstein M. Making trials matter: pragmatic and explanatory trials and the problem of applicability. Trials. 2009;10:37.PubMedCentralPubMedCrossRefGoogle Scholar
  28. 28.
    Thorpe KE, Zwarenstein M, Oxman AD, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. CMAJ. 2009;180:E47–57.PubMedCentralPubMedCrossRefGoogle Scholar
  29. 29.
    Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA. 2003;290:1624–32.PubMedCrossRefGoogle Scholar
  30. 30.
    Kent DM, Hayward RA. Limitations of applying summary results of clinical trials to individual patients: the need for risk stratification. JAMA. 2007;298:1209–12.PubMedCrossRefGoogle Scholar
  31. 31.
    Rothwell PM. Can overall results of clinical trials be applied to all patients? Lancet. 1995;345:1616–9.PubMedCrossRefGoogle Scholar
  32. 32.
    Bach PB, Gould MK. When the Average Applies to No One: Personalized Decision Making About Potential Benefits of Lung Cancer Screening. Ann Intern Med. 2012;157:571–3.PubMedCrossRefGoogle Scholar
  33. 33.
    Furberg CD, Byington RP. What do subgroup analyses reveal about differential response to beta-blocker therapy? The Beta-Blocker Heart Attack Trial experience. Circulation. 1983;67:I98–101.PubMedGoogle Scholar
  34. 34.
    Brookes ST, Whitley E, Peters TJ, Mulheran PA, Egger M, Davey SG. Subgroup analyses in randomised controlled trials: quantifying the risks of false-positives and false-negatives. Health Technol Assess. 2001;5:1–56.PubMedGoogle Scholar
  35. 35.
    Tannock IF. False-positive results in clinical trials: multiple significance tests and the problem of unreported comparisons. J Natl Cancer Inst. 1996;88:206–7.PubMedCrossRefGoogle Scholar
  36. 36.
    Kent DM, Rothwell PM, Ioannidis JP, Altman DG, Hayward RA. Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal. Trials. 2010;11:85.PubMedCentralPubMedCrossRefGoogle Scholar
  37. 37.
    Weiss CO, Segal JB, Boyd CM, Wu A, Varadhan R. A Framework to Identify and Address Heterogeneity of Treatment Effect in Comparative Effectiveness Research. Rockville: MD; 2010.Google Scholar
  38. 38.
    Varadhan R, Segal JB, Boyd CM, Wu AW, Weiss CO. A framework for the analysis of heterogeneity of treatment effect in patient-centered outcomes research. Journal of Clinical Epidemiology. 2013. (In Press).Google Scholar
  39. 39.
    Varadhan R, Stuart EA, Louis TA, Segal JB, Weiss CO. Review of Guidance Documents for Selected Methods in Patient Centered Outcomes Research: Standards in Addressing Heterogeneity of Treatment Effectiveness in Observational and Experimental Patient Centered Outcomes Research. A Report to the PCORI Methodology Committee Research Methods Working Group. 2012. March 29, 2012.Google Scholar
  40. 40.
    Vickers AJ, Scardino PT. The clinically-integrated randomized trial: proposed novel method for conducting large trials at low cost. Trials. 2009;10:14.PubMedCentralPubMedCrossRefGoogle Scholar
  41. 41.
    Wizeman TM IoM. Sex-Specific Reporting of Scientific Research: A Workshop Summary. Washington DC: The National Academies Press; 2012.Google Scholar
  42. 42.
    Califf RM, Zarin DA, Kramer JM, Sherman RE, Aberle LH, Tasneem A. Characteristics of clinical trials registered in, 2007–2010. JAMA. 2012;307:1838–47.PubMedCrossRefGoogle Scholar
  43. 43.
    Kent DM, Kitsios G. Against pragmatism: on efficacy, effectiveness and the real world. Trials. 2009;10:48.PubMedCentralPubMedCrossRefGoogle Scholar
  44. 44.
    Califf RM, Filerman GL, Murray RK, Rosenblatt M, Merck, Co I. The Clinical Trials Enterprise in the United States: A Call for Disruptive Innovation. Washington: Institute of Medicine; 2012. Discussion Paper.Google Scholar
  45. 45.
    Sabin JE, Mazor K, Meterko V, Goff SL, Platt R. Comparing drug effectiveness at health plans: the ethics of cluster randomized trials. Hastings Cent Rep. 2008;38:39–48.PubMedCrossRefGoogle Scholar
  46. 46.
    Vickers AJ. Whose data set is it anyway? Sharing raw data from randomized trials. Trials. 2006;7:15.PubMedCentralPubMedCrossRefGoogle Scholar
  47. 47.
    Vickers AJ. Making raw data more widely available. BMJ. 2011;342:d2323.PubMedCrossRefGoogle Scholar
  48. 48.
    Ross JS, Krumholz HM. Ushering in a new era of open science through data sharing: the wall must come down. JAMA. 2013;309:1355–6.PubMedCrossRefGoogle Scholar
  49. 49.
    Sandercock PA, Niewada M, Czlonkowska A. The International Stroke Trial database. Trials. 2011;12:101.PubMedCentralPubMedCrossRefGoogle Scholar
  50. 50.
    Specks U, Merkel PA, Seo P, et al. Efficacy of remission-induction regimens for ANCA-associated vasculitis. N Engl J Med. 2013;369:417–27.PubMedCrossRefGoogle Scholar
  51. 51.
    National Heart Lung and Blood Institute. NHLBI. Available at: Accessed August 15, 2013.
  52. 52.
    National Institute of Diabetes and Dignestive and Kidney Diseases. National Institute of Diabetes and Dignestive and Kidney Diseases (NIDDK) Central Data Repository (CDR). Available at: Accessed August 13, 2013.
  53. 53.
    GlaxoSmithKline. Clinical study requests—initial members of the Independent Review Panel. Available at: Accessed August 15, 2013.
  54. 54.
    European Medicines Agency. Release of data from clinical trials. Available at: Accessed August 15, 2013.
  55. 55.
    United States Department of Health and Human Services. Strategic Framework on Multiple Chronic Conditions, 2010.Google Scholar

Copyright information

© Society of General Internal Medicine 2013

Authors and Affiliations

  • Carlos O. Weiss
    • 1
    • 2
  • Ravi Varadhan
    • 2
    • 3
    • 4
  • Milo A. Puhan
    • 5
  • Andrew Vickers
    • 6
  • Karen Bandeen-Roche
    • 3
    • 4
  • Cynthia M. Boyd
    • 2
  • David M. Kent
    • 7
    Email author
  1. 1.Department of Family Medicine, College of Human MedicineMichigan State UniversityEast LansingUSA
  2. 2.Division of Geriatric Medicine and GerontologyJohns Hopkins UniversityBaltimoreUSA
  3. 3.The Johns Hopkins Center on Aging and HealthBaltimoreUSA
  4. 4.Department of BiostatisticsJohns Hopkins Bloomberg School of Public HealthBaltimoreUSA
  5. 5.Department of EpidemiologyJohns Hopkins Bloomberg School of Public HealthBaltimoreUSA
  6. 6.Department of Epidemiology & BiostatisticsMemorial Sloan-Kettering Cancer CenterNew YorkUSA
  7. 7.Predictive Analytics and Comparative Effectiveness (PACE) Center, Institute for Clinical Research and Health Policy StudiesTufts Medical CenterBostonUSA

Personalised recommendations