Journal of General Internal Medicine

, Volume 33, Issue 12, pp 2120–2126 | Cite as

Identifying Latent Subgroups of High-Risk Patients Using Risk Score Trajectories

  • Edwin S. WongEmail author
  • Jean Yoon
  • Rebecca I. Piegari
  • Ann-Marie M. Rosland
  • Stephan D. Fihn
  • Evelyn T. Chang
Original Research



Many healthcare systems employ population-based risk scores to prospectively identify patients at high risk of poor outcomes, but it is unclear whether single point-in-time scores adequately represent future risk. We sought to identify and characterize latent subgroups of high-risk patients based on risk score trajectories.

Study Design

Observational study of 7289 patients discharged from Veterans Health Administration (VA) hospitals during a 1-week period in November 2012 and categorized in the top 5th percentile of risk for hospitalization.


Using VA administrative data, we calculated weekly risk scores using the validated Care Assessment Needs model, reflecting the predicted probability of hospitalization. We applied the non-parametric k-means algorithm to identify latent subgroups of patients based on the trajectory of patients’ hospitalization probability over a 2-year period. We then compared baseline sociodemographic characteristics, comorbidities, health service use, and social instability markers between identified latent subgroups.


The best-fitting model identified two subgroups: moderately high and persistently high risk. The moderately high subgroup included 65% of patients and was characterized by moderate subgroup-level hospitalization probability decreasing from 0.22 to 0.10 between weeks 1 and 66, then remaining constant through the study end. The persistently high subgroup, comprising the remaining 35% of patients, had a subgroup-level probability increasing from 0.38 to 0.41 between weeks 1 and 52, and declining to 0.30 at study end. Persistently high-risk patients were older, had higher prevalence of social instability and comorbidities, and used more health services.


On average, one third of patients initially identified as high risk stayed at very high risk over a 2-year follow-up period, while risk for the other two thirds decreased to a moderately high level. This suggests that multiple approaches may be needed to address high-risk patient needs longitudinally or intermittently.


high risk risk stratification trajectory latent subgroups machine learning patient-centered medical home 



The authors acknowledge helpful comments from Matthew Maciejewski, Sandeep Vijan, Donna Zulman, Karin Nelson, and Joshua Thorpe. Preliminary findings from this work were presented at the 2017 AcademyHealth Annual Research Meeting on June 26, 2017 in New Orleans, LA and at the 2017 VA HSR&D National Conference on July 19, 2017 in Arlington, VA.

Funding information

This study was funded by the Veterans Health Administration Patient Aligned Care Team Demonstration Laboratory Coordination Center (XVA-61-041). Dr. Wong is supported by VA Health Services Research and Development Career Development Award 13-024.

Compliance with Ethical Standards


The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs, the University of Washington, the University of California, and the University of Pittsburgh.

Conflict of Interest

Dr. Wong reports ownership of common stock in UnitedHealth Group Inc. All other authors have no conflicts of interest to report.

Supplementary material

11606_2018_4653_MOESM1_ESM.pdf (281 kb)
ESM 1 (PDF 281 kb)


  1. 1.
    Edwards ST, Peterson K, Chan B, Anderson J, Helfand M. Effectiveness of Intensive Primary Care Interventions: A Systematic Review. J Gen Intern Med. 2017;32(12):1377–1386.CrossRefGoogle Scholar
  2. 2.
    Peterson K, Helfand M, Humphrey L, Christensen V, Carson S. Evidence Brief: Effectiveness of Intensive Primary Care Programs. Accessed Nov 1, 2017.
  3. 3.
    Powers BW, Chaguturu SK. ACOs and High-Cost Patients. N Engl J Med. 2016;374(3):203–205.CrossRefGoogle Scholar
  4. 4.
    Lee NS, Whitman N, Vakharia N, Ph DG, Rothberg MB. High-Cost Patients: Hot-Spotters Don’t Explain the Half of It. J Gen Intern Med. 2017;32(1):28–34.CrossRefGoogle Scholar
  5. 5.
    Hochman M, Asch SM. Disruptive Models in Primary Care: Caring for High-Needs, High-Cost Populations. J Gen Intern Med. 2017;32(4):392–397.CrossRefGoogle Scholar
  6. 6.
    Blumenthal D, Chernof B, Fulmer T, Lumpkin J, Selberg J. Caring for High-Need, High-Cost Patients - An Urgent Priority. N Engl J Med. 2016;375(10):909–911.CrossRefGoogle Scholar
  7. 7.
    Gill TM, Gahbauer EA, Han L, Allore HG. Trajectories of disability in the last year of life. N Engl J Med. 2010;362(13):1173–1180.CrossRefGoogle Scholar
  8. 8.
    Franklin JM, Shrank WH, Pakes J, et al. Group-based trajectory models: a new approach to classifying and predicting long-term medication adherence. Med Care. 2013;51(9):789–796.CrossRefGoogle Scholar
  9. 9.
    Ravona-Springer R, Heymann A, Schmeidler J, et al. Trajectories in glycemic control over time are associated with cognitive performance in elderly subjects with type 2 diabetes. PLoS One. 2014;9(6):e97384.CrossRefGoogle Scholar
  10. 10.
    Lauffenburger JC, Franklin JM, Krumme AA, et al. Longitudinal Patterns of Spending Enhance the Ability to Predict Costly Patients: A Novel Approach to Identify Patients for Cost Containment. Med Care. 2017;55(1):64–73.CrossRefGoogle Scholar
  11. 11.
    O'Hare AM, Batten A, Burrows NR, et al. Trajectories of kidney function decline in the 2 years before initiation of long-term dialysis. Am J Kidney Dis. 2012;59(4):513–522.CrossRefGoogle Scholar
  12. 12.
    Nagin DS, Odgers CL. Group-based trajectory modeling in clinical research. Annu Rev Clin Psychol. 2010;6:109–138.CrossRefGoogle Scholar
  13. 13.
    Lanza ST, Rhoades BL. Latent class analysis: an alternative perspective on subgroup analysis in prevention and treatment. Prev Sci. 2013;14(2):157–168.CrossRefGoogle Scholar
  14. 14.
    Institute of Medicine. Capturing Social and Behavioral Domains in Electronic Health Records: Phase 1. Washington, DC 2014.Google Scholar
  15. 15.
    Yoon J, Chee CP, Su P, et al. Persistence of High Health Care Costs among VA Patients. Health Serv Res. 2018.Google Scholar
  16. 16.
    Wang L, Porter B, Maynard C, et al. Predicting risk of hospitalization or death among patients receiving primary care in the Veterans Health Administration. Med Care. 2013;51(4):368–373.CrossRefGoogle Scholar
  17. 17.
    Hartigan JA, Wong MA. Algorithm AS 136: A K-Means Clustering Algorithm. Journal of the Royal Statistical Society. Series C (Applied Statistics). 1979;28(1):100–108.Google Scholar
  18. 18.
    Genolini C, Alacoque X, Sentenac M, Arnaud C. kml and kml3d: R Packages to Cluster Longitudinal Data. Journal of Statistical Software. 2016;65(4).Google Scholar
  19. 19.
    Gagne JJ, Glynn RJ, Avorn J, Levin R, Schneeweiss S. A combined comorbidity score predicted mortality in elderly patients better than existing scores. J Clin Epidemiol. 2011;64(7):749–759.CrossRefGoogle Scholar
  20. 20.
    Sloan KL, Montez-Rath ME, Spiro A, 3rd, et al. Development and validation of a psychiatric case-mix system. Med Care. 2006;44(6):568–580.CrossRefGoogle Scholar
  21. 21.
    Amarasingham R, Moore BJ, Tabak YP, et al. An automated model to identify heart failure patients at risk for 30-day readmission or death using electronic medical record data. Med Care. 2010;48(11):981–988.CrossRefGoogle Scholar
  22. 22.
    Peterson R, Gundlapalli AV, Metraux S, et al. Identifying Homelessness among Veterans Using VA Administrative Data: Opportunities to Expand Detection Criteria. PLoS One. 2015;10(7):e0132664.CrossRefGoogle Scholar
  23. 23.
    Liu CF, Manning WG, Burgess JF, Jr., et al. Reliance on Veterans Affairs outpatient care by Medicare-eligible veterans. Med Care. 2011;49(10):911–917.CrossRefGoogle Scholar
  24. 24.
    Nelson K, Schwartz G, Hernandez S, et al. The Association Between Neighborhood Environment and Mortality: Results from a National Study of Veterans. J Gen Intern Med. 2017;32(4):416–422.CrossRefGoogle Scholar
  25. 25.
    Normand ST, Landrum MB, Guadagnoli E, et al. Validating recommendations for coronary angiography following acute myocardial infarction in the elderly: a matched analysis using propensity scores. J Clin Epidemiol. 2001;54(4):387–398.CrossRefGoogle Scholar
  26. 26.
    Austin PC. An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies. Multivariate Behav Res. 2011;46(3):399–424.CrossRefGoogle Scholar
  27. 27.
    Hynes DM, Koelling K, Stroupe K, et al. Veterans’ access to and use of Medicare and Veterans Affairs health care. Med Care. 2007;45(3):214–223.CrossRefGoogle Scholar
  28. 28.
    Petersen LA, Byrne MM, Daw CN, et al. Relationship between clinical conditions and use of Veterans Affairs health care among Medicare-enrolled veterans. Health Serv Res. 2010;45(3):762–791.CrossRefGoogle Scholar
  29. 29.
    Department of Veterans Affairs. Home Based Primary Care. Accessed Nov 1, 2017.
  30. 30.
    U.S. Department of Veterans Affairs. Improving the Health and Care for Veterans with Cardiovascular Disease. Accessed Nov 1, 2017.
  31. 31.
    Rinne ST, Elwy AR, Bastian LA, et al. Impact of Multisystem Health Care on Readmission and Follow-up Among Veterans Hospitalized for Chronic Obstructive Pulmonary Disease. Med Care. 2017;55 Suppl 7 Suppl 1:S20-S25.CrossRefGoogle Scholar
  32. 32.
    Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998;36(1):8–27.CrossRefGoogle Scholar
  33. 33.
    Fortin Y, Crispo JA, Cohen D, et al. External validation and comparison of two variants of the Elixhauser comorbidity measures for all-cause mortality. PLoS One. 2017;12(3):e0174379.CrossRefGoogle Scholar
  34. 34.
    Prenovost K, Maciejewski ML, Vijan S, et al. Mixture Distribution Item Response Theory As a Method for Identifying Clinically Relevant Subgroups of Patients Accessed Jun 12, 2018.
  35. 35.
    Porter ME, Pabo EA, Lee TH. Redesigning primary care: a strategic vision to improve value by organizing around patients' needs. Health Aff (Millwood). 2013;32(3):516–525.CrossRefGoogle Scholar
  36. 36.
    Scholle SH, Saunders RC, Tirodkar MA, Torda P, Pawlson LG. Patient-centered medical homes in the United States. J Ambul Care Manage. 2011;34(1):20–32.CrossRefGoogle Scholar
  37. 37.
    National Committee for Quality Assurance. PCMH Recognition. Accessed Aug 1, 2017.
  38. 38.
    Sinaiko AD, Landrum MB, Meyers DJ, et al. Synthesis Of Research On Patient-Centered Medical Homes Brings Systematic Differences Into Relief. Health Aff (Millwood). 2017;36(3):500–508.CrossRefGoogle Scholar
  39. 39.
    Ewigman NL, Vella L, Eng JA. Implementing evidence-based interventions to prevent readmissions in the real world. J Gen Intern Med. 2018;33(5):578–580.CrossRefGoogle Scholar
  40. 40.
    Yoon J, Chang E, Rubenstein LV, et al. Impact of Primary Care Intensive Management on High-Risk Veterans’ Costs and Utilization: A Randomized Quality Improvement Trial. Ann Intern Med. 2018;168(12):846–854.CrossRefGoogle Scholar
  41. 41.
    Zulman DM, Pal Chee C, Ezeji-Okoye SC, et al. Effect of an Intensive Outpatient Program to Augment Primary Care for High-Need Veterans Affairs Patients: A Randomized Clinical Trial. JAMA Intern Med. 2017;177(2):166–175.CrossRefGoogle Scholar
  42. 42.
    Wong ES, Wang V, Liu CF, Hebert PL, Maciejewski ML. Do Veterans Health Administration Enrollees Generalize to Other Populations? Med Care Res Rev. 2016;73(4):493–507.CrossRefGoogle Scholar
  43. 43.
    Johnson TL, Rinehart DJ, Durfee J, et al. For many patients who use large amounts of health care services, the need is intense yet temporary. Health Aff (Millwood). 2015;34(8):1312–1319.CrossRefGoogle Scholar
  44. 44.
    Bodenheimer T. Strategies to Reduce Costs and Improve Care for High-Utilizing Medicaid Patients: Reflections on Pioneering Programs: Center for Health Care Strategies; 2013.Google Scholar
  45. 45.
    Hasselman D. Super-Utilizer Summit: Common Themes from Innovative Complex Care Management Programs: Center for Health Care Strategies;2013.Google Scholar

Copyright information

© Society of General Internal Medicine (This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply) 2018

Authors and Affiliations

  • Edwin S. Wong
    • 1
    • 2
    Email author
  • Jean Yoon
    • 3
    • 4
  • Rebecca I. Piegari
    • 5
  • Ann-Marie M. Rosland
    • 6
    • 7
  • Stephan D. Fihn
    • 5
    • 8
  • Evelyn T. Chang
    • 9
    • 10
  1. 1.Center of Innovation for Veteran-Centered and Value-Driven CareVA Puget Sound Health Care SystemSeattleUSA
  2. 2.Department of Health ServicesUniversity of WashingtonSeattleUSA
  3. 3.Health Economics Resource CenterVA Palo Alto Healthcare SystemLivermoreUSA
  4. 4.Department of General Internal MedicineUCSF School of MedicineSan FranciscoUSA
  5. 5.Office of Clinical Systems Development and EvaluationVeterans Health AdministrationSeattleUSA
  6. 6.Center for Health Equity Research and PromotionVA Pittsburgh Healthcare SystemPittsburghUSA
  7. 7.Department of Internal MedicineUniversity of PittsburghPittsburghUSA
  8. 8.Department of MedicineUniversity of WashingtonSeattleUSA
  9. 9.Center for the Study of Healthcare Innovation, Implementation and PolicyVA Greater Los Angeles Health Care SystemLos AngelesUSA
  10. 10.David Geffen School of MedicineUniversity of CaliforniaLos AngelesUSA

Personalised recommendations