Journal of General Internal Medicine

, Volume 32, Issue 6, pp 640–647 | Cite as

A Multilevel Analysis of Patient Engagement and Patient-Reported Outcomes in Primary Care Practices of Accountable Care Organizations

  • Stephen M. Shortell
  • Bing Ying Poon
  • Patricia P. Ramsay
  • Hector P. Rodriguez
  • Susan L. Ivey
  • Thomas Huber
  • Jeremy Rich
  • Tom Summerfelt
Original Research



The growing movement toward more accountable care delivery and the increasing number of people with chronic illnesses underscores the need for primary care practices to engage patients in their own care.


For adult primary care practices seeing patients with diabetes and/or cardiovascular disease, we examined the relationship between selected practice characteristics, patient engagement, and patient-reported outcomes of care.


Cross-sectional multilevel observational study of 16 randomly selected practices in two large accountable care organizations (ACOs).


Patients with diabetes and/or cardiovascular disease (CVD) who met study eligibility criteria (n = 4368) and received care in 2014 were randomly selected to complete a patient activation and PRO survey (51% response rate; n = 2176). Primary care team members of the 16 practices completed surveys that assessed practice culture, relational coordination, and teamwork (86% response rate; n = 411).

Main Measures

Patient-reported outcomes included depression (PHQ-4), physical functioning (PROMIS SF12a), and social functioning (PROMIS SF8a), the Patient Assessment of Chronic Illness Care instrument (PACIC-11), and the Patient Activation Measure instrument (PAM-13). Patient-level covariates included patient age, gender, education, insurance coverage, limited English language proficiency, blood pressure, HbA1c, LDL-cholesterol, and disease comorbidity burden. For each of the 16 practices, patient-centered culture and the degree of relational coordination among team members were measured using a clinician and staff survey. The implementation of shared decision-making activities in each practice was assessed using an operational leader survey.

Key Results

Having a patient-centered culture was positively associated with fewer depression symptoms (odds ratio [OR] = 1.51; confidence interval [CI] 1.04, 2.19) and better physical function scores (OR = 1.85; CI 1.25, 2.73). Patient activation was positively associated with fewer depression symptoms (OR = 2.26; CI 1.79, 2.86), better physical health (OR = 2.56; CI 2.00, 3.27), and better social health functioning (OR = 4.12; CI 3.21, 5.29). Patient activation (PAM-13) mediated the positive association between patients’ experience of chronic illness care and each of the three patient-reported outcome measures—fewer depression symptoms, better physical health, and better social health. Relational coordination and shared decision-making activities reported by practices were not significantly associated with higher patient-reported outcome scores.


Diabetic and CVD patients who received care from ACO-affiliated practices with more developed patient-centered cultures reported lower PHQ-4 depression symptom scores and better physical functioning. Diabetic and CVD patients who were more highly activated to participate in their care reported lower PHQ-4 scores and better physical and social outcomes of care.


patient engagement patient-reported outcomes accountable care organizations 

Supplementary material

11606_2016_3980_MOESM1_ESM.docx (16 kb)
ESM 1(DOCX 16 kb)


  1. 1.
    American Diabetes Association. Data from the 2011 national diabetes fact sheet 2013 [accessed 2016, December 22]. Available from:
  2. 2.
    Heidenreich PA, Trogdon JG, Khavjou OA, Butler J, Dracup K, Ezekowitz MD, et al. Forecasting the future of cardiovascular disease in the United States: a policy statement from the American Heart Association. Circulation. 2011;123(8):933–44.CrossRefPubMedGoogle Scholar
  3. 3.
    Centers for Disease Control and Prevention. National Diabetes Fact Sheet, 2011. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention; 2011.Google Scholar
  4. 4.
    Coleman K, Austin BT, Brach C, Wagner EH. Evidence on the chronic care model in the new millennium. Health Aff (Millwood). 2009;28(1):75–85. doi:10.1377/hlthaff.28.1.75.CrossRefGoogle Scholar
  5. 5.
    Lin GA, Halley M, Rendle KA, Tietbohl C, May SG, Trujillo L, et al. An effort to spread decision aids in five California primary care practices yielded low distribution, highlighting hurdles. Health Aff (Millwood). 2013;32(2):311–20.CrossRefGoogle Scholar
  6. 6.
    Hibbard JH, Mahoney ER, Stock R, Tusler M. Do increases in patient activation result in improved self-management behaviors? Health Serv Res. 2007;42(4):1443–63.CrossRefPubMedPubMedCentralGoogle Scholar
  7. 7.
    Hibbard JH, Stockard J, Mahoney ER, Tusler M. Development of the patient activation measure (PAM): Conceptualizing and measuring activation in patients and consumers. Health Serv Res. 2004;39(4):1005–26.CrossRefPubMedPubMedCentralGoogle Scholar
  8. 8.
    Mosen DM, Schmittdiel J, Hibbard J, Sobel D, Remmers C, Bellows J. Is patient activation associated with outcomes of care for adults with chronic conditions? J Ambulatory Care Manage. 2007;30(1):21–9.CrossRefPubMedGoogle Scholar
  9. 9.
    Delbanco T, Walker J, Bell SK, Darer JD, Elmore JG, Farag N, et al. Inviting patients to read their doctors’ notes: a quasi-experimental study and a look ahead. Ann Intern Med. 2012;157(7):461–70. doi:10.7326/0003-4819-157-7-201210020-00002.CrossRefPubMedPubMedCentralGoogle Scholar
  10. 10.
    Juul L, Maindal HT, Zoffmann V, Frydenberg M, Sandbaek A. A cluster randomised pragmatic trial applying self-determination theory to type 2 diabetes care in general practice. BMC Fam Pract. 2011;12:130. doi:10.1186/1471-2296-12-130.CrossRefPubMedPubMedCentralGoogle Scholar
  11. 11.
    Zoffmann V, Kirkevold M. Realizing empowerment in difficult diabetes care: a guided self-determination intervention. Qual Health Res. 2012;22(1):103–18. doi:10.1177/1049732311420735.CrossRefPubMedGoogle Scholar
  12. 12.
    Williams GC, Lynch M, Glasgow RE. Computer-assisted intervention improves patient-centered diabetes care by increasing autonomy support. Health Psychol. 2007;26(6):728–34. doi:10.1037/0278-6133.26.6.728.CrossRefPubMedGoogle Scholar
  13. 13.
    Gagliardi AR, Legare F, Brouwers MC, Webster F, Badley E, Straus S. Patient-mediated knowledge translation (pkt) interventions for clinical encounters: a systematic review. Implement Sci. 2016;11:26. doi:10.1186/s13012-016-0389-3.CrossRefPubMedPubMedCentralGoogle Scholar
  14. 14.
    Health policy brief: Patient engagement. Health Aff (Millwood). February 14, 2013.Google Scholar
  15. 15.
    LeBlanc ES, Rosales AG, Kachroo S, Mukherjee J, Funk KL, Nichols GA. Do patient or provider characteristics impact management of diabetes? Am J Manag Care. 2015;21(9):597–606.PubMedGoogle Scholar
  16. 16.
    Rodriguez HP, Rogers WH, Marshall RE, Safran DG. Multidisciplinary primary care teams: effects on the quality of clinician-patient interactions and organizational features of care. Med Care. 2007;45(1):19–27.CrossRefPubMedGoogle Scholar
  17. 17.
    Cella D, Riley W, Stone A, Rothrock N, Reeve B, Yount S, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. J Clin Epidemiol. 2010;63(11):1179–94. doi:10.1016/j.jclinepi.2010.04.011.CrossRefPubMedPubMedCentralGoogle Scholar
  18. 18.
    Gittell JH, Fairfield KM, Bierbaum B, Head W, Jackson R, Kelly M, et al. Impact of relational coordination on quality of care, postoperative pain and functioning, and length of stay: a nine-hospital study of surgical patients. Med Care. 2000;38(8):807–19.CrossRefPubMedGoogle Scholar
  19. 19.
    O’Toole TP, Cabral R, Blumen JM, Blake DA. Building high functioning clinical teams through quality improvement initiatives. Qual Prim Care. 2011;19(1):13–22.PubMedGoogle Scholar
  20. 20.
    Deutsch A, Gage B, Smith L, Kelleher C. Patient-reported outcomes in performance measurement. Washington, DC: National Quality Forum (NQF); 2012.Google Scholar
  21. 21.
    Rose M, Bjorner JB, Gandek B, Bruce B, Fries JF, Ware JE Jr. The promis physical function item bank was calibrated to a standardized metric and shown to improve measurement efficiency. J Clin Epidemiol. 2014;67(5):516–26. doi:10.1016/j.jclinepi.2013.10.024.CrossRefPubMedPubMedCentralGoogle Scholar
  22. 22.
    Kaplan RM, Saccuzzo DP. Psychological testing: principles, applications, and issues. 7th ed. Brooks/Cole: Monterey, CA; 1982.Google Scholar
  23. 23.
    Wiley JA, Rittenhouse D, Shortell SM, Casalino L, Ramsay PP, Bibi S, et al. Managing chronic illness: Physician practices increased the use of care management and medical home processes. Health Aff (Millwood). 2015; Forthcoming.Google Scholar
  24. 24.
    Gittell JH, Beswick J, Goldmann D, Wallack SS. Teamwork methods for accountable care: relational coordination and TeamSTEPPS®. Health Care Manag Rev. 2015;40(2):116–25. doi:10.1097/HMR.0000000000000021.CrossRefGoogle Scholar
  25. 25.
    Gittell JH, Weinberg DB, Bennett AL, Miller JA. Is the doctor in? A relational approach to job design and the coordination of work. Hum Resour Manag. 2008;47(4):729–55. doi:10.1002/hrm.20242.CrossRefGoogle Scholar
  26. 26.
    Glasgow RE, Wagner EH, Schaefer J, Mahoney LD, Reid RJ, Greene SM. Development and validation of the patient assessment of chronic illness care (PACIC). Med Care. 2005;43(5):436–44.CrossRefPubMedGoogle Scholar
  27. 27.
    Glasgow RE, Whitesides H, Nelson CC, King DK. Use of the Patient Assessment of Chronic Illness Care (PACIC) with diabetic patients: relationship to patient characteristics, receipt of care, and self-management. Diabetes Care. 2005;28(11):2655–61.CrossRefPubMedGoogle Scholar
  28. 28.
    Gugiu PC, Coryn C, Clark R, Kuehn A. Development and evaluation of the short version of the Patient Assessment of Chronic Illness Care instrument. Chronic Illness. 2009;5(4):268–76. doi:10.1177/1742395309348072.CrossRefPubMedGoogle Scholar
  29. 29.
    Hibbard JH, Mahoney ER, Stockard J, Tusler M. Development and testing of a short form of the patient activation measure. Health Serv Res. 2005;40(6p1):1918–30.CrossRefPubMedPubMedCentralGoogle Scholar
  30. 30.
    Greene J, Hibbard JH, Sacks R, Overton V, Parrotta CD. When patient activation levels change, health outcomes and costs change, too. Health Aff (Millwood). 2015;34(3):431–7. doi:10.1377/hlthaff.2014.0452.CrossRefGoogle Scholar
  31. 31.
    Bryk AS, Raudenbush SW. Hierarchical linear models: applications and data analysis methods. Newbury Park: Sage Publications; 1992.Google Scholar
  32. 32.
    Krull JL, MacKinnon DP. Multilevel modeling of individual and group level mediated effects. Multivar Behav Res. 2001;36(2):249–77. doi:10.1207/S15327906MBR3602_06.CrossRefGoogle Scholar
  33. 33.
    Hare DL, Toukhsati SR, Johansson P, Jaarsma T. Depression and cardiovascular disease: a clinical review. Eur Heart J. 2014;35(21):1365–72. doi:10.1093/eurheartj/eht462.CrossRefPubMedGoogle Scholar
  34. 34.
    Fiedorowicz JG. Depression and cardiovascular disease: an update on how course of illness may influence risk. Curr Psychiatry Rep. 2014;16(10):492. doi:10.1007/s11920-014-0492-6.CrossRefPubMedPubMedCentralGoogle Scholar
  35. 35.
    Semenkovich K, Brown ME, Svrakic DM, Lustman PJ. Depression in type 2 diabetes mellitus: prevalence, impact, and treatment. Drugs. 2015;75(6):577–87. doi:10.1007/s40265-015-0347-4.CrossRefPubMedGoogle Scholar
  36. 36.
    Vancampfort D, Mitchell AJ, De Hert M, Sienaert P, Probst M, Buys R, et al. Type 2 diabetes in patients with major depressive disorder: a meta-analysis of prevalence estimates and predictors. Depress Anxiety. 2015;32(10):763–73. doi:10.1002/da.22387.CrossRefPubMedGoogle Scholar
  37. 37.
    Centers for Disease Control and Prevention. The state of aging and health in America 2013. Atlanta, GA: Centers for Disease Control and Prevention, US Dept of Health and Human Services; 2013.Google Scholar
  38. 38.
    Aung E, Donald M, Williams GM, Coll JR, Doi SA. Influence of patient-assessed quality of chronic illness care and patient activation on health-related quality of life. Int J Qual Health Care. 2016. doi:10.1093/intqhc/mzw023.PubMedGoogle Scholar
  39. 39.
    Weinberg DB, Lusenhop RW, Gittell JH, Kautz CM. Coordination between formal providers and informal caregivers. Health Care Manag Rev. 2007;32(2):140–9. doi:10.1097/01.HMR.0000267790.24933.4c.CrossRefGoogle Scholar
  40. 40.
    Cramm JM, Nieboer AP. Relational coordination promotes quality of chronic care delivery in Dutch disease-management programs. Health Care Manag Rev. 2012;37(4):301–9. doi:10.1097/HMR.0b013e3182355ea4.CrossRefGoogle Scholar
  41. 41.
    Leykum LK, Lanham HJ, Pugh JA, Parchman M, Anderson RA, Crabtree BF, et al. Manifestations and implications of uncertainty for improving healthcare systems: an analysis of observational and interventional studies grounded in complexity science. Implement Sci. 2014;9:165. doi:10.1186/s13012-014-0165-1.CrossRefPubMedPubMedCentralGoogle Scholar
  42. 42.
    Rathert C, Wyrwich MD, Boren SA. Patient-centered care and outcomes: a systematic review of the literature. Med Care Res Rev. 2013;70(4):351–79. doi:10.1177/1077558712465774.CrossRefPubMedGoogle Scholar
  43. 43.
    Hahn J, Blom KB. The Medicare Access and CHIP Reauthorization Act of 2015 (MACRA; P.L. 114-10). Congressional Research Service, 2015 Nov 10, 2015.Google Scholar
  44. 44.
    Weldring T, Smith SM. Patient-reported outcomes (pros) and patient-reported outcome measures (proms). Health Serv Insights. 2013;6:61–8. doi:10.4137/HSI.S11093.PubMedPubMedCentralGoogle Scholar
  45. 45.
    Lavallee DC, Chenok KE, Love RM, Petersen C, Holve E, Segal CD, et al. Incorporating patient-reported outcomes into health care to engage patients and enhance care. Health Aff (Millwood). 2016;35(4):575–82. doi:10.1377/hlthaff.2015.1362.CrossRefGoogle Scholar
  46. 46.
    Rothman ML, Beltran P, Cappelleri JC, Lipscomb J, Teschendorf B. Patient-reported outcomes: conceptual issues. Value Health. 2007;10(Suppl 2):S66–75. doi:10.1111/j.1524-4733.2007.00269.x.CrossRefPubMedGoogle Scholar
  47. 47.
    Revicki D, Hays RD, Cella D, Sloan J. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. J Clin Epidemiol. 2008;61(2):102–9. doi:10.1016/j.jclinepi.2007.03.012.CrossRefPubMedGoogle Scholar
  48. 48.
    Greenhalgh J. The applications of pros in clinical practice: what are they, do they work, and why? Qual Life Res. 2009;18(1):115–23. doi:10.1007/s11136-008-9430-6.CrossRefPubMedGoogle Scholar
  49. 49.
    Nelson EC, Hvitfeldt HF, Reid R, Grossman D, Lindblad S, Mastanduno MP, et al. Using patient-reported information to improve health outcomes and health care value: case studies from Dartmouth, Karolinska and Group Health. The Dartmouth Institute for Health Policy and Clinical Practice, 2012.Google Scholar

Copyright information

© Society of General Internal Medicine 2017

Authors and Affiliations

  • Stephen M. Shortell
    • 1
  • Bing Ying Poon
    • 1
  • Patricia P. Ramsay
    • 1
  • Hector P. Rodriguez
    • 1
  • Susan L. Ivey
    • 1
  • Thomas Huber
    • 1
  • Jeremy Rich
    • 2
  • Tom Summerfelt
    • 3
  1. 1.School of Public HealthUniversity of California BerkeleyBerkeleyUSA
  2. 2.HealthCare Partners Institute for Applied Research and EducationLos AngelesUSA
  3. 3.Advocate HealthChicagoUSA

Personalised recommendations