INTRODUCTION

Improving the patient experience, an element of the IHI (Institute for Healthcare Improvement) Triple Aim health care quality improvement initiative, is a priority for clinical leaders and policymakers.1 The Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) scores are standardized measures of patient experience used to determine reimbursement in hospital value-based purchasing (VBP) arrangements.2,3 Since 2013, performance on HCAHPS domain scores have determined 0.3 % of participating hospitals’ diagnosis-related group (DRG) payments.4 Hospitals are expected to demonstrate improvement in measures or meet minimum quality thresholds to receive payment. In general, patient experience scores in safety-net hospitals are lower than those in non-safety-net hospitals,5–7 and the gap is widening.5 Safety-net hospitals serve patients of lower socioeconomic status, low health literacy, and limited English proficiency (LEP); these factors may negatively affect the provider–patient relationship and contribute to lower patient experience scores.8–10 As such, some concerns have been raised about the impact these policies have on safety-net hospitals and whether adjusting for socioeconomic status and social risk factors, including limited English proficiency and depression status, is appropriate for quality measurement and comparison.11

Transition-of-care activities present an opportunity to improve communication-related patient experience. These activities include educating patients about diagnoses, counseling them about medications, assessing discharge needs, and reinforcing follow-up. Poor communication in the peri-discharge period can lead to negative outcomes such as medication errors and readmissions.12–17 In recognition of the importance of transitions of care, a three-item version of the Care Transitions Measure (CTM-3) is now included in the HCAHPS survey.18,19

Few studies have examined the effect of transition interventions on the patient experience. Project RED (Re-Engineered Discharge),20 a randomized controlled trial of an in-hospital patient education and discharge planning intervention, and the IMPaCT (Individualized Management for Patient-Centered Targets) study, a randomized controlled trial of a community health worker (CHW) intervention,21 both demonstrated improvements in patient experience measures. The C-TraIn (Care Transitions Innovation) study,22 a cluster-randomized controlled trial studying the effects of intensive transitional nurse coaching, pharmacy care, and community payment for primary care intervention showed improved CTM-3 scores in low-income English-speaking participants.

No studies to date have evaluated the impact of transition interventions on the patient experience in diverse older non-English-speaking populations. The Support from Hospitalization to Home for Elders (SHHE) study, a randomized controlled trial of a nurse-led transition-of-care intervention with post-discharge follow-up phone calls, found no reductions in 30-day readmission rates, and noted possible increases in ED visits.23 However, we posited that this intervention might improve patient experience through improved communication and the use of patient-centered discharge care planning. Therefore, we assessed the effect of the SHHE intervention on the patient experience in older English, Chinese- and Spanish-speaking adults hospitalized at a safety-net hospital, hypothesizing that participants receiving the intervention would have higher CTM-3 and HCAHPS scores in discharge communication, medication counseling, and nursing communication domains.

METHODS

Study Design

This study was part of the SHHE, a randomized controlled trial of a hospital-based care transition intervention designed to reduce ED use and hospital readmissions for adults aged 55 and older hospitalized at a safety-net hospital.23 The study was approved by the UCSF institutional review board.

Setting and Participants

Patients admitted to the internal medicine, family medicine, cardiology, or neurology services at San Francisco General Hospital and Trauma Center (SFGH) who were aged 55 and older and who spoke English, Spanish, or Chinese (Mandarin or Cantonese) were eligible for enrollment. We excluded patients who had been transferred from an outside hospital, were admitted for a planned hospitalization, were likely to be discharged to hospice, nursing home, or other institutional settings, were unable to consent due to severe cognitive impairment, delirium, or severe mental illness, or were unable to participate in telephone follow-up.23

Enrollment, Randomization, and Follow-up

Study staff received a daily list of all patients admitted in the previous 24 h. After a preliminary review, study staff discussed eligibility with the patients’ attending physicians. Patients who were eligible were then approached by a language-concordant research assistant, who explained the study and obtained consent. Research assistants administered a baseline assessment prior to randomization. We used a parallel-group randomized design, stratified by language. Research assistants were blinded to participants’ randomization status during follow-up telephone interviews 30 days after hospital discharge.

Intervention

Usual Care

All study participants received usual care. The patient’s bedside registered nurse (RN) provided structured education, including information about follow-up appointments, and reviewed a discharge medication list, which was reconciled with pre-hospital medications. The RN gave the patient instructions on what symptoms should prompt return to the hospital. Social workers were available to assist with discharge needs. The inpatient team was responsible for transmitting the discharge summary to the patient's primary care provider within 3 days of discharge, and attempted to make post-hospitalization primary care visits within 2 weeks of discharge.

Intervention

In addition to receiving usual care, intervention group participants were visited by a study RN on the day of study enrollment and again within 24 h of discharge. The study RNs included native Spanish and Chinese speakers who were matched to study participants by language. Once the participant was randomized to the intervention group, the study RN notified the primary care provider by email to inform them that the patient had been admitted, along with contact information for the study RN and the primary medical team.

Disease-specific patient education, including symptom recognition, medication reconciliation, and strategies for navigating the health system, were provided by the study RN in the participant’s preferred language. Study RNs also organized post-discharge services for participants, confirming patients' understanding of services and reviewing time, location, and transportation plans for scheduled appointments, including instructions for arranging follow-up services, if necessary. Study RNs reconciled discharge medication regimens and explained medication instructions, emphasizing any changes, proper administration of medications, and potential side effects. They also reviewed with participants the appropriate steps to take if problems arose, and provided contact numbers and instructions for contacting primary care providers or emergency services. The study RNs used motivational interviewing24 techniques to foster positive self-care and health behaviors, and used Krames Patient Education25 language-concordant written educational materials to supplement verbal instructions. They reinforced their teaching using the “teach-back” method to ensure patient comprehension.26 In order to improve intervention participants’ ability to receive the intervention, study RNs worked with caregivers (both family and non-family) to include them in the education and training activities.

At discharge, participants were given a booklet, the "After Hospital Care Plan" (AHCP), which was written in their native language and was conceptually similar to that used in Project RED.20 The AHCP included the reason for hospitalization, principal diagnoses and important findings, a reconciled discharge medication list, and primary care and pharmacy contact information and upcoming appointments. Based on feedback from pilot testing, we simplified the AHCP by changing the presentation of medications.

After hospital discharge, nurse practitioners (NPs) from the study team contacted intervention participants via telephone once on post-discharge days 1–3 and again on days 6–10. These calls were made by language-concordant NPs or, if a language-concordant NP was not available, using a trained medical telephone interpreter. The NPs provided patient education on symptoms, assessed adherence to medications and treatment plans, helped patients resolve barriers to attending follow-up appointments, and discussed other issues identified in the participants' personalized discharge plans. They answered questions about and adjusted medications, worked with pharmacies to resolve prescription problems, and, if necessary, referred patients to their primary care provider, urgent health clinic, or emergency department. Study NPs contacted patients’ primary care providers if there were any changes in clinical status, new reported symptoms, or medication adjustments. Participants had access to a phone support line staffed by an NP who returned phone calls within 24 h.

Dependent Variables

HCAHPS Domains of Patient Experience

Developed by the Agency for Healthcare Research and Quality (AHRQ) and introduced in 2008, the HCAHPS asks patients 25 questions regarding their experience in the hospital. Patients are asked to respond to questions either using a four-point Likert scale (“always,” “usually,” “sometimes,” or “never”) or with “yes” or “no.”3 Of these 25 questions, 14 are summarized and reported in six summary domain scores, which include communication with nurses, communication with physicians, communication about medications, adequacy of planning for discharge, responsiveness of staff, and pain management.27 We measured the impact of the intervention on the discharge communication domain, the outcome most likely to have been affected by the intervention. We also assessed medication and nursing domains, recognizing that these encompass the entire hospitalization, including the discharge period. We calculated the HCAHPS domain scores by determining the “top-box” score that CMS uses to report HCAHPS scores. Top-box scores represent the proportion of participants who responded with “always” to all questions in a particular domain, with a higher proportion indicating better quality.3 For the HCAHPS questions regarding discharge communication, we determined the proportion of respondents responding “yes” to both questions (Text Box 1).

Text Box 1 HCAHPS Domains, CTM-3 Questions, and Discharge Medication Counseling Questions Asked at 30-Day Follow-Up Interview

Discharge Communication*

1. During this hospital stay, did doctors, nurses or other hospital staff talk with you about whether you would have the help you needed when you left the hospital? (Y/N)

2. During this hospital stay, did you get information in writing about what symptoms or health problems to look out for after you left the hospital? (Y/N)

Medication Communication†

1. Before giving you any new medicine, how often did hospital staff tell you what the medicine was for?

2. Before giving you any new medicine, how often did hospital staff describe possible side effects in a way you could understand?

Nursing Communication†

1. During this hospital stay, how often did nurses treat you with courtesy and respect?

2. During this hospital stay, how often did nurses listen carefully to you?

3. During this hospital stay, how often did nurses explain things in a way you could understand?

Care Transitions Measure-3 Questions‡

1. During this hospital stay, staff took my preferences and those of my family or caregiver into account in deciding what my health care needs would be when I left.

2. When I left the hospital, I had a good understanding of the things I was responsible for in managing my health.

3. When I left the hospital, I clearly understood the purpose for taking each of my medications.

Discharge Medication Counseling Question*

1. During this hospital stay, did someone on the hospital staff explain the purpose of the medicines you were to take at home in a way you could understand?

* Yes or No

† Four-point Likert scale: 1, Never; 2, Sometimes; 3, Usually; 4, Always

‡ Four-point Likert scale: 1, Strongly disagree; 2, Disagree; 3, Agree; 4, Strongly agree

Three-Item Care Transitions Measure (CTM-3)

The CTM-3,18 which consists of three questions, assesses the quality of the transitional care experience (Text Box 1).28 The CTM-3 was incorporated into the HCAHPS survey questionnaire beginning in January 2013.19 We summed and linearly transformed mean scores for each CTM-3 question to obtain a score ranging from 1 to 100. We also calculated a “top-box” score for each individual CTM-3 question.

Baseline Measures

At the baseline interview, participants reported date of birth, sex, race/ethnicity, total household income (≤$20,000 per year vs >$20,000 per year),29 and last completed grade in school (categorized as 0–6 years, 7–11 years, 12 years, and >12 years). We asked what language the participant spoke at home, and how well the participant spoke English (we defined low English proficiency for participants indicating “not at all/not well”).30 We measured substance use (using the World Health Organization [WHO] Alcohol, Smoking and Substance Involvement Screening Test [ASSIST]),31,32 depression symptom severity using the Patient Health Questionnaire-9 (PHQ-9) (we defined moderate-severe depressive symptoms by a PHQ-9 score ≥10),33,34 and self-reported health literacy (a score ≥ 9 was defined as an adequate health literacy score).35,36 We asked participants whether they identified a usual place of care and regular health care provider, and whether they had experienced emergency department visits or hospitalizations in the 6 months prior to study enrollment. Research assistants administered the Telephone Interview for Cognitive Status (TICS) to evaluate cognitive impairment, defined as a TICS score less than 20.37 We calculated the Charlson comorbidity index using administrative ICD-9 codes from the index hospitalization (scores 0, 1–2, 3–4, 5+, higher scores indicate higher mortality risk).38

Statistical Analysis

While our study was originally powered for readmission rate reductions, we estimated that our sample size was large enough to detect a 5 % effect size difference between HCAHPS and CTM scores based on a prior published study.18 We compared patient characteristics between treatment and control groups using chi-square tests. As the CTM-3 scores were not normally distributed, we used nonparametric tests of significance to compare scores between groups. Because we used stratified randomization by language a priori, we conducted a secondary analysis examining the outcome measures by language. We conducted sensitivity analyses to account for communication experience not affected by the intervention by comparing the proportions reporting “usually” or “always” versus other answers between groups. We accounted for missing responses by conducting sensitivity analyses coding missing data as “no” or “never” responses. Because we hypothesized that patients with depression might respond to the intervention differently from patients without depression, we conducted a post hoc analysis stratified by depression. Data analysis was conducted using the Stata version 13 software program (StataCorp, College Station, TX, USA).

Role of the Funding Source

The Gordon and Betty Moore Foundation funded this study. The funding source had no access to the data or role in design, data collection, management, analysis, interpretation, or manuscript preparation, review, or approval of the manuscript.

RESULTS

Study Patients

During the study period, 6384 inpatients met basic inclusion criteria, from which 1781 were eligible for screening (Fig. 1). After screening, 912 inpatients were ineligible, and an additional 169 declined to participate. We randomized the remaining 700 participants. Of the 685 who survived to 30 days, 90 % (n = 616) completed follow-up research interviews.

Fig. 1
figure 1

Study eligibility, enrollment, and telephone interview completion. §Excluded prior to screening: non-study language (n = 466); planned admission (n = 101); screened out by team (n = 917); previously enrolled in pilot or RCT (n = 551); previously refused (n = 82); less than 24-h stay (n = 1350); discharged before assessment (n = 94); transfer to other service (n = 233); transfer to/from institution (n = 809). †Excluded at screening: no phone/homeless (n = 583); lives elsewhere (n = 46); left hospital before enrollment completed (n = 111); failed teach-back (n = 96); declined (n = 169); other (n = 76)

Full size image

The study sample was an ethnically diverse low-income population with limited educational background and health literacy. The mean age was 66.2 years (SD 9.0). Blacks made up 24.4 % of the population, 19.0 % were white, 32.2 % were Asian, and 19.6 % were Latino. Over half (62.1 %) of the interviews were conducted in English; 23.3 % were in Chinese and 14.6 % in Spanish. Over one-third of participants noted limited English proficiency (37.1 %), and more than half (51.6 %) had limited health literacy. Over half (54.5 %) of the study population met criteria for cognitive impairment at baseline, and one-third (33.8 %) met criteria for moderate to severe depression. More than one-third (35.1 %) of participants lived alone. Over one-third (35.9 %)of the study population had a Charlson score of 3 or above. Overall, 77.2 % of the participants could name a regular health care provider. There were no significant differences between the intervention and control groups (Table 1).

Table 1 Characteristics of SHHE Study Participants*
Full size table

Process Measures

We measured several aspects of intervention implementation. The in-hospital study RN spent a total of 156.6 min (SD 102.0) on one-on-one patient care and care coordination issues. The post-discharge study NP completed at least one follow-up phone call for 94.5 % of patients, and completed both follow-up phone calls for 82.8 % of the intervention participants.

Outcomes Measures

At 30-day follow-up, there were no statistically significant differences between the intervention and usual care groups in the CTM-3 score (80.5 % vs 78.5 %; p = 0.18) or the HCAHPS discharge communication domain score (74.8 % vs 68.7 %; p = 0.11). There were also no differences in HCAHPS scores in medicine communication (44.5 % vs 53.1 %; p = 0.13), or nurse communication (67.9 % vs 64.9 %; p = 0.43) (Table 2). No differences between groups were found when the results were stratified by interview language (Table 3). There was also no significant difference between groups when participants reporting “usually” or above were included as a positive response, or when missing data were recoded as a negative response (data not shown). When we stratified by depression status, we found that among non-depressed participants, those in the intervention had higher patient experience scores than those in usual care, although these did not always reach statistical significance. Patients with depression showed no difference (data not shown).

Table 2 Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) Domain, Care Transitions Measure (CTM-3), and Discharge Medication Counseling Scores assessed at 30 days after discharge
Full size table
Table 3 Patient Experience Outcomes, Stratified by Language of Interview
Full size table

DISCUSSION

This study found that a nurse-led transition-of-care intervention in older linguistically diverse adults admitted to a safety-net hospital did not impact communication-related patient experience. We hypothesized that the intervention’s specially trained study RNs working one-on-one with the patient to devise a patient-centered discharge plan would improve communication-related patient experience measures, but it did not. We suggest that high rates of cognitive impairment and depression may have interfered with participants’ ability to benefit from the intervention. While other studies have shown modest effects on patient experience, our results reflect the challenges in improving patient experience in older adults with limited English proficiency, cognitive impairment, and depression.

While it is clear that the patient experience is an important component in providing quality care,27,39,40 it is not clear what interventions are effective in improving the patient experience. Our intervention was modeled after Project RED,20 which showed improvement in patient experience measures that were similar to the HCAHPS and CTM-3 measures. In Project RED, the study population was younger, was English-speaking, and had less cognitive impairment, which could have made the participants more amenable to the intervention. The IMPaCT trial with a community health worker intervention reported higher scores on one HCAHPS question (“Did you receive high-quality verbal discharge communication?”).21 The differences in interventions may explain why the IMPaCT study demonstrated improved patient experience and ours did not. The use of lay persons who shared socioeconomic backgrounds with them and who continued working with patients post-hospitalization may have been more effective in communicating and reinforcing patient-centered discharge information. Similarly, the improved CTM-3 scores in the C-TraIn study,13 which involved a high-intensity nurse coaching and post-discharge linkage intervention in a younger, low-income population, may have been due to the continuity of care provided by the study nurse and post-discharge home visits to reinforce the care plan.

There are several reasons why our intervention may not have worked. The patient experience is likely influenced by the quality of interactions between patients and their providers, individual patient factors and health expectations, and the hospital environment. While the HCAHPS questionnaire focuses on the in-hospital experience, participant responses may be influenced by post-hospitalization interactions with health care providers. Our intervention’s follow-up phone calls and access to “warm lines” may not have been enough to reinforce participants' understanding of care plans.

Our study population had high rates of cognitive impairment and depression and low levels of social support, and this may have affected the ability of the intervention to improve the patient experience. Depression is associated with difficulties related to health care 41 and patient-physician communication.42,43 In addition to coloring patients’ view of the care that they received, depression may interfere with patients’ ability to respond to interventions that focus on self-efficacy and education. While our findings on depression are exploratory, they provide insight into why we found no improvement in patient experience and may guide future research efforts.

Our intervention was a standalone enhancement provided alongside usual care, and did not address aspects of hospital culture such as timeliness of care delivery and nurse staffing ratios.27,44–47 We did not alter the usual care provider–patient interactions, which may have been necessary to improve HCAHPS nursing communication and medication communication domain scores that reflect the participants’ experience throughout the entire hospitalization.

This study has several limitations. Primary medical teams were not blinded to the presence of the study RN intervention; it is possible that the bedside nurses adopted some of the teaching techniques used by the study RN, improving the care of all patients and impeding our ability to find an effect. The measures of patient experience we included, though widely used, have not been validated in low-income, limited-literacy populations, and thus may not have been well-understood by participants.

CONCLUSIONS

A nurse-led education-based transition-of-care intervention did not improve the patient experience at discharge. Our finding on the lack of improvement in patient experience in this population underscores the importance of designing pay-for-performance programs such that they do not widen existing health disparities. Designs of future care transition interventions in older safety-net populations with cognitive impairment and depression may require additional supportive post-discharge intervention in order to improve the patient experience.