In 2010, 5% of the US population accounted for 50% of medical costs.1 Most of these “high-cost” patients have multiple chronic conditions. Indeed, nearly one in seven Medicare beneficiaries has six or more chronic conditions.2 This multimorbidity creates problems related to polypharmacy,3 adherence to appointments, scheduled tests, and medications,4,5,-6 and places financial and other resource-related burdens on family members and informal care givers.7 Many complex patients also have social, mental health, and economic stressors that can contribute to increased care utilization.8 The effective management of these complex patients represents a major challenge to our health care system.

Integrated care systems and accountable care organizations increasingly rely on intensive care management for their complex high-cost patients in an effort to decrease barriers to care and thereby reduce costs.9 With its intensive focus on patients with multiple chronic conditions, care management is distinct from less resource-intensive disease management programs that generally oversee larger, less complex patient populations defined by a single disease (e.g., diabetes or congestive heart failure disease management programs).10,11,-12 Complex patient care management teams are typically led by a nurse and/or a licensed clinical social worker and include a comprehensive initial patient assessment to create a care plan, frequent monitoring of the patient’s condition, support for transitions between care settings (e.g., hospital to home), coordination between different care providers, and education and support for patients, family members, and caregivers.13,14

When directed towards the appropriate patients, care management is considered a powerful tool for preventing clinical deterioration and reducing future hospitalizations and costs.15 However, published evidence demonstrating clinical and cost benefits of care management are sparse, and many complex patients identified as high risk using existing referral mechanisms do not appear to derive measurable benefit.16,17 The inability to distinguish which complex high-cost patients are most amenable to intensive care management using current practices may be one reason why overall clinical benefits have been difficult to observe. Given the scarcity of available resources, research is therefore needed to more effectively identify which complex patients are most likely to benefit from care management.18

To gain new insight into optimal patient selection for intensive care management programs, we studied the clinicians most directly involved in providing their care. We hypothesized that clinically experienced nurse and social worker care managers (CMs) are ideally positioned to identify the factors (or combination of factors) that identify which complex patients are best suited for intensive care management. We conducted a mixed-methods study involving 35 care managers at 17 medical centers across 11 counties within the Kaiser Permanente Northern California region. Relying on the rich clinical knowledge of these providers, we sought to identify characteristics of high-risk patients who CMs believe would most benefit from care management.

METHODS

Setting and Study Participants

Kaiser Permanente Northern California (KPNC) is a non-profit integrated care system caring for over 4.1 million members in 17 geographically diverse medical centers throughout Northern California. KPNC members are similar to the local geographic population and include patients insured by Medicare and Medicaid.19

Within each of the 17 medical centers, intensive care management programs consisting of nurses and social workers are deployed to support the management of the highest-cost and most complex patients, the majority of whom have multiple medical, social, and behavioral barriers to care. In the current system, as with most other care systems, patients are identified for enrollment into intensive care management via referral by an outpatient physician or at the time of hospital discharge without standardized referral criteria. Care management interventions at KPNC are typical of such programs nationwide and include an initial needs assessment, self-care education and skills building, and linking patients to appropriate internal and external resources. These interventions are delivered during clinic visits, telephone appointments, family conferences and home visits, if indicated, and typically run over a 2–6-month period (although some patients may remain under care management for up to 1 year).

We contacted 51 eligible nurses and licensed clinical social workers who were actively involved in the intensive care management programs at each of the 17 Northern California medical centers. Of these 51 CMs, 35 provided informed consent (69% response rate) and are the subject of this analysis. The institutional review board of the Kaiser Foundation Research Institute approved the study protocol.

Conceptual Model

Among complex patients who could be potential candidates for care management, we conceptualized a spectrum of care management need that would range from patients with sufficient existing skills and resources who do not require care management (“not needing”) to patients requiring more specialized or intensive interventions beyond those offered by traditional care management (“needing more”). These more specialized interventions could be by psychiatric nurses or substance abuse counselors, roles that are generally out of the usual scope of care managers for patients whose psychiatric or behavioral issues are more severe. Between these two extremes of the spectrum would be the patients with needs that can be effectively addressed by care management (good candidates).

Data Collection

With this framework in mind, we asked CMs to review a list of up to 50 of their own patients seen within the last 12 months. A patient list for each CM was randomly generated by the research team based on the CM’s prior year schedule of patient contacts. Patients reviewed in the study were clinically and demographically similar to the overall population of patients (n = 2230) referred to care managers during the same period. After excluding patients that were not theirs, CMs placed each of the remaining patients into one of three mutually exclusive categories: (1) not needing intensive care management (e.g., sufficient resources and support already available, not needing), (2) able to be helped by care management (good candidates), and (3) intractable to the existing care management program (needing more). For this step of data collection, our trained research interviewers presented the spectrum of care management conceptual model, which our participating care managers quickly and easily understood.

Structured Qualitative Analysis

After categorizing the patients on their lists, CMs were asked to discuss several patients from each category who exemplified the characteristics typical of the patients in that category. Individual interviews included at least two research staff members, with one designated as the active listener and another as the note taker. This part of the data collection included an interview guide with appropriate prompts to encourage CMs to fully explore their reasons for the categories chosen with each patient example. After each interview, field notes were reviewed and discussed. Interviews were professionally transcribed then read independently by three investigators with the goal of identifying common themes for why specific patients were placed into each of the three CM-defined referral categories (i.e., not needing, good candidate, and needs more). This part of the interview was the basis for structured qualitative analysis.

Data were analyzed using content analysis to identify major concepts. The constant comparative method of qualitative data analysis was used to develop new themes identified through iterative transcript review and discussion among the investigators.20 Coding of these data was accomplished in a series of iterative steps. An initial code list was used to organize transcripts of the first 4 interviews and was then refined during review and analysis of transcripts from subsequent site visits. During its development, the code structure was reviewed three times by the full research team for logic and breadth. After establishing consensus on code definitions, each interview was coded by two researchers and any coding discrepancies (fewer than 15%) were resolved by consensus in a negotiated, group process.

We used the following techniques to ensure that data analysis was systematic and verifiable, as commonly recommended by experts in qualitative research:21,22 consistent use of the interview guide, discussion and debriefing by team members after each interview, audio-recording and professional preparation of the transcripts, standardized coding and analysis of the data, and the creation of an analysis audit trail to document analytic decisions. We stopped generating new themes once we reached thematic saturation. Using this approach, we developed 12 specific themes (with modifiers), organized within 4 broad domains. These themes served as the basis for final text review and organization of the transcript data. Coded data were entered into a database and analyzed to examine grouping patterns within each CM-defined referral category.

Statistical Methods for Quantitative Analyses

We conducted two types of quantitative analysis. For the overall cohort of complex patients reviewed by their CMs (n = 1178 patients), we compared patient, care manager, and practice characteristics between the three CM-defined referral categories using ANOVA or chi-square tests as indicated. Patient data were derived from electronic clinical care records for the 12 months preceding the CM interview date. Using these data, we developed binary logistic regression models to estimate value of available electronic health record (EHR) data to predict whether a complex patient would be defined as a “good candidate” based on our CM gold standard. We used a hybrid modeling approach recommended by many statisticians, in which we (1) created a sequence of models from null to full using the stepwise procedure to prioritize model variables, and then (2) selected the optimal model from this set based on differences in the Akaike information criteria (AIC) for predictive models.23 We used PROC GLIMMIX (SAS 9.3, North Carolina) to account for patient clustering within CM and used the Hosmer-Lemeshow goodness-of-fit and Pearson goodness-of-fit deviance statistics to arrive at our final optimal model.

For the subset of individual patients specifically discussed by each CM in the qualitative interviews (n = 346), we also examined the distribution of themes developed in our qualitative analysis. We first identified the top three most prevalent themes for each category. In an exploratory analysis, we then sought to identify the subset of CM-derived themes that (1) tended to be specific to a single referral category (“category specific”, e.g., theme seem primarily [> 90%] in a single category) or (2) were commonly found in more than one category (“category nonspecific”, e.g., theme seen with > 25% prevalence in more than one category). Our goal was to investigate whether we could apply the insights derived from the study CMs to inform what data elements would be most useful for future complex patient care management referral guidelines.

RESULTS

Participant Characteristics

We interviewed 35 care managers from 13 non-overlapping practice locations within KPNC. All except one of the CMs were female (97%) and 60% had been working as CMs for greater than 10 years. Twenty-five (71%) of participating CMs were registered nurses and the other 10 were licensed clinical social workers (LCSWs).

Patient Categorization

CMs assigned 1178 patients into the 3 referral categories. Sixty-two percent of reviewed patients were considered to be good candidate for care management, 18% were categorized as not needing advanced care management, and 19% categorized as needs more than what care management could provide. Small but statistically significant differences by category included patient age; diagnosis of depression; number of prescribed medications; family as first contact; and outpatient, hospital, and emergency department utilization (Table 1). Patients in the three referral categories did not differ by gender, race, income, primary language spoken, or diagnosis of dementia.

Table 1 Patient Demographics and Prior Year Utilization by Care Manager Defined Category of Referral Appropriateness
Full size table

Using Predictive Modeling to Distinguish Good Candidates

Our optimal logistic regression model included categorized variables derived from the EHR for age, gender, medication count, visits to emergency department without in-patient hospitalization, and having a family member listed as first contact in the medical record. Variables that remained independently associated with good candidate included age > 75 years (adjusted odds ratio 1.9; 95% CI 1.4–2.5 with age ≤ 75 as referent) and number of admissions to emergency department with discharge to home rather than hospital in prior year (aOR 1.5; 95%CI 1.2–2.0 for 1–4 admissions, referent: 0 admissions). This final full model had a c statistic of 0.75 (considered moderately good prediction model) in predicting good candidate vs. the other two categories.

Themes Developed from Care Managers

The 35 interview transcripts included discussion of 346 unique patients from the initial patient lists, with 76 patients exemplifying the not needing category, 176 patients the good candidate category, and 94 patients the needing more category. Qualitative analysis of reasons given by CMs for including individual patients into each of the three categories resulted in 12 themes (with modifiers) grouped into the following four domains: patient medical factors, patient non-medical factors, clinical trajectory, and system factors. The themes and their modifiers and definitions, with illustrative quotes, are listed in Table 2.

Table 2 Themes Used to Determine Appropriateness of Care Management Referral, with Modifiers and Examples
Full size table

Prevalence of the 12 themes identified in the qualitative analysis varied by referral category. Among the 176 exemplary good candidate patients, the three most common themes were inadequate but available social support (49% of good candidates), high health care need (28%), and inadequate medical system access (16%). These themes represented 40% of the themes in the good candidate category, and at least one of these themes was present in 72% of good candidate patients. Among the 76 exemplary patients in the not needing care management category, the top three themes were adequate social support (51%), low health care need (38%), and adequate agency (18%). These three themes represented 55% of all themes for this category, and at least one of the themes was present in 91% of the not needing patients. Among the 94 exemplary patients in the needing more category, the top three themes were psychiatric illness or active substance abuse (45%), inadequate motivation (32%), and inadequate and unavailable social support (22%). These three themes represented 51% of all themes for individuals deemed to need more than traditional intensive care management, and at least one of these three themes was present in 79% of the needing more patients.

Using Care Manager Themes to Distinguish Good Candidates

Several themes tended to be specific (> 90%) for one of the three categories of referral. Major non-medical transition, end-of-life stage, and low physical function themes were applied only to good candidates more than 90% of the time and low health care need and stable trajectory to not needing, and psychiatric or substance abuse, low motivation, and lack of specific system resources were generally applied to the needing more group. Conversely, several themes tended to cross categories (> 25% prevalence in more than one referral category). For example, adequate motivation and limited or low mental function were commonly (> 25%) seen in both good candidate and not needing categories, while high health care need, lack of agency, and unavailable social support were seen in both good candidate and needing more categories (Fig. 1).

Fig. 1
figure 1

Care managers reviewed complex patients referred to them for intensive case management in the prior year. Appropriateness of these referrals was classified into one of three mutually exclusive categories: “not needing,” “good candidates,” or “needs more.” Themes were developed based on reasons given by case managers for their classifications. The figure depicts themes that tended to be specific to one referral category (> 90% of patients with theme found in one category) vs. themes that were not specific to one referral category (> 25% of patients with theme found in at least two categories).

Full size image

DISCUSSION

Finding the “right” medically complex patients for intensive care management remains an important and frequently unrealized goal for health care systems. Current referral strategies include quantitative risk prediction tools, health risk assessments, prior care utilization, referral by staff members, and a focus on high-risk conditions.16,18 In our study, we found that less than two thirds of referred patients were considered to be appropriate for intensive care management by their care managers, a clear indication that there remains substantial room for improvement. Our results indicate that factors such as motivation and availability of social support are important considerations in the referral process. Because these predictors are not often readily available in the electronic medical record, efforts to use predictive modeling to identify high-risk patients will likely continue to fall short.

Among patients not considered good candidates in our study, roughly equal proportions were considered either not needing care management or needing something more than traditional care management, underscoring the concept that referrals can err on either side of the complex care needs spectrum. Our EHR-based model to predict whether a study patient would be considered a good candidate had a c statistic of 0.75, indicating good but not excellent model prediction. This result emphasizes that there remain “electronically unmeasured” factors that are important contributors to defining good referral candidates. Many of these factors fall within the domain of social and behavioral determinants of health, such as social support, health trajectory, non-medical transitions, and patient motivation. Finding better ways to define (or systematically collect) these variables would be the first step towards building better predictive models using EHR data.24,25,-26

We delved further into these missing data elements by asking CMs why they placed individual patients into each of the referral categories. This qualitative analysis identified 12 themes grouped into four domains (patient medical factors, patient non-medical factors, patient trajectory, and care system factors). These results highlight the potential for themes to interact with each other and the role of variables (e.g., motivation, availability of social support, trajectories over time) that are generally neglected by traditional diagnosis and utilization-based patient identification algorithms. One prominent criterion not commonly measured in clinical care that helped identify good candidates was the availability of social supports. Specifically, complex patients who did not have current social support but could have supports recruited to help with their care were often the best candidates for care management. This finding supports prior work that highlighted the impact of social isolation, which in one study was found to have a comparable association with mortality as traditional clinical risk factors.27

Patient “intangibles” such as motivation and agency often determined whether a patient would benefit from intensive care management. In the view of the CMs, patients with adequate agency could avail themselves of existing resources and did not require intensive care management. Conversely, patients who lacked motivation to engage in improving their own health were often considered intractable to the efforts of care management. These patients often remain a considerable source of professional stress for providers and underscore the need to develop new strategies beyond traditional care management. Recent research has shown that patient activation, a concept which captures both agency and motivation, can be reliably measured and can be used to both predict and tailor types of interventions.28

Several limitations of our study design should be taken into consideration. Although there was wide geographic and socio-economic diversity represented by the 17 clinical centers within our study, all patients were members of a single integrated health care system. While we are confident that patient-level factors are likely generalizable across systems, the role of specific system-level resource barriers and facilitators will likely vary depending on the local care systems. In addition, our gold standard of referral appropriateness was based on care manager experience and assessment. One strength of this approach is that CMs are considering their own patients rather than hypothetical or unfamiliar patients. However, this post hoc categorization may be subject to unmeasured bias. Also missing was information from the clinicians who initiated the referrals. Future work to understand the initial referral process and using clinical outcomes as an additional or alternate gold standard is warranted.

Effective interventions may look ineffective if implemented in the wrong patents. Identifying the optimal patients for referral remains a key unresolved challenge for care management programs. Complex patients have high heterogeneity in their mix of addressable and non-addressable individual barriers to care, leading to corresponding variation in the type of interventions that can be offered within a care system. Given this heterogeneity among otherwise apparently similarly complex patients, identifying which of these patients are most likely to benefit from intensive care management remains a critical challenge. Based on the insights of care managers and their experiences with patients referred to them for intensive care management, we have found that (1) data currently available from existing electronic health record data sources are likely insufficient to optimally identify good candidates for referral; (2) frequently unmeasured factors such as motivation, availability of social support, recent non-medically related life changes (e.g., death of a spouse), and current health trajectories are all influential factors in identifying which patients would benefit from care management; and 3) factors such as health care utilization and mental function may be insufficient to distinguish good referral candidates among medically complex patients. Future efforts to more optimally identify which complex patients are most likely to benefit from care management will require systematic collection of data beyond the traditional domains of medical conditions and prior utilization.