INTRODUCTION

Unhealthy alcohol use, ranging from risky drinking to alcohol use disorder (AUD),1 is highly prevalent2 and associated with consequences at both individual and societal levels.3,4 Most at-risk individuals, however, receive no clinical care for their unhealthy drinking.2,5,6 Primary care settings hold promise for engaging a larger proportion of patients with unhealthy drinking in treatment. This study aims to test the link between delivery of guideline-based alcohol care among routine primary care patients and reduced alcohol consumption.

Quality of care for unhealthy alcohol use can be assessed using quality measures that describe the extent to which care is consistent with clinical practice guideline recommendations. These guidelines combine scientific evidence and expert opinion to guide providers in delivering appropriate care for specific conditions. Current guidelines on substance use disorders include recommended care for the full spectrum of unhealthy alcohol use.7,8 Most alcohol-related quality measures, in contrast, focus on patients with an AUD diagnosis and on care delivered in specialty care settings9,10 and are not applicable to patients with undiagnosed or milder, but potentially harmful, alcohol use.11 This study uses a set of measures that assess alcohol-related care in primary care or specialty care settings and applies the measures across the full spectrum of primary care patients screening positive for unhealthy alcohol in 11 Veterans Health Administration (VHA) primary care practices.

Historically, quality measures have often been evaluated based on face validity relative to scientific evidence and expert opinion and the feasibility of measure implementation. With increased use of quality measures,12 validating process-based quality measures in terms of their ability to predict meaningful improvements in patient outcomes has become increasingly important.13 Prior studies have evaluated the predictive validity of some alcohol-related quality measures and have demonstrated a relationship between better care and improved clinical outcomes,14 increased employment,15 decreased arrests,16 and decreased mortality.15,17 However, we know of no prior studies that focused on assessing whether a comprehensive set of alcohol quality of care measures predicted subsequent changes in alcohol use when applied to a broad primary care population.

The objectives of this study were to describe quality of care for unhealthy alcohol use among patients visiting primary care settings and to determine whether better quality of alcohol care was linked to reduced alcohol consumption 6 months later.

METHODS

Research Design and Study Cohort

Participants were recruited following a positive screen for unhealthy alcohol use during routine screening at one of 11 primary care practices in the Greater Los Angeles VA Healthcare System (GLAVA) from February 2013 to January 2014, and were followed prospectively for 6 months. The clinical reminder system within the electronic medical record prompts providers to perform screening using the Alcohol Use Disorders Identification Test – Consumption (AUDIT-C).18 A positive screen was defined as a score of 5 or higher, the threshold used by VHA to indicate follow-up interventions.19 Additional study eligibility criteria included no AUD outpatient or inpatient encounter in the prior 90 days, at least 18 years of age, ability to speak and understand English, have a valid telephone, no cognitive impairment, and engaged in care (at least one outpatient visit within GLAVA in the prior 12 months). The RAND Human Subjects Protection Committee and the GLAVA Institutional Review Board approved all procedures.

Data Collection

Eligible patients received a letter followed by a telephone invitation. Interviewers explained study participation and obtained verbal informed consent. Baseline telephone interviews were completed an average of 15.3 days after the positive screen visit (range six to 87). Follow-up telephone interviews were conducted 6 months after the positive screen visit (mean 187.2 days, range 174 to 299). Participants received incentives for baseline ($50) and follow-up ($60).

Measures

Patient Characteristics

Gender and age were obtained from administrative records. Marital status, education, employment, race/ethnicity, income, receiving all or most care at VHA, depression symptoms,20 physical and mental health functioning,21 and drug use (illegal and prescription misuse) were collected from the baseline survey.

Process Measures

Candidate process measures were based on a review of existing measures and clinical practice guidelines and were selected and refined using a modified RAND/UCLA expert consensus panel approach22 and then operationalized, yielding a total of 21 measures.23 We applied the measures using administrative data (i.e., outpatient visits, pharmacy, laboratory) and medical record review to assess outpatient care across both primary and specialty settings in the 6 months following identification of unhealthy alcohol use (Table 1). Six measures required only administrative data, while 15 measures integrated both administrative and medical record review data. Two registered nurses used a structured abstraction tool to collect data elements from medical record text notes. Abstractors achieved high interrater reliability (percent agreement 99.1%, kappa 98.0%).

Table 1 Performance on Quality Measures for Unhealthy Alcohol Use
Full size table

We defined a formative composite24 to summarize documented care processes identified by the expert panel that each patient received for unhealthy alcohol use. We created an overall percentage composite measure, defined as the proportion of individual process-based quality measures (described below) for which each patient received the recommended care relative to all for which the patient was eligible. Each eligible measure is weighted equally for each patient. Composite measures have several advantages,24,25 including the potential to increase reliability and maximize the number of patients included in the analyses relative to individual measure analyses that are limited to patients in the denominator of an individual measure. For inclusion in our composite measures, individual measures needed to have a denominator of at least 20 eligible patients, a numerator of at least four, and a pass rate no smaller than 2% and no larger than 98%. Using these 17 qualifying measures, we developed two sub-composites: (1) screening including screening and assessment for unhealthy alcohol use (eight measures: 1–7, 24; see Tables 1 and 2) and (2) treatment including brief intervention, initial discussion of treatment options, initiation of treatment, and receipt of treatment for unhealthy alcohol use (nine measures: 8–11, 15, 18, 19, 21, 25).

Table 2 Demographic and Clinical Characteristics of Study Sample at Baseline (n = 719)
Full size table

Alcohol Use Measures

We assessed past 30-day drinking using the Timeline Followback (TLFB) at both baseline and follow-up.26 We examined two drinking outcomes: percent of heavy drinking days in the past 30 days out of days available (i.e., not incarcerated or hospitalized) and mean drinks per week in the past 30 days. A heavy drinking day was defined as five or more drinks for men and four or more drinks for women. Drinking above recommended weekly limits was defined as mean drinks per week of more than 14 drinks per week for men and more than seven per week for women.27 These measures are distinct from the positive alcohol screen for study eligibility, which was an AUDIT-C score of 5 or higher during a routine screen.

The goal of the baseline interview was to capture alcohol use prior to the screen-positive visit, as interventions delivered immediately following the positive screen could affect alcohol use. Past 30-day alcohol use included some days after the screen-positive visit, so we used drinking data for all days prior to the screen-positive visit for each patient to characterize baseline drinking. We imputed missing daily alcohol consumption data, both at baseline and 6-month follow-up, using a hot-deck multiple imputation approach28 (see Appendix).

Response Rate and Analytic Sample

The adjusted response rate (AAPOR response rate 129) at baseline was 54% (raw rate: 51%; 973 completes of 1922 potentially eligible). The probability of baseline non-response was not significantly associated with any characteristics available for all eligible patients (age, gender, income, marital status, or service era; results not shown). Of 1032 who consented to participate, 46 were excluded for reporting alcohol-related encounters within the past 90 days and 13 declined further participation. The follow-up adjusted response rate was 82% (raw rate, 79%; 770 completes of 973 baseline completers). Removal of patients sampled in error (e.g., updated administrative data indicated non-eligibility) resulted in a sample of 913 at baseline and 719 at follow-up. Two patients reported no days available for drinking in the past 30 days at follow-up and were excluded from modeling analyses (n = 717).

Statistical Analyses

Patient Characteristics and Receipt of Recommended Care

We used descriptive statistics, weighted for survey non-response, to describe the characteristics of the study sample and the proportion of patients who received recommended care on each quality measure.

Composite Measure Analyses

To assess the association between composite measures and risk of heavy drinking, we fit quasi-Poisson models to account for overdispersion and estimated robust standard errors for regression coefficients.30,31 We regressed the number of heavy drinking days in the 30 days preceding the follow-up interview on the target composite measure. Models were run using the overall composite as the predictor of interest, as well as separate models containing both sub-composites (assessment; treatment). All models included age, marital status, education, race/ethnicity, income, mental and physical functioning, receiving all or most care in VHA, and the baseline value of alcohol use, derived from daily drinking data preceding the positive screening visit. Since 47.6% (n = 342) reported no heavy drinking days at baseline, we also fit these models including only a targeted subset of 375 patients who had at least one heavy drinking day prior to positive screening.

We fit linear regression models for the average number of drinks per week outcome. We transformed the outcome (taking the log of the average number of drinks per week + 1) so the model would be better aligned with linear regression assumptions. Under this model, the composite quality coefficient estimate can be expressed in terms of the expected percent change in drinks per week associated with receiving all versus none of the care recommended for a patient. We adjusted for the same covariates described above. We also fit these models on a targeted subsample of 357 patients who drank above recommended weekly limits in the period prior to positive screening (i.e., > 14 drinks per week for men; > seven per week for women).

Individual Process Measure Analyses

For both alcohol consumption outcomes, we similarly examined nine individual measures: Assess for Alcohol Use Disorders, Evaluate for Suicide Risk, Screen for Other Substance Use, Test Liver Function, Conduct Brief Intervention, Discuss Treatment Options, Offer Psychosocial Intervention, Reassess Alcohol Use, and Repeat Brief Intervention. These were the subset of measures for which the sample size was sufficient to detect a medium-sized association (Cohen’s d = 0.5 standard deviations32 (SD)) between the individual measure and the logarithm of average number of drinks per week. We fit models using these measures in place of the composites for both full sample and targeted subsample analyses as described above.

RESULTS

Sample Characteristics

Patient demographics and clinical characteristics are provided in Table 2. Almost all patients were male (97%), and nearly half reported a race/ethnicity other than White (48%). On average, participants were 55 years old. Nearly two fifths of the sample was married (36%), two fifths were employed (40%), and one third earned $15,000 or less a year (31%).

Alcohol use at baseline and 6-months is described in Table 3. Despite all patients screening positive for unhealthy alcohol use in the routine clinical setting, only 83% of patients had an AUDIT-C of 5 or higher during the baseline interview. One-quarter had been diagnosed with an AUD in the past 2 years (26%). Overall, there was little change in drinking between baseline and follow-up (e.g., Cohen’s d = 0.11 SD and d = 0.12 SD differences for percent heavy drinking days and drinks per week, respectively).

Table 3 Characteristics of Alcohol Use at Baseline and 6-Month Follow-up (n = 719)
Full size table

Care for Alcohol Misuse

Table 1 describes care received for unhealthy alcohol use among patients followed at 6 months. Five patients (0.7%) had documentation of receiving all recommended care, and 28% had documentation of receiving at least half of the recommended care processes. Performance on the process measures varied widely, ranging from less than 1 to 93% (median, 33%). Processes of care with performance rates higher than 75% included screening for co-occurring depression (93%), delivering a brief intervention (84%), and testing patients’ liver function (81%). One measure of whether patients who received specialty mental health care were given a diagnostic assessment for depression had moderate performance (61%). The remaining 17 measures had lower performance (i.e., fewer than 50% of patients received recommended care). Among these, the lowest measures were treatment initiation (5.0%), offering pharmacotherapy (3.1%), treatment engagement (1.5%), and receipt of pharmacotherapy (0.5%).

Multivariable Model Results

Table 4 presents estimated changes in the percentage of heavy drinking days and percent changes in drinks per week in the 30 days prior to the follow-up interview after adjustment for baseline drinking, demographics, mental and physical functioning, and receiving all or most of care from VHA. In analyses of the full sample and of targeted subsamples, neither the overall composite nor either of the two sub-composites were associated with statistically significant changes in the percentage of heavy drinking days or changes in drinks per week.

Table 4 Association Between Quality of Care for Unhealthy Alcohol Use Composite Measures and Change in Drinking Outcomes
Full size table

Table 5 presents model results for nine selected individual quality measures. Brief Intervention was associated with a statistically significant reduction in drinks per week within the targeted subsample that had exceeded recommended drinking limits in the 30 days preceding the positive screening visit. Those in the subsample who received a brief intervention reduced their adjusted drinks per week by 25.4% (95% CI 6.0, 40.8; p = 0.013) on average. Reassessment of Alcohol Use was associated with a significant increase in the percent of heavy drinking days in both the full sample and the targeted subsample. In the full sample, patients with documentation of reassessment increased their adjusted percent of heavy drinking days by 30.75% (95% CI 5.3, 62.4; p = 0.015), and among those who had at least one heavy drinking day prior to the positive screen visit, the increase was 36.3% (95% CI 8.1, 71.8; p = 0.009).

Table 5 Association Between Individual Measures of Quality of Care for Unhealthy Alcohol Use and Change in Drinking Outcomes
Full size table

DISCUSSION

This study comprehensively applied measures of the quality of alcohol-related processes of care to an alcohol screen–positive primary care population seeking care in one of 11 primary care practices. The process measures span care received in both primary and specialty care settings, were based on the best available evidence for importance and feasibility, used both administrative encounter data and medical review data, and were the result of an expert panel process. We observed wide variability in the proportion of patients receiving recommended care, and the median performance rate across these measures was only 33%. Further, we observed negligible change in alcohol use in the 6 months following the positive screen. These findings highlight the need for improvements in the quality of care delivered to patients with unhealthy alcohol use (or at least care that is documented).

We found receipt of higher overall quality of care for unhealthy alcohol use among a diverse population of patients who screened positive on the AUDIT-C in primary care clinics was not associated with significant reductions in drinking during our follow-up period. Composite measures separately capturing screening/assessment and treatment similarly were not associated with reduced drinking. Subgroup analyses of patients with at least one heavy drinking day or who were drinking above recommended weekly limits at baseline yielded similar results. Receipt of brief intervention significantly predicted decreased drinking in patients who were drinking above recommended weekly limits. However, cautious interpretation is warranted, as up to three significant findings (p < 0.05) would be expected by chance alone.33

It remains unclear why we did not detect an overall process-outcome link between higher-quality alcohol care and reduced drinking, particularly when we used composites designed to capture multiple aspects of recommended care—an approach similar to prior work that found a process-outcome link in depression care.34 Low rates of receipt of effective interventions (e.g., pharmacotherapy) prevented us from adequately testing some measures. Perhaps evidence-based interventions identified in research and assessed by our measures were not applied with sufficient adherence. Alternatively, the approach to capture whether recommended care occurred may not reflect actual care delivered. For example, providers may document in the medical record that an assessment or intervention was conducted when it was not done with adequate fidelity to evidence-based practice.

Some limitations should be noted. Almost half of patients at baseline reported no heavy drinking days, limiting the amount of change we could detect and reducing the sample size of those with more severe drinking. There was negligible change in drinking outcomes overall (e.g., Cohen’s d = 0.11–0.12), leaving little room for reductions in drinking to be associated with composite and individual measures. Few patients received specialty care, limiting our ability to test composites and certain measures. VHA uses clinical reminders to prompt screenings, which may result in detecting a higher proportion of patients with unhealthy alcohol use. The focus on VHA clinics also resulted in fewer women and younger men being included in our sample, particularly because we used the VHA’s AUDIT-C positive screen cut point of 5 or greater for both men and women. Further, although we adjusted for observable risk factors, there could be residual confounding of our estimates by other unmeasured factors. For example, 95% confidence intervals for some measures (Test Liver Function, Reassess Alcohol Use, Discuss Treatment Options) suggested receipt of recommended care was associated with increased drinking. Perhaps providers detected and responded to signals of increased alcohol use that were not detected by our severity measures or other observable characteristics. Finally, our response rate (54% at baseline) may impact generalizability of our findings; there could be unobserved differences between study participants and the population despite the lack of significant differences on observed demographics.

Results indicated that few patients received recommended care following a positive screen for unhealthy alcohol use in routine practice. Our analyses did not demonstrate that overall receipt of a greater percentage of recommended care processes predicted decreases in drinking at 6-month follow-up among patients in our sample. Subgroup analyses suggest, however, that providing brief interventions to higher-risk patients is linked to reduced drinking. Results highlight a critical need for greater focus on how to provide comprehensive alcohol care to patients across both primary care and specialty care settings.