Skip to main content
Download PDF

Practical Assessment of Alcohol Use Disorder in Routine Primary Care: Performance of an Alcohol Symptom Checklist

Journal of General Internal Medicine volume 37, pages 1885–1893 (2022)Cite this article

Abstract

Background

Alcohol use disorder (AUD) is highly prevalent but underrecognized and undertreated in primary care settings. Alcohol Symptom Checklists can engage patients and providers in discussions of AUD-related care. However, the performance of Alcohol Symptom Checklists when they are used in routine care and documented in electronic health records (EHRs) remains unevaluated.

Objective

To evaluate the psychometric performance of an Alcohol Symptom Checklist in routine primary care.

Design

Cross-sectional study using item response theory (IRT) and differential item functioning analyses of measurement consistency across age, sex, race, and ethnicity.

Patients

Patients seen in primary care in the Kaiser Permanente Washington Healthcare System who reported high-risk drinking on the Alcohol Use Disorder Identification Test Consumption screening measure (AUDIT-C ≥ 7) and subsequently completed an Alcohol Symptom Checklist between October 2015 and February 2020.

Main Measure

Alcohol Symptom Checklists with 11 items assessing AUD criteria defined in the Diagnostic and Statistical Manual for Mental Disorders, 5th edition (DSM-5), completed by patients during routine medical care and documented in EHRs.

Key Results

Among 11,464 patients who screened positive for high-risk drinking and completed an Alcohol Symptom Checklist (mean age 43.6 years, 30.5% female), 54.1% reported ≥ 2 DSM-5 AUD criteria (threshold for AUD diagnosis). IRT analyses demonstrated that checklist items measured a unidimensional continuum of AUD severity. Differential item functioning was observed for some demographic subgroups but had minimal impact on accurate measurement of AUD severity, with differences between demographic subgroups attributable to differential item functioning never exceeding 0.42 points of the total symptom count (of a possible range of 0–11).

Conclusions

Alcohol Symptom Checklists used in routine care discriminated AUD severity consistently with current definitions of AUD and performed equitably across age, sex, race, and ethnicity. Integrating symptom checklists into routine care may help inform clinical decision-making around diagnosing and managing AUD.

Practical Assessment of Alcohol Use Disorder in Routine Primary Care: Performance of an Alcohol Symptom Checklist

BACKGROUND

Sixty-one million people in the United States (US) drink alcohol above recommended limits1 and 13.9% of US adults meet past-year criteria for alcohol use disorder (AUD).2 In the US, AUD is associated with 3% of deaths annually and 5% of years of life lost due to premature mortality.3,4 Despite this large burden of disease, up to 78% of patients with AUD who visit primary care go undiagnosed5 and 80% or more go untreated1,2,6,7,8,9 even though effective AUD treatments can be delivered in primary care.10,11,12

A lack of practical, validated measures for diagnosing AUD in routine medical care perpetuates this treatment gap. Although brief screening measures, such as the Alcohol Use Disorders Identification Test–Consumption screen (AUDIT-C),13,14 can detect potentially harmful levels of alcohol consumption, there is currently a lack of practical measures for assessing AUD symptoms. Experts recommend using an Alcohol Symptom Checklist, where patients self-report the presence or absence of the 11 AUD symptoms defined by the Diagnostic and Statistical Manual of Mental Disorders, 5th Edition (DSM-5)15 to facilitate patient-provider discussions about AUD and other substance use disorders.16,17 However, the psychometric performance of such a measure when used in routine care has never been tested.

In 2015, Kaiser Permanente (KP) Washington began implementing an Alcohol Symptom Checklist,18,19,20 which is now used in all primary care clinics and has been completed by over 11,000 patients who reported high-risk drinking on an alcohol screening measure (AUDIT-C scores of 7–12). The present study evaluated the psychometric performance of this Alcohol Symptom Checklist by assessing its ability to measure AUD symptoms along a unidimensional continuum, discriminate AUD severity consistently with scientifically accepted views of AUD,15 and measure AUD severity similarly across subgroups based on age, sex, race, and ethnicity.

METHODS

Study Setting

This cross-sectional study used secondary clinical data from electronic health records (EHRs) and insurance claims of KP Washington, an integrated health system in Washington State.

Screening and Assessment Procedures

As part of an effort to integrate behavioral health care within primary care, all 32 KP Washington primary care clinics implemented annual alcohol screening for adult patients, followed by an EHR prompt to have patients complete an Alcohol Symptom Checklist if the screening indicated high-risk drinking (i.e., AUDIT-C scores of 7–12).20 As of February 2020, alcohol screens were completed by 91% of adult primary care patients. Most alcohol screens were completed during primary care appointments (65.0%); others were completed during mental health (28.5%) and other types of visits (e.g., obstetrics, urgent care; 6.5%). In the most common case where screening occurred in primary care, the EHR automatically prompted check-in staff and medical assistants to give the paper screening questionnaire if it had not been completed within the past year. Primary care patients then completed the screening questionnaire in waiting areas or exam rooms and medical assistants entered responses into the EHR prior to the patient’s appointment with primary care providers. If a primary care patient screened positive for high-risk drinking within the past year without completing a subsequent Alcohol Symptom Checklist, the EHR prompted medical assistants to give them a paper-based Alcohol Symptom Checklist, which medical assistants entered into the EHR.

Patient Population

Patient eligibility for this study was identified based on EHR data and included (a) ≥ 1 visit to a KP Washington primary care clinic between October 1, 2015, and February 29, 2020, (b) screened positive for high-risk drinking (AUDIT-C score of 7–12), (c) completed the Alcohol Symptom Checklist 0–365 days after the positive alcohol screen, and (e) at least 18 years old when the checklist was completed. The study was approved by the KP Washington Health Research Institute’s Institutional Review Board with a waiver of consent and HIPAA authorization to use existing EHR data.

Measures

Alcohol Screening Measure

The AUDIT-C13,14 is a validated21,22 self-report measure of alcohol consumption included in KP Washington’s annual behavioral health screen. It includes three items that assess the frequency of drinking, typical drinks per drinking day, and frequency of heavy drinking (eSupplement). Items are answered on a 5-point scale (0–4 points) and then summed (total score 0–12 points). If AUDIT-C scores were 7–12,23 the EHR prompted staff to ask patients to complete an Alcohol Symptom Checklist.

Alcohol Symptom Checklist

KP Washington’s Alcohol Symptom Checklist (eSupplement) is an 11-item self-report questionnaire that asks patients whether they have experienced each of the 11 DSM-5 AUD criteria within the past year, a timeframe consistent with diagnostic standards.15 Summed scores reflect criteria counts (0–11) that may aid the determination of an AUD diagnosis (≥ 2 criteria endorsed) and its severity as defined in DSM-5 (mild: 2–3 criteria; moderate: 4–5 criteria; severe: 6–11 criteria), if clinicians determine symptoms are recurrent.

Demographics and Other Measures

Age was grouped into similar categories as a previous study evaluating an AUD diagnostic measure in a national epidemiological sample (18–24, 25–44, 45–64, and 65+ years).24 Sex was coded as male or female based on available EHR data. Race and ethnicity were coded to align with categories defined by the US National Institutes of Health (race: White, Black/African American, Asian/Asian American, American Indian/Alaska Native, or Native Hawaiian/Pacific Islander; ethnicity: non-Hispanic or Hispanic). Participants with other (4.5%) or unknown race (5.2%) or unknown ethnicity (5.4%) were included in full-sample analyses but excluded from race- and ethnicity-specific analyses. Medicaid and Medicare insurance statuses were obtained from enrollment records. AUD diagnoses up to 2 years before completing Alcohol Symptom Checklists were identified using ICD-9 and ICD-10 codes from EHRs and insurance claims.

Analytic Approach

Psychometric analyses tested whether the Alcohol Symptom Checklist performed in a manner consistent with current scientific conceptualizations of AUD15 and with the performance of AUD diagnostic measures tested in previous epidemiological studies.24,25,26

Unidimensionality

Current diagnostic standards15 and population-based studies24,25,26 characterize AUD as a brain disorder with a unidimensional continuum of severity reflected by the number of AUD criteria present. This contrasts popular perceptions of AUD as a binary or categorical condition that is often assumed to be severe (e.g., “alcoholism”)27,28 and also contrasts historical diagnostic standards that distinguished alcohol abuse and alcohol dependence as separate diagnoses without severity designations.29

In this study, confirmatory factor analysis tested whether the Alcohol Symptom Checklist measured a single, continuous underlying dimension of AUD severity (vs. multiple dimensions) by specifying a model with all 11 AUD criteria loading onto a single, continuous latent variable. The factor model was fit using maximum likelihood within a two-parameter logistic item response theory (IRT) framework via the mirt package in R.30,31,32,33 Adequacy of the unidimensional factor model was evaluated by comparing model fit indices to a priori cutoffs for acceptable fit: comparative fit index >.95, root mean square error of approximation <.05, standardized root mean square residual <.05 (possible range for all indices: 0–1).34,35

Item Characteristics

IRT analyses characterized how well the Alcohol Symptom Checklist items discriminated higher versus lower latent AUD severity. Specifically, the IRT model36 estimated each item’s ability to discriminate whether a patient has higher versus lower latent AUD severity (discrimination parameter) and where along the continuum of latent AUD severity the item best discriminates (severity parameter). These parameters were illustrated graphically using item characteristic curves to show the probabilities of endorsing each of the 11 items based on a patient’s underlying latent AUD severity.

Differential Item Functioning

To ensure equitable interpretation of Alcohol Symptom Checklists, it is important to understand whether the number of AUD criteria endorsed on the checklist is influenced by other factors besides AUD severity, such as the patient’s age, sex, race, or ethnicity. If a patient’s likelihood of endorsing specific Alcohol Symptom Checklist items is influenced by demographic factors, rather than by their AUD severity, those items are said to have differential item functioning, which may suggest the instrument measures AUD severity differently for different subgroups.

Differential item functioning was tested using established procedures37,38,39 that compared the fit of an IRT model that assumed identical item discrimination and severity parameters for all demographic subgroups to models that estimated discrimination and severity parameters separately for each subgroup. Differential item functioning was indicated when the latter model resulted in significantly better fit via likelihood ratio testing.

Assessing the Clinical Impact of Differential Item Functioning

Importantly, differential item functioning can be statistically significant without having a clinically meaningful impact on the determination of AUD severity.37,40 Because AUD diagnosis and its severity are determined by the number of AUD criteria present,15 knowing the impact of differential item functioning on the number of criteria endorsed is more clinically important than knowing whether there is differential item functioning for individual items within the checklist.40

We evaluated the cumulative impact of differential item functioning in three ways, namely (1) examining how the expected number of criteria endorsed by patients with the same latent AUD severity increased or decreased due to differential item functioning across different demographic subgroups,37 (2) comparing estimates of latent AUD severity (factor scores) with versus without correction for differential item functioning,41 and (3) examining differences in fit indices of models with versus without correction for differential item functioning for each demographic subgroup.42,43

RESULTS

Sample Description

Of 449,795 primary care patients who completed AUDIT-C screens, 17,760 (3.9%) had AUDIT-C screening scores of 7–12 indicating high-risk drinking at any point during the study period. Within this subgroup, 11,464 (64.5%) completed an Alcohol Symptom Checklist and were included in subsequent analyses (88.5% were completed the same day as the AUDIT-C screening). Among those with high-risk drinking, patients were more likely to subsequently complete an Alcohol Symptom Checklist if they were male, had an AUDIT-C score 10–12, or did not have a past 2-year AUD diagnosis from a healthcare provider; however, rates of Alcohol Symptom Checklist completion did not differ by age, race, ethnicity, or Medicaid/Medicare insurance status (see Table 1). There were 2064 patients (18.0% of those with Alcohol Symptom Checklists) who completed more than one checklist; for these patients, only the first checklist was retained to maintain independent observations.

Table 1 Characteristics of Sample: Patients With and Without Alcohol Symptom Checklists After Screening Positive for High-Risk Drinking on the AUDIT-C
Full size table

Characteristics of the sample that completed the Alcohol Symptom Checklist are shown in Table 1. On average, participants were 43.6 years old and 30.5% were female. Most participants were White and non-Hispanic, but racial and ethnic minority subgroups were large enough (n = 163 to 780) to adequately test differential item functioning.44

Rates of AUD Criteria

Of patients who completed an Alcohol Symptom Checklist, 54.1% reported ≥ 2 AUD criteria, exceeding the threshold for an AUD diagnosis, including 20.1%, 12.1%, and 21.8% reporting symptoms consistent with mild, moderate, and severe AUD, respectively. In contrast, only 19.6% of patients who completed the Alcohol Symptom Checklist had AUD diagnosed within the 2-year period before completing the checklist based on EHR diagnoses and insurance claims. The 11 AUD criteria were endorsed at varying rates, ranging from 16.2% reporting being intoxicated more than once in situations where it was dangerous to 44.9% reporting continued drinking despite knowing or suspecting it creates or worsens mental or physical problems (Table 2).

Table 2 Prevalence of Specific Alcohol Use Disorder (AUD) Criteria and Item Response Theory (IRT) Parameter Estimates
Full size table

Tests of Unidimensionality

Confirmatory factor analysis indicated that the Alcohol Symptom Checklist measured latent AUD severity along a unidimensional continuum. The unidimensional factor model provided excellent fit34,35 to the 11 Alcohol Symptom Checklist items: comparative fit index = 0.99, root mean square error of approximation = 0.04, standardized root mean square residual = 0.02.

Item Characteristics

Item characteristic curves (Fig. 1) show the probabilities of each AUD criterion being reported with respect to a patient’s latent AUD severity. IRT discrimination and severity parameters are shown in Table 2. Most items had high discrimination parameters, indicating they were strongly associated with AUD severity (Table 2). Comparatively, discrimination parameters were somewhat lower for the two items assessing tolerance and hazardous use. Some items discriminated AUD severity better when latent AUD was mild to moderate (e.g., physical/psychological problems, larger/longer, craving) as demonstrated by lower severity parameters; other items discriminated AUD severity better when AUD was moderate to severe (e.g., hazardous use, withdrawal, neglect roles) as demonstrated by higher severity parameters (Table 2).

Fig. 1
figure 1

Item characteristic curves for the eleven alcohol use disorder (AUD) criteria assessed on the Alcohol Symptom Checklist. Latent AUD severity (x-axis) is distributed such that 0 corresponds to the sample mean (approximately 2.98 AUD criteria) and ±1 corresponds with ±1 standard deviation (SD) from the mean (SD = 3.23 AUD criteria).

Full size image

Differential Item Functioning

Seven items had statistically significant differential item functioning associated with age (tolerance, larger/longer, quit/control, hazardous use, social/interpersonal problems, craving, activities given up). Three items had significant differential item functioning associated with sex (larger/longer, hazardous use, craving) and race (tolerance, larger/longer, quit/control). One item had significant differential item functioning associated with ethnicity (quit/control; see eSupplement for detailed reporting).

Clinical Impact of Differential Item Functioning

Figure 2 displays the expected number of AUD criteria endorsed (y-axis), based on a patient’s latent AUD severity (x-axis) for all demographic subgroups (shown as separate curves that nearly overlap). Differences in expected criteria counts for individuals from different demographic subgroups with the same latent AUD severity were small and never diverged by more than half of one criterion, suggesting minimal cumulative impact of differential item functioning on total criteria counts and therefore minimal impact on clinical interpretation. When AUD severity was held constant, differential item functioning was expected to produce differences in AUD criteria counts that never exceeded 0.38 criteria for age (patients aged 18–24 reporting fewer criteria), < 0.1 criteria for sex, < 0.42 criteria for race (white patients reporting fewer criteria), and < 0.1 criteria for Hispanic ethnicity (Fig. 2; see supplement for additional details).

Fig. 2
figure 2

Analyses of differential test functioning for demographic subgroups. Latent alcohol use disorder (AUD) severity (x-axis) is distributed such that 0 corresponds to the sample mean (2.98 AUD criteria) and ±1 corresponds with ±1 standard deviation (SD) from the mean (SD = 3.23 criteria). Separate curves are shown to illustrate the expected number of AUD criteria endorsed (y-axis) for each demographic subgroup. Vertical distances between the plotted lines indicate the expected differences in the number of AUD criteria endorsed that are attributable to differential item functioning, when latent AUD severity is held constant between members of different subgroups. In all cases, the expected differences between expected number of AUD criteria endorsed attributable to differential item functioning (i.e., vertical distances between lines) were less than half of one criterion, which is not a clinically meaningful difference. More detailed differential item functioning results are presented in the online supplement.

Full size image

Estimated values of latent AUD severity (i.e., factor scores) with versus without correction for differential item functioning were similar and highly correlated (r ≥ 0.996), indicating that correction for differential item functioning typically had minimal impact on estimated latent AUD severity. Differences in the comparative fit indices of models with identical versus independently estimated discrimination and severity parameters for each subgroup were never larger than 0.01, indicating that correction for differential item functioning had minimal effect on absolute model fit (see eSupplement).42

DISCUSSION

Currently, there are no validated, practical measures available to assess AUD symptoms in routine primary care despite the considerable disease burden1,2,3,4 and ability to treat AUD.10,11,12 Prior research has shown that implementation of routine alcohol screening and assessment of AUD symptoms among patients with high-risk drinking as part of behavioral health integration is feasible19 and affordable,45 and that patients are often willing to report AUD symptoms on Alcohol Symptom Checklists.18 The current study further shows that the Alcohol Symptom Checklist, administered in routine care and entered into patients’ EHRs, is psychometrically valid, performs equitably across age, sex, race, and ethnicity subgroups, and can detect AUD symptoms for many patients who report high-risk drinking but have not been previously diagnosed with AUD.

The two items that exhibited somewhat lower discrimination parameters (tolerance, hazardous use) also had lower discrimination parameters in prior studies of AUD symptom measures,24,26,46 suggesting these criteria may discriminate AUD severity less strongly than other criteria. Similar to prior epidemiological studies,24,25,26 some items demonstrated differential item functioning, but the cumulative effect of differential item functioning on overall AUD criteria counts and estimated AUD severity was small and not of great enough magnitude to have a clinically meaningful impact.24,26

Most patients with AUD contact the health system through primary care47,48 and only 8% of them receive care in traditional addiction treatment settings.49 Primary care is therefore uniquely positioned to provide AUD-related care that many patients will not otherwise seek or receive.10,50,51,52 Primary care settings are increasingly adopting practices to screen patients for high-risk drinking,53 which is recommended by the US Preventive Services Task Force followed by more in-depth risk assessment when high-risk drinking is detected.54

Alcohol Symptom Checklists can be implemented as part of a broader effort to address behavioral health within primary care and can systematically assess AUD symptoms among patients who report high-risk drinking. Using the checklist after patients report high-risk drinking may help prompt patient-centered discussions about the negative consequences of alcohol use (e.g., “you indicated wanting to cut down or stop drinking but being unable to do so – can you tell me more about that?”), while also potentially helping improve problem recognition28 and guiding AUD diagnosis, treatment, and management.16,17,18 The use of a written checklist (versus verbal assessment of AUD symptoms) may help facilitate more valid responding, mitigate potential patient discomfort, and avoid stigma.55,56 Additional research may test whether Alcohol Symptom Checklists improve rates of AUD diagnosis and treatment engagement.

Limitations and Strengths

Our study has limitations. Patients may have underestimated or underreported their drinking on screens and/or underreported AUD symptoms. Additionally, 35.4% of patients with high-risk drinking did not have an EHR-documented Alcohol Symptom Checklist and reasons why are unknown (e.g., clinic staff may have had limited time or discomfort administering it; patients may have refused or presented with more urgent concerns). Results should therefore be interpreted as supporting the performance of the Alcohol Symptom Checklist among patients who report high-risk drinking and who complete the checklist as part of routine care. Sex, race, and ethnicity were modeled as categorical variables that do not capture the dimensional, intersectional, social, historical, and cultural factors that could potentially affect drinking and AUD (e.g., sexism, racism, social determinants of health).57,58 Our use of a routine care sample also prevented us from administering structured diagnostic interviews of AUD symptoms to test criterion validity. Future studies should compare the checklist to a gold standard criterion.

The study also has several strengths. Evaluating data collected in routine care (rather than during research participation) greatly enhances external validity by focusing on the performance of the Alcohol Symptom Checklist completed clinically where results are entered into EHRs and available to healthcare providers. The routine care sample also reduces the risk of sampling bias by including individuals who might not enroll in research studies. The psychometric evaluation used rigorous methods. The AUDIT-C is widely used and freely available for public use, along with the Alcohol Symptom Checklist (eSupplement), improving the potential to disseminate these tools in practice.

CONCLUSION

This study validated the Alcohol Symptom Checklist as a practical measure of AUD symptoms that provides scaled, unidimensional information to help primary care clinicians assess the presence of AUD and its severity, doing so equitably across age, sex, race, and ethnicity subgroups. Routinely screening for high-risk drinking and assessing AUD symptoms when patients report high-risk drinking may help primary care providers diagnose AUD, inform clinical decision-making, and promote patient-centered discussions about alcohol.

References

  1. Center for Behavioral Health Statistics and Quality. 2018 National Survey on Drug Use and Health: Detailed Tables. Published online 2019.

    Google Scholar 

  2. Grant BF, Goldstein RB, Saha TD, et al. Epidemiology of DSM-5 Alcohol Use Disorder: Results from the National Epidemiologic Survey on Alcohol and Related Conditions III. JAMA Psychiatry. 2015;72(8):757-766. doi:https://doi.org/10.1001/jamapsychiatry.2015.0584

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  3. Rehm J, Mathers C, Popova S, Thavorncharoensap M, Teerawattananon Y, Patra J. Global Burden of Disease and Injury and Economic Cost Attributable to Alcohol Use and Alcohol-Use Disorders. The Lancet. 2009;373(9682):2223-2233. doi:https://doi.org/10.1016/S0140-6736(09)60746-7

    Article  Google Scholar 

  4. Rehm J, Dawson D, Frick U, et al. Burden of Disease Associated with Alcohol Use Disorders in the United States. Alcoholism: Clinical and Experimental Research. 2014;38(4):1068-1077. doi:https://doi.org/10.1111/acer.12331

    Article  Google Scholar 

  5. Williams EC, Rubinsky AD, Lapham GT, et al. Prevalence of Clinically Recognized Alcohol and Other Substance Use Disorders Among VA Outpatients with Unhealthy Alcohol Use Identified by Routine Alcohol Screening. Drug Alcohol Depend. 2014;135:95-103. doi:https://doi.org/10.1016/j.drugalcdep.2013.11.016

    Article  PubMed  Google Scholar 

  6. Williams EC, Gupta S, Rubinsky AD, et al. Variation in Receipt of Pharmacotherapy for Alcohol Use Disorders Across Racial/Ethnic Groups: a National Study in the U.S. Veterans Health Administration. Drug Alcohol Depend. 2017;178:527-533. doi:https://doi.org/10.1016/j.drugalcdep.2017.06.011

    Article  PubMed  Google Scholar 

  7. Rieckmann T, Muench J, McBurnie MA, et al. Medication-Assisted Treatment for Substance Use Disorders Within a National Community Health Center Research Network. Subst Abus. 2016;37(4):625-634. doi:https://doi.org/10.1080/08897077.2016.1189477

    Article  PubMed  Google Scholar 

  8. Glass JE, Bohnert KM, Brown RL. Alcohol Screening and Intervention Among United States Adults who Attend Ambulatory Healthcare. J Gen Intern Med. 2016;31(7):739-745. doi:https://doi.org/10.1007/s11606-016-3614-5

    Article  PubMed  PubMed Central  Google Scholar 

  9. Hallgren KA, Witwer E, West I, et al. Prevalence of Documented Alcohol and Opioid Use Disorder Diagnoses and Treatments in a Regional Primary Care Practice-Based Research Network. J Subst Abuse Treat. 2020;110:18-27. doi:https://doi.org/10.1016/j.jsat.2019.11.008

    Article  PubMed  Google Scholar 

  10. Oslin DW, Lynch KG, Maisto SA, et al. A Randomized Clinical Trial of Alcohol Care Management Delivered in Department of Veterans Affairs Primary Care Clinics Versus Specialty Addiction Treatment. J GEN INTERN MED. 2014;29(1):162-168. doi:https://doi.org/10.1007/s11606-013-2625-8

    Article  PubMed  Google Scholar 

  11. Jonas DE, Amick HR, Feltner C, et al. Pharmacotherapy for Adults with Alcohol Use Disorders in Outpatient Settings: a Systematic Review and Meta-analysis. JAMA. 2014;311(18):1889-1900. doi:https://doi.org/10.1001/jama.2014.3628

    CAS  Article  PubMed  Google Scholar 

  12. Watkins KE, Ober AJ, Lamp K, et al. Collaborative Care for Opioid and Alcohol Use Disorders in Primary Care: the SUMMIT Randomized Clinical Trial. JAMA Intern Med. 2017;177(10):1480-1488. doi:https://doi.org/10.1001/jamainternmed.2017.3947

    Article  PubMed  PubMed Central  Google Scholar 

  13. Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA. The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Alcohol Use Disorders Identification Test. Arch Intern Med. 1998;158(16):1789-1795.

    CAS  Article  Google Scholar 

  14. Bradley KA, Bush KR, Epler AJ, et al. Two Brief Alcohol-Screening Tests from the Alcohol Use Disorders Identification Test (AUDIT): Validation in a Female Veterans Affairs Patient Population. Arch Intern Med. 2003;163(7):821-829. doi:https://doi.org/10.1001/archinte.163.7.821

    Article  PubMed  Google Scholar 

  15. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (DSM-5®). American Psychiatric Pub; 2013.

    Book  Google Scholar 

  16. Marsden J, Tai B, Ali R, Hu L, Rush AJ, Volkow N. Measurement-Based Care Using DSM-5 for Opioid Use Disorder: Can We Make Opioid Medication Treatment More Effective? Addiction. 2019;114(8):1346-1353. doi:https://doi.org/10.1111/add.14546

    Article  PubMed  PubMed Central  Google Scholar 

  17. Bradley KA, Caldeiro RM, Hallgren KA, Kivlahan DR. Making measurement-based care for addictions a reality in primary care. Addiction. 2019;114(8):1355-1356.

    Article  Google Scholar 

  18. Sayre M, Lapham GT, Lee AK, et al. Routine Assessment of Symptoms of Substance Use Disorders in Primary Care: Prevalence and Severity of Reported Symptoms. J Gen Intern Med. 2020;35(4):1111-1119. doi:https://doi.org/10.1007/s11606-020-05650-3

    Article  PubMed  PubMed Central  Google Scholar 

  19. Bobb JF, Lee AK, Lapham GT, et al. Evaluation of a Pilot Implementation to Integrate Alcohol-Related Care Within Primary Care. Int J Environ Res Public Health. 2017;14(9)1030. doi:https://doi.org/10.3390/ijerph14091030

    Article  PubMed Central  Google Scholar 

  20. Glass JE, Bobb JF, Lee AK, et al. Study Protocol: a cluster-Randomized Trial Implementing Sustained Patient-centered Alcohol-related Care (SPARC Trial). Implement Sci. 2018;13(1):108. doi:https://doi.org/10.1186/s13012-018-0795-9

    Article  PubMed  PubMed Central  Google Scholar 

  21. Bradley KA, DeBenedetti AF, Volk RJ, Williams EC, Frank D, Kivlahan DR. AUDIT-C as a Brief Screen for Alcohol Misuse in Primary Care. Alcohol Clin Exp Res. 2007;31(7):1208-1217. doi:https://doi.org/10.1111/j.1530-0277.2007.00403.x

    Article  PubMed  Google Scholar 

  22. National Council for Behavioral Health. Implementing care for alcohol & other drug use in medical setting: an extension of SBIRT. Published online 2018. https://www.thenationalcouncil.org/wp-content/uploads/2018/03/021518_NCBH_ASPTReport-FINAL.pdf. Accessed 2021 June

  23. Rubinsky AD, Dawson DA, Williams EC, Kivlahan DR, Bradley KA. AUDIT-C Scores as a Scaled Marker of Mean Daily Drinking, Alcohol Use Disorder Severity, and Probability of Alcohol Dependence in a U.S. General Population Sample of Drinkers. Alcohol Clin Exp Res. 2013;37(8):1380-1390. https://doi.org/10.1111/acer.12092. Accessed 2019

  24. Saha TD, Chou SP, Grant BF. Toward an Alcohol Use Disorder Continuum Using Item Response Theory: Results from the National Epidemiologic Survey on Alcohol and Related Conditions. Psychological Medicine. 2006;36(7):931-941. doi:https://doi.org/10.1017/S003329170600746X

    Article  PubMed  Google Scholar 

  25. Dawson DA, Saha TD, Grant BF. A Multidimensional Assessment of the Validity and Utility of Alcohol Use Disorder Severity as Determined by Item Response Theory Models. Drug and Alcohol Dependence. 2010;107(1):31-38. doi:https://doi.org/10.1016/j.drugalcdep.2009.08.019

    Article  PubMed  Google Scholar 

  26. Harford TC, Yi H, Faden VB, Chen CM. The Dimensionality of DSM-IV Alcohol Use Disorders Among Adolescent and Adult Drinkers and Symptom Patterns by Age, Gender, and Race/Ethnicity. Alcohol Clin Exp Res. 2009;33(5):868-878. doi:https://doi.org/10.1111/j.1530-0277.2009.00910.x

    Article  PubMed  PubMed Central  Google Scholar 

  27. Schomerus G, Matschinger H, Angermeyer MC. Continuum beliefs and stigmatizing attitudes towards persons with schizophrenia, depression and alcohol dependence. Psychiatry Research. 2013;209(3):665-669.

    Article  Google Scholar 

  28. Morris J, Albery IP, Heather N, Moss AC. Continuum Beliefs Are Associated with Higher Problem Recognition than Binary Beliefs Among Harmful Drinkers Without Addiction Experience. Addict Behav. 2020;105:106292. doi:https://doi.org/10.1016/j.addbeh.2020.106292

    CAS  Article  PubMed  Google Scholar 

  29. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders: DSM-IV-TR. American Psychiatric Association; 2000.

    Google Scholar 

  30. Takane Y, de Leeuw J. On the Relationship Between Item Response Theory and Factor Analysis of Discretized Variables. Psychometrika. 1987;52(3):393-408. doi:https://doi.org/10.1007/BF02294363

    Article  Google Scholar 

  31. Kamata A, Bauer DJ. A Note on the Relation Between Factor Analytic and Item Response Theory Models. Structural Equation Modeling: A Multidisciplinary Journal. 2008;15(1):136-153. doi:https://doi.org/10.1080/10705510701758406

    Article  Google Scholar 

  32. Chalmers RP. mirt: A Multidimensional Item Response Theory Package for the R Environment. Journal of Statistical Software. 2012;48(1):1-29. doi:https://doi.org/10.18637/jss.v048.i06

    Article  Google Scholar 

  33. R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing; 2019. https://www.R-project.org

  34. Iacobucci D. Structural Equations Modeling: Fit Indices, Sample Size, and Advanced Topics. Journal of Consumer Psychology. 2010;20(1):90-98. doi:https://doi.org/10.1016/j.jcps.2009.09.003

    Article  Google Scholar 

  35. Jöreskog KG, Moustaki I. Factor Analysis of Ordinal Variables: a Comparison of Three Approaches. Multivariate Behav Res. 2001;36(3):347-387. doi:https://doi.org/10.1207/S15327906347-387

    Article  PubMed  Google Scholar 

  36. De Ayala RJ. The Theory and Practice of Item Response Theory. Guilford Publications; 2013.

    Google Scholar 

  37. Chalmers RP, Counsell A, Flora DB. It Might Not Make a Big DIF: Improved Differential Test Functioning Statistics That Account for Sampling Variability. Educ Psychol Meas. 2016;76(1):114-140. doi:https://doi.org/10.1177/0013164415584576

    Article  PubMed  Google Scholar 

  38. Rivas GEL, Stark S, Chernyshenko OS. The Effects of Referent Item Parameters on Differential Item Functioning Detection Using the Free Baseline Likelihood Ratio Test: Applied Psychological Measurement. Published online, 2008. doi:https://doi.org/10.1177/0146621608321760

  39. Kopf J, Zeileis A, Strobl C. Anchor Selection Strategies for DIF Analysis: Review, Assessment, and New Approaches. Educational and Psychological Measurement. Published online, 2014. doi:https://doi.org/10.1177/0013164414529792

  40. Lai MHC, Richardson GB, Mak HW. Quantifying the Impact of Partial Measurement Invariance in Diagnostic Research: an Application to Addiction Research. Addictive Behaviors. Published online, 2018. doi:https://doi.org/10.1016/j.addbeh.2018.11.029

  41. Cook KF, Bombardier CH, Bamer AM, Choi SW, Kroenke K, Fann JR. Do somatic and cognitive symptoms of traumatic brain injury confound depression screening? Archives of physical medicine and rehabilitation. 2011;92(5):818-823.

    Article  Google Scholar 

  42. Putnick DL, Bornstein MH. Measurement Invariance Conventions and Reporting: the State of the Art and Future Directions for Psychological Research. Dev Rev. 2016;41:71-90. doi:https://doi.org/10.1016/j.dr.2016.06.004

    Article  PubMed  PubMed Central  Google Scholar 

  43. Cheung GW, Rensvold RB. Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance. Structural Equation Modeling: A Multidisciplinary Journal. 2002;9(2):233-255. doi:https://doi.org/10.1207/S15328007SEM0902_5

    Article  Google Scholar 

  44. Flora DB, Curran PJ. An Empirical Evaluation of Alternative Methods of Estimation for Confirmatory Factor Analysis with Ordinal Data. Psychol Methods. 2004;9(4):466-491. doi:https://doi.org/10.1037/1082-989X.9.4.466

    Article  PubMed  PubMed Central  Google Scholar 

  45. Yeung K, Richards J, Goemer E, et al. Costs of Using Evidence-Based Implementation Strategies for Behavioral Health Integration in a Large Primary Care System. Health Services Research. 2020;55(6):913-923. doi:https://doi.org/10.1111/1475-6773.13592

    Article  PubMed  PubMed Central  Google Scholar 

  46. Serier KN, Venner KL, Sarafin RE. Evaluating the Validity of DSM-5 Alcohol Use Disorder Diagnostic Criteria in a Sample of Treatment Seeking Native Americans. J Addict Med. 2019;13(1):35-40. doi:https://doi.org/10.1097/ADM.0000000000000452

    Article  PubMed  PubMed Central  Google Scholar 

  47. Tai B, Wu L-T, Clark HW. Electronic Health Records: Essential Tools in Integrating Substance Abuse Treatment with Primary Care. Subst Abuse Rehabil. 2012;3:1-8. doi:https://doi.org/10.2147/SAR.S22575

    Article  PubMed  PubMed Central  Google Scholar 

  48. Wang PS, Lane M, Olfson M, Pincus HA, Wells KB, Kessler RC. Twelve-Month Use of Mental Health Services in the United States: Results from the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62(6):629-640. doi:https://doi.org/10.1001/archpsyc.62.6.629

    Article  PubMed  Google Scholar 

  49. Park-Lee E, Lipari RN, Hedden SL, Kroutil LA, Porter JD. Receipt of services for substance use and mental health issues among adults: results from the 2016 National Survey on Drug Use and Health. NSDUH Data Review. Substance Abuse and Mental Health Services Administration, Rockville, MD. Published online 2017.

  50. McQuade WH, Levy SM, Yanek LR, Davis SW, Liepman MR. Detecting symptoms of alcohol abuse in primary care settings. Arch Fam Med. 2000;9(9):814-821.

    CAS  Article  Google Scholar 

  51. Rehm J, Anderson P, Manthey J, et al. Alcohol Use Disorders in Primary Health Care: What Do We Know and Where Do We Go? Alcohol Alcohol. 2016;51(4):422-427. doi:https://doi.org/10.1093/alcalc/agv127

    Article  PubMed  Google Scholar 

  52. Anderson P, O’Donnell A, Kaner E. Managing Alcohol Use Disorder in Primary Health Care. Curr Psychiatry Rep. 2017;19(11). doi:https://doi.org/10.1007/s11920-017-0837-z

  53. Sahker E, Arndt S. Alcohol Use Screening and Intervention by American Primary Care Providers. International Journal of Drug Policy. 2017;41:29-33. doi:https://doi.org/10.1016/j.drugpo.2016.11.013

    Article  PubMed  Google Scholar 

  54. US Preventive Services Task Force, Curry SJ, Krist AH, et al. Screening and Behavioral Counseling Interventions to Reduce Unhealthy Alcohol Use in Adolescents and Adults: US Preventive Services Task Force Recommendation Statement. JAMA. 2018;320(18):1899-1909. doi:https://doi.org/10.1001/jama.2018.16789

    Article  Google Scholar 

  55. Williams EC, Achtmeyer CE, Thomas RM, et al. Factors Underlying Quality Problems with Alcohol Screening Prompted by a Clinical Reminder in Primary Care: a Multi-site Qualitative Study. J Gen Intern Med. 2015;30(8):1125-1132. doi:https://doi.org/10.1007/s11606-015-3248-z

    Article  PubMed  PubMed Central  Google Scholar 

  56. McNeely J, Adam A, Rotrosen J, et al. Comparison of Methods for Alcohol and Drug Screening in Primary Care Clinics. JAMA Network Open. 2021;4(5):e2110721. doi:https://doi.org/10.1001/jamanetworkopen.2021.10721

    Article  PubMed  PubMed Central  Google Scholar 

  57. Keyes KM, Hatzenbuehler ML, McLaughlin KA, et al. Stigma and Treatment for Alcohol Disorders in the United States. Am J Epidemiol. 2010;172(12):1364-1372. doi:https://doi.org/10.1093/aje/kwq304

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  58. Smith SM, Dawson DA, Goldstein RB, Grant BF. Examining Perceived Alcoholism Stigma Effect on Racial-Ethnic Disparities in Treatment and Quality of Life Among Alcoholics. J Stud Alcohol Drugs. 2010;71(2):231-236. doi:https://doi.org/10.15288/jsad.2010.71.231

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

Research reported in this publication was supported by the National Institute on Alcohol Abuse and Alcoholism (NIAAA) of the National Institutes of Health (NIH) under award numbers R21AA028073, K01AA024796, and K24AA022128.

Author information

Authors and Affiliations

  1. Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA, USA

    Kevin A. Hallgren PhD

  2. Department of Health Systems and Population Health, University of Washington, Seattle, WA, USA

    Kevin A. Hallgren PhD, Theresa E. Matson MPH & Katharine A. Bradley MD, MPH

  3. Kaiser Permanente Washington Health Research Institute, Seattle, WA, USA

    Theresa E. Matson MPH, Malia Oliver BA, Jennifer F. Bobb PhD, Amy K. Lee MPH, Ryan M. Caldeiro MD & Katharine A. Bradley MD, MPH

  4. Department of Psychology and Center on Alcohol, Substance Use, and Addictions, University of New Mexico, Albuquerque, NM, USA

    Katie Witkiewitz PhD

  5. Center of Innovation for Veteran-Centered and Value-Driven Care, Health Services Research and Development, Veteran Affairs Puget Sound Health Care System, Seattle, WA, USA

    Daniel Kivlahan PhD & Katharine A. Bradley MD, MPH

  6. Department of Medicine, University of Washington, Seattle, WA, USA

    Katharine A. Bradley MD, MPH

Authors
  1. Kevin A. Hallgren PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Theresa E. Matson MPH
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Malia Oliver BA
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Katie Witkiewitz PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jennifer F. Bobb PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Amy K. Lee MPH
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ryan M. Caldeiro MD
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Daniel Kivlahan PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Katharine A. Bradley MD, MPH
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kevin A. Hallgren PhD.

Ethics declarations

Disclaimer

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

ESM 1

(DOCX 541 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hallgren, K.A., Matson, T.E., Oliver, M. et al. Practical Assessment of Alcohol Use Disorder in Routine Primary Care: Performance of an Alcohol Symptom Checklist. J GEN INTERN MED 37, 1885–1893 (2022). https://doi.org/10.1007/s11606-021-07038-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11606-021-07038-3

KEY WORDS

  • addiction
  • alcohol
  • diagnosis
  • differential item functioning
  • item response theory
  • primary care
  • screening