Background

Emergency departments (EDs) provide a crucial opportunity to screen for substance use disorders and to provide essential care for people with substance use-related concerns [1]. In the last decade, ED visits related to substance use have increased substantially in North America [2,3,4]. Annual ED and inpatient costs of substance use disorder exceeded $13 billion in 2017 in the United States, and in Canada, per capita costs increased from $321 to $353 between 2007 and 2020 [5, 6]. Between 2014 and 2018, there were 9.3 million ED visits by patients with substance use disorders in the United States; during this period, substance-use related visits increased by 30% relative to baseline. Overall, one in 11 patients visiting an ED had a co-morbid alcohol or other substance use disorder [3]. Alcohol-related ED visits increased by 47% between 2006 and 2014 in the United States [4]. Similarly, in the Canadian province of Ontario, ED visits for alcohol-related concerns increased by 440% from 2003 to 2016 [7]. In the Canadian province of Alberta, ED visits related to any substance use increased by 38%, and those related to opioid use increased by 57.3% from 2010 to 2015 [8]. Alberta also saw a 168% increase in ED visits related to stimulant use from 2010 to 2017 [9]. These statistics emphasize the enormous and increasing impact of substance use in EDs, and the important role that emergency physicians play in identification, risk stratification, and management of substance use disorders.

ED visits are key contact points with high-risk patients with substance use and opportunities to provide life-saving interventions [1]. Data from British Columbia, Canada indicate that 60% of people who overdosed in 2015–2016 visited an ED in the year prior to their overdose event [10]. This highlights that ED visits are critical, and often missed, opportunities to identify at-risk individuals. The ED provides a unique opportunity for screening and identification of harmful substance use, initiating pharmacological and psychosocial interventions, and making referrals to outpatient addictions care [11]. A 2017 American College of Emergency Physicians position statement affirmed that “emergency medical professionals are positioned and qualified to mitigate the consequences of alcohol abuse through screening programs, brief intervention, and referral to treatment” [12]. This was similarly reflected in a 2020 position statement from the Canadian Association of Emergency Physicians, which recommended that emergency providers use case-finding strategies to identify opioid and other substance use disorders [13].

Although the need and opportunity for ED substance use screening is recognized and endorsed, there is no accepted recommendation on preferred screening methods. Numerous screening tools have been adapted for use in EDs, however their comparative performance is poorly understood. Determining the most sensitive and specific screening tools to identify individuals with harmful substance use will provide crucial information to inform ED guidelines and recommendations for best practices.

Methods

Aim

The primary objective of this systematic review is to synthesize available research to identify the diagnostic accuracy of screening tools in detecting harmful substance use and substance use disorders in an ED setting. We evaluated screening tools designed to detect both general and specific substance use-related harms.

Design

Our systematic review meets 2020 Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [14].

Registration and protocol

We submitted a systematic review protocol to PROSPERO prior to study initiation. Due to COVID-19-related delays, our protocol was not published prior to completion of our review.

Eligibility criteria

We included studies evaluating the following:

  • Population: Adults (≥ 18 years) presenting to EDs in any country, for any reason.

  • Intervention: Screening tools to identify substance use disorders or harmful use, for general or specific substances.

  • Outcomes: Identification or diagnosis of a substance use disorder or harmful substance use-related health outcome.

  • Study Design: derivation and/or validation studies with or without a comparator group. We only included interventional studies with integrated screening if they evaluated accuracy of screening compared to a gold standard substance use-related diagnosis. We excluded reviews.

Information sources and search strategy

We developed a search strategy that combined concepts Emergency Department AND Screening AND Substance Use using Medical Subject Headings (MeSH), keywords, and author-assigned terms informed by a previous literature review [11]. Studies referenced in this review were hand-searched for potential eligibility in the present systematic review. We limited our search to publications on or after January 1, 2000, adults, English language, and human studies. We applied our search to Medical Literature Analysis and Retrieval System Online (MEDLINE [Ovid]) and Embase (Ovid) (to January 5, 2021), PsycINFO (EBSCO) (to February 8, 2021), HaPI – Health and Psychosocial Instruments database (Ovid) (to February 8, 2021), Web of Science (Clarivate Analytics) (to February 26, 2021), and CINAHL – Cumulative Index to Nursing and Allied Health Literature (EBSCO) (to March 10, 2021). We report our full MEDLINE search strategy in Additional file 1.

Selection and data collection

We exported citations into Covidence and removed duplicates [15]. Two trained reviewers independently assessed title/abstracts (CR, KM) and excluded articles that were obviously irrelevant. Each potentially eligible title/abstract underwent full-text eligibility review by two of five authors independently and in duplicate (JMa, EM, JKe, KM, CR). For both title/abstract and full-text screening, reviewers met to discuss eligibility decisions after assessing an initial 20 citations to ensure consistency, then completed review of the remaining citations independently. Reviewers resolved disagreements by discussion, with primary investigators (JMo, JKo) adjudicating if they could not reach consensus.

Two of four reviewers then extracted data independently and in duplicate from eligible articles (JMa, EM, JKe, JMo). Reviewers completed all assessments and extractions using standardized forms that were pilot tested among independent research colleagues for face validity. Reviewers discussed and resolved discrepancies and involved the primary investigators (JMo, JKo) to adjudicate when they could not reach consensus. When data were missing or ambiguous, we emailed authors up to two times to request additional information.

Data items

We extracted information relating to the study characteristics (authors, publication year, design, location, time period, follow-up period if applicable, data sources); study participants (inclusion/exclusion criteria, age, sex, gender, ethnicity, education, occupational status, marital status, income, ED presentation, comorbidities, number of participants in main analysis, losses to follow-up); details about screening (methods of determining eligibility, screening tools, substances screened for, person administering screening, person interpreting screening results, follow-up after ED visit, and descriptions of associated interventions where applicable); and patient outcomes (definitions/thresholds for “screen positive,” numbers screened positive and negative, gold standard definition, methods of ascertainment, person applying the screening tool and gold standard, sensitivity and specificity of the screening tool).

Study risk of bias

We assessed risk of bias in included studies using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool specifically developed for diagnostic accuracy studies [16]. QUADAS-2 comprises four domains of patient selection, index test, reference standard, and flow and timing. Two of four reviewers (JMa, EM, JKe, JMo) performed assessments independently and in duplicate. Reviewers met to discuss risk of bias decisions after an initial 12 appraisals to ensure consistency, then completed the assessments independently. Reviewers resolved discrepancies by discussion and involved the primary investigators (JMo, JKo) to adjudicate if they could not reach consensus.

Effect measures

For the diagnosis of a substance use disorder or a substance use-related patient outcome, effect measures were diagnostic accuracy (e.g., sensitivity, specificity).

Synthesis methods

We attempted to group papers assessing identical screening tools, “screen positive” thresholds, substances, and gold standard outcome definitions to meta-analyze sensitivity and specificity in ED populations. Due to a limited number of studies that could be grouped, we were unable to proceed with meta-analysis.

We descriptively summarized results in forest plots. We limited our visual summaries to studies assessing outcomes of alcohol abuse and/or dependence since this comprised the majority, and to North American studies to support comparability across screening tools. We did not limit our visual summaries to studies with low risk of bias, as many performed variably on the distinct domains of the QUADAS-2 tool. We instead present all studies meeting the above criteria, along with risk of bias assessments (Table 2), enabling clinicians and decision makers to interpret summative visual results in the context of studies’ quality assessments. We extracted information from each study to generate a 2 × 2 table and used the R package “meta” to obtain 95% confidence intervals for sensitivity and specificity. We used the Clopper-Pearson method to calculate confidence intervals. We plotted these values for outcomes of Diagnostic and Statistical Manual of Mental Disorders, 4th Edition (DSM-IV) alcohol abuse and/or dependence, as these were the reference standards most commonly reported in the included studies meeting criteria for visual summarization.

Results

Study selection

After removing duplicates, our search strategy yielded 2696 citations. We excluded 2328 after title/abstract review and 322 after full-text review, most commonly for ineligible outcome (n = 175) and population (n = 67). Five of 33 included articles evaluated subsets of the same population. For these studies, we included data evaluating the same tool(s) only once [17,18,19,20,21]. See Fig. 1 for the study flow diagram.

Fig. 1
figure 1

Study flow diagram

Full size image

Study characteristics

We summarize study characteristics, including participants, screening tools, “screen positive” cut-offs, reference standard, and tool performance in Table 1. We report on 33 included studies [17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49]. All were cohort studies. Twenty-one studies (64%) evaluated a North American population. A minority of studies were set in Europe (n = 6), South America (n = 4 [N.B. some overlap with North American studies]), Asia (n = 2), and Africa [1]. Studies included a median of 530 patients (IQR: 200, 1492). Fourteen studies (42%) applied screening among a general ED population, whereas others evaluated patients with specific presentations (trauma/injury [n = 6], psychiatric [n = 3], alcohol intoxication [n = 2], opioid prescription request and/or pain [n = 3]). Three studies applied further screening to ED patients who reported alcohol use in the last 12 months [30, 31, 45]. Where reported, tools were applied by patient surveys or questionnaires (n = 10), or by interviewers with research or clinical backgrounds (n = 22). In most studies, trained research staff conducted screening (n = 21); in one study, physicians on duty administered the screening tool.

Table 1 Study characteristics and screening tool performance
Full size table

Most studies evaluated tools designed to screen for alcohol use problems (n = 26). The most commonly evaluated tools were the Alcohol Use Disorders Identification Test (AUDIT; 14 studies), Cut down/Annoyed/Guilty/Eye-opener (CAGE; 13 studies), and the Rapid Alcohol Problems Screen and its derivatives (RAPS, RAPS4, RAPS4-QF; 12 studies). A minority of studies addressed drugs/substances more generally (n = 5 [N.B. some overlap with alcohol studies]), opioids (n = 3), and cocaine (n = 1).

Results of individual studies and syntheses

We summarize tool sensitivities and specificities in Table 1. Studies evaluated screening tools’ test characteristics against a range of reference standards, most commonly DSM-IV alcohol abuse, dependence, and/or use disorder (n = 22); some studies reported on the same study population and are only summarized once. Other studies evaluated screening tool characteristics against reference standards of illicit substance use, dependence, and/or substance use disorder (DSM-IV); opioid use disorder (DSM-V); alcohol dependence, harmful alcohol use, or high-risk drinking (International Classification of Diseases 10th revision [ICD-10]); drug abuse (Composite International Diagnostic Interview [CIDI]); alcohol abuse (CIDI); and illicit drug use (MINI). Reported sensitivities amongst all screening tools ranged from 4 to 100%, and specificities from 43 to 100%.

Alcohol abuse/dependence

Figure 2 displays sensitivity and specificity for six studies assessing screening tools for alcohol abuse and/or dependence in North American ED patients. Four studies assessing six tools and screening thresholds (AUDIT ≥ 8; CAGE ≥ 2; DSM-IV-2 ≥ 1; RAPS ≥ 1; National Institute on Alcohol Abuse and Alcoholism [NIAAA]; Tolerance/Worry/Eye-opener/Amnesia/K-Cut down [TWEAK] ≥ 3) reported sensitivities and specificities that were both ≥ 83% [19, 22, 32, 37]. Overall, AUDIT ≥ 8 and RAPS ≥ 1 demonstrated the highest sensitivities (95%) [18]. The tools that demonstrated the lowest sensitivities were breathalyzer (21%) [19], quantity-frequency (25%) [32], and reporting drinking within 6 h prior to event (45%) [19]. Notably, two studies evaluated a single alcohol screening question (SASQ) for problem drinking (“When was the last time you had more than X drinks in 1 day?,” where X = 4 for women and X = 5 for men; within 3 months considered positive). The first study reported sensitivities of 85% and 82%, and specificities of 70% and 77% among men and women, respectively [26]. The second study reported a sensitivity of 83% and specificity of 72% among all screened patients [48].

Fig. 2
figure 2

Forest plot of screening tool sensitivity and specificity for detection of DSM-IV alcohol abuse/dependence

Full size image

Other substances

Five studies evaluated screening tools for general substance abuse and/or dependence (Relax/Alone/Friends/Family/Trouble [RAFFT] ≥ 3; Drug Abuse Screening Test [DAST] ≥ 4; single drug screening question; Alcohol, Smoking and Substance Involvement Screening Test [ASSIST] ≥ 42/18), and reported sensitivities ranging from 64%-90% and specificities from 61%-100% [22, 35, 43, 44, 46]. The ASSIST tool at a threshold of 18 performed best in one study, which reported sensitivity of 90% and specificity of 87% at detecting illicit substance abuse and dependence [46]. One study assessed screening for opioid misuse or use disorder using the Prescription Drug Use Questionnaire Patient Version tool (PDUQp; reported sensitivities 38%-56%, and specificities 75%-81%) [23], and two studies evaluated tools predicting aberrant behaviors related to opioid prescriptions (reported sensitivities 38%-93%, and specificities 45%-60%) [27, 49]. One study examined a screening tool for repeat ED visits among people with cocaine-related ED visits (reported sensitivity 46% and specificity 83%) [33].

We summarize components of the six alcohol screening tools that had both sensitivities and specificities ≥ 83%, the single alcohol screening question, and the ASSIST tool that performed best for general substance use and/or dependence in Additional file 2.

Risk of bias

We summarize studies’ risk of bias in Table 2. The large majority of studies (73%, n = 24/33) ranked as high risk of bias on at least one domain. In the domain of patient selection, 13 studies were rated low risk of bias, seven high risk, and 13 unclear. In 19 studies, the conduct or interpretation of the screening test were high risk of bias. Most studies did not provide enough information to determine whether application of the reference standard was likely to introduce bias (n = 26 unclear). In the domain of patient flow (e.g., whether all patients received the reference standard, and appropriateness of interval between index test and reference standard), 15 studies were rated high risk, 15 low risk, and three unclear risk of bias.

Table 2 Study risk of bias
Full size table

Discussion

Interpretation of findings

In our systematic review, six screening tools at specific thresholds concurrently demonstrated both sensitivities and specificities ≥ 83% in identifying ED patients with alcohol abuse and/or dependence. Although study heterogeneity precluded meta-analysis, our descriptive summaries show that multiple tools performed comparably. Given that no single tool appeared superior to others, the feasibility and logistics of ED screening approaches are important considerations. Simplicity, ease of recall, and clinicians’ ability to apply tools consistently and to efficiently integrate them into existing workflows are paramount (e.g., single question screens). Additionally, screening tools with higher sensitivity level should be prioritized to avoid false negatives and to increase detection of people at risk from harmful substance use who could benefit from interventions initiated or arranged from EDs. Based on our review, ED clinicians should prioritize the six tools that concurrently demonstrated both sensitivities and specificities ≥ 83% in ED settings, particularly those with the highest reported sensitivities of 95% (AUDIT ≥ 8 and RAPS ≥ 1). Screening tools that reported the lowest sensitivities (breathalyzer, quantity-frequency, and reporting drinking within 6 h prior to event) should be discontinued due to unacceptable risk of false negatives. Due to simplicity and efficiency, we recommend that ED clinicians consider integrating the single alcohol screening question (SASQ) for problem alcohol use into ED patients’ assessments (When was the last time you had more than X drinks in 1 day?,” where X = 4 for women and X = 5 for men; within 3 months considered positive). SASQ demonstrated sensitivities of 82–85% and specificities of 70–77% in two studies.

Five studies examined screening tools for general substance use. Although variable, reported sensitivities up to 90% and specificities up to 100% indicate that tools can accurately rule out and rule in harmful substance use in EDs. One tool (ASSIST ≥ 18) demonstrated sensitivity of 90% and specificity of 87% at detecting illicit substance use and dependence and could be considered in the ED setting on the basis of this data. However, a lack of studies evaluating screening tools for general substance use with common reference standards preclude recommendations. Furthermore, all results must be interpreted with caution given that included studies had predominantly high or uncertain risk of bias in key domains related to diagnostic studies. Low numbers of studies examining other specific substances preclude further interpretations.

ED providers face multiple competing demands, staffing shortages [50], resource strains [51], and burnout [52]. Screening must therefore seamlessly pair identification of high-risk patients with improved access to addictions resources that support frontline providers’ ability to provide improved patient care, rather than increase their workload.

Comparison to previous studies

Our results corroborate findings from a previous review supporting the utility of ED screening for substance use disorders, particularly for alcohol [11]. Our findings provide renewed support for recommendations from the American College of Emergency Physicians and Canadian Association of Emergency Physicians emphasizing importance of ED substance use screening and treatment initiation [12, 13]. Our results should be interpreted in light of evidence supporting the effectiveness of brief substance use interventions in EDs [53], as screening is a crucial step in initiating a spectrum of supports for high-risk individuals once identified.

Strengths and limitations

Our extraction of sensitivity and specificity data from included studies, either reported directly by authors or that we have derived from the presented data, is a strength. By collating data on comparative performance, we hope to inform clinicians’ and policy-makers’ decisions regarding which tools may be applicable to their local settings. Our systematic methodology, adherence to PRISMA guidelines, and use of the QUADAS-2 quality assessment tool specific to diagnostic accuracy studies strengthen our study’s rigour. Our review is limited by included studies’ heterogeneity, which prevented meta-analysis. Studies covered very different populations from various countries, and evaluated different subsets of ED patients (e.g., general versus specific presentations [psychiatric, intoxication, injury]), which limit generalizability. Moreover, only two studies evaluated screening by ED staff (most evaluated screening by trained research personnel), limiting real-world generalizability. Moreover, the majority of studies were ranked as uncertain or high risk of bias in one or more of the specific QUADAS-2 domains. Clinicians should consider studies’ risk of bias assessments globally in judging the applicability of reported screening tools to their own setting. For example, EDs seeking to implement strategies reported in studies that have high risk of bias in the domain of conduct or interpretation of the index test should carefully consider how their implementation of the reported tools (e.g., personnel applying the screens, and screening procedures/settings) compare to the original studies’ approaches. Furthermore, a paucity of studies evaluating substances other than alcohol limit conclusions. Finally, the impact of screening in the ED on patient substance use is not known. Nonetheless, our results offer a crucial update on ED approaches to screening, which is particularly needed given rising substance-related ED presentations in North America [2].

Clinical implications

Our results demonstrate that screening can accurately identify people with harmful alcohol and substance use in EDs. Interpreted in the context that one in 11 ED visits are made by people with substance use disorders [3], ED screening may capitalize on crucial opportunities to identify high-risk individuals who may not present elsewhere to the healthcare system. In addition to brief intervention and referral [54], emerging evidence supports the feasibility and effectiveness of ED treatment initiation (e.g., naltrexone for alcohol use disorder, buprenorphine/naloxone for opioid use disorder, linkage to urgent follow-up and community services [55,56,57]. The use of accurate screening tools will likely increase the number of patients who are initiated on such treatments in the ED.

Research implications

Future research should address gaps we have identified, particularly evaluation of screening tools for substances other than alcohol (e.g., opioids, stimulants). Additionally, in most studies screening was applied by trained research staff: research must validate whether tools perform well when applied by frontline practitioners, and how they can be best integrated into actual ED workflows from an implementation and quality improvement perspective. Data on patient outcomes following ED screening and initiation or referral to treatment are also lacking.

Conclusions

Six screening tools at various thresholds concurrently demonstrated both sensitivities and specificities ≥ 83% at detecting alcohol abuse and/or dependence when applied in EDs. Based on our review, tools with the highest sensitivities (AUDIT ≥ 8 and RAPS ≥ 1) and that prioritize simplicity and efficiency (single screening question for problem alcohol use [SASQ]) should be prioritized, while those with unacceptably low sensitivities (breathalyzer, quantity-frequency, and reporting drinking within 6 h prior to event) should be discontinued due to risk of false negatives. Practitioners and policy-makers should consider integrating substance use screening into workflows that combine identification of high-risk patients with improved access to addictions resources and enhanced supports for frontline clinicians.