Background

Tobacco, alcohol and drug use drive poor health outcomes and are associated with substantial societal costs [1,2,3,4,5]. The World Health Organization (WHO), and the U.S. Preventive Service Task Force (USPSTF) recommend screening for tobacco and alcohol use in adult primary care patients [5,6,7]. Screening for drug use is recommended in the US Surgeon General’s report on addiction, and by the Substance Abuse and Mental Health Services Administration (SAMHSA) [8, 9]. Screening for opioid use has gained increased attention in light of the US opioid crisis, as individuals who are identified as having problem use of opioids could be targeted for overdose prevention and treatment interventions. New SAMHSA guidelines specifically recommend screening for opioid use in general medical settings [10]. Doing so requires the use of a screening tool that identifies illicit and prescription opioid use as a component of a general screen for problem use of tobacco, alcohol, and other drugs, while still remaining brief enough to fit into routine clinical care.

Despite the existing recommendations, unhealthy alcohol and drug use remain largely undetected in health care settings [1, 11]. Many challenges to implementing screening are related to clinical workflow and time pressures [12, 13], as well as to the stigma associated with substance use [14, 15]. An electronic self-administered screening tool has the potential to address several of these barriers that are encountered in medical settings. First, patients may feel more comfortable disclosing stigmatized behavior when it is self-reported instead of asked face-to-face, and this can lead to more accurate disclosure of substance use [16,17,18]. Further, electronic screening may allow patients to complete screening in the privacy of their own home, (for example through a patient portal into their electronic health record (EHR)), and be linked immediately to a patient-facing electronic intervention that does not require them to interact with clinical staff. Second, electronic screening tools can reduce barriers to screening in medical settings because they can be completed in the clinic waiting room using a tablet or kiosk computer, with results transmitted directly into the EHR, thus minimizing intrusion into clinical workflows. Screening results may be paired with clinical decision supports tools in the EHR, to help providers offer adequate interventions to their patients (i.e. brief intervention for unhealthy use or treatment for substance use disorder). Third, an electronic approach can improve the quality of screening. Because self-administered questionnaires consistently deliver screening items exactly as written, electronic screening may have higher fidelity and reliability than an interviewer-administered approach [19,20,21,22,23]. Delivering screening in an electronic format, as opposed to paper, also makes it possible to deliver sophisticated instruments that may require complex skip patterns or computer adaptive testing approaches, (for example, the World Health Organization Alcohol, Smoking, and Substance Involvement Screening Test (ASSIST), or the Patient-Reported Outcomes Measurement Information System (PROMIS) instruments) because the computer efficiently delivers only the items that are required based on the patient’s prior responses [23,24,25].

Yet electronic screening could also be challenging. Patients may have trouble reading or understanding the questions, or difficulty navigating the computer interface. While electronic self-administered screening tools can incorporate audio guidance to accommodate low literacy users, some patients may still have difficulty operating the technology [21, 26,27,28]. While some patients appreciate the privacy of self-administered screening, others may prefer the human touch of an interviewer [29].

The tobacco, alcohol, prescription medication, and other substance use (TAPS) Tool is a two-step screening (TAPS-1) and brief assessment (TAPS-2) instrument [30, 31] that identifies unhealthy use of tobacco, alcohol, prescription medications (used nonmedically) and illicit drugs. The TAPS Tool was specifically developed for adult primary care, was designed to be used in either an electronic self-administered format (myTAPS) or a more traditional interviewer-administered format, and both formats were validated in a large study conducted by the National Institute of Drug Abuse (NIDA) Clinical Trials Network [30, 31]. The TAPS Tool performed well for identifying problem use of tobacco (sensitivity 0.92, specificity 0.87), alcohol (sensitivity 0.77, specificity 0.77), and commonly used classes of illicit drugs (sensitivity ranging from 0.73 to 0.79, specificity ranging from 0.93 to 1.0) [31]. For detecting nonmedical use of prescription drugs, sensitivity was lower (ranging from 0.61 to 0.66) but still comparable to other screening instruments [32], and specificity was high (0.97–0.98). Based on the results of this validation study, the TAPS Tool is among the instruments recommended by the National Institute on Drug Abuse and by SAMHSA to screen for opioid and other substance use in medical settings [10, 33].

While the primary focus of the TAPS Tool study was to evaluate the accuracy of the TAPS for identifying problem use and substance use disorders, data was also collected on operational characteristics (time and assistance required) and patient attitudes toward the TAPS Tool, for the purpose of informing its future integration into primary care practice. This study presents the planned secondary outcomes analysis of these data. The aims of our analysis were to determine the feasibility and acceptability of myTAPS among primary care patients, including in specific subpopulations that may have greater difficulty using an electronic self-administered screening tool.

We were interested in how our two outcomes (feasibility and acceptability) may differ among subpopulations of patients who may have greater difficulty completing an electronic self-administered instrument. We examined the hypothesis that myTAPS, in comparison to the interviewer format, would be less feasible and acceptable for individuals who have greater difficulty completing an electronic screener, and may thus appreciate the assistance that an interviewer can provide. Based on the existing literature on electronic substance use screening, we hypothesized that individuals from the following groups may have greater difficulty completing the myTAPS format, and would find the interviewer format more acceptable: males; older participants (age > 65 years); Hispanic participants; and individuals with less than high school education [21, 26, 28, 34,35,36,37,38.] Conversely, we hypothesized that the myTAPS format would be more feasible and acceptable, in comparison to the interviewer format, for those with current alcohol or drug use, females, racial minorities (African American), and younger participants (age 18–25 years), because self-administered screeners are typically preferred by individuals who are reporting stigmatized behavior or are from groups who suffer from high levels of substance use-associated stigma [39,40,41,42,43,44]. An electronic screening instrument was also hypothesized to be preferable for younger participants, who are highly acclimated to this technology. We further examined whether feasibility and acceptability differed based on the order in which the TAPS was administered, with a hypothesis that the first format received (whether it was myTAPS or interviewer-administered format) would be less feasible and acceptable than the second format, because participants would be more familiar with the TAPS Tool items on the second administration.

Methods

The methods of the parent validation study (Clinical Trial registration: ClinicalTrials.gov identifier NCT02110693) are fully detailed in two preceding manuscripts [30, 31], and summarized here. The TAPS tool items are presented in a prior publication [31] and the instrument is available on the NIDA website (https://www.drugabuse.gov/taps/#/) [45].

Participants and Recruitment

In a study of the National Drug Abuse Treatment Clinical Trials Network, between August 2014 and April 2015, 2000 participants, from five primary care clinics located in urban and suburban areas in the Eastern U.S., completed the TAPS Tool validation study. Eligible individuals were adults (18 years or older) who were current patients of one of the participating clinics. Individuals were excluded if they could not understand spoken English or were physically unable to use a tablet computer. Research assistants (RAs) consecutively approached patients in the waiting room to invite them to participate, and obtained verbal informed consent. Institutional review boards of the sites involved in the study (blinded for review) approved all study procedures.

Study procedures

Participants completed the TAPS tool and other study assessments in a private room, and were informed that answers were confidential. All participants completed the TAPS Tool in both an interviewer-administered (administered by the RA) and an electronic self-administered (myTAPS) format. Each participant was randomly assigned to have the TAPS administered in one of two sequential orders (either interviewer-administered first followed by myTAPS, or myTAPS first followed by the interviewer-administered format). The myTAPS was delivered on a tablet computer (iPad), and participants had the option of hearing the question and response options read verbatim by a recorded female voice. Before starting myTAPS, participants were given the option of viewing, on the tablet, a brief tutorial on how to operate the electronic screening tool, include use of the touchscreen buttons to select responses and advance from one item to the next. The RA noted any participant requests for assistance, and recorded the reason(s) for the request, for both formats. The time required for completing myTAPS was recorded by the computer, and for the interviewer-administered format it was recorded in 1-min increments by the RA, using a stopwatch. Following completion of both formats of the TAPS, the RA verbally administered a brief survey. The survey was developed by the investigators to assess participant views on the tool’s feasibility and acceptability. Its items were informed by prior work on patient attitudes toward substance use screening [29].

Outcomes and measure

We assessed two types of outcomes: (1) feasibility (ease of use); and (2) acceptability (preference for the electronic versus interviewer-administered format). Survey responses used a 5-point Likert Scale (with options ranging from strongly agree to strongly disagree).

Feasibility of the TAPS tool was assessed by two self-reported questions from the survey, as well as RA-observed requests for assistance and the time (in minutes) required to complete the TAPS Tool. The survey questions addressed (1) the ease of use of the myTAPS tablet (“The iPad touch screen was easy to use”) and (2) the usefulness of the voice recording on the myTAPS (“The voice recording was helpful”). The participant’s response to the voice recording question was entered as ‘not applicable’ for those who did not use the voice recording; this allowed us to additionally use the response to this item as a measure of the number of participants who used the voice recording option. For those who requested assistance, the RAs recorded the reason(s); these reasons were further categorized into three categories: comprehension (e.g. problems understanding the meaning of a TAPS Tool item), difficulty using the tablet (e.g. trouble using the touchscreen), or technical issue(s) (e.g. tablet not operating correctly). Acceptability was assessed with two survey items: “I would prefer that a person asked me these questions in the doctor’s office instead of answering them myself on the iPad”; and “I would prefer answering these questions on an iPad instead of having a person ask me.”

Statistical analyses

Demographic characteristics of participants and the three outcomes were summarized, for continuous variables, by their mean, median, and standard deviation (SD), and for categorical variables by their frequencies and percentages. To assess acceptability, survey collected measures were collapsed into three categories (1—strongly agree/agree, 2—neither agree nor disagree and 3—strongly disagree/disagree). Chi squared statistics were used to test whether the distribution of outcomes differed between subpopulations. For example, we tested whether the distribution of format preference differed between younger (18–25 years) and older (> 25 years) participants. Instead of assessing age as a single ordinal variable, we used age to examine whether older participants (> 65 years old) would prefer the interviewer-administered format because of greater difficulty using technology. Two logistic regression analyses were run with the dependent outcome variable dichotomized into agree (‘strongly agree’ and ‘agree’) versus disagree (‘neither agree nor disagree’, ‘disagree,’ and ‘strongly disagree’). The regression analysis assessed the association of each categorical outcome with each of the subpopulations; adjusted odds-ratios, 95% confidence intervals and p-values were obtained.

The feasibility measure ‘assistance requested’ was summarized by the number and proportion of participants who asked for assistance at least once while completing the myTAPS and the interviewed-administered format. The time to complete the two formats was summarized descriptively. The Chi squared test was used to evaluate ‘assistance requested’ to compare the subpopulation distributions. The Wilcoxon rank-sum test was used to evaluate time required to complete the TAPS tool for each subpopulation. Logistic regression was applied in the same manner as described above. Finally, we conducted analyses to determine if the order in which the two TAPS formats were administered affected feasibility. For these analyses, we compared the odds of requesting assistance and the time needed to complete the TAPS for those who received myTAPS first versus those who received the interviewed-administered format first. Format preference was measured through two separate items in the survey; one asked if the myTAPS format was preferred, and the other asked if the interviewer format was preferred. Among all participants, 69.8% (n = 1395) gave concordant answers to these two items. Concordance (i.e. those who preferred the myTAPS format did not prefer the interviewer format), is required in order to interpret results. Therefore, only participants with concordant responses were included in the analysis of format preference. For these analyses, illicit and nonmedical prescription drug uses were combined into a single variable, to maximize the available sample for the planned analyses. All regression models included age, education level, race, ethnicity, sex, and substance use (alcohol, drug, and nonmedical prescription drug use). Analyses were performed using STATA 14 software (StataCorp. 2015. Stata Statistical Software: Release 14. College Station, TX: StataCorp LP).

Results

Participant characteristics are summarized in Table 1.

Table 1 Demographic characteristics of the 2000 participants
Full size table

A majority of participants (56.2%) were women, and the mean age was 46 years (SD = 14.7); 11.3% were 18–25 years old, and 8.0% were over 65. Just over half (55.6%) of the participants were Black/African American and 11.7% were Hispanic. Nineteen percent had less than high school education. Based on responses to the myTAPS (TAPS-1 items), 42.9% had unhealthy alcohol use, 24.6% used illicit drugs, and 12.3% had nonmedical use of prescription drugs, in the past year. There was some overlap between illicit and nonmedical prescription drug use, with 91 participants (4.6% of the sample) screening positive for both.

Feasibility

Throughout the results section, results of the adjusted analyses are presented in the text, while results of bivariate analyses can be found in the specified tables.

Self-reported feasibility (Table 2)

The measures of feasibility showed that most participants (98.3%) found the tablet (myTAPS) easy to use. In the multivariate analysis, women had two times the odds of reporting that the tablet was easy to use, in comparison to men (OR = 2.09, 95% CI 1.01–4.33). Those who screened positive for prescription drug use had lower odds of reporting that the tablet was easy to use (OR = 0.36, 95% CI 0.15–0.83).

Table 2 Self-reported feasibility of the myTAPS (N = 2000)
Full size table

The myTAPS audio guidance was used by a minority (18.3%) of participants. Participants over age 65 more frequently reported use of the audio guidance (29.4% versus 17.3%, OR = 1.79, 95% CI 1.22–2.61), as did those who were Hispanic (26.2% versus 17.2%, OR = 1.92, 95% CI 1.36–2.74), those who were Black/African-American (19.9% versus 16.2%, OR = 1.30, 95% CI 1.01–1.68), and those who had less education (30% versus 15.5%, OR = 2.01, 95% CI 1.54–2.63). Conversely, the audio guidance was used less by women (OR = 0.52, 95% CI 0.41–0.66), and by young participants (OR = 0.30, 95% CI 0.16–0.54).

Requests for assistance (Table 3)

One-quarter (25.0%) of the study population (n = 500) requested assistance with myTAPS, while 8.1% (n = 162) requested assistance with the interviewed-administered format. Participants most frequently requested assistance with myTAPS because of difficulty using the tablet (7.8% of the sample, n = 155), followed by comprehension problems (6.9%, n = 137), and technical issues (6.5%, n = 129). An additional 75 (3.8%) requested assistance for more than one reason, and for 4 participants the reason for requesting assistance was not recorded.

Table 3 Assistance requested during the completion of the myTAPS Tool (N = 2000)
Full size table

Examining results by subpopulation, participants over 65 years old requested assistance with myTAPS at twice the rate of younger participants (48.1% versus 23.0%, OR = 2.79, 95% CI 1.98–3.93). Assistance was also more frequently requested by participants having less than high school education, in comparison with those having high school or higher level education (38.1% versus 21.9%, OR = 2.08, 95% CI 1.62–2.67). Participants more frequently requested assistance on the first format they completed, regardless of whether it was the myTAPS (27.7% versus 22.2%, OR = 1.35, 95% CI 1.09–1.67) or the interviewer format (11.5% versus 4.7%, OR = 3.17, 95% CI 2.20–4.57).

Time to complete the TAPS tool (Table 4)

Across all participants, the median time to complete myTAPS was 4.00 min (SD = 2.57, range 1–27 min), and it was completed by 90% of participants in 7 min or less. By comparison, the interviewer format had a median completion time of 2.00 min (SD = 1.00, range < 1 to 11 min), and was completed by 90% of participants in 3 min or less. Based on results of the Wilcoxon test, the time to complete myTAPS was higher for participants who were older (median = 5.00, mean = 6.14, SD = 3.30) Black/African American (median = 4.00, mean = 4.73, SD = 2.58), Hispanic (median = 4.00, mean = 5.03, SD = 3.00), had lower education (median = 5.00, mean = 6.00, SD = 3.26), or screened positive for illicit (median = 4.00, mean = 5.31, SD = 2.52) or prescription drug use (median = 5.00, mean = 4.93, SD = 2.72), in comparison to participants without these characteristics. Following a similar pattern to requests for assistance, more time was required to complete the format of the TAPS tool that was received first, whether it was the myTAPS (median = 4.00, mean = 4.85, SD = 2.84) or the interviewer-administered (median = 4.00, mean = 4.11, SD = 2.21) version. Less time was needed by women (median = 3.00, mean = 3.95, SD = 2.00), and by younger participants (median = 3.00, mean = 3.00, SD = 1.08) to complete myTAPS.

Table 4 Time to complete the myTAPS (N = 2000)
Full size table

Acceptability (Table 5)

A majority (52.7%) of participants had no preference regarding the myTAPS versus the interviewer-administered format of the TAPS Tool. Thee myTAPS format was preferred by 27.5% (n = 383) of participants, while the interviewer-administered format was preferred by 19.9% (n = 277). Participants with less than a high school education had almost three times greater odds of preferring the interviewer-administered format (OR = 2.75, 95% CI 2.00–3.78). In comparison to men, women were somewhat more likely to prefer myTAPS (OR = 1.29, 95% CI 1.00–1.66). Those who screened positive for drug use were also more likely to prefer the myTAPS over the interviewer-administered format (OR = 1.43, 95% CI 1.09–1.88).

Table 5 Format preference (N = 1395)
Full size table

Discussion

The myTAPS Tool, which efficiently screens and assesses tobacco, alcohol, and illicit or nonmedical prescription drug use (including opioids), was feasible and acceptable for the majority of participants in this diverse sample of adult primary care patients. While we detected small differences among some subpopulations who may be expected to have difficulty with an electronic self-administered screener, the overarching finding of our analysis was that most patients would be able and willing to complete screening with the myTAPS tool.

The myTAPS required more time to complete than the interviewer format, but as a self-administered screener, (which could be completed in the waiting room or prior to the clinic visit), requiring a median time of 4.0 min, myTAPS would be feasible in most primary care settings. Overall, one-quarter of participants requested some assistance with myTAPS. This is comparable to the rate of assistance found in a study of electronic self-administered formats of the Single-Item Screening Questions for alcohol and drugs, in which 29% of participants needed assistance [21], but higher than what has been reported for some other electronic self-administered screening tools. In a prior study of adult patients enrolled from public primary care clinics, 11% needed assistance with the Substance Use Brief Screen (SUBS) [32], and 5.3% needed assistance with the more complex ACASI-ASSIST [23]. Participants with lower education, as well as those over 65 years old, more frequently requested assistance and required more time to complete myTAPS screening. Prior studies have similarly shown that electronic and self-administered questionnaires may be more difficult for primary care patients having less computer familiarity or lower literacy [26, 28]. Our study thus indicates that while the majority of primary care patients can complete the myTAPS without assistance, clinical settings serving primarily older and less educated patients should have an interviewer-administered screening approach available, and all settings should be prepared to offer assistance to some individuals. Given that most patients would be able to complete the myTAPS on their own, adoption of a predominantly electronic screening approach may free up the time for clinical staff to help those patients who need it.

The use of audio guidance during the completion of myTAPS screening was frequent (18%), and significantly higher among participants with lower education, age > 65 years, and participants who were Hispanic or Black/African American. The higher use of audio guidance by these participants could potentially be explained by their lack of confidence in using new technology, or by low literacy [21, 26, 28]. Prior studies have also found that patients who are non-native English speakers could have difficulty understanding electronic self-administered screening questions [34, 36, 46]. Because our study did not collect data on primary language, we were unable to assess whether language was the reason that members of racial and ethnic minorities in our sample reported more use of the audio guidance.

Interestingly, we found that participants who screened positive for nonmedical use of prescription drugs requested more assistance, and required more time to complete screening in both formats of the TAPS. Our previous studies found that primary care patients had difficulty understanding screening items about prescription medications [29, 47]. Confusion about how to report nonmedical prescription drug use could underlie the lower feasibility among individuals who reported prescription drug use in our sample. In settings where nonmedical use of prescription medications is of particular concern, practices may consider adding an introduction to the screening questions, similar to that used for the ACASI ASSIST:Footnote 1 [23].

Regarding acceptability, most participants (52.7%) had no preference regarding electronic versus interviewer-administered screening. The interviewer-administered format was preferred by those having lower levels of education, perhaps because of the feasibility issues discussed above. Women and individuals who screened positive for drug use preferred the myTAPS format. Generally, self-administered questionnaires are preferred and are more accurate when asking about a stigmatized behavior [16, 48]. This finding is consistent with prior studies reporting that individuals with substance use, and especially women who use drugs, feel highly stigmatized [29, 40, 49,50,51,52], which could cause them to be less comfortable answering screening questions face-to-face.

Limitations

Our study has some limitations. While it is not surprising that individuals who reported substance use required more time to complete the TAPS (since they received more questions), for all participants the time required to complete the TAPS Tool was likely overestimated. Time was measured as the time to complete both the TAPS-1 and TAPS-2, and the recommended skip pattern was not followed because the validation study sought to study the performance of the TAPS-1 and TAPS-2 both separately and in combination. The time required to complete the TAPS Tool that we reported here should thus be viewed as the maximum time for patients to complete screening. Nonetheless, we found that the time was brief and would likely be feasible in a primary care setting (myTAPS: median time 4.0 min, and interviewer-administered format: median time 2.0 min). In comparison, the time to complete the longer ACASI-ASSIST was found in a prior study to be 3 to 5 min, [23, 29] while the interviewer-ASSIST required 5–15 min [53, 54].

Our survey regarding acceptability of the TAPS tool was not validated, and richer information about patient attitudes and preferences might have been gained through qualitative interviews. Having the RA administer the survey could have introduced social desirability bias. As detailed in the Methods section, there was inconsistency of responses regarding preference for interviewer-administered format versus myTAPS that required us to exclude 30% of the sample from the format preference analysis. Nevertheless, the survey gave consistent results on related items assessed and its findings were consistent with prior literature [16, 26, 28, 48]. Although the study was conducted in a general adult primary care population, the prevalence of substance use in our sample was somewhat higher than may be found in other primary care settings [55]. The study sites were only in urban and suburban areas, which may limit the generalizability of our findings to other populations, including those living in rural areas. Acceptability and feasibility may differ depending on characteristics of the population and the setting in which screening is being conducted. Finally, because the TAPS Tool was only available in English at the time of our study, we are unable to evaluate its feasibility and acceptability in other languages. A Spanish version has since been developed and pilot tested [56].

Conclusions

The myTAPS Tool would be feasible in most primary care clinical settings, and was well accepted by patients in this large and diverse sample of adult primary care patients. However, primary care practices that choose this format should be prepared to offer assistance to some patients, particularly those who are older or have less than a high school level of education, and should have the capacity to screen using an interviewer-administered approach when required. Future studies should assess the acceptability and the feasibility of myTAPS screening when it is implemented in routine practice.